2025-05-26

Title: Generalizing Large Language Model Usability Across Resource-Constrained

• Authors: Yun-Da Tsai
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.17040
• Pdf URL: https://arxiv.org/pdf/2505.17040
• Copy Paste: [[2505.17040]] Generalizing Large Language Model Usability Across Resource-Constrained(https://arxiv.org/abs/2505.17040)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have achieved remarkable success across a wide range of natural language tasks, and recent efforts have sought to extend their capabilities to multimodal domains and resource-constrained environments. However, existing approaches often rely on costly supervised fine-tuning or assume fixed training conditions, limiting their generalization when facing unseen modalities, limited data, or restricted compute resources. This dissertation presents a systematic study toward generalizing LLM usability under real-world constraints. First, it introduces a robust text-centric alignment framework that enables LLMs to seamlessly integrate diverse modalities-including text, images, tables, and any modalities - via natural language interfaces. This approach supports in-context adaptation to unseen or dynamically changing modalities without requiring retraining. To enhance robustness against noisy and missing modalities, an adversarial prompting technique is proposed, generating semantically challenging perturbations at the prompt level to stress-test model reliability. Beyond multimodal setting, the dissertation investigates inference-time optimization strategies for LLMs, leveraging prompt search and uncertainty quantification to improve performance without additional model training. This perspective offers an efficient alternative to scaling model parameters or retraining from scratch. Additionally, the work addresses low-resource domains such as Verilog code generation by designing correct-by-construction synthetic data pipelines and logic-enhanced reasoning models, achieving state-of-the-art performance with minimal data. Together, these contributions form a unified effort to enhance the adaptability, scalability, and efficiency of large language models under practical constraints.
• 摘要：大型语言模型（LLM）在各种自然语言任务中取得了巨大的成功，最近的努力试图将其功能扩展到多模式领域和资源约束环境。但是，现有的方法通常依赖于昂贵的监督微调或假设固定的培训条件，从而在面对看不见的方式，有限的数据或限制计算资源时限制其概括。本论文提出了一项系统的研究，旨在在现实世界中的约束下概括LLM可用性。首先，它引入了一个强大的以文本为中心的对齐框架，该框架使LLM可以通过自然语言接口无缝整合多种模态 - 包括文本，图像，表格和任何模态。这种方法支持在不需要重新训练的情况下未见或动态变化的模态的文化适应。为了增强对嘈杂和缺失模式的鲁棒性，提出了一种对抗性提示技术，从而在迅速的级别上产生了具有挑战性的障碍，以使压力测试模型的可靠性。除了多模式设置之外，论文还研究了LLMS的推理时间优化策​​略，利用及时的搜索和不确定性量化以提高性能而没有进行其他模型培训。该视角为缩放模型参数提供了有效的替代方案或从头开始进行重新训练。此外，该工作通过设计正确的合成数据管道和逻辑增强的推理模型来解决低资源域，例如Verilog代码生成，从而实现了最低的数据。这些贡献共同构成了统一的努力，以提高实际限制下大语言模型的适应性，可伸缩性和效率。

Title: RAP: Runtime-Adaptive Pruning for LLM Inference

• Authors: Huanrong Liu, Chunlin Tian, Xuyang Wei, Jiaheng Dai, Qin Liu, Tianqi Wei, Qingbiao Li, Li Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17138
• Pdf URL: https://arxiv.org/pdf/2505.17138
• Copy Paste: [[2505.17138]] RAP: Runtime-Adaptive Pruning for LLM Inference(https://arxiv.org/abs/2505.17138)
• Keywords: generation
• Abstract: Large language models (LLMs) excel at language understanding and generation, but their enormous computational and memory requirements hinder deployment. Compression offers a potential solution to mitigate these constraints. However, most existing methods rely on fixed heuristics and thus fail to adapt to runtime memory variations or heterogeneous KV-cache demands arising from diverse user requests. To address these limitations, we propose RAP, an elastic pruning framework driven by reinforcement learning (RL) that dynamically adjusts compression strategies in a runtime-aware manner. Specifically, RAP dynamically tracks the evolving ratio between model parameters and KV-cache across practical execution. Recognizing that FFNs house most parameters, whereas parameter -light attention layers dominate KV-cache formation, the RL agent retains only those components that maximize utility within the current memory budget, conditioned on instantaneous workload and device state. Extensive experiments results demonstrate that RAP outperforms state-of-the-art baselines, marking the first time to jointly consider model weights and KV-cache on the fly.
• 摘要：大型语言模型（LLMS）在语言理解和生成方面表现出色，但它们的巨大计算和内存要求阻碍了部署。压缩提供了缓解这些约束的潜在解决方案。但是，大多数现有方法都依赖于固定的启发式方法，因此无法适应运行时内存的变化或由不同的用户请求引起的异质KV-CACHE要求。为了解决这些局限性，我们提出了RAP，这是一个由强化学习（RL）驱动的弹性修剪框架，该框架以运行时感知的方式动态调整压缩策略。具体而言，RAP会在实际执行中动态跟踪模型参数和KV-CACHE之间的发展比率。认识到FFNS容纳大多数参数，而参数 - 注意层主导了KV -CACHE形成，RL代理仅保留那些以瞬时工作负载和设备状态为条件的当前存储器预算中最大化效用的组件。广泛的实验结果表明，RAP的表现优于最先进的基线，这标志着第一次共同考虑模型权重和飞行的KV-CACHE。

Title: REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge

• Authors: Siyang Song, Micol Spitale, Xiangyu Kong, Hengde Zhu, Cheng Luo, Cristina Palmero, German Barquero, Sergio Escalera, Michel Valstar, Mohamed Daoudi, Tobias Baur, Fabien Ringeval, Andrew Howes, Elisabeth Andre, Hatice Gunes
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17223
• Pdf URL: https://arxiv.org/pdf/2505.17223
• Copy Paste: [[2505.17223]] REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge(https://arxiv.org/abs/2505.17223)
• Keywords: generation
• Abstract: In dyadic interactions, a broad spectrum of human facial reactions might be appropriate for responding to each human speaker behaviour. Following the successful organisation of the REACT 2023 and REACT 2024 challenges, we are proposing the REACT 2025 challenge encouraging the development and benchmarking of Machine Learning (ML) models that can be used to generate multiple appropriate, diverse, realistic and synchronised human-style facial reactions expressed by human listeners in response to an input stimulus (i.e., audio-visual behaviours expressed by their corresponding speakers). As a key of the challenge, we provide challenge participants with the first natural and large-scale multi-modal MAFRG dataset (called MARS) recording 137 human-human dyadic interactions containing a total of 2856 interaction sessions covering five different topics. In addition, this paper also presents the challenge guidelines and the performance of our baselines on the two proposed sub-challenges: Offline MAFRG and Online MAFRG, respectively. The challenge baseline code is publicly available at this https URL
• 摘要：在二元相互作用中，各种各样的人面部反应可能适合对每种人的行为做出反应。在成功组织了2023年的React 2024挑战之后，我们提出了React 2025挑战，鼓励了机器学习（ML）模型的发展和基准测试，这些挑战可用于产生多种适当，多样，现实和同步的人类式的面部反应，以响应输入刺激的人（即表达式刺激者）表达的人（即助听器）表达的人类反应（即响应）。作为挑战的关键，我们为挑战参与者提供了第一个自然和大规模多模式的MAFRG数据集（称为MARS）记录137个人类人类二元相互作用，其中包含2856个互动会话，涵盖了五个不同的主题。此外，本文还介绍了挑战指南和基线对两个提议的子挑战的表现：离线MAFRG和在线MAFRG。挑战基线代码在此HTTPS URL上公开可用

Title: Optimal Policy Minimum Bayesian Risk

• Authors: Ramón Fernandez Astudillo, Md Arafat Sultan, Aashka Trivedi, Yousef El-Kurdi, Tahira Naseem, Radu Florian, Salim Roukos
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17242
• Pdf URL: https://arxiv.org/pdf/2505.17242
• Copy Paste: [[2505.17242]] Optimal Policy Minimum Bayesian Risk(https://arxiv.org/abs/2505.17242)
• Keywords: generation
• Abstract: Inference scaling can help LLMs solve complex reasoning problems through extended runtime computation. On top of targeted supervision for long chain-of-thought (long-CoT) generation, purely inference-time techniques such as best-of-N (BoN) sampling, majority voting, or more generally, minimum Bayes risk decoding (MBRD), can further improve LLM accuracy by generating multiple candidate solutions and aggregating over them. These methods typically leverage additional signals in the form of reward models and risk/similarity functions that compare generated samples, e.g., exact match in some normalized space or standard similarity metrics such as Rouge. Here we present a novel method for incorporating reward and risk/similarity signals into MBRD. Based on the concept of optimal policy in KL-controlled reinforcement learning, our framework provides a simple and well-defined mechanism for leveraging such signals, offering several advantages over traditional inference-time methods: higher robustness, improved accuracy, and well-understood asymptotic behavior. In addition, it allows for the development of a sample-efficient variant of MBRD that can adjust the number of samples to generate according to the difficulty of the problem, without relying on majority vote counts. We empirically demonstrate the advantages of our approach on math (MATH-$500$) and coding (HumanEval) tasks using recent open-source models. We also present a comprehensive analysis of its accuracy-compute trade-offs.
• 摘要：推理缩放可以通过扩展的运行时计算来帮助LLMS解决复杂的推理问题。除了针对长期思考（长期稳定）生成的有针对性的监督外，纯粹的推理时间技术，例如最佳N（BON）采样，多数投票或更普遍的一般情况，最小贝叶斯风险解码（MBRD）可以通过产生多个候选解决方案并汇总它们，从而进一步提高LLM准确性。这些方法通常以奖励模型的形式和风险/相似性功能的形式利用其他信号，这些功能比较生成的样品，例如，在某些归一化空间或标准相似性指标（例如rouge）中的精确匹配。在这里，我们提出了一种新颖的方法，将奖励和风险/相似性信号纳入MBRD。基于KL控制的强化学习中最佳政策的概念，我们的框架为利用此类信号提供了一种简单且定义明确的机制，比传统推理时间方法具有多个优势：更高的鲁棒性，提高的准确性和良好理解的渐近性行为。此外，它允许开发MBRD的样本效率变体，该变体可以根据问题的难度调整样本数量，而无需依赖多数投票计数。我们从经验上证明了我们使用最近的开源模型在数学（数学 - $ 500 $）和编码（HumaneVal）任务方面的优势。我们还对其准确性计算的权衡进行了全面分析。

Title: ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation

• Authors: Liang Shi, Yun Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17256
• Pdf URL: https://arxiv.org/pdf/2505.17256
• Copy Paste: [[2505.17256]] ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation(https://arxiv.org/abs/2505.17256)
• Keywords: generation
• Abstract: Recent advances in diffusion models have significantly improved text-to-face generation, but achieving fine-grained control over facial features remains a challenge. Existing methods often require training additional modules to handle specific controls such as identity, attributes, or age, making them inflexible and resource-intensive. We propose ExpertGen, a training-free framework that leverages pre-trained expert models such as face recognition, facial attribute recognition, and age estimation networks to guide generation with fine control. Our approach uses a latent consistency model to ensure realistic and in-distribution predictions at each diffusion step, enabling accurate guidance signals to effectively steer the diffusion process. We show qualitatively and quantitatively that expert models can guide the generation process with high precision, and multiple experts can collaborate to enable simultaneous control over diverse facial aspects. By allowing direct integration of off-the-shelf expert models, our method transforms any such model into a plug-and-play component for controllable face generation.
• 摘要：扩散模型的最新进展显着改善了文本到面的生成，但是对面部特征的细粒度控制仍然是一个挑战。现有方法通常需要培训其他模块来处理特定的控件，例如身份，属性或年龄，使它们变得不灵活且资源密集。我们提出了专家Gen，这是一个无培训的框架，利用了预培训的专家模型，例如面部识别，面部属性识别和年龄估计网络，以指导具有良好控制的生成。我们的方法使用潜在的一致性模型来确保在每个扩散步骤中进行现实和分配预测，从而使准确的指导信号有效地引导扩散过程。我们在定性和定量上表明专家模型可以高精度地指导生成过程，并且多个专家可以协作以同时控制各种面部方面。通过允许直接集成现成的专家模型，我们的方法将任何此类模型都转换为插件组件，以产生可控的面部生成。

Title: Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models

• Authors: Pushkar Shukla, Aditya Chinchure, Emily Diana, Alexander Tolbert, Kartik Hosanagar, Vineeth N Balasubramanian, Leonid Sigal, Matthew Turk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17280
• Pdf URL: https://arxiv.org/pdf/2505.17280
• Copy Paste: [[2505.17280]] Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models(https://arxiv.org/abs/2505.17280)
• Keywords: generative
• Abstract: The biases exhibited by text-to-image (TTI) models are often treated as independent, though in reality, they may be deeply interrelated. Addressing bias along one dimension - such as ethnicity or age - can inadvertently affect another, like gender, either mitigating or exacerbating existing disparities. Understanding these interdependencies is crucial for designing fairer generative models, yet measuring such effects quantitatively remains a challenge. To address this, we introduce BiasConnect, a novel tool for analyzing and quantifying bias interactions in TTI models. BiasConnect uses counterfactual interventions along different bias axes to reveal the underlying structure of these interactions and estimates the effect of mitigating one bias axis on another. These estimates show strong correlation (+0.65) with observed post-mitigation outcomes. Building on BiasConnect, we propose InterMit, an intersectional bias mitigation algorithm guided by user-defined target distributions and priority weights. InterMit achieves lower bias (0.33 vs. 0.52) with fewer mitigation steps (2.38 vs. 3.15 average steps), and yields superior image quality compared to traditional techniques. Although our implementation is training-free, InterMit is modular and can be integrated with many existing debiasing approaches for TTI models, making it a flexible and extensible solution.
• 摘要：文本对图像（TTI）模型所表现出的偏见通常被视为独立，尽管实际上，它们可能与之有着密切的相互关联。解决一个维度的偏见（例如种族或年龄）可能会无意中影响另一个维度，例如性别，减轻或加剧现有差异。了解这些相互依存关系对于设计更公平的生成模型至关重要，但是定量测量这种效果仍然是一个挑战。为了解决这个问题，我们引入了BiasConnect，这是一种用于分析和量化TTI模型中偏置相互作用的新工具。 BiasConnect在不同的偏置轴上使用反事实干预措施来揭示这些相互作用的基本结构，并估计减轻一个偏置轴对另一个偏置轴的影响。这些估计值显示出强烈的相关性（+0.65）与观察到的降低后结果。在BiasConnect的基础上，我们提出了Intermit，这是一种以用户定义的目标分布和优先级权重为指导的相互偏置缓解算法。 Intermit实现了较低的偏差（0.33 vs. 0.52），缓解步骤较少（2.38 vs. 3.15平均步骤），与传统技术相比，图像质量较高。尽管我们的实施是无训练的，但Intermit是模块化的，可以与许多现有的tti模型中的偏见方法集成在一起，从而使其成为灵活且可扩展的解决方案。

Title: Harnessing EHRs for Diffusion-based Anomaly Detection on Chest X-rays

• Authors: Harim Kim, Yuhan Wang, Minkyu Ahn, Heeyoul Choi, Yuyin Zhou, Charmgil Hong
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17311
• Pdf URL: https://arxiv.org/pdf/2505.17311
• Copy Paste: [[2505.17311]] Harnessing EHRs for Diffusion-based Anomaly Detection on Chest X-rays(https://arxiv.org/abs/2505.17311)
• Keywords: generation
• Abstract: Unsupervised anomaly detection (UAD) in medical imaging is crucial for identifying pathological abnormalities without requiring extensive labeled data. However, existing diffusion-based UAD models rely solely on imaging features, limiting their ability to distinguish between normal anatomical variations and pathological anomalies. To address this, we propose Diff3M, a multi-modal diffusion-based framework that integrates chest X-rays and structured Electronic Health Records (EHRs) for enhanced anomaly detection. Specifically, we introduce a novel image-EHR cross-attention module to incorporate structured clinical context into the image generation process, improving the model's ability to differentiate normal from abnormal features. Additionally, we develop a static masking strategy to enhance the reconstruction of normal-like images from anomalies. Extensive evaluations on CheXpert and MIMIC-CXR/IV demonstrate that Diff3M achieves state-of-the-art performance, outperforming existing UAD methods in medical imaging. Our code is available at this http URL this https URL
• 摘要：医学成像中无监督的异常检测（UAD）对于鉴定病理异常而无需广泛标记的数据至关重要。但是，现有的基于扩散的UAD模型仅依赖成像特征，从而限制了它们区分正常解剖变异和病理异常的能力。为了解决这个问题，我们提出了DIFF3M，这是一种基于多模式扩散的框架，该框架集成了胸部X射线和结构化电子健康记录（EHR），以增强异常检测。具体而言，我们引入了一个新型的图像-EHR跨意义模块，以将结构化的临床环境纳入图像生成过程，从而提高了模型将正常与异常特征区分开的能力。此外，我们制定了一种静态掩蔽策略，以增强异常现象的正常图像的重建。对CHEXPERT和MIMIC-CXR/IV的广泛评估表明，DIFF3M实现了最先进的性能，超过了医学成像中现有的UAD方法。我们的代码可在此HTTP URL上找到此https url

Title: FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems

• Authors: N. Benjamin Erichson, Vinicius Mikuni, Dongwei Lyu, Yang Gao, Omri Azencot, Soon Hoe Lim, Michael W. Mahoney
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17351
• Pdf URL: https://arxiv.org/pdf/2505.17351
• Copy Paste: [[2505.17351]] FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems(https://arxiv.org/abs/2505.17351)
• Keywords: super-resolution, generative
• Abstract: We introduce FLEX (FLow EXpert), a backbone architecture for generative modeling of spatio-temporal physical systems using diffusion models. FLEX operates in the residual space rather than on raw data, a modeling choice that we motivate theoretically, showing that it reduces the variance of the velocity field in the diffusion model, which helps stabilize training. FLEX integrates a latent Transformer into a U-Net with standard convolutional ResNet layers and incorporates a redesigned skip connection scheme. This hybrid design enables the model to capture both local spatial detail and long-range dependencies in latent space. To improve spatio-temporal conditioning, FLEX uses a task-specific encoder that processes auxiliary inputs such as coarse or past snapshots. Weak conditioning is applied to the shared encoder via skip connections to promote generalization, while strong conditioning is applied to the decoder through both skip and bottleneck features to ensure reconstruction fidelity. FLEX achieves accurate predictions for super-resolution and forecasting tasks using as few as two reverse diffusion steps. It also produces calibrated uncertainty estimates through sampling. Evaluations on high-resolution 2D turbulence data show that FLEX outperforms strong baselines and generalizes to out-of-distribution settings, including unseen Reynolds numbers, physical observables (e.g., fluid flow velocity fields), and boundary conditions.
• 摘要：我们介绍了使用扩散模型对时空物理系统进行生成建模的骨干架构Flex（Flow Expert）。 FLEX在剩余空间而不是在原始数据中运行，这是我们从理论上激励的建模选择，表明它降低了扩散模型中速度场的差异，这有助于稳定训练。 Flex将潜在变压器集成到具有标准卷积重新连接层的U-NET中，并结合了重新设计的跳过连接方案。这种混合设计使该模型能够捕获潜在空间中的局部空间细节和远程依赖性。为了改善时空调节，Flex使用特定于任务的编码器，该编码器处理辅助输入，例如粗或过去的快照。通过跳过连接将弱条件应用于共享编码器以促进概括，而通过跳过和瓶颈功能将强条适用于解码器，以确保重建保真度。 Flex可以实现超分辨率和预测任务的准确预测，并使用较少的两个反向扩散步骤。它还通过抽样产生校准的不确定性估计。对高分辨率2D湍流数据的评估表明，Flex胜过强大的基准，并概括为分布的设置，包括看不见的雷诺数，物理可观察物（例如，流体流动速度场）和边界条件。

Title: Alignment and Safety of Diffusion Models via Reinforcement Learning and Reward Modeling: A Survey

• Authors: Preeti Lamba, Kiran Ravish, Ankita Kushwaha, Pawan Kumar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17352
• Pdf URL: https://arxiv.org/pdf/2505.17352
• Copy Paste: [[2505.17352]] Alignment and Safety of Diffusion Models via Reinforcement Learning and Reward Modeling: A Survey(https://arxiv.org/abs/2505.17352)
• Keywords: generative
• Abstract: Diffusion models have emerged as leading generative models for images and other modalities, but aligning their outputs with human preferences and safety constraints remains a critical challenge. This thesis proposal investigates methods to align diffusion models using reinforcement learning (RL) and reward modeling. We survey recent advances in fine-tuning text-to-image diffusion models with human feedback, including reinforcement learning from human and AI feedback, direct preference optimization, and differentiable reward approaches. We classify these methods based on the type of feedback (human, automated, binary or ranked preferences), the fine-tuning technique (policy gradient, reward-weighted likelihood, direct backpropagation, etc.), and their efficiency and safety outcomes. We compare key algorithms and frameworks, highlighting how they improve alignment with user intent or safety standards, and discuss inter-relationships such as how newer methods build on or diverge from earlier ones. Based on the survey, we identify five promising research directions for the next two years: (1) multi-objective alignment with combined rewards, (2) efficient human feedback usage and active learning, (3) robust safety alignment against adversarial inputs, (4) continual and online alignment of diffusion models, and (5) interpretable and trustworthy reward modeling for generative images. Each direction is elaborated with its problem statement, challenges, related work, and a proposed research plan. The proposal is organized as a comprehensive document with literature review, comparative tables of methods, and detailed research plans, aiming to contribute new insights and techniques for safer and value-aligned diffusion-based generative AI.
• 摘要：扩散模型已成为图像和其他方式的领先生成模型，但是将其产出与人类的偏好和安全约束保持一致仍然是一个关键的挑战。该论文提案研究了使用增强学习（RL）和奖励建模对齐扩散模型的方法。我们调查了通过人类反馈的微调文本到图像扩散模型的最新进展，包括从人类和AI反馈中学习，直接优先优化和可区分的奖励方法。我们根据反馈的类型（人，自动化，二进制或排名偏好），微调技术（策略梯度，奖励加权的可能性，直接反向传播等）以及它们的效率和安全结果对这些方法进行分类。我们比较关键算法和框架，突出了它们如何改善用户意图或安全标准的一致性，并讨论诸如较新的方法与早期较新方法的相互关系。根据调查，我们在接下来的两年中确定了五个有前途的研究方向：（1）具有共同奖励，（2）有效的人类反馈用法和积极学习的多目标对准，（3）针对对抗性输入的稳健安全对准，（4）持续和在线和在线对差异模型的持续和在线对齐，以及（5）可解释的奖励模型。每个方向的问题声明，挑战，相关工作和拟议的研究计划都详细阐述了。该提案是通过文献综述，方法的比较表和详细的研究计划组织的综合文档，旨在为基于更安全和价值的基于扩散的生成AI提供新的见解和技术。

Title: Dual Ascent Diffusion for Inverse Problems

• Authors: Minseo Kim, Axel Levy, Gordon Wetzstein
• Subjects: cs.CV, cs.AI, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.17353
• Pdf URL: https://arxiv.org/pdf/2505.17353
• Copy Paste: [[2505.17353]] Dual Ascent Diffusion for Inverse Problems(https://arxiv.org/abs/2505.17353)
• Keywords: restoration
• Abstract: Ill-posed inverse problems are fundamental in many domains, ranging from astrophysics to medical imaging. Emerging diffusion models provide a powerful prior for solving these problems. Existing maximum-a-posteriori (MAP) or posterior sampling approaches, however, rely on different computational approximations, leading to inaccurate or suboptimal samples. To address this issue, we introduce a new approach to solving MAP problems with diffusion model priors using a dual ascent optimization framework. Our framework achieves better image quality as measured by various metrics for image restoration problems, it is more robust to high levels of measurement noise, it is faster, and it estimates solutions that represent the observations more faithfully than the state of the art.
• 摘要：在许多领域中，不良逆问题是基本的，从天体物理学到医学成像。新兴的扩散模型为解决这些问题提供了有力的先验。但是，现有的最大A-posteriori（MAP）或后采样方法依赖于不同的计算近似值，从而导致不准确或次优的样本。为了解决这个问题，我们引入了一种新方法，以使用双重升高优化框架来解决扩散模型先验的地图问题。我们的框架可实现更好的图像质量，这些图像质量通过各种指标来用于图像恢复问题，它对高度测量噪声的稳健性更高，它更快，并且估计解决方案比最忠实地代表观测值的解决方案比最忠实的现状。

Title: FRIREN: Beyond Trajectories -- A Spectral Lens on Time

• Authors: Qilin Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17370
• Pdf URL: https://arxiv.org/pdf/2505.17370
• Copy Paste: [[2505.17370]] FRIREN: Beyond Trajectories -- A Spectral Lens on Time(https://arxiv.org/abs/2505.17370)
• Keywords: generative
• Abstract: Long-term time-series forecasting (LTSF) models are often presented as general-purpose solutions that can be applied across domains, implicitly assuming that all data is pointwise predictable. Using chaotic systems such as Lorenz-63 as a case study, we argue that geometric structure - not pointwise prediction - is the right abstraction for a dynamic-agnostic foundational model. Minimizing the Wasserstein-2 distance (W2), which captures geometric changes, and providing a spectral view of dynamics are essential for long-horizon forecasting. Our model, FRIREN (Flow-inspired Representations via Interpretable Eigen-networks), implements an augmented normalizing-flow block that embeds data into a normally distributed latent representation. It then generates a W2-efficient optimal path that can be decomposed into rotation, scaling, inverse rotation, and translation. This architecture yields locally generated, geometry-preserving predictions that are independent of the underlying dynamics, and a global spectral representation that functions as a finite Koopman operator with a small modification. This enables practitioners to identify which modes grow, decay, or oscillate, both locally and system-wide. FRIREN achieves an MSE of 11.4, MAE of 1.6, and SWD of 0.96 on Lorenz-63 in a 336-in, 336-out, dt=0.01 setting, surpassing TimeMixer (MSE 27.3, MAE 2.8, SWD 2.1). The model maintains effective prediction for 274 out of 336 steps, approximately 2.5 Lyapunov times. On Rossler (96-in, 336-out), FRIREN achieves an MSE of 0.0349, MAE of 0.0953, and SWD of 0.0170, outperforming TimeMixer's MSE of 4.3988, MAE of 0.886, and SWD of 3.2065. FRIREN is also competitive on standard LTSF datasets such as ETT and Weather. By connecting modern generative flows with classical spectral analysis, FRIREN makes long-term forecasting both accurate and interpretable, setting a new benchmark for LTSF model design.
• 摘要：长期时间序列预测（LTSF）模型通常被视为可以跨域应用的通用解决方案，隐含地假设所有数据都是可以预测的。我们认为，使用诸如lorenz -63的混沌系统作为案例研究，我们认为几何结构（不是刻薄的预测）是动态 - 静态基础模型的正确抽象。将捕获几何变化的Wasserstein-2距离（W2）最小化，并提供动力学光谱对于长途预测至关重要。我们的模型Friren（通过可解释的特征网络的流动启发表示）实现了增强的归一化块，该块将数据嵌入正态分布的潜在表示中。然后，它生成一个W2高效的最佳路径，可以将其分解为旋转，缩放，逆旋转和翻译。该体系结构产生了本地生成的，几何形状的预测，这些预测独立于潜在的动力学，以及具有较小修改的有限的Koopman操作员的全局光谱表示。这使实践者能够识别哪些模式在本地和系统范围内增长，衰减或振荡。 Friren在336英寸，336-out，DT = 0.01设置的Lorenz-63上达到11.4，MAE为1.6，SWD为0.96，超过了Timemixer（MSE 27.3，MAE 2.8，SWD 2.1）。该模型在336个步骤中保持有效的预测，约为2.5次Lyapunov时间。在Rossler（96英寸，336淘汰）上，Friren的MSE为0.0349，MAE为0.0953，SWD为0.0170，表现优于Timemixer的4.3988的MSE，MAE为0.886，SWD为3.2065。 Friren在标准LTSF数据集（例如ETT和天气）上也具有竞争力。通过将现代生成流与经典的光谱分析联系起来，Friren可以长期预测准确和可解释的，从而为LTSF模型设计树立了新的基准。

Title: Value-Guided Search for Efficient Chain-of-Thought Reasoning

• Authors: Kaiwen Wang, Jin Peng Zhou, Jonathan Chang, Zhaolin Gao, Nathan Kallus, Kianté Brantley, Wen Sun
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.17373
• Pdf URL: https://arxiv.org/pdf/2505.17373
• Copy Paste: [[2505.17373]] Value-Guided Search for Efficient Chain-of-Thought Reasoning(https://arxiv.org/abs/2505.17373)
• Keywords: generation
• Abstract: In this paper, we propose a simple and efficient method for value model training on long-context reasoning traces. Compared to existing process reward models (PRMs), our method does not require a fine-grained notion of "step," which is difficult to define for long-context reasoning models. By collecting a dataset of 2.5 million reasoning traces, we train a 1.5B token-level value model and apply it to DeepSeek models for improved performance with test-time compute scaling. We find that block-wise value-guided search (VGS) with a final weighted majority vote achieves better test-time scaling than standard methods such as majority voting or best-of-n. With an inference budget of 64 generations, VGS with DeepSeek-R1-Distill-1.5B achieves an average accuracy of 45.7% across four competition math benchmarks (AIME 2024 & 2025, HMMT Feb 2024 & 2025), reaching parity with o3-mini-medium. Moreover, VGS significantly reduces the inference FLOPs required to achieve the same performance of majority voting. Our dataset, model and codebase are open-sourced.
• 摘要：在本文中，我们提出了一种简单有效的方法，用于在长篇文章推理轨迹上进行价值模型培训。与现有的流程奖励模型（PRM）相比，我们的方法不需要“步骤”的细粒度概念，这对于长篇文化推理模型很难定义。通过收集250万个推理迹线的数据集，我们训练了1.5B令牌级别的值模型，并将其应用于DeepSeek模型，以通过测试时间计算缩放来提高性能。我们发现，最终加权多数投票的块价值指导搜索（VGS）比多数投票或最佳N中的标准方法更好地提高了测试时间缩放。具有DeepSeek-R1-Distill-1.5b的VGS的推理预算为64代，在四个竞争数学基准（Aime 2024＆2025，HMMT 2024＆2025年2月和2025年）中，平均准确度为45.7％，达到了O3-Mini-Mini-Mini-medium。此外，VGS大大减少了实现多数投票表现所需的推理拖船。我们的数据集，模型和代码库是开源的。

Title: Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling

• Authors: Tianyu Xie, Shuchen Xue, Zijin Feng, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Cheng Zhang
• Subjects: cs.LG, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.17384
• Pdf URL: https://arxiv.org/pdf/2505.17384
• Copy Paste: [[2505.17384]] Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling(https://arxiv.org/abs/2505.17384)
• Keywords: generation
• Abstract: Discrete diffusion models have recently shown great promise for modeling complex discrete data, with masked diffusion models (MDMs) offering a compelling trade-off between quality and generation speed. MDMs denoise by progressively unmasking multiple dimensions from an all-masked input, but their performance can degrade when using few denoising steps due to limited modeling of inter-dimensional dependencies. In this paper, we propose Variational Autoencoding Discrete Diffusion (VADD), a novel framework that enhances discrete diffusion with latent variable modeling to implicitly capture correlations among dimensions. By introducing an auxiliary recognition model, VADD enables stable training via variational lower bounds maximization and amortized inference over the training set. Our approach retains the efficiency of traditional MDMs while significantly improving sample quality, especially when the number of denoising steps is small. Empirical results on 2D toy data, pixel-level image generation, and text generation demonstrate that VADD consistently outperforms MDM baselines.
• 摘要：离散扩散模型最近显示了建模复杂数据的巨大希望，掩盖扩散模型（MDMS）在质量和发电速度之间提供了令人信服的权衡。 MDMS DENOISE通过逐步从全面掩盖的输入中逐步揭示多个维度，但是由于使用二米间依赖性建模有限，它们的性能会降低。在本文中，我们提出了变异自动编码离散扩散（VADD），这是一个新型框架，通过潜在变量建模增强离散扩散，以隐式捕获维度之间的相关性。通过引入辅助识别模型，VADD可以通过各种下限进行稳定的培训，可最大化并摊销对训练集的推断。我们的方法保留了传统MDM的效率，同时显着提高了样品质量，尤其是当降解步骤的数量很少时。 2D玩具数据，像素级图像生成和文本生成的经验结果表明，VADD始终优于MDM基线。

Title: Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention

• Authors: Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Philip Torr, Xun Cao, Yao Yao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17412
• Pdf URL: https://arxiv.org/pdf/2505.17412
• Copy Paste: [[2505.17412]] Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention(https://arxiv.org/abs/2505.17412)
• Keywords: generation
• Abstract: Generating high resolution 3D shapes using volumetric representations such as Signed Distance Functions presents substantial computational and memory challenges. We introduce Direct3D S2, a scalable 3D generation framework based on sparse volumes that achieves superior output quality with dramatically reduced training costs. Our key innovation is the Spatial Sparse Attention mechanism, which greatly enhances the efficiency of Diffusion Transformer computations on sparse volumetric data. SSA allows the model to effectively process large token sets within sparse volumes, significantly reducing computational overhead and achieving a 3.9x speedup in the forward pass and a 9.6x speedup in the backward pass. Our framework also includes a variational autoencoder that maintains a consistent sparse volumetric format across input, latent, and output stages. Compared to previous methods with heterogeneous representations in 3D VAE, this unified design significantly improves training efficiency and stability. Our model is trained on public available datasets, and experiments demonstrate that Direct3D S2 not only surpasses state-of-the-art methods in generation quality and efficiency, but also enables training at 1024 resolution using only 8 GPUs, a task typically requiring at least 32 GPUs for volumetric representations at 256 resolution, thus making gigascale 3D generation both practical and accessible. Project page: this https URL.
• 摘要：使用体积表示（例如签名距离函数）产生高分辨率3D形状，列出了实质性的计算和内存挑战。我们介绍了Direct3D S2，这是一个基于稀疏量的可扩展3D生成框架，可通过大幅降低培训成本，从而达到较高的输出质量。我们的关键创新是空间稀疏注意机制，它极大地提高了扩散变压器计算在稀疏体积数据上的效率。 SSA允许该模型有效地处理稀疏体积内的大令牌集，从而大大降低了计算开销，并在向前传球中实现了3.9倍的速度，在向后通行证中的速度为9.6倍。我们的框架还包括一个变异自动编码器，该自动编码器在输入，潜在和输出阶段保持一致的稀疏体积格式。与以前在3D VAE中具有异质表示的方法相比，这种统一的设计显着提高了训练效率和稳定性。我们的模型在公共可用数据集中进行了培训，实验表明，直接3D S2不仅超过了发电质量和效率的最先进方法，而且还可以仅使用8个GPU以1024分辨率进行培训，这项任务通常需要至少32 GPU，至少需要32 GPU以256分辨率以256的分辨率，从而使Gigascale 3D代表均可实现实用，从而使Gigascale 3d coessional cossects coessional cossects coessional cossects coessial and coessional cossects coessial and coessial cossect coessial cossect coessial cossect coessial coessial cossect。项目页面：此HTTPS URL。

Title: VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Models

• Authors: Hefei Mei, Zirui Wang, Shen You, Minjing Dong, Chang Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17440
• Pdf URL: https://arxiv.org/pdf/2505.17440
• Copy Paste: [[2505.17440]] VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Models(https://arxiv.org/abs/2505.17440)
• Keywords: generation
• Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding and generation, yet their vulnerability to adversarial attacks raises significant robustness concerns. While existing effective attacks always focus on task-specific white-box settings, these approaches are limited in the context of LVLMs, which are designed for diverse downstream tasks and require expensive full-model gradient computations. Motivated by the pivotal role and wide adoption of the vision encoder in LVLMs, we propose a simple yet effective Vision Encoder Attack (VEAttack), which targets the vision encoder of LVLMs only. Specifically, we propose to generate adversarial examples by minimizing the cosine similarity between the clean and perturbed visual features, without accessing the following large language models, task information, and labels. It significantly reduces the computational overhead while eliminating the task and label dependence of traditional white-box attacks in LVLMs. To make this simple attack effective, we propose to perturb images by optimizing image tokens instead of the classification token. We provide both empirical and theoretical evidence that VEAttack can easily generalize to various tasks. VEAttack has achieved a performance degradation of 94.5% on image caption task and 75.7% on visual question answering task. We also reveal some key observations to provide insights into LVLM attack/defense: 1) hidden layer variations of LLM, 2) token attention differential, 3) Möbius band in transfer attack, 4) low sensitivity to attack steps. The code is available at this https URL
• 摘要：大型视觉模型（LVLM）在多模式的理解和产生中表现出显着的能力，但是它们对对抗性攻击的脆弱性引起了重大的鲁棒性关注。尽管现有的有效攻击始终集中在特定于任务的白色盒子设置上，但这些方法在LVLM的背景下受到限制，该方法是针对各种下游任务而设计的，需要昂贵的全模型梯度计算。由LVLM中的视觉编码器的关键作用和广泛采用的动机，我们提出了一种简单而有效的视觉编码器攻击（VEATTACK），该攻击仅针对LVLMS的视觉编码器。具体而言，我们建议通过最大程度地减少干净和干扰视觉特征之间的余弦相似性来生成对抗性示例，而无需访问以下大型语言模型，任务信息和标签。它大大降低了计算开销，同时消除了LVLMS中传统白盒攻击的任务和标签依赖性。为了使这种简单的攻击有效，我们建议通过优化图像令牌而不是分类令牌来扰动图像。我们提供经验和理论证据，即Veattack可以轻松地将其推广到各种任务。 VeatTack在图像字幕任务上达到了94.5％的性能降解，视觉问题回答任务的降级为75.7％。我们还揭示了一些关键观察，以提供有关LVLM攻击/防御的见解：1）LLM的隐藏层变化，2）令牌注意力差异，3）MöbiusBand在转移攻击中，4）对攻击步骤的低灵敏度。该代码可在此HTTPS URL上找到

Title: Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression

• Authors: Yuning Shen, Lihao Wang, Huizhuo Yuan, Yan Wang, Bangji Yang, Quanquan Gu
• Subjects: cs.LG, cs.AI, physics.bio-ph, q-bio.BM, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2505.17478
• Pdf URL: https://arxiv.org/pdf/2505.17478
• Copy Paste: [[2505.17478]] Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression(https://arxiv.org/abs/2505.17478)
• Keywords: generation, generative
• Abstract: Understanding protein dynamics is critical for elucidating their biological functions. The increasing availability of molecular dynamics (MD) data enables the training of deep generative models to efficiently explore the conformational space of proteins. However, existing approaches either fail to explicitly capture the temporal dependencies between conformations or do not support direct generation of time-independent samples. To address these limitations, we introduce ConfRover, an autoregressive model that simultaneously learns protein conformation and dynamics from MD trajectories, supporting both time-dependent and time-independent sampling. At the core of our model is a modular architecture comprising: (i) an encoding layer, adapted from protein folding models, that embeds protein-specific information and conformation at each time frame into a latent space; (ii) a temporal module, a sequence model that captures conformational dynamics across frames; and (iii) an SE(3) diffusion model as the structure decoder, generating conformations in continuous space. Experiments on ATLAS, a large-scale protein MD dataset of diverse structures, demonstrate the effectiveness of our model in learning conformational dynamics and supporting a wide range of downstream tasks. ConfRover is the first model to sample both protein conformations and trajectories within a single framework, offering a novel and flexible approach for learning from protein MD data.
• 摘要：了解蛋白质动力学对于阐明其生物学功能至关重要。分子动力学（MD）数据的可用性增加，可以训练深层生成模型，以有效探索蛋白质的构象空间。但是，现有方法要么无法明确捕获构象之间的时间依赖性，要么不支持时间无关的样品的直接生成。为了解决这些局限性，我们介绍了Confrover，这是一种自回归模型，同时从MD轨迹中学习蛋白质构象和动力学，从而支持时间依赖性和时间无关的采样。我们模型的核心是一个模块化体系结构，其中包括：（i）一个编码层，该层改编自蛋白质折叠模型，该模型将蛋白质特异性信息和构象嵌入每个时间范围； （ii）一个时间模块，一种序列模型，捕获跨帧的构象动力学； （iii）SE（3）扩散模型作为结构解码器，在连续空间中产生构象。大规模蛋白质MD数据集Atlas上的实验证明了我们模型在学习构象动力学和支持各种下游任务方面的有效性。 Confrover是第一个在单个框架内采样蛋白质构象和轨迹的模型，提供了一种从蛋白质MD数据中学习的新颖而灵活的方法。

Title: What You Read Isn't What You Hear: Linguistic Sensitivity in Deepfake Speech Detection

• Authors: Binh Nguyen, Shuji Shi, Ryan Ofman, Thai Le
• Subjects: cs.LG, cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2505.17513
• Pdf URL: https://arxiv.org/pdf/2505.17513
• Copy Paste: [[2505.17513]] What You Read Isn't What You Hear: Linguistic Sensitivity in Deepfake Speech Detection(https://arxiv.org/abs/2505.17513)
• Keywords: generation
• Abstract: Recent advances in text-to-speech technologies have enabled realistic voice generation, fueling audio-based deepfake attacks such as fraud and impersonation. While audio anti-spoofing systems are critical for detecting such threats, prior work has predominantly focused on acoustic-level perturbations, leaving the impact of linguistic variation largely unexplored. In this paper, we investigate the linguistic sensitivity of both open-source and commercial anti-spoofing detectors by introducing transcript-level adversarial attacks. Our extensive evaluation reveals that even minor linguistic perturbations can significantly degrade detection accuracy: attack success rates surpass 60% on several open-source detector-voice pairs, and notably one commercial detection accuracy drops from 100% on synthetic audio to just 32%. Through a comprehensive feature attribution analysis, we identify that both linguistic complexity and model-level audio embedding similarity contribute strongly to detector vulnerability. We further demonstrate the real-world risk via a case study replicating the Brad Pitt audio deepfake scam, using transcript adversarial attacks to completely bypass commercial detectors. These results highlight the need to move beyond purely acoustic defenses and account for linguistic variation in the design of robust anti-spoofing systems. All source code will be publicly available.
• 摘要：文本到语音技术的最新进展使现实的语音产生，助长了基于音频的深层攻击，例如欺诈和模仿。尽管音频反欺骗系统对于检测这种威胁至关重要，但先前的工作主要集中在声学级别的扰动上，从而使语言差异的影响在很大程度上没有探索。在本文中，我们通过引入笔录级对抗性攻击来研究开源和商业反欺骗检测器的语言敏感性。我们广泛的评估表明，即使是较小的语言扰动也可以显着降低检测准确性：攻击成功率在几个开源探测器配对上超过60％，尤其是一种商业检测准确性从合成音频的100％下降到仅32％。通过全面的特征归因分析，我们确定语言复杂性和模型级音频嵌入相似性都对检测器脆弱性有很大贡献。我们通过案例研究进一步证明了现实世界中的风险，该案例研究复制了Brad Pitt Audio Deepfake骗局，并使用转录本的对抗攻击来完全绕过商业探测器。这些结果表明，有必要超越纯声防御能力，并解释强大的反欺骗系统设计中语言差异。所有源代码将公开可用。

Title: Spacetime Geometry of Denoising in Diffusion Models

• Authors: Rafał Karczewski, Markus Heinonen, Alison Pouplin, Søren Hauberg, Vikas Garg
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17517
• Pdf URL: https://arxiv.org/pdf/2505.17517
• Copy Paste: [[2505.17517]] Spacetime Geometry of Denoising in Diffusion Models(https://arxiv.org/abs/2505.17517)
• Keywords: generation
• Abstract: We present a novel perspective on diffusion models using the framework of information geometry. We show that the set of noisy samples, taken across all noise levels simultaneously, forms a statistical manifold -- a family of denoising probability distributions. Interpreting the noise level as a temporal parameter, we refer to this manifold as spacetime. This manifold naturally carries a Fisher-Rao metric, which defines geodesics -- shortest paths between noisy points. Notably, this family of distributions is exponential, enabling efficient geodesic computation even in high-dimensional settings without retraining or fine-tuning. We demonstrate the practical value of this geometric viewpoint in transition path sampling, where spacetime geodesics define smooth sequences of Boltzmann distributions, enabling the generation of continuous trajectories between low-energy metastable states. Code is available at: this https URL.
• 摘要：我们使用信息几何形状框架对扩散模型进行了新的视角。我们表明，同时在所有噪声水平上采集的一组嘈杂的样本构成了统计歧管，这是一个剥夺概率分布的家族。将噪声水平解释为时间参数，我们将此歧管称为时空。该歧管自然带有一个Fisher-Rao度量，该指标定义了大地测量学 - 嘈杂点之间的最短路径。值得注意的是，这个分布家族是指数级的，即使在高维设置中，也可以在不进行重新调整或微调的情况下实现有效的测量计算。我们证明了这种几何观点在过渡路径采样中的实用值，其中时空测量学定义了玻尔兹曼分布的平滑序列，从而使低能肌稳定状态之间的连续轨迹产生。代码可用：此HTTPS URL。

Title: Co-Reinforcement Learning for Unified Multimodal Understanding and Generation

• Authors: Jingjing Jiang, Chongjie Si, Jun Luo, Hanwang Zhang, Chao Ma
• Subjects: cs.CV, cs.CL, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.17534
• Pdf URL: https://arxiv.org/pdf/2505.17534
• Copy Paste: [[2505.17534]] Co-Reinforcement Learning for Unified Multimodal Understanding and Generation(https://arxiv.org/abs/2505.17534)
• Keywords: generation
• Abstract: This paper presents a pioneering exploration of reinforcement learning (RL) via group relative policy optimization for unified multimodal large language models (ULMs), aimed at simultaneously reinforcing generation and understanding capabilities. Through systematic pilot studies, we uncover the significant potential of ULMs to enable the synergistic co-evolution of dual capabilities within a shared policy optimization framework. Building on this insight, we introduce \textbf{CoRL}, a co-reinforcement learning framework comprising a unified RL stage for joint optimization and a refined RL stage for task-specific enhancement. With the proposed CoRL, our resulting model, \textbf{ULM-R1}, achieves average improvements of \textbf{7%} on three text-to-image generation datasets and \textbf{23%} on nine multimodal understanding benchmarks. These results demonstrate the effectiveness of CoRL and highlight the substantial benefit of reinforcement learning in facilitating cross-task synergy and optimization for ULMs.
• 摘要：本文通过统一多模式大型语言模型（ULMS）的小组相对政策优化对加强学习（RL）进行了开创性的探索，旨在同时增强产生和理解能力。通过系统的试点研究，我们发现了ULMS在共享政策优化框架内实现双重能力的协同共同发展的巨大潜力。在此洞察力的基础上，我们介绍了\ textbf {corl}，这是一个共同增强的学习框架，该框架包括一个统一的RL阶段，用于关节优化，以及用于特定任务增强的精制RL阶段。借助提出的CORL，我们所得的模型\ textBf {ulm-r1}可以在三个文本到图像生成数据集上的平均改进和\ textbf {23％}上的平均改进。这些结果证明了CORL的有效性，并突出了增强学习在促进跨任务协同作用和对ULM的优化方面的实质性好处。

Title: RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning

• Authors: Mingrui Wu, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, Rongrong Ji
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17540
• Pdf URL: https://arxiv.org/pdf/2505.17540
• Copy Paste: [[2505.17540]] RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning(https://arxiv.org/abs/2505.17540)
• Keywords: generation
• Abstract: Despite recent progress in text-to-image (T2I) generation, existing models often struggle to faithfully capture user intentions from short and under-specified prompts. While prior work has attempted to enhance prompts using large language models (LLMs), these methods frequently generate stylistic or unrealistic content due to insufficient grounding in visual semantics and real-world composition. Inspired by recent advances in reasoning for language model, we propose RePrompt, a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning. Instead of relying on handcrafted rules or stylistic rewrites, our method trains a language model to generate structured, self-reflective prompts by optimizing for image-level outcomes. The tailored reward models assesse the generated images in terms of human preference, semantic alignment, and visual composition, providing indirect supervision to refine prompt generation. Our approach enables end-to-end training without human-annotated data. Experiments on GenEval and T2I-Compbench show that RePrompt significantly boosts spatial layout fidelity and compositional generalization across diverse T2I backbones, establishing new state-of-the-art results.
• 摘要：尽管最近的文本图像（T2I）一代取得了进展，但现有模型通常很难忠实地从简短和未指定的提示中捕获用户意图。尽管先前的工作试图使用大型语言模型（LLM）增强提示，但由于视觉语义和现实世界中的接地不足，这些方法经常产生风格或不切实际的内容。受语言模型推理推理的最新进展的启发，我们提出了Reprompt，这是一个新颖的重复框架，通过增强学习将明确的推理引入了及时的增强过程中。我们的方法不依赖手工制作的规则或风格改写，而是训练一种语言模型来通过优化图像级别的结果来生成结构化的自我反射提示。量身定制的奖励模型根据人类的偏好，语义一致性和视觉组成评估了生成的图像，从而提供了间接的监督来完善及时生成。我们的方法可以实现无人通知数据的端到端培训。关于Geneval和T2i-Compbench的实验表明，重复重复可显着提高各种T2I骨架之间的空间布局保真度和组成概括，从而确立新的最新结果。

Title: Graph Style Transfer for Counterfactual Explainability

• Authors: Bardh Prenkaj, Efstratios Zaradoukas, Gjergji Kasneci
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17542
• Pdf URL: https://arxiv.org/pdf/2505.17542
• Copy Paste: [[2505.17542]] Graph Style Transfer for Counterfactual Explainability(https://arxiv.org/abs/2505.17542)
• Keywords: generation
• Abstract: Counterfactual explainability seeks to uncover model decisions by identifying minimal changes to the input that alter the predicted outcome. This task becomes particularly challenging for graph data due to preserving structural integrity and semantic meaning. Unlike prior approaches that rely on forward perturbation mechanisms, we introduce Graph Inverse Style Transfer (GIST), the first framework to re-imagine graph counterfactual generation as a backtracking process, leveraging spectral style transfer. By aligning the global structure with the original input spectrum and preserving local content faithfulness, GIST produces valid counterfactuals as interpolations between the input style and counterfactual content. Tested on 8 binary and multi-class graph classification benchmarks, GIST achieves a remarkable +7.6% improvement in the validity of produced counterfactuals and significant gains (+45.5%) in faithfully explaining the true class distribution. Additionally, GIST's backtracking mechanism effectively mitigates overshooting the underlying predictor's decision boundary, minimizing the spectral differences between the input and the counterfactuals. These results challenge traditional forward perturbation methods, offering a novel perspective that advances graph explainability.
• 摘要：反事实解释性旨在通过确定改变预测结果的输入的最小变化来发现模型决策。由于保留结构完整性和语义含义，对于图形数据而言，此任务变得尤其具有挑战性。与依靠前进扰动机制的先前方法不同，我们引入了图形逆向传输（GIST），这是重新想象图形反事实生成的第一个框架，作为回溯过程，利用光谱样式传输。通过将全球结构与原始输入频谱保持一致并保留本地内容的忠诚，GIST会产生有效的反事实，作为输入样式和反事实内容之间的插值。在8个二元和多级图分类基准测试基准下，GIST在忠实地解释真实的类别分布方面，产生的反事实和显着增长（ +45.5％）的有效性提高了7.6％。此外，GIST的回溯机制有效地减轻了基础预测器的决策边界的超越，从而最大程度地减少了输入和反事实之间的频谱差异。这些结果挑战了传统的前向扰动方法，提供了一种新颖的观点，可以提高图形解释性。

Title: T2VUnlearning: A Concept Erasing Method for Text-to-Video Diffusion Models

• Authors: Xiaoyu Ye, Songjie Cheng, Yongtao Wang, Yajiao Xiong, Yishen Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17550
• Pdf URL: https://arxiv.org/pdf/2505.17550
• Copy Paste: [[2505.17550]] T2VUnlearning: A Concept Erasing Method for Text-to-Video Diffusion Models(https://arxiv.org/abs/2505.17550)
• Keywords: generation
• Abstract: Recent advances in text-to-video (T2V) diffusion models have significantly enhanced the quality of generated videos. However, their ability to produce explicit or harmful content raises concerns about misuse and potential rights violations. Inspired by the success of unlearning techniques in erasing undesirable concepts from text-to-image (T2I) models, we extend unlearning to T2V models and propose a robust and precise unlearning method. Specifically, we adopt negatively-guided velocity prediction fine-tuning and enhance it with prompt augmentation to ensure robustness against LLM-refined prompts. To achieve precise unlearning, we incorporate a localization and a preservation regularization to preserve the model's ability to generate non-target concepts. Extensive experiments demonstrate that our method effectively erases a specific concept while preserving the model's generation capability for all other concepts, outperforming existing methods. We provide the unlearned models in \href{this https URL}{this https URL}.
• 摘要：文本到视频（T2V）扩散模型的最新进展显着提高了生成的视频的质量。但是，它们产生明确或有害内容的能力引起了人们对滥用和潜在权利的关注。受到未学习技术在从文本到图像（T2I）模型中删除不良概念的成功的启发，我们将未学习的文化扩展到T2V模型，并提出了一种坚固且精确的未学习方法。具体而言，我们采用负面速度预测微调并通过迅速增强来增强它，以确保针对LLM改良提示的鲁棒性。为了实现精确的学习，我们结合了定位和保存正则化，以保留模型生成非目标概念的能力。广泛的实验表明，我们的方法有效地擦除了一个特定的概念，同时保留了模型的所有其他概念的生成能力，表现优于现有方法。我们在\ href {this https url} {this HTTPS url}中提供了未学习的模型。

Title: Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing

• Authors: Zijie Qiu, Jiaqi Wei, Xiang Zhang, Sheng Xu, Kai Zou, Zhi Jin, Zhiqiang Gao, Nanqing Dong, Siqi Sun
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17552
• Pdf URL: https://arxiv.org/pdf/2505.17552
• Copy Paste: [[2505.17552]] Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing(https://arxiv.org/abs/2505.17552)
• Keywords: generation
• Abstract: De novo peptide sequencing is a critical task in proteomics. However, the performance of current deep learning-based methods is limited by the inherent complexity of mass spectrometry data and the heterogeneous distribution of noise signals, leading to data-specific biases. We present RankNovo, the first deep reranking framework that enhances de novo peptide sequencing by leveraging the complementary strengths of multiple sequencing models. RankNovo employs a list-wise reranking approach, modeling candidate peptides as multiple sequence alignments and utilizing axial attention to extract informative features across candidates. Additionally, we introduce two new metrics, PMD (Peptide Mass Deviation) and RMD (residual Mass Deviation), which offer delicate supervision by quantifying mass differences between peptides at both the sequence and residue levels. Extensive experiments demonstrate that RankNovo not only surpasses its base models used to generate training candidates for reranking pre-training, but also sets a new state-of-the-art benchmark. Moreover, RankNovo exhibits strong zero-shot generalization to unseen models whose generations were not exposed during training, highlighting its robustness and potential as a universal reranking framework for peptide sequencing. Our work presents a novel reranking strategy that fundamentally challenges existing single-model paradigms and advances the frontier of accurate de novo sequencing. Our source code is provided on GitHub.
• 摘要：从头肽测序是蛋白质组学中的关键任务。但是，当前基于深度学习的方法的性能受到质谱数据的固有复杂性和噪声信号的异质分布的限制，从而导致特定于数据的偏见。我们提出了Ranknovo，这是第一个通过利用多个测序模型的互补强度来增强从头肽测序的第一个深层重读框架。 Ranknovo采用列表的重读方法，将候选肽建模为多个序列比对，并利用轴向注意来提取各个候选者的信息特征。此外，我们介绍了两个新的指标，PMD（肽质量偏差）和RMD（残留质量偏差），它们通过量化序列和残基水平的肽之间的质量差异来提供微妙的监督。广泛的实验表明，Ranknovo不仅超过了其基本模型，该模型用于生成训练候选者，用于重新训练预训练，而且还设定了新的最新基准。此外，兰克诺沃（Ranknovo）对看不见的模型表现出强烈的零拍概括，这些模型在训练过程中没有暴露出来，突出了其稳健性和潜力，作为肽测序的通用reranking框架。我们的工作提出了一种新颖的重新策略，从根本上挑战了现有的单模范式，并提高了准确的从头测序的前沿。我们的源代码在GitHub上提供。

Title: Deeper Diffusion Models Amplify Bias

• Authors: Shahin Hakemi, Naveed Akhtar, Ghulam Mubashar Hassan, Ajmal Mian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17560
• Pdf URL: https://arxiv.org/pdf/2505.17560
• Copy Paste: [[2505.17560]] Deeper Diffusion Models Amplify Bias(https://arxiv.org/abs/2505.17560)
• Keywords: generation, generative
• Abstract: Despite the impressive performance of generative Diffusion Models (DMs), their internal working is still not well understood, which is potentially problematic. This paper focuses on exploring the important notion of bias-variance tradeoff in diffusion models. Providing a systematic foundation for this exploration, it establishes that at one extreme the diffusion models may amplify the inherent bias in the training data and, on the other, they may compromise the presumed privacy of the training samples. Our exploration aligns with the memorization-generalization understanding of the generative models, but it also expands further along this spectrum beyond ``generalization'', revealing the risk of bias amplification in deeper models. Building on the insights, we also introduce a training-free method to improve output quality in text-to-image and image-to-image generation. By progressively encouraging temporary high variance in the generation process with partial bypassing of the mid-block's contribution in the denoising process of DMs, our method consistently improves generative image quality with zero training cost. Our claims are validated both theoretically and empirically.
• 摘要：尽管生成扩散模型（DMS）的表现令人印象深刻，但它们的内部工作仍未得到充分了解，这可能是有问题的。本文着重于探讨扩散模型中偏见变化权衡的重要概念。为这项探索提供了系统的基础，它确定了一个极端的扩散模型可能会扩大培训数据中的固有偏见，而另一方面，它们可能会损害培训样本的假定隐私。我们的探索与对生成模型的记忆中的理解相吻合，但它也沿着该频谱进一步扩展了``概括''，从而揭示了在更深层次的模型中偏向放大的风险。在洞察力的基础上，我们还引入了一种无培训的方法，以提高文本对图像和图像形象生成的产量质量。通过逐步鼓励生成过程中的暂时较高的差异，并部分绕过中间块在DMS的转化过程中的贡献，我们的方法始终以零培训成本提高生成图像质量。我们的主张在理论上和经验上都得到了验证。

Title: Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model

• Authors: Kwanyoung Kim, Sanghyun Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17561
• Pdf URL: https://arxiv.org/pdf/2505.17561
• Copy Paste: [[2505.17561]] Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model(https://arxiv.org/abs/2505.17561)
• Keywords: generation
• Abstract: The choice of initial noise significantly affects the quality and prompt alignment of video diffusion models, where different noise seeds for the same prompt can lead to drastically different generations. While recent methods rely on externally designed priors such as frequency filters or inter-frame smoothing, they often overlook internal model signals that indicate which noise seeds are inherently preferable. To address this, we propose ANSE (Active Noise Selection for Generation), a model-aware framework that selects high-quality noise seeds by quantifying attention-based uncertainty. At its core is BANSA (Bayesian Active Noise Selection via Attention), an acquisition function that measures entropy disagreement across multiple stochastic attention samples to estimate model confidence and consistency. For efficient inference-time deployment, we introduce a Bernoulli-masked approximation of BANSA that enables score estimation using a single diffusion step and a subset of attention layers. Experiments on CogVideoX-2B and 5B demonstrate that ANSE improves video quality and temporal coherence with only an 8% and 13% increase in inference time, respectively, providing a principled and generalizable approach to noise selection in video diffusion. See our project page: this https URL
• 摘要：初始噪声的选择显着影响视频扩散模型的质量和迅速对齐，在同一提示中，不同的噪声种子可能会导致截然不同的世代。尽管最近的方法依赖于外部设计的先验，例如频过滤器或框架间平滑，但它们通常会忽略内部模型信号，这些信号表明哪些噪声种子本质上是可取的。为了解决这个问题，我们提出了ANSE（Anenthing Andies选择），这是一种模型感知的框架，通过量化基于注意力的不确定性来选择高质量的噪声种子。 BANSA的核心是（通过注意力选择贝叶斯主动噪声），这是一种采集函数，可测量多个随机注意样本的熵分歧，以估计模型的置信度和一致性。为了有效的推理时间部署，我们引入了BERNOULLI掩盖的BANSA近似值，该近似可以使用单个扩散步骤和注意力层的子集实现得分估计。关于Cogvideox-2b和5b的实验表明，ANSE分别提高了视频质量和时间连贯性，而推理时间仅增加了8％和13％，从而提供了一种原则性且可推广的视频扩散中噪声选择方法。请参阅我们的项目页面：此HTTPS URL

Title: Enhancing Fourier-based Doppler Resolution with Diffusion Models

• Authors: Denisa Qosja, Kilian Barth, Simon Wagner
• Subjects: cs.CV, eess.SP
• Abstract URL: https://arxiv.org/abs/2505.17567
• Pdf URL: https://arxiv.org/pdf/2505.17567
• Copy Paste: [[2505.17567]] Enhancing Fourier-based Doppler Resolution with Diffusion Models(https://arxiv.org/abs/2505.17567)
• Keywords: generative
• Abstract: In radar systems, high resolution in the Doppler dimension is important for detecting slow-moving targets as it allows for more distinct separation between these targets and clutter, or stationary objects. However, achieving sufficient resolution is constrained by hardware capabilities and physical factors, leading to the development of processing techniques to enhance the resolution after acquisition. In this work, we leverage artificial intelligence to increase the Doppler resolution in range-Doppler maps. Based on a zero-padded FFT, a refinement via the generative neural networks of diffusion models is achieved. We demonstrate that our method overcomes the limitations of traditional FFT, generating data where closely spaced targets are effectively separated.
• 摘要：在雷达系统中，多普勒维度中的高分辨率对于检测缓慢移动目标很重要，因为它可以在这些目标和混乱或固定物体之间进行更明显的分离。但是，实现足够的分辨率受到硬件能力和物理因素的限制，从而导致加工技术的发展以增强收购后的分辨率。在这项工作中，我们利用人工智能来增加范围多普勒图中的多普勒分辨率。基于零填充的FFT，通过扩散模型的生成神经网络进行了完善。我们证明我们的方法克服了传统FFT的局限性，从而生成了有效分离目标的数据。

Title: InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO

• Authors: Xueji Fang, Liyuan Ma, Zhiyang Chen, Mingyuan Zhou, Guo-jun Qi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17574
• Pdf URL: https://arxiv.org/pdf/2505.17574
• Copy Paste: [[2505.17574]] InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO(https://arxiv.org/abs/2505.17574)
• Keywords: generation
• Abstract: Recent advances in text-to-video generation, particularly with autoregressive models, have enabled the synthesis of high-quality videos depicting individual scenes. However, extending these models to generate long, cross-scene videos remains a significant challenge. As the context length grows during autoregressive decoding, computational costs rise sharply, and the model's ability to maintain consistency and adhere to evolving textual prompts deteriorates. We introduce InfLVG, an inference-time framework that enables coherent long video generation without requiring additional long-form video data. InfLVG leverages a learnable context selection policy, optimized via Group Relative Policy Optimization (GRPO), to dynamically identify and retain the most semantically relevant context throughout the generation process. Instead of accumulating the entire generation history, the policy ranks and selects the top-$K$ most contextually relevant tokens, allowing the model to maintain a fixed computational budget while preserving content consistency and prompt alignment. To optimize the policy, we design a hybrid reward function that jointly captures semantic alignment, cross-scene consistency, and artifact reduction. To benchmark performance, we introduce the Cross-scene Video Benchmark (CsVBench) along with an Event Prompt Set (EPS) that simulates complex multi-scene transitions involving shared subjects and varied actions/backgrounds. Experimental results show that InfLVG can extend video length by up to 9$\times$, achieving strong consistency and semantic fidelity across scenes. Our code is available at this https URL.
• 摘要：文本到视频生成的最新进展，尤其是在自回归模型的情况下，已经使描绘了各个场景的高质量视频综合了。但是，扩展这些模型以产生长期的跨场景视频仍然是一个重大挑战。随着自回归解码期间上下文长度的增长，计算成本急剧上升，并且模型保持一致性并遵守不断发展的文本提示的能力会导致变质。我们介绍了Inflvg，这是一个推理时间框架，可实现连贯的长视频生成，而无需额外的长期视频数据。 Affvg利用通过小组相对策略优化（GRPO）优化的可学习上下文选择策略，以动态识别并保留整个生成过程中最相关的上下文。该策略没有累积整个一代历史，而是排名最高的$ k $最相关的代币，从而使模型可以维护固定的计算预算，同时保留内容一致性和及时对齐。为了优化政策，我们设计了一个混合奖励功能，该功能共同捕获语义对齐，跨场景一致性和减少工件。为了进行基准性能，我们介绍了跨场景视频基准（CSVBENCE）以及事件提示集（EPS），该设置模拟了涉及共同主题和各种操作/背景的复杂多场景过渡。实验结果表明，Inflvg可以将视频长度延长到9 $ \ times $，从而实现了强大的一致性和跨场景的语义保真度。我们的代码可在此HTTPS URL上找到。

Title: MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery

• Authors: Hainuo Wang, Qiming Hu, Xiaojie Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17581
• Pdf URL: https://arxiv.org/pdf/2505.17581
• Copy Paste: [[2505.17581]] MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery(https://arxiv.org/abs/2505.17581)
• Keywords: restoration
• Abstract: Restoring images degraded by adverse weather remains a significant challenge due to the highly non-uniform and spatially heterogeneous nature of weather-induced artifacts, e.g., fine-grained rain streaks versus widespread haze. Accurately estimating the underlying degradation can intuitively provide restoration models with more targeted and effective guidance, enabling adaptive processing strategies. To this end, we propose a Morton-Order Degradation Estimation Mechanism (MODEM) for adverse weather image restoration. Central to MODEM is the Morton-Order 2D-Selective-Scan Module (MOS2D), which integrates Morton-coded spatial ordering with selective state-space models to capture long-range dependencies while preserving local structural coherence. Complementing MOS2D, we introduce a Dual Degradation Estimation Module (DDEM) that disentangles and estimates both global and local degradation priors. These priors dynamically condition the MOS2D modules, facilitating adaptive and context-aware restoration. Extensive experiments and ablation studies demonstrate that MODEM achieves state-of-the-art results across multiple benchmarks and weather types, highlighting its effectiveness in modeling complex degradation dynamics. Our code will be released at this https URL.
• 摘要：由于天气引起的人工制品的高度不均匀和空间异质性质，例如细粒度的雨条与广泛的雾霾，恢复不利天气降解的图像仍然是一个重大挑战。准确地估算潜在的降解可以直观地为恢复模型提供更有针对性和有效的指导，从而实现自适应处理策略。为此，我们提出了一种不良天气图像恢复的降落降解估计机制（调制解调器）。调制解调器的中心是Morton-order-2D-选择性 - 扫描模块（MOS2D），它将莫尔顿编码的空间排序与选择性状态空间模型集成在一起，以捕获长期依赖性，同时保留局部结构相干性。补充MOS2D，我们引入了一个双重降解估计模块（DDEM），该模块解散和估计全球和局部退化先验。这些先验动态调节MOS2D模块，促进自适应和背景感知的恢复。广泛的实验和消融研究表明，调制解调器在多种基准和天气类型之间取得了最新的结果，突出了其在建模复杂降解动态方面的有效性。我们的代码将在此HTTPS URL上发布。

Title: CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesis

• Authors: Florian Barthel, Wieland Morgenstern, Paul Hinzer, Anna Hilsmann, Peter Eisert
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17590
• Pdf URL: https://arxiv.org/pdf/2505.17590
• Copy Paste: [[2505.17590]] CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesis(https://arxiv.org/abs/2505.17590)
• Keywords: generation
• Abstract: Recently, 3D GANs based on 3D Gaussian splatting have been proposed for high quality synthesis of human heads. However, existing methods stabilize training and enhance rendering quality from steep viewpoints by conditioning the random latent vector on the current camera position. This compromises 3D consistency, as we observe significant identity changes when re-synthesizing the 3D head with each camera shift. Conversely, fixing the camera to a single viewpoint yields high-quality renderings for that perspective but results in poor performance for novel views. Removing view-conditioning typically destabilizes GAN training, often causing the training to collapse. In response to these challenges, we introduce CGS-GAN, a novel 3D Gaussian Splatting GAN framework that enables stable training and high-quality 3D-consistent synthesis of human heads without relying on view-conditioning. To ensure training stability, we introduce a multi-view regularization technique that enhances generator convergence with minimal computational overhead. Additionally, we adapt the conditional loss used in existing 3D Gaussian splatting GANs and propose a generator architecture designed to not only stabilize training but also facilitate efficient rendering and straightforward scaling, enabling output resolutions up to $2048^2$. To evaluate the capabilities of CGS-GAN, we curate a new dataset derived from FFHQ. This dataset enables very high resolutions, focuses on larger portions of the human head, reduces view-dependent artifacts for improved 3D consistency, and excludes images where subjects are obscured by hands or other objects. As a result, our approach achieves very high rendering quality, supported by competitive FID scores, while ensuring consistent 3D scene generation. Check our our project page here: this https URL
• 摘要：最近，已经提出了基于3D高斯碎屑的3D甘甘式，用于高质量的人头合成。但是，现有的方法通过在当前相机位置调节随机潜在向量来稳定训练并从陡峭的角度提高渲染质量。这损害了3D一致性，因为我们在每个相机移动的3D头重新合成时会观察到重大的身份变化。相反，将相机固定在单个角度上可以为该角度提供高质量的效果图，但对于新颖的看法，摄像机的表现差。删除视图条件通常会破坏GAN训练的稳定，通常会导致培训崩溃。为了应对这些挑战，我们引入了CGS-GAN，这是一种新颖的3D高斯脱落GAN框架，可实现稳定的训练和高质量的3D一致综合人类头的合成，而无需依赖视图条件。为了确保训练稳定性，我们引入了一种多视图正则化技术，该技术可以增强发电机的收敛，并使用最小的计算开销。此外，我们适应了现有的3D高斯脱衣gan中使用的条件损失，并提出了一种发电机体系结构，旨在稳定训练，还可以促进有效的渲染和直截了当的缩放，使输出分辨率最高为$ 2048^2 $。为了评估CGS-GAN的功能，我们策划了一个来自FFHQ的新数据集。该数据集实现了非常高的分辨率，专注于人头的较大部分，减少了与视图相关的伪像以提高3D一致性，并排除了受试者被手或其他物体遮盖的图像。结果，我们的方法在竞争性的FID分数的支持下，达到了非常高的渲染质量，同时确保了一致的3D场景产生。在此处查看我们的项目页面：此HTTPS URL

Title: PathoSCOPE: Few-Shot Pathology Detection via Self-Supervised Contrastive Learning and Pathology-Informed Synthetic Embeddings

• Authors: Sinchee Chin, Yinuo Ma, Xiaochen Yang, Jing-Hao Xue, Wenming Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17614
• Pdf URL: https://arxiv.org/pdf/2505.17614
• Copy Paste: [[2505.17614]] PathoSCOPE: Few-Shot Pathology Detection via Self-Supervised Contrastive Learning and Pathology-Informed Synthetic Embeddings(https://arxiv.org/abs/2505.17614)
• Keywords: generation
• Abstract: Unsupervised pathology detection trains models on non-pathological data to flag deviations as pathologies, offering strong generalizability for identifying novel diseases and avoiding costly annotations. However, building reliable normality models requires vast healthy datasets, as hospitals' data is inherently biased toward symptomatic populations, while privacy regulations hinder the assembly of representative healthy cohorts. To address this limitation, we propose PathoSCOPE, a few-shot unsupervised pathology detection framework that requires only a small set of non-pathological samples (minimum 2 shots), significantly improving data efficiency. We introduce Global-Local Contrastive Loss (GLCL), comprised of a Local Contrastive Loss to reduce the variability of non-pathological embeddings and a Global Contrastive Loss to enhance the discrimination of pathological regions. We also propose a Pathology-informed Embedding Generation (PiEG) module that synthesizes pathological embeddings guided by the global loss, better exploiting the limited non-pathological samples. Evaluated on the BraTS2020 and ChestXray8 datasets, PathoSCOPE achieves state-of-the-art performance among unsupervised methods while maintaining computational efficiency (2.48 GFLOPs, 166 FPS).
• 摘要：无监督的病理检测训练非病理数据的模型将偏差作为病理标记，为识别新型疾病和避免昂贵的注释提供了强大的普遍性。但是，建立可靠的正态模型需要大量的健康数据集，因为医院的数据固有地偏向于有症状的人群，而隐私法规则阻碍了代表性健康同伙的组装。为了解决这一限制，我们提出了病原体，这是几个无监督的病理检测框架，该框架仅需要一小部分非病理样本（至少2张照片），从而显着提高了数据效率。我们引入了全球局部对比损失（GLCL），该损失由局部对比损失组成，以减少非病理嵌入的变异性和全球对比度损失，以增强病理区域的歧视。我们还提出了一个信息信息的嵌入产生（PIEG）模块，该模块综合了以全球损失为指导的病理嵌入，更好地利用有限的非病理样本。病原体在BRATS2020和CHESTXRAY8数据集上进行了评估，在保持计算效率的同时，在无监督方法之间实现了最新的性能（2.48 GFLOPS，166 fps）。

Title: Large language model as user daily behavior data generator: balancing population diversity and individual personality

• Authors: Haoxin Li, Jingtao Ding, Jiahui Gong, Yong Li
• Subjects: cs.LG, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2505.17615
• Pdf URL: https://arxiv.org/pdf/2505.17615
• Copy Paste: [[2505.17615]] Large language model as user daily behavior data generator: balancing population diversity and individual personality(https://arxiv.org/abs/2505.17615)
• Keywords: generation
• Abstract: Predicting human daily behavior is challenging due to the complexity of routine patterns and short-term fluctuations. While data-driven models have improved behavior prediction by leveraging empirical data from various platforms and devices, the reliance on sensitive, large-scale user data raises privacy concerns and limits data availability. Synthetic data generation has emerged as a promising solution, though existing methods are often limited to specific applications. In this work, we introduce BehaviorGen, a framework that uses large language models (LLMs) to generate high-quality synthetic behavior data. By simulating user behavior based on profiles and real events, BehaviorGen supports data augmentation and replacement in behavior prediction models. We evaluate its performance in scenarios such as pertaining augmentation, fine-tuning replacement, and fine-tuning augmentation, achieving significant improvements in human mobility and smartphone usage predictions, with gains of up to 18.9%. Our results demonstrate the potential of BehaviorGen to enhance user behavior modeling through flexible and privacy-preserving synthetic data generation.
• 摘要：由于常规模式和短期波动的复杂性，预测人类日常行为是具有挑战性的。尽管数据驱动的模型通过利用来自各种平台和设备的经验数据改善了行为预测，但依赖敏感的大规模用户数据引起了隐私问题，并限制了数据可用性。尽管现有方法通常仅限于特定的应用程序，但合成数据生成已成为有前途的解决方案。在这项工作中，我们介绍了行为，该框架使用大型语言模型（LLMS）生成高质量的合成行为数据。通过基于配置文件和真实事件模拟用户行为，行为GEN支持行为预测模型中的数据增强和替换。我们在诸如增强，微调替代品和微调增强方面的情况下评估了其性能，从而取得了显着改善人类移动性和智能手机使用预测，增长高达18.9％。我们的结果证明了行为的潜力通过灵活和隐私的合成数据生成来增强用户行为建模。

Title: Scaling Image and Video Generation via Test-Time Evolutionary Search

• Authors: Haoran He, Jiajun Liang, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Ling Pan
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17618
• Pdf URL: https://arxiv.org/pdf/2505.17618
• Copy Paste: [[2505.17618]] Scaling Image and Video Generation via Test-Time Evolutionary Search(https://arxiv.org/abs/2505.17618)
• Keywords: generation, generative
• Abstract: As the marginal cost of scaling computation (data and parameters) during model pre-training continues to increase substantially, test-time scaling (TTS) has emerged as a promising direction for improving generative model performance by allocating additional computation at inference time. While TTS has demonstrated significant success across multiple language tasks, there remains a notable gap in understanding the test-time scaling behaviors of image and video generative models (diffusion-based or flow-based models). Although recent works have initiated exploration into inference-time strategies for vision tasks, these approaches face critical limitations: being constrained to task-specific domains, exhibiting poor scalability, or falling into reward over-optimization that sacrifices sample diversity. In this paper, we propose \textbf{Evo}lutionary \textbf{Search} (EvoSearch), a novel, generalist, and efficient TTS method that effectively enhances the scalability of both image and video generation across diffusion and flow models, without requiring additional training or model expansion. EvoSearch reformulates test-time scaling for diffusion and flow models as an evolutionary search problem, leveraging principles from biological evolution to efficiently explore and refine the denoising trajectory. By incorporating carefully designed selection and mutation mechanisms tailored to the stochastic differential equation denoising process, EvoSearch iteratively generates higher-quality offspring while preserving population diversity. Through extensive evaluation across both diffusion and flow architectures for image and video generation tasks, we demonstrate that our method consistently outperforms existing approaches, achieves higher diversity, and shows strong generalizability to unseen evaluation metrics. Our project is available at the website this https URL.
• 摘要：随着模型预训练期间缩放计算（数据和参数）的边际成本继续大大增加，测试时间缩放（TTS）已成为通过在推理时分配其他计算来改善生成模型性能的有希望的方向。尽管TTS在多种语言任务之间取得了显着的成功，但在理解图像和视频生成模型的测试时间缩放行为（基于扩散或基于流程的模型）方面仍然存在显着差距。尽管最近的工作已经开始探索视力任务的推理时间策略，但这些方法面临着临界局限性：被限制在特定于任务的领域，表现出较差的可伸缩性或陷入奖励过度优化，从而牺牲了样本多样性。在本文中，我们提出了\ textbf {evo} lutionary \ textbf {search}（evosearch），这是一种新颖，通才和有效的TTS方法，可有效增强跨扩散和流程模型的图像和视频生成的可扩展性，而无需进行其他培训或模型扩展。 EvoSearch将扩散和流模型的测试时间缩放重新定义为进化搜索问题，从生物进化中利用原理来有效地探索和完善脱氧轨迹。通过纳入针对随机微分方程降级过程量身定制的精心设计的选择和突变机制，EvoSearch迭代会产生更高质量的后代，同时保持人口多样性。通过对图像和视频生成任务的扩散和流架构进行广泛的评估，我们证明我们的方法始终超过现有方法，实现了更高的多样性，并显示出强大的普遍性对于看不见的评估指标。我们的项目可在此HTTPS URL网站上找到。

Title: CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment

• Authors: Bo Wang, De-Xing Huang, Xiao-Hu Zhou, Mei-Jiang Gui, Nu-Fang Xiao, Jian-Long Hao, Ming-Yuan Liu, Zeng-Guang Hou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17619
• Pdf URL: https://arxiv.org/pdf/2505.17619
• Copy Paste: [[2505.17619]] CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment(https://arxiv.org/abs/2505.17619)
• Keywords: generative, quality assessment
• Abstract: Synthetic X-ray angiographies generated by modern generative models hold great potential to reduce the use of contrast agents in vascular interventional procedures. However, low-quality synthetic angiographies can significantly increase procedural risk, underscoring the need for reliable image quality assessment (IQA) methods. Existing IQA models, however, fail to leverage auxiliary images as references during evaluation and lack fine-grained, task-specific metrics necessary for clinical relevance. To address these limitations, this paper proposes CAS-IQA, a vision-language model (VLM)-based framework that predicts fine-grained quality scores by effectively incorporating auxiliary information from related images. In the absence of angiography datasets, CAS-3K is constructed, comprising 3,565 synthetic angiographies along with score annotations. To ensure clinically meaningful assessment, three task-specific evaluation metrics are defined. Furthermore, a Multi-path featUre fuSion and rouTing (MUST) module is designed to enhance image representations by adaptively fusing and routing visual tokens to metric-specific branches. Extensive experiments on the CAS-3K dataset demonstrate that CAS-IQA significantly outperforms state-of-the-art IQA methods by a considerable margin.
• 摘要：现代生成模型产生的合成X射线血管造影具有巨大的潜力，可以减少对比度在血管介入过程中的使用。但是，低质量的合成血管造影可以显着增加程序风险，强调对可靠图像质量评估（IQA）方法的需求。但是，现有的IQA模型无法利用辅助图像作为评估期间的参考，并且缺乏临床相关性所需的细粒度，特定于任务的指标。为了解决这些局限性，本文提出了CAS-IQA，这是一个基于视觉语言模型（VLM）的框架，该框架通过有效地合并了相关图像中的辅助信息来预测细粒度的质量得分。在没有血管造影数据集的情况下，构建CAS-3K，包括3,565个合成血管造影以及分数注释。为了确保临床有意义的评估，定义了三个特定于任务的评估指标。此外，多条路径特征融合和路由（必须）模块旨在通过自适应融合和路由视觉令牌与指标特异性分支来增强图像表示。在CAS-3K数据集上进行的广泛实验表明，CAS-IQA以相当大的余量优于最先进的IQA方法。

Title: Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training

• Authors: Tony Bonnaire, Raphaël Urfin, Giulio Biroli, Marc Mézard
• Subjects: cs.LG, cond-mat.dis-nn, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.17638
• Pdf URL: https://arxiv.org/pdf/2505.17638
• Copy Paste: [[2505.17638]] Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training(https://arxiv.org/abs/2505.17638)
• Keywords: generative
• Abstract: Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of training data and allow generalization. In this work, we investigate the role of the training dynamics in the transition from generalization to memorization. Through extensive experiments and theoretical analysis, we identify two distinct timescales: an early time $\tau_\mathrm{gen}$ at which models begin to generate high-quality samples, and a later time $\tau_\mathrm{mem}$ beyond which memorization emerges. Crucially, we find that $\tau_\mathrm{mem}$ increases linearly with the training set size $n$, while $\tau_\mathrm{gen}$ remains constant. This creates a growing window of training times with $n$ where models generalize effectively, despite showing strong memorization if training continues beyond it. It is only when $n$ becomes larger than a model-dependent threshold that overfitting disappears at infinite training times. These findings reveal a form of implicit dynamical regularization in the training dynamics, which allow to avoid memorization even in highly overparameterized settings. Our results are supported by numerical experiments with standard U-Net architectures on realistic and synthetic datasets, and by a theoretical analysis using a tractable random features model studied in the high-dimensional limit.
• 摘要：扩散模型在各种生成任务中取得了巨大的成功。一个关键的挑战是了解阻止其记忆训练数据并允许概括的机制。在这项工作中，我们研究了训练动力学在从概括到记忆的过渡中的作用。通过广泛的实验和理论分析，我们确定了两个不同的时间表：早期$ \ tau_ \ mathrm {genRm {gen} $，其中模型开始生成高质量的样本，而较晚的时间$ \ tau_ \ tau_ \ mathrm {mem} $以后出现了记忆。至关重要的是，我们发现$ \ tau_ \ mathrm {mem} $随着训练集尺寸$ n $而线性增加，而$ \ tau_ \ mathrm {gen} $保持恒定。尽管如果训练继续超出了训练，则可以有效地将训练时间越来越有效地概括，从而创造了越来越多的训练时间。只有当$ n $变得大于模型依赖性阈值时，在无限训练时间中，过度拟合才能消失。这些发现揭示了训练动力学中隐性动力正则化的一种形式，即使在高度参数化的设置中，也可以避免记忆。我们的结果得到了具有现实和合成数据集的标准U-NET体系结构的数值实验，以及使用在高维限制中研究的可拖动随机特征模型的理论分析。

Title: Instruct2See: Learning to Remove Any Obstructions Across Distributions

• Authors: Junhang Li, Yu Guo, Chuhua Xian, Shengfeng He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17649
• Pdf URL: https://arxiv.org/pdf/2505.17649
• Copy Paste: [[2505.17649]] Instruct2See: Learning to Remove Any Obstructions Across Distributions(https://arxiv.org/abs/2505.17649)
• Keywords: restoration
• Abstract: Images are often obstructed by various obstacles due to capture limitations, hindering the observation of objects of interest. Most existing methods address occlusions from specific elements like fences or raindrops, but are constrained by the wide range of real-world obstructions, making comprehensive data collection impractical. To overcome these challenges, we propose Instruct2See, a novel zero-shot framework capable of handling both seen and unseen obstacles. The core idea of our approach is to unify obstruction removal by treating it as a soft-hard mask restoration problem, where any obstruction can be represented using multi-modal prompts, such as visual semantics and textual instructions, processed through a cross-attention unit to enhance contextual understanding and improve mode control. Additionally, a tunable mask adapter allows for dynamic soft masking, enabling real-time adjustment of inaccurate masks. Extensive experiments on both in-distribution and out-of-distribution obstacles show that Instruct2See consistently achieves strong performance and generalization in obstruction removal, regardless of whether the obstacles were present during the training phase. Code and dataset are available at this https URL.
• 摘要：由于捕获的局限性，图像通常被各种障碍所阻碍，从而阻碍了对目标对象的观察。大多数现有的方法都涉及来自栅栏或雨滴等特定元素的阻塞，但受到广泛的现实障碍的限制，使全面的数据收集不切实际。为了克服这些挑战，我们提出了指令2See，这是一个可处理和看不见的障碍的新型零击框架。我们方法的核心思想是通过将障碍物视为柔软的面膜恢复问题来统一障碍物，在这种情况下，可以使用多模式提示来表示任何障碍物，例如视觉语义和文本说明，通过交叉注意单元处理，以增强上下文理解并改善模式控制。此外，可调蒙版适配器允许动态软掩模，从而实现不准确掩模的实时调整。关于分发和分布外障碍物的广泛实验表明，指令2分别在训练阶段是否存在障碍，始终如一地实现强大的性能和概括。代码和数据集可在此HTTPS URL上找到。

Title: DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification

• Authors: Chenyang Li, Jinsong Chen, John E. Hopcroft, Kun He
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17660
• Pdf URL: https://arxiv.org/pdf/2505.17660
• Copy Paste: [[2505.17660]] DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification(https://arxiv.org/abs/2505.17660)
• Keywords: generation
• Abstract: Neighborhood-aware tokenized graph Transformers have recently shown great potential for node classification tasks. Despite their effectiveness, our in-depth analysis of neighborhood tokens reveals two critical limitations in the existing paradigm. First, current neighborhood token generation methods fail to adequately capture attribute correlations within a neighborhood. Second, the conventional self-attention mechanism suffers from attention diversion when processing neighborhood tokens, where high-hop neighborhoods receive disproportionate focus, severely disrupting information interactions between the target node and its neighborhood tokens. To address these challenges, we propose DAM-GT, Dual positional encoding-based Attention Masking graph Transformer. DAM-GT introduces a novel dual positional encoding scheme that incorporates attribute-aware encoding via an attribute clustering strategy, effectively preserving node correlations in both topological and attribute spaces. In addition, DAM-GT formulates a new attention mechanism with a simple yet effective masking strategy to guide interactions between target nodes and their neighborhood tokens, overcoming the issue of attention diversion. Extensive experiments on various graphs with different homophily levels as well as different scales demonstrate that DAM-GT consistently outperforms state-of-the-art methods in node classification tasks.
• 摘要：邻里感知的令牌化图形变压器最近显示了节点分类任务的巨大潜力。尽管它们有效，但我们对邻里代币的深入分析揭示了现有范式的两个关键局限性。首先，当前的邻里代币生成方法无法充分捕获社区内的属性相关性。其次，在处理社区令牌时，常规的自我发场机制会受到关注转移，那里的高跳跃社区得到了不成比例的重点，从而严重破坏了目标节点与其社区令牌之间的信息相互作用。为了应对这些挑战，我们提出了DAM-GT，双位置编码的基于双位置掩盖图形变压器。 DAM-GT引入了一种新颖的双位置编码方案，该方案通过属性聚类策略结合了属性意识，可以有效地保留拓扑和属性空间中的节点相关性。此外，DAM-GT通过一种简单而有效的掩盖策略制定了一种新的注意机制，以指导目标节点及其邻里令牌之间的相互作用，从而克服了注意力转移的问题。在不同层次层的各种图表上进行的广泛实验以及不同的量表表明，大坝GT在节点分类任务中始终优于最先进的方法。

Title: Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs

• Authors: Tianheng Ling, Chao Qian, Lukas Johannes Haßler, Gregor Schiele
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17662
• Pdf URL: https://arxiv.org/pdf/2505.17662
• Copy Paste: [[2505.17662]] Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs(https://arxiv.org/abs/2505.17662)
• Keywords: generation
• Abstract: Transformer-based models have shown strong performance across diverse time-series tasks, but their deployment on resource-constrained devices remains challenging due to high memory and computational demand. While prior work targeting Microcontroller Units (MCUs) has explored hardware-specific optimizations, such approaches are often task-specific and limited to 8-bit fixed-point precision. Field-Programmable Gate Arrays (FPGAs) offer greater flexibility, enabling fine-grained control over data precision and architecture. However, existing FPGA-based deployments of Transformers for time-series analysis typically focus on high-density platforms with manual configuration. This paper presents a unified and fully automated deployment framework for Tiny Transformers on embedded FPGAs. Our framework supports a compact encoder-only Transformer architecture across three representative time-series tasks (forecasting, classification, and anomaly detection). It combines quantization-aware training (down to 4 bits), hardware-aware hyperparameter search using Optuna, and automatic VHDL generation for seamless deployment. We evaluate our framework on six public datasets across two embedded FPGA platforms. Results show that our framework produces integer-only, task-specific Transformer accelerators achieving as low as 0.033 mJ per inference with millisecond latency on AMD Spartan-7, while also providing insights into deployment feasibility on Lattice iCE40. All source code will be released in the GitHub repository (this https URL).
• 摘要：基于变压器的模型在不同的时间序列任务中表现出强大的性能，但是由于高内存和计算需求，它们在资源受限设备上的部署仍然具有挑战性。虽然先前针对微控制器单元（MCUS）的工作已经探索了特定于硬件的优化，但这种方法通常是特定于任务的，并且仅限于8位固定点精度。现场编程的门阵列（FPGA）提供了更大的灵活性，从而可以对数据精度和体系结构进行细粒度的控制。但是，用于时间序列分析的变压器的现有部署通常集中在具有手动配置的高密度平台上。本文为嵌入式FPGA上的小型变压器提供了一个统一且完全自动化的部署框架。我们的框架支持三个代表性的时间序列任务（预测，分类和异常检测）的紧凑型编码变压器体系结构。它结合了量化感知训练（降至4位），使用Optuna的硬件感知的超参数搜索以及自动VHDL生成无缝部署。我们在两个嵌入式FPGA平台上的六个公共数据集上评估了我们的框架。结果表明，我们的框架可产生仅在AMD Spartan-7上使用毫秒潜伏期的每次推断，可实现仅限的，特定于任务的特定于任务的变压器加速器，同时也提供了对Lattice Ice40的部署可行性的见解。所有源代码将在GitHub存储库（此HTTPS URL）中发布。

Title: FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving

• Authors: Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, Xing Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17685
• Pdf URL: https://arxiv.org/pdf/2505.17685
• Copy Paste: [[2505.17685]] FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving(https://arxiv.org/abs/2505.17685)
• Keywords: generation
• Abstract: Visual language models (VLMs) have attracted increasing interest in autonomous driving due to their powerful reasoning capabilities. However, existing VLMs typically utilize discrete text Chain-of-Thought (CoT) tailored to the current scenario, which essentially represents highly abstract and symbolic compression of visual information, potentially leading to spatio-temporal relationship ambiguity and fine-grained information loss. Is autonomous driving better modeled on real-world simulation and imagination than on pure symbolic logic? In this paper, we propose a spatio-temporal CoT reasoning method that enables models to think visually. First, VLM serves as a world model to generate unified image frame for predicting future world states: where perception results (e.g., lane divider and 3D detection) represent the future spatial relationships, and ordinary future frame represent the temporal evolution relationships. This spatio-temporal CoT then serves as intermediate reasoning steps, enabling the VLM to function as an inverse dynamics model for trajectory planning based on current observations and future predictions. To implement visual generation in VLMs, we propose a unified pretraining paradigm integrating visual generation and understanding, along with a progressive visual CoT enhancing autoregressive image generation. Extensive experimental results demonstrate the effectiveness of the proposed method, advancing autonomous driving towards visual reasoning.
• 摘要：视觉语言模型（VLM）由于其强大的推理能力而引起了人们对自主驾驶的兴趣。但是，现有的VLM通常利用针对当前场景量身定制的离散文本链链（COT），这基本上代表了视觉信息的高度抽象和符号的压缩，可能导致时空关系歧义和细粒度的信息丢失。在现实世界模拟和想象力上，自主驾驶是否比纯符号逻辑更好地建模？在本文中，我们提出了一种时空的COT推理方法，该方法使模型可以视觉思考。首先，VLM是一种世界模型，以生成统一的图像框架来预测未来的世界状态：在哪里感知结果（例如，车道分隔线和3D检测）代表未来的空间关系，而普通的未来框架代表了时间的演变关系。然后，该时空COT充当中间推理步骤，使VLM能够基于当前的观察结果和未来的预测作为轨迹计划的逆动力学模型。为了在VLMS中实施视觉生成，我们提出了一个统一的预处理范式，将视觉生成和理解整合在一起，以及渐进的视觉COT增强自回归图像的生成。广泛的实验结果证明了所提出的方法的有效性，从而促进了自主驱动视觉推理。

Title: Seek-CAD: A Self-refined Generative Modeling for 3D Parametric CAD Using Local Inference via DeepSeek

• Authors: Xueyang Li, Jiahao Li, Yu Song, Yunzhong Lou, Xiangdong Zhou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17702
• Pdf URL: https://arxiv.org/pdf/2505.17702
• Copy Paste: [[2505.17702]] Seek-CAD: A Self-refined Generative Modeling for 3D Parametric CAD Using Local Inference via DeepSeek(https://arxiv.org/abs/2505.17702)
• Keywords: generation, generative
• Abstract: The advent of Computer-Aided Design (CAD) generative modeling will significantly transform the design of industrial products. The recent research endeavor has extended into the realm of Large Language Models (LLMs). In contrast to fine-tuning methods, training-free approaches typically utilize the advanced closed-source LLMs, thereby offering enhanced flexibility and efficiency in the development of AI agents for generating CAD parametric models. However, the substantial cost and limitations of local deployment of the top-tier closed-source LLMs pose challenges in practical applications. The Seek-CAD is the pioneer exploration of locally deployed open-source inference LLM DeepSeek-R1 for CAD parametric model generation with a training-free methodology. This study is the first investigation to incorporate both visual and Chain-of-Thought (CoT) feedback within the self-refinement mechanism for generating CAD models. Specifically, the initial generated parametric CAD model is rendered into a sequence of step-wise perspective images, which are subsequently processed by a Vision Language Model (VLM) alongside the corresponding CoTs derived from DeepSeek-R1 to assess the CAD model generation. Then, the feedback is utilized by DeepSeek-R1 to refine the initial generated model for the next round of generation. Moreover, we present an innovative 3D CAD model dataset structured around the SSR (Sketch, Sketch-based feature, and Refinements) triple design paradigm. This dataset encompasses a wide range of CAD commands, thereby aligning effectively with industrial application requirements and proving suitable for the generation of LLMs. Extensive experiments validate the effectiveness of Seek-CAD under various metrics.
• 摘要：计算机辅助设计（CAD）生成建模的出现将显着改变工业产品的设计。最近的研究努力已经扩展到了大语言模型（LLM）的领域。与微调方法相反，无训练方法通常利用先进的封闭源LLM，从而为生成CAD参数模型的AI代理的开发提供了增强的灵活性和效率。但是，顶级封闭源LLMS的本地部署的大量成本和局限性在实际应用中构成了挑战。 Seek-CAD是对本地部署的开源推理LLM DeepSeek-R1的先驱探索，用于使用无培训方法来生成CAD参数模型。这项研究是首次将视觉和思想链（COT）反馈纳入生成CAD模型的自我注册机制中的研究。具体而言，将初始生成的参数CAD模型渲染为一系列逐步的透视图像，随后通过视觉语言模型（VLM）以及从DeepSeek-R1得出的相应的COTS处理，以评估CAD模型的生成。然后，DeepSeek-R1利用反馈来完善下一轮发电的初始生成模型。此外，我们提出了一个创新的3D CAD模型数据集，该数据集围绕SSR（基于草图，基于草图的功能和改进）三重设计范式构建。该数据集涵盖了各种CAD命令，从而有效地与工业应用要求有效地保持一致，并证明适合于LLM的生成。广泛的实验验证了各种指标下Seek-CAD的有效性。

Title: SeaLion: Semantic Part-Aware Latent Point Diffusion Models for 3D Generation

• Authors: Dekai Zhu, Yan Di, Stefan Gavranovic, Slobodan Ilic
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17721
• Pdf URL: https://arxiv.org/pdf/2505.17721
• Copy Paste: [[2505.17721]] SeaLion: Semantic Part-Aware Latent Point Diffusion Models for 3D Generation(https://arxiv.org/abs/2505.17721)
• Keywords: generation, generative
• Abstract: Denoising diffusion probabilistic models have achieved significant success in point cloud generation, enabling numerous downstream applications, such as generative data augmentation and 3D model editing. However, little attention has been given to generating point clouds with point-wise segmentation labels, as well as to developing evaluation metrics for this task. Therefore, in this paper, we present SeaLion, a novel diffusion model designed to generate high-quality and diverse point clouds with fine-grained segmentation labels. Specifically, we introduce the semantic part-aware latent point diffusion technique, which leverages the intermediate features of the generative models to jointly predict the noise for perturbed latent points and associated part segmentation labels during the denoising process, and subsequently decodes the latent points to point clouds conditioned on part segmentation labels. To effectively evaluate the quality of generated point clouds, we introduce a novel point cloud pairwise distance calculation method named part-aware Chamfer distance (p-CD). This method enables existing metrics, such as 1-NNA, to measure both the local structural quality and inter-part coherence of generated point clouds. Experiments on the large-scale synthetic dataset ShapeNet and real-world medical dataset IntrA demonstrate that SeaLion achieves remarkable performance in generation quality and diversity, outperforming the existing state-of-the-art model, DiffFacto, by 13.33% and 6.52% on 1-NNA (p-CD) across the two datasets. Experimental analysis shows that SeaLion can be trained semi-supervised, thereby reducing the demand for labeling efforts. Lastly, we validate the applicability of SeaLion in generative data augmentation for training segmentation models and the capability of SeaLion to serve as a tool for part-aware 3D shape editing.
• 摘要：去核扩散概率模型已在点云的产生中取得了重大成功，从而实现了许多下游应用，例如生成数据增强和3D模型编辑。但是，很少关注具有点细分标签的点云，以及为此任务开发评估指标。因此，在本文中，我们提出了一种新型扩散模型，旨在产生具有细粒度分割标签的高质量和不同的点云。具体而言，我们介绍了语义零件感知的潜在扩散技术，该技术利用生成模型的中间特征共同预测了deNOOSing过程中扰动的潜在点和相关零件分段标签的噪声，并随后将潜在点解码为在部分分割标签上的潜在云。为了有效评估生成点云的质量，我们引入了一种新的点云成对距离计算方法，名为零件吸引的倒角距离（P-CD）。该方法使现有的指标（例如1-NNA）能够测量生成点云的局部结构质量和零件间的连贯性。大规模合成数据集Shapenet和现实世界医学数据集的实验表明，密封剂在发电质量和多样性方面的表现出色，在两个数据集中，在1-NNA（P-CD）上的实验表现优于现有的最新模型，DiFffacto，Difffacto，均优于13.33％和6.52％。实验分析表明，可以对海豹进行半监督训练，从而减少对标签工作的需求。最后，我们验证了密封在生成数据增强中的适用性，用于训练分割模型，以及密封能力作为零件感知3D形状编辑工具的能力。

Title: Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLM

• Authors: Donghwan Chi, Hyomin Kim, Yoonjin Oh, Yongjin Kim, Donghoon Lee, Daejin Jo, Jongmin Kim, Junyeob Baek, Sungjin Ahn, Sungwoong Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17726
• Pdf URL: https://arxiv.org/pdf/2505.17726
• Copy Paste: [[2505.17726]] Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLM(https://arxiv.org/abs/2505.17726)
• Keywords: generation
• Abstract: Recently, multimodal large language models (MLLMs) have emerged as a key approach in achieving artificial general intelligence. In particular, vision-language MLLMs have been developed to generate not only text but also visual outputs from multimodal inputs. This advancement requires efficient image tokens that LLMs can process effectively both in input and output. However, existing image tokenization methods for MLLMs typically capture only global abstract concepts or uniformly segmented image patches, restricting MLLMs' capability to effectively understand or generate detailed visual content, particularly at the object level. To address this limitation, we propose an object-centric visual tokenizer based on Slot Attention specifically for MLLMs. In particular, based on the Q-Former encoder, diffusion decoder, and residual vector quantization, our proposed discretized slot tokens can encode local visual details while maintaining high-level semantics, and also align with textual data to be integrated seamlessly within a unified next-token prediction framework of LLMs. The resulting Slot-MLLM demonstrates significant performance improvements over baselines with previous visual tokenizers across various vision-language tasks that entail local detailed comprehension and generation. Notably, this work is the first demonstration of the feasibility of object-centric slot attention performed with MLLMs and in-the-wild natural images.
• 摘要：最近，多模式的大语言模型（MLLM）已成为实现人工通用智能的关键方法。特别是，已经开发了视觉mllms，不仅可以生成文本，而且还产生了来自多模式输入的视觉输出。此进步需要LLM可以在输入和输出中有效处理的有效图像令牌。但是，现有的MLLM的现有图像令牌化方法通常仅捕获全局抽象概念或均匀分割的图像补丁，从而限制了MLLM的能力有效理解或生成详细的视觉内容，尤其是在对象级别上。为了解决此限制，我们根据专门针对MLLM的插槽注意提出了一个以对象为中心的视觉令牌仪。特别是，基于Q形式编码器，扩散解码器和残留矢量量化，我们提议的离散的插槽令牌可以编码本地视觉细节，同时保持高级语义，并且还与统一的LLMS的统一的下一步预测框架在统一的统一的接缝框架中保持一致。由此产生的插槽-MLLM表现出对基线的显着改进，并且在各种视觉语言任务中，先前的视觉令牌器需要进行本地详细的理解和生成。值得注意的是，这项工作是用MLLM和野外自然图像进行的以对象为中心的插槽注意的可行性的首次演示。

Title: SafeMVDrive: Multi-view Safety-Critical Driving Video Synthesis in the Real World Domain

• Authors: Jiawei Zhou, Linye Lyu, Zhuotao Tian, Cheng Zhuo, Yu Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17727
• Pdf URL: https://arxiv.org/pdf/2505.17727
• Copy Paste: [[2505.17727]] SafeMVDrive: Multi-view Safety-Critical Driving Video Synthesis in the Real World Domain(https://arxiv.org/abs/2505.17727)
• Keywords: generation
• Abstract: Safety-critical scenarios are rare yet pivotal for evaluating and enhancing the robustness of autonomous driving systems. While existing methods generate safety-critical driving trajectories, simulations, or single-view videos, they fall short of meeting the demands of advanced end-to-end autonomous systems (E2E AD), which require real-world, multi-view video data. To bridge this gap, we introduce SafeMVDrive, the first framework designed to generate high-quality, safety-critical, multi-view driving videos grounded in real-world domains. SafeMVDrive strategically integrates a safety-critical trajectory generator with an advanced multi-view video generator. To tackle the challenges inherent in this integration, we first enhance scene understanding ability of the trajectory generator by incorporating visual context -- which is previously unavailable to such generator -- and leveraging a GRPO-finetuned vision-language model to achieve more realistic and context-aware trajectory generation. Second, recognizing that existing multi-view video generators struggle to render realistic collision events, we introduce a two-stage, controllable trajectory generation mechanism that produces collision-evasion trajectories, ensuring both video quality and safety-critical fidelity. Finally, we employ a diffusion-based multi-view video generator to synthesize high-quality safety-critical driving videos from the generated trajectories. Experiments conducted on an E2E AD planner demonstrate a significant increase in collision rate when tested with our generated data, validating the effectiveness of SafeMVDrive in stress-testing planning modules. Our code, examples, and datasets are publicly available at: this https URL.
• 摘要：安全 - 关键的情况很少见，但对于评估和增强自动驾驶系统的鲁棒性而言是关键的。尽管现有方法会产生安全至关重要的驾驶轨迹，模拟或单视图视频，但它们无法满足高级端到端自治系统（E2E AD）的需求，这些系统需要真实世界，多视频视频数据。为了弥合这一差距，我们介绍了Safemvdrive，这是第一个框架，旨在生成基于现实世界域中的高质量，至关重要的多视图驾驶视频。 Safemvidrive战略性地将安全关键轨迹生成器与高级多视频视频生成器集成在一起。为了应对这种整合中固有的挑战，我们首先通过合并视觉上下文来增强场景理解能力（以前是此类发电机不可用的视觉上下文），并利用GRPO-FINETNETNETENEDEDINED视觉语言模型来实现更现实和上下文意识到的轨迹产生。其次，认识到现有的多视频发电机难以实现现实的碰撞事件，我们引入了一种两阶段，可控制的轨迹生成机制，从而产生碰撞探测轨迹，从而确保视频质量和安全关键的忠诚度。最后，我们采用基于扩散的多视频视频生成器来合成生成轨迹的高质量安全关键驾驶视频。对E2E AD计划进行的实验表明，使用我们的生成数据进行测试时，碰撞率显着提高，从而验证了Safemvdrive在应力测试计划模块中的有效性。我们的代码，示例和数据集可公开可用：此HTTPS URL。

Title: A tensor network approach for chaotic time series prediction

• Authors: Rodrigo Martínez-Peña, Román Orús
• Subjects: cs.LG, cs.NE, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2505.17740
• Pdf URL: https://arxiv.org/pdf/2505.17740
• Copy Paste: [[2505.17740]] A tensor network approach for chaotic time series prediction(https://arxiv.org/abs/2505.17740)
• Keywords: generation
• Abstract: Making accurate predictions of chaotic time series is a complex challenge. Reservoir computing, a neuromorphic-inspired approach, has emerged as a powerful tool for this task. It exploits the memory and nonlinearity of dynamical systems without requiring extensive parameter tuning. However, selecting and optimizing reservoir architectures remains an open problem. Next-generation reservoir computing simplifies this problem by employing nonlinear vector autoregression based on truncated Volterra series, thereby reducing hyperparameter complexity. Nevertheless, the latter suffers from exponential parameter growth in terms of the maximum monomial degree. Tensor networks offer a promising solution to this issue by decomposing multidimensional arrays into low-dimensional structures, thus mitigating the curse of dimensionality. This paper explores the application of a previously proposed tensor network model for predicting chaotic time series, demonstrating its advantages in terms of accuracy and computational efficiency compared to conventional echo state networks. Using a state-of-the-art tensor network approach enables us to bridge the gap between the tensor network and reservoir computing communities, fostering advances in both fields.
• 摘要：对混乱时间序列进行准确的预测是一个复杂的挑战。储层计算是一种神经形态启发的方法，已成为该任务的强大工具。它利用动态系统的内存和非线性，而无需进行广泛的参数调整。但是，选择和优化储层体系结构仍然是一个开放的问题。下一代储层计算通过基于截短的Volterra系列采用非线性矢量自动进度来简化此问题，从而降低了超参数复杂性。然而，后者以最大单一程度的方式遭受了指数参数的增长。张量网络通过将多维阵列分解为低维结构，从而减轻维度的诅咒，从而为这个问题提供了有希望的解决方案。本文探讨了先前提出的张量网络模型在预测混乱时间序列中的应用，与传统的回声状态网络相比，它在准确性和计算效率方面证明了其优势。使用最先进的张量网络方法，我们可以弥合张量网络和储层计算社区之间的差距，从而促进了这两个领域的进步。

Title: R-Genie: Reasoning-Guided Generative Image Editing

• Authors: Dong Zhang, Lingfeng He, Rui Yan, Fei Shen, Jinhui Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17768
• Pdf URL: https://arxiv.org/pdf/2505.17768
• Copy Paste: [[2505.17768]] R-Genie: Reasoning-Guided Generative Image Editing(https://arxiv.org/abs/2505.17768)
• Keywords: generation, generative
• Abstract: While recent advances in image editing have enabled impressive visual synthesis capabilities, current methods remain constrained by explicit textual instructions and limited editing operations, lacking deep comprehension of implicit user intentions and contextual reasoning. In this work, we introduce a new image editing paradigm: reasoning-guided generative editing, which synthesizes images based on complex, multi-faceted textual queries accepting world knowledge and intention inference. To facilitate this task, we first construct a comprehensive dataset featuring over 1,000 image-instruction-edit triples that incorporate rich reasoning contexts and real-world knowledge. We then propose R-Genie: a reasoning-guided generative image editor, which synergizes the generation power of diffusion models with advanced reasoning capabilities of multimodal large language models. R-Genie incorporates a reasoning-attention mechanism to bridge linguistic understanding with visual synthesis, enabling it to handle intricate editing requests involving abstract user intentions and contextual reasoning relations. Extensive experimental results validate that R-Genie can equip diffusion models with advanced reasoning-based editing capabilities, unlocking new potentials for intelligent image synthesis.
• 摘要：尽管图像编辑的最新进展使令人印象深刻的视觉合成功能仍然受到明确的文本指令和有限的编辑操作的限制，因此缺乏对隐式用户意图和上下文推理的深入理解。在这项工作中，我们介绍了一个新的图像编辑范式：推理引导的生成编辑，该编辑基于接受世界知识和意图推论的复杂，多方面的文本查询综合图像。为了促进这项任务，我们首先构建了一个全面的数据集，其中包含1,000多个图像实施者编辑三元组，这些数据集包含丰富的推理环境和现实世界知识。然后，我们提出了R-Genie：一种推理引导的生成图像编辑器，该编辑器通过多模式大语言模型的高级推理能力协同扩散模型的生成能力。 R-Genie结合了一种推理意见机制，可以将语言理解与视觉合成桥接，从而使其能够处理涉及抽象用户意图和上下文推理关系的复杂编辑请求。广泛的实验结果验证了R-Genie可以为扩散模型提供基于先进的基于推理的编辑功能，从而为智能图像合成提供了新的潜力。

Title: TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesis

• Authors: Yu Xie, Jielei Zhang, Pengyu Chen, Ziyue Wang, Weihang Wang, Longwen Gao, Peiyi Li, Huyang Sun, Qiang Zhang, Qian Qiao, Jiaqing Fan, Zhouhui Lian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17778
• Pdf URL: https://arxiv.org/pdf/2505.17778
• Copy Paste: [[2505.17778]] TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesis(https://arxiv.org/abs/2505.17778)
• Keywords: generation
• Abstract: Diffusion-based scene text synthesis has progressed rapidly, yet existing methods commonly rely on additional visual conditioning modules and require large-scale annotated data to support multilingual generation. In this work, we revisit the necessity of complex auxiliary modules and further explore an approach that simultaneously ensures glyph accuracy and achieves high-fidelity scene integration, by leveraging diffusion models' inherent capabilities for contextual reasoning. To this end, we introduce TextFlux, a DiT-based framework that enables multilingual scene text synthesis. The advantages of TextFlux can be summarized as follows: (1) OCR-free model architecture. TextFlux eliminates the need for OCR encoders (additional visual conditioning modules) that are specifically used to extract visual text-related features. (2) Strong multilingual scalability. TextFlux is effective in low-resource multilingual settings, and achieves strong performance in newly added languages with fewer than 1,000 samples. (3) Streamlined training setup. TextFlux is trained with only 1% of the training data required by competing methods. (4) Controllable multi-line text generation. TextFlux offers flexible multi-line synthesis with precise line-level control, outperforming methods restricted to single-line or rigid layouts. Extensive experiments and visualizations demonstrate that TextFlux outperforms previous methods in both qualitative and quantitative evaluations.
• 摘要：基于扩散的场景文本综合的进展迅速，但现有的方法通常依赖于其他视觉调节模块，并且需要大规模注释的数据来支持多语言生成。在这项工作中，我们通过利用扩散模型的固有功能来实现上下文推理的固有功能，从而重新审视复杂的辅助模块的必要性，并进一步探索一种同时确保字形准确性并实现高效率场景集成的方法。为此，我们介绍了TextFlux，这是一个基于DIT的框架，可实现多种语言场景文本综合。文本流量的优点可以总结如下：（1）无OCR模型体系结构。 TextFlux消除了专门用于提取视觉文本相关特征的OCR编码器（其他视觉调节模块）的需求。 （2）强大的多语言可伸缩性。 TextFlux在低资源的多语言设置中有效，并且在新添加的语言中实现了较少的样本的良好性能。 （3）简化的培训设置。 TextFlux仅接受竞争方法所需的培训数据的1％培训。 （4）可控的多行文本生成。 TextFlux提供具有精确的线路级控件的灵活多行合成，优于仅限单线或刚性布局的方法。广泛的实验和可视化表明，在定性和定量评估中，TextFlux优于先前的方法。

Title: U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding

• Authors: Anjie Le, Henan Liu, Yue Wang, Zhenyu Liu, Rongkun Zhu, Taohan Weng, Jinze Yu, Boyang Wang, Yalun Wu, Kaiwen Yan, Quanlin Sun, Meirui Jiang, Jialun Pei, Siya Liu, Haoyun Zheng, Zhoujun Li, Alison Noble, Jacques Souquet, Xiaoqing Guo, Manxi Lin, Hongcheng Guo
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17779
• Pdf URL: https://arxiv.org/pdf/2505.17779
• Copy Paste: [[2505.17779]] U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding(https://arxiv.org/abs/2505.17779)
• Keywords: generation
• Abstract: Ultrasound is a widely-used imaging modality critical to global healthcare, yet its interpretation remains challenging due to its varying image quality on operators, noises, and anatomical structures. Although large vision-language models (LVLMs) have demonstrated impressive multimodal capabilities across natural and medical domains, their performance on ultrasound remains largely unexplored. We introduce U2-BENCH, the first comprehensive benchmark to evaluate LVLMs on ultrasound understanding across classification, detection, regression, and text generation tasks. U2-BENCH aggregates 7,241 cases spanning 15 anatomical regions and defines 8 clinically inspired tasks, such as diagnosis, view recognition, lesion localization, clinical value estimation, and report generation, across 50 ultrasound application scenarios. We evaluate 20 state-of-the-art LVLMs, both open- and closed-source, general-purpose and medical-specific. Our results reveal strong performance on image-level classification, but persistent challenges in spatial reasoning and clinical language generation. U2-BENCH establishes a rigorous and unified testbed to assess and accelerate LVLM research in the uniquely multimodal domain of medical ultrasound imaging.
• 摘要：超声是一种对全球医疗保健至关重要的广泛使用的成像方式，但由于其对操作员，噪音和解剖结构的图像质量的不同，其解释仍然具有挑战性。尽管大型视觉模型（LVLM）在天然和医疗领域表现出令人印象深刻的多模式能力，但它们在超声波上的性能仍然在很大程度上没有探索。我们介绍了U2 Bench，这是第一个评估LVLM在分类，检测，回归和文本生成任务的超声理解的全面基准。 U2板凳汇总了7,241例跨越15个解剖区域的病例并定义了8个临床灵感的任务，例如诊断，视图识别，病变定位，临床价值估计和报告生成，跨50个超声应用程序场景。我们评估了20个最先进的LVLM，无论是开放式和封闭式，通用和特定于医学的。我们的结果揭示了图像级分类的强烈表现，但是在空间推理和临床语言产生方面持续挑战。 U2板台建立了严格而统一的测试台，以评估和加速医疗超声成像的独特多模式结构域中的LVLM研究。

Title: Generative Data Augmentation for Object Point Cloud Segmentation

• Authors: Dekai Zhu, Stefan Gavranovic, Flavien Boussuge, Benjamin Busam, Slobodan Ilic
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17783
• Pdf URL: https://arxiv.org/pdf/2505.17783
• Copy Paste: [[2505.17783]] Generative Data Augmentation for Object Point Cloud Segmentation(https://arxiv.org/abs/2505.17783)
• Keywords: generation, generative
• Abstract: Data augmentation is widely used to train deep learning models to address data scarcity. However, traditional data augmentation (TDA) typically relies on simple geometric transformation, such as random rotation and rescaling, resulting in minimal data diversity enrichment and limited model performance improvement. State-of-the-art generative models for 3D shape generation rely on the denoising diffusion probabilistic models and manage to generate realistic novel point clouds for 3D content creation and manipulation. Nevertheless, the generated 3D shapes lack associated point-wise semantic labels, restricting their usage in enlarging the training data for point cloud segmentation tasks. To bridge the gap between data augmentation techniques and the advanced diffusion models, we extend the state-of-the-art 3D diffusion model, Lion, to a part-aware generative model that can generate high-quality point clouds conditioned on given segmentation masks. Leveraging the novel generative model, we introduce a 3-step generative data augmentation (GDA) pipeline for point cloud segmentation training. Our GDA approach requires only a small amount of labeled samples but enriches the training data with generated variants and pseudo-labeled samples, which are validated by a novel diffusion-based pseudo-label filtering method. Extensive experiments on two large-scale synthetic datasets and a real-world medical dataset demonstrate that our GDA method outperforms TDA approach and related semi-supervised and self-supervised methods.
• 摘要：数据增强被广泛用于训练深度学习模型以解决数据稀缺性。但是，传统数据增强（TDA）通常依赖于简单的几何变换，例如随机旋转和重新缩放，从而导致数据多样性富集和有限的模型性能改善。用于3D形状生成的最新生成模型依赖于扩散概率模型，并设法生成了现实的新点云来创建3D内容和操纵。然而，生成的3D形状缺乏关联的点语义标签，从而限制了它们在扩大点云分段任务的训练数据方面的用法。为了弥合数据增强技术与高级扩散模型之间的差距，我们将最新的3D扩散模型（狮子）扩展到可以生成在给定的分段蒙版条件下的高质量点云的部分感知生成模型。利用新颖的生成模型，我们引入了三步生成数据增强（GDA）管道，以进行点云分割训练。我们的GDA方法仅需要少量标记的样品，但使用生成的变体和伪标记的样品丰富了训练数据，这些样品通过一种新型的基于基于扩散的伪标签的滤波方法验证。对两个大规模合成数据集和一个现实世界医学数据集进行的广泛实验表明，我们的GDA方法的表现优于TDA方法和相关的半监督和自我监督方法。

Title: RECIPE-TKG: From Sparse History to Structured Reasoning for LLM-based Temporal Knowledge Graph Completion

• Authors: Ömer Faruk Akgül, Feiyu Zhu, Yuxin Yang, Rajgopal Kannan, Viktor Prasanna
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.17794
• Pdf URL: https://arxiv.org/pdf/2505.17794
• Copy Paste: [[2505.17794]] RECIPE-TKG: From Sparse History to Structured Reasoning for LLM-based Temporal Knowledge Graph Completion(https://arxiv.org/abs/2505.17794)
• Keywords: generation
• Abstract: Temporal Knowledge Graphs (TKGs) represent dynamic facts as timestamped relations between entities. TKG completion involves forecasting missing or future links, requiring models to reason over time-evolving structure. While LLMs show promise for this task, existing approaches often overemphasize supervised fine-tuning and struggle particularly when historical evidence is limited or missing. We introduce RECIPE-TKG, a lightweight and data-efficient framework designed to improve accuracy and generalization in settings with sparse historical context. It combines (1) rule-based multi-hop retrieval for structurally diverse history, (2) contrastive fine-tuning of lightweight adapters to encode relational semantics, and (3) test-time semantic filtering to iteratively refine generations based on embedding similarity. Experiments on four TKG benchmarks show that RECIPE-TKG outperforms previous LLM-based approaches, achieving up to 30.6\% relative improvement in Hits@10. Moreover, our proposed framework produces more semantically coherent predictions, even for the samples with limited historical context.
• 摘要：时间知识图（TKG）表示动态事实，作为实体之间的时间戳关系。 TKG完成涉及预测丢失或将来的链接，要求模型推理过时的时间不断发展的结构。尽管LLM对这项任务表现出希望，但现有的方法通常过分强调受监督的微调和斗争，尤其是在历史证据有限或缺失时。我们介绍了食谱-TKG，这是一个轻巧且具有数据效率的框架，旨在提高历史上下文稀疏的设置中的准确性和概括。它结合了（1）基于规则的多跳检索，用于结构多样化的历史，（2）轻巧适配器的对比度微调以编码关系语义，以及（3）基于嵌入相似性的基于嵌入的迭代性迭代性效果的测试时间语义过滤。在四个TKG基准测试的实验表明，配方-TKG的表现优于以前的基于LLM的方法，可达到高达30.6 \％的命中率相对改善@10。此外，即使对于历史上下文有限的样本，我们提出的框架也会产生更具语义上的一致性预测。

Title: Hyperparameter Optimization via Interacting with Probabilistic Circuits

• Authors: Jonas Seng, Fabrizio Ventola, Zhongjie Yu, Kristian Kersting
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17804
• Pdf URL: https://arxiv.org/pdf/2505.17804
• Copy Paste: [[2505.17804]] Hyperparameter Optimization via Interacting with Probabilistic Circuits(https://arxiv.org/abs/2505.17804)
• Keywords: generation
• Abstract: Despite the growing interest in designing truly interactive hyperparameter optimization (HPO) methods, to date, only a few allow to include human feedback. Existing interactive Bayesian optimization (BO) methods incorporate human beliefs by weighting the acquisition function with a user-defined prior distribution. However, in light of the non-trivial inner optimization of the acquisition function prevalent in BO, such weighting schemes do not always accurately reflect given user beliefs. We introduce a novel BO approach leveraging tractable probabilistic models named probabilistic circuits (PCs) as a surrogate model. PCs encode a tractable joint distribution over the hybrid hyperparameter space and evaluation scores. They enable exact conditional inference and sampling. Based on conditional sampling, we construct a novel selection policy that enables an acquisition function-free generation of candidate points (thereby eliminating the need for an additional inner-loop optimization) and ensures that user beliefs are reflected accurately in the selection policy. We provide a theoretical analysis and an extensive empirical evaluation, demonstrating that our method achieves state-of-the-art performance in standard HPO and outperforms interactive BO baselines in interactive HPO.
• 摘要：尽管对设计真正的交互式超参数优化（HPO）方法的兴趣日益增加，但迄今为止，只有少数人允许包括人类的反馈。现有的交互式贝叶斯优化（BO）方法通过用用户定义的先前分布加权采集功能来结合人类的信念。但是，鉴于BO中采集功能的非平凡内部优化，这种加权方案并不总是准确地反映出给定用户信念的。我们介绍了一种新型的BO方法，利用了称为概率电路（PC）作为替代模型的可拖动概率模型。 PC在混合体高参数空间和评估分数上编码可拖动的关节分布。它们可以确切的有条件推断和采样。基于条件抽样，我们构建了一种新颖的选择策略，该策略可以实现无动函数生成候选点（从而消除对额外的内环优化的需求），并确保在选择策略中准确反映用户信念。我们提供了理论分析和广泛的经验评估，表明我们的方法在标准HPO中实现了最先进的性能，并且在交互式HPO中的交互式BO基准均超过了交互式BO基线。

Title: VIBE: Vector Index Benchmark for Embeddings

• Authors: Elias Jääsaari, Ville Hyvönen, Matteo Ceccarello, Teemu Roos, Martin Aumüller
• Subjects: cs.LG, cs.IR
• Abstract URL: https://arxiv.org/abs/2505.17810
• Pdf URL: https://arxiv.org/pdf/2505.17810
• Copy Paste: [[2505.17810]] VIBE: Vector Index Benchmark for Embeddings(https://arxiv.org/abs/2505.17810)
• Keywords: generation
• Abstract: Approximate nearest neighbor (ANN) search is a performance-critical component of many machine learning pipelines. Rigorous benchmarking is essential for evaluating the performance of vector indexes for ANN search. However, the datasets of the existing benchmarks are no longer representative of the current applications of ANN search. Hence, there is an urgent need for an up-to-date set of benchmarks. To this end, we introduce Vector Index Benchmark for Embeddings (VIBE), an open source project for benchmarking ANN algorithms. VIBE contains a pipeline for creating benchmark datasets using dense embedding models characteristic of modern applications, such as retrieval-augmented generation (RAG). To replicate real-world workloads, we also include out-of-distribution (OOD) datasets where the queries and the corpus are drawn from different distributions. We use VIBE to conduct a comprehensive evaluation of SOTA vector indexes, benchmarking 21 implementations on 12 in-distribution and 6 out-of-distribution datasets.
• 摘要：大约最近的邻居（ANN）搜索是许多机器学习管道的性能至关重要的组成部分。严格的基准测试对于评估ANN搜索的向量索引的性能至关重要。但是，现有基准的数据集不再代表ANN搜索的当前应用程序。因此，迫切需要建立最新的基准测试。为此，我们引入了嵌入式矢量索引基准（Vibe），这是一个用于基准ANN算法的开源项目。 Vibe包含使用现代应用程序的密集嵌入模型来创建基准数据集的管道，例如检索增强生成（RAG）。为了复制现实世界的工作负载，我们还包括分布（OOD）数据集，其中查询和语料库从不同的分布中汲取。我们使用Vibe对SOTA矢量索引进行全面评估，对12个分发和6个分发数据集进行了基准测试。

Title: Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimization

• Authors: Francois Chaubard, Mykel Kochenderfer
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17852
• Pdf URL: https://arxiv.org/pdf/2505.17852
• Copy Paste: [[2505.17852]] Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimization(https://arxiv.org/abs/2505.17852)
• Keywords: generation
• Abstract: During inference, Recurrent Neural Networks (RNNs) scale constant in both FLOPs and GPU memory with increasing context length, as they compress all prior tokens into a fixed-size memory. In contrast, transformers scale linearly in FLOPs and, at best, linearly in memory during generation, since they must attend to all previous tokens explicitly. Despite this inference-time advantage, training large RNNs on long contexts remains impractical because standard optimization methods depend on Backpropagation Through Time (BPTT). BPTT requires retention of all intermediate activations during the forward pass, causing memory usage to scale linearly with both context length and model size. In this paper, we show that Zero-Order Optimization (ZOO) methods such as Random-vector Gradient Estimation (RGE) can successfully replace BPTT to train RNNs with convergence rates that match, or exceed BPTT by up to 19 fold, while using orders of magnitude less memory and cost, as the model remains in inference mode throughout training. We further demonstrate that Central-Difference RGE (CD-RGE) corresponds to optimizing a smoothed surrogate loss, inherently regularizing training and improving generalization. Our method matches or outperforms BPTT across three settings: (1) overfitting, (2) transduction, and (3) language modeling. Across all tasks, with sufficient perturbations, our models generalize as well as or better than those trained with BPTT, often in fewer steps. Despite the need for more forward passes per step, we can surpass BPTT wall-clock time per step using recent advancements such as FlashRNN and distributed inference.
• 摘要：在推断期间，在拖船和GPU内存中，复发性神经网络（RNN）尺度尺度均匀，上下文长度增加，因为它们在代币中压缩到固定尺寸的内存之前。相比之下，变形金刚在拖鞋中线性缩放，充其量是在发电期间在内存中线性线性的，因为它们必须明确地进行所有以前的令牌。尽管有这种推理时间优势，但在长篇小说中训练大型RNN仍然不切实际，因为标准优化方法取决于通过时间（BPTT）的反向传播。 BPTT需要在向前传球期间保留所有中间激活，从而导致记忆使用量与上下文长度和模型大小线性扩展。在本文中，我们表明零级优化（ZOO）方法，例如随机矢量梯度估计（RGE）可以成功替换BPTT以培训RNN的收敛速率，或者最多超过BPTT，同时使用较大的数量级，而使用较小的内存和成本，因为在整个培训过程中，模型仍然保持在培训模型中。我们进一步证明，中央差异RGE（CD-RGE）对应于优化平滑的替代损失，固有地正规化训练并改善概括。我们的方法在三个设置上匹配或胜过BPTT：（1）过度拟合，（2）转导和（3）语言建模。在所有任务中，具有足够的扰动，我们的模型通常比接受BPTT培训的模型概括或更好，通常以更少的步骤。尽管需要每个步骤进行更多的向前传球，但我们可以使用诸如FlashRNN和分布式推理之类的最新进步每步超越BPTT壁挂时间。

Title: DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization

• Authors: Hongshu Guo, Zeyuan Ma, Yining Ma, Xinglin Zhang, Wei-Neng Chen, Yue-Jiao Gong
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2505.17866
• Pdf URL: https://arxiv.org/pdf/2505.17866
• Copy Paste: [[2505.17866]] DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization(https://arxiv.org/abs/2505.17866)
• Keywords: generation
• Abstract: Designing effective black-box optimizers is hampered by limited problem-specific knowledge and manual control that spans months for almost every detail. In this paper, we present DesignX, the first automated algorithm design framework that generates an effective optimizer specific to a given black-box optimization problem within seconds. Rooted in the first principles, we identify two key sub-tasks: 1) algorithm structure generation and 2) hyperparameter control. To enable systematic construction, a comprehensive modular algorithmic space is first built, embracing hundreds of algorithm components collected from decades of research. We then introduce a dual-agent reinforcement learning system that collaborates on structural and parametric design through a novel cooperative training objective, enabling large-scale meta-training across 10k diverse instances. Remarkably, through days of autonomous learning, the DesignX-generated optimizers continuously surpass human-crafted optimizers by orders of magnitude, either on synthetic testbed or on realistic optimization scenarios such as Protein-docking, AutoML and UAV path planning. Further in-depth analysis reveals DesignX's capability to discover non-trivial algorithm patterns beyond expert intuition, which, conversely, provides valuable design insights for the optimization community. We provide DesignX's inference code at this https URL.
• 摘要：有限的特定问题知识和手动控制跨越了几个月的几乎每个细节，因此设计有效的黑盒优化器受到了有限的特定问题知识和手动控制的阻碍。在本文中，我们介绍了Designx，这是第一个自动化算法设计框架，该框架在几秒钟内生成了特定于给定的黑盒优化问题的有效优化器。植根于第一原理，我们确定两个关键子任务：1）算法结构生成和2）超参数控制。为了实现系统的结构，首先建立了一个全面的模块化算法空间，它包含数百种研究中收集的数百种算法组件。然后，我们引入了一个双重试剂增强学习系统，该系统通过新颖的合作培训目标在结构和参数设计上进行合作，从而实现了10K不同实例的大规模元训练。值得注意的是，在自主学习的日子中，DesignX生成的优化器在合成测试床上或现实的优化场景（例如蛋白质借入，AutoML和UAV路径计划）上，通过数量级不断超过人工制作的优化器。进一步的深入分析揭示了Designx发现超出专家直觉的非平凡算法模式的能力，相反，该模式为优化社区提供了宝贵的设计见解。我们在此HTTPS URL上提供Designx的推理代码。

Title: Track Anything Annotate: Video annotation and dataset generation of computer vision models

• Authors: Nikita Ivanov, Mark Klimov, Dmitry Glukhikh, Tatiana Chernysheva, Igor Glukhikh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17884
• Pdf URL: https://arxiv.org/pdf/2505.17884
• Copy Paste: [[2505.17884]] Track Anything Annotate: Video annotation and dataset generation of computer vision models(https://arxiv.org/abs/2505.17884)
• Keywords: generation
• Abstract: Modern machine learning methods require significant amounts of labelled data, making the preparation process time-consuming and resource-intensive. In this paper, we propose to consider the process of prototyping a tool for annotating and generating training datasets based on video tracking and segmentation. We examine different approaches to solving this problem, from technology selection through to final implementation. The developed prototype significantly accelerates dataset generation compared to manual annotation. All resources are available at this https URL
• 摘要：现代机器学习方法需要大量的标记数据，使准备过程耗时和资源密集。在本文中，我们建议考虑根据视频跟踪和分割来注释和生成培训数据集的工具的过程。我们研究了解决此问题的不同方法，从技术选择到最终实施。与手动注释相比，开发的原型显着加速了数据集的生成。所有资源都可以在此HTTPS URL上找到

Title: DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning

• Authors: Bin Wu, Wei Wang, Yahui Liu, Zixiang Li, Yao Zhao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17910
• Pdf URL: https://arxiv.org/pdf/2505.17910
• Copy Paste: [[2505.17910]] DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning(https://arxiv.org/abs/2505.17910)
• Keywords: restoration, generative
• Abstract: Reward Feedback Learning (ReFL) has recently shown great potential in aligning model outputs with human preferences across various generative tasks. In this work, we introduce a ReFL framework, named DiffusionReward, to the Blind Face Restoration task for the first time. DiffusionReward effectively overcomes the limitations of diffusion-based methods, which often fail to generate realistic facial details and exhibit poor identity consistency. The core of our framework is the Face Reward Model (FRM), which is trained using carefully annotated data. It provides feedback signals that play a pivotal role in steering the optimization process of the restoration network. In particular, our ReFL framework incorporates a gradient flow into the denoising process of off-the-shelf face restoration methods to guide the update of model parameters. The guiding gradient is collaboratively determined by three aspects: (i) the FRM to ensure the perceptual quality of the restored faces; (ii) a regularization term that functions as a safeguard to preserve generative diversity; and (iii) a structural consistency constraint to maintain facial fidelity. Furthermore, the FRM undergoes dynamic optimization throughout the process. It not only ensures that the restoration network stays precisely aligned with the real face manifold, but also effectively prevents reward hacking. Experiments on synthetic and wild datasets demonstrate that our method outperforms state-of-the-art methods, significantly improving identity consistency and facial details. The source codes, data, and models are available at: this https URL.
• 摘要：奖励反馈学习（REFL）最近在将模型输出与各种生成任务的人类偏好保持一致方面表现出巨大的潜力。在这项工作中，我们将一个名为diffusionreward的REFL框架首次介绍了盲人恢复任务。扩散性有效地克服了基于扩散的方法的局限性，这些方法通常无法产生逼真的面部细节并表现出较差的身份一致性。我们框架的核心是面部奖励模型（FRM），该模型是使用精心注释的数据训练的。它提供了反馈信号，该信号在转向恢复网络的优化过程中起着关键作用。特别是，我们的REFL框架将梯度流纳入了现成的面部恢复方法的转换过程，以指导模型参数的更新。指导梯度由三个方面进行协作：（i）FRM确保恢复的面孔的感知质量； （ii）一个正规化术语，可作为保障生成多样性的保障； （iii）保持面部忠诚的结构一致性约束。此外，FRM在整个过程中都进行了动态优化。它不仅可以确保恢复网络与真实面部多种多样保持一致，还可以有效地防止奖励黑客入侵。关于合成和野生数据集的实验表明，我们的方法表现优于最先进的方法，从而显着提高了身份一致性和面部细节。源代码，数据和模型可在以下网址提供：此HTTPS URL。

Title: Diffusion Classifiers Understand Compositionality, but Conditions Apply

• Authors: Yujin Jeong, Arnas Uselis, Seong Joon Oh, Anna Rohrbach
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.17955
• Pdf URL: https://arxiv.org/pdf/2505.17955
• Copy Paste: [[2505.17955]] Diffusion Classifiers Understand Compositionality, but Conditions Apply(https://arxiv.org/abs/2505.17955)
• Keywords: generative
• Abstract: Understanding visual scenes is fundamental to human intelligence. While discriminative models have significantly advanced computer vision, they often struggle with compositional understanding. In contrast, recent generative text-to-image diffusion models excel at synthesizing complex scenes, suggesting inherent compositional capabilities. Building on this, zero-shot diffusion classifiers have been proposed to repurpose diffusion models for discriminative tasks. While prior work offered promising results in discriminative compositional scenarios, these results remain preliminary due to a small number of benchmarks and a relatively shallow analysis of conditions under which the models succeed. To address this, we present a comprehensive study of the discriminative capabilities of diffusion classifiers on a wide range of compositional tasks. Specifically, our study covers three diffusion models (SD 1.5, 2.0, and, for the first time, 3-m) spanning 10 datasets and over 30 tasks. Further, we shed light on the role that target dataset domains play in respective performance; to isolate the domain effects, we introduce a new diagnostic benchmark Self-Bench comprised of images created by diffusion models themselves. Finally, we explore the importance of timestep weighting and uncover a relationship between domain gap and timestep sensitivity, particularly for SD3-m. To sum up, diffusion classifiers understand compositionality, but conditions apply! Code and dataset are available at this https URL.
• 摘要：了解视觉场景是人类智能的基础。尽管判别模型具有显着高级的计算机视觉，但它们通常在构图理解方面挣扎。相反，最近生成的文本对图像扩散模型在合成复杂场景方面表现出色，表明固有的组成能力。在此基础上，已经提出了零射击扩散分类器来重新利用歧视任务的扩散模型。虽然先前的工作在歧视性组成方案中提供了有希望的结果，但由于少量基准和对模型成功的条件的相对较浅的分析，这些结果仍然是初步的。为了解决这个问题，我们介绍了在各种组成任务上扩散分类器的判别能力的全面研究。具体而言，我们的研究涵盖了三个扩散模型（SD 1.5、2.0，第一次3米）涵盖10个数据集和30多个任务。此外，我们阐明了目标数据集域在各自的性能中的作用。为了隔离域效应，我们引入了一个新的诊断基准自我基础，由扩散模型本身创建的图像组成。最后，我们探讨了时间段加权的重要性，并发现了域间隙和时间段灵敏度之间的关系，尤其是对于SD3-M。总而言之，扩散分类器了解组成性，但条件适用！代码和数据集可在此HTTPS URL上找到。

Title: ADLGen: Synthesizing Symbolic, Event-Triggered Sensor Sequences for Human Activity Modeling

• Authors: Weihang You, Hanqi Jiang, Zishuai Liu, Zihang Xie, Tianming Liu, Jin Lu, Fei Dou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.17987
• Pdf URL: https://arxiv.org/pdf/2505.17987
• Copy Paste: [[2505.17987]] ADLGen: Synthesizing Symbolic, Event-Triggered Sensor Sequences for Human Activity Modeling(https://arxiv.org/abs/2505.17987)
• Keywords: generation, generative
• Abstract: Real world collection of Activities of Daily Living data is challenging due to privacy concerns, costly deployment and labeling, and the inherent sparsity and imbalance of human behavior. We present ADLGen, a generative framework specifically designed to synthesize realistic, event triggered, and symbolic sensor sequences for ambient assistive environments. ADLGen integrates a decoder only Transformer with sign based symbolic temporal encoding, and a context and layout aware sampling mechanism to guide generation toward semantically rich and physically plausible sensor event sequences. To enhance semantic fidelity and correct structural inconsistencies, we further incorporate a large language model into an automatic generate evaluate refine loop, which verifies logical, behavioral, and temporal coherence and generates correction rules without manual intervention or environment specific tuning. Through comprehensive experiments with novel evaluation metrics, ADLGen is shown to outperform baseline generators in statistical fidelity, semantic richness, and downstream activity recognition, offering a scalable and privacy-preserving solution for ADL data synthesis.
• 摘要：由于隐私问题，昂贵的部署和标签以及人类行为的固有的稀疏性和不平衡，日常生活数据活动的现实世界集合具有挑战性。我们提出了ADLGEN，这是一种专门设计的生成框架，该框架旨在为环境辅助环境综合现实，事件触发和符号传感器序列。 ADLGEN通过基于符号的符号时间编码集成了仅解码器的变压器，以及一个上下文和布局意识到的采样机制，以指导生成语义上富含且物理上合理的传感器事件序列。为了增强语义保真度并纠正结构上的不一致，我们将大型语言模型进一步纳入自动生成的评估循环，该循环验证了逻辑，行为和时间连贯性，并生成无手动干预或特定于环境调整的校正规则。通过具有新颖的评估指标的全面实验，ADLGEN显示出在统计保真度，语义丰富度和下游活动识别方面的表现优于基线发生器，为ADL数据综合提供了可扩展且具有隐私性的解决方案。

Title: Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling

• Authors: Matthieu Blanke, Yongquan Qu, Sara Shamekh, Pierre Gentine
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.18017
• Pdf URL: https://arxiv.org/pdf/2505.18017
• Copy Paste: [[2505.18017]] Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling(https://arxiv.org/abs/2505.18017)
• Keywords: generative
• Abstract: Deep generative models hold great promise for representing complex physical systems, but their deployment is currently limited by the lack of guarantees on the physical plausibility of the generated outputs. Ensuring that known physical constraints are enforced is therefore critical when applying generative models to scientific and engineering problems. We address this limitation by developing a principled framework for sampling from a target distribution while rigorously satisfying physical constraints. Leveraging the variational formulation of Langevin dynamics, we propose Split Augmented Langevin (SAL), a novel primal-dual sampling algorithm that enforces constraints progressively through variable splitting, with convergence guarantees. While the method is developed theoretically for Langevin dynamics, we demonstrate its effective applicability to diffusion models. In particular, we use constrained diffusion models to generate physical fields satisfying energy and mass conservation laws. We apply our method to diffusion-based data assimilation on a complex physical system, where enforcing physical constraints substantially improves both forecast accuracy and the preservation of critical conserved quantities. We also demonstrate the potential of SAL for challenging feasibility problems in optimal control.
• 摘要：深层生成模型对代表复杂的物理系统具有巨大的希望，但是目前它们的部署受到生成产出的物理合理性的保证的限制。因此，在将生成模型应用于科学和工程问题时，确保实施已知的物理约束至关重要。我们通过开发一个从目标分布中取样的原则框架来解决此限制，同时严格满足物理约束。利用Langevin动力学的变异配方，我们提出了拆分增强的Langevin（SAL），这是一种新型的原始二重式采样算法，该算法通过可变分割逐渐强制执行约束，并具有收敛的保证。在理论上为Langevin动力学开发该方法时，我们证明了其对扩散模型的有效适用性。特别是，我们使用受约束的扩散模型来生成满足能量和质量保护定律的物理领域。我们将方法应用于基于扩散的数据同化在复杂的物理系统上，在该系统中，实施物理约束基本上可提高预测准确性和保留关键保守量。我们还证明了SAL在最佳控制中有挑战性的可行性问题的潜力。

Title: CAMME: Adaptive Deepfake Image Detection with Multi-Modal Cross-Attention

• Authors: Naseem Khan, Tuan Nguyen, Amine Bermak, Issa Khalil
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.18035
• Pdf URL: https://arxiv.org/pdf/2505.18035
• Copy Paste: [[2505.18035]] CAMME: Adaptive Deepfake Image Detection with Multi-Modal Cross-Attention(https://arxiv.org/abs/2505.18035)
• Keywords: generative
• Abstract: The proliferation of sophisticated AI-generated deepfakes poses critical challenges for digital media authentication and societal security. While existing detection methods perform well within specific generative domains, they exhibit significant performance degradation when applied to manipulations produced by unseen architectures--a fundamental limitation as generative technologies rapidly evolve. We propose CAMME (Cross-Attention Multi-Modal Embeddings), a framework that dynamically integrates visual, textual, and frequency-domain features through a multi-head cross-attention mechanism to establish robust cross-domain generalization. Extensive experiments demonstrate CAMME's superiority over state-of-the-art methods, yielding improvements of 12.56% on natural scenes and 13.25% on facial deepfakes. The framework demonstrates exceptional resilience, maintaining (over 91%) accuracy under natural image perturbations and achieving 89.01% and 96.14% accuracy against PGD and FGSM adversarial attacks, respectively. Our findings validate that integrating complementary modalities through cross-attention enables more effective decision boundary realignment for reliable deepfake detection across heterogeneous generative architectures.
• 摘要：复杂的AI生成的深烟的扩散为数字媒体身份验证和社会安全带来了关键的挑战。尽管现有的检测方法在特定的生成域中表现良好，但当应用于不看见的体系结构产生的操作时，它们表现出明显的性能降解，这是生成技术迅速发展的基本限制。我们提出了Camme（跨注意的多模式嵌入），该框架通过多头跨注意机制动态整合视觉，文本和频域特征，以建立强大的跨域概括。广泛的实验表明，Camme比最新方法的优越性在自然场景中的提高为12.56％，面部深层摄影的提高了13.25％。该框架表现出了出色的弹性，在自然图像扰动下（超过91％）的精度（超过91％），分别针对PGD和FGSM对抗性攻击达到89.01％和96.14％的精度。我们的发现验证了通过交叉注意通过互补方式整合互补模式的，可以使更有效的决策边界重新调整，以跨异构生成体系结构进行可靠的深层检测。

Title: Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions

• Authors: Yizhou Xu, Florent Krzakala, Lenka Zdeborová
• Subjects: cs.LG, cond-mat.dis-nn, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.18046
• Pdf URL: https://arxiv.org/pdf/2505.18046
• Copy Paste: [[2505.18046]] Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions(https://arxiv.org/abs/2505.18046)
• Keywords: generative
• Abstract: The Restricted Boltzmann Machine (RBM) is one of the simplest generative neural networks capable of learning input distributions. Despite its simplicity, the analysis of its performance in learning from the training data is only well understood in cases that essentially reduce to singular value decomposition of the data. Here, we consider the limit of a large dimension of the input space and a constant number of hidden units. In this limit, we simplify the standard RBM training objective into a form that is equivalent to the multi-index model with non-separable regularization. This opens a path to analyze training of the RBM using methods that are established for multi-index models, such as Approximate Message Passing (AMP) and its state evolution, and the analysis of Gradient Descent (GD) via the dynamical mean-field theory. We then give rigorous asymptotics of the training dynamics of RBM on data generated by the spiked covariance model as a prototype of a structure suitable for unsupervised learning. We show in particular that RBM reaches the optimal computational weak recovery threshold, aligning with the BBP transition, in the spiked covariance model.
• 摘要：受限的玻尔兹曼机器（RBM）是能够学习输入分布的最简单的生成神经网络之一。尽管它很简单，但只有在基本上减少数据的奇异价值分解的情况下，才对其从培训数据中学习的性能进行分析。在这里，我们考虑了输入空间的较大维度和恒定数量的隐藏单元的极限。在此限制中，我们将标准RBM训练目标简化为一种形式，该形式等同于具有不可分割的正则化的多指数模型。这开辟了一条使用针对多指数模型建立的方法（例如近似消息传递（AMP）及其状态进化）以及通过动态平均场理论对梯度下降（GD）的分析来分析RBM训练的途径。然后，我们将RBM训练动力学的严格渐近造型对峰值协方差模型产生的数据进行，作为适合无监督学习的结构的原型。我们特别表明，RBM在尖峰协方差模型中达到了最佳计算弱恢复阈值，与BBP转变保持一致。

Title: RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration

• Authors: Sudarshan Rajagopalan, Kartik Narayan, Vishal M. Patel
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.18047
• Pdf URL: https://arxiv.org/pdf/2505.18047
• Copy Paste: [[2505.18047]] RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration(https://arxiv.org/abs/2505.18047)
• Keywords: restoration, generation, generative
• Abstract: The use of latent diffusion models (LDMs) such as Stable Diffusion has significantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM-based frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. To address this, we propose RestoreVAR, a novel generative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over $\mathbf{10\times}$ faster inference. RestoreVAR leverages visual autoregressive modeling (VAR), a recently introduced approach which performs scale-space autoregression for image generation. VAR achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. To optimally exploit these advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed cross-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-art performance among generative AiOR methods, while also exhibiting strong generalization capabilities.
• 摘要：潜在扩散模型（LDM）（例如稳定扩散）的使用显着提高了多合一图像恢复（AIOR）方法的感知质量，同时还增强了其泛化能力。但是，这些基于LDM的框架由于其迭代降解过程而遭受缓慢的推理，因此使它们不切实际地用于时间敏感的应用。为了解决这个问题，我们提出了RestoreVar，这是一种新颖的Aior生成方法，它在恢复性能方面显着优于基于LDM的模型，同时实现$ \ Mathbf {10 \ times} $更快的推断。 RestoreVar利用了视觉自动回归建模（VAR），这是一种最近引入的方法，该方法可为图像生成进行规模空间自动化。 VAR的性能与最先进的扩散变压器具有可比的性能，其计算成本大大降低。为了最佳利用Aior的VAR的这些优势，我们提出了体系结构的修改和改进，包括精心设计的跨注意机制和潜在空间改进模块，该模块是针对AIOR任务量身定制的。广泛的实验表明，RestoreVar在生成Aior方法中实现了最先进的性能，同时也表现出强大的概括能力。

Title: SpikeGen: Generative Framework for Visual Spike Stream Processing

• Authors: Gaole Dai, Menghang Dong, Rongyu Zhang, Ruichuan An, Shanghang Zhang, Tiejun Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.18049
• Pdf URL: https://arxiv.org/pdf/2505.18049
• Copy Paste: [[2505.18049]] SpikeGen: Generative Framework for Visual Spike Stream Processing(https://arxiv.org/abs/2505.18049)
• Keywords: generation, generative
• Abstract: Neuromorphic Visual Systems, such as spike cameras, have attracted considerable attention due to their ability to capture clear textures under dynamic conditions. This capability effectively mitigates issues related to motion and aperture blur. However, in contrast to conventional RGB modalities that provide dense spatial information, these systems generate binary, spatially sparse frames as a trade-off for temporally rich visual streams. In this context, generative models emerge as a promising solution to address the inherent limitations of sparse data. These models not only facilitate the conditional fusion of existing information from both spike and RGB modalities but also enable the conditional generation based on latent priors. In this study, we introduce a robust generative processing framework named SpikeGen, designed for visual spike streams captured by spike cameras. We evaluate this framework across multiple tasks involving mixed spike-RGB modalities, including conditional image/video deblurring, dense frame reconstruction from spike streams, and high-speed scene novel-view synthesis. Supported by comprehensive experimental results, we demonstrate that leveraging the latent space operation abilities of generative models allows us to effectively address the sparsity of spatial information while fully exploiting the temporal richness of spike streams, thereby promoting a synergistic enhancement of different visual modalities.
• 摘要：神经形态的视觉系统（例如尖峰摄像机）由于能够在动态条件下捕获清晰纹理的能力而引起了很大的关注。该能力有效地减轻了与运动和光圈模糊有关的问题。但是，与提供密集的空间信息的常规RGB模式相反，这些系统在空间上产生了二进制的稀疏框架，作为时间富含时间丰富的视觉流的权衡。在这种情况下，生成模型作为解决稀疏数据固有局限性的有前途解决方案。这些模型不仅促进了来自Spike和RGB模式的现有信息的条件融合，而且还可以基于潜在的先验来实现条件生成。在这项研究中，我们介绍了一个名为Spikegen的强大生成处理框架，该框架是为Spike摄像机捕获的视觉尖峰流而设计的。我们在涉及混合Spike-RGB模式的多个任务中评估了此框架，包括有条件的图像/视频脱毛，尖峰流的密集框架重建以及高速场景Novel-View-View-View-View-view-view综合。在全面的实验结果的支持下，我们证明了利用生成模型的潜在空间操作能力使我们能够有效地解决空间信息的稀疏性，同时充分利用尖峰流的时间丰富性，从而促进不同视觉方式的协同增强。

Title: Reward Model Generalization for Compute-Aware Test-Time Reasoning

• Authors: Zeen Song, Wenwen Qiang, Siyu Zhao, Changwen Zheng, Gang Hua
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.18065
• Pdf URL: https://arxiv.org/pdf/2505.18065
• Copy Paste: [[2505.18065]] Reward Model Generalization for Compute-Aware Test-Time Reasoning(https://arxiv.org/abs/2505.18065)
• Keywords: generation
• Abstract: External test-time reasoning enhances large language models (LLMs) by decoupling generation and selection. At inference time, the model generates multiple reasoning paths, and an auxiliary process reward model (PRM) is used to score and select the best one. A central challenge in this setting is test-time compute optimality (TCO), i.e., how to maximize answer accuracy under a fixed inference budget. In this work, we establish a theoretical framework to analyze how the generalization error of the PRM affects compute efficiency and reasoning performance. Leveraging PAC-Bayes theory, we derive generalization bounds and show that a lower generalization error of PRM leads to fewer samples required to find correct answers. Motivated by this analysis, we propose Compute-Aware Tree Search (CATS), an actor-critic framework that dynamically controls search behavior. The actor outputs sampling hyperparameters based on reward distributions and sparsity statistics, while the critic estimates their utility to guide budget allocation. Experiments on the MATH and AIME benchmarks with various LLMs and PRMs demonstrate that CATS consistently outperforms other external TTS methods, validating our theoretical predictions.
• 摘要：外部测试时间推理通过解耦生成和选择增强了大型语言模型（LLM）。在推理时，该模型会生成多个推理路径，并且使用辅助过程奖励模型（PRM）来评分并选择最佳奖励模型。在这种情况下，一个核心挑战是测试时间计算最优性（TCO），即如何在固定推理预算下最大化答案准确性。在这项工作中，我们建立了一个理论框架，以分析PRM的概括误差如何影响计算效率和推理性能。利用Pac-Bayes理论，我们得出了泛化界限，并表明PRM的概括误差较低会导致找到正确答案所需的样本更少。在此分析的激励下，我们提出了Compute-Aware Tree Search（CAT），这是一个动态控制搜索行为的参与者批评框架。演员根据奖励分布和稀疏统计数据对超参数进行抽样，而评论家则估计其指导预算分配的效用。具有各种LLM和PRM的数学和AIME基准测试的实验表明，猫始终优于其他外部TTS方法，从而验证了我们的理论预测。

Title: DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation

• Authors: Junhao Chen, Mingjin Chen, Jianjin Xu, Xiang Li, Junting Dong, Mingze Sun, Puhua Jiang, Hongxiang Li, Yuhang Yang, Hao Zhao, Xiaoxiao Long, Ruqi Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.18078
• Pdf URL: https://arxiv.org/pdf/2505.18078
• Copy Paste: [[2505.18078]] DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation(https://arxiv.org/abs/2505.18078)
• Keywords: generation
• Abstract: Controllable video generation (CVG) has advanced rapidly, yet current systems falter when more than one actor must move, interact, and exchange positions under noisy control signals. We address this gap with DanceTogether, the first end-to-end diffusion framework that turns a single reference image plus independent pose-mask streams into long, photorealistic videos while strictly preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at every denoising step by fusing robust tracking masks with semantically rich-but noisy-pose heat-maps, eliminating the identity drift and appearance bleeding that plague frame-wise pipelines. To train and evaluate at scale, we introduce (i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii) HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a significant margin. Moreover, we show that a one-hour fine-tune yields convincing human-robot videos, underscoring broad generalization to embodied-AI and HRI tasks. Extensive ablations confirm that persistent identity-action binding is critical to these gains. Together, our model, datasets, and benchmark lift CVG from single-subject choreography to compositionally controllable, multi-actor interaction, opening new avenues for digital production, simulation, and embodied intelligence. Our video demos and code are available at this https URL.
• 摘要：可控的视频生成（CVG）已迅速发展，但是当一个以上的演员必须在嘈杂的控制信号下移动，互动和交换位置时，​​当前的系统会步履蹒跚。我们用Dancetogether解决了这一差距，这是第一个将单个参考图像加上独立姿势掩码流变成长时间逼真的视频的端到端扩散框架，同时严格保留了每个身份。一个新颖的面具放映器通过将强大的跟踪口罩与语义上丰富但嘈杂的姿势热图融合来绑定“谁”和“如何”，从而消除了身份漂移和外观，使瘟疫框架在管道上消失。 To train and evaluate at scale, we introduce (i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii) HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the DanceTogEval-100 test suite covering dance, boxing, wrestling,瑜伽和花样滑冰。在VideObench上，Dancetogether的表现优于先前的艺术。此外，我们表明一个小时的微调产生了令人信服的人类机器人视频，强调了对体现的AI和HRI任务的广泛概括。广泛的消融证实，持续的身份行动结合对这些收益至关重要。我们的模型，数据集和基准提升CVG从单个主体编排到构图可控的多动互动，开辟了用于数字生产，模拟和体现智能的新途径。我们的视频演示和代码可在此HTTPS URL上找到。

Title: An Iterative Framework for Generative Backmapping of Coarse Grained Proteins

• Authors: Georgios Kementzidis, Erin Wong, John Nicholson, Ruichen Xu, Yuefan Deng
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.18082
• Pdf URL: https://arxiv.org/pdf/2505.18082
• Copy Paste: [[2505.18082]] An Iterative Framework for Generative Backmapping of Coarse Grained Proteins(https://arxiv.org/abs/2505.18082)
• Keywords: generative
• Abstract: The techniques of data-driven backmapping from coarse-grained (CG) to fine-grained (FG) representation often struggle with accuracy, unstable training, and physical realism, especially when applied to complex systems such as proteins. In this work, we introduce a novel iterative framework by using conditional Variational Autoencoders and graph-based neural networks, specifically designed to tackle the challenges associated with such large-scale biomolecules. Our method enables stepwise refinement from CG beads to full atomistic details. We outline the theory of iterative generative backmapping and demonstrate via numerical experiments the advantages of multistep schemes by applying them to proteins of vastly different structures with very coarse representations. This multistep approach not only improves the accuracy of reconstructions but also makes the training process more computationally efficient for proteins with ultra-CG representations.
• 摘要：数据驱动的反向图从粗粒（CG）到细粒度（FG）表示的技术通常会在准确性，不稳定的训练和物理现实主义方面挣扎，尤其是当应用于诸如蛋白质等复杂系统时。在这项工作中，我们通过使用条件变分自动编码器和基于图的神经网络引入了一个新颖的迭代框架，该框架专门设计，以应对与此类大规模生物分子相关的挑战。我们的方法使从CG珠到完整的原子细节可以逐步完善。我们概述了迭代生成背景的理论，并通过数值实验将多步项的优势应用于具有非常粗糙的表示非常粗的蛋白质的蛋白质，从而概述了多步骤方案的优势。这种多步方法不仅提高了重建的准确性，而且还使训练过程对具有超CG表示的蛋白质在计算上更有效。

Title: What Do You Need for Diverse Trajectory Stitching in Diffusion Planning?

• Authors: Quentin Clark, Florian Shkurti
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.18083
• Pdf URL: https://arxiv.org/pdf/2505.18083
• Copy Paste: [[2505.18083]] What Do You Need for Diverse Trajectory Stitching in Diffusion Planning?(https://arxiv.org/abs/2505.18083)
• Keywords: generative
• Abstract: In planning, stitching is an ability of algorithms to piece together sub-trajectories of data they are trained on to generate new and diverse behaviours. While stitching is historically a strength of offline reinforcement learning, recent generative behavioural cloning (BC) methods have also shown proficiency at stitching. However, the main factors behind this are poorly understood, hindering the development of new algorithms that can reliably stitch. Focusing on diffusion planners trained via BC, we find two properties are needed to compose: \emph{positional equivariance} and \emph{local receptiveness}. We use these two properties to explain architecture, data, and inference choices in existing generative BC methods based on diffusion planning, including replanning frequency, data augmentation, and data scaling. Experimental comparisions show that (1) while locality is more important than positional equivariance in creating a diffusion planner capable of composition, both are crucial (2) enabling these properties through relatively simple architecture choices can be competitive with more computationally expensive methods such as replanning or scaling data, and (3) simple inpainting-based guidance can guide architecturally compositional models to enable generalization in goal-conditioned settings.
• 摘要：在计划中，缝线是算法的一种能力，将他们经过培训以产生新的和多样化的行为进行培训。虽然缝线在历史上是离线增强学习的强度，但最近的生成行为克隆（BC）方法也表明了缝线的熟练程度。但是，这背后的主要因素是鲜为人知的，阻碍了可以可靠缝制的新算法的发展。专注于通过BC训练的扩散计划者，我们发现需要两个属性来组成：\ emph {positial equivariance}和\ emph {local Focections}。我们使用这两个属性来解释基于扩散计划的现有生成BC方法中的体系结构，数据和推理选择，包括重新启动频率，数据扩展和数据扩展。实验比较表明，虽然（1）在创建能够组成的扩散计划者方面比位置均衡性更为重要，但两者都是至关重要的（2）通过相对简单的体系结构选择使这些属性具有相对简单的架构选择，可以通过更昂贵的方法竞争更昂贵的方法，例如更换或扩展数据，以及基于基于Inspainting Intage Indection的构图构建模型，可以启动构造构图。

Title: CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays

• Authors: Hyungyung Lee, Geon Choi, Jung-Oh Lee, Hangyul Yoon, Hyuk Gi Hong, Edward Choi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.18087
• Pdf URL: https://arxiv.org/pdf/2505.18087
• Copy Paste: [[2505.18087]] CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays(https://arxiv.org/abs/2505.18087)
• Keywords: generation
• Abstract: Recent progress in Large Vision-Language Models (LVLMs) has enabled promising applications in medical tasks, such as report generation and visual question answering. However, existing benchmarks focus mainly on the final diagnostic answer, offering limited insight into whether models engage in clinically meaningful reasoning. To address this, we present CheXStruct and CXReasonBench, a structured pipeline and benchmark built on the publicly available MIMIC-CXR-JPG dataset. CheXStruct automatically derives a sequence of intermediate reasoning steps directly from chest X-rays, such as segmenting anatomical regions, deriving anatomical landmarks and diagnostic measurements, computing diagnostic indices, and applying clinical thresholds. CXReasonBench leverages this pipeline to evaluate whether models can perform clinically valid reasoning steps and to what extent they can learn from structured guidance, enabling fine-grained and transparent assessment of diagnostic reasoning. The benchmark comprises 18,988 QA pairs across 12 diagnostic tasks and 1,200 cases, each paired with up to 4 visual inputs, and supports multi-path, multi-stage evaluation including visual grounding via anatomical region selection and diagnostic measurements. Even the strongest of 10 evaluated LVLMs struggle with structured reasoning and generalization, often failing to link abstract knowledge with anatomically grounded visual interpretation. The code is available at this https URL
• 摘要：大型视觉模型（LVLM）的最新进展已实现了医疗任务中有希望的应用程序，例如报告生成和视觉问题答案。但是，现有的基准主要关注最终诊断答案，从而有限地了解模型是否参与临床意义的推理。为了解决这个问题，我们提出了ChexStruct和CXREASONBENCH，这是一种结构化管道和基准测试，建立在公开可用的Mimic-CXR-JPG数据集上。 CHEXSTRUCT会自动从胸部X射线中自动得出一系列中间推理步骤，例如分割解剖区域，得出解剖学地标和诊断测量，计算诊断指数以及应用临床阈值。 CXREASON BENCH利用该管道来评估模型是否可以执行临床上有效的推理步骤，以及他们可以从结构化指导中学习的程度，从而对诊断推理进行细粒度和透明的评估。该基准包括在12个诊断任务和1200个案例中包含18,988个QA对，每个案例最多可与多达4个视觉输入配对，并支持多路径，多阶段评估，包括通过解剖区域选择和诊断测量值进行视觉接地。即使是10个评估的LVLM中最强大的LVLM在结构化的推理和概括方面都挣扎，通常无法将抽象知识与解剖学上的视觉解释联系起来。该代码可在此HTTPS URL上找到

Title: DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations

• Authors: Ziqiao Peng, Yanbo Fan, Haoyu Wu, Xuan Wang, Hongyan Liu, Jun He, Zhaoxin Fan
• Subjects: cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2505.18096
• Pdf URL: https://arxiv.org/pdf/2505.18096
• Copy Paste: [[2505.18096]] DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations(https://arxiv.org/abs/2505.18096)
• Keywords: generation
• Abstract: In face-to-face conversations, individuals need to switch between speaking and listening roles seamlessly. Existing 3D talking head generation models focus solely on speaking or listening, neglecting the natural dynamics of interactive conversation, which leads to unnatural interactions and awkward transitions. To address this issue, we propose a new task -- multi-round dual-speaker interaction for 3D talking head generation -- which requires models to handle and generate both speaking and listening behaviors in continuous conversation. To solve this task, we introduce DualTalk, a novel unified framework that integrates the dynamic behaviors of speakers and listeners to simulate realistic and coherent dialogue interactions. This framework not only synthesizes lifelike talking heads when speaking but also generates continuous and vivid non-verbal feedback when listening, effectively capturing the interplay between the roles. We also create a new dataset featuring 50 hours of multi-round conversations with over 1,000 characters, where participants continuously switch between speaking and listening roles. Extensive experiments demonstrate that our method significantly enhances the naturalness and expressiveness of 3D talking heads in dual-speaker conversations. We recommend watching the supplementary video: this https URL.
• 摘要：在面对面的对话中，个人需要在说话和聆听角色之间切换。现有的3D谈话校长模型仅着眼于说话或聆听，忽略了交互式对话的自然动态，这导致了不自然的互动和尴尬的过渡。为了解决这个问题，我们提出了一项新任务 - 3D Talking Head Generation的多轮双扬声器交互 - 这需要模型来处理和生成不断对话的口语和听力行为。为了解决这项任务，我们介绍了DualTalk，这是一个新颖的统一框架，该框架整合了说话者和听众的动态行为，以模拟现实且连贯的对话交互。该框架不仅可以在说话时综合栩栩如生的说话头，而且在聆听时会产生连续而生动的非语言反馈，从而有效地捕获了角色之间的相互作用。我们还创建了一个新的数据集，其中包含50小时的多轮对话，其中有1,000多个字符，参与者在讲话和听力角色之间不断切换。广泛的实验表明，我们的方法显着增强了双扬声器对话中3D讲话头的自然性和表现力。我们建议观看补充视频：此HTTPS URL。

Title: F-ANcGAN: An Attention-Enhanced Cycle Consistent Generative Adversarial Architecture for Synthetic Image Generation of Nanoparticles

• Authors: Varun Ajith, Anindya Pal, Saumik Bhattacharya, Sayantari Ghosh
• Subjects: cs.CV, cond-mat.mtrl-sci, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.18106
• Pdf URL: https://arxiv.org/pdf/2505.18106
• Copy Paste: [[2505.18106]] F-ANcGAN: An Attention-Enhanced Cycle Consistent Generative Adversarial Architecture for Synthetic Image Generation of Nanoparticles(https://arxiv.org/abs/2505.18106)
• Keywords: generation, generative
• Abstract: Nanomaterial research is becoming a vital area for energy, medicine, and materials science, and accurate analysis of the nanoparticle topology is essential to determine their properties. Unfortunately, the lack of high-quality annotated datasets drastically hinders the creation of strong segmentation models for nanoscale imaging. To alleviate this problem, we introduce F-ANcGAN, an attention-enhanced cycle consistent generative adversarial system that can be trained using a limited number of data samples and generates realistic scanning electron microscopy (SEM) images directly from segmentation maps. Our model uses a Style U-Net generator and a U-Net segmentation network equipped with self-attention to capture structural relationships and applies augmentation methods to increase the variety of the dataset. The architecture reached a raw FID score of 17.65 for TiO$_2$ dataset generation, with a further reduction in FID score to nearly 10.39 by using efficient post-processing techniques. By facilitating scalable high-fidelity synthetic dataset generation, our approach can improve the effectiveness of downstream segmentation task training, overcoming severe data shortage issues in nanoparticle analysis, thus extending its applications to resource-limited fields.
• 摘要：纳米材料研究正成为能源，医学和材料科学的重要领域，对纳米颗粒拓扑的准确分析对于确定其特性至关重要。不幸的是，缺乏高质量的注释数据集极大地阻碍了纳米级成像的强分段模型的创建。为了减轻此问题，我们引入了F-Ancgan，这是一种注意力增强的循环一致的生成对抗系统，可以使用有限的数据样本训练，并直接从分段图中生成逼真的扫描电子显微镜（SEM）图像。我们的模型使用配备自我注意力的U-NET生成器和U-NET分割网络，以捕获结构关系并应用增强方法来增加数据集的多样性。对于TIO $ _2 $数据集生成的架构达到了17.65的原始FID得分，通过使用有效的后处理技术，FID分数将FID得分进一步降低到近10.39。通过促进可扩展的高保真性合成数据集生成，我们的方法可以提高下游分割任务训练的有效性，克服纳米粒子分析中严重的数据短缺问题，从而将其应用扩展到资源有限的领域。

Title: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning

• Authors: Huayu Chen, Kaiwen Zheng, Qinsheng Zhang, Ganqu Cui, Yin Cui, Haotian Ye, Tsung-Yi Lin, Ming-Yu Liu, Jun Zhu, Haoxiang Wang
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.18116
• Pdf URL: https://arxiv.org/pdf/2505.18116
• Copy Paste: [[2505.18116]] Bridging Supervised Learning and Reinforcement Learning in Math Reasoning(https://arxiv.org/abs/2505.18116)
• Keywords: generation
• Abstract: Reinforcement Learning (RL) has played a central role in the recent surge of LLMs' math abilities by enabling self-improvement through binary verifier signals. In contrast, Supervised Learning (SL) is rarely considered for such verification-driven training, largely due to its heavy reliance on reference answers and inability to reflect on mistakes. In this work, we challenge the prevailing notion that self-improvement is exclusive to RL and propose Negative-aware Fine-Tuning (NFT) -- a supervised approach that enables LLMs to reflect on their failures and improve autonomously with no external teachers. In online training, instead of throwing away self-generated negative answers, NFT constructs an implicit negative policy to model them. This implicit policy is parameterized with the same positive LLM we target to optimize on positive data, enabling direct policy optimization on all LLMs' generations. We conduct experiments on 7B and 32B models in math reasoning tasks. Results consistently show that through the additional leverage of negative feedback, NFT significantly improves over SL baselines like Rejection sampling Fine-Tuning, matching or even surpassing leading RL algorithms like GRPO and DAPO. Furthermore, we demonstrate that NFT and GRPO are actually equivalent in strict-on-policy training, even though they originate from entirely different theoretical foundations. Our experiments and theoretical findings bridge the gap between SL and RL methods in binary-feedback learning systems.
• 摘要：强化学习（RL）通过通过二元验证器信号实现自我完善，在最近的LLMS数学能力激增中发挥了核心作用。相比之下，很少考虑监督的学习（SL）用于这种验证驱动的培训，这在很大程度上是由于其对参考答案的严重依赖以及无法反思错误。在这项工作中，我们挑战了普遍的观念，即自我完善是RL独有的，并提出了负面意识的微调（NFT） - 一种有监督的方法，使LLM可以反思自己的失败并没有外部教师自主进行自主改善。在在线培训中，NFT并没有丢弃自我生成的负面答案，而是构建了一个隐含的负面政策来对其进行建模。该隐式策略的参数为我们针对的相同的正面LLM，以优化正面数据，从而对所有LLMS的几代进行了直接的策略优化。我们在数学推理任务中对7B和32B模型进行实验。结果一致地表明，通过负面反馈的额外杠杆作用，NFT显着改善了SL基准，例如拒绝采样，匹配，甚至超过了GRPO和DAPO等领先的RL算法。此外，我们证明了NFT和GRPO实际上在严格的政策培训中是等效的，即使它们源于完全不同的理论基础。我们的实验和理论发现弥合了二进制反馈学习系统中SL和RL方法之间的差距。

Title: TokBench: Evaluating Your Visual Tokenizer before Visual Generation

• Authors: Junfeng Wu, Dongliang Luo, Weizhi Zhao, Zhihao Xie, Yuanhao Wang, Junyi Li, Xudong Xie, Yuliang Liu, Xiang Bai
• Subjects: cs.CV, cs.DB
• Abstract URL: https://arxiv.org/abs/2505.18142
• Pdf URL: https://arxiv.org/pdf/2505.18142
• Copy Paste: [[2505.18142]] TokBench: Evaluating Your Visual Tokenizer before Visual Generation(https://arxiv.org/abs/2505.18142)
• Keywords: generation
• Abstract: In this work, we reveal the limitations of visual tokenizers and VAEs in preserving fine-grained features, and propose a benchmark to evaluate reconstruction performance for two challenging visual contents: text and face. Image tokenization has significantly advanced visual generation and multimodal modeling, particularly with autoregressive models due to the modeling simplicity of discrete tokens. Autoregressive models typically rely on image tokenizers to compress images into discrete tokens for sequential prediction, whereas diffusion models often operate on continuous latent space to reduce computational costs. However, both visual compression approaches inevitably lose visual information, thereby limiting the upper bound of visual generation quality. To evaluate how these compression losses affect text and faces, the most human-sensitive visual elements, we first collect and curate a collection of text and faces images from existing datasets, ensuring clarity and diversity. For text reconstruction, we employ OCR models to assess the recognition accuracy of the reconstructed text, and then we measure feature similarity between original and reconstructed faces thereby quantifying faces reconstruction fidelity. Our method is highly lightweight, requiring just 2GB memory and 4 minutes to complete evaluations. With our benchmark, we analyze the reconstruction quality of text and faces at various scales across different image tokenizers and VAEs. Our results demonstrate that modern visual tokenizers still struggle to preserve fine-grained features, particularly at smaller scales. Furthermore, we extend this evaluation framework to the video, conducting a comprehensive analysis of video tokenizers. Additionally, we find that traditional metrics fail to accurately reflect the reconstruction performance for faces and text, while our proposed metrics serve as an effective complement.
• 摘要：在这项工作中，我们揭示了视觉引物和VAE的局限性在保留细粒度的功能中，并提出了一个基准，以评估两个具有挑战性的视觉内容：文本和面部的重建性能。图像令牌化具有显着高级的视觉生成和多模式建模，尤其是由于离散令牌的简单性而引起的自回归模型。自回归模型通常依靠图像令牌来将图像压缩为离散令牌以进行顺序预测，而扩散模型通常在连续的延伸空间上运行以降低计算成本。但是，两种视觉压缩方法都不可避免地会失去视觉信息，从而限制了视觉生成质量的上限。为了评估这些压缩损失如何影响文本和面部，是最敏感的视觉元素，我们首先收集和策划了文本的集合，并面对现有数据集的图像，从而确保清晰度和多样性。对于文本重建，我们采用OCR模型来评估重建文本的识别精度，然后测量原始面孔和重建面之间的特征相似性，从而量化了面孔的重建保真度。我们的方法非常轻巧，仅需要2GB的内存和4分钟才能完成评估。通过我们的基准，我们分析了不同图像令牌和VAE的各种尺度的文本的重建质量和面孔的重建质量。我们的结果表明，现代视觉令牌器仍然难以保留细粒度的特征，尤其是在较小的尺度下。此外，我们将此评估框架扩展到视频，对视频令牌进行了全面分析。此外，我们发现传统指标无法准确反映面部和文本的重建性能，而我们建议的指标则是有效的补充。

Title: Generative Distribution Embeddings

• Authors: Nic Fishman, Gokul Gowri, Peng Yin, Jonathan Gootenberg, Omar Abudayyeh
• Subjects: cs.LG, q-bio.QM, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.18150
• Pdf URL: https://arxiv.org/pdf/2505.18150
• Copy Paste: [[2505.18150]] Generative Distribution Embeddings(https://arxiv.org/abs/2505.18150)
• Keywords: generative
• Abstract: Many real-world problems require reasoning across multiple scales, demanding models which operate not on single data points, but on entire distributions. We introduce generative distribution embeddings (GDE), a framework that lifts autoencoders to the space of distributions. In GDEs, an encoder acts on sets of samples, and the decoder is replaced by a generator which aims to match the input distribution. This framework enables learning representations of distributions by coupling conditional generative models with encoder networks which satisfy a criterion we call distributional invariance. We show that GDEs learn predictive sufficient statistics embedded in the Wasserstein space, such that latent GDE distances approximately recover the $W_2$ distance, and latent interpolation approximately recovers optimal transport trajectories for Gaussian and Gaussian mixture distributions. We systematically benchmark GDEs against existing approaches on synthetic datasets, demonstrating consistently stronger performance. We then apply GDEs to six key problems in computational biology: learning representations of cell populations from lineage-tracing data (150K cells), predicting perturbation effects on single-cell transcriptomes (1M cells), predicting perturbation effects on cellular phenotypes (20M single-cell images), modeling tissue-specific DNA methylation patterns (253M sequences), designing synthetic yeast promoters (34M sequences), and spatiotemporal modeling of viral protein sequences (1M sequences).
• 摘要：许多现实世界中的问题需要跨多个量表进行推理，要求在单个数据点上而是在整个分布上运行的模型。我们引入了生成分配嵌入（GDE），该框架将自动编码器提升到分布空间。在GDES中，编码器作用于样品集，并且解码器被旨在匹配输入分布的发电机代替。该框架通过将条件生成模型与编码网络耦合，可以满足我们称为分布不变性的标准，从而实现了分布的学习表示。我们表明，GDES学习了嵌入在Wasserstein空间中的足够的足够统计数据，因此潜在的GDE距离大约恢复了$ W_2 $距离，而潜在的插值大致恢复了高斯和高斯混合物分布的最佳运输轨迹。我们从系统地基准了GDES，可以针对合成数据集上的现有方法，表明性能始终如一。 We then apply GDEs to six key problems in computational biology: learning representations of cell populations from lineage-tracing data (150K cells), predicting perturbation effects on single-cell tr​​anscriptomes (1M cells), predicting perturbation effects on cellular phenotypes (20M single-cell images), modeling tissue-specific DNA methylation patterns (253M sequences), designing synthetic yeast promoters (34M序列）和病毒蛋白序列的时空建模（1M序列）。

Title: REN: Fast and Efficient Region Encodings from Patch-Based Image Encoders

• Authors: Savya Khosla, Sethuraman TV, Barnett Lee, Alexander Schwing, Derek Hoiem
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.18153
• Pdf URL: https://arxiv.org/pdf/2505.18153
• Copy Paste: [[2505.18153]] REN: Fast and Efficient Region Encodings from Patch-Based Image Encoders(https://arxiv.org/abs/2505.18153)
• Keywords: generation
• Abstract: We introduce the Region Encoder Network (REN), a fast and effective model for generating region-based image representations using point prompts. Recent methods combine class-agnostic segmenters (e.g., SAM) with patch-based image encoders (e.g., DINO) to produce compact and effective region representations, but they suffer from high computational cost due to the segmentation step. REN bypasses this bottleneck using a lightweight module that directly generates region tokens, enabling 60x faster token generation with 35x less memory, while also improving token quality. It uses a few cross-attention blocks that take point prompts as queries and features from a patch-based image encoder as keys and values to produce region tokens that correspond to the prompted objects. We train REN with three popular encoders-DINO, DINOv2, and OpenCLIP-and show that it can be extended to other encoders without dedicated training. We evaluate REN on semantic segmentation and retrieval tasks, where it consistently outperforms the original encoders in both performance and compactness, and matches or exceeds SAM-based region methods while being significantly faster. Notably, REN achieves state-of-the-art results on the challenging Ego4D VQ2D benchmark and outperforms proprietary LMMs on Visual Haystacks' single-needle challenge. Code and models are available at: this https URL.
• 摘要：我们介绍了区域编码网络（REN），这是一种使用点提示来生成基于区域的图像表示的快速有效模型。最近的方法将类不足的细分器（例如SAM）与基于贴片的图像编码器（例如Dino）结合在一起，以产生紧凑而有效的区域表示，但由于分段步骤，它们的计算成本很高。 Ren使用直接生成区域令牌的轻量级模块绕过这一瓶颈，从而使60倍的代币产生能减少35倍，同时也提高了令牌质量。它使用一些跨注意区块，这些块将点提示作为查询和基于补丁的图像编码器的特征作为键和值，以产生与提示对象相对应的区域令牌。我们使用三个受欢迎的编码器，Dinov2和OpenClip培训REN，并表明它可以将其扩展到其他编码者而无需专门的培训。我们在语义细分和检索任务上评估了REN，在该任务中，它始终优于性能和紧凑性的原始编码器，并且匹配或超过基于SAM的区域方法，同时又要快得多。值得注意的是，Ren在具有挑战性的EGO4D VQ2D基准中取得了最先进的结果，并且在Visual Haystacks的单针挑战上优于专有LMM。代码和模型可在以下网址提供：此HTTPS URL。