2026-03-20

Title: Enhancing Reinforcement Learning Fine-Tuning with an Online Refiner

• Authors: Hao Ma, Zhiqiang Pu, Yang Liu, Xiaolin Ai
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18088
• Pdf URL: https://arxiv.org/pdf/2603.18088
• Copy Paste: [[2603.18088]] Enhancing Reinforcement Learning Fine-Tuning with an Online Refiner(https://arxiv.org/abs/2603.18088)
• Keywords: generation
• Abstract: Constraints are essential for stabilizing reinforcement learning fine-tuning (RFT) and preventing degenerate outputs, yet they inherently conflict with the optimization objective because stronger constraints limit the ability of a fine-tuned model to discover better solutions. We propose \textit{dynamic constraints} that resolve this tension by adapting to the evolving capabilities of the fine-tuned model based on the insight that constraints should only intervene when degenerate outputs occur. We implement this by using a reference model as an \textit{online refiner} that takes the response from the fine-tuned model and generates a minimally corrected version which preserves correct content verbatim while fixing errors. A supervised fine-tuning loss then trains the fine-tuned model to produce the refined output. This mechanism yields a constraint that automatically strengthens or relaxes based on output quality. Experiments on dialogue and code generation show that dynamic constraints outperform both KL regularization and unconstrained baselines, achieving substantially higher task rewards while maintaining training stability.
• 摘要：约束对于稳定强化学习微调（RFT）和防止退化输出至关重要，但它们本质上与优化目标相冲突，因为更强的约束限制了微调模型发现更好解决方案的能力。我们提出 \textit{动态约束}，通过适应微调模型的不断发展的能力来解决这种紧张局势，该模型基于这样的见解：约束只应在发生退化输出时进行干预。我们通过使用参考模型作为 \textit{在线精炼器} 来实现这一点，该模型从微调模型中获取响应并生成最小校正版本，该版本在修复错误的同时逐字保留正确的内容。然后，有监督的微调损失训练微调模型以产生精细的输出。该机制产生一个根据输出质量自动加强或放松的约束。对话和代码生成实验表明，动态约束优于 KL 正则化和无约束基线，在保持训练稳定性的同时实现了更高的任务奖励。

Title: CytoSyn: a Foundation Diffusion Model for Histopathology -- Tech Report

• Authors: Thomas Duboudin, Xavier Fontaine, Etienne Andrier, Lionel Guillou, Alexandre Filiot, Thalyssa Baiocco-Rodrigues, Antoine Olivier, Alberto Romagnoni, John Klein, Jean-Baptiste Schiratti
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18089
• Pdf URL: https://arxiv.org/pdf/2603.18089
• Copy Paste: [[2603.18089]] CytoSyn: a Foundation Diffusion Model for Histopathology -- Tech Report(https://arxiv.org/abs/2603.18089)
• Keywords: generation, generative
• Abstract: Computational pathology has made significant progress in recent years, fueling advances in both fundamental disease understanding and clinically ready tools. This evolution is driven by the availability of large amounts of digitized slides and specialized deep learning methods and models. Multiple self-supervised foundation feature extractors have been developed, enabling downstream predictive applications from cell segmentation to tumor sub-typing and survival analysis. In contrast, generative foundation models designed specifically for histopathology remain scarce. Such models could address tasks that are beyond the capabilities of feature extractors, such as virtual staining. In this paper, we introduce CytoSyn, a state-of-the-art foundation latent diffusion model that enables the guided generation of highly realistic and diverse histopathology H&E-stained images, as shown in an extensive benchmark. We explored methodological improvements, training set scaling, sampling strategies and slide-level overfitting, culminating in the improved CytoSyn-v2, and compared our work to PixCell, a state-of-the-art model, in an in-depth manner. This comparison highlighted the strong sensitivity of both diffusion models and performance metrics to preprocessing-specific details such as JPEG compression. Our model has been trained on a dataset obtained from more than 10,000 TCGA diagnostic whole-slide images of 32 different cancer types. Despite being trained only on oncology slides, it maintains state-of-the-art performance generating inflammatory bowel disease images. To support the research community, we publicly release CytoSyn's weights, its training and validation datasets, and a sample of synthetic images in this repository: this https URL.
• 摘要：近年来，计算病理学取得了重大进展，推动了基础疾病理解和临床工具的进步。这种演变是由大量数字化幻灯片以及专门的深度学习方法和模型的可用性推动的。多个自监督基础特征提取器已经开发出来，支持从细胞分割到肿瘤亚型和生存分析的下游预测应用。相比之下，专门为组织病理学设计的生成基础模型仍然很少。这些模型可以解决超出特征提取器能力的任务，例如虚拟染色。在本文中，我们介绍了 CytoSyn，这是一种最先进的基础潜在扩散模型，能够引导生成高度真实且多样化的组织病理学 H&E 染色图像，如广泛的基准测试所示。我们探索了方法改进、训练集缩放、采样策略和幻灯片级过度拟合，最终改进了 CytoSyn-v2，并将我们的工作与最先进的模型 PixCell 进行了深入的比较。这一比较凸显了扩散模型和性能指标对预处理特定细节（例如 JPEG 压缩）的强烈敏感性。我们的模型已经在从 32 种不同癌症类型的 10,000 多张 TCGA 诊断全幻灯片图像中获得的数据集上进行了训练。尽管仅接受了肿瘤学幻灯片的培训，但它仍保持了生成炎症性肠病图像的最先进的性能。为了支持研究社区，我们在此存储库中公开发布 CytoSyn 的权重、训练和验证数据集以及合成图像示例：此 https URL。

Title: Action Draft and Verify: A Self-Verifying Framework for Vision-Language-Action Model

• Authors: Chen Zhao, Zhuoran Wang, Haoyang Li, Shifeng Bao, Guanlin Li, Youhe Feng, Yang Li, Jie Tang, Jing Zhang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2603.18091
• Pdf URL: https://arxiv.org/pdf/2603.18091
• Copy Paste: [[2603.18091]] Action Draft and Verify: A Self-Verifying Framework for Vision-Language-Action Model(https://arxiv.org/abs/2603.18091)
• Keywords: generation
• Abstract: Vision-Language-Action (VLA) models have recently demonstrated strong performance across embodied tasks. Modern VLAs commonly employ diffusion action experts to efficiently generate high-precision continuous action chunks, while auto-regressive generation can be slower and less accurate at low-level control. Yet auto-regressive paradigms still provide complementary priors that can improve robustness and generalization in out-of-distribution environments. To leverage both paradigms, we propose Action-Draft-and-Verify (ADV): diffusion action expert drafts multiple candidate action chunks, and the VLM selects one by scoring all candidates in a single forward pass with a perplexity-style metric. Under matched backbones, training data, and action-chunk length, ADV improves success rate by +4.3 points in simulation and +19.7 points in real-world over diffusion-based baseline, with a single-pass VLM reranking overhead.
• 摘要：视觉-语言-动作（VLA）模型最近在具体任务中表现出了强大的性能。现代 VLA 通常采用扩散动作专家来有效生成高精度连续动作块，而自回归生成在低级控制下可能会较慢且不太准确。然而，自回归范式仍然提供了补充先验，可以提高分布外环境中的鲁棒性和泛化性。为了利用这两种范式，我们提出了行动起草和验证（ADV）：扩散行动专家起草多个候选行动块，VLM 通过使用困惑类型指标在一次前向传递中对所有候选者进行评分来选择一个行动块。在匹配主干网、训练数据和动作块长度的情况下，ADV 在模拟中将成功率提高了 4.3 点，在现实世界中比基于扩散的基线提高了 19.7 点，并且具有单通道 VLM 重新排序开销。

Title: One-to-More: High-Fidelity Training-Free Anomaly Generation with Attention Control

• Authors: Haoxiang Rao, Zhao Wang, Chenyang Si, Yan Lyu, Yuanyi Duan, Fang Zhao, Caifeng Shan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18093
• Pdf URL: https://arxiv.org/pdf/2603.18093
• Copy Paste: [[2603.18093]] One-to-More: High-Fidelity Training-Free Anomaly Generation with Attention Control(https://arxiv.org/abs/2603.18093)
• Keywords: generation
• Abstract: Industrial anomaly detection (AD) is characterized by an abundance of normal images but a scarcity of anomalous ones. Although numerous few-shot anomaly synthesis methods have been proposed to augment anomalous data for downstream AD tasks, most existing approaches require time-consuming training and struggle to learn distributions that are faithful to real anomalies, thereby restricting the efficacy of AD models trained on such data. To address these limitations, we propose a training-free few-shot anomaly generation method, namely O2MAG, which leverages the self-attention in One reference anomalous image to synthesize More realistic anomalies, supporting effective downstream anomaly detection. Specifically, O2MAG manipulates three parallel diffusion processes via self-attention grafting and incorporates the anomaly mask to mitigate foreground-background query confusion, synthesizing text-guided anomalies that closely adhere to real anomalous distributions. To bridge the semantic gap between the encoded anomaly text prompts and the true anomaly semantics, Anomaly-Guided Optimization is further introduced to align the synthesis process with the target anomalous distribution, steering the generation toward realistic and text-consistent anomalies. Moreover, to mitigate faint anomaly synthesis inside anomaly masks, Dual-Attention Enhancement is adopted during generation to reinforce both self- and cross-attention on masked regions. Extensive experiments validate the effectiveness of O2MAG, demonstrating its superior performance over prior state-of-the-art methods on downstream AD tasks.
• 摘要：工业异常检测（AD）的特点是正常图像丰富，异常图像稀缺。尽管已经提出了许多小样本异常合成方法来增强下游 AD 任务的异常数据，但大多数现有方法需要耗时的训练，并且很难学习忠实于真实异常的分布，从而限制了在此类数据上训练的 AD 模型的有效性。为了解决这些限制，我们提出了一种免训练的少样本异常生成方法，即 O2MAG，它利用一个参考异常图像中的自注意力来合成更真实的异常，支持有效的下游异常检测。具体来说，O2MAG 通过自注意力嫁接操纵三个并行扩散过程，并结合异常掩模来减轻前景-背景查询混淆，合成紧密遵循真实异常分布的文本引导异常。为了弥合编码的异常文本提示和真实的异常语义之间的语义差距，进一步引入了异常引导优化，以使合成过程与目标异常分布保持一致，从而引导生成真实且文本一致的异常。此外，为了减轻异常掩模内的微弱异常合成，在生成过程中采用双重注意力增强来增强掩模区域的自注意力和交叉注意力。大量实验验证了 O2MAG 的有效性，证明其在下游 AD 任务上比先前最先进的方法具有优越的性能。

Title: Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling

• Authors: Sooyoung Ryu, Mathieu Salzmann, Saqib Javed
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18095
• Pdf URL: https://arxiv.org/pdf/2603.18095
• Copy Paste: [[2603.18095]] Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling(https://arxiv.org/abs/2603.18095)
• Keywords: generation
• Abstract: Post-training quantization (PTQ) is a practical path to deploy large diffusion models, but quantization noise can accumulate over the denoising trajectory and degrade generation quality. We propose Q-Drift, a principled sampler-side correction that treats quantization error as an implicit stochastic perturbation on each denoising step and derives a marginal-distribution-preserving drift adjustment. Q-Drift estimates a timestep-wise variance statistic from calibration, in practice requiring as few as 5 paired full-precision/quantized calibration runs. The resulting sampler correction is plug-and-play with common samplers, diffusion models, and PTQ methods, while incurring negligible overhead at inference. Across six diverse text-to-image models (spanning DiT and U-Net), three samplers (Euler, flow-matching, DPM-Solver++), and two PTQ methods (SVDQuant, MixDQ), Q-Drift improves FID over the corresponding quantized baseline in most settings, with up to 4.59 FID reduction on PixArt-Sigma (SVDQuant W3A4), while preserving CLIP scores.
• 摘要：训练后量化（PTQ）是部署大型扩散模型的实用途径，但量化噪声可能会在去噪轨迹上累积并降低生成质量。我们提出了 Q-Drift，这是一种有原则的采样器端校正，它将量化误差视为每个去噪步骤的隐式随机扰动，并得出保留边际分布的漂移调整。 Q-Drift 估计校准的时间步方差统计量，实际上只需要 5 配对的全精度/量化校准运行。由此产生的采样器校正可与常见采样器、扩散模型和 PTQ 方法即插即用，同时在推理时产生的开销可以忽略不计。在六种不同的文本到图像模型（涵盖 DiT 和 U-Net）、三种采样器（Euler、flow-matching、DPM-Solver++）和两种 PTQ 方法（SVDQuant、MixDQ）中，Q-Drift 在大多数设置中提高了相应量化基线上的 FID，在 PixArt-Sigma (SVDQuant W3A4) 上 FID 降低高达 4.59，同时保留 CLIP 分数。

Title: BoundAD: Boundary-Aware Negative Generation for Time Series Anomaly Detection

• Authors: Xiancheng Wang, Lin Wang, Zhibo Zhang, Rui Wang, Minghang Zhao
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2603.18111
• Pdf URL: https://arxiv.org/pdf/2603.18111
• Copy Paste: [[2603.18111]] BoundAD: Boundary-Aware Negative Generation for Time Series Anomaly Detection(https://arxiv.org/abs/2603.18111)
• Keywords: generation
• Abstract: Contrastive learning methods for time series anomaly detection (TSAD) heavily depend on the quality of negative sample construction. However, existing strategies based on random perturbations or pseudo-anomaly injection often struggle to simultaneously preserve temporal semantic consistency and provide effective decision-boundary supervision. Most existing methods rely on prior anomaly injection, while overlooking the potential of generating hard negatives near the data manifold boundary directly from normal samples themselves. To address this issue, we propose a reconstruction-driven boundary negative generation framework that automatically constructs hard negatives through the reconstruction process of normal samples. Specifically, the method first employs a reconstruction network to capture normal temporal patterns, and then introduces a reinforcement learning strategy to adaptively adjust the optimization update magnitude according to the current reconstruction state. In this way, boundary-shifted samples close to the normal data manifold can be induced along the reconstruction trajectory and further used for subsequent contrastive representation learning. Unlike existing methods that depend on explicit anomaly injection, the proposed framework does not require predefined anomaly patterns, but instead mines more challenging boundary negatives from the model's own learning dynamics. Experimental results show that the proposed method effectively improves anomaly representation learning and achieves competitive detection performance on the current dataset.
• 摘要：时间序列异常检测（TSAD）的对比学习方法在很大程度上取决于负样本构建的质量。然而，基于随机扰动或伪异常注入的现有策略通常难以同时保持时间语义一致性并提供有效的决策边界监督。大多数现有方法依赖于先前的异常注入，同时忽略了直接从正常样本本身在数据流形边界附近生成硬负例的可能性。为了解决这个问题，我们提出了一种重建驱动的边界负片生成框架，该框架通过正常样本的重建过程自动构造硬负片。具体来说，该方法首先采用重建网络来捕获正常时间模式，然后引入强化学习策略来根据当前重建状态自适应地调整优化更新幅度。这样，可以沿着重建轨迹导出接近正常数据流形的边界移位样本，并进一步用于后续的对比表示学习。与依赖于显式异常注入的现有方法不同，所提出的框架不需要预定义的异常模式，而是从模型自身的学习动态中挖掘更具挑战性的边界负数。实验结果表明，该方法有效改进了异常表示学习，并在当前数据集上实现了有竞争力的检测性能。

Title: VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models

• Authors: Hefei Xu, Le Wu, Yu Wang, Min Hou, Han Wu, Zhen Zhang, Meng Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18113
• Pdf URL: https://arxiv.org/pdf/2603.18113
• Copy Paste: [[2603.18113]] VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models(https://arxiv.org/abs/2603.18113)
• Keywords: generation
• Abstract: As large language models (LLMs) increasingly shape content generation, interaction, and decision-making across the Web, aligning them with human values has become a central objective in trustworthy AI. This challenge becomes even more pronounced when aligning multiple, potentially conflicting human values. Although recent approaches, such as reward reweighting, prompt-based supervised fine-tuning, and model merging, attempt to tackle multi-value alignment, they still face two major limitations: (1) training separate models for each value combination is prohibitively expensive; (2) value conflicts substantially degrade alignment performance. These limitations make it difficult to achieve favorable trade-offs across diverse human values. To address these challenges, we revisit multi-value alignment from the perspective of value consistency in data and propose VC-soup, a data filtering and parameter merging framework grounded in value-consistent learning. We first design a value consistency metric based on the cosine similarity between the reward-gap vector of each preference pair and an all-ones vector, which quantifies its cross-value coherence. We then filter out low-consistency preference pairs in each value dataset and train on the remaining data to obtain smooth, value-consistent policy models that better preserve linear mode connectivity. Finally, we linearly combine these policies and apply Pareto filtering across values to obtain solutions with balanced multi-value performance. Extensive experiments and theoretical analysis demonstrate that VC-soup effectively mitigates conflicts and consistently outperforms existing multi-value alignment methods.
• 摘要：随着大型语言模型 (LLM) 越来越多地影响整个网络的内容生成、交互和决策，使它们与人类价值观保持一致已成为值得信赖的人工智能的核心目标。当协调多个可能相互冲突的人类价值观时，这一挑战变得更加明显。尽管最近的方法，例如奖励重新加权、基于提示的监督微调和模型合并，试图解决多值对齐问题，但它们仍然面临两个主要限制：（1）为每个值组合训练单独的模型成本高昂； (2) 值冲突会显着降低对齐性能。这些限制使得在不同的人类价值观之间实现有利的权衡变得困难。为了应对这些挑战，我们从数据价值一致性的角度重新审视多值对齐，并提出了 VC-soup，一种基于价值一致性学习的数据过滤和参数合并框架。我们首先根据每个偏好对的奖励差距向量与全一向量之间的余弦相似性设计一个价值一致性度量，该度量量化其跨值一致性。然后，我们过滤掉每个值数据集中的低一致性偏好对，并对剩余数据进行训练，以获得平滑、值一致的策略模型，从而更好地保留线性模式连接性。最后，我们线性组合这些策略，并在值之间应用帕累托过滤，以获得具有平衡的多值性能的解决方案。大量的实验和理论分析表明，VC-soup 有效地缓解了冲突，并且始终优于现有的多值对齐方法。

Title: LLM-Augmented Computational Phenotyping of Long Covid

• Authors: Jing Wang, Jie Shen, Amar Sra, Qiaomin Xie, Jeremy C Weiss
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18115
• Pdf URL: https://arxiv.org/pdf/2603.18115
• Copy Paste: [[2603.18115]] LLM-Augmented Computational Phenotyping of Long Covid(https://arxiv.org/abs/2603.18115)
• Keywords: generation
• Abstract: Phenotypic characterization is essential for understanding heterogeneity in chronic diseases and for guiding personalized interventions. Long COVID, a complex and persistent condition, yet its clinical subphenotypes remain poorly understood. In this work, we propose an LLM-augmented computational phenotyping framework ``Grace Cycle'' that iteratively integrates hypothesis generation, evidence extraction, and feature refinement to discover clinically meaningful subgroups from longitudinal patient data. The framework identifies three distinct clinical phenotypes, Protected, Responder, and Refractory, based on 13,511 Long Covid participants. These phenotypes exhibit pronounced separation in peak symptom severity, baseline disease burden, and longitudinal dose-response patterns, with strong statistical support across multiple independent dimensions. This study illustrates how large language models can be integrated into a principled, statistically grounded pipeline for phenotypic screening from complex longitudinal data. Note that the proposed framework is disease-agnostic and offers a general approach for discovering clinically interpretable subphenotypes.
• 摘要：表型特征对于理解慢性疾病的异质性和指导个性化干预至关重要。长效新冠病毒是一种复杂且持续的疾病，但其临床亚表型仍知之甚少。在这项工作中，我们提出了一个法学硕士增强的计算表型框架“Grace Cycle”，它迭代地整合了假设生成、证据提取和特征细化，以从纵向患者数据中发现有临床意义的亚组。该框架基于 13,511 名 Long Covid 参与者，确定了三种不同的临床表型：受保护型、应答型和难治型。这些表型在峰值症状严重程度、基线疾病负担和纵向剂量反应模式方面表现出明显的分离，并在多个独立维度上提供了强有力的统计支持。这项研究说明了如何将大型语言模型集成到有原则的、基于统计的管道中，以便从复杂的纵向数据中进行表型筛选。请注意，所提出的框架与疾病无关，并提供了发现临床可解释的亚表型的通用方法。

Title: Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models

• Authors: Yuhao Dong, Zuyan Liu, Shulin Tian, Yongming Rao, Ziwei Liu
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18118
• Pdf URL: https://arxiv.org/pdf/2603.18118
• Copy Paste: [[2603.18118]] Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models(https://arxiv.org/abs/2603.18118)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have achieved remarkable reliability and advanced capabilities through extended test-time reasoning. However, extending these capabilities to Multi-modal Large Language Models (MLLMs) remains a significant challenge due to a critical scarcity of high-quality, long-chain reasoning data and optimized training pipelines. To bridge this gap, we present a unified multi-agent visual reasoning framework that systematically evolves from our foundational image-centric model, Insight-V, into a generalized spatial-temporal architecture, Insight-V++. We first propose a scalable data generation pipeline equipped with multi-granularity assessment that autonomously synthesizes structured, complex reasoning trajectories across image and video domains without human intervention. Recognizing that directly supervising MLLMs with such intricate data yields sub-optimal results, we design a dual-agent architecture comprising a reasoning agent to execute extensive analytical chains, and a summary agent to critically evaluate and distill final outcomes. While our initial framework utilized Direct Preference Optimization (DPO), its off-policy nature fundamentally constrained reinforcement learning potential. To overcome these limitations, particularly for long-horizon video understanding, Insight-V++ introduces two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness. Crucially, by leveraging reliable feedback from the summary agent, we guide an iterative reasoning path generation process, retraining the entire multi-agent system in a continuous, self-improving loop. Extensive experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks while preserving strong capabilities on traditional perception-focused tasks.
• 摘要：大型语言模型 (LLM) 通过延长测试时间推理实现了卓越的可靠性和先进的功能。然而，由于高质量、长链推理数据和优化的训练管道的严重缺乏，将这些功能扩展到多模态大型语言模型（MLLM）仍然是一个重大挑战。为了弥补这一差距，我们提出了一个统一的多智能体视觉推理框架，该框架系统地从我们以图像为中心的基础模型 Insight-V 演变为广义的时空架构 Insight-V++。我们首先提出了一种配备多粒度评估的可扩展数据生成管道，可以在无需人工干预的情况下自动合成跨图像和视频领域的结构化、复杂的推理轨迹。认识到用如此复杂的数据直接监督 MLLM 会产生次优结果，我们设计了一种双代理架构，其中包括一个用于执行广泛分析链的推理代理和一个用于严格评估和提取最终结果的摘要代理。虽然我们最初的框架利用了直接偏好优化（DPO），但其离策略性质从根本上限制了强化学习的潜力。为了克服这些限制，特别是对于长视距视频理解，Insight-V++ 引入了两种新颖的算法：ST-GRPO 和 J-GRPO，它们增强了时空推理并提高了评估的鲁棒性。至关重要的是，通过利用摘要代理的可靠反馈，我们指导迭代推理路径生成过程，在连续的自我改进循环中重新训练整个多代理系统。在 LLaVA-NeXT 和 Qwen2.5-VL 等基础模型上进行的大量实验表明，在具有挑战性的图像和视频推理基准测试中，性能显着提升，同时保留了传统的以感知为中心的任务的强大功能。

Title: MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models

• Authors: Philippe Formont, Maxime Darrin, Ismail Ben Ayed, Pablo Piantanida
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18256
• Pdf URL: https://arxiv.org/pdf/2603.18256
• Copy Paste: [[2603.18256]] MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models(https://arxiv.org/abs/2603.18256)
• Keywords: generation
• Abstract: Recent advances in reasoning-based large language models (LLMs) have demonstrated substantial improvements in complex problem-solving tasks. Motivated by these advances, several works have explored the application of reasoning LLMs to drug discovery and molecular design. However, most existing approaches either focus on evaluation or rely on training setups that require ground-truth labels, such as molecule pairs with known property modifications. Such supervision is unavailable in \textit{de novo} molecular generation, where the objective is to generate novel molecules that optimize a desirability score without prior knowledge of high-scoring candidates. To bridge this gap, we introduce MolRGen, a large-scale benchmark and dataset for training and evaluating reasoning-based LLMs on \textit{de novo} molecular generation. Our contributions are threefold. First, we propose a setting to evaluate and train models for \textit{de novo} molecular generation and property prediction. Second, we introduce a novel diversity-aware top-$k$ score that captures both the quality and diversity of generated molecules. Third, we show our setting can be used to train LLMs for molecular generation, training a 24B LLM with reinforcement learning, and we provide a detailed analysis of its performance and limitations.
• 摘要：基于推理的大型语言模型 (LLM) 的最新进展已经证明了复杂问题解决任务的显着改进。受这些进步的推动，一些著作探索了推理法学硕士在药物发现和分子设计中的应用。然而，大多数现有方法要么侧重于评估，要么依赖于需要真实标签的训练设置，例如具有已知属性修饰的分子对。这种监督在 \textit{de novo} 分子生成中是不可用的，其中的目标是生成新颖的分子，在不事先了解高分候选者的情况下优化合意性分数。为了弥补这一差距，我们引入了 MolRGen，这是一个大型基准和数据集，用于训练和评估基于 \textit{de novo} 分子生成的推理的 LLM。我们的贡献是三重的。首先，我们提出了一种评估和训练 \textit{de novo} 分子生成和属性预测模型的设置。其次，我们引入了一种新颖的多样性感知 top-$k$ 分数，它可以捕获生成分子的质量和多样性。第三，我们展示了我们的设置可用于训练分子生成的 LLM，通过强化学习训练 24B LLM，并且我们提供了对其性能和局限性的详细分析。

Title: Enactor: From Traffic Simulators to Surrogate World Models

• Authors: Yash Ranjan, Rahul Sengupta, Anand Rangarajan, Sanjay Ranka
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18266
• Pdf URL: https://arxiv.org/pdf/2603.18266
• Copy Paste: [[2603.18266]] Enactor: From Traffic Simulators to Surrogate World Models(https://arxiv.org/abs/2603.18266)
• Keywords: generative
• Abstract: Traffic microsimulators are widely used to evaluate road network performance under various ``what-if" conditions. However, the behavior models controlling the actions of the actors are overly simplistic and fails to capture realistic actor-actor interactions. Deep learning-based methods have been applied to model vehicles and pedestrians as ``agents" responding to their surrounding ``environment" (including lanes, signals, and neighboring agents). Although effective in learning actor-actor interaction, these approaches fail to generate physically consistent trajectories over long time periods, and they do not explicitly address the complex dynamics that arise at traffic intersections which is a critical location in urban networks. Inspired by the World Model paradigm, we have developed an actor centric generative model using transformer-based architecture that is able to capture the actor-actor interaction, at the same time understanding the geometry to the traffic intersection to generate physically grounded trajectories that are based on learned behavior. Moreover, we test the model in a live ``simulation-in-the-loop" setting, where we generate the initial conditions of the actors using SUMO and then let the model control the dynamics of the actors. We let the simulation run for 40000 timesteps (4000 seconds), testing the performance of the model on long timerange and evaluating the trajectories on traffic engineering related metrics. Experimental results demonstrate that the proposed framework effectively captures complex actor-actor interactions and generates long-horizon, physically consistent trajectories, while requiring significantly fewer training samples than traditional agent-centric generative approaches. Our model is able to outperform the baseline in traffic related as well as aggregate metrics where our model beats the baseline by more than 10x on the KL-Divergence.
• 摘要：交通微模拟器被广泛用于评估各种“假设”条件下的道路网络性能。然而，控制参与者行为的行为模型过于简单化，无法捕捉真实的参与者之间的交互。基于深度学习的方法已被应用于将车辆和行人建模为响应周围“环境”（包括车道、信号和邻近代理）的“代理”。虽然在学习参与者之间的交互方面有效，但这些方法无法长时间生成物理上一致的轨迹受世界模型范例的启发，我们使用基于变压器的架构开发了一种以参与者为中心的生成模型，该模型能够捕获参与者之间的交互，同时了解交通交叉口的几何形状，以生成基于学习行为的物理接地轨迹。此外，我们在实时“循环模拟”设置中测试模型，在其中生成初始条件。使用 SUMO 对演员进行建模，然后让模型控制演员的动态。我们让模拟运行 40000 个时间步长（4000 秒），在长时间范围内测试模型的性能并评估流量工程相关指标的轨迹。实验结果表明，所提出的框架有效地捕获了复杂的参与者之间的交互，并生成长范围的、物理一致的轨迹，同时比传统的以代理为中心的生成方法需要更少的训练样本。我们的模型能够在流量相关以及聚合指标方面超越基线，其中我们的模型在 KL 散度上比基线高出 10 倍以上。

Title: On Additive Gaussian Processes for Wind Farm Power Prediction

• Authors: Simon M. Brealy, Lawrence A. Bull, Daniel S. Brennan, Pauline Beltrando, Anders Sommer, Nikolaos Dervilis, Keith Worden
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18281
• Pdf URL: https://arxiv.org/pdf/2603.18281
• Copy Paste: [[2603.18281]] On Additive Gaussian Processes for Wind Farm Power Prediction(https://arxiv.org/abs/2603.18281)
• Keywords: generation
• Abstract: Population-based Structural Health Monitoring (PBSHM) aims to share information between similar machines or structures. This paper takes a population-level perspective, exploring the use of additive Gaussian processes to reveal variations in turbine-specific and farm-level power models over a collected wind farm dataset. The predictions illustrate patterns in wind farm power generation, which follow intuition and should enable more informed control and decision-making.
• 摘要：基于群体的结构健康监测（PBSHM）旨在在相似的机器或结构之间共享信息。本文从人口层面的角度出发，探索使用加性高斯过程来揭示收集的风电场数据集上涡轮机特定和农场级功率模型的变化。这些预测说明了风电场发电的模式，这些模式遵循直觉，应该能够实现更明智的控制和决策。

Title: Unrolled Reconstruction with Integrated Super-Resolution for Accelerated 3D LGE MRI

• Authors: Md Hasibul Husain Hisham, Shireen Elhabian, Ganesh Adluru, Jason Mendes, Andrew Arai, Eugene Kholmovski, Ravi Ranjan, Edward DiBella
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18309
• Pdf URL: https://arxiv.org/pdf/2603.18309
• Copy Paste: [[2603.18309]] Unrolled Reconstruction with Integrated Super-Resolution for Accelerated 3D LGE MRI(https://arxiv.org/abs/2603.18309)
• Keywords: super-resolution
• Abstract: Accelerated 3D late gadolinium enhancement (LGE) MRI requires robust reconstruction methods to recover thin atrial structures from undersampled k-space data. While unrolled model-based networks effectively integrate physics-driven data consistency with learned priors, they operate at the acquired resolution and may fail to fully recover high-frequency detail. We propose a hybrid unrolled reconstruction framework in which an Enhanced Deep Super-Resolution (EDSR) network replaces the proximal operator within each iteration of the optimization loop, enabling joint super-resolution enhancement and data consistency enforcement. The model is trained end-to-end on retrospectively undersampled preclinical 3D LGE datasets and compared against compressed sensing, Model-Based Deep Learning (MoDL), and self-guided Deep Image Prior (DIP) baselines. Across acceleration factors, the proposed method consistently improves PSNR and SSIM over standard unrolled reconstruction and better preserves fine cardiac structures, leading to improved LA (left atrium) segmentation performance. These results demonstrate that integrating super-resolution priors directly within model-based reconstruction provides measurable gains in accelerated 3D LGE MRI.
• 摘要：加速 3D 晚期钆增强 (LGE) MRI 需要强大的重建方法来从欠采样的 k 空间数据中恢复薄心房结构。虽然展开的基于模型的网络有效地将物理驱动的数据一致性与学习的先验集成在一起，但它们以获取的分辨率运行，可能无法完全恢复高频细节。我们提出了一种混合展开重建框架，其中增强型深度超分辨率（EDSR）网络取代了优化循环每次迭代中的近端算子，从而实现了联合超分辨率增强和数据一致性执行。该模型在回顾性欠采样临床前 3D LGE 数据集上进行端到端训练，并与压缩感知、基于模型的深度学习 (MoDL) 和自引导深度图像先验 (DIP) 基线进行比较。在加速因素中，所提出的方法比标准展开重建一致地提高了 PSNR 和 SSIM，并更好地保留了精细的心脏结构，从而提高了 LA（左心房）分割性能。这些结果表明，将超分辨率先验直接集成到基于模型的重建中可以为加速 3D LGE MRI 提供可测量的增益。

Title: VISTA: Validation-Guided Integration of Spatial and Temporal Foundation Models with Anatomical Decoding for Rare-Pathology VCE Event Detection

• Authors: Bo-Cheng Qiu, Yu-Fan Lin, Yu-Zhe Pien, Chia-Ming Lee, Fu-En Yang, Yu-Chiang Frank Wang, Chih-Chung Hsu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18343
• Pdf URL: https://arxiv.org/pdf/2603.18343
• Copy Paste: [[2603.18343]] VISTA: Validation-Guided Integration of Spatial and Temporal Foundation Models with Anatomical Decoding for Rare-Pathology VCE Event Detection(https://arxiv.org/abs/2603.18343)
• Keywords: generation
• Abstract: Capsule endoscopy event detection is challenging because diagnostically relevant findings are sparse, visually heterogeneous, and embedded in long, noisy video streams, while evaluation is performed at the event level rather than by frame accuracy alone. We therefore formulate the RARE-VISION task as a metric-aligned event detection problem instead of a purely frame-wise classification task. Our framework combines two complementary backbones, EndoFM-LV for local temporal context and DINOv3 ViT-L/16 for strong frame-level visual semantics, followed by a Diverse Head Ensemble, Validation-Guided Hierarchical Fusion, and Anatomy-Aware Temporal Event Decoding. The fusion stage uses validation-derived class-wise model weighting, backbone weighting, and probability calibration, while the decoding stage applies temporal smoothing, anatomical constraints, threshold refinement, and per-label event generation to produce stable event predictions. Validation ablations indicate that complementary backbones, validation-guided fusion, and anatomy-aware temporal decoding all contribute to event-level performance. On the official hidden test set, the proposed method achieved an overall temporal mAP@0.5 of 0.3530 and temporal mAP@0.95 of 0.3235.
• 摘要：胶囊内窥镜事件检测具有挑战性，因为诊断相关的发现稀疏、视觉上异质，并且嵌入在长而嘈杂的视频流中，而评估是在事件级别而不是仅根据帧精度进行的。因此，我们将 RARE-VISION 任务制定为度量对齐的事件检测问题，而不是纯粹的逐帧分类任务。我们的框架结合了两个互补的骨干网，用于局部时间上下文的 EndoFM-LV 和用于强大的帧级视觉语义的 DINOv3 ViT-L/16，其次是多样化的头部集成、验证引导的层次融合和解剖感知时间事件解码。融合阶段使用验证派生的分类模型加权、骨干加权和概率校准，而解码阶段应用时间平滑、解剖约束、阈值细化和每个标签事件生成来产生稳定的事件预测。验证消融表明互补主干、验证引导融合和解剖感知时间解码都有助于事件级性能。在官方隐藏测试集上，该方法的总体时间 mAP@0.5 为 0.3530，时间 mAP@0.95 为 0.3235。

Title: Epistemic Generative Adversarial Networks

• Authors: Muhammad Mubashar, Fabio Cuzzolin
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18348
• Pdf URL: https://arxiv.org/pdf/2603.18348
• Copy Paste: [[2603.18348]] Epistemic Generative Adversarial Networks(https://arxiv.org/abs/2603.18348)
• Keywords: generation, generative
• Abstract: Generative models, particularly Generative Adversarial Networks (GANs), often suffer from a lack of output diversity, frequently generating similar samples rather than a wide range of variations. This paper introduces a novel generalization of the GAN loss function based on Dempster-Shafer theory of evidence, applied to both the generator and discriminator. Additionally, we propose an architectural enhancement to the generator that enables it to predict a mass function for each image pixel. This modification allows the model to quantify uncertainty in its outputs and leverage this uncertainty to produce more diverse and representative generations. Experimental evidence shows that our approach not only improves generation variability but also provides a principled framework for modeling and interpreting uncertainty in generative processes.
• 摘要：生成模型，特别是生成对抗网络（GAN），通常缺乏输出多样性，经常生成相似的样本，而不是广泛的变化。本文介绍了一种基于 Dempster-Shafer 证据理论的 GAN 损失函数的新颖推广，应用于生成器和鉴别器。此外，我们提出了对生成器的架构增强，使其能够预测每个图像像素的质量函数。这种修改允许模型量化其输出的不确定性，并利用这种不确定性来产生更加多样化和代表性的世代。实验证据表明，我们的方法不仅改善了生成变异性，而且还为建模和解释生成过程中的不确定性提供了原则框架。

Title: Mathematical Foundations of Deep Learning

• Authors: Xiaojing Ye
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2603.18387
• Pdf URL: https://arxiv.org/pdf/2603.18387
• Copy Paste: [[2603.18387]] Mathematical Foundations of Deep Learning(https://arxiv.org/abs/2603.18387)
• Keywords: generative
• Abstract: This draft book offers a comprehensive and rigorous treatment of the mathematical principles underlying modern deep learning. The book spans core theoretical topics, from the approximation capabilities of deep neural networks, the theory and algorithms of optimal control and reinforcement learning integrated with deep learning techniques, to contemporary generative models that drive today's advances in artificial intelligence.
• 摘要：这本书草稿对现代深度学习背后的数学原理进行了全面而严格的处理。本书涵盖了核心理论主题，从深度神经网络的逼近能力、与深度学习技术相结合的最优控制和强化学习的理论和算法，到推动当今人工智能进步的当代生成模型。

Title: FlowMS: Flow Matching for De Novo Structure Elucidation from Mass Spectra

• Authors: Jianan Nie, Peng Gao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18397
• Pdf URL: https://arxiv.org/pdf/2603.18397
• Copy Paste: [[2603.18397]] FlowMS: Flow Matching for De Novo Structure Elucidation from Mass Spectra(https://arxiv.org/abs/2603.18397)
• Keywords: generation
• Abstract: Mass spectrometry (MS) stands as a cornerstone analytical technique for molecular identification, yet de novo structure elucidation from spectra remains challenging due to the combinatorial complexity of chemical space and the inherent ambiguity of spectral fragmentation patterns. Recent deep learning approaches, including autoregressive sequence models, scaffold-based methods, and graph diffusion models, have made progress. However, diffusion-based generation for this task remains computationally demanding. Meanwhile, discrete flow matching, which has shown strong performance for graph generation, has not yet been explored for spectrum-conditioned structure elucidation. In this work, we introduce FlowMS, the first discrete flow matching framework for spectrum-conditioned de novo molecular generation. FlowMS generates molecular graphs through iterative refinement in probability space, enforcing chemical formula constraints while conditioning on spectral embeddings from a pretrained formula transformer encoder. Notably, it achieves state-of-the-art performance on 5 out of 6 metrics on the NPLIB1 benchmark: 9.15% top-1 accuracy (9.7% relative improvement over DiffMS) and 7.96 top-10 MCES (4.2% improvement over MS-BART). We also visualize the generated molecules, which further demonstrate that FlowMS produces structurally plausible candidates closely resembling ground truth structures. These results establish discrete flow matching as a promising paradigm for mass spectrometry-based structure elucidation in metabolomics and natural product discovery.
• 摘要：质谱 (MS) 是分子鉴定的基石分析技术，但由于化学空间的组合复杂性和光谱碎片模式固有的模糊性，从光谱中从头解析结构仍然具有挑战性。最近的深度学习方法，包括自回归序列模型、基于支架的方法和图扩散模型，已经取得了进展。然而，该任务的基于扩散的生成仍然需要大量计算。与此同时，离散流匹配在图生成方面表现出了强大的性能，但尚未被探索用于谱条件结构阐明。在这项工作中，我们介绍了 FlowMS，这是第一个用于光谱条件从头分子生成的离散流匹配框架。 FlowMS 通过概率空间中的迭代细化生成分子图，强制化学式约束，同时根据预训练公式转换器编码器的光谱嵌入进行调节。值得注意的是，它在 NPLIB1 基准的 6 个指标中的 5 个上实现了最先进的性能：9.15% top-1 准确度（比 DiffMS 相对提高 9.7%）和 7.96 top-10 MCES（比 MS-BART 提高 4.2%）。我们还可视化生成的分子，这进一步证明 FlowMS 产生的结构合理的候选分子与地面真实结构非常相似。这些结果将离散流匹配确立为代谢组学和天然产物发现中基于质谱的结构阐明的有前景的范例。

Title: R&D: Balancing Reliability and Diversity in Synthetic Data Augmentation for Semantic Segmentation

• Authors: Huy Che, Dinh-Duy Phan, Duc-Khai Lam
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18427
• Pdf URL: https://arxiv.org/pdf/2603.18427
• Copy Paste: [[2603.18427]] R&D: Balancing Reliability and Diversity in Synthetic Data Augmentation for Semantic Segmentation(https://arxiv.org/abs/2603.18427)
• Keywords: generative
• Abstract: Collecting and annotating datasets for pixel-level semantic segmentation tasks are highly labor-intensive. Data augmentation provides a viable solution by enhancing model generalization without additional real-world data collection. Traditional augmentation techniques, such as translation, scaling, and color transformations, create geometric variations but fail to generate new structures. While generative models have been employed to extend semantic information of datasets, they often struggle to maintain consistency between the original and generated images, particularly for pixel-level tasks. In this work, we propose a novel synthetic data augmentation pipeline that integrates controllable diffusion models. Our approach balances diversity and reliability data, effectively bridging the gap between synthetic and real data. We utilize class-aware prompting and visual prior blending to improve image quality further, ensuring precise alignment with segmentation labels. By evaluating benchmark datasets such as PASCAL VOC and BDD100K, we demonstrate that our method significantly enhances semantic segmentation performance, especially in data-scarce scenarios, while improving model robustness in real-world applications. Our code is available at \href{this https URL}{this https URL}.
• 摘要：为像素级语义分割任务收集和注释数据集是高度劳动密集型的。数据增强通过增强模型泛化而无需额外收集实际数据，从而提供了可行的解决方案。传统的增强技术，例如平移、缩放和颜色变换，会产生几何变化，但无法生成新的结构。虽然生成模型已被用来扩展数据集的语义信息，但它们通常很难保持原始图像和生成图像之间的一致性，特别是对于像素级任务。在这项工作中，我们提出了一种新颖的合成数据增强管道，它集成了可控扩散模型。我们的方法平衡了数据的多样性和可靠性，有效地弥合了合成数据和真实数据之间的差距。我们利用类别感知提示和视觉优先混合来进一步提高图像质量，确保与分割标签的精确对齐。通过评估 PASCAL VOC 和 BDD100K 等基准数据集，我们证明我们的方法显着增强了语义分割性能，特别是在数据稀缺的场景中，同时提高了模型在实际应用中的鲁棒性。我们的代码位于 \href{此 https URL}{此 https URL}。

Title: Learning Consistent Temporal Grounding between Related Tasks in Sports Coaching

• Authors: Arushi Rai, Adriana Kovashka
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18453
• Pdf URL: https://arxiv.org/pdf/2603.18453
• Copy Paste: [[2603.18453]] Learning Consistent Temporal Grounding between Related Tasks in Sports Coaching(https://arxiv.org/abs/2603.18453)
• Keywords: generation
• Abstract: Video-LLMs often attend to irrelevant frames, which is especially detrimental for sports coaching tasks requiring precise temporal grounding. Yet obtaining frame-level supervision is challenging: expensive to collect from humans and unreliable from other models. We improve temporal grounding without additional annotations by exploiting the observation that related tasks, such as generation and verification, must attend to the same frames. We enforce this via a self-consistency objective over select visual attention maps of tightly-related tasks. Using VidDiffBench, which provides ground-truth keyframe annotations, we first validate that attention misallocation is a significant bottleneck. We then show that training with our objective yields gains of +3.0%, +14.1% accuracy and +0.9 BERTScore over supervised finetuning across three sports coaching tasks: Exact, FitnessQA, and ExpertAF, even surpassing closed-source models.
• 摘要：视频法学硕士经常关注不相关的帧，这对于需要精确时间基础的体育教练任务尤其不利。然而，获得帧级监督具有挑战性：从人类那里收集数据成本高昂，并且从其他模型中收集数据并不可靠。我们通过利用相关任务（例如生成和验证）必须关注相同帧的观察来改进时间基础，而无需额外注释。我们通过对紧密相关任务的选定视觉注意力图的自我一致性目标来强制执行这一点。使用提供真实关键帧注释的 VidDiffBench，我们首先验证注意力分配不当是一个重要的瓶颈。然后，我们表明，与 Exact、FitnessQA 和 ExpertAF 三个运动教练任务的监督微调相比，按照我们的目标进行训练的效果提高了 +3.0%、+14.1% 准确率和 +0.9 BERTScore，甚至超过了闭源模型。

Title: MedQ-UNI: Toward Unified Medical Image Quality Assessment and Restoration via Vision-Language Modeling

• Authors: Jiyao Liu, Junzhi Ning, Wanying Qu, Lihao Liu, Chenglong Ma, Junjun He, Ningsheng Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18465
• Pdf URL: https://arxiv.org/pdf/2603.18465
• Copy Paste: [[2603.18465]] MedQ-UNI: Toward Unified Medical Image Quality Assessment and Restoration via Vision-Language Modeling(https://arxiv.org/abs/2603.18465)
• Keywords: restoration, quality assessment
• Abstract: Existing medical image restoration (Med-IR) methods are typically modality-specific or degradation-specific, failing to generalize across the heterogeneous degradations encountered in clinical practice. We argue this limitation stems from the isolation of Med-IR from medical image quality assessment (Med-IQA), as restoration models without explicit quality understanding struggle to adapt to diverse degradation types across modalities. To address these challenges, we propose MedQ-UNI, a unified vision-language model that follows an assess-then-restore paradigm, explicitly leveraging Med-IQA to guide Med-IR across arbitrary modalities and degradation types. MedQ-UNI adopts a multimodal autoregressive dual-expert architecture with shared attention: a quality assessment expert first identifies degradation issues through structured natural language descriptions, and a restoration expert then conditions on these descriptions to perform targeted image restoration. To support this paradigm, we construct a large-scale dataset of approximately 50K paired samples spanning three imaging modalities and five restoration tasks, each annotated with structured quality descriptions for joint Med-IQA and Med-IR training, along with a 2K-sample benchmark for evaluation. Extensive experiments demonstrate that a single MedQ-UNI model, without any task-specific adaptation, achieves state-of-the-art restoration performance across all tasks while generating superior descriptions, confirming that explicit quality understanding meaningfully improves restoration fidelity and interpretability.
• 摘要：现有的医学图像恢复（Med-IR）方法通常是特定于模态或特定于退化的，无法概括临床实践中遇到的异质退化。我们认为这种限制源于 Med-IR 与医学图像质量评估 (Med-IQA) 的隔离，因为没有明确质量理解的恢复模型很难适应跨模态的不同退化类型。为了应对这些挑战，我们提出了 MedQ-UNI，这是一种统一的视觉语言模型，遵循评估然后恢复范式，明确利用 Med-IQA 来指导 Med-IR 跨越任意模式和降解类型。 MedQ-UNI 采用共享注意力的多模态自回归双专家架构：质量评估专家首先通过结构化自然语言描述识别退化问题，然后恢复专家根据这些描述进行有针对性的图像恢复。为了支持这种范例，我们构建了一个包含大约 50K 配对样本的大型数据集，涵盖三种成像模式和五个恢复任务，每个数据集都注释有用于联合 Med-IQA 和 Med-IR 训练的结构化质量描述，以及用于评估的 2K 样本基准。大量实验表明，单个 MedQ-UNI 模型在没有任何特定于任务的适应的情况下，可以在所有任务中实现最先进的恢复性能，同时生成卓越的描述，证实明确的质量理解有意义地提高恢复保真度和可解释性。

Title: Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion

• Authors: Yuqi Yang, Dongliang Chang, Yijia Ling, Ruoyi Du, Zhanyu Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18466
• Pdf URL: https://arxiv.org/pdf/2603.18466
• Copy Paste: [[2603.18466]] Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion(https://arxiv.org/abs/2603.18466)
• Keywords: generation
• Abstract: Colour is one of the most perceptually salient yet least controllable attributes in image generation. Although recent diffusion models can modify object colours from user instructions, their results often deviate from the intended hue, especially for fine-grained and local edits. Early text-driven methods rely on discrete language descriptions that cannot accurately represent continuous chromatic variations. To overcome this limitation, we propose ColourCrafter, a unified diffusion framework that transforms colour editing from global tone transfer into a structured, region-aware generation process. Unlike traditional colour driven methods, ColourCrafter performs token-level fusion of RGB colour tokens and image tokens in latent space, selectively propagating colour information to semantically relevant regions while preserving structural fidelity. A perceptual Lab-space Loss further enhances pixel-level precision by decoupling luminance and chrominance and constraining edits within masked areas. Additionally, we build ColourfulSet, a largescale dataset of high-quality image pairs with continuous and diverse colour variations. Extensive experiments demonstrate that ColourCrafter achieves state-of-the-art colour accuracy, controllability and perceptual fidelity in fine-grained colour editing. Our project is available at this https URL.
• 摘要：颜色是图像生成中感知上最显着但最不可控的属性之一。尽管最近的扩散模型可以根据用户指令修改对象颜色，但它们的结果通常会偏离预期的色调，特别是对于细粒度和局部编辑。早期的文本驱动方法依赖于离散的语言描述，无法准确地表示连续的色彩变化。为了克服这一限制，我们提出了 ColourCrafter，这是一个统一的扩散框架，它将颜色编辑从全局色调传输转变为结构化的、区域感知的生成过程。与传统的颜色驱动方法不同，ColourCrafter 在潜在空间中执行 RGB 颜色标记和图像标记的标记级融合，选择性地将颜色信息传播到语义相关区域，同时保持结构保真度。感知实验室空间损失通过解耦亮度和色度以及限制遮罩区域内的编辑来进一步提高像素级精度。此外，我们还构建了 ColourfulSet，这是一个具有连续且多样化颜色变化的高质量图像对的大规模数据集。大量实验表明，ColourCrafter 在细粒度颜色编辑中实现了最先进的颜色准确性、可控性和感知保真度。我们的项目可以通过此 https URL 获取。

Title: Efficient Video Diffusion with Sparse Information Transmission for Video Compression

• Authors: Mingde Zhou, Zheng Chen, Yulun Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18501
• Pdf URL: https://arxiv.org/pdf/2603.18501
• Copy Paste: [[2603.18501]] Efficient Video Diffusion with Sparse Information Transmission for Video Compression(https://arxiv.org/abs/2603.18501)
• Keywords: generative
• Abstract: Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at this https URL.
• 摘要：视频压缩旨在以最小的比特率最大限度地提高重建质量。除了标准失真指标之外，感知质量和时间一致性也至关重要。然而，在超低比特率下，传统的端到端压缩模型往往会产生感知质量较差的模糊图像。此外，现有的生成压缩方法通常独立处理视频帧，并且在时间一致性和效率方面表现出局限性。为了应对这些挑战，我们提出了具有稀疏信息传输的高效视频扩散（Diff-SIT），其中包括稀疏时间编码模块（STEM）和具有帧类型嵌入器的一步视频扩散（ODFTE）。 STEM 将原始帧序列稀疏编码为信息丰富的中间序列，从而实现显着的比特率节省。随后，ODFTE 将这个中间序列作为一个整体进行处理，从而利用时间相关性。在此过程中，我们提出的帧类型嵌入器（FTE）引导扩散模型根据不同的帧类型进行自适应重建，以优化整体质量。对多个数据集的大量实验表明，Diff-SIT 在感知质量和时间一致性方面建立了新的最先进技术，特别是在具有挑战性的超低比特率情况下。代码在此 https URL 发布。

Title: From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactions

• Authors: Jingzhi Chen, Lijian Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18505
• Pdf URL: https://arxiv.org/pdf/2603.18505
• Copy Paste: [[2603.18505]] From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactions(https://arxiv.org/abs/2603.18505)
• Keywords: generative
• Abstract: The protein folding problem has been fundamentally transformed by artificial intelligence, evolving from static structure prediction toward the modeling of dynamic conformational ensembles and complex biomolecular interactions. This review systematically examines the paradigm shift in AI driven protein science across five interconnected dimensions: unified multimodal representations that integrate sequences, geometries, and textual knowledge; refinement of static prediction through MSA free architectures and all atom complex modeling; generative frameworks, including diffusion models and flow matching, that capture conformational distributions consistent with thermodynamic ensembles; prediction of heterogeneous interactions spanning protein ligand, protein nucleic acid, and protein protein complexes; and functional inference of fitness landscapes, mutational effects, and text guided property prediction. We critically analyze current bottlenecks, including data distribution biases, limited mechanistic interpretability, and the disconnect between geometric metrics and biophysical reality, while identifying future directions toward physically consistent generative models, multimodal foundation architectures, and experimental closed loop systems. This methodological transformation marks artificial intelligence's transition from a structural analysis tool into a universal simulator capable of understanding and ultimately rewriting the dynamic language of life.
• 摘要：人工智能从根本上改变了蛋白质折叠问题，从静态结构预测发展到动态构象集合和复杂生物分子相互作用的建模。这篇综述系统地研究了人工智能驱动的蛋白质科学在五个相互关联的维度上的范式转变：整合序列、几何和文本知识的统一多模态表示；通过 MSA 免费架构和全原子复杂建模改进静态预测；生成框架，包括扩散模型和流动匹配，捕获与热力学系综一致的构象分布；预测蛋白质配体、蛋白质核酸和蛋白质复合物的异质相互作用；以及适应度景观、突变效应和文本引导属性预测的功能推理。我们批判性地分析当前的瓶颈，包括数据分布偏差、有限的机械解释性以及几何度量与生物物理现实之间的脱节，同时确定物理一致的生成模型、多模态基础架构和实验闭环系统的未来方向。这种方法论的转变标志着人工智能从结构分析工具转变为能够理解并最终重写生命动态语言的通用模拟器。

Title: CAFlow: Adaptive-Depth Single-Step Flow Matching for Efficient Histopathology Super-Resolution

• Authors: Elad Yoshai, Ariel D. Yoshai, Natan T. Shaked
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18513
• Pdf URL: https://arxiv.org/pdf/2603.18513
• Copy Paste: [[2603.18513]] CAFlow: Adaptive-Depth Single-Step Flow Matching for Efficient Histopathology Super-Resolution(https://arxiv.org/abs/2603.18513)
• Keywords: super-resolution, generative
• Abstract: In digital pathology, whole-slide images routinely exceed gigapixel resolution, making computationally intensive generative super-resolution (SR) impractical for routine deployment. We introduce CAFlow, an adaptive-depth single-step flow-matching framework that routes each image tile to the shallowest network exit that preserves reconstruction quality. CAFlow performs flow matching in pixel-unshuffled rearranged space, reducing spatial computation by 16x while enabling direct inference. We show that dedicating half of training to exact t=0 samples is essential for single-step quality (-1.5 dB without it). The backbone, FlowResNet (1.90M parameters), mixes convolution and window self-attention blocks across four early exits spanning 3.1 to 13.3 GFLOPs. A lightweight exit classifier (~6K parameters) achieves 33% compute savings at only 0.12 dB cost. On multi-organ histopathology x4 SR, adaptive routing achieves 31.72 dB PSNR versus 31.84 dB at full depth, while the shallowest exit exceeds bicubic by +1.9 dB at 2.8x less compute than SwinIR-light. The method generalizes to held-out colon tissue with minimal quality loss (-0.02 dB), and at x8 upscaling it outperforms all comparable-compute baselines while remaining competitive with the much larger SwinIR-Medium model. Downstream nuclei segmentation confirms preservation of clinically relevant structure. The model trains in under 5 hours on a single GPU, and adaptive routing can reduce whole-slide inference from minutes to seconds.
• 摘要：在数字病理学中，整个幻灯片图像的分辨率通常超过十亿像素，这使得计算密集型生成超分辨率（SR）对于常规部署来说不切实际。我们引入了 CAFlow，这是一种自适应深度单步流匹配框架，它将每个图像块路由到最浅的网络出口，以保持重建质量。 CAFlow 在像素未洗牌的重新排列空间中执行流匹配，将空间计算减少 16 倍，同时实现直接推理。我们表明，将一半的训练时间用于精确的 t=0 样本对于单步质量至关重要（没有它则为 -1.5 dB）。主干 FlowResNet（1.90M 参数）在跨越 3.1 至 13.3 GFLOP 的四个早期出口中混合了卷积和窗口自注意力块。轻量级出口分类器（约 6K 个参数）仅以 0.12 dB 成本即可节省 33% 的计算量。在多器官组织病理学 x4 SR 上，自适应路由实现了 31.72 dB PSNR，而全深度为 31.84 dB，而最浅出口超过双三次，计算量比 SwinIR-light 少 2.8 倍。该方法以最小的质量损失 (-0.02 dB) 推广到保留的结肠组织，并且在 x8 升级时，它优于所有可比计算基线，同时与更大的 SwinIR-Medium 模型保持竞争。下游细胞核分割证实了临床相关结构的保存。该模型在单个 GPU 上的训练时间不到 5 小时，自适应路由可以将整个幻灯片推理从几分钟缩短到几秒钟。

Title: 3DreamBooth: High-Fidelity 3D Subject-Driven Video Generation Model

• Authors: Hyun-kyu Ko, Jihyeon Park, Younghyun Kim, Dongheok Park, Eunbyung Park
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18524
• Pdf URL: https://arxiv.org/pdf/2603.18524
• Copy Paste: [[2603.18524]] 3DreamBooth: High-Fidelity 3D Subject-Driven Video Generation Model(https://arxiv.org/abs/2603.18524)
• Keywords: generation
• Abstract: Creating dynamic, view-consistent videos of customized subjects is highly sought after for a wide range of emerging applications, including immersive VR/AR, virtual production, and next-generation e-commerce. However, despite rapid progress in subject-driven video generation, existing methods predominantly treat subjects as 2D entities, focusing on transferring identity through single-view visual features or textual prompts. Because real-world subjects are inherently 3D, applying these 2D-centric approaches to 3D object customization reveals a fundamental limitation: they lack the comprehensive spatial priors necessary to reconstruct the 3D geometry. Consequently, when synthesizing novel views, they must rely on generating plausible but arbitrary details for unseen regions, rather than preserving the true 3D identity. Achieving genuine 3D-aware customization remains challenging due to the scarcity of multi-view video datasets. While one might attempt to fine-tune models on limited video sequences, this often leads to temporal overfitting. To resolve these issues, we introduce a novel framework for 3D-aware video customization, comprising 3DreamBooth and 3Dapter. 3DreamBooth decouples spatial geometry from temporal motion through a 1-frame optimization paradigm. By restricting updates to spatial representations, it effectively bakes a robust 3D prior into the model without the need for exhaustive video-based training. To enhance fine-grained textures and accelerate convergence, we incorporate 3Dapter, a visual conditioning module. Following single-view pre-training, 3Dapter undergoes multi-view joint optimization with the main generation branch via an asymmetrical conditioning strategy. This design allows the module to act as a dynamic selective router, querying view-specific geometric hints from a minimal reference set. Project page: this https URL
• 摘要：创建定制主题的动态、视图一致的视频受到各种新兴应用的高度追捧，包括沉浸式 VR/AR、虚拟制作和下一代电子商务。然而，尽管主题驱动的视频生成取得了快速进展，但现有方法主要将主题视为 2D 实体，侧重于通过单视图视觉特征或文本提示来传递身份。由于现实世界的主题本质上是 3D 的，因此将这些以 2D 为中心的方法应用于 3D 对象定制揭示了一个基本限制：它们缺乏重建 3D 几何所需的全面空间先验。因此，在合成新颖的视图时，他们必须依赖于为看不见的区域生成看似合理但任意的细节，而不是保留真实的 3D 身份。由于多视图视频数据集的稀缺，实现真正的 3D 感知定制仍然具有挑战性。虽然人们可能会尝试在有限的视频序列上微调模型，但这通常会导致时间过度拟合。为了解决这些问题，我们引入了一种新颖的 3D 感知视频定制框架，包括 3DreamBooth 和 3Dapter。 3DreamBooth 通过 1 帧优化范例将空间几何与时间运动解耦。通过限制对空间表示的更新，它可以有效地将强大的 3D 先验烘焙到模型中，而无需进行详尽的基于视频的训练。为了增强细粒度纹理并加速收敛，我们采用了视觉调节模块 3Dapter。在单视图预训练之后，3Dapter 通过不对称调节策略与主生成分支进行多视图联合优化。这种设计允许模块充当动态选择性路由器，从最小参考集中查询特定于视图的几何提示。项目页面：此 https URL

Title: Data-efficient pre-training by scaling synthetic megadocs

• Authors: Konwoo Kim, Suhas Kotha, Yejin Choi, Tatsunori Hashimoto, Nick Haber, Percy Liang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18534
• Pdf URL: https://arxiv.org/pdf/2603.18534
• Copy Paste: [[2603.18534]] Data-efficient pre-training by scaling synthetic megadocs(https://arxiv.org/abs/2603.18534)
• Keywords: generation
• Abstract: Synthetic data augmentation has emerged as a promising solution when pre-training is constrained by data rather than compute. We study how to design synthetic data algorithms that achieve better loss scaling: not only lowering loss at finite compute but especially as compute approaches infinity. We first show that pre-training on web data mixed with synthetically generated rephrases improves i.i.d. validation loss on the web data, despite the synthetic data coming from an entirely different distribution. With optimal mixing and epoching, loss and benchmark accuracy improve without overfitting as the number of synthetic generations grows, plateauing near $1.48\times$ data efficiency at 32 rephrases per document. We find even better loss scaling under a new perspective: synthetic generations from the same document can form a single substantially longer megadocument instead of many short documents. We show two ways to construct megadocs: stitching synthetic rephrases from the same web document or stretching a document by inserting rationales. Both methods improve i.i.d. loss, downstream benchmarks, and especially long-context loss relative to simple rephrasing, increasing data efficiency from $1.48\times$ to $1.80\times$ at $32$ generations per document. Importantly, the improvement of megadocs over simple rephrasing widens as more synthetic data is generated. Our results show how to design synthetic data algorithms that benefit more from increasing compute when data-constrained.
• 摘要：当预训练受到数据而不是计算的限制时，合成数据增强已成为一种有前景的解决方案。我们研究如何设计实现更好的损失缩放的合成数据算法：不仅降低有限计算时的损失，而且特别是当计算接近无穷大时。我们首先表明，对网络数据进行预训练与综合生成的改写相结合可以提高 i.i.d。尽管合成数据来自完全不同的分布，但网络数据的验证丢失。通过最佳混合和纪元，随着合成代数量的增长，损失和基准准确性得到改善，而不会过度拟合，每个文档 32 次改写时数据效率稳定在接近 1.48 美元\倍的水平。我们在新的视角下发现了更好的损失缩放：同一文档的合成生成可以形成一个更长的巨型文档，而不是许多短文档。我们展示了两种构建巨型文档的方法：从同一网络文档中拼接合成改写或通过插入基本原理来扩展文档。两种方法都提高了 i.i.d.损失、下游基准，尤其是相对于简单改写的长上下文损失，将每个文档生成 $32$ 代的数据效率从 $1.48\times$ 提高到 $1.80\times$。重要的是，随着更多综合数据的产生，巨文档相对于简单改写的改进不断扩大。我们的结果展示了如何设计合成数据算法，以便在数据受限时从增加计算中获得更多好处。

Title: HEP Statistical Inference for UAV Fault Detection: CLs, LRT, and SBI Applied to Blade Damage

• Authors: Khushiyant
• Subjects: cs.LG, cs.RO, eess.SY
• Abstract URL: https://arxiv.org/abs/2603.18546
• Pdf URL: https://arxiv.org/pdf/2603.18546
• Copy Paste: [[2603.18546]] HEP Statistical Inference for UAV Fault Detection: CLs, LRT, and SBI Applied to Blade Damage(https://arxiv.org/abs/2603.18546)
• Keywords: generative
• Abstract: This paper transfers three statistical methods from particle physics to multirotor propeller fault detection: the likelihood ratio test (LRT) for binary detection, the CLs modified frequentist method for false alarm rate control, and sequential neural posterior estimation (SNPE) for quantitative fault characterization. Operating on spectral features tied to rotor harmonic physics, the system returns three outputs: binary detection, controlled false alarm rates, and calibrated posteriors over fault severity and motor location. On UAV-FD, a hexarotor dataset of 18 real flights with 5% and 10% blade damage, leave-one-flight-out cross-validation gives AUC 0.862 +/- 0.007 (95% CI: 0.849--0.876), outperforming CUSUM (0.708 +/- 0.010), autoencoder (0.753 +/- 0.009), and LSTM autoencoder (0.551). At 5% false alarm rate the system detects 93% of significant and 81% of subtle blade damage. On PADRE, a quadrotor platform, AUC reaches 0.986 after refitting only the generative models. SNPE gives a full posterior over fault severity (90% credible interval coverage 92--100%, MAE 0.012), so the output includes uncertainty rather than just a point estimate or fault flag. Per-flight sequential detection achieves 100% fault detection with 94% overall accuracy.
• 摘要：本文将三种统计方法从粒子物理学转移到多旋翼螺旋桨故障检测：用于二元检测的似然比检验（LRT）、用于误报率控制的 CLs 改进频率论方法以及用于定量故障表征的序贯神经后验估计（SNPE）。该系统根据与转子谐波物理相关的频谱特征进行操作，返回三个输出：二进制检测、受控误报率以及针对故障严重性和电机位置的校准后验。在 UAV-FD 上，18 次真实飞行的六旋翼数据集有 5% 和 10% 叶片损坏，留一飞行交叉验证给出 AUC 0.862 +/- 0.007 (95% CI: 0.849--0.876)，优于 CUSUM (0.708 +/- 0.010)、自动编码器 (0.753 +/-) 0.009）和 LSTM 自动编码器（0.551）。在误报率为 5% 的情况下，系统可检测到 93% 的重大叶片损坏和 81% 的细微叶片损坏。在四旋翼飞行器 PADRE 平台上，仅改装生成模型后，AUC 达到 0.986。 SNPE 给出故障严重性的完整后验（90% 可信区间覆盖范围 92--100%，MAE 0.012），因此输出包括不确定性而不仅仅是点估计或故障标记。每个航班的顺序检测实现了 100% 的故障检测，总体准确率达到 94%。

Title: Color image restoration based on nonlocal saturation-value similarity

• Authors: Wei Wang, Yakun Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18586
• Pdf URL: https://arxiv.org/pdf/2603.18586
• Copy Paste: [[2603.18586]] Color image restoration based on nonlocal saturation-value similarity(https://arxiv.org/abs/2603.18586)
• Keywords: restoration
• Abstract: In this paper, we propose and develop a novel nonlocal variational technique based on saturation-value similarity for color image restoration. In traditional nonlocal methods, image patches are extracted from red, green and blue channels of a color image directly, and the color information can not be described finely because the patch similarity is mainly based on the grayscale value of independent channel. The main aim of this paper is to propose and develop a novel nonlocal regularization method by considering the similarity of image patches in saturation-value channel of a color image. In particular, we first establish saturation-value similarity based nonlocal total variation by incorporating saturation-value similarity of color image patches into the proposed nonlocal gradients, which can describe the saturation and value similarity of two adjacent color image patches. The proposed nonlocal variational models are then formulated based on saturation-value similarity based nonlocal total variation. Moreover, we design an effective and efficient algorithm to solve the proposed optimization problem numerically by employing bregmanized operator splitting method, and we also study the convergence of the proposed algorithms. Numerical examples are presented to demonstrate that the performance of the proposed models is better than that of other testing methods in terms of visual quality and some quantitative metrics including peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), quaternion structural similarity index (QSSIM) and S-CIELAB color error.
• 摘要：在本文中，我们提出并开发了一种基于饱和度值相似性的新型非局部变分技术，用于彩色图像恢复。传统的非局部方法直接从彩色图像的红、绿、蓝通道中提取图像块，由于块相似度主要基于独立通道的灰度值，无法精细描述颜色信息。本文的主要目的是通过考虑彩色图像饱和度值通道中图像块的相似性，提出并开发一种新颖的非局部正则化方法。特别是，我们首先通过将彩色图像块的饱和度值相似度纳入所提出的非局部梯度中，建立基于饱和度值相似度的非局部全变分，它可以描述两个相邻彩色图像块的饱和度和值相似度。然后，基于饱和值相似性的非局部总变分，制定了所提出的非局部变分模型。此外，我们设计了一种有效且高效的算法，通过采用bregmanized算子分裂方法来数值求解所提出的优化问题，并且我们还研究了所提出算法的收敛性。数值例子表明，所提出的模型在视觉质量和一些定量指标方面的性能优于其他测试方法，包括峰值信噪比（PSNR）、结构相似性指数（SSIM）、四元数结构相似性指数（QSSIM）和S-CIELAB颜色误差。

Title: AU Codes, Language, and Synthesis: Translating Anatomy to Text for Facial Behavior Synthesis

• Authors: Jiahe Wang, Cong Liang, Xuandong Huang, Yuxin Wang, Xin Yun, Yi Wu, Yanan Chang, Shangfei Wang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2603.18588
• Pdf URL: https://arxiv.org/pdf/2603.18588
• Copy Paste: [[2603.18588]] AU Codes, Language, and Synthesis: Translating Anatomy to Text for Facial Behavior Synthesis(https://arxiv.org/abs/2603.18588)
• Keywords: generative
• Abstract: Facial behavior synthesis remains a critical yet underexplored challenge. While text-to-face models have made progress, they often rely on coarse emotion categories, which lack the nuance needed to capture the full spectrum of human nonverbal communication. Action Units (AUs) provide a more precise and anatomically grounded alternative. However, current AU-based approaches typically encode AUs as one-hot vectors, modeling compound expressions as simple linear combinations of individual AUs. This linearity becomes problematic when handling conflicting AUs--defined as those which activate the same facial muscle with opposing actions. Such cases lead to anatomically implausible artifacts and unnatural motion superpositions. To address this, we propose a novel method that represents facial behavior through natural language descriptions of AUs. This approach preserves the expressiveness of the AU framework while enabling explicit modeling of complex and conflicting AUs. It also unlocks the potential of modern text-to-image models for high-fidelity facial synthesis. Supporting this direction, we introduce BP4D-AUText, the first large-scale text-image paired dataset for complex facial behavior. It is synthesized by applying a rule-based Dynamic AU Text Processor to the BP4D and BP4D+ datasets. We further propose VQ-AUFace, a generative model that leverages facial structural priors to synthesize realistic and diverse facial behaviors from text. Extensive quantitative experiments and user studies demonstrate that our approach significantly outperforms existing methods. It excels in generating facial expressions that are anatomically plausible, behaviorally rich, and perceptually convincing, particularly under challenging conditions involving conflicting AUs.
• 摘要：面部行为合成仍然是一个关键但尚未充分探索的挑战。虽然文本到面对面的模型取得了进展，但它们通常依赖于粗略的情感类别，而缺乏捕捉人类非语言交流的全部范围所需的细微差别。动作单元 (AU) 提供了更精确、更符合解剖学原理的替代方案。然而，当前基于 AU 的方法通常将 AU 编码为 one-hot 向量，将复合表达式建模为各个 AU 的简单线性组合。当处理冲突的 AU（定义为通过相反的动作激活相同的面部肌肉的 AU）时，这种线性就会出现问题。这种情况会导致解剖学上不可信的伪影和不自然的运动叠加。为了解决这个问题，我们提出了一种通过 AU 的自然语言描述来表示面部行为的新方法。这种方法保留了 AU 框架的表达能力，同时支持对复杂且冲突的 AU 进行显式建模。它还释放了现代文本到图像模型在高保真面部合成方面的潜力。为了支持这个方向，我们引入了 BP4D-AUText，这是第一个用于复杂面部行为的大规模文本图像配对数据集。它是通过将基于规则的动态 AU 文本处理器应用于 BP4D 和 BP4D+ 数据集来合成的。我们进一步提出了 VQ-AUFace，一种利用面部结构先验从文本中合成真实且多样化的面部行为的生成模型。广泛的定量实验和用户研究表明，我们的方法明显优于现有方法。它擅长生成解剖学上合理、行为丰富且感知上令人信服的面部表情，特别是在涉及 AU 冲突的挑战性条件下。

Title: SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation

• Authors: Jialiang Kang, Han Shu, Wenshuo Li, Yingjie Zhai, Xinghao Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18599
• Pdf URL: https://arxiv.org/pdf/2603.18599
• Copy Paste: [[2603.18599]] SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation(https://arxiv.org/abs/2603.18599)
• Keywords: generation
• Abstract: Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-entropy nature of visual generation yields low draft-token acceptance rates in complex regions, creating a bottleneck that severely limits overall throughput. To overcome this, we introduce SJD-PAC, an enhanced SJD framework. First, SJD-PAC employs a proactive drafting strategy to improve local acceptance rates in these challenging high-entropy regions. Second, we introduce an adaptive continuation mechanism that sustains sequence validation after an initial rejection, bypassing the need for full resampling. Working in tandem, these optimizations significantly increase the average acceptance length per step, boosting inference speed while strictly preserving the target distribution. Experiments on standard text-to-image benchmarks demonstrate that SJD-PAC achieves a $3.8\times$ speedup with lossless image quality.
• 摘要：推测雅可比解码 (SJD) 提供了一种无草稿模型的方法来加速自回归文本到图像的合成。然而，视觉生成的高熵性质导致复杂区域中的草案令牌接受率较低，从而形成严重限制整体吞吐量的瓶颈。为了克服这个问题，我们引入了 SJD-PAC，一个增强的 SJD 框架。首先，SJD-PAC 采用主动起草策略来提高这些具有挑战性的高熵区域的本地接受率。其次，我们引入了一种自适应延续机制，该机制可以在初始拒绝后维持序列验证，从而绕过完全重采样的需要。这些优化协同工作，显着增加了每步的平均接受长度，提高了推理速度，同时严格保留了目标分布。标准文本到图像基准的实验表明，SJD-PAC 在无损图像质量的情况下实现了 3.8 美元的加速。

Title: Improving Joint Audio-Video Generation with Cross-Modal Context Learning

• Authors: Bingqi Ma, Linlong Lang, Ming Zhang, Dailan He, Xingtong Ge, Yi Zhang, Guanglu Song, Yu Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18600
• Pdf URL: https://arxiv.org/pdf/2603.18600
• Copy Paste: [[2603.18600]] Improving Joint Audio-Video Generation with Cross-Modal Context Learning(https://arxiv.org/abs/2603.18600)
• Keywords: generation
• Abstract: The dual-stream transformer architecture-based joint audio-video generation method has become the dominant paradigm in current research. By incorporating pre-trained video diffusion models and audio diffusion models, along with a cross-modal interaction attention module, high-quality, temporally synchronized audio-video content can be generated with minimal training data. In this paper, we first revisit the dual-stream transformer paradigm and further analyze its limitations, including model manifold variations caused by the gating mechanism controlling cross-modal interactions, biases in multi-modal background regions introduced by cross-modal attention, and the inconsistencies in multi-modal classifier-free guidance (CFG) during training and inference, as well as conflicts between multiple conditions. To alleviate these issues, we propose Cross-Modal Context Learning (CCL), equipped with several carefully designed modules. Temporally Aligned RoPE and Partitioning (TARP) effectively enhances the temporal alignment between audio latent and video latent representations. The Learnable Context Tokens (LCT) and Dynamic Context Routing (DCR) in the Cross-Modal Context Attention (CCA) module provide stable unconditional anchors for cross-modal information, while dynamically routing based on different training tasks, further enhancing the model's convergence speed and generation quality. During inference, Unconditional Context Guidance (UCG) leverages the unconditional support provided by LCT to facilitate different forms of CFG, improving train-inference consistency and further alleviating conflicts. Through comprehensive evaluations, CCL achieves state-of-the-art performance compared with recent academic methods while requiring substantially fewer resources.
• 摘要：基于双流变压器架构的联合音视频生成方法已成为当前研究的主导范式。通过结合预先训练的视频扩散模型和音频扩散模型，以及跨模式交互注意模块，可以用最少的训练数据生成高质量、时间同步的音视频内容。在本文中，我们首先重新审视双流变压器范式并进一步分析其局限性，包括控制跨模态交互的门控机制引起的模型流形变化、跨模态注意力引入的多模态背景区域的偏差、训练和推理过程中多模态无分类器指导（CFG）的不一致以及多个条件之间的冲突。为了缓解这些问题，我们提出了跨模态上下文学习（CCL），配备了几个精心设计的模块。时间对齐 RoPE 和分区 (TARP) 有效增强了音频潜在表示和视频潜在表示之间的时间对齐。跨模态上下文注意力（CCA）模块中的可学习上下文令牌（LCT）和动态上下文路由（DCR）为跨模态信息提供稳定的无条件锚定，同时根据不同的训练任务进行动态路由，进一步提升模型的收敛速度和生成质量。在推理过程中，无条件上下文指导（UCG）利用LCT提供的无条件支持来促进不同形式的CFG，提高训练推理的一致性并进一步缓解冲突。通过综合评估，与最近的学术方法相比，CCL 实现了最先进的性能，同时需要的资源却少得多。

Title: OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data

• Authors: Bin Cao, Sipeng Zheng, Hao Luo, Boyuan Li, Jing Liu, Zongqing Lu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18623
• Pdf URL: https://arxiv.org/pdf/2603.18623
• Copy Paste: [[2603.18623]] OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data(https://arxiv.org/abs/2603.18623)
• Keywords: generation
• Abstract: Text-to-motion (T2M) generation aims to create realistic human movements from text descriptions, with promising applications in animation and robotics. Despite recent progress, current T2M models perform poorly on unseen text descriptions due to the small scale and limited diversity of existing motion datasets. To address this problem, we introduce OpenT2M, a million-level, high-quality, and open-source motion dataset containing over 2800 hours of human motion. Each sequence undergoes rigorous quality control through physical feasibility validation and multi-granularity filtering, with detailed second-wise text annotations. We also develop an automated pipeline for creating long-horizon sequences, enabling complex motion generation. Building upon OpenT2M, we introduce MonoFrill, a pretrained motion model that achieves compelling T2M results without complicated designs or technique tricks as "frills". Its core component is 2D-PRQ, a novel motion tokenizer that captures spatiotemporal dependencies by dividing the human body into biology parts. Experiments show that OpenT2M significantly improves generalization of existing T2M models, while 2D-PRQ achieves superior reconstruction and strong zero-shot performance. We expect OpenT2M and MonoFrill will advance the T2M field by addressing longstanding data quality and benchmarking challenges.
• 摘要：文本到运动（T2M）生成旨在根据文本描述创建逼真的人类动作，在动画和机器人技术中具有广阔的应用前景。尽管最近取得了进展，但由于现有运动数据集的规模小且多样性有限，当前的 T2M 模型在看不见的文本描述上表现不佳。为了解决这个问题，我们引入了 OpenT2M，这是一个百万级、高质量、开源的运动数据集，包含超过 2800 小时的人体运动。每个序列都经过物理可行性验证和多粒度过滤的严格质量控制，并带有详细的二级文本注释。我们还开发了一个自动化管道，用于创建长视野序列，从而实现复杂的运动生成。在 OpenT2M 的基础上，我们引入了 MonoFrill，这是一种预训练的运动模型，无需复杂的设计或“花哨”的技术技巧即可实现引人注目的 T2M 结果。其核心组件是 2D-PRQ，这是一种新颖的运动标记器，通过将人体划分为生物部分来捕获时空依赖性。实验表明，OpenT2M 显着提高了现有 T2M 模型的泛化能力，而 2D-PRQ 实现了卓越的重建和强大的零样本性能。我们预计 OpenT2M 和 MonoFrill 将通过解决长期存在的数据质量和基准测试挑战来推动 T2M 领域的发展。

Title: Training-Free Sparse Attention for Fast Video Generation via Offline Layer-Wise Sparsity Profiling and Online Bidirectional Co-Clustering

• Authors: Jiayi Luo, Jiayu Chen, Jiankun Wang, Cong Wang, Hanxin Zhu, Qingyun Sun, Chen Gao, Zhibo Chen, Jianxin Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18636
• Pdf URL: https://arxiv.org/pdf/2603.18636
• Copy Paste: [[2603.18636]] Training-Free Sparse Attention for Fast Video Generation via Offline Layer-Wise Sparsity Profiling and Online Bidirectional Co-Clustering(https://arxiv.org/abs/2603.18636)
• Keywords: generation
• Abstract: Diffusion Transformers (DiTs) achieve strong video generation quality but suffer from high inference cost due to dense 3D attention, leading to the development of sparse attention technologies to improve efficiency. However, existing training-free sparse attention methods in video generation still face two unresolved limitations: ignoring layer heterogeneity in attention pruning and ignoring query-key coupling in block partitioning, which hinder a better quality-speedup trade-off. In this work, we uncover a critical insight that the attention sparsity of each layer is its intrinsic property, with minor effects across different inputs. Motivated by this, we propose SVOO, a training-free Sparse attention framework for fast Video generation via Offline layer-wise sparsity profiling and Online bidirectional co-clustering. Specifically, SVOO adopts a two-stage paradigm: (i) offline layer-wise sensitivity profiling to derive intrinsic per-layer pruning levels, and (ii) online block-wise sparse attention via a novel bidirectional co-clustering algorithm. Extensive experiments on seven widely used video generation models demonstrate that SVOO achieves a superior quality-speedup trade-off over state-of-the-art methods, delivering up to $1.93\times$ speedup while maintaining a PSNR of up to 29 dB on Wan2.1.
• 摘要：扩散变压器 (DiT) 实现了强大的视频生成质量，但由于密集的 3D 注意力而导致推理成本较高，从而导致稀疏注意力技术的发展以提高效率。然而，视频生成中现有的免训练稀疏注意力方法仍然面临两个未解决的局限性：忽略注意力修剪中的层异质性和忽略块分区中的查询键耦合，这阻碍了更好的质量加速权衡。在这项工作中，我们发现了一个重要的见解，即每一层的注意力稀疏性是其内在属性，对不同输入的影响较小。受此启发，我们提出了 SVOO，这是一种免训练的稀疏注意力框架，用于通过离线分层稀疏分析和在线双向联合聚类来快速生成视频。具体来说，SVOO 采用两阶段范式：(i) 离线逐层敏感度分析以导出内在的每层剪枝级别，以及 (ii) 通过新颖的双向共聚类算法进行在线块式稀疏注意力。对七种广泛使用的视频生成模型进行的大量实验表明，SVOO 相对于最先进的方法实现了卓越的质量与加速权衡，在 Wan2.1 上保持高达 29 dB 的 PSNR 的同时，提供高达 1.93 美元\times$ 的加速。

Title: PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance

• Authors: Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Fei-Yue Wang, Zhibo Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18639
• Pdf URL: https://arxiv.org/pdf/2603.18639
• Copy Paste: [[2603.18639]] PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance(https://arxiv.org/abs/2603.18639)
• Keywords: generation
• Abstract: Recent progress in video generation has led to substantial improvements in visual fidelity, yet ensuring physically consistent motion remains a fundamental challenge. Intuitively, this limitation can be attributed to the fact that real-world object motion unfolds in three-dimensional space, while video observations provide only partial, view-dependent projections of such dynamics. To address these issues, we propose PhysVideo, a two-stage framework that first generates physics-aware orthogonal foreground videos and then synthesizes full videos with background. In the first stage, Phys4View leverages physics-aware attention to capture the influence of physical attributes on motion dynamics, and enhances spatio-temporal consistency by incorporating geometry-enhanced cross-view attention and temporal attention. In the second stage, VideoSyn uses the generated foreground videos as guidance and learns the interactions between foreground dynamics and background context for controllable video synthesis. To support training, we construct PhysMV, a dataset containing 40K scenes, each consisting of four orthogonal viewpoints, resulting in a total of 160K video sequences. Extensive experiments demonstrate that PhysVideo significantly improves physical realism and spatial-temporal coherence over existing video generation methods. Home page: this https URL.
• 摘要：视频生成领域的最新进展极大地提高了视觉保真度，但确保物理上一致的运动仍然是一个基本挑战。直观上，这种限制可归因于现实世界的物体运动在三维空间中展开，而视频观察仅提供这种动态的部分、依赖于视图的投影。为了解决这些问题，我们提出了 PhysVideo，这是一个两阶段框架，首先生成物理感知的正交前景视频，然后合成带有背景的完整视频。在第一阶段，Phys4View利用物理感知注意力来捕捉物理属性对运动动力学的影响，并通过结合几何增强的跨视图注意力和时间注意力来增强时空一致性。在第二阶段，VideoSyn使用生成的前景视频作为指导，学习前景动态和背景上下文之间的交互，以实现可控的视频合成。为了支持训练，我们构建了 PhysMV，一个包含 40K 场景的数据集，每个场景由四个正交视点组成，总共产生 160K 视频序列。大量实验表明，与现有视频生成方法相比，PhysVideo 显着提高了物理真实感和时空一致性。主页：此 https URL。

Title: MeInTime: Bridging Age Gap in Identity-Preserving Face Restoration

• Authors: Teer Song, Yue Zhang, Yu Tian, Ziyang Wang, Xianlin Zhang, Guixuan Zhang, Xuan Liu, Xueming Li, Yasen Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18645
• Pdf URL: https://arxiv.org/pdf/2603.18645
• Copy Paste: [[2603.18645]] MeInTime: Bridging Age Gap in Identity-Preserving Face Restoration(https://arxiv.org/abs/2603.18645)
• Keywords: restoration
• Abstract: To better preserve an individual's identity, face restoration has evolved from reference-free to reference-based approaches, which leverage high-quality reference images of the same identity to enhance identity fidelity in the restored outputs. However, most existing methods implicitly assume that the reference and degraded input are age-aligned, limiting their effectiveness in real-world scenarios where only cross-age references are available, such as historical photo restoration. This paper proposes MeInTime, a diffusion-based face restoration method that extends reference-based restoration from same-age to cross-age settings. Given one or few reference images along with an age prompt corresponding to the degraded input, MeInTime achieves faithful restoration with both identity fidelity and age consistency. Specifically, we decouple the modeling of identity and age conditions. During training, we focus solely on effectively injecting identity features through a newly introduced attention mechanism and introduce Gated Residual Fusion modules to facilitate the integration between degraded features and identity representations. At inference, we propose Age-Aware Gradient Guidance, a training-free sampling strategy, using an age-driven direction to iteratively nudge the identity-aware denoising latent toward the desired age semantic manifold. Extensive experiments demonstrate that MeInTime outperforms existing face restoration methods in both identity preservation and age consistency. Our code is available at: this https URL
• 摘要：为了更好地保留个人的身份，面部恢复已经从无参考方法发展到基于参考的方法，该方法利用相同身份的高质量参考图像来增强恢复输出中的身份保真度。然而，大多数现有方法隐含地假设参考和降级输入是与年龄一致的，这限制了它们在只有跨年龄参考可用的现实场景中的有效性，例如历史照片恢复。本文提出了MeInTime，一种基于扩散的人脸恢复方法，将基于参考的恢复从同龄设置扩展到跨年龄设置。给定一张或几张参考图像以及与降级输入相对应的年龄提示，MeInTime 即可实现身份保真度和年龄一致性的忠实恢复。具体来说，我们将身份和年龄条件的建模解耦。在训练过程中，我们只专注于通过新引入的注意力机制有效地注入身份特征，并引入门控残差融合模块以促进退化特征和身份表示之间的集成。在推理中，我们提出了年龄感知梯度指导，这是一种免训练的采样策略，使用年龄驱动的方向迭代地将身份感知去噪潜向所需的年龄语义流形。大量实验表明，MeInTime 在身份保存和年龄一致性方面均优于现有的人脸恢复方法。我们的代码位于：此 https URL

Title: Multimodal Model for Computational Pathology:Representation Learning and Image Compression

• Authors: Peihang Wu, Zehong Chen, Lijian Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18660
• Pdf URL: https://arxiv.org/pdf/2603.18660
• Copy Paste: [[2603.18660]] Multimodal Model for Computational Pathology:Representation Learning and Image Compression(https://arxiv.org/abs/2603.18660)
• Keywords: generation
• Abstract: Whole slide imaging (WSI) has transformed digital pathology by enabling computational analysis of gigapixel histopathology images. Recent foundation model advances have accelerated progress in computational pathology, facilitating joint reasoning across pathology images, clinical reports, and structured data. Despite this progress, challenges remain: the extreme resolution of WSIs creates computational hurdles for visual learning; limited expert annotations constrain supervised approaches; integrating multimodal information while preserving biological interpretability remains difficult; and the opacity of modeling ultra-long visual sequences hinders clinical transparency. This review comprehensively surveys recent advances in multimodal computational pathology. We systematically analyze four research directions: (1) self-supervised representation learning and structure-aware token compression for WSIs; (2) multimodal data generation and augmentation; (3) parameter-efficient adaptation and reasoning-enhanced few-shot learning; and (4) multi-agent collaborative reasoning for trustworthy diagnosis. We specifically examine how token compression enables cross-scale modeling and how multi-agent mechanisms simulate a pathologist's "Chain of Thought" across magnifications to achieve uncertainty-aware evidence fusion. Finally, we discuss open challenges and argue that future progress depends on unified multimodal frameworks integrating high-resolution visual data with clinical and biomedical knowledge to support interpretable and safe AI-assisted diagnosis.
• 摘要：全玻片成像 (WSI) 通过对十亿像素组织病理学图像进行计算分析，改变了数字病理学。最近的基础模型进展加速了计算病理学的进展，促进了病理图像、临床报告和结构化数据的联合推理。尽管取得了这些进展，挑战仍然存在：WSI 的极端分辨率给视觉学习带来了计算障碍；有限的专家注释限制了监督方法；在保持生物学可解释性的同时整合多模态信息仍然很困难；超长视觉序列建模的不透明性阻碍了临床透明度。这篇综述全面调查了多模态计算病理学的最新进展。我们系统地分析了四个研究方向：（1）WSI 的自监督表示学习和结构感知令牌压缩； (2) 多模态数据生成和增强； （3）参数高效自适应和推理增强的小样本学习； (4)多智能体协作推理以实现可信的诊断。我们特别研究了令牌压缩如何实现跨尺度建模，以及多代理机制如何跨放大倍数模拟病理学家的“思维链”，以实现不确定性感知的证据融合。最后，我们讨论了开放的挑战，并认为未来的进展取决于统一的多模式框架，将高分辨率视觉数据与临床和生物医学知识相结合，以支持可解释和安全的人工智能辅助诊断。

Title: CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks

• Authors: Hao Wang, Licheng Pan, Zhichao Chen, Chunyuan Zheng, Zhixuan Chu, Xiaoxi Li, Yuan Lu, Xinggao Liu, Haoxuan Li, Zhouchen Lin
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2603.18736
• Pdf URL: https://arxiv.org/pdf/2603.18736
• Copy Paste: [[2603.18736]] CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks(https://arxiv.org/abs/2603.18736)
• Keywords: generation
• Abstract: Despite the success of reinforcement learning from human feedback (RLHF) in aligning language models, current reward modeling heavily relies on experimental feedback data collected from human annotators under controlled and costly conditions. In this work, we introduce observational reward modeling -- learning reward models with observational user feedback (e.g., clicks, copies, and upvotes) -- as a scalable and cost-effective alternative. We identify two fundamental challenges in this setting: (1) observational feedback is noisy due to annotation errors, which deviates it from true user preference; (2) observational feedback is biased by user preference, where users preferentially provide feedback on responses they feel strongly about, which creats a distribution shift between training and inference data. To address these challenges, we propose CausalRM, a causal-theoretic reward modeling framework that aims to learn unbiased reward models from observational feedback. To tackle challenge (1), CausalRM introduces a noise-aware surrogate loss term that is provably equivalent to the primal loss under noise-free conditions by explicitly modeling the annotation error generation process. To tackle challenge (2), CausalRM uses propensity scores -- the probability of a user providing feedback for a given response -- to reweight training samples, yielding a loss function that eliminates user preference bias. Extensive experiments across diverse LLM backbones and benchmark datasets validate that CausalRM effectively learns accurate reward signals from noisy and biased observational feedback and delivers substantial performance improvements on downstream RLHF tasks -- including a 49.2% gain on WildGuardMix and a 32.7% improvement on HarmBench. Code is available on our project website.
• 摘要：尽管基于人类反馈的强化学习（RLHF）在调整语言模型方面取得了成功，但当前的奖励建模在很大程度上依赖于在受控和昂贵的条件下从人类注释者收集的实验反馈数据。在这项工作中，我们引入了观察奖励模型——通过观察用户反馈（例如点击、复制和点赞）来学习奖励模型——作为一种可扩展且经济高效的替代方案。我们在此设置中确定了两个基本挑战：（1）由于注释错误，观察反馈存在噪音，这偏离了真实的用户偏好； (2) 观察反馈因用户偏好而存在偏差，用户优先针对他们强烈感受的响应提供反馈，这会造成训练数据和推理数据之间的分布变化。为了应对这些挑战，我们提出了 CausalRM，一种因果理论奖励建模框架，旨在从观察反馈中学习无偏见的奖励模型。为了应对挑战 (1)，CausalRM 引入了一种噪声感知代理损失项，通过对注释错误生成过程进行显式建模，可证明该代理损失项与无噪声条件下的原始损失等效。为了应对挑战 (2)，CausalRM 使用倾向得分（用户针对给定响应提供反馈的概率）来重新加权训练样本，从而产生消除用户偏好偏差的损失函数。跨不同 LLM 主干和基准数据集的广泛实验验证了 CausalRM 可以有效地从噪声和有偏差的观察反馈中学习准确的奖励信号，并为下游 RLHF 任务带来显着的性能改进，包括在 WildGuardMix 上提高 49.2%，在 HarmBench 上提高 32.7%。代码可在我们的项目网站上找到。

Title: 6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models

• Authors: Rundong Su, Jintao Zhang, Zhihang Yuan, Haojie Duanmu, Jianfei Chen, Jun Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18742
• Pdf URL: https://arxiv.org/pdf/2603.18742
• Copy Paste: [[2603.18742]] 6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models(https://arxiv.org/abs/2603.18742)
• Keywords: generation
• Abstract: Diffusion transformers have demonstrated remarkable capabilities in generating videos. However, their practical deployment is severely constrained by high memory usage and computational cost. Post-Training Quantization provides a practical way to reduce memory usage and boost computation speed. Existing quantization methods typically apply a static bit-width allocation, overlooking the quantization difficulty of activations across diffusion timesteps, leading to a suboptimal trade-off between efficiency and quality. In this paper, we propose a inference time NVFP4/INT8 Mixed-Precision Quantization framework. We find a strong linear correlation between a block's input-output difference and the quantization sensitivity of its internal linear layers. Based on this insight, we design a lightweight predictor that dynamically allocates NVFP4 to temporally stable layers to maximize memory compression, while selectively preserving INT8 for volatile layers to ensure robustness. This adaptive precision strategy enables aggressive quantization without compromising generation quality. Beside this, we observe that the residual between the input and output of a Transformer block exhibits high temporal consistency across timesteps. Leveraging this temporal redundancy, we introduce Temporal Delta Cache (TDC) to skip computations for these invariant blocks, further reducing the computational cost. Extensive experiments demonstrate that our method achieves 1.92$\times$ end-to-end acceleration and 3.32$\times$ memory reduction, setting a new baseline for efficient inference in Video DiTs.
• 摘要：扩散变压器在生成视频方面表现出了卓越的能力。然而，它们的实际部署受到高内存使用和计算成本的严重限制。训练后量化提供了一种减少内存使用和提高计算速度的实用方法。现有的量化方法通常采用静态位宽分配，忽略了跨扩散时间步长的激活的量化难度，导致效率和质量之间的权衡不理想。在本文中，我们提出了一种推理时间 NVFP4/INT8 混合精度量化框架。我们发现块的输入输出差异与其内部线性层的量化灵敏度之间存在很强的线性相关性。基于这一见解，我们设计了一个轻量级预测器，可以动态地将 NVFP4 分配给临时稳定层以最大化内存压缩，同时有选择地为易失性层保留 INT8 以确保鲁棒性。这种自适应精度策略可以在不影响生成质量的情况下实现积极的量化。除此之外，我们观察到 Transformer 块的输入和输出之间的残差在时间步长上表现出高度的时间一致性。利用这种时间冗余，我们引入了时间增量缓存（TDC）来跳过这些不变块的计算，进一步降低了计算成本。大量实验表明，我们的方法实现了 1.92$\times$ 端到端加速和 3.32$\times$ 内存减少，为视频 DiT 中的高效推理设定了新的基准。

Title: WeNLEX: Weakly Supervised Natural Language Explanations for Multilabel Chest X-ray Classification

• Authors: Isabel Rio-Torto, Jaime S. Cardoso, Luís F. Teixeira
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18752
• Pdf URL: https://arxiv.org/pdf/2603.18752
• Copy Paste: [[2603.18752]] WeNLEX: Weakly Supervised Natural Language Explanations for Multilabel Chest X-ray Classification(https://arxiv.org/abs/2603.18752)
• Keywords: generation
• Abstract: Natural language explanations provide an inherently human-understandable way to explain black-box models, closely reflecting how radiologists convey their diagnoses in textual reports. Most works explicitly supervise the explanation generation process using datasets annotated with explanations. Thus, though plausible, the generated explanations are not faithful to the model's reasoning. In this work, we propose WeNLEX, a weakly supervised model for the generation of natural language explanations for multilabel chest X-ray classification. Faithfulness is ensured by matching images generated from their corresponding natural language explanations with original images, in the black-box model's feature space. Plausibility is maintained via distribution alignment with a small database of clinician-annotated explanations. We empirically demonstrate, through extensive validation on multiple metrics to assess faithfulness, simulatability, diversity, and plausibility, that WeNLEX is able to produce faithful and plausible explanations, using as little as 5 ground-truth explanations per diagnosis. Furthermore, WeNLEX can operate in both post-hoc and in-model settings. In the latter, i.e., when the multilabel classifier is trained together with the rest of the network, WeNLEX improves the classification AUC of the standalone classifier by 2.21%, thus showing that adding interpretability to the training process can actually increase the downstream task performance. Additionally, simply by changing the database, WeNLEX explanations are adaptable to any target audience, and we showcase this flexibility by training a layman version of WeNLEX, where explanations are simplified for non-medical users.
• 摘要：自然语言解释提供了一种本质上人类可理解的方式来解释黑盒模型，密切反映了放射科医生如何在文本报告中传达他们的诊断。大多数作品使用带有解释注释的数据集明确地监督解释生成过程。因此，虽然看似合理，但生成的解释并不忠实于模型的推理。在这项工作中，我们提出了 WeNLEX，这是一种弱监督模型，用于生成多标签胸部 X 射线分类的自然语言解释。通过在黑盒模型的特征空间中将根据相应的自然语言解释生成的图像与原始图像进行匹配来确保忠实性。通过与临床医生注释解释的小型数据库进行分布对齐来维持合理性。我们通过对多个指标进行广泛验证来评估真实性、可模拟性、多样性和合理性，凭经验证明，WeNLEX 能够在每个诊断中仅使用 5 个真实解释来生成忠实且合理的解释。此外，WeNLEX 可以在事后和模型内设置中运行。在后者中，即当多标签分类器与网络的其余部分一起训练时，WeNLEX 将独立分类器的分类 AUC 提高了 2.21%，从而表明在训练过程中增加可解释性实际上可以提高下游任务性能。此外，只需更改数据库，WeNLEX 解释就可以适应任何目标受众，并且我们通过训练 WeNLEX 的外行版本来展示这种灵活性，其中针对非医疗用户的解释得到了简化。

Title: Enhancing the Parameterization of Reservoir Properties for Data Assimilation Using Deep VAE-GAN

• Authors: Marcio Augusto Sampaio, Paulo Henrique Ranazzi, Martin Julian Blunt
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18766
• Pdf URL: https://arxiv.org/pdf/2603.18766
• Copy Paste: [[2603.18766]] Enhancing the Parameterization of Reservoir Properties for Data Assimilation Using Deep VAE-GAN(https://arxiv.org/abs/2603.18766)
• Keywords: generative
• Abstract: Currently, the methods called Iterative Ensemble Smoothers, especially the method called Ensemble Smoother with Multiple Data Assimilation (ESMDA) can be considered state-of-the-art for history matching in petroleum reservoir simulation. However, this approach has two important limitations: the use of an ensemble with finite size to represent the distributions and the Gaussian assumption in parameter and data uncertainties. This latter is particularly important because many reservoir properties have non-Gaussian distributions. Parameterization involves mapping non-Gaussian parameters to a Gaussian field before the update and then mapping them back to the original domain to forward the ensemble through the reservoir simulator. A promising approach to perform parameterization is through deep learning models. Recent studies have shown that Generative Adversarial Networks (GAN) performed poorly concerning data assimilation, but generated more geologically plausible realizations of the reservoir, while the Variational Autoencoder (VAE) performed better than the GAN in data assimilation, but generated less geologically realistic models. This work is innovative in combining the strengths of both to implement a deep learning model called Variational Autoencoder Generative Adversarial Network (VAE-GAN) integrated with ESMDA. The methodology was applied in two case studies, one case being categorical and the other with continuous values of permeability. Our findings demonstrate that by applying the VAE-GAN model we can obtain high quality reservoir descriptions (just like GANs) and a good history matching on the production curves (just like VAEs) simultaneously.
• 摘要：目前，迭代集成平滑器的方法，特别是具有多重数据同化的集成平滑器（ESMDA）的方法可以被认为是石油储层模拟中历史匹配的最先进方法。然而，这种方法有两个重要的局限性：使用有限大小的集合来表示分布以及参数和数据不确定性的高斯假设。后者尤其重要，因为许多储层特性具有非高斯分布。参数化涉及在更新之前将非高斯参数映射到高斯场，然后将它们映射回原始域以通过油藏模拟器转发集合。执行参数化的一种有前途的方法是通过深度学习模型。最近的研究表明，生成对抗网络（GAN）在数据同化方面表现不佳，但生成了地质上更合理的储层实现，而变分自动编码器（VAE）在数据同化方面表现优于 GAN，但生成的地质模型不太真实。这项工作的创新之处在于结合了两者的优势，实现了与 ESMDA 集成的称为变分自动编码器生成对抗网络（VAE-GAN）的深度学习模型。该方法应用于两个案例研究，一个是分类案例，另一个是连续渗透率值。我们的研究结果表明，通过应用 VAE-GAN 模型，我们可以同时获得高质量的油藏描述（就像 GAN 一样）和生产曲线上良好的历史匹配（就像 VAE 一样）。

Title: Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors

• Authors: Jiatong Xia, Zicheng Duan, Anton van den Hengel, Lingqiao Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18782
• Pdf URL: https://arxiv.org/pdf/2603.18782
• Copy Paste: [[2603.18782]] Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors(https://arxiv.org/abs/2603.18782)
• Keywords: generation
• Abstract: Recent progress in 3D generation has been driven largely by models conditioned on images or text, while readily available 3D priors are still underused. In many real-world scenarios, the visible-region point cloud are easy to obtain from active sensors such as LiDAR or from feed-forward predictors like VGGT, offering explicit geometric constraints that current methods fail to exploit. In this work, we introduce Points-to-3D, a diffusion-based framework that leverages point cloud priors for geometry-controllable 3D asset and scene generation. Built on a latent 3D diffusion model TRELLIS, Points-to-3D first replaces pure-noise sparse structure latent initialization with a point cloud priors tailored input formulation.A structure inpainting network, trained within the TRELLIS framework on task-specific data designed to learn global structural inpainting, is then used for inference with a staged sampling strategy (structural inpainting followed by boundary refinement), completing the global geometry while preserving the visible regions of the input this http URL practice, Points-to-3D can take either accurate point-cloud priors or VGGT-estimated point clouds from single images as input. Experiments on both objects and scene scenarios consistently demonstrate superior performance over state-of-the-art baselines in terms of rendering quality and geometric fidelity, highlighting the effectiveness of explicitly embedding point-cloud priors for achieving more accurate and structurally controllable 3D generation.
• 摘要：3D 生成的最新进展主要是由以图像或文本为条件的模型推动的，而现成的 3D 先验仍然没有得到充分利用。在许多现实场景中，可见区域点云很容易从主动传感器（例如 LiDAR）或前馈预测器（例如 VGGT）获得，从而提供当前方法无法利用的显式几何约束。在这项工作中，我们引入了 Points-to-3D，这是一种基于扩散的框架，它利用点云先验来生成几何可控的 3D 资产和场景。 Points-to-3D 基于潜在 3D 扩散模型 TRELLIS 构建，首先用点云先验定制输入公式取代纯噪声稀疏结构潜在初始化。结构修复网络在 TRELLIS 框架内针对特定任务数据进行训练，旨在学习全局结构修复，然后使用分阶段采样策略（结构修复后进行边界细化）进行推理，完成全局几何形状，同时保留此 http URL 实践中输入的可见区域， Points-to-3D 可以采用来自单个图像的准确点云先验或 VGGT 估计点云作为输入。对对象和场景场景的实验一致证明，在渲染质量和几何保真度方面，其性能优于最先进的基线，凸显了显式嵌入点云先验对于实现更准确和结构可控的 3D 生成的有效性。

Title: Perceptio: Perception Enhanced Vision Language Models via Spatial Token Generation

• Authors: Yuchen Li, Amanmeet Garg, Shalini Chaudhuri, Rui Zhao, Garin Kessler
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18795
• Pdf URL: https://arxiv.org/pdf/2603.18795
• Copy Paste: [[2603.18795]] Perceptio: Perception Enhanced Vision Language Models via Spatial Token Generation(https://arxiv.org/abs/2603.18795)
• Keywords: generation
• Abstract: Large Vision Language Models (LVLMs) excel at semantic understanding but struggle with fine grained spatial grounding, as the model must implicitly infer complex geometry without ever producing a spatial interpretation. We present Perceptio, a perception enhanced LVLM with 2D and 3D spatial reasoning abilities, enabled via explicit semantic segmentation tokens and depth tokens generated directly within the autoregressive sequence. Concretely, we (i) distill a VQVAE depth codebook from a strong monocular teacher to tokenize dense depth into compact sequences, and (ii) integrate SAM2 based semantic segmentation tokens and VQ-VAE depth tokens inside the LLM so the model first emits spatial tokens and then answers. To stabilize depth token generation, we introduce novel composite depth-token objectives (marker, token, and count losses) and a soft-merging technique for differentiable reconstruction. We adopt a multi-task co-training strategy across diverse datasets, letting the model learn perception tokens to tackle multiple downstream tasks. Building on InternVL, Perceptio achieves state-of-the-art performance across benchmarks: improving referring expression segmentation by +0.8/+1.4/+1.1 cIoU on RefCOCO/+/g HardBLINK spatial understanding accuracy by 10.3%, and MMBench accuracy by 1.0%, demonstrating that explicit spatial chain-of-thought materially strengthens spatial grounding in LVLMs.
• 摘要：大视觉语言模型 (LVLM) 擅长语义理解，但难以处理细粒度的空间基础，因为该模型必须隐式推断复杂的几何形状，而不会产生空间解释。我们提出了 Perceptio，一种具有 2D 和 3D 空间推理能力的感知增强 LVLM，通过在自回归序列中直接生成的显式语义分割标记和深度标记来实现。具体来说，我们 (i) 从强大的单目教师那里提取 VQVAE 深度密码本，将密集深度标记为紧凑序列，并且 (ii) 将基于 SAM2 的语义分割标记和 VQ-VAE 深度标记集成到 LLM 中，因此模型首先发出空间标记，然后给出答案。为了稳定深度标记的生成，我们引入了新颖的复合深度标记目标（标记、标记和计数损失）和用于可微重建的软合并技术。我们在不同的数据集上采用多任务协同训练策略，让模型学习感知标记来处理多个下游任务。以 InternVL 为基础，Perceptio 在各个基准测试中实现了最先进的性能：在 RefCOCO/+/g HardBLINK 上将指代表达分割提高了 +0.8/+1.4/+1.1 cIoU，空间理解精度提高了 10.3%，MMBench 精度提高了 1.0%，这表明明确的空间思想链实质上增强了 LVLM 中的空间基础。

Title: VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation

• Authors: Chinmay Prabhakar, Bastian Wittmann, Tamaz Amiranashvili, Paul Büschl, Ezequiel de la Rosa, Julian McGinnis, Benedikt Wiestler, Bjoern Menze, Suprosanna Shit
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.18797
• Pdf URL: https://arxiv.org/pdf/2603.18797
• Copy Paste: [[2603.18797]] VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation(https://arxiv.org/abs/2603.18797)
• Keywords: generation, generative
• Abstract: Spatial graphs provide a lightweight and elegant representation of curvilinear anatomical structures such as blood vessels, lung airways, and neuronal networks. Accurately modeling these graphs is crucial in clinical and (bio-)medical research. However, the high spatial resolution of large networks drastically increases their complexity, resulting in significant computational challenges. In this work, we aim to tackle these challenges by proposing VesselTok, a framework that approaches spatially dense graphs from a parametric shape perspective to learn latent representations (tokens). VesselTok leverages centerline points with a pseudo radius to effectively encode tubular geometry. Specifically, we learn a novel latent representation conditioned on centerline points to encode neural implicit representations of vessel-like, tubular structures. We demonstrate VesselTok's performance across diverse anatomies, including lung airways, lung vessels, and brain vessels, highlighting its ability to robustly encode complex topologies. To prove the effectiveness of VesselTok's learnt latent representations, we show that they (i) generalize to unseen anatomies, (ii) support generative modeling of plausible anatomical graphs, and (iii) transfer effectively to downstream inverse problems, such as link prediction.
• 摘要：空间图提供了曲线解剖结构（例如血管、肺气道和神经网络）的轻量级且优雅的表示。准确地对这些图进行建模对于临床和（生物）医学研究至关重要。然而，大型网络的高空间分辨率极大地增加了其复杂性，导致了巨大的计算挑战。在这项工作中，我们的目标是通过提出 VesselTok 来应对这些挑战，这是一个从参数形状角度处理空间密集图以学习潜在表示（标记）的框架。 VesselTok 利用具有伪半径的中心线点来有效地编码管状几何形状。具体来说，我们学习了一种以中心线点为条件的新颖的潜在表示，以编码血管状管状结构的神经隐式表示。我们展示了 VesselTok 在不同解剖结构（包括肺气道、肺血管和脑血管）中的性能，强调了其稳健编码复杂拓扑的能力。为了证明 VesselTok 学习的潜在表示的有效性，我们证明它们（i）泛化到看不见的解剖结构，（ii）支持合理解剖图的生成模型，以及（iii）有效地转移到下游逆问题，例如链接预测。

Title: Seasoning Generative Models for a Generalization Aftertaste

• Authors: Hisham Husain, Valentin De Bortoli, Richard Nock
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.18817
• Pdf URL: https://arxiv.org/pdf/2603.18817
• Copy Paste: [[2603.18817]] Seasoning Generative Models for a Generalization Aftertaste(https://arxiv.org/abs/2603.18817)
• Keywords: generative
• Abstract: The use of discriminators to train or fine-tune generative models has proven to be a rather successful framework. A notable example is Generative Adversarial Networks (GANs) that minimize a loss incurred by training discriminators along with other paradigms that boost generative models via discriminators that satisfy weak learner constraints. More recently, even diffusion models have shown advantages with some kind of discriminator guidance. In this work, we extend a strong-duality result related to $f$-divergences which gives rise to a discriminator-guided recipe that allows us to \textit{refine} any generative model. We then show that the refined generative models provably improve generalization, compared to its non-refined counterpart. In particular, our analysis reveals that the gap in generalization is improved based on the Rademacher complexity of the discriminator set used for refinement. Our recipe subsumes a recently introduced score-based diffusion approach (Kim et al., 2022) that has shown great empirical success, however allows us to shed light on the generalization guarantees of this method by virtue of our analysis. Thus, our work provides a theoretical validation for existing work, suggests avenues for new algorithms, and contributes to our understanding of generalization in generative models at large.
• 摘要：使用判别器来训练或微调生成模型已被证明是一个相当成功的框架。一个值得注意的例子是生成对抗网络（GAN），它最大限度地减少了训练判别器以及其他通过满足弱学习器约束的判别器来增强生成模型的范式所带来的损失。最近，甚至扩散模型也显示出了某种鉴别器指导的优势。在这项工作中，我们扩展了与 $f$-divergences 相关的强对偶结果，这产生了判别器引导的方法，使我们能够\textit{refine}任何生成模型。然后我们证明，与非精炼的生成模型相比，精炼的生成模型可以提高泛化能力。特别是，我们的分析表明，基于用于细化的判别器集的 Rademacher 复杂度，泛化方面的差距得到了改善。我们的方案包含了最近引入的基于分数的扩散方法（Kim 等人，2022），该方法已显示出巨大的经验成功，但使我们能够通过我们的分析阐明该方法的泛化保证。因此，我们的工作为现有工作提供了理论验证，为新算法提出了途径，并有助于我们对生成模型泛化的理解。

Title: Translating MRI to PET through Conditional Diffusion Models with Enhanced Pathology Awareness

• Authors: Yitong Li, Igor Yakushev, Dennis M. Hedderich, Christian Wachinger
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18896
• Pdf URL: https://arxiv.org/pdf/2603.18896
• Copy Paste: [[2603.18896]] Translating MRI to PET through Conditional Diffusion Models with Enhanced Pathology Awareness(https://arxiv.org/abs/2603.18896)
• Keywords: generation, generative
• Abstract: Positron emission tomography (PET) is a widely recognized technique for diagnosing neurodegenerative diseases, offering critical functional insights. However, its high costs and radiation exposure hinder its widespread use. In contrast, magnetic resonance imaging (MRI) does not involve such limitations. While MRI also detects neurodegenerative changes, it is less sensitive for diagnosis compared to PET. To overcome such limitations, one approach is to generate synthetic PET from MRI. Recent advances in generative models have paved the way for cross-modality medical image translation; however, existing methods largely emphasize structural preservation while neglecting the critical need for pathology awareness. To address this gap, we propose PASTA, a novel image translation framework built on conditional diffusion models with enhanced pathology awareness. PASTA surpasses state-of-the-art methods by preserving both structural and pathological details through its highly interactive dual-arm architecture and multi-modal condition integration. Additionally, we introduce a novel cycle exchange consistency and volumetric generation strategy that significantly enhances PASTA's ability to produce high-quality 3D PET images. Our qualitative and quantitative results demonstrate the high quality and pathology awareness of the synthesized PET scans. For Alzheimer's diagnosis, the performance of these synthesized scans improves over MRI by 4%, almost reaching the performance of actual PET. Our code is available at this https URL.
• 摘要：正电子发射断层扫描 (PET) 是一种广泛认可的诊断神经退行性疾病的技术，可提供重要的功能见解。然而，其高成本和辐射暴露阻碍了其广泛使用。相比之下，磁共振成像（MRI）则不存在此类限制。虽然 MRI 也可以检测神经退行性变化，但与 PET 相比，其诊断敏感性较低。为了克服这些限制，一种方法是从 MRI 生成合成 PET。生成模型的最新进展为跨模态医学图像翻译铺平了道路；然而，现有的方法主要强调结构保存，而忽视了病理意识的迫切需要。为了解决这一差距，我们提出了 PASTA，这是一种基于条件扩散模型构建的新型图像翻译框架，具有增强的病理学意识。 PASTA 通过其高度交互的双臂架构和多模态条件集成保留结构和病理细节，超越了最先进的方法。此外，我们还引入了一种新颖的循环交换一致性和体积生成策略，可显着增强 PASTA 生成高质量 3D PET 图像的能力。我们的定性和定量结果证明了合成 PET 扫描的高质量和病理学意识。对于阿尔茨海默病的诊断，这些合成扫描的性能比 MRI 提高了 4%，几乎达到了实际 PET 的性能。我们的代码可以在这个 https URL 上找到。

Title: Neural Galerkin Normalizing Flow for Transition Probability Density Functions of Diffusion Models

• Authors: Riccardo Saporiti, Fabio Nobile
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2603.18907
• Pdf URL: https://arxiv.org/pdf/2603.18907
• Copy Paste: [[2603.18907]] Neural Galerkin Normalizing Flow for Transition Probability Density Functions of Diffusion Models(https://arxiv.org/abs/2603.18907)
• Keywords: generation
• Abstract: We propose a new Neural Galerkin Normalizing Flow framework to approximate the transition probability density function of a diffusion process by solving the corresponding Fokker-Planck equation with an atomic initial distribution, parametrically with respect to the location of the initial mass. By using Normalizing Flows, we look for the solution as a transformation of the transition probability density function of a reference stochastic process, ensuring that our approximation is structure-preserving and automatically satisfies positivity and mass conservation constraints. By extending Neural Galerkin schemes to the context of Normalizing Flows, we derive a system of ODEs for the time evolution of the Normalizing Flow's parameters. Adaptive sampling routines are used to evaluate the Fokker-Planck residual in meaningful locations, which is of vital importance to address high-dimensional PDEs. Numerical results show that this strategy captures key features of the true solution and enforces the causal relationship between the initial datum and the density function at subsequent times. After completing an offline training phase, online evaluation becomes significantly more cost-effective than solving the PDE from scratch. The proposed method serves as a promising surrogate model, which could be deployed in many-query problems associated with stochastic differential equations, like Bayesian inference, simulation, and diffusion bridge generation.
• 摘要：我们提出了一种新的神经伽辽金归一流框架，通过求解具有原子初始分布的相应福克-普朗克方程（相对于初始质量的位置进行参数化）来近似扩散过程的转移概率密度函数。通过使用归一化流，我们将解决方案视为参考随机过程的转移概率密度函数的变换，确保我们的近似保持结构并自动满足正性和质量守恒约束。通过将神经伽辽金方案扩展到归一化流的背景下，我们导出了归一化流参数随时间演化的 ODE 系统。自适应采样例程用于评估有意义位置的 Fokker-Planck 残差，这对于解决高维偏微分方程至关重要。数值结果表明，该策略捕获了真实解决方案的关键特征，并在后续时间强化了初始数据和密度函数之间的因果关系。完成离线训练阶段后，在线评估比从头开始求解偏微分方程更具成本效益。所提出的方法是一种有前途的代理模型，可以应用于与随机微分方程相关的许多查询问题，例如贝叶斯推理、模拟和扩散桥生成。

Title: Foundations of Schrödinger Bridges for Generative Modeling

• Authors: Sophia Tang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.18992
• Pdf URL: https://arxiv.org/pdf/2603.18992
• Copy Paste: [[2603.18992]] Foundations of Schrödinger Bridges for Generative Modeling(https://arxiv.org/abs/2603.18992)
• Keywords: generative
• Abstract: At the core of modern generative modeling frameworks, including diffusion models, score-based models, and flow matching, is the task of transforming a simple prior distribution into a complex target distribution through stochastic paths in probability space. Schrödinger bridges provide a unifying principle underlying these approaches, framing the problem as determining an optimal stochastic bridge between marginal distribution constraints with minimal-entropy deviations from a pre-defined reference process. This guide develops the mathematical foundations of the Schrödinger bridge problem, drawing on optimal transport, stochastic control, and path-space optimization, and focuses on its dynamic formulation with direct connections to modern generative modeling. We build a comprehensive toolkit for constructing Schrödinger bridges from first principles, and show how these constructions give rise to generalized and task-specific computational methods.
• 摘要：现代生成建模框架（包括扩散模型、基于评分的模型和流匹配）的核心是通过概率空间中的随机路径将简单的先验分布转换为复杂的目标分布。薛定谔桥提供了这些方法背后的统一原则，将问题定义为确定边缘分布约束与预定义参考过程的最小熵偏差之间的最佳随机桥。本指南利用最优传输、随机控制和路径空间优化，发展了薛定谔桥问题的数学基础，并重点关注其与现代生成模型直接联系的动态公式。我们构建了一个综合工具包，用于从第一原理构建薛定谔桥，并展示这些结构如何产生通用和特定于任务的计算方法。

Title: AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science

• Authors: An Luo, Jin Du, Xun Xian, Robert Specht, Fangqiao Tian, Ganghua Wang, Xuan Bi, Charles Fleming, Ashish Kundu, Jayanth Srinivasa, Mingyi Hong, Rui Zhang, Tianxi Li, Galin Jones, Jie Ding
• Subjects: cs.LG, cs.AI, stat.ME
• Abstract URL: https://arxiv.org/abs/2603.19005
• Pdf URL: https://arxiv.org/pdf/2603.19005
• Copy Paste: [[2603.19005]] AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science(https://arxiv.org/abs/2603.19005)
• Keywords: generation
• Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains. Recent developments in large language models (LLMs) and artificial intelligence (AI) agents have significantly automated data science workflow. However, it remains unclear to what extent AI agents can match the performance of human experts on domain-specific data science tasks, and in which aspects human expertise continues to provide advantages. We introduce AgentDS, a benchmark and competition designed to evaluate both AI agents and human-AI collaboration performance in domain-specific data science. AgentDS consists of 17 challenges across six industries: commerce, food production, healthcare, insurance, manufacturing, and retail banking. We conducted an open competition involving 29 teams and 80 participants, enabling systematic comparison between human-AI collaborative approaches and AI-only baselines. Our results show that current AI agents struggle with domain-specific reasoning. AI-only baselines perform near or below the median of competition participants, while the strongest solutions arise from human-AI collaboration. These findings challenge the narrative of complete automation by AI and underscore the enduring importance of human expertise in data science, while illuminating directions for the next generation of AI. Visit the AgentDS website here: this https URL and open source datasets here: this https URL .
• 摘要：数据科学在将复杂数据转化为跨多个领域的可行见解方面发挥着关键作用。大型语言模型 (LLM) 和人工智能 (AI) 代理的最新发展显着实现了数据科学工作流程的自动化。然而，目前尚不清楚人工智能代理在特定领域的数据科学任务上能够在多大程度上与人类专家的表现相媲美，以及人类专业知识在哪些方面继续提供优势。我们推出了 AgentDS，这是一个基准和竞赛，旨在评估特定领域数据科学中的人工智能代理和人机协作性能。 AgentDS 包含 17 项挑战，涉及六个行业：商业、食品生产、医疗保健、保险、制造和零售银行业务。我们举办了一场由 29 个团队和 80 名参与者参与的公开竞赛，对人类与人工智能协作方法和纯人工智能基线进行系统比较。我们的结果表明，当前的人工智能代理在特定领域的推理方面遇到了困难。仅人工智能基线的表现接近或低于竞赛参与者的中位数，而最强大的解决方案来自人类与人工智能的协作。这些发现挑战了人工智能完全自动化的叙述，强调了人类专业知识在数据科学中的持久重要性，同时阐明了下一代人工智能的方向。请访问此处的 AgentDS 网站：此 https URL 和此处的开源数据集：此 https URL 。

Title: Measuring 3D Spatial Geometric Consistency in Dynamic Generated Videos

• Authors: Weijia Dou, Wenzhao Zheng, Weiliang Chen, Yu Zheng, Jie Zhou, Jiwen Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19048
• Pdf URL: https://arxiv.org/pdf/2603.19048
• Copy Paste: [[2603.19048]] Measuring 3D Spatial Geometric Consistency in Dynamic Generated Videos(https://arxiv.org/abs/2603.19048)
• Keywords: generative
• Abstract: Recent generative models can produce high-fidelity videos, yet they often exhibit 3D spatial geometric inconsistencies. Existing evaluation methods fail to accurately characterize these inconsistencies: fidelity-centric metrics like FVD are insensitive to geometric distortions, while consistency-focused benchmarks often penalize valid foreground dynamics. To address this gap, we introduce SGC, a metric for evaluating 3D \textbf{S}patial \textbf{G}eometric \textbf{C}onsistency in dynamically generated videos. We quantify geometric consistency by measuring the divergence among multiple camera poses estimated from distinct local regions. Our approach first separates static from dynamic regions, then partitions the static background into spatially coherent sub-regions. We predict depth for each pixel, estimate a local camera pose for each subregion, and compute the divergence among these poses to quantify geometric consistency. Experiments on real and generative videos demonstrate that SGC robustly quantifies geometric inconsistencies, effectively identifying critical failures missed by existing metrics.
• 摘要：最近的生成模型可以生成高保真视频，但它们经常表现出 3D 空间几何不一致。现有的评估方法无法准确描述这些不一致的特征：以保真度为中心的指标（如 FVD）对几何失真不敏感，而以一致性为中心的基准通常会影响有效的前景动态。为了解决这一差距，我们引入了 SGC，这是一种用于评估动态生成视频中 3D \textbf{S}patial \textbf{G}eometric \textbf{C} 一致性的指标。我们通过测量从不同局部区域估计的多个相机姿势之间的差异来量化几何一致性。我们的方法首先将静态区域与动态区域分开，然后将静态背景划分为空间相干的子区域。我们预测每个像素的深度，估计每个子区域的局部相机姿态，并计算这些姿态之间的散度以量化几何一致性。对真实视频和生成视频的实验表明，SGC 可以稳健地量化几何不一致，从而有效识别现有指标遗漏的关键故障。

Title: SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation

• Authors: Phuc Pham, Uy Dieu Tran, Binh-Son Hua, Phong Nguyen
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2603.19053
• Pdf URL: https://arxiv.org/pdf/2603.19053
• Copy Paste: [[2603.19053]] SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation(https://arxiv.org/abs/2603.19053)
• Keywords: generation
• Abstract: Realistic and efficient 3D garment generation remains a longstanding challenge in computer vision and digital fashion. Existing methods typically rely on large vision- language models to produce serialized representations of 2D sewing patterns, which are then transformed into simulation-ready 3D meshes using garment modeling framework such as GarmentCode. Although these approaches yield high-quality results, they often suffer from slow inference times, ranging from 30 seconds to a minute. In this work, we introduce SwiftTailor, a novel two-stage framework that unifies sewing-pattern reasoning and geometry-based mesh synthesis through a compact geometry image representation. SwiftTailor comprises two lightweight modules: PatternMaker, an efficient vision-language model that predicts sewing patterns from diverse input modalities, and GarmentSewer, an efficient dense prediction transformer that converts these patterns into a novel Garment Geometry Image, encoding the 3D surface of all garment panels in a unified UV space. The final 3D mesh is reconstructed through an efficient inverse mapping process that incorporates remeshing and dynamic stitching algorithms to directly assemble the garment, thereby amortizing the cost of physical simulation. Extensive experiments on the Multimodal GarmentCodeData demonstrate that SwiftTailor achieves state-of-the-art accuracy and visual fidelity while significantly reducing inference time. This work offers a scalable, interpretable, and high-performance solution for next-generation 3D garment generation.
• 摘要：真实高效的 3D 服装生成仍然是计算机视觉和数字时尚领域的长期挑战。现有方法通常依赖于大型视觉语言模型来生成 2D 缝纫图案的序列化表示，然后使用 GarmentCode 等服装建模框架将其转换为可用于模拟的 3D 网格。尽管这些方法可以产生高质量的结果，但它们的推理时间通常很慢，从 30 秒到一分钟不等。在这项工作中，我们介绍了 SwiftTailor，这是一种新颖的两阶段框架，它通过紧凑的几何图像表示将缝纫图案推理和基于几何的网格合成统一起来。 SwiftTailor 包含两个轻量级模块：PatternMaker，一种高效的视觉语言模型，可根据不同的输入模式预测缝纫图案；GarmentSewer，一种高效的密集预测转换器，可将这些图案转换为新颖的服装几何图像，在统一的 UV 空间中对所有服装面板的 3D 表面进行编码。最终的 3D 网格通过高效的逆映射过程进行重建，该过程结合了重新网格化和动态缝合算法来直接组装服装，从而分摊物理模拟的成本。对多模式 GarmentCodeData 进行的大量实验表明，SwiftTailor 实现了最先进的准确性和视觉保真度，同时显着缩短了推理时间。这项工作为下一代 3D 服装生成提供了可扩展、可解释且高性能的解决方案。

Title: Revisiting Autoregressive Models for Generative Image Classification

• Authors: Ilia Sudakov, Artem Babenko, Dmitry Baranchuk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19122
• Pdf URL: https://arxiv.org/pdf/2603.19122
• Copy Paste: [[2603.19122]] Revisiting Autoregressive Models for Generative Image Classification(https://arxiv.org/abs/2603.19122)
• Keywords: generative
• Abstract: Class-conditional generative models have emerged as accurate and robust classifiers, with diffusion models demonstrating clear advantages over other visual generative paradigms, including autoregressive (AR) models. In this work, we revisit visual AR-based generative classifiers and identify an important limitation of prior approaches: their reliance on a fixed token order, which imposes a restrictive inductive bias for image understanding. We observe that single-order predictions rely more on partial discriminative cues, while averaging over multiple token orders provides a more comprehensive signal. Based on this insight, we leverage recent any-order AR models to estimate order-marginalized predictions, unlocking the high classification potential of AR models. Our approach consistently outperforms diffusion-based classifiers across diverse image classification benchmarks, while being up to 25x more efficient. Compared to state-of-the-art self-supervised discriminative models, our method delivers competitive classification performance - a notable achievement for generative classifiers.
• 摘要：类条件生成模型已经成为准确而强大的分类器，扩散模型比其他视觉生成范例（包括自回归（AR）模型）表现出明显的优势。在这项工作中，我们重新审视基于视觉 AR 的生成分类器，并确定了先前方法的一个重要限制：它们依赖于固定的标记顺序，这对图像理解施加了限制性归纳偏差。我们观察到，单阶预测更多地依赖于部分判别线索，而对多个令牌阶进行平均可以提供更全面的信号。基于这一见解，我们利用最新的任意阶 AR 模型来估计阶边缘化预测，从而释放 AR 模型的高分类潜力。我们的方法在不同的图像分类基准中始终优于基于扩散的分类器，同时效率提高了 25 倍。与最先进的自监督判别模型相比，我们的方法提供了有竞争力的分类性能——这是生成分类器的显着成就。

Title: ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation

• Authors: Kwanyoung Lee, Hyunwoo Oh, SeungJu Cha, Sungho Koh, Dong-Jin Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19157
• Pdf URL: https://arxiv.org/pdf/2603.19157
• Copy Paste: [[2603.19157]] ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation(https://arxiv.org/abs/2603.19157)
• Keywords: generation
• Abstract: Generating rare compositional concepts in text-to-image synthesis remains a challenge for diffusion models, particularly for attributes that are uncommon in the training data. While recent approaches, such as R2F, address this challenge by utilizing LLM for prompt scheduling, they suffer from inherent variance due to the randomness of language models and suboptimal guidance from iterative text embedding switching. To address these problems, we propose the ADAPT framework, a training-free framework that deterministically plans and semantically aligns prompt schedules, providing consistent guidance to enhance the composition of rare concepts. By leveraging attention scores and orthogonal components, ADAPT significantly enhances compositional generation of rare concepts in the RareBench benchmark without additional training or fine-tuning. Through comprehensive experiments, we demonstrate that ADAPT achieves superior performance in RareBench and accurately reflects the semantic information of rare attributes, providing deterministic and precise control over the generation of rare compositions without compromising visual integrity.
• 摘要：在文本到图像的合成中生成罕见的组合概念对于扩散模型来说仍然是一个挑战，特别是对于训练数据中不常见的属性。虽然最近的方法（例如 R2F）通过利用 LLM 进行即时调度来解决这一挑战，但由于语言模型的随机性和迭代文本嵌入切换的次优指导，它们受到固有的方差的影响。为了解决这些问题，我们提出了 ADAPT 框架，这是一个无需培训的框架，可以确定性地规划​​和语义上调整提示时间表，为增强稀有概念的构成提供一致的指导。通过利用注意力分数和正交分量，ADAPT 显着增强了 RareBench 基准中稀有概念的组合生成，无需额外的训练或微调。通过全面的实验，我们证明 ADAPT 在 RareBench 中实现了卓越的性能，准确地反映了稀有属性的语义信息，在不影响视觉完整性的情况下，对稀有成分的生成提供了确定性和精确的控制。

Title: Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation

• Authors: Kwanyoung Lee, SeungJu Cha, Yebin Ahn, Hyunwoo Oh, Sungho Koh, Dong-Jin Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19158
• Pdf URL: https://arxiv.org/pdf/2603.19158
• Copy Paste: [[2603.19158]] Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation(https://arxiv.org/abs/2603.19158)
• Keywords: generation
• Abstract: Diffusion-based text-to-image (T2I) models have made remarkable progress in generating photorealistic and semantically rich images. However, when the target concepts lie in low-density regions of the training distribution, these models often produce semantically misaligned or structurally inconsistent results. This limitation arises from the long-tailed nature of text-image datasets, where rare concepts or editing instructions are underrepresented. To address this, we introduce Adaptive Auxiliary Prompt Blending (AAPB) - a unified framework that stabilizes the diffusion process in low-density regions. AAPB leverages auxiliary anchor prompts to provide semantic support in rare concept generation and structural support in image editing, ensuring faithful guidance toward the target prompt. Unlike prior heuristic prompt alternation methods, AAPB derives a closed-form adaptive coefficient that optimally balances the influence between the auxiliary anchor and the target prompt at each diffusion step. Grounded in Tweedie's identity, our formulation provides a principled and training-free framework for adaptive prompt blending, ensuring stable and target-faithful generation. We demonstrate the effectiveness of adaptive interpolation over fixed interpolation through controlled experiments and empirically show consistent improvements on the RareBench and FlowEdit datasets, achieving superior semantic accuracy and structural fidelity compared to prior training-free baselines.
• 摘要：基于扩散的文本到图像（T2I）模型在生成逼真且语义丰富的图像方面取得了显着进展。然而，当目标概念位于训练分布的低密度区域时，这些模型通常会产生语义不一致或结构不一致的结果。这种限制源于文本图像数据集的长尾性质，其中罕见的概念或编辑指令代表性不足。为了解决这个问题，我们引入了自适应辅助提示混合（AAPB）——一个稳定低密度区域扩散过程的统一框架。 AAPB 利用辅助锚点提示为罕见概念生成提供语义支持，并为图像编辑提供结构支持，确保忠实引导目标提示。与之前的启发式提示交替方法不同，AAPB 导出一个封闭式自适应系数，在每个扩散步骤中最佳地平衡辅助锚点和目标提示之间的影响。基于 Tweedie 的身份，我们的公式为自适应即时混合提供了一个有原则且无需培训的框架，确保稳定且忠实于目标的生成。我们通过受控实验证明了自适应插值相对于固定插值的有效性，并凭经验证明了 RareBench 和 FlowEdit 数据集的一致改进，与之前的免训练基线相比，实现了卓越的语义准确性和结构保真度。

Title: MIDST Challenge at SaTML 2025: Membership Inference over Diffusion-models-based Synthetic Tabular data

• Authors: Masoumeh Shafieinejad, Xi He, Mahshid Alinoori, John Jewell, Sana Ayromlou, Wei Pang, Veronica Chatrath, Garui Sharma, Deval Pandya
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2603.19185
• Pdf URL: https://arxiv.org/pdf/2603.19185
• Copy Paste: [[2603.19185]] MIDST Challenge at SaTML 2025: Membership Inference over Diffusion-models-based Synthetic Tabular data(https://arxiv.org/abs/2603.19185)
• Keywords: generative
• Abstract: Synthetic data is often perceived as a silver-bullet solution to data anonymization and privacy-preserving data publishing. Drawn from generative models like diffusion models, synthetic data is expected to preserve the statistical properties of the original dataset while remaining resilient to privacy attacks. Recent developments of diffusion models have been effective on a wide range of data types, but their privacy resilience, particularly for tabular formats, remains largely unexplored. MIDST challenge sought a quantitative evaluation of the privacy gain of synthetic tabular data generated by diffusion models, with a specific focus on its resistance to membership inference attacks (MIAs). Given the heterogeneity and complexity of tabular data, multiple target models were explored for MIAs, including diffusion models for single tables of mixed data types and multi-relational tables with interconnected constraints. MIDST inspired the development of novel black-box and white-box MIAs tailored to these target diffusion models as a key outcome, enabling a comprehensive evaluation of their privacy efficacy. The MIDST GitHub repository is available at this https URL
• 摘要：合成数据通常被认为是数据匿名化和隐私保护数据发布的灵丹妙药。从扩散模型等生成模型中提取的合成数据有望保留原始数据集的统计属性，同时保持对隐私攻击的弹性。扩散模型的最新发展对多种数据类型都有效，但其隐私弹性，特别是表格格式，在很大程度上仍未得到探索。 MIDST 挑战赛寻求对扩散模型生成的合成表格数据的隐私增益进行定量评估，特别关注其对成员推理攻击 (MIA) 的抵抗力。考虑到表格数据的异质性和复杂性，MIA 探索了多个目标模型，包括混合数据类型的单个表的扩散模型和具有互连约束的多关系表。 MIDST 启发了针对这些目标扩散模型定制的新型黑盒和白盒 MIA 的开发，作为一个关键成果，从而能够对其隐私功效进行全面评估。 MIDST GitHub 存储库可通过此 https URL 获取

Title: Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting

• Authors: Yiren Lu, Xin Ye, Burhaneddin Yaman, Jingru Luo, Zhexiao Xiong, Liu Ren, Yu Yin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19193
• Pdf URL: https://arxiv.org/pdf/2603.19193
• Copy Paste: [[2603.19193]] Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting(https://arxiv.org/abs/2603.19193)
• Keywords: generation
• Abstract: Bird's-Eye-View (BEV) perception serves as a cornerstone for autonomous driving, offering a unified spatial representation that fuses surrounding-view images to enable reasoning for various downstream tasks, such as semantic segmentation, 3D object detection, and motion prediction. However, most existing BEV perception frameworks adopt an end-to-end training paradigm, where image features are directly transformed into the BEV space and optimized solely through downstream task supervision. This formulation treats the entire perception process as a black box, often lacking explicit 3D geometric understanding and interpretability, leading to suboptimal performance. In this paper, we claim that an explicit 3D representation matters for accurate BEV perception, and we propose Splat2BEV, a Gaussian Splatting-assisted framework for BEV tasks. Splat2BEV aims to learn BEV feature representations that are both semantically rich and geometrically precise. We first pre-train a Gaussian generator that explicitly reconstructs 3D scenes from multi-view inputs, enabling the generation of geometry-aligned feature representations. These representations are then projected into the BEV space to serve as inputs for downstream tasks. Extensive experiments on nuScenes and argoverse dataset demonstrate that Splat2BEV achieves state-of-the-art performance and validate the effectiveness of incorporating explicit 3D reconstruction into BEV perception.
• 摘要：鸟瞰 (BEV) 感知是自动驾驶的基石，提供统一的空间表示，融合周围视图图像，从而支持各种下游任务的推理，例如语义分割、3D 对象检测和运动预测。然而，大多数现有的 BEV 感知框架采用端到端训练范例，其中图像特征直接转换到 BEV 空间，并仅通过下游任务监督进行优化。该公式将整个感知过程视为黑匣子，通常缺乏明确的 3D 几何理解和可解释性，从而导致性能不佳。在本文中，我们声称明确的 3D 表示对于准确的 BEV 感知很重要，并且我们提出了 Splat2BEV，一种用于 BEV 任务的高斯 Splatting 辅助框架。 Splat2BEV 旨在学习语义丰富且几何精确的 BEV 特征表示。我们首先预训练一个高斯生成器，它可以根据多视图输入显式重建 3D 场景，从而生成几何对齐的特征表示。然后，这些表示被投影到 BEV 空间中，作为下游任务的输入。在 nuScenes 和 argoverse 数据集上进行的大量实验表明，Splat2BEV 实现了最先进的性能，并验证了将显式 3D 重建纳入 BEV 感知的有效性。

Title: RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editing

• Authors: Yue Gong, Hongyu Li, Shanyuan Liu, Bo Cheng, Yuhang Ma, Liebucha Wu, Xiaoyu Wu, Manyuan Zhang, Dawei Leng, Yuhui Yin, Lijun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19206
• Pdf URL: https://arxiv.org/pdf/2603.19206
• Copy Paste: [[2603.19206]] RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editing(https://arxiv.org/abs/2603.19206)
• Keywords: generation, generative
• Abstract: Diffusion models have become the dominant paradigm for image generation and editing, with latent diffusion models shifting denoising to a compact latent space for efficiency and scalability. Recent attempts to leverage pretrained visual representation models as tokenizer priors either align diffusion features to representation features or directly reuse representation encoders as frozen tokenizers. Although such approaches can improve generation metrics, they often suffer from limited reconstruction fidelity due to frozen encoders, which in turn degrades editing quality, as well as overly high-dimensional latents that make diffusion modeling difficult. To address these limitations, We propose Representation-Pivoted AutoEncoder, a representation-based tokenizer that improves both generation and editing. We introduce Representation-Pivot Regularization, a training strategy that enables a representation-initialized encoder to be fine-tuned for reconstruction while preserving the semantic structure of the pretrained representation space, followed by a variational bridge which compress latent space into a compact one for better diffusion modeling. We adopt an objective-decoupled stage-wise training strategy that sequentially optimizes generative tractability and reconstruction-fidelity objectives. Together, these components yield a tokenizer that preserves strong semantics, reconstructs faithfully, and produces latents with reduced diffusion modeling complexity. Experiments demonstrate that RPiAE outperforms other visual tokenizers on text-to-image generation and image editing, while delivering the best reconstruction fidelity among representation-based tokenizers.
• 摘要：扩散模型已成为图像生成和编辑的主导范例，潜在扩散模型将去噪转移到紧凑的潜在空间，以提高效率和可扩展性。最近尝试利用预训练的视觉表示模型作为分词器先验，将扩散特征与表示特征对齐，或者直接重用表示编码器作为冻结分词器。尽管此类方法可以改善生成指标，但由于编码器冻结，它们通常会受到有限的重建保真度的影响，这反过来又会降低编辑质量，并且潜在维度过高，使扩散建模变得困难。为了解决这些限制，我们提出了 Representation-Pivoted AutoEncoder，这是一种基于表示的分词器，可以改进生成和编辑。我们引入了表示枢轴正则化，这是一种训练策略，可以对表示初始化的编码器进行微调以进行重建，同时保留预训练表示空间的语义结构，然后是一个变分桥，将潜在空间压缩为紧凑的空间，以实现更好的扩散建模。我们采用目标解耦的分阶段训练策略，依次优化生成可处理性和重建保真度目标。这些组件共同产生了一个分词器，它保留了强大的语义，忠实地重建，并以降低的扩散建模复杂性产生潜在的。实验表明，RPiAE 在文本到图像生成和图像编辑方面优于其他视觉分词器，同时在基于表示的分词器中提供最佳的重建保真度。

Title: DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising

• Authors: Tianjiao Yu, Xinzhuo Li, Muntasir Wahed, Jerry Xiong, Yifan Shen, Ying Shen, Ismini Lourentzou
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.19216
• Pdf URL: https://arxiv.org/pdf/2603.19216
• Copy Paste: [[2603.19216]] DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising(https://arxiv.org/abs/2603.19216)
• Keywords: generation
• Abstract: Understanding and generating 3D objects as compositions of meaningful parts is fundamental to human perception and reasoning. However, most text-to-3D methods overlook the semantic and functional structure of parts. While recent part-aware approaches introduce decomposition, they remain largely geometry-focused, lacking semantic grounding and failing to model how parts align with textual descriptions or their inter-part relations. We propose DreamPartGen, a framework for semantically grounded, part-aware text-to-3D generation. DreamPartGen introduces Duplex Part Latents (DPLs) that jointly model each part's geometry and appearance, and Relational Semantic Latents (RSLs) that capture inter-part dependencies derived from language. A synchronized co-denoising process enforces mutual geometric and semantic consistency, enabling coherent, interpretable, and text-aligned 3D synthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity and text-shape alignment.
• 摘要：将 3D 对象理解和生成为有意义的部分的组合是人类感知和推理的基础。然而，大多数文本转 3D 方法都忽略了零件的语义和功能结构。虽然最近的零件感知方法引入了分解，但它们仍然主要以几何为中心，缺乏语义基础，并且无法对零件如何与文本描述或其零件间关系保持一致进行建模。我们提出了 DreamPartGen，一个基于语义、零件感知的文本到 3D 生成的框架。 DreamPartGen 引入了对每个部件的几何形状和外观进行联合建模的双工部件潜在 (DPL)，以及捕获源自语言的部件间依赖关系的关系语义潜在 (RSL)。同步的共同去噪过程增强了相互的几何和语义一致性，从而实现连贯、可解释和文本对齐的 3D 合成。在多个基准测试中，DreamPartGen 在几何保真度和文本形状对齐方面提供了最先进的性能。

Title: Rethinking Vector Field Learning for Generative Segmentation

• Authors: Chaoyang Wang, Yaobo Liang, Boci Peng, Fan Duan, Jingdong Wang, Yunhai Tong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19218
• Pdf URL: https://arxiv.org/pdf/2603.19218
• Copy Paste: [[2603.19218]] Rethinking Vector Field Learning for Generative Segmentation(https://arxiv.org/abs/2603.19218)
• Keywords: generative
• Abstract: Taming diffusion models for generative segmentation has attracted increasing attention. While existing approaches primarily focus on architectural tweaks or training heuristics, there remains a limited understanding of the intrinsic mismatch between continuous flow matching objectives and discrete perception tasks. In this work, we revisit diffusion segmentation from the perspective of vector field learning. We identify two key limitations of the commonly used flow matching objective: gradient vanishing and trajectory traversing, which result in slow convergence and poor class separation. To tackle these issues, we propose a principled vector field reshaping strategy that augments the learned velocity field with a detached distance-aware correction term. This correction introduces both attractive and repulsive interactions, enhancing gradient magnitudes near centroids while preserving the original diffusion training framework. Furthermore, we design a computationally efficient, quasi-random category encoding scheme inspired by Kronecker sequences, which integrates seamlessly with an end-to-end pixel neural field framework for pixel-level semantic alignment. Extensive experiments consistently demonstrate significant improvements over vanilla flow matching approaches, substantially narrowing the performance gap between generative segmentation and strong discriminative specialists.
• 摘要：用于生成分割的驯化扩散模型引起了越来越多的关注。虽然现有方法主要侧重于架构调整或训练启发式，但对连续流匹配目标和离散感知任务之间的内在不匹配仍然存在有限的理解。在这项工作中，我们从矢量场学习的角度重新审视扩散分割。我们确定了常用的流匹配目标的两个关键限制：梯度消失和轨迹遍历，这导致收敛速度慢和类别分离差。为了解决这些问题，我们提出了一种原则性的矢量场重塑策略，该策略通过分离的距离感知校正项来增强学习的速度场。这种校正引入了吸引和排斥相互作用，增强了质心附近的梯度幅度，同时保留了原始的扩散训练框架。此外，我们受克罗内克序列的启发，设计了一种计算高效的准随机类别编码方案，该方案与用于像素级语义对齐的端到端像素神经场框架无缝集成。大量的实验一致证明了与普通流匹配方法相比的显着改进，大大缩小了生成分割和强判别专家之间的性能差距。

Title: Spectrally-Guided Diffusion Noise Schedules

• Authors: Carlos Esteves, Ameesh Makadia
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.19222
• Pdf URL: https://arxiv.org/pdf/2603.19222
• Copy Paste: [[2603.19222]] Spectrally-Guided Diffusion Noise Schedules(https://arxiv.org/abs/2603.19222)
• Keywords: generation, generative
• Abstract: Denoising diffusion models are widely used for high-quality image and video generation. Their performance depends on noise schedules, which define the distribution of noise levels applied during training and the sequence of noise levels traversed during sampling. Noise schedules are typically handcrafted and require manual tuning across different resolutions. In this work, we propose a principled way to design per-instance noise schedules for pixel diffusion, based on the image's spectral properties. By deriving theoretical bounds on the efficacy of minimum and maximum noise levels, we design ``tight'' noise schedules that eliminate redundant steps. During inference, we propose to conditionally sample such noise schedules. Experiments show that our noise schedules improve generative quality of single-stage pixel diffusion models, particularly in the low-step regime.
• 摘要：去噪扩散模型广泛用于高质量图像和视频生成。它们的性能取决于噪声时间表，噪声时间表定义了训练期间应用的噪声级别的分布以及采样期间遍历的噪声级别的序列。噪声表通常是手工制作的，需要在不同的分辨率下进行手动调整。在这项工作中，我们提出了一种基于图像的光谱特性来设计像素扩散的每个实例噪声表的原则方法。通过推导最小和最大噪声水平功效的理论界限，我们设计了“严格”​​的噪声时间表，消除了冗余步骤。在推理过程中，我们建议对此类噪声表进行条件采样。实验表明，我们的噪声计划提高了单级像素扩散模型的生成质量，特别是在低步状态下。

Title: Under One Sun: Multi-Object Generative Perception of Materials and Illumination

• Authors: Nobuo Yoshii, Xinran Nicole Han, Ryo Kawahara, Todd Zickler, Ko Nishino
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19226
• Pdf URL: https://arxiv.org/pdf/2603.19226
• Copy Paste: [[2603.19226]] Under One Sun: Multi-Object Generative Perception of Materials and Illumination(https://arxiv.org/abs/2603.19226)
• Keywords: generative
• Abstract: We introduce Multi-Object Generative Perception (MultiGP), a generative inverse rendering method for stochastic sampling of all radiometric constituents -- reflectance, texture, and illumination -- underlying object appearance from a single image. Our key idea to solve this inherently ambiguous radiometric disentanglement is to leverage the fact that while their texture and reflectance may differ, objects in the same scene are all lit by the same illumination. MultiGP exploits this consensus to produce samples of reflectance, texture, and illumination from a single image of known shapes based on four key technical contributions: a cascaded end-to-end architecture that combines image-space and angular-space disentanglement; Coordinated Guidance for diffusion convergence to a single consistent illumination estimate; Axial Attention applied to facilitate ``cross-talk'' between objects of different reflectance; and a Texture Extraction ControlNet to preserve high-frequency texture details while ensuring decoupling from estimated lighting. Experimental results demonstrate that MultiGP effectively leverages the complementary spatial and frequency characteristics of multiple object appearances to recover individual texture and reflectance as well as the common illumination.
• 摘要：我们引入了多对象生成感知 (MultiGP)，这是一种生成逆渲染方法，用于对单个图像中的所有辐射成分（反射率、纹理和照明）进行随机采样，作为基础对象外观。我们解决这种本质上不明确的辐射解缠结的关键想法是利用这样一个事实：虽然它们的纹理和反射率可能不同，但同一场景中的物体都被相同的照明照亮。 MultiGP 利用这一共识，基于四个关键技术贡献，从已知形状的单个图像中生成反射率、纹理和照明的样本：结合图像空间和角度空间解缠结的级联端到端架构；扩散收敛到单一一致照明估计的协调指南；轴向注意力用于促进不同反射率的物体之间的“串扰”；纹理提取 ControlNet 可保留高频纹理细节，同时确保与估计照明的解耦。实验结果表明，MultiGP 有效地利用多个对象外观的互补空间和频率特性来恢复个体纹理和反射率以及共同照明。

Title: Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer

• Authors: Chenyang Gu, Mingyuan Zhang, Haozhe Xie, Zhongang Cai, Lei Yang, Ziwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19227
• Pdf URL: https://arxiv.org/pdf/2603.19227
• Copy Paste: [[2603.19227]] Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer(https://arxiv.org/abs/2603.19227)
• Keywords: generation
• Abstract: Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.
• 摘要：先前的运动生成主要遵循两个范式：擅长运动控制的连续扩散模型，以及对语义条件有效的基于离散标记的生成器。为了结合它们的优势，我们提出了一个三阶段框架，包括条件特征提取（感知）、离散标记生成（规划）和基于扩散的运动合成（控制）。该框架的核心是 MoTok，它是一种基于扩散的离散运动标记器，通过将运动恢复委托给扩散解码器，将语义抽象与细粒度重建分离，从而在保持运动保真度的同时实现紧凑的单层标记。对于运动学条件，粗粒度约束在规划过程中指导令牌生成，而细粒度约束在控制过程中通过基于扩散的优化来强制执行。这种设计可以防止运动细节破坏语义标记规划。在 HumanML3D 上，我们的方法显着提高了 MaskControl 的可控性和保真度，同时仅使用六分之一的标记，将轨迹误差从 0.72 cm 减少到 0.08 cm，将 FID 从 0.083 减少到 0.029。与在更强的运动学约束下性能下降的先前方法不同，我们的方法提高了保真度，将 FID 从 0.033 降低到 0.014。

Title: SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing

• Authors: Xinyao Zhang, Wenkai Dong, Yuxin Song, Bo Fang, Qi Zhang, Jing Wang, Fan Chen, Hui Zhang, Haocheng Feng, Yu Lu, Hang Zhou, Chun Yuan, Jingdong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19228
• Pdf URL: https://arxiv.org/pdf/2603.19228
• Copy Paste: [[2603.19228]] SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing(https://arxiv.org/abs/2603.19228)
• Keywords: restoration
• Abstract: Current instruction-guided video editing models struggle to simultaneously balance precise semantic modifications with faithful motion preservation. While existing approaches rely on injecting explicit external priors (e.g., VLM features or structural conditions) to mitigate these issues, this reliance severely bottlenecks model robustness and generalization. To overcome this limitation, we present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling. First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames, enabling purely instruction-aware structural planning. Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks (cube inpainting, speed perturbation, and tube shuffle), enabling the model to internalize temporal dynamics directly from raw videos. SAMA is optimized with a two-stage pipeline: a factorized pre-training stage that learns inherent semantic-motion representations without paired video-instruction editing data, followed by supervised fine-tuning on paired editing data. Remarkably, the factorized pre-training alone already yields strong zero-shot video editing ability, validating the proposed factorization. SAMA achieves state-of-the-art performance among open-source models and is competitive with leading commercial systems (e.g., Kling-Omni). Code, models, and datasets will be released.
• 摘要：当前的指令引导视频编辑模型很难同时平衡精确的语义修改和忠实的运动保留。虽然现有方法依靠注入显式外部先验（例如，VLM 特征或结构条件）来缓解这些问题，但这种依赖严重阻碍了模型的稳健性和泛化性。为了克服这一限制，我们提出了 SAMA（分解语义锚定和运动对齐），这是一个将视频编辑分解为语义锚定和运动建模的框架。首先，我们介绍语义锚定，它通过在稀疏锚帧上联合预测语义标记和视频潜伏来建立可靠的视觉锚定，从而实现纯粹的指令感知结构规划。其次，运动对齐在以运动为中心的视频恢复借口任务（立方体修复、速度扰动和管洗牌）上预训练相同的骨干网，使模型能够直接从原始视频中内化时间动态。 SAMA 通过两阶段管道进行了优化：分解预训练阶段，无需配对视频指令编辑数据即可学习固有的语义运动表示，然后对配对编辑数据进行监督微调。值得注意的是，仅因式分解预训练就已经产生了强大的零镜头视频编辑能力，验证了所提出的因式分解。 SAMA 在开源模型中实现了最先进的性能，并且与领先的商业系统（例如 Kling-Omni）具有竞争力。代码、模型和数据集将被发布。

Title: MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstruction

• Authors: Haitian Li, Haozhe Xie, Junxiang Xu, Beichen Wen, Fangzhou Hong, Ziwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19231
• Pdf URL: https://arxiv.org/pdf/2603.19231
• Copy Paste: [[2603.19231]] MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstruction(https://arxiv.org/abs/2603.19231)
• Keywords: generation
• Abstract: Reconstructing articulated 3D objects from a single image requires jointly inferring object geometry, part structure, and motion parameters from limited visual evidence. A key difficulty lies in the entanglement between motion cues and object structure, which makes direct articulation regression unstable. Existing methods address this challenge through multi-view supervision, retrieval-based assembly, or auxiliary video generation, often sacrificing scalability or efficiency. We present MonoArt, a unified framework grounded in progressive structural reasoning. Rather than predicting articulation directly from image features, MonoArt progressively transforms visual observations into canonical geometry, structured part representations, and motion-aware embeddings within a single architecture. This structured reasoning process enables stable and interpretable articulation inference without external motion templates or multi-stage pipelines. Extensive experiments on PartNet-Mobility demonstrate that OM achieves state-of-the-art performance in both reconstruction accuracy and inference speed. The framework further generalizes to robotic manipulation and articulated scene reconstruction.
• 摘要：从单个图像重建铰接的 3D 对象需要从有限的视觉证据中联合推断对象几何形状、零件结构和运动参数。一个关键的困难在于运动线索和物体结构之间的纠缠，这使得直接清晰度回归不稳定。现有方法通过多视图监督、基于检索的组装或辅助视频生成来解决这一挑战，但通常会牺牲可扩展性或效率。我们提出了 MonoArt，一个基于渐进结构推理的统一框架。 MonoArt 不是直接根据图像特征预测清晰度，而是逐步将视觉观察转化为单一架构中的规范几何、结构化零件表示和运动感知嵌入。这种结构化推理过程无需外部运动模板或多级管道即可实现稳定且可解释的发音推理。 PartNet-Mobility 上的大量实验表明，OM 在重建精度和推理速度方面都实现了最先进的性能。该框架进一步推广到机器人操作和铰接式场景重建。

Title: Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens

• Authors: Yuqing Wang, Chuofan Ma, Zhijie Lin, Yao Teng, Lijun Yu, Shuai Wang, Jiaming Han, Jiashi Feng, Yi Jiang, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2603.19232
• Pdf URL: https://arxiv.org/pdf/2603.19232
• Copy Paste: [[2603.19232]] Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens(https://arxiv.org/abs/2603.19232)
• Keywords: generation
• Abstract: Visual generation with discrete tokens has gained significant attention as it enables a unified token prediction paradigm shared with language models, promising seamless multimodal architectures. However, current discrete generation methods remain limited to low-dimensional latent tokens (typically 8-32 dims), sacrificing the semantic richness essential for understanding. While high-dimensional pretrained representations (768-1024 dims) could bridge this gap, their discrete generation poses fundamental challenges. In this paper, we present Cubic Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations. CubiD performs fine-grained masking throughout the high-dimensional discrete representation -- any dimension at any position can be masked and predicted from partial observations. This enables the model to learn rich correlations both within and across spatial positions, with the number of generation steps fixed at $T$ regardless of feature dimensionality, where $T \ll hwd$. On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters. Crucially, we validate that these discretized tokens preserve original representation capabilities, demonstrating that the same discrete tokens can effectively serve both understanding and generation tasks. We hope this work will inspire future research toward unified multimodal architectures. Code is available at: this https URL.
• 摘要：使用离散标记的视觉生成受到了极大的关注，因为它实现了与语言模型共享的统一标记预测范式，有望实现无缝的多模式架构。然而，当前的离散生成方法仍然仅限于低维潜在标记（通常为 8-32 维），牺牲了理解所必需的语义丰富性。虽然高维预训练表示（768-1024 维）可以弥补这一差距，但它们的离散生成带来了根本性的挑战。在本文中，我们提出了三次离散扩散（CubiD），这是第一个用于高维表示的离散生成模型。 CubiD 在整个高维离散表示中执行细粒度掩蔽 - 任何位置的任何维度都可以通过部分观察进行掩蔽和预测。这使得模型能够学习空间位置内和跨空间位置的丰富相关性，无论特征维度如何，生成步骤数都固定为 $T$，其中 $T \ll hwd$。在 ImageNet-256 上，CubiD 实现了最先进的离散生成，具有从 900M 到 3.7B 参数的强大缩放行为。至关重要的是，我们验证这些离散标记保留了原始表示能力，证明相同的离散标记可以有效地服务于理解和生成任务。我们希望这项工作能够激发未来对统一多模式架构的研究。代码可在以下位置获得：此 https URL。

Title: Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

• Authors: Xianjin Wu, Dingkang Liang, Tianrui Feng, Kui Xia, Yumeng Zhang, Xiaofan Li, Xiao Tan, Xiang Bai
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2603.19235
• Pdf URL: https://arxiv.org/pdf/2603.19235
• Copy Paste: [[2603.19235]] Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding(https://arxiv.org/abs/2603.19235)
• Keywords: generation, generative
• Abstract: While Multimodal Large Language Models demonstrate impressive semantic capabilities, they often suffer from spatial blindness, struggling with fine-grained geometric reasoning and physical dynamics. Existing solutions typically rely on explicit 3D modalities or complex geometric scaffolding, which are limited by data scarcity and generalization challenges. In this work, we propose a paradigm shift by leveraging the implicit spatial prior within large-scale video generation models. We posit that to synthesize temporally coherent videos, these models inherently learn robust 3D structural priors and physical laws. We introduce VEGA-3D (Video Extracted Generative Awareness), a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator. By extracting spatiotemporal features from intermediate noise levels and integrating them with semantic representations via a token-level adaptive gated fusion mechanism, we enrich MLLMs with dense geometric cues without explicit 3D supervision. Extensive experiments across 3D scene understanding, spatial reasoning, and embodied manipulation benchmarks demonstrate that our method outperforms state-of-the-art baselines, validating that generative priors provide a scalable foundation for physical-world understanding. Code is publicly available at this https URL.
• 摘要：虽然多模态大语言模型表现出令人印象深刻的语义能力，但它们经常遭受空间盲症，难以应对细粒度的几何推理和物理动力学。现有的解决方案通常依赖于显式 3D 模式或复杂的几何脚手架，而这受到数据稀缺和泛化挑战的限制。在这项工作中，我们通过利用大规模视频生成模型中的隐式空间先验提出了范式转变。我们假设，为了合成时间相干的视频，这些模型本质上学习了强大的 3D 结构先验和物理定律。我们引入了 VEGA-3D（视频提取生成意识），这是一种即插即用框架，可将预先训练的视频扩散模型重新用作潜在世界模拟器。通过从中间噪声水平中提取时空特征，并通过令牌级自适应门控融合机制将它们与语义表示集成，我们在没有显式 3D 监督的情况下用密集的几何线索丰富了 MLLM。 3D 场景理解、空间推理和具体操作基准的广泛实验表明，我们的方法优于最先进的基线，验证生成先验为物理世界理解提供了可扩展的基础。代码可通过此 https URL 公开获取。