2025-12-15

Title: MolSculpt: Sculpting 3D Molecular Geometries from Chemical Syntax

• Authors: Zhanpeng Chen, Weihao Gao, Shunyu Wang, Yanan Zhu, Hong Meng, Yuexian Zou
• Subjects: cs.LG, cs.AI, physics.chem-ph, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2512.10991
• Pdf URL: https://arxiv.org/pdf/2512.10991
• Copy Paste: [[2512.10991]] MolSculpt: Sculpting 3D Molecular Geometries from Chemical Syntax(https://arxiv.org/abs/2512.10991)
• Keywords: generation
• Abstract: Generating precise 3D molecular geometries is crucial for drug discovery and material science. While prior efforts leverage 1D representations like SELFIES to ensure molecular validity, they fail to fully exploit the rich chemical knowledge entangled within 1D models, leading to a disconnect between 1D syntactic generation and 3D geometric realization. To bridge this gap, we propose MolSculpt, a novel framework that "sculpts" 3D molecular geometries from chemical syntax. MolSculpt is built upon a frozen 1D molecular foundation model and a 3D molecular diffusion model. We introduce a set of learnable queries to extract inherent chemical knowledge from the foundation model, and a trainable projector then injects this cross-modal information into the conditioning space of the diffusion model to guide the 3D geometry generation. In this way, our model deeply integrates 1D latent chemical knowledge into the 3D generation process through end-to-end optimization. Experiments demonstrate that MolSculpt achieves state-of-the-art (SOTA) performance in \textit{de novo} 3D molecule generation and conditional 3D molecule generation, showing superior 3D fidelity and stability on both the GEOM-DRUGS and QM9 datasets. Code is available at this https URL.
• 摘要：生成精确的 3D 分子几何形状对于药物发现和材料科学至关重要。虽然之前的努力利用 SELFIES 等 1D 表示来确保分子有效性，但它们未能充分利用 1D 模型中丰富的化学知识，导致 1D 句法生成和 3D 几何实现之间脱节。为了弥补这一差距，我们提出了 MolSculpt，这是一种新颖的框架，可以根据化学语法“雕刻”3D 分子几何形状。 MolSculpt 建立在冷冻的 1D 分子基础模型和 3D 分子扩散模型之上。我们引入了一组可学习的查询来从基础模型中提取固有的化学知识，然后可训练的投影仪将这种跨模态信息注入扩散模型的条件空间中以指导 3D 几何生成。这样，我们的模型通过端到端优化将 1D 潜在化学知识深度集成到 3D 生成过程中。实验表明，MolSculpt 在 \textit{de novo} 3D 分子生成和条件 3D 分子生成方面实现了最先进的 (SOTA) 性能，在 GEOM-DRUGS 和 QM9 数据集上显示出卓越的 3D 保真度和稳定性。代码可从此 https URL 获取。

Title: VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation

• Authors: Felix O'Mahony, Roberto Cipolla, Ayush Tewari
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11061
• Pdf URL: https://arxiv.org/pdf/2512.11061
• Copy Paste: [[2512.11061]] VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation(https://arxiv.org/abs/2512.11061)
• Keywords: generative
• Abstract: Generative video models, a leading approach to world modeling, face fundamental limitations. They often violate physical and logical rules, lack interactivity, and operate as opaque black boxes ill-suited for building structured, queryable worlds. To overcome these challenges, we propose a new paradigm focused on distilling an image caption pair into a tractable, abstract representation optimized for simulation. We introduce VDAWorld, a framework where a Vision-Language Model (VLM) acts as an intelligent agent to orchestrate this process. The VLM autonomously constructs a grounded (2D or 3D) scene representation by selecting from a suite of vision tools, and accordingly chooses a compatible physics simulator (e.g., rigid body, fluid) to act upon it. VDAWorld can then infer latent dynamics from the static scene to predict plausible future states. Our experiments show that this combination of intelligent abstraction and adaptive simulation results in a versatile world model capable of producing high quality simulations across a wide range of dynamic scenarios.
• 摘要：生成视频模型是一种领先的世界建模方法，但面临着根本性的限制。它们经常违反物理和逻辑规则，缺乏交互性，并且作为不透明的黑匣子运行，不适合构建结构化的、可查询的世界。为了克服这些挑战，我们提出了一种新的范式，重点是将图像标题对提炼成易于处理的抽象表示，并针对模拟进行了优化。我们引入了 VDAWorld，这是一个框架，其中视觉语言模型 (VLM) 充当智能代理来协调此过程。 VLM 通过从一套视觉工具中进行选择，自主构建接地（2D 或 3D）场景表示，并相应地选择兼容的物理模拟器（例如，刚体、流体）对其进行操作。然后，VDAWorld 可以从静态场景推断潜在动态，以预测可能的未来状态。我们的实验表明，智能抽象和自适应模拟的结合产生了一个多功能的世界模型，能够在各种动态场景中生成高质量的模拟。

Title: Investigating ECG Diagnosis with Ambiguous Labels using Partial Label Learning

• Authors: Sana Rahmani, Javad Hashemi, Ali Etemad
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11095
• Pdf URL: https://arxiv.org/pdf/2512.11095
• Copy Paste: [[2512.11095]] Investigating ECG Diagnosis with Ambiguous Labels using Partial Label Learning(https://arxiv.org/abs/2512.11095)
• Keywords: generation
• Abstract: Label ambiguity is an inherent problem in real-world electrocardiogram (ECG) diagnosis, arising from overlapping conditions and diagnostic disagreement. However, current ECG models are trained under the assumption of clean and non-ambiguous annotations, which limits both the development and the meaningful evaluation of models under real-world conditions. Although Partial Label Learning (PLL) frameworks are designed to learn from ambiguous labels, their effectiveness in medical time-series domains, ECG in particular, remains largely unexplored. In this work, we present the first systematic study of PLL methods for ECG diagnosis. We adapt nine PLL algorithms to multi-label ECG diagnosis and evaluate them using a diverse set of clinically motivated ambiguity generation strategies, capturing both unstructured (e.g., random) and structured ambiguities (e.g., cardiologist-derived similarities, treatment relationships, and diagnostic taxonomies). Our experiments on the PTB-XL and Chapman datasets demonstrate that PLL methods vary substantially in their robustness to different types and degrees of ambiguity. Through extensive analysis, we identify key limitations of current PLL approaches in clinical settings and outline future directions for developing robust and clinically aligned ambiguity-aware learning frameworks for ECG diagnosis.
• 摘要：标签模糊性是现实世界心电图 (ECG) 诊断中的一个固有问题，由重叠条件和诊断分歧引起。然而，当前的心电图模型是在干净且明确的注释假设下进行训练的，这限制了模型在现实条件下的开发和有意义的评估。尽管部分标签学习（PLL）框架旨在从不明确的标签中学习，但它们在医学时间序列领域（尤其是心电图）中的有效性在很大程度上仍未得到探索。在这项工作中，我们首次系统地研究了用于心电图诊断的 PLL 方法。我们采用九种 PLL 算法来进行多标签心电图诊断，并使用多种临床动机的模糊性生成策略对其进行评估，捕获非结构化（例如随机）和结构化模糊性（例如心脏病专家得出的相似性、治疗关系和诊断分类法）。我们在 PTB-XL 和 Chapman 数据集上的实验表明，PLL 方法对于不同类型和程度的模糊性的鲁棒性差异很大。通过广泛的分析，我们确定了当前 PLL 方法在临床环境中的主要局限性，并概述了为心电图诊断开发强大且临床一致的模糊性感知学习框架的未来方向。

Title: Learning from a Generative Oracle: Domain Adaptation for Restoration

• Authors: Yuyang Hu, Mojtaba Sahraee-Ardakan, Arpit Bansal, Kangfu Mei, Christian Qi, Peyman Milanfar, Mauricio Delbracio
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2512.11121
• Pdf URL: https://arxiv.org/pdf/2512.11121
• Copy Paste: [[2512.11121]] Learning from a Generative Oracle: Domain Adaptation for Restoration(https://arxiv.org/abs/2512.11121)
• Keywords: restoration, generative
• Abstract: Pre-trained image restoration models often fail on real-world, out-of-distribution degradations due to significant domain gaps. Adapting to these unseen domains is challenging, as out-of-distribution data lacks ground truth, and traditional adaptation methods often require complex architectural changes. We propose LEGO (Learning from a Generative Oracle), a practical three-stage framework for post-training domain adaptation without paired data. LEGO converts this unsupervised challenge into a tractable pseudo-supervised one. First, we obtain initial restorations from the pre-trained model. Second, we leverage a frozen, large-scale generative oracle to refine these estimates into high-quality pseudo-ground-truths. Third, we fine-tune the original model using a mixed-supervision strategy combining in-distribution data with these new pseudo-pairs. This approach adapts the model to the new distribution without sacrificing its original robustness or requiring architectural modifications. Experiments demonstrate that LEGO effectively bridges the domain gap, significantly improving performance on diverse real-world benchmarks.
• 摘要：由于显着的域间隙，预训练的图像恢复模型通常会在现实世界的分布外退化方面失败。适应这些看不见的领域具有挑战性，因为分布外数据缺乏基本事实，而传统的适应方法通常需要复杂的架构更改。我们提出了 LEGO（从生成预言机学习），这是一种实用的三阶段框架，用于无需配对数据的训练后领域适应。乐高将这种无监督的挑战转化为易于处理的伪监督挑战。首先，我们从预训练模型中获得初始恢复。其次，我们利用冻结的大规模生成预言机将这些估计细化为高质量的伪地面事实。第三，我们使用混合监督策略将分布数据与这些新的伪对相结合来微调原始模型。这种方法使模型适应新的分布，而不牺牲其原始的稳健性或需要架构修改。实验表明，乐高有效地弥合了领域差距，显着提高了各种现实世界基准的性能。

Title: Lightweight 3D Gaussian Splatting Compression via Video Codec

• Authors: Qi Yang, Geert Van Der Auwera, Zhu Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11186
• Pdf URL: https://arxiv.org/pdf/2512.11186
• Copy Paste: [[2512.11186]] Lightweight 3D Gaussian Splatting Compression via Video Codec(https://arxiv.org/abs/2512.11186)
• Keywords: generation
• Abstract: Current video-based GS compression methods rely on using Parallel Linear Assignment Sorting (PLAS) to convert 3D GS into smooth 2D maps, which are computationally expensive and time-consuming, limiting the application of GS on lightweight devices. In this paper, we propose a Lightweight 3D Gaussian Splatting (GS) Compression method based on Video codec (LGSCV). First, a two-stage Morton scan is proposed to generate blockwise 2D maps that are friendly for canonical video codecs in which the coding units (CU) are square blocks. A 3D Morton scan is used to permute GS primitives, followed by a 2D Morton scan to map the ordered GS primitives to 2D maps in a blockwise style. However, although the blockwise 2D maps report close performance to the PLAS map in high-bitrate regions, they show a quality collapse at medium-to-low bitrates. Therefore, a principal component analysis (PCA) is used to reduce the dimensionality of spherical harmonics (SH), and a MiniPLAS, which is flexible and fast, is designed to permute the primitives within certain block sizes. Incorporating SH PCA and MiniPLAS leads to a significant gain in rate-distortion (RD) performance, especially at medium and low bitrates. MiniPLAS can also guide the setting of the codec CU size configuration and significantly reduce encoding time. Experimental results on the MPEG dataset demonstrate that the proposed LGSCV achieves over 20% RD gain compared with state-of-the-art methods, while reducing 2D map generation time to approximately 1 second and cutting encoding time by 50%. The code is available at this https URL .
• 摘要：当前基于视频的GS压缩方法依赖于使用并行线性分配排序（PLAS）将3D GS转换为平滑的2D地图，这在计算上昂贵且耗时，限制了GS在轻量级设备上的应用。在本文中，我们提出了一种基于视频编解码器（LGSCV）的轻量级3D高斯分布（GS）压缩方法。首先，提出了两阶段 Morton 扫描来生成分块 2D 映射，该映射对于编码单元 (CU) 为方形块的规范视频编解码器是友好的。 3D Morton 扫描用于排列 GS 图元，然后使用 2D Morton 扫描以块方式将有序 GS 图元映射到 2D 地图。然而，尽管块状 2D 地图在高比特率区域报告的性能与 PLAS 地图接近，但它们在中低比特率下表现出质量崩溃。因此，使用主成分分析（PCA）来降低球谐函数（SH）的维数，并且灵活且快速的MiniPLAS被设计用于在一定的块大小内排列图元。 SH PCA 和 MiniPLAS 的结合可显着提高速率失真 (RD) 性能，尤其是在中低比特率下。 MiniPLAS还可以指导编解码器CU大小配置的设置并显着减少编码时间。 MPEG 数据集上的实验结果表明，与最先进的方法相比，所提出的 LGSCV 实现了超过 20% 的 RD 增益，同时将 2D 地图生成时间减少到大约 1 秒，并将编码时间减少 50%。该代码可在此 https URL 获取。

Title: Beyond Memorization: Gradient Projection Enables Selective Learning in Diffusion Models

• Authors: Divya Kothandaraman, Jaclyn Pytlarz
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11194
• Pdf URL: https://arxiv.org/pdf/2512.11194
• Copy Paste: [[2512.11194]] Beyond Memorization: Gradient Projection Enables Selective Learning in Diffusion Models(https://arxiv.org/abs/2512.11194)
• Keywords: generation, generative
• Abstract: Memorization in large-scale text-to-image diffusion models poses significant security and intellectual property risks, enabling adversarial attribute extraction and the unauthorized reproduction of sensitive or proprietary features. While conventional dememorization techniques, such as regularization and data filtering, limit overfitting to specific training examples, they fail to systematically prevent the internalization of prohibited concept-level features. Simply discarding all images containing a sensitive feature wastes invaluable training data, necessitating a method for selective unlearning at the concept level. To address this, we introduce a Gradient Projection Framework designed to enforce a stringent requirement of concept-level feature exclusion. Our defense operates during backpropagation by systematically identifying and excising training signals aligned with embeddings of prohibited attributes. Specifically, we project each gradient update onto the orthogonal complement of the sensitive feature's embedding space, thereby zeroing out its influence on the model's weights. Our method integrates seamlessly into standard diffusion model training pipelines and complements existing defenses. We analyze our method against an adversary aiming for feature extraction. In extensive experiments, we demonstrate that our framework drastically reduces memorization while rigorously preserving generation quality and semantic fidelity. By reframing memorization control as selective learning, our approach establishes a new paradigm for IP-safe and privacy-preserving generative AI.
• 摘要：大规模文本到图像扩散模型中的记忆带来了重大的安全和知识产权风险，使得对抗性属性提取和敏感或专有特征的未经授权的复制成为可能。虽然传统的去记忆技术（例如正则化和数据过滤）限制了对特定训练示例的过度拟合，但它们无法系统地防止禁止的概念级特征的内化。简单地丢弃包含敏感特征的所有图像会浪费宝贵的训练数据，因此需要一种在概念级别选择性遗忘的方法。为了解决这个问题，我们引入了一个梯度投影框架，旨在强制执行概念级特征排除的严格要求。我们的防御在反向传播过程中通过系统地识别和删除与禁止属性的嵌入一致的训练信号来进行。具体来说，我们将每个梯度更新投影到敏感特征嵌入空间的正交补集上，从而将其对模型权重的影响归零。我们的方法无缝集成到标准扩散模型训练流程中，并补充了现有的防御措施。我们针对旨在提取特征的对手来分析我们的方法。在大量的实验中，我们证明我们的框架大大减少了记忆，同时严格保持生成质量和语义保真度。通过将记忆控制重新定义为选择性学习，我们的方法为 IP 安全和隐私保护的生成人工智能建立了新的范例。

Title: CADKnitter: Compositional CAD Generation from Text and Geometry Guidance

• Authors: Tri Le, Khang Nguyen, Baoru Huang, Tung D. Ta, Anh Nguyen
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11199
• Pdf URL: https://arxiv.org/pdf/2512.11199
• Copy Paste: [[2512.11199]] CADKnitter: Compositional CAD Generation from Text and Geometry Guidance(https://arxiv.org/abs/2512.11199)
• Keywords: generation
• Abstract: Crafting computer-aided design (CAD) models has long been a painstaking and time-intensive task, demanding both precision and expertise from designers. With the emergence of 3D generation, this task has undergone a transformative impact, shifting not only from visual fidelity to functional utility but also enabling editable CAD designs. Prior works have achieved early success in single-part CAD generation, which is not well-suited for real-world applications, as multiple parts need to be assembled under semantic and geometric constraints. In this paper, we propose CADKnitter, a compositional CAD generation framework with a geometry-guided diffusion sampling strategy. CADKnitter is able to generate a complementary CAD part that follows both the geometric constraints of the given CAD model and the semantic constraints of the desired design text prompt. We also curate a dataset, so-called KnitCAD, containing over 310,000 samples of CAD models, along with textual prompts and assembly metadata that provide semantic and geometric constraints. Intensive experiments demonstrate that our proposed method outperforms other state-of-the-art baselines by a clear margin.
• 摘要：长期以来，制作计算机辅助设计 (CAD) 模型一直是一项艰巨且耗时的任务，需要设计师的精度和专业知识。随着 3D 生成的出现，这项任务经历了变革性的影响，不仅从视觉保真度转变为功能实用性，而且还支持可编辑的 CAD 设计。先前的工作在单零件 CAD 生成方面取得了早期成功，但它不太适合实际应用，因为需要在语义和几何约束下组装多个零件。在本文中，我们提出了 CADKnitter，一种具有几何引导扩散采样策略的组合 CAD 生成框架。 CADKnitter 能够生成补充的 CAD 零件，该零件遵循给定 CAD 模型的几何约束和所需设计文本提示的语义约束。我们还整理了一个数据集，即所谓的 KnitCAD，其中包含超过 310,000 个 CAD 模型样本，以及提供语义和几何约束的文本提示和装配元数据。密集的实验表明，我们提出的方法明显优于其他最先进的基线。

Title: Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery: Sublinear Memory Growth for Efficient LLM Inference

• Authors: Adilet Metinov, Gulida M. Kudakeeva, Bolotbek uulu Nursultan, Gulnara D. Kabaeva
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2512.11221
• Pdf URL: https://arxiv.org/pdf/2512.11221
• Copy Paste: [[2512.11221]] Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery: Sublinear Memory Growth for Efficient LLM Inference(https://arxiv.org/abs/2512.11221)
• Keywords: generation
• Abstract: We present Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery (ASR-KF-EGR), a training-free inference-time framework for efficient large language model generation. Our method introduces a reversible soft-freeze mechanism that temporarily suspends key-value (KV) updates for low-importance tokens identified within a sliding attention window. Unlike eviction-based approaches that permanently discard context, ASR-KF-EGR preserves all tokens in off-GPU storage and restores them on demand. We extend the framework with sublinear freeze scheduling, where freeze duration grows sublinearly with repeated low-importance detections, preventing over-aggressive compression. Preliminary experiments on LLaMA-3 8B demonstrate 55-67% reduction in active KV cache size while maintaining generation quality and passing needle-in-haystack retrieval tests. The method is architecture-agnostic, requires no fine-tuning, and provides a practical solution for memory-constrained deployment of long-context LLMs.
• 摘要：我们提出了具有熵引导恢复功能的自适应软滚动 KV 冻结 (ASR-KF-EGR)，这是一种用于高效生成大型语言模型的免训练推理时间框架。我们的方法引入了一种可逆的软冻结机制，该机制可以暂时暂停在滑动注意窗口内识别的低重要性令牌的键值（KV）更新。与永久丢弃上下文的基于驱逐的方法不同，ASR-KF-EGR 将所有令牌保留在 GPU 外存储中，并根据需要恢复它们。我们通过亚线性冻结调度扩展了框架，其中冻结持续时间随着重复的低重要性检测而亚线性增长，从而防止过度压缩。 LLaMA-3 8B 的初步实验表明，活动 KV 缓存大小减少了 55-67%，同时保持生成质量并通过大海捞针检索测试。该方法与架构无关，不需要微调，并为长上下文 LLM 的内存受限部署提供了实用的解决方案。

Title: VFMF: World Modeling by Forecasting Vision Foundation Model Features

• Authors: Gabrijel Boduljak, Yushi Lan, Christian Rupprecht, Andrea Vedaldi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11225
• Pdf URL: https://arxiv.org/pdf/2512.11225
• Copy Paste: [[2512.11225]] VFMF: World Modeling by Forecasting Vision Foundation Model Features(https://arxiv.org/abs/2512.11225)
• Keywords: generation, generative
• Abstract: Forecasting from partial observations is central to world modeling. Many recent methods represent the world through images, and reduce forecasting to stochastic video generation. Although such methods excel at realism and visual fidelity, predicting pixels is computationally intensive and not directly useful in many applications, as it requires translating RGB into signals useful for decision making. An alternative approach uses features from vision foundation models (VFMs) as world representations, performing deterministic regression to predict future world states. These features can be directly translated into actionable signals such as semantic segmentation and depth, while remaining computationally efficient. However, deterministic regression averages over multiple plausible futures, undermining forecast accuracy by failing to capture uncertainty. To address this crucial limitation, we introduce a generative forecaster that performs autoregressive flow matching in VFM feature space. Our key insight is that generative modeling in this space requires encoding VFM features into a compact latent space suitable for diffusion. We show that this latent space preserves information more effectively than previously used PCA-based alternatives, both for forecasting and other applications, such as image generation. Our latent predictions can be easily decoded into multiple useful and interpretable output modalities: semantic segmentation, depth, surface normals, and even RGB. With matched architecture and compute, our method produces sharper and more accurate predictions than regression across all modalities. Our results suggest that stochastic conditional generation of VFM features offers a promising and scalable foundation for future world models.
• 摘要：根据部分观测进行预测是世界建模的核心。最近的许多方法通过图像来表示世界，并将预测减少到随机视频生成。尽管此类方法在真实感和视觉保真度方面表现出色，但预测像素的计算量很大，并且在许多应用中并不直接有用，因为它需要将 RGB 转换为对决策有用的信号。另一种方法使用视觉基础模型 (VFM) 的特征作为世界表示，执行确定性回归来预测未来的世界状态。这些特征可以直接转化为可操作的信号，例如语义分割和深度，同时保持计算效率。然而，确定性回归对多个可能的未来进行平均，由于未能捕获不确定性而破坏了预测的准确性。为了解决这个关键的限制，我们引入了一个生成预测器，它在 VFM 特征空间中执行自回归流匹配。我们的主要见解是，该空间中的生成建模需要将 VFM 特征编码到适合扩散的紧凑潜在空间中。我们证明，这种潜在空间比以前使用的基于 PCA 的替代方案更有效地保存信息，无论是用于预测还是其他应用，例如图像生成。我们的潜在预测可以轻松解码为多种有用且可解释的输出模式：语义分割、深度、表面法线，甚至 RGB。通过匹配的架构和计算，我们的方法可以比所有模态的回归产生更清晰、更准确的预测。我们的结果表明，VFM 特征的随机条件生成为未来世界模型提供了有前途且可扩展的基础。

Title: REST: Diffusion-based Real-time End-to-end Streaming Talking Head Generation via ID-Context Caching and Asynchronous Streaming Distillation

• Authors: Haotian Wang, Yuzhe Weng, Xinyi Yu, Jun Du, Haoran Xu, Xiaoyan Wu, Shan He, Bing Yin, Cong Liu, Qingfeng Liu
• Subjects: cs.CV, cs.SD
• Abstract URL: https://arxiv.org/abs/2512.11229
• Pdf URL: https://arxiv.org/pdf/2512.11229
• Copy Paste: [[2512.11229]] REST: Diffusion-based Real-time End-to-end Streaming Talking Head Generation via ID-Context Caching and Asynchronous Streaming Distillation(https://arxiv.org/abs/2512.11229)
• Keywords: generation
• Abstract: Diffusion models have significantly advanced the field of talking head generation. However, the slow inference speeds and non-autoregressive paradigms severely constrain the application of diffusion-based THG models. In this study, we propose REST, the first diffusion-based, real-time, end-to-end streaming audio-driven talking head generation framework. To support real-time end-to-end generation, a compact video latent space is first learned through high spatiotemporal VAE compression. Additionally, to enable autoregressive streaming within the compact video latent space, we introduce an ID-Context Cache mechanism, which integrates ID-Sink and Context-Cache principles to key-value caching for maintaining temporal consistency and identity coherence during long-time streaming generation. Furthermore, an Asynchronous Streaming Distillation (ASD) training strategy is proposed to mitigate error accumulation in autoregressive generation and enhance temporal consistency, which leverages a non-streaming teacher with an asynchronous noise schedule to supervise the training of the streaming student model. REST bridges the gap between autoregressive and diffusion-based approaches, demonstrating substantial value for applications requiring real-time talking head generation. Experimental results demonstrate that REST outperforms state-of-the-art methods in both generation speed and overall performance.
• 摘要：扩散模型极大地推进了头部特写生成领域的发展。然而，缓慢的推理速度和非自回归范式严重限制了基于扩散的 THG 模型的应用。在这项研究中，我们提出了 REST，这是第一个基于扩散的、实时的、端到端的流音频驱动的头部说话生成框架。为了支持实时端到端生成，首先通过高时空 VAE 压缩来学习紧凑的视频潜在空间。此外，为了在紧凑的视频潜在空间内实现自回归流，我们引入了 ID-Context Cache 机制，它将 ID-Sink 和 Context-Cache 原理集成到键值缓存中，以在长时间流生成期间保持时间一致性和身份一致性。此外，提出了一种异步流蒸馏（ASD）训练策略，以减轻自回归生成中的错误积累并增强时间一致性，该策略利用具有异步噪声计划的非流教师来监督流学生模型的训练。 REST 弥合了自回归和基于扩散的方法之间的差距，为需要实时头部说话生成的应用程序展示了巨大的价值。实验结果表明，REST 在生成速度和整体性能方面均优于最先进的方法。

Title: RoomPilot: Controllable Synthesis of Interactive Indoor Environments via Multimodal Semantic Parsing

• Authors: Wentang Chen, Shougao Zhang, Yiman Zhang, Tianhao Zhou, Ruihui Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11234
• Pdf URL: https://arxiv.org/pdf/2512.11234
• Copy Paste: [[2512.11234]] RoomPilot: Controllable Synthesis of Interactive Indoor Environments via Multimodal Semantic Parsing(https://arxiv.org/abs/2512.11234)
• Keywords: generation
• Abstract: Generating controllable and interactive indoor scenes is fundamental to applications in game development, architectural visualization, and embodied AI training. Yet existing approaches either handle a narrow range of input modalities or rely on stochastic processes that hinder controllability. To overcome these limitations, we introduce RoomPilot, a unified framework that parses diverse multi-modal inputs--textual descriptions or CAD floor plans--into an Indoor Domain-Specific Language (IDSL) for indoor structured scene generation. The key insight is that a well-designed IDSL can act as a shared semantic representation, enabling coherent, high-quality scene synthesis from any single modality while maintaining interaction semantics. In contrast to conventional procedural methods that produce visually plausible but functionally inert layouts, RoomPilot leverages a curated dataset of interaction-annotated assets to synthesize environments exhibiting realistic object behaviors. Extensive experiments further validate its strong multi-modal understanding, fine-grained controllability in scene generation, and superior physical consistency and visual fidelity, marking a significant step toward general-purpose controllable 3D indoor scene generation.
• 摘要：生成可控和交互式的室内场景是游戏开发、建筑可视化和具体人工智能训练应用的基础。然而，现有的方法要么处理范围狭窄的输入模式，要么依赖阻碍可控性的随机过程。为了克服这些限制，我们引入了 RoomPilot，这是一个统一的框架，可将不同的多模式输入（文本描述或 CAD 平面图）解析为室内特定领域语言 (IDSL)，以生成室内结构化场景。关键的见解是，精心设计的 IDSL 可以充当共享语义表示，从而能够从任何单一模态进行连贯、高质量的场景合成，同时保持交互语义。与产生视觉上合理但功能上惰性布局的传统程序方法相比，RoomPilot 利用交互注释资产的精选数据集来合成表现出真实对象行为的环境。大量实验进一步验证了其强大的多模态理解能力、场景生成的细粒度可控性以及卓越的物理一致性和视觉保真度，标志着向通用可控3D室内场景生成迈出了重要一步。

Title: Cross-modal Prompting for Balanced Incomplete Multi-modal Emotion Recognition

• Authors: Wen-Jue He, Xiaofeng Zhu, Zheng Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11239
• Pdf URL: https://arxiv.org/pdf/2512.11239
• Copy Paste: [[2512.11239]] Cross-modal Prompting for Balanced Incomplete Multi-modal Emotion Recognition(https://arxiv.org/abs/2512.11239)
• Keywords: generation
• Abstract: Incomplete multi-modal emotion recognition (IMER) aims at understanding human intentions and sentiments by comprehensively exploring the partially observed multi-source data. Although the multi-modal data is expected to provide more abundant information, the performance gap and modality under-optimization problem hinder effective multi-modal learning in practice, and are exacerbated in the confrontation of the missing data. To address this issue, we devise a novel Cross-modal Prompting (ComP) method, which emphasizes coherent information by enhancing modality-specific features and improves the overall recognition accuracy by boosting each modality's performance. Specifically, a progressive prompt generation module with a dynamic gradient modulator is proposed to produce concise and consistent modality semantic cues. Meanwhile, cross-modal knowledge propagation selectively amplifies the consistent information in modality features with the delivered prompts to enhance the discrimination of the modality-specific output. Additionally, a coordinator is designed to dynamically re-weight the modality outputs as a complement to the balance strategy to improve the model's efficacy. Extensive experiments on 4 datasets with 7 SOTA methods under different missing rates validate the effectiveness of our proposed method.
• 摘要：不完全多模态情感识别（IMER）旨在通过综合探索部分观察到的多源数据来理解人类的意图和情感。尽管多模态数据有望提供更丰富的信息，但性能差距和模态欠优化问题阻碍了实践中有效的多模态学习，并且在缺失数据的对抗中加剧。为了解决这个问题，我们设计了一种新颖的跨模态提示（ComP）方法，该方法通过增强模态特定特征来强调连贯信息，并通过提高每种模态的性能来提高整体识别精度。具体来说，提出了一种具有动态梯度调制器的渐进提示生成模块，以产生简洁且一致的模态语义提示。同时，跨模态知识传播有选择地放大模态特征中与所传递的提示一致的信息，以增强模态特定输出的辨别力。此外，协调器旨在动态地重新加权模态输出，作为平衡策略的补充，以提高模型的效率。在不同缺失率下使用 7 种 SOTA 方法在 4 个数据集上进行的广泛实验验证了我们提出的方法的有效性。

Title: PersonaLive! Expressive Portrait Image Animation for Live Streaming

• Authors: Zhiyuan Li, Chi-Man Pun, Chen Fang, Jue Wang, Xiaodong Cun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11253
• Pdf URL: https://arxiv.org/pdf/2512.11253
• Copy Paste: [[2512.11253]] PersonaLive! Expressive Portrait Image Animation for Live Streaming(https://arxiv.org/abs/2512.11253)
• Keywords: generation
• Abstract: Current diffusion-based portrait animation models predominantly focus on enhancing visual quality and expression realism, while overlooking generation latency and real-time performance, which restricts their application range in the live streaming scenario. We propose PersonaLive, a novel diffusion-based framework towards streaming real-time portrait animation with multi-stage training recipes. Specifically, we first adopt hybrid implicit signals, namely implicit facial representations and 3D implicit keypoints, to achieve expressive image-level motion control. Then, a fewer-step appearance distillation strategy is proposed to eliminate appearance redundancy in the denoising process, greatly improving inference efficiency. Finally, we introduce an autoregressive micro-chunk streaming generation paradigm equipped with a sliding training strategy and a historical keyframe mechanism to enable low-latency and stable long-term video generation. Extensive experiments demonstrate that PersonaLive achieves state-of-the-art performance with up to 7-22x speedup over prior diffusion-based portrait animation models.
• 摘要：目前基于扩散的人像动画模型主要注重增强视觉质量和表达真实感，而忽视了生成延迟和实时性能，这限制了其在直播场景中的应用范围。我们提出了 PersonaLive，一种新颖的基于扩散的框架，用于通过多阶段训练配方来流式传输实时肖像动画。具体来说，我们首先采用混合隐式信号，即隐式面部表示和 3D 隐式关键点，来实现富有表现力的图像级运动控制。然后，提出了少步外观蒸馏策略来消除去噪过程中的外观冗余，大大提高了推理效率。最后，我们引入了一种自回归微块流生成范例，配备滑动训练策略和历史关键帧机制，以实现低延迟和稳定的长期视频生成。大量实验表明，PersonaLive 实现了最先进的性能，与之前基于扩散的肖像动画模型相比，速度提高了 7-22 倍。

Title: FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion

• Authors: Xiangyang Luo, Qingyu Li, Xiaokun Liu, Wenyu Qin, Miao Yang, Meng Wang, Pengfei Wan, Di Zhang, Kun Gai, Shao-Lun Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11274
• Pdf URL: https://arxiv.org/pdf/2512.11274
• Copy Paste: [[2512.11274]] FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion(https://arxiv.org/abs/2512.11274)
• Keywords: generation
• Abstract: Current video generation models perform well at single-shot synthesis but struggle with multi-shot videos, facing critical challenges in maintaining character and background consistency across shots and flexibly generating videos of arbitrary length and shot count. To address these limitations, we introduce \textbf{FilmWeaver}, a novel framework designed to generate consistent, multi-shot videos of arbitrary length. First, it employs an autoregressive diffusion paradigm to achieve arbitrary-length video generation. To address the challenge of consistency, our key insight is to decouple the problem into inter-shot consistency and intra-shot coherence. We achieve this through a dual-level cache mechanism: a shot memory caches keyframes from preceding shots to maintain character and scene identity, while a temporal memory retains a history of frames from the current shot to ensure smooth, continuous motion. The proposed framework allows for flexible, multi-round user interaction to create multi-shot videos. Furthermore, due to this decoupled design, our method demonstrates high versatility by supporting downstream tasks such as multi-concept injection and video extension. To facilitate the training of our consistency-aware method, we also developed a comprehensive pipeline to construct a high-quality multi-shot video dataset. Extensive experimental results demonstrate that our method surpasses existing approaches on metrics for both consistency and aesthetic quality, opening up new possibilities for creating more consistent, controllable, and narrative-driven video content. Project Page: this https URL
• 摘要：当前的视频生成模型在单镜头合成方面表现良好，但在多镜头视频方面表现不佳，在保持镜头之间的角色和背景一致性以及灵活生成任意长度和镜头数量的视频方面面临着严峻的挑战。为了解决这些限制，我们引入了 \textbf{FilmWeaver}，这是一种新颖的框架，旨在生成任意长度的一致的多镜头视频。首先，它采用自回归扩散范式来实现任意长度的视频生成。为了解决一致性的挑战，我们的主要见解是将问题分解为镜头间一致性和镜头内一致性。我们通过双层缓存机制来实现这一点：镜头内存缓存先前镜头中的关键帧，以保持角色和场景的身份，而时间内存则保留当前镜头中的帧的历史记录，以确保平滑、连续的运动。所提出的框架允许灵活的多轮用户交互来创建多镜头视频。此外，由于这种解耦设计，我们的方法通过支持多概念注入和视频扩展等下游任务表现出高度的多功能性。为了促进一致性意识方法的训练，我们还开发了一个全面的管道来构建高质量的多镜头视频数据集。大量的实验结果表明，我们的方法在一致性和审美质量方面都超越了现有的指标方法，为创建更加一致、可控和叙事驱动的视频内容开辟了新的可能性。项目页面：此 https URL

Title: Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context

• Authors: Cuifeng Shen, Lumin Xu, Xingguo Zhu, Gengdai Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11293
• Pdf URL: https://arxiv.org/pdf/2512.11293
• Copy Paste: [[2512.11293]] Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context(https://arxiv.org/abs/2512.11293)
• Keywords: generation
• Abstract: Video autoencoders compress videos into compact latent representations for efficient reconstruction, playing a vital role in enhancing the quality and efficiency of video generation. However, existing video autoencoders often entangle spatial and temporal information, limiting their ability to capture temporal consistency and leading to suboptimal performance. To address this, we propose Autoregressive Video Autoencoder (ARVAE), which compresses and reconstructs each frame conditioned on its predecessor in an autoregressive manner, allowing flexible processing of videos with arbitrary lengths. ARVAE introduces a temporal-spatial decoupled representation that combines downsampled flow field for temporal coherence with spatial relative compensation for newly emerged content, achieving high compression efficiency without information loss. Specifically, the encoder compresses the current and previous frames into the temporal motion and spatial supplement, while the decoder reconstructs the original frame from the latent representations given the preceding frame. A multi-stage training strategy is employed to progressively optimize the model. Extensive experiments demonstrate that ARVAE achieves superior reconstruction quality with extremely lightweight models and small-scale training data. Moreover, evaluations on video generation tasks highlight its strong potential for downstream applications.
• 摘要：视频自动编码器将视频压缩为紧凑的潜在表示以进行高效重建，在提高视频生成的质量和效率方面发挥着至关重要的作用。然而，现有的视频自动编码器经常纠缠空间和时间信息，限制了它们捕获时间一致性的能力并导致性能不佳。为了解决这个问题，我们提出了自回归视频自动编码器（ARVAE），它以自回归方式压缩和重建以其前身为条件的每个帧，从而允许灵活处理任意长度的视频。 ARVAE 引入了一种时空解耦表示，它将用于时间相干性的下采样流场与新出现的内容的空间相对补偿相结合，实现了高压缩效率而不丢失信息。具体来说，编码器将当前帧和先前帧压缩为时间运动和空间补充，而解码器根据给定先前帧的潜在表示重建原始帧。采用多阶段训练策略来逐步优化模型。大量实验表明，ARVAE 通过极其轻量级的模型和小规模的训练数据实现了卓越的重建质量。此外，对视频生成任务的评估凸显了其下游应用的强大潜力。

Title: Few-Shot VLM-Based G-Code and HMI Verification in CNC Machining

• Authors: Yasaman Hashem Pour, Nazanin Mahjourian, Vinh Nguyen
• Subjects: cs.CV, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2512.11296
• Pdf URL: https://arxiv.org/pdf/2512.11296
• Copy Paste: [[2512.11296]] Few-Shot VLM-Based G-Code and HMI Verification in CNC Machining(https://arxiv.org/abs/2512.11296)
• Keywords: generation
• Abstract: Manual generation of G-code is important for learning the operation of CNC machines. Prior work in G-code verification uses Large-Language Models (LLMs), which primarily examine errors in the written programming. However, CNC machining requires extensive use and knowledge of the Human-Machine Interface (HMI), which displays machine status and errors. LLMs currently lack the capability to leverage knowledge of HMIs due to their inability to access the vision modality. This paper proposes a few-shot VLM-based verification approach that simultaneously evaluates the G-code and the HMI display for errors and safety status. The input dataset includes paired G-code text and associated HMI screenshots from a 15-slant-PRO lathe, including both correct and error-prone cases. To enable few-shot learning, the VLM is provided with a structured JSON schema based on prior heuristic knowledge. After determining the prompts, instances of G-code and HMI that either contain errors or are error free are used as few-shot examples to guide the VLM. The model was then evaluated in comparison to a zero-shot VLM through multiple scenarios of incorrect G-code and HMI errors with respect to per-slot accuracy. The VLM showed that few-shot prompting led to overall enhancement of detecting HMI errors and discrepancies with the G-code for more comprehensive debugging. Therefore, the proposed framework was demonstrated to be suitable for verification of manually generated G-code that is typically developed in CNC training.
• 摘要：手动生成G代码对于学习数控机床的操作非常重要。先前的 G 代码验证工作使用大语言模型 (LLM)，主要检查书面编程中的错误。然而，CNC 加工需要广泛使用并了解人机界面 (HMI)，它显示机器状态和错误。由于无法访问视觉模态，法学硕士目前缺乏利用 HMI 知识的能力。本文提出了一种基于 VLM 的几次验证方法，可同时评估 G 代码和 HMI 显示的错误和安全状态。输入数据集包括配对的 G 代码文本和来自 15 倾斜 PRO 车床的相关 HMI 屏幕截图，包括正确和容易出错的情况。为了实现少量学习，VLM 提供了基于先验启发式知识的结构化 JSON 模式。确定提示后，包含错误或无错误的 G 代码和 HMI 实例将用作引导 VLM 的少数示例。然后，通过不正确的 G 代码和 HMI 错误的多种场景，与零样本 VLM 进行比较，评估该模型的每槽精度。 VLM 表明，几次提示可以全面增强对 HMI 错误和 G 代码差异的检测，从而实现更全面的调试。因此，所提出的框架被证明适用于验证通常在 CNC 培训中开发的手动生成的 G 代码。

Title: QGEC : Quantum Golay Code Error Correction

• Authors: Hideo Mukai, Hoshitaro Ohnishi
• Subjects: cs.LG, quant-ph
• Abstract URL: https://arxiv.org/abs/2512.11307
• Pdf URL: https://arxiv.org/pdf/2512.11307
• Copy Paste: [[2512.11307]] QGEC : Quantum Golay Code Error Correction(https://arxiv.org/abs/2512.11307)
• Keywords: generative
• Abstract: Quantum computers have the possibility of a much reduced calculation load compared with classical computers in specific problems. Quantum error correction (QEC) is vital for handling qubits, which are vulnerable to external noise. In QEC, actual errors are predicted from the results of syndrome measurements by stabilizer generators, in place of making direct measurements of the data qubits. Here, we propose Quantum Golay code Error Correction (QGEC), a QEC method using Golay code, which is an efficient coding method in classical information theory. We investigated our method's ability in decoding calculations with the Transformer. We evaluated the accuracy of the decoder in a code space defined by the generative polynomials with three different weights sets and three noise models with different correlations of bit-flip error and phase-flip error. Furthermore, under a noise model following a discrete uniform distribution, we compared the decoding performance of Transformer decoders with identical architectures trained respectively on Golay and toric codes. The results showed that the noise model with the smaller correlation gave better accuracy, while the weights of the generative polynomials had little effect on the accuracy of the decoder. In addition, they showed that Golay code requiring 23 data qubits and having a code distance of 7 achieved higher decoding accuracy than toric code which requiring 50 data qubits and having a code distance of 5. This suggests that implementing quantum error correction using a Transformer may enable the Golay code to realize fault-tolerant quantum computation more efficiently.
• 摘要：与经典计算机相比，量子计算机在特定问题上有可能大大减少计算负荷。量子纠错 (QEC) 对于处理容易受到外部噪声影响的量子位至关重要。在 QEC 中，实际误差是根据稳定器生成器的校正子测量结果来预测的，而不是直接测量数据量子位。在这里，我们提出了量子格雷码纠错（QGEC），这是一种使用格雷码的QEC方法，格雷码是经典信息论中的一种高效编码方法。我们研究了我们的方法使用 Transformer 解码计算的能力。我们在由具有三个不同权重集的生成多项式和具有不同位翻转误差和相位翻转误差相关性的三个噪声模型定义的代码空间中评估了解码器的准确性。此外，在遵循离散均匀分布的噪声模型下，我们比较了分别在 Golay 和 toric 码上训练的具有相同架构的 Transformer 解码器的解码性能。结果表明，相关性较小的噪声模型具有较好的精度，而生成多项式的权重对解码器的精度影响很小。此外，他们还表明，需要 23 个数据量子位、码距为 7 的 Golay 码比需要 50 个数据量子位、码距为 5 的 Toric 码具有更高的解码精度。这表明使用 Transformer 实现量子纠错可以使 Golay 码更有效地实现容错量子计算。

Title: KeyframeFace: From Text to Expressive Facial Keyframes

• Authors: Jingchao Wu, Zejian Kang, Haibo Liu, Yuanchen Fei, Xiangru Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11321
• Pdf URL: https://arxiv.org/pdf/2512.11321
• Copy Paste: [[2512.11321]] KeyframeFace: From Text to Expressive Facial Keyframes(https://arxiv.org/abs/2512.11321)
• Keywords: generation
• Abstract: Generating dynamic 3D facial animation from natural language requires understanding both temporally structured semantics and fine-grained expression changes. Existing datasets and methods mainly focus on speech-driven animation or unstructured expression sequences and therefore lack the semantic grounding and temporal structures needed for expressive human performance generation. In this work, we introduce KeyframeFace, a large-scale multimodal dataset designed for text-to-animation research through keyframe-level supervision. KeyframeFace provides 2,100 expressive scripts paired with monocular videos, per-frame ARKit coefficients, contextual backgrounds, complex emotions, manually defined keyframes, and multi-perspective annotations based on ARKit coefficients and images via Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs). Beyond the dataset, we propose the first text-to-animation framework that explicitly leverages LLM priors for interpretable facial motion synthesis. This design aligns the semantic understanding capabilities of LLMs with the interpretable structure of ARKit's coefficients, enabling high-fidelity expressive animation. KeyframeFace and our LLM-based framework together establish a new foundation for interpretable, keyframe-guided, and context-aware text-to-animation. Code and data are available at this https URL.
• 摘要：从自然语言生成动态 3D 面部动画需要理解时间结构语义和细粒度的表情变化。现有的数据集和方法主要关注语音驱动的动画或非结构化表达序列，因此缺乏表达性人类表演生成所需的语义基础和时间结构。在这项工作中，我们介绍了 KeyframeFace，这是一个大型多模态数据集，旨在通过关键帧级监督进行文本到动画研究。 KeyframeFace 提供 2,100 个表达脚本，搭配单目视频、每帧 ARKit 系数、上下文背景、复杂情感、手动定义的关键帧以及通过大语言模型 (LLM) 和多模态大语言模型 (MLLM) 基于 ARKit 系数和图像的多视角注释。除了数据集之外，我们还提出了第一个文本到动画框架，该框架明确利用 LLM 先验来进行可解释的面部运动合成。该设计将 LLM 的语义理解能力与 ARKit 系数的可解释结构结合起来，从而实现高保真表达动画。 KeyframeFace 和我们基于 LLM 的框架共同为可解释、关键帧引导和上下文感知的文本到动画奠定了新的基础。代码和数据可从此 https URL 获取。

Title: Physics-Informed Video Flare Synthesis and Removal Leveraging Motion Independence between Flare and Scene

• Authors: Junqiao Wang, Yuanfei Huang, Hua Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11327
• Pdf URL: https://arxiv.org/pdf/2512.11327
• Copy Paste: [[2512.11327]] Physics-Informed Video Flare Synthesis and Removal Leveraging Motion Independence between Flare and Scene(https://arxiv.org/abs/2512.11327)
• Keywords: restoration
• Abstract: Lens flare is a degradation phenomenon caused by strong light sources. Existing researches on flare removal have mainly focused on images, while the spatiotemporal characteristics of video flare remain largely unexplored. Video flare synthesis and removal pose significantly greater challenges than in image, owing to the complex and mutually independent motion of flare, light sources, and scene content. This motion independence further affects restoration performance, often resulting in flicker and artifacts. To address this issue, we propose a physics-informed dynamic flare synthesis pipeline, which simulates light source motion using optical flow and models the temporal behaviors of both scattering and reflective flares. Meanwhile, we design a video flare removal network that employs an attention module to spatially suppress flare regions and incorporates a Mamba-based temporal modeling component to capture long range spatio-temporal dependencies. This motion-independent spatiotemporal representation effectively eliminates the need for multi-frame alignment, alleviating temporal aliasing between flares and scene content and thereby improving video flare removal performance. Building upon this, we construct the first video flare dataset to comprehensively evaluate our method, which includes a large set of synthetic paired videos and additional real-world videos collected from the Internet to assess generalization capability. Extensive experiments demonstrate that our method consistently outperforms existing video-based restoration and image-based flare removal methods on both real and synthetic videos, effectively removing dynamic flares while preserving light source integrity and maintaining spatiotemporal consistency of scene.
• 摘要：镜头眩光是由强光源引起的退化现象。现有的耀斑去除研究主要集中在图像上，而视频耀斑的时空特征很大程度上尚未被探索。由于耀斑、光源和场景内容的复杂且相互独立的运动，视频耀斑的合成和去除比图像中的挑战要大得多。这种运动独立性进一步影响恢复性能，通常会导致闪烁和伪影。为了解决这个问题，我们提出了一种基于物理的动态耀斑合成管道，它使用光流模拟光源运动，并对散射和反射耀斑的时间行为进行建模。同时，我们设计了一个视频耀斑去除网络，该网络采用注意模块在空间上抑制耀斑区域，并结合基于 Mamba 的时间建模组件来捕获长范围的时空依赖性。这种与运动无关的时空表示有效地消除了多帧对齐的需要，减轻了耀斑和场景内容之间的时间混叠，从而提高了视频耀斑去除性能。在此基础上，我们构建了第一个视频耀斑数据集来全面评估我们的方法，其中包括大量合成配对视频和从互联网收集的其他真实世界视频，以评估泛化能力。大量实验表明，我们的方法在真实和合成视频上始终优于现有的基于视频的恢复和基于图像的耀斑去除方法，有效去除动态耀斑，同时保持光源完整性并保持场景的时空一致性。

Title: Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits

• Authors: Minwoo Park, Junwoo Chang, Jongeun Choi, Roberto Horowitz
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2512.11345
• Pdf URL: https://arxiv.org/pdf/2512.11345
• Copy Paste: [[2512.11345]] Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits(https://arxiv.org/abs/2512.11345)
• Keywords: generative
• Abstract: Equivariant diffusion policies (EDPs) combine the generative expressivity of diffusion models with the strong generalization and sample efficiency afforded by geometric symmetries. While steering these policies with reinforcement learning (RL) offers a promising mechanism for fine-tuning beyond demonstration data, directly applying standard (non-equivariant) RL can be sample-inefficient and unstable, as it ignores the symmetries that EDPs are designed to exploit. In this paper, we theoretically establish that the diffusion process of an EDP is equivariant, which in turn induces a group-invariant latent-noise MDP that is well-suited for equivariant diffusion steering. Building on this theory, we introduce a principled symmetry-aware steering framework and compare standard, equivariant, and approximately equivariant RL strategies through comprehensive experiments across tasks with varying degrees of symmetry. While we identify the practical boundaries of strict equivariance under symmetry breaking, we show that exploiting symmetry during the steering process yields substantial benefits-enhancing sample efficiency, preventing value divergence, and achieving strong policy improvements even when EDPs are trained from extremely limited demonstrations.
• 摘要：等变扩散策略（EDP）将扩散模型的生成表达能力与几何对称性提供的强泛化性和样本效率结合起来。虽然通过强化学习 (RL) 来引导这些策略提供了一种有前途的机制，可以在演示数据之外进行微调，但直接应用标准（非等变）强化学习可能会导致样本效率低下且不稳定，因为它忽略了 EDP 旨在利用的对称性。在本文中，我们从理论上建立了 EDP 的扩散过程是等变的，这反过来又产生了非常适合等变扩散控制的群不变潜在噪声 MDP。在此理论的基础上，我们引入了一个有原则的对称感知控制框架，并通过对不同对称程度的任务进行综合实验来比较标准、等变和近似等变的强化学习策略。虽然我们确定了对称性破缺下严格等方差的实际边界，但我们表明，在引导过程中利用对称性会产生巨大的好处——提高样本效率，防止价值发散，并实现强有力的政策改进，即使 EDP 是通过极其有限的演示进行训练的。

Title: JoyAvatar: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion

• Authors: Chaochao Li, Ruikui Wang, Liangbo Zhou, Jinheng Feng, Huaishao Luo, Huan Zhang, Youzheng Wu, Xiaodong He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11423
• Pdf URL: https://arxiv.org/pdf/2512.11423
• Copy Paste: [[2512.11423]] JoyAvatar: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion(https://arxiv.org/abs/2512.11423)
• Keywords: generation
• Abstract: Existing DiT-based audio-driven avatar generation methods have achieved considerable progress, yet their broader application is constrained by limitations such as high computational overhead and the inability to synthesize long-duration videos. Autoregressive methods address this problem by applying block-wise autoregressive diffusion methods. However, these methods suffer from the problem of error accumulation and quality degradation. To address this, we propose JoyAvatar, an audio-driven autoregressive model capable of real-time inference and infinite-length video generation with the following contributions: (1) Progressive Step Bootstrapping (PSB), which allocates more denoising steps to initial frames to stabilize generation and reduce error accumulation; (2) Motion Condition Injection (MCI), enhancing temporal coherence by injecting noise-corrupted previous frames as motion condition; and (3) Unbounded RoPE via Cache-Resetting (URCR), enabling infinite-length generation through dynamic positional encoding. Our 1.3B-parameter causal model achieves 16 FPS on a single GPU and achieves competitive results in visual quality, temporal consistency, and lip synchronization.
• 摘要：现有的基于 DiT 的音频驱动头像生成方法已经取得了相当大的进步，但其更广泛的应用受到高计算开销和无法合成长时间视频等限制。自回归方法通过应用分块自回归扩散方法来解决这个问题。然而，这些方法存在错误累积和质量下降的问题。为了解决这个问题，我们提出了 JoyAvatar，一种音频驱动的自回归模型，能够实时推理和无限长度的视频生成，具有以下贡献：（1）渐进式引导（PSB），它为初始帧分配更多的去噪步骤以稳定生成并减少错误累积； （2）运动条件注入（MCI），通过注入噪声损坏的先前帧作为运动条件来增强时间相干性； (3) 通过缓存重置 (URCR) 实现无界 RoPE，通过动态位置编码实现无限长度生成。我们的 1.3B 参数因果模型在单个 GPU 上实现了 16 FPS，并在视觉质量、时间一致性和唇形同步方面取得了有竞争力的结果。

Title: Flowception: Temporally Expansive Flow Matching for Video Generation

• Authors: Tariq Berrada Ifriqi, John Nguyen, Karteek Alahari, Jakob Verbeek, Ricky T. Q. Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.11438
• Pdf URL: https://arxiv.org/pdf/2512.11438
• Copy Paste: [[2512.11438]] Flowception: Temporally Expansive Flow Matching for Video Generation(https://arxiv.org/abs/2512.11438)
• Keywords: generation
• Abstract: We present Flowception, a novel non-autoregressive and variable-length video generation framework. Flowception learns a probability path that interleaves discrete frame insertions with continuous frame denoising. Compared to autoregressive methods, Flowception alleviates error accumulation/drift as the frame insertion mechanism during sampling serves as an efficient compression mechanism to handle long-term context. Compared to full-sequence flows, our method reduces FLOPs for training three-fold, while also being more amenable to local attention variants, and allowing to learn the length of videos jointly with their content. Quantitative experimental results show improved FVD and VBench metrics over autoregressive and full-sequence baselines, which is further validated with qualitative results. Finally, by learning to insert and denoise frames in a sequence, Flowception seamlessly integrates different tasks such as image-to-video generation and video interpolation.
• 摘要：我们推出 Flowception，一种新颖的非自回归和可变长度视频生成框架。 Flowception 学习一条将离散帧插入与连续帧去噪交织在一起的概率路径。与自回归方法相比，Flowception 减轻了错误累积/漂移，因为采样期间的帧插入机制可作为处理长期上下文的有效压缩机制。与全序列流相比，我们的方法将训练的 FLOP 减少了三倍，同时也更适合局部注意力变量，并允许联合学习视频的长度及其内容。定量实验结果表明，FVD 和 VBench 指标优于自回归和全序列基线，并通过定性结果进一步验证。最后，通过学习在序列中插入帧和降噪，Flowception 无缝集成了不同的任务，例如图像到视频生成和视频插值。

Title: Exploring MLLM-Diffusion Information Transfer with MetaCanvas

• Authors: Han Lin, Xichen Pan, Ziqi Huang, Ji Hou, Jialiang Wang, Weifeng Chen, Zecheng He, Felix Juefei-Xu, Junzhe Sun, Zhipeng Fan, Ali Thabet, Mohit Bansal, Chu Wang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11464
• Pdf URL: https://arxiv.org/pdf/2512.11464
• Copy Paste: [[2512.11464]] Exploring MLLM-Diffusion Information Transfer with MetaCanvas(https://arxiv.org/abs/2512.11464)
• Keywords: generation
• Abstract: Multimodal learning has rapidly advanced visual understanding, largely via multimodal large language models (MLLMs) that use powerful LLMs as cognitive cores. In visual generation, however, these powerful core models are typically reduced to global text encoders for diffusion models, leaving most of their reasoning and planning ability unused. This creates a gap: current multimodal LLMs can parse complex layouts, attributes, and knowledge-intensive scenes, yet struggle to generate images or videos with equally precise and structured control. We propose MetaCanvas, a lightweight framework that lets MLLMs reason and plan directly in spatial and spatiotemporal latent spaces and interface tightly with diffusion generators. We empirically implement MetaCanvas on three different diffusion backbones and evaluate it across six tasks, including text-to-image generation, text/image-to-video generation, image/video editing, and in-context video generation, each requiring precise layouts, robust attribute binding, and reasoning-intensive control. MetaCanvas consistently outperforms global-conditioning baselines, suggesting that treating MLLMs as latent-space planners is a promising direction for narrowing the gap between multimodal understanding and generation.
• 摘要：多模态学习快速推进了视觉理解，主要是通过使用强大的法学硕士作为认知核心的多模态大语言模型 (MLLM)。然而，在视觉生成中，这些强大的核心模型通常被简化为扩散模型的全局文本编码器，而使它们的大部分推理和规划能力未被使用。这就造成了一个差距：当前的多模式法学硕士可以解析复杂的布局、属性和知识密集型场景，但很难生成具有同样精确和结构化控制的图像或视频。我们提出了 MetaCanvas，这是一个轻量级框架，可以让 MLLM 直接在空间和时空潜在空间中进行推理和规划，并与扩散生成器紧密结合。我们根据经验在三个不同的扩散主干上实现 MetaCanvas，并在六个任务中对其进行评估，包括文本到图像生成、文本/图像到视频生成、图像/视频编辑和上下文视频生成，每个任务都需要精确的布局、强大的属性绑定和推理密集型控制。 MetaCanvas 始终优于全局调节基线，这表明将 MLLM 视为潜在空间规划器是缩小多模态理解和生成之间差距的一个有希望的方向。

Title: CADMorph: Geometry-Driven Parametric CAD Editing via a Plan-Generate-Verify Loop

• Authors: Weijian Ma, Shizhao Sun, Ruiyu Wang, Jiang Bian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11480
• Pdf URL: https://arxiv.org/pdf/2512.11480
• Copy Paste: [[2512.11480]] CADMorph: Geometry-Driven Parametric CAD Editing via a Plan-Generate-Verify Loop(https://arxiv.org/abs/2512.11480)
• Keywords: generation
• Abstract: A Computer-Aided Design (CAD) model encodes an object in two coupled forms: a parametric construction sequence and its resulting visible geometric shape. During iterative design, adjustments to the geometric shape inevitably require synchronized edits to the underlying parametric sequence, called geometry-driven parametric CAD editing. The task calls for 1) preserving the original sequence's structure, 2) ensuring each edit's semantic validity, and 3) maintaining high shape fidelity to the target shape, all under scarce editing data triplets. We present CADMorph, an iterative plan-generate-verify framework that orchestrates pretrained domain-specific foundation models during inference: a parameter-to-shape (P2S) latent diffusion model and a masked-parameter-prediction (MPP) model. In the planning stage, cross-attention maps from the P2S model pinpoint the segments that need modification and offer editing masks. The MPP model then infills these masks with semantically valid edits in the generation stage. During verification, the P2S model embeds each candidate sequence in shape-latent space, measures its distance to the target shape, and selects the closest one. The three stages leverage the inherent geometric consciousness and design knowledge in pretrained priors, and thus tackle structure preservation, semantic validity, and shape fidelity respectively. Besides, both P2S and MPP models are trained without triplet data, bypassing the data-scarcity bottleneck. CADMorph surpasses GPT-4o and specialized CAD baselines, and supports downstream applications such as iterative editing and reverse-engineering enhancement.
• 摘要：计算机辅助设计 (CAD) 模型以两种耦合形式对对象进行编码：参数化构造序列及其产生的可见几何形状。在迭代设计过程中，对几何形状的调整不可避免地需要对底层参数序列进行同步编辑，称为几何驱动参数化 CAD 编辑。该任务要求 1) 保留原始序列的结构，2) 确保每次编辑的语义有效性，3) 保持对目标形状的高形状保真度，所有这些都在稀缺的编辑数据三元组下进行。我们提出了 CADMorph，这是一个迭代计划-生成-验证框架，可在推理过程中编排预训练的特定领域基础模型：参数到形状 (P2S) 潜在扩散模型和掩蔽参数预测 (MPP) 模型。在规划阶段，P2S 模型的交叉注意力图可以查明需要修改的片段并提供编辑掩模。然后，MPP 模型在生成阶段用语义上有效的编辑来填充这些掩码。在验证过程中，P2S模型将每个候选序列嵌入到形状潜在空间中，测量其与目标形状的距离，并选择最接近的一个。这三个阶段利用预先训练的先验中固有的几何意识和设计知识，从而分别解决结构保存、语义有效性和形状保真度。此外，P2S和MPP模型都是在没有三元组数据的情况下进行训练的，绕过了数据稀缺的瓶颈。 CADMorph 超越了 GPT-4o 和专门的 CAD 基线，并支持迭代编辑和逆向工程增强等下游应用程序。

Title: VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

• Authors: Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan, Yonghao Xu, Michael Felsberg
• Subjects: cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.11490
• Pdf URL: https://arxiv.org/pdf/2512.11490
• Copy Paste: [[2512.11490]] VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing(https://arxiv.org/abs/2512.11490)
• Keywords: generative
• Abstract: Satellite imagery differs fundamentally from natural images: its aerial viewpoint, very high resolution, diverse scale variations, and abundance of small objects demand both region-level spatial reasoning and holistic scene understanding. Current remote-sensing approaches remain fragmented between dual-encoder retrieval models, which excel at large-scale cross-modal search but cannot interleave modalities, and generative assistants, which support region-level interpretation but lack scalable retrieval capabilities. We propose $\textbf{VLM2GeoVec}$, an instruction-following, single-encoder vision-language model trained contrastively to embed interleaved inputs (images, text, bounding boxes, and geographic coordinates) in a unified vector space. Our single encoder interleaves all inputs into one joint embedding trained with a contrastive loss, eliminating multi-stage pipelines and task-specific modules. To evaluate its versatility, we introduce $\textbf{RSMEB}$, a novel benchmark covering key remote-sensing embedding applications: scene classification; cross-modal search; compositional retrieval; visual-question answering; visual grounding and region-level reasoning; and semantic geospatial retrieval. On RSMEB, it achieves $\textbf{26.6%}$ P@1 on region-caption retrieval (+25 pp vs. dual-encoder baselines), $\textbf{32.5%}$ P@1 on referring-expression retrieval (+19 pp), and $\textbf{17.8%}$ P@1 on semantic geo-localization retrieval (over $3\times$ prior best), while matching or exceeding specialized baselines on conventional tasks such as scene classification and cross-modal retrieval. VLM2GeoVec unifies scalable retrieval with region-level spatial reasoning, enabling cohesive multimodal analysis in remote sensing. We will publicly release the code, checkpoints, and data upon acceptance.
• 摘要：卫星图像与自然图像有着根本的不同：​​它的空中视角、非常高分辨率、不同的尺度变化以及大量的小物体需要区域级的空间推理和整体场景理解。当前的遥感方法在双编码器检索模型和生成助理之间仍然碎片化，双编码器检索模型擅长大规模跨模态搜索，但不能交错模态，而生成助理则支持区域级解释，但缺乏可扩展的检索能力。我们提出了 $\textbf{VLM2GeoVec}$，一种遵循指令的单编码器视觉语言模型，经过对比训练，将交错输入（图像、文本、边界框和地理坐标）嵌入到统一的向量空间中。我们的单个编码器将所有输入交织到一个联合嵌入中，并通过对比损失进行训练，从而消除了多级管道和特定于任务的模块。为了评估其多功能性，我们引入了 $\textbf{RSMEB}$，这是一个涵盖关键遥感嵌入应用的新颖基准：场景分类；跨模式搜索；构图检索；视觉问答；视觉基础和区域级推理；和语义地理空间检索。在 RSMEB 上，它在区域标题检索上实现 $\textbf{26.6%}$ P@1（与双编码器基线相比，+25 pp），在指称表达检索上实现 $\textbf{32.5%}$ P@1（+19 pp），在语义地理定位检索上实现 $\textbf{17.8%}$ P@1（超过 $3\times$ 先前最佳值），同时匹配或超过专业化场景分类和跨模式检索等传统任务的基线。 VLM2GeoVec 将可扩展检索与区域级空间推理相结合，从而实现遥感中的内聚多模态分析。我们将在接受后公开发布代码、检查点和数据。

Title: SSA3D: Text-Conditioned Assisted Self-Supervised Framework for Automatic Dental Abutment Design

• Authors: Mianjie Zheng, Xinquan Yang, Along He, Xuguang Li, Feilie Zhong, Xuefen Liu, Kun Tang, Zhicheng Zhang, Linlin Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11507
• Pdf URL: https://arxiv.org/pdf/2512.11507
• Copy Paste: [[2512.11507]] SSA3D: Text-Conditioned Assisted Self-Supervised Framework for Automatic Dental Abutment Design(https://arxiv.org/abs/2512.11507)
• Keywords: restoration
• Abstract: Abutment design is a critical step in dental implant restoration. However, manual design involves tedious measurement and fitting, and research on automating this process with AI is limited, due to the unavailability of large annotated datasets. Although self-supervised learning (SSL) can alleviate data scarcity, its need for pre-training and fine-tuning results in high computational costs and long training times. In this paper, we propose a Self-supervised assisted automatic abutment design framework (SS$A^3$D), which employs a dual-branch architecture with a reconstruction branch and a regression branch. The reconstruction branch learns to restore masked intraoral scan data and transfers the learned structural information to the regression branch. The regression branch then predicts the abutment parameters under supervised learning, which eliminates the separate pre-training and fine-tuning process. We also design a Text-Conditioned Prompt (TCP) module to incorporate clinical information (such as implant location, system, and series) into SS$A^3$D. This guides the network to focus on relevant regions and constrains the parameter predictions. Extensive experiments on a collected dataset show that SS$A^3$D saves half of the training time and achieves higher accuracy than traditional SSL methods. It also achieves state-of-the-art performance compared to other methods, significantly improving the accuracy and efficiency of automated abutment design.
• 摘要：基台设计是牙种植体修复的关键步骤。然而，手动设计涉及繁琐的测量和拟合，并且由于缺乏大型注释数据集，利用人工智能自动化该过程的研究是有限的。虽然自监督学习（SSL）可以缓解数据稀缺性，但其需要预训练和微调，导致计算成本高、训练时间长。在本文中，我们提出了一种自监督辅助自动基台设计框架（SS$A^3$D），该框架采用具有重建分支和回归分支的双分支架构。重建分支学习恢复蒙版口内扫描数据并将学习到的结构信息传输到回归分支。然后，回归分支在监督学习下预测基台参数，从而消除了单独的预训练和微调过程。我们还设计了一个文本条件提示 (TCP) 模块，将临床信息（例如种植体位置、系统和系列）合并到 SS$A^3$D 中。这引导网络关注相关区域并限制参数预测。对收集的数据集进行的大量实验表明，SS$A^3$D 比传统 SSL 方法节省了一半的训练时间，并且获得了更高的准确度。与其他方法相比，它还实现了最先进的性能，显着提高了自动化基台设计的准确性和效率。

Title: Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using LiDAR HD Reference Data across Metropolitan France

• Authors: Ekaterina Kalinicheva, Florian Helen, Stéphane Mermoz, Florian Mouret, Milena Planells
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11524
• Pdf URL: https://arxiv.org/pdf/2512.11524
• Copy Paste: [[2512.11524]] Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using LiDAR HD Reference Data across Metropolitan France(https://arxiv.org/abs/2512.11524)
• Keywords: super-resolution
• Abstract: Fine-scale forest monitoring is essential for understanding canopy structure and its dynamics, which are key indicators of carbon stocks, biodiversity, and forest health. Deep learning is particularly effective for this task, as it integrates spectral, temporal, and spatial signals that jointly reflect the canopy structure. To address this need, we introduce THREASURE-Net, a novel end-to-end framework for Tree Height Regression And Super-Resolution. The model is trained on Sentinel-2 time series using reference height metrics derived from LiDAR HD data at multiple spatial resolutions over Metropolitan France to produce annual height maps. We evaluate three model variants, producing tree-height predictions at 2.5 m, 5 m, and 10 m resolution. THREASURE-Net does not rely on any pretrained model nor on reference very high resolution optical imagery to train its super-resolution module; instead, it learns solely from LiDAR-derived height information. Our approach outperforms existing state-of-the-art methods based on Sentinel data and is competitive with methods based on very high resolution imagery. It can be deployed to generate high-precision annual canopy-height maps, achieving mean absolute errors of 2.62 m, 2.72 m, and 2.88 m at 2.5 m, 5 m, and 10 m resolution, respectively. These results highlight the potential of THREASURE-Net for scalable and cost-effective structural monitoring of temperate forests using only freely available satellite data. The source code for THREASURE-Net is available at: this https URL.
• 摘要：精细化森林监测对于了解冠层结构及其动态至关重要，而冠层结构是碳储量、生物多样性和森林健康的关键指标。深度学习对于这项任务特别有效，因为它集成了共同反映冠层结构的光谱、时间和空间信号。为了满足这一需求，我们引入了 THREASURE-Net，这是一种用于树高回归和超分辨率的新型端到端框架。该模型在 Sentinel-2 时间序列上进行训练，使用源自法国大都市多个空间分辨率的 LiDAR HD 数据的参考高度指标，以生成年度高度图。我们评估了三种模型变体，以 2.5 m、5 m 和 10 m 分辨率生成树高预测。 THREASURE-Net 不依赖任何预训练模型，也不依赖参考超高分辨率光学图像来训练其超分辨率模块；相反，它仅从激光雷达获得的高度信息中学习。我们的方法优于基于 Sentinel 数据的现有最先进方法，并且与基于超高分辨率图像的方法具有竞争力。它可以用于生成高精度的年度冠层高度地图，在 2.5 m、5 m 和 10 m 分辨率下，平均绝对误差分别为 2.62 m、2.72 m 和 2.88 m。这些结果凸显了 THREASURE-Net 在仅使用免费卫星数据对温带森林进行可扩展且具有成本效益的结构监测方面的潜力。 THREASURE-Net 的源代码位于：此 https URL。

Title: xGR: Efficient Generative Recommendation Serving at Scale

• Authors: Qingxiao Sun, Tongxuan Liu, Shen Zhang, Siyu Wu, Peijun Yang, Haotian Liang, Menxin Li, Xiaolong Ma, Zhiwei Liang, Ziyi Ren, Minchao Zhang, Xinyu Liu, Ke Zhang, Depei Qian, Hailong Yang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.11529
• Pdf URL: https://arxiv.org/pdf/2512.11529
• Copy Paste: [[2512.11529]] xGR: Efficient Generative Recommendation Serving at Scale(https://arxiv.org/abs/2512.11529)
• Keywords: generative
• Abstract: Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of long user-item sequences. Despite employing attention-based architectures, GR's workload differs markedly from that of LLM serving. GR typically processes long prompt while producing short, fixed-length outputs, yet the computational cost of each decode phase is especially high due to the large beam width. In addition, since the beam search involves a vast item space, the sorting overhead becomes particularly time-consuming. We propose xGR, a GR-oriented serving system that meets strict low-latency requirements under highconcurrency scenarios. First, xGR unifies the processing of prefill and decode phases through staged computation and separated KV cache. Second, xGR enables early sorting termination and mask-based item filtering with data structure reuse. Third, xGR reconstructs the overall pipeline to exploit multilevel overlap and multi-stream parallelism. Our experiments with real-world recommendation service datasets demonstrate that xGR achieves at least 3.49x throughput compared to the state-of-the-art baseline under strict latency constraints.
• 摘要：推荐系统通过提供个性化预测来带来可观的经济效益。生成推荐 (GR) 集成了法学硕士，以增强对长用户项目序列的理解。尽管采用基于注意力的架构，GR 的工作量与 LLM 服务的工作量明显不同。 GR 通常会处理长提示，同时产生短的固定长度输出，但由于波束宽度较大，每个解码阶段的计算成本特别高。此外，由于波束搜索涉及巨大的项目空间，因此排序开销变得特别耗时。我们提出了xGR，一个面向GR的服务系统，满足高并发场景下严格的低延迟要求。首先，xGR通过分级计算和分离的KV缓存来统一预填充和解码阶段的处理。其次，xGR 通过数据结构重用实现早期排序终止和基于掩码的项目过滤。第三，xGR 重建整体管道以利用多级重叠和多流并行性。我们对真实推荐服务数据集的实验表明，在严格的延迟限制下，与最先进的基线相比，xG​​R 的吞吐量至少提高了 3.49 倍。

Title: SSL-MedSAM2: A Semi-supervised Medical Image Segmentation Framework Powered by Few-shot Learning of SAM2

• Authors: Zhendi Gong, Xin Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11548
• Pdf URL: https://arxiv.org/pdf/2512.11548
• Copy Paste: [[2512.11548]] SSL-MedSAM2: A Semi-supervised Medical Image Segmentation Framework Powered by Few-shot Learning of SAM2(https://arxiv.org/abs/2512.11548)
• Keywords: generation
• Abstract: Despite the success of deep learning based models in medical image segmentation, most state-of-the-art (SOTA) methods perform fully-supervised learning, which commonly rely on large scale annotated training datasets. However, medical image annotation is highly time-consuming, hindering its clinical applications. Semi-supervised learning (SSL) has been emerged as an appealing strategy in training with limited annotations, largely reducing the labelling cost. We propose a novel SSL framework SSL-MedSAM2, which contains a training-free few-shot learning branch TFFS-MedSAM2 based on the pretrained large foundation model Segment Anything Model 2 (SAM2) for pseudo label generation, and an iterative fully-supervised learning branch FSL-nnUNet based on nnUNet for pseudo label refinement. The results on MICCAI2025 challenge CARE-LiSeg (Liver Segmentation) demonstrate an outstanding performance of SSL-MedSAM2 among other methods. The average dice scores on the test set in GED4 and T1 MRI are 0.9710 and 0.9648 respectively, and the Hausdorff distances are 20.07 and 21.97 respectively. The code is available via this https URL.
• 摘要：尽管基于深度学习的模型在医学图像分割中取得了成功，但大多数最先进的（SOTA）方法执行完全监督学习，通常依赖于大规模带注释的训练数据集。然而，医学图像标注非常耗时，阻碍了其临床应用。半监督学习（SSL）已成为一种在注释有限的训练中颇具吸引力的策略，大大降低了标签成本。我们提出了一种新颖的 SSL 框架 SSL-MedSAM2，其中包含一个基于预训练大型基础模型 Segment Anything Model 2 (SAM2) 的免训练小样本学习分支 TFFS-MedSAM2，用于伪标签生成，以及一个基于 nnUNet 的迭代全监督学习分支 FSL-nnUNet，用于伪标签细化。 MICCAI2025 挑战 CARE-LiSeg（肝脏分割）的结果表明 SSL-MedSAM2 在其他方法中具有出色的性能。 GED4 和 T1 MRI 测试集上的平均骰子分数分别为 0.9710 和 0.9648，豪斯多夫距离分别为 20.07 和 21.97。该代码可通过此 https URL 获取。

Title: Fast and Explicit: Slice-to-Volume Reconstruction via 3D Gaussian Primitives with Analytic Point Spread Function Modeling

• Authors: Maik Dannecker, Steven Jia, Nil Stolt-Ansó, Nadine Girard, Guillaume Auzias, François Rousseau, Daniel Rueckert
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11624
• Pdf URL: https://arxiv.org/pdf/2512.11624
• Copy Paste: [[2512.11624]] Fast and Explicit: Slice-to-Volume Reconstruction via 3D Gaussian Primitives with Analytic Point Spread Function Modeling(https://arxiv.org/abs/2512.11624)
• Keywords: super-resolution
• Abstract: Recovering high-fidelity 3D images from sparse or degraded 2D images is a fundamental challenge in medical imaging, with broad applications ranging from 3D ultrasound reconstruction to MRI super-resolution. In the context of fetal MRI, high-resolution 3D reconstruction of the brain from motion-corrupted low-resolution 2D acquisitions is a prerequisite for accurate neurodevelopmental diagnosis. While implicit neural representations (INRs) have recently established state-of-the-art performance in self-supervised slice-to-volume reconstruction (SVR), they suffer from a critical computational bottleneck: accurately modeling the image acquisition physics requires expensive stochastic Monte Carlo sampling to approximate the point spread function (PSF). In this work, we propose a shift from neural network based implicit representations to Gaussian based explicit representations. By parameterizing the HR 3D image volume as a field of anisotropic Gaussian primitives, we leverage the property of Gaussians being closed under convolution and thus derive a \textit{closed-form analytical solution} for the forward model. This formulation reduces the previously intractable acquisition integral to an exact covariance addition ($\mathbf{\Sigma}_{obs} = \mathbf{\Sigma}_{HR} + \mathbf{\Sigma}_{PSF}$), effectively bypassing the need for compute-intensive stochastic sampling while ensuring exact gradient propagation. We demonstrate that our approach matches the reconstruction quality of self-supervised state-of-the-art SVR frameworks while delivering a 5$\times$--10$\times$ speed-up on neonatal and fetal data. With convergence often reached in under 30 seconds, our framework paves the way towards translation into clinical routine of real-time fetal 3D MRI. Code will be public at {this https URL}.
• 摘要：从稀疏或退化的 2D 图像中恢复高保真 3D 图像是医学成像领域的一项基本挑战，其应用范围广泛，从 3D 超声重建到 MRI 超分辨率。在胎儿 MRI 背景下，从运动损坏的低分辨率 2D 采集中对大脑进行高分辨率 3D 重建是准确神经发育诊断的先决条件。虽然隐式神经表示（INR）最近在自监督切片到体积重建（SVR）中建立了最先进的性能，但它们遇到了关键的计算瓶颈：精确建模图像采集物理需要昂贵的随机蒙特卡罗采样来近似点扩散函数（PSF）。在这项工作中，我们提出从基于神经网络的隐式表示到基于高斯的显式表示的转变。通过将 HR 3D 图像体积参数化为各向异性高斯基元的场，我们利用高斯在卷积下封闭的特性，从而导出前向模型的 \textit{封闭形式解析解}。该公式将之前难以处理的采集积分简化为精确的协方差加法 ($\mathbf{\Sigma}_{obs} = \mathbf{\Sigma}_{HR} + \mathbf{\Sigma}_{PSF}$)，有效地绕过了计算密集型随机采样的需要，同时确保了精确的梯度传播。我们证明，我们的方法与最先进的自监督 SVR 框架的重建质量相匹配，同时对新生儿和胎儿数据提供 5$\times$--10$\times$ 加速。由于通常在 30 秒内即可达到收敛，我们的框架为转化为实时胎儿 3D MRI 的临床常规铺平了道路。代码将在 {this https URL} 公开。

Title: FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpoint

• Authors: Jiapeng Tang, Kai Li, Chengxiang Yin, Liuhao Ge, Fei Jiang, Jiu Xu, Matthias Nießner, Christian Häne, Timur Bagautdinov, Egor Zakharov, Peihong Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11645
• Pdf URL: https://arxiv.org/pdf/2512.11645
• Copy Paste: [[2512.11645]] FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpoint(https://arxiv.org/abs/2512.11645)
• Keywords: generation
• Abstract: We introduce FactorPortrait, a video diffusion method for controllable portrait animation that enables lifelike synthesis from disentangled control signals of facial expressions, head movement, and camera viewpoints. Given a single portrait image, a driving video, and camera trajectories, our method animates the portrait by transferring facial expressions and head movements from the driving video while simultaneously enabling novel view synthesis from arbitrary viewpoints. We utilize a pre-trained image encoder to extract facial expression latents from the driving video as control signals for animation generation. Such latents implicitly capture nuanced facial expression dynamics with identity and pose information disentangled, and they are efficiently injected into the video diffusion transformer through our proposed expression controller. For camera and head pose control, we employ Plücker ray maps and normal maps rendered from 3D body mesh tracking. To train our model, we curate a large-scale synthetic dataset containing diverse combinations of camera viewpoints, head poses, and facial expression dynamics. Extensive experiments demonstrate that our method outperforms existing approaches in realism, expressiveness, control accuracy, and view consistency.
• 摘要：我们介绍了 FactorPortrait，这是一种用于可控肖像动画的视频扩散方法，可以根据面部表情、头部运动和相机视点的解开控制信号进行逼真的合成。给定单个肖像图像、驾驶视频和相机轨迹，我们的方法通过传输驾驶视频中的面部表情和头部运动来动画肖像，同时从任意视点实现新颖的视图合成。我们利用预先训练的图像编码器从驾驶视频中提取潜在的面部表情作为动画生成的控制信号。这些潜伏隐式地捕获了细致入微的面部表情动态，并解开了身份和姿势信息，并且它们通过我们提出的表情控制器有效地注入到视频扩散变压器中。对于相机和头部姿势控制，我们采用 Plücker 射线贴图和通过 3D 身体网格跟踪渲染的法线贴图。为了训练我们的模型，我们策划了一个大规模的合成数据集，其中包含相机视点、头部姿势和面部表情动态的不同组合。大量的实验表明，我们的方法在真实性、表现力、控制精度和视图一致性方面优于现有方法。

Title: Kinetic Mining in Context: Few-Shot Action Synthesis via Text-to-Motion Distillation

• Authors: Luca Cazzola, Ahed Alboody
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11654
• Pdf URL: https://arxiv.org/pdf/2512.11654
• Copy Paste: [[2512.11654]] Kinetic Mining in Context: Few-Shot Action Synthesis via Text-to-Motion Distillation(https://arxiv.org/abs/2512.11654)
• Keywords: generative
• Abstract: The acquisition cost for large, annotated motion datasets remains a critical bottleneck for skeletal-based Human Activity Recognition (HAR). Although Text-to-Motion (T2M) generative models offer a compelling, scalable source of synthetic data, their training objectives, which emphasize general artistic motion, and dataset structures fundamentally differ from HAR's requirements for kinematically precise, class-discriminative actions. This disparity creates a significant domain gap, making generalist T2M models ill-equipped for generating motions suitable for HAR classifiers. To address this challenge, we propose KineMIC (Kinetic Mining In Context), a transfer learning framework for few-shot action synthesis. KineMIC adapts a T2M diffusion model to an HAR domain by hypothesizing that semantic correspondences in the text encoding space can provide soft supervision for kinematic distillation. We operationalize this via a kinetic mining strategy that leverages CLIP text embeddings to establish correspondences between sparse HAR labels and T2M source data. This process guides fine-tuning, transforming the generalist T2M backbone into a specialized few-shot Action-to-Motion generator. We validate KineMIC using HumanML3D as the source T2M dataset and a subset of NTU RGB+D 120 as the target HAR domain, randomly selecting just 10 samples per action class. Our approach generates significantly more coherent motions, providing a robust data augmentation source that delivers a +23.1% accuracy points improvement. Animated illustrations and supplementary materials are available at (this https URL).
• 摘要：大型带注释运动数据集的获取成本仍然是基于骨骼的人类活动识别（HAR）的关键瓶颈。尽管文本到动作 (T2M) 生成模型提供了令人信服的、可扩展的合成数据源，但它们强调一般艺术动作的训练目标和数据集结构与 HAR 对运动学精确、类别区分动作的要求存在根本差异。这种差异造成了显着的域差距，使得通用 T2M 模型无法生成适合 HAR 分类器的运动。为了应对这一挑战，我们提出了 KineMIC（上下文动态挖掘），这是一种用于少镜头动作合成的迁移学习框架。 KineMIC 通过假设文本编码空间中的语义对应可以为运动蒸馏提供软监督，将 T2M 扩散模型应用于 HAR 域。我们通过动态挖掘策略来实现这一点，该策略利用 CLIP 文本嵌入来建立稀疏 HAR 标签和 T2M 源数据之间的对应关系。此过程指导微调，将通用的 T2M 主干网络转变为专门的小镜头动作到动作生成器。我们使用 HumanML3D 作为源 T2M 数据集和 NTU RGB+D 120 的子集作为目标 HAR 域来验证 KineMIC，每个动作类仅随机选择 10 个样本。我们的方法可生成更加连贯的运动，提供强大的数据增强源，使准确度提高 23.1%。动画插图和补充材料可在（此 https URL）获取。

Title: EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing

• Authors: Wei Chow, Linfeng Li, Lingdong Kong, Zefeng Li, Qi Xu, Hang Song, Tian Ye, Xian Wang, Jinbin Bai, Shilin Xu, Xiangtai Li, Junting Pan, Shaoteng Liu, Ran Zhou, Tianshu Yang, Songhua Liu
• Subjects: cs.CV, cs.MM, eess.IV
• Abstract URL: https://arxiv.org/abs/2512.11715
• Pdf URL: https://arxiv.org/pdf/2512.11715
• Copy Paste: [[2512.11715]] EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing(https://arxiv.org/abs/2512.11715)
• Keywords: generative
• Abstract: Recent advances in diffusion models (DMs) have achieved exceptional visual quality in image editing tasks. However, the global denoising dynamics of DMs inherently conflate local editing targets with the full-image context, leading to unintended modifications in non-target regions. In this paper, we shift our attention beyond DMs and turn to Masked Generative Transformers (MGTs) as an alternative approach to tackle this challenge. By predicting multiple masked tokens rather than holistic refinement, MGTs exhibit a localized decoding paradigm that endows them with the inherent capacity to explicitly preserve non-relevant regions during the editing process. Building upon this insight, we introduce the first MGT-based image editing framework, termed EditMGT. We first demonstrate that MGT's cross-attention maps provide informative localization signals for localizing edit-relevant regions and devise a multi-layer attention consolidation scheme that refines these maps to achieve fine-grained and precise localization. On top of these adaptive localization results, we introduce region-hold sampling, which restricts token flipping within low-attention areas to suppress spurious edits, thereby confining modifications to the intended target regions and preserving the integrity of surrounding non-target areas. To train EditMGT, we construct CrispEdit-2M, a high-resolution dataset spanning seven diverse editing categories. Without introducing additional parameters, we adapt a pre-trained text-to-image MGT into an image editing model through attention injection. Extensive experiments across four standard benchmarks demonstrate that, with fewer than 1B parameters, our model achieves similarity performance while enabling 6 times faster editing. Moreover, it delivers comparable or superior editing quality, with improvements of 3.6% and 17.6% on style change and style transfer tasks, respectively.
• 摘要：扩散模型 (DM) 的最新进展在图像编辑任务中实现了卓越的视觉质量。然而，DM 的全局去噪动态本质上将局部编辑目标与全图像上下文混为一谈，导致非目标区域发生意外修改。在本文中，我们将注意力从 DM 转移到 Masked Generative Transformers (MGT)，作为应对这一挑战的替代方法。通过预测多个屏蔽标记而不是整体细化，MGT 展示了一种本地化解码范例，赋予它们在编辑过程中显式保留不相关区域的固有能力。基于这一见解，我们引入了第一个基于 MGT 的图像编辑框架，称为 EditMGT。我们首先证明 MGT 的交叉注意力图为本地化编辑相关区域提供了信息丰富的定位信号，并设计了一种多层注意力整合方案来细化这些图以实现细粒度和精确的定位。除了这些自适应定位结果之外，我们还引入了区域保持采样，它限制低关注区域内的令牌翻转以抑制虚假编辑，从而限制对预期目标区域的修改并保持周围非目标区域的完整性。为了训练 EditMGT，我们构建了 CrispEdit-2M，这是一个涵盖七个不同编辑类别的高分辨率数据集。在不引入额外参数的情况下，我们通过注意力注入将预先训练的文本到图像 MGT 调整为图像编辑模型。跨越四个标准基准的大量实验表明，我们的模型在参数少于 1B 的情况下实现了相似性能，同时实现了 6 倍的编辑速度。此外，它还提供相当或更高的编辑质量，风格更改和风格转移任务分别提高了 3.6% 和 17.6%。

Title: Referring Change Detection in Remote Sensing Imagery

• Authors: Yilmaz Korkmaz, Jay N. Paranjape, Celso M. de Melo, Vishal M. Patel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11719
• Pdf URL: https://arxiv.org/pdf/2512.11719
• Copy Paste: [[2512.11719]] Referring Change Detection in Remote Sensing Imagery(https://arxiv.org/abs/2512.11719)
• Keywords: generation
• Abstract: Change detection in remote sensing imagery is essential for applications such as urban planning, environmental monitoring, and disaster management. Traditional change detection methods typically identify all changes between two temporal images without distinguishing the types of transitions, which can lead to results that may not align with specific user needs. Although semantic change detection methods have attempted to address this by categorizing changes into predefined classes, these methods rely on rigid class definitions and fixed model architectures, making it difficult to mix datasets with different label sets or reuse models across tasks, as the output channels are tightly coupled with the number and type of semantic classes. To overcome these limitations, we introduce Referring Change Detection (RCD), which leverages natural language prompts to detect specific classes of changes in remote sensing images. By integrating language understanding with visual analysis, our approach allows users to specify the exact type of change they are interested in. However, training models for RCD is challenging due to the limited availability of annotated data and severe class imbalance in existing datasets. To address this, we propose a two-stage framework consisting of (I) \textbf{RCDNet}, a cross-modal fusion network designed for referring change detection, and (II) \textbf{RCDGen}, a diffusion-based synthetic data generation pipeline that produces realistic post-change images and change maps for a specified category using only pre-change image, without relying on semantic segmentation masks and thereby significantly lowering the barrier to scalable data creation. Experiments across multiple datasets show that our framework enables scalable and targeted change detection. Project website is here: this https URL.
• 摘要：遥感图像中的变化检测对于城市规划、环境监测和灾害管理等应用至关重要。传统的变化检测方法通常识别两个时间图像之间的所有变化，而不区分过渡的类型，这可能导致结果可能与特定用户需求不符。尽管语义变化检测方法试图通过将变化分类为预定义的类来解决这个问题，但这些方法依赖于严格的类定义和固定的模型架构，使得很难将数据集与不同的标签集混合或跨任务重用模型，因为输出通道与语义类的数量和类型紧密耦合。为了克服这些限制，我们引入了参考变化检测（RCD），它利用自然语言提示来检测遥感图像中特定类别的变化。通过将语言理解与视觉分析相结合，我们的方法允许用户指定他们感兴趣的确切变化类型。然而，由于注释数据的可用性有限以及现有数据集中严重的类别不平衡，RCD 的训练模型具有挑战性。为了解决这个问题，我们提出了一个两阶段框架，其中包括（I）\textbf{RCDNet}，一个为引用变化检测而设计的跨模态融合网络，以及（II）\textbf{RCDGen}，一个基于扩散的合成数据生成管道，仅使用变化前图像生成真实的变化后图像和指定类别的变化图，而不依赖语义分割掩模，从而显着降低可扩展数据创建的障碍。跨多个数据集的实验表明，我们的框架能够实现可扩展且有针对性的变化检测。项目网站在这里：https URL。

Title: Reframing Music-Driven 2D Dance Pose Generation as Multi-Channel Image Generation

• Authors: Yan Zhang, Han Zou, Lincong Feng, Cong Xie, Ruiqi Yu, Zhenpeng Zhan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11720
• Pdf URL: https://arxiv.org/pdf/2512.11720
• Copy Paste: [[2512.11720]] Reframing Music-Driven 2D Dance Pose Generation as Multi-Channel Image Generation(https://arxiv.org/abs/2512.11720)
• Keywords: generation
• Abstract: Recent pose-to-video models can translate 2D pose sequences into photorealistic, identity-preserving dance videos, so the key challenge is to generate temporally coherent, rhythm-aligned 2D poses from music, especially under complex, high-variance in-the-wild distributions. We address this by reframing music-to-dance generation as a music-token-conditioned multi-channel image synthesis problem: 2D pose sequences are encoded as one-hot images, compressed by a pretrained image VAE, and modeled with a DiT-style backbone, allowing us to inherit architectural and training advances from modern text-to-image models and better capture high-variance 2D pose distributions. On top of this formulation, we introduce (i) a time-shared temporal indexing scheme that explicitly synchronizes music tokens and pose latents over time and (ii) a reference-pose conditioning strategy that preserves subject-specific body proportions and on-screen scale while enabling long-horizon segment-and-stitch generation. Experiments on a large in-the-wild 2D dance corpus and the calibrated AIST++2D benchmark show consistent improvements over representative music-to-dance methods in pose- and video-space metrics and human preference, and ablations validate the contributions of the representation, temporal indexing, and reference conditioning. See supplementary videos at this https URL
• 摘要：最近的姿势到视频模型可以将 2D 姿势序列转换为逼真、保留身份的舞蹈视频，因此关键挑战是从音乐中生成时间连贯、节奏对齐的 2D 姿势，尤其是在复杂、高方差的野外分布下。我们通过将音乐到舞蹈的生成重新定义为音乐令牌条件的多通道图像合成问题来解决这个问题：2D 姿势序列被编码为单热图像，由预训练图像 VAE 压缩，并使用 DiT 风格的主干进行建模，使我们能够继承现代文本到图像模型的架构和训练进步，并更好地捕获高方差的 2D 姿势分布。在此公式的基础上，我们引入了（i）一种分时时间索引方案，该方案可以随着时间的推移显式地同步音乐标记和潜在姿势；（ii）一种参考姿势调节策略，可以保留特定于主题的身体比例和屏幕比例，同时实现长视野分段和缝合生成。在大型野外 2D 舞蹈语料库和校准的 AIST++2D 基准上进行的实验表明，在姿势和视频空间指标以及人类偏好方面，与代表性的音乐舞蹈方法相比，有一致的改进，并且消融验证了表示、时间索引和参考调节的贡献。在此 https URL 观看补充视频

Title: SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder

• Authors: Minglei Shi, Haolin Wang, Borui Zhang, Wenzhao Zheng, Bohan Zeng, Ziyang Yuan, Xiaoshi Wu, Yuanxing Zhang, Huan Yang, Xintao Wang, Pengfei Wan, Kun Gai, Jie Zhou, Jiwen Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11749
• Pdf URL: https://arxiv.org/pdf/2512.11749
• Copy Paste: [[2512.11749]] SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder(https://arxiv.org/abs/2512.11749)
• Keywords: generation, generative
• Abstract: Visual generation grounded in Visual Foundation Model (VFM) representations offers a highly promising unified pathway for integrating visual understanding, perception, and generation. Despite this potential, training large-scale text-to-image diffusion models entirely within the VFM representation space remains largely unexplored. To bridge this gap, we scale the SVG (Self-supervised representations for Visual Generation) framework, proposing SVG-T2I to support high-quality text-to-image synthesis directly in the VFM feature domain. By leveraging a standard text-to-image diffusion pipeline, SVG-T2I achieves competitive performance, reaching 0.75 on GenEval and 85.78 on DPG-Bench. This performance validates the intrinsic representational power of VFMs for generative tasks. We fully open-source the project, including the autoencoder and generation model, together with their training, inference, evaluation pipelines, and pre-trained weights, to facilitate further research in representation-driven visual generation.
• 摘要：基于视觉基础模型 (VFM) 表示的视觉生成为集成视觉理解、感知和生成提供了一种非常有前途的统一途径。尽管有这种潜力，但完全在 VFM 表示空间内训练大规模文本到图像扩散模型仍然很大程度上尚未探索。为了弥补这一差距，我们扩展了 SVG（视觉生成的自监督表示）框架，提出 SVG-T2I 来支持直接在 VFM 特征域中进行高质量文本到图像的合成。通过利用标准的文本到图像扩散管道，SVG-T2I 实现了具有竞争力的性能，在 GenEval 上达到 0.75，在 DPG-Bench 上达到 85.78。这种性能验证了 VFM 对于生成任务的内在表征能力。我们完全开源该项目，包括自动编码器和生成模型，以及它们的训练、推理、评估管道和预训练权重，以促进表示驱动的视觉生成的进一步研究。

Title: MatAnyone 2: Scaling Video Matting via a Learned Quality Evaluator

• Authors: Peiqing Yang, Shangchen Zhou, Kai Hao, Qingyi Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11782
• Pdf URL: https://arxiv.org/pdf/2512.11782
• Copy Paste: [[2512.11782]] MatAnyone 2: Scaling Video Matting via a Learned Quality Evaluator(https://arxiv.org/abs/2512.11782)
• Keywords: quality assessment
• Abstract: Video matting remains limited by the scale and realism of existing datasets. While leveraging segmentation data can enhance semantic stability, the lack of effective boundary supervision often leads to segmentation-like mattes lacking fine details. To this end, we introduce a learned Matting Quality Evaluator (MQE) that assesses semantic and boundary quality of alpha mattes without ground truth. It produces a pixel-wise evaluation map that identifies reliable and erroneous regions, enabling fine-grained quality assessment. The MQE scales up video matting in two ways: (1) as an online matting-quality feedback during training to suppress erroneous regions, providing comprehensive supervision, and (2) as an offline selection module for data curation, improving annotation quality by combining the strengths of leading video and image matting models. This process allows us to build a large-scale real-world video matting dataset, VMReal, containing 28K clips and 2.4M frames. To handle large appearance variations in long videos, we introduce a reference-frame training strategy that incorporates long-range frames beyond the local window for effective training. Our MatAnyone 2 achieves state-of-the-art performance on both synthetic and real-world benchmarks, surpassing prior methods across all metrics.
• 摘要：视频抠图仍然受到现有数据集的规模和真实性的限制。虽然利用分割数据可以增强语义稳定性，但缺乏有效的边界监督通常会导致类似分割的遮罩缺乏精细细节。为此，我们引入了一个学习的抠图质量评估器（MQE），它可以在没有地面事实的情况下评估 alpha 遮罩的语义和边界质量。它生成一个像素级的评估图，可以识别可靠和错误的区域，从而实现细粒度的质量评估。 MQE 通过两种方式扩展视频抠图：(1) 作为训练期间的在线抠图质量反馈，以抑制错误区域，提供全面的监督；(2) 作为数据管理的离线选择模块，通过结合领先视频和图像抠图模型的优势来提高注释质量。这个过程使我们能够构建一个大规模的真实世界视频抠图数据集 VMReal，其中包含 28K 剪辑和 240 万帧。为了处理长视频中较大的外观变化，我们引入了一种参考帧训练策略，该策略结合了本地窗口之外的远程帧以进行有效的训练。我们的 MatAnyone 2 在综合基准和实际基准上都实现了最先进的性能，在所有指标上都超越了之前的方法。

Title: Structure From Tracking: Distilling Structure-Preserving Motion for Video Generation

• Authors: Yang Fei, George Stoica, Jingyuan Liu, Qifeng Chen, Ranjay Krishna, Xiaojuan Wang, Benlin Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11792
• Pdf URL: https://arxiv.org/pdf/2512.11792
• Copy Paste: [[2512.11792]] Structure From Tracking: Distilling Structure-Preserving Motion for Video Generation(https://arxiv.org/abs/2512.11792)
• Keywords: generation
• Abstract: Reality is a dance between rigid constraints and deformable structures. For video models, that means generating motion that preserves fidelity as well as structure. Despite progress in diffusion models, producing realistic structure-preserving motion remains challenging, especially for articulated and deformable objects such as humans and animals. Scaling training data alone, so far, has failed to resolve physically implausible transitions. Existing approaches rely on conditioning with noisy motion representations, such as optical flow or skeletons extracted using an external imperfect model. To address these challenges, we introduce an algorithm to distill structure-preserving motion priors from an autoregressive video tracking model (SAM2) into a bidirectional video diffusion model (CogVideoX). With our method, we train SAM2VideoX, which contains two innovations: (1) a bidirectional feature fusion module that extracts global structure-preserving motion priors from a recurrent model like SAM2; (2) a Local Gram Flow loss that aligns how local features move together. Experiments on VBench and in human studies show that SAM2VideoX delivers consistent gains (+2.60\% on VBench, 21-22\% lower FVD, and 71.4\% human preference) over prior baselines. Specifically, on VBench, we achieve 95.51\%, surpassing REPA (92.91\%) by 2.60\%, and reduce FVD to 360.57, a 21.20\% and 22.46\% improvement over REPA- and LoRA-finetuning, respectively. The project website can be found at this https URL .
• 摘要：现实是刚性约束和可变形结构之间的舞蹈。对于视频模型，这意味着生成保持保真度和结构的运动。尽管扩散模型取得了进展，但产生真实的结构保持运动仍然具有挑战性，特别是对于人类和动物等铰接和可变形物体。到目前为止，仅靠缩放训练数据未能解决物理上不合理的转变。现有方法依赖于噪声运动表示的调节，例如使用外部不完美模型提取的光流或骨架。为了应对这些挑战，我们引入了一种算法，将自回归视频跟踪模型 (SAM2) 中的结构保持运动先验提取为双向视频扩散模型 (CogVideoX)。通过我们的方法，我们训练了 SAM2VideoX，它包含两项创新：（1）双向特征融合模块，从 SAM2 这样的循环模型中提取全局结构保持运动先验； (2) 局部 Gram Flow 损失，用于调整局部特征如何一起移动。 VBench 和人类研究中的实验表明，与之前的基线相比，SAM2VideoX 提供了一致的增益（在 VBench 上增加 2.60%，FVD 降低 21-22%，人类偏好降低 71.4%）。具体来说，在 VBench 上，我们达到了 95.51%，比 REPA (92.91%) 提高了 2.60%，并将 FVD 降低到 360.57，分别比 REPA 和 LoRA 微调提高了 21.20% 和 22.46%。可以在此 https URL 找到该项目网站。

Title: V-RGBX: Video Editing with Accurate Controls over Intrinsic Properties

• Authors: Ye Fang, Tong Wu, Valentin Deschaintre, Duygu Ceylan, Iliyan Georgiev, Chun-Hao Paul Huang, Yiwei Hu, Xuelin Chen, Tuanfeng Yang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.11799
• Pdf URL: https://arxiv.org/pdf/2512.11799
• Copy Paste: [[2512.11799]] V-RGBX: Video Editing with Accurate Controls over Intrinsic Properties(https://arxiv.org/abs/2512.11799)
• Keywords: generation
• Abstract: Large-scale video generation models have shown remarkable potential in modeling photorealistic appearance and lighting interactions in real-world scenes. However, a closed-loop framework that jointly understands intrinsic scene properties (e.g., albedo, normal, material, and irradiance), leverages them for video synthesis, and supports editable intrinsic representations remains unexplored. We present V-RGBX, the first end-to-end framework for intrinsic-aware video editing. V-RGBX unifies three key capabilities: (1) video inverse rendering into intrinsic channels, (2) photorealistic video synthesis from these intrinsic representations, and (3) keyframe-based video editing conditioned on intrinsic channels. At the core of V-RGBX is an interleaved conditioning mechanism that enables intuitive, physically grounded video editing through user-selected keyframes, supporting flexible manipulation of any intrinsic modality. Extensive qualitative and quantitative results show that V-RGBX produces temporally consistent, photorealistic videos while propagating keyframe edits across sequences in a physically plausible manner. We demonstrate its effectiveness in diverse applications, including object appearance editing and scene-level relighting, surpassing the performance of prior methods.
• 摘要：大规模视频生成模型在模拟真实场景中的真实外观和灯光交互方面表现出了巨大的潜力。然而，共同理解内在场景属性（例如反照率、法线、材质和辐照度）、利用它们进行视频合成并支持可编辑内在表示的闭环框架仍未被探索。我们推出了 V-RGBX，这是第一个用于内在感知视频编辑的端到端框架。 V-RGBX 统一了三个关键功能：(1) 视频逆渲染到内在通道中，(2) 从这些内在表示进行逼真的视频合成，以及 (3) 以内在通道为条件的基于关键帧的视频编辑。 V-RGBX 的核心是交错调节机制，可通过用户选择的关键帧实现直观、基于物理的视频编辑，支持对任何固有模态的灵活操作。广泛的定性和定量结果表明，V-RGBX 可以生成时间一致、逼真的视频，同时以物理上合理的方式跨序列传播关键帧编辑。我们展示了其在各种应用中的有效性，包括对象外观编辑和场景级重新照明，超越了先前方法的性能。