2025-12-22

Title: Lights, Camera, Consistency: A Multistage Pipeline for Character-Stable AI Video Stories

• Authors: Chayan Jain, Rishant Sharma, Archit Garg, Ishan Bhanuka, Pratik Narang, Dhruv Kumar
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16954
• Pdf URL: https://arxiv.org/pdf/2512.16954
• Copy Paste: [[2512.16954]] Lights, Camera, Consistency: A Multistage Pipeline for Character-Stable AI Video Stories(https://arxiv.org/abs/2512.16954)
• Keywords: generation
• Abstract: Generating long, cohesive video stories with consistent characters is a significant challenge for current text-to-video AI. We introduce a method that approaches video generation in a filmmaker-like manner. Instead of creating a video in one step, our proposed pipeline first uses a large language model to generate a detailed production script. This script guides a text-to-image model in creating consistent visuals for each character, which then serve as anchors for a video generation model to synthesize each scene individually. Our baseline comparisons validate the necessity of this multi-stage decomposition; specifically, we observe that removing the visual anchoring mechanism results in a catastrophic drop in character consistency scores (from 7.99 to 0.55), confirming that visual priors are essential for identity preservation. Furthermore, we analyze cultural disparities in current models, revealing distinct biases in subject consistency and dynamic degree between Indian vs Western-themed generations.
• 摘要：生成具有一致字符的长而有凝聚力的视频故事是当前文本转视频人工智能面临的重大挑战。我们介绍了一种以类似电影制作人的方式生成视频的方法。我们提出的管道不是一步创建视频，而是首先使用大型语言模型来生成详细的制作脚本。该脚本指导文本到图像模型为每个角色创建一致的视觉效果，然后作为视频生成模型的锚点来单独合成每个场景。我们的基线比较验证了这种多阶段分解的必要性；具体来说，我们观察到消除视觉锚定机制会导致字符一致性得分灾难性下降（从 7.99 降至 0.55），这证实了视觉先验对于身份保存至关重要。此外，我们分析了当前模型中的文化差异，揭示了印度与西方主题世代之间在主题一致性和动态程度方面的明显偏差。

Title: Compression is Routing: Reconstruction Error as an Intrinsic Signal for Modular Language Models

• Authors: Zhongpan Tang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.16963
• Pdf URL: https://arxiv.org/pdf/2512.16963
• Copy Paste: [[2512.16963]] Compression is Routing: Reconstruction Error as an Intrinsic Signal for Modular Language Models(https://arxiv.org/abs/2512.16963)
• Keywords: generation
• Abstract: Current Large Language Models (LLMs) face three major challenges: context length limitations, high inference costs, and catastrophic forgetting during continual learning. While Mixture-of-Experts (MoE) architectures mitigate some of these conflicts, their routing mechanisms typically rely on explicitly trained auxiliary classifiers. This not only increases system complexity but also often lacks interpretability when handling mixed-domain inputs. Building upon the premise that ``Compression is Intelligence,'' this paper proposes a novel architectural philosophy: \textbf{``Compression is Routing.''} We trained an 87M-parameter end-to-end Transformer Autoencoder, achieving a \textbf{64x sequence length compression} (compressing 512 tokens into 8 latent vectors). Experimental results demonstrate that this compressor possesses extreme domain discriminative capability: it achieves a reconstruction accuracy of \textbf{99.47\%} on the in-domain (code) validation set; accuracy drops sharply to \textbf{47.76\%} on a semi-out-of-distribution domain (Wiki text); and further plummets to just \textbf{0.57\%} on a fully out-of-distribution domain (random sequences). This extreme and systematic performance discrepancy establishes the validity of reconstruction error as an \textbf{Intrinsic Distribution Fingerprint}. Based on this, we propose that expert modules can be automatically scheduled using reconstruction residuals directly, without the need for explicit gating networks. This mechanism offers excellent scalability. Furthermore, this architecture provides a new perspective on ``VRAM compression'' for handling ultra-long contexts. This report aims to verify the physical validity of this foundational architecture, offering a new research perspective for the next generation of scalable modular neural networks.
• 摘要：当前的大型语言模型（LLM）面临三大挑战：上下文长度限制、推理成本高以及持续学习过程中的灾难性遗忘。虽然专家混合 (MoE) 架构缓解了其中一些冲突，但它们的路由机制通常依赖于显式训练的辅助分类器。这不仅增加了系统的复杂性，而且在处理混合域输入时通常缺乏可解释性。基于“压缩就是智能”的前提，本文提出了一种新颖的架构理念：\textbf{“压缩就是路由”。我们训练了一个 87M 参数的端到端 Transformer 自动编码器，实现了 \textbf{64 倍序列长度压缩}（将 512 个令牌压缩为 8 个潜在向量）。实验结果表明，该压缩器具有极端的域判别能力：在域内（代码）验证集上实现了 \textbf{99.47\%} 的重建精度；在半分布外域（Wiki 文本）上，准确率急剧下降至 \textbf{47.76\%}；并在完全超出分布的域（随机序列）上进一步下降到 \textbf{0.57\%}。这种极端和系统的性能差异将重建误差的有效性确立为 \textbf{内在分布指纹}。基于此，我们提出可以直接使用重建残差自动调度专家模块，而不需要显式的门控网络。该机制提供了出色的可扩展性。此外，该架构为处理超长上下文的“VRAM 压缩”提供了新的视角。本报告旨在验证这一基础架构的物理有效性，为下一代可扩展模块化神经网络提供新的研究视角。

Title: Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation

• Authors: Min-Jung Kim, Jeongho Kim, Hoiyeong Jin, Junha Hyung, Jaegul Choo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17040
• Pdf URL: https://arxiv.org/pdf/2512.17040
• Copy Paste: [[2512.17040]] Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation(https://arxiv.org/abs/2512.17040)
• Keywords: generation
• Abstract: Recent progress in video diffusion models has spurred growing interest in camera-controlled novel-view video generation for dynamic scenes, aiming to provide creators with cinematic camera control capabilities in post-production. A key challenge in camera-controlled video generation is ensuring fidelity to the specified camera pose, while maintaining view consistency and reasoning about occluded geometry from limited observations. To address this, existing methods either train trajectory-conditioned video generation model on trajectory-video pair dataset, or estimate depth from the input video to reproject it along a target trajectory and generate the unprojected regions. Nevertheless, existing methods struggle to generate camera-pose-faithful, high-quality videos for two main reasons: (1) reprojection-based approaches are highly susceptible to errors caused by inaccurate depth estimation; and (2) the limited diversity of camera trajectories in existing datasets restricts learned models. To address these limitations, we present InfCam, a depth-free, camera-controlled video-to-video generation framework with high pose fidelity. The framework integrates two key components: (1) infinite homography warping, which encodes 3D camera rotations directly within the 2D latent space of a video diffusion model. Conditioning on this noise-free rotational information, the residual parallax term is predicted through end-to-end training to achieve high camera-pose fidelity; and (2) a data augmentation pipeline that transforms existing synthetic multiview datasets into sequences with diverse trajectories and focal lengths. Experimental results demonstrate that InfCam outperforms baseline methods in camera-pose accuracy and visual fidelity, generalizing well from synthetic to real-world data. Link to our project page:this https URL
• 摘要：视频扩散模型的最新进展激发了人们对动态场景的摄像机控制新视点视频生成的兴趣日益浓厚，旨在为创作者在后期制作中提供电影摄像机控制功能。相机控制视频生成的一个关键挑战是确保指定相机姿势的保真度，同时保持视图一致性并从有限的观察中推理出遮挡的几何形状。为了解决这个问题，现有的方法要么在轨迹视频对数据集上训练轨迹条件视频生成模型，要么估计输入视频的深度以沿着目标轨迹重新投影它并生成未投影的区域。然而，现有的方法很难生成忠实于相机姿势的高质量视频，主要原因有两个：（1）基于重投影的方法很容易受到不准确的深度估计引起的错误的影响； （2）现有数据集中相机轨迹的有限多样性限制了学习模型。为了解决这些限制，我们提出了 InfCam，一种无深度、相机控制的视频到视频生成框架，具有高姿态保真度。该框架集成了两个关键组件：(1) 无限单应性变形，它直接在视频扩散模型的 2D 潜在空间内对 3D 相机旋转进行编码。以这种无噪声旋转信息为条件，通过端到端训练来预测残余视差项，以实现高相机姿态保真度； (2) 数据增强管道，将现有的合成多视图数据集转换为具有不同轨迹和焦距的序列。实验结果表明，InfCam 在相机姿态准确性和视觉保真度方面优于基线方法，可以很好地从合成数据推广到现实世界数据。链接到我们的项目页面：这个 https URL

Title: SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples

• Authors: Haoye Lu, Yaoliang Yu, Darren Ho
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.17051
• Pdf URL: https://arxiv.org/pdf/2512.17051
• Copy Paste: [[2512.17051]] SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples(https://arxiv.org/abs/2512.17051)
• Keywords: restoration
• Abstract: In many real-world scenarios, obtaining fully observed samples is prohibitively expensive or even infeasible, while partial and noisy observations are comparatively easy to collect. In this work, we study distribution restoration with abundant noisy samples, assuming the corruption process is available as a black-box generator. We show that this task can be framed as a one-sided entropic optimal transport problem and solved via an EM-like algorithm. We further provide a test criterion to determine whether the true underlying distribution is recoverable under per-sample information loss, and show that in otherwise unrecoverable cases, a small number of clean samples can render the distribution largely recoverable. Building on these insights, we introduce SFBD-OMNI, a bridge model-based framework that maps corrupted sample distributions to the ground-truth distribution. Our method generalizes Stochastic Forward-Backward Deconvolution (SFBD; Lu et al., 2025) to handle arbitrary measurement models beyond Gaussian corruption. Experiments across benchmark datasets and diverse measurement settings demonstrate significant improvements in both qualitative and quantitative performance.
• 摘要：在许多现实场景中，获得完全观察到的样本非常昂贵甚至不可行，而部分和有噪声的观察则相对容易收集。在这项工作中，我们研究了具有大量噪声样本的分布恢复，假设腐败过程可用作黑盒生成器。我们表明，这一任务可以被构建为单边熵最优传输问题，并通过类似 EM 的算法来解决。我们进一步提供了一个测试标准来确定真实的基础分布在每个样本信息丢失的情况下是否可恢复，并表明在其他不可恢复的情况下，少量的干净样本可以使分布在很大程度上可恢复。基于这些见解，我们引入了 SFBD-OMNI，这是一个基于桥接模型的框架，可将损坏的样本分布映射到真实分布。我们的方法概括了随机前向-后向反卷积（SFBD；Lu et al., 2025）来处理高斯腐败之外的任意测量模型。跨基准数据集和不同测量设置的实验证明了定性和定量性能的显着改进。

Title: Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation

• Authors: Zhenyu Liu, Yunzhen Liu, Zehao Fan, Garrett Gagnon, Yayue Hou, Nan Wu, Yangwook Kang, Liu Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.17073
• Pdf URL: https://arxiv.org/pdf/2512.17073
• Copy Paste: [[2512.17073]] Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation(https://arxiv.org/abs/2512.17073)
• Keywords: restoration
• Abstract: Mixture-of-Experts (MoE) models scale capacity via sparse activation but stress memory and bandwidth. Offloading alleviates GPU memory by fetching experts on demand, yet token-level routing causes irregular transfers that make inference I/O-bound. Static uniform quantization reduces traffic but degrades accuracy under aggressive compression by ignoring expert heterogeneity. We present Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation, which performs router-guided precision restoration using precomputed low-rank compensators. At inference time, our method transfers compact low-rank factors with Top-n (n
• 摘要：专家混合 (MoE) 模型通过稀疏激活来扩展容量，但强调内存和带宽。卸载通过按需获取专家来减少 GPU 内存，但令牌级路由会导致不规则传输，从而导致推理 I/O 受限。静态均匀量化会减少流量，但会通过忽略专家异构性而降低激进压缩下的准确性。我们通过低秩补偿提出了带宽高效的自适应专家混合，它使用预先计算的低秩补偿器执行路由器引导的精度恢复。在推理时，我们的方法将紧凑的低等级因子与每个代币的前 n (n

Title: The Effect of Negation on CLIP in Medical Imaging: Limitations of Contrastive Language-Image Pretraining

• Authors: Jasmine Vu, Shivanand Sheshappanavar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.17121
• Pdf URL: https://arxiv.org/pdf/2512.17121
• Copy Paste: [[2512.17121]] The Effect of Negation on CLIP in Medical Imaging: Limitations of Contrastive Language-Image Pretraining(https://arxiv.org/abs/2512.17121)
• Keywords: generation
• Abstract: Large vision-language models like CLIP are increasingly used in medical imaging tasks due to their ability to align images and text without the need for extensive labeled data. This makes them particularly useful for applications like image retrieval, report generation, and classification in clinical settings. A potential issue to this approach is that CLIP-based models often under perform when interpreting negated phrases, which is especially problematic in the context of medical diagnosing. In this study, we evaluate the Stanford AIMI CheXagent model on its ability to correctly retrieve chest X-ray images using prompts with and without negation. The goal of this project is to understand where this model fails and then use it as a base model to improve its retrieval accuracy by fine tuning methods outlined in previous work. Results from this study show improvement in handling of negation in the CLIP model with a slight decrease in accuracy of positive prompt evaluation. Alongside retrieval accuracy, we examined internal model behavior through token attribution, t-SNE projection, and attention-head ablation to better characterize how each fine tuning approach reshaped the text encoders representation of negated clinical language. Through this work, we hope to better understand the internal behavior of CLIP and improve its handling of negation using clinically relevant language for improving its reliability in medical AI devices.
• 摘要：像 CLIP 这样的大型视觉语言模型越来越多地用于医学成像任务，因为它们能够在不需要大量标记数据的情况下对齐图像和文本。这使得它们对于临床环境中的图像检索、报告生成和分类等应用特别有用。这种方法的一个潜在问题是，基于 CLIP 的模型在解释否定短语时通常表现不佳，这在医学诊断的背景下尤其成问题。在这项研究中，我们评估了斯坦福 AIMI CheXagent 模型使用带否定和不带否定的提示正确检索胸部 X 射线图像的能力。该项目的目标是了解该模型在哪里失败，然后将其用作基础模型，通过先前工作中概述的微调方法来提高其检索准确性。这项研究的结果表明，CLIP 模型在处理否定方面有所改进，但积极提示评估的准确性略有下降。除了检索准确性之外，我们还通过标记归因、t-SNE 投影和注意力头消融检查了内部模型行为，以更好地表征每种微调方法如何重塑文本编码器对否定临床语言的表示。通过这项工作，我们希望更好地理解 CLIP 的内部行为，并使用临床相关语言改进其对否定的处理，以提高其在医疗人工智能设备中的可靠性。

Title: Text-Conditioned Background Generation for Editable Multi-Layer Documents

• Authors: Taewon Kang, Joseph K J, Chris Tensmeyer, Jihyung Kil, Wanrong Zhu, Ming C. Lin, Vlad I. Morariu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17151
• Pdf URL: https://arxiv.org/pdf/2512.17151
• Copy Paste: [[2512.17151]] Text-Conditioned Background Generation for Editable Multi-Layer Documents(https://arxiv.org/abs/2512.17151)
• Keywords: generation, generative
• Abstract: We present a framework for document-centric background generation with multi-page editing and thematic continuity. To ensure text regions remain readable, we employ a \emph{latent masking} formulation that softly attenuates updates in the diffusion space, inspired by smooth barrier functions in physics and numerical optimization. In addition, we introduce \emph{Automated Readability Optimization (ARO)}, which automatically places semi-transparent, rounded backing shapes behind text regions. ARO determines the minimal opacity needed to satisfy perceptual contrast standards (WCAG 2.2) relative to the underlying background, ensuring readability while maintaining aesthetic harmony without human intervention. Multi-page consistency is maintained through a summarization-and-instruction process, where each page is distilled into a compact representation that recursively guides subsequent generations. This design reflects how humans build continuity by retaining prior context, ensuring that visual motifs evolve coherently across an entire document. Our method further treats a document as a structured composition in which text, figures, and backgrounds are preserved or regenerated as separate layers, allowing targeted background editing without compromising readability. Finally, user-provided prompts allow stylistic adjustments in color and texture, balancing automated consistency with flexible customization. Our training-free framework produces visually coherent, text-preserving, and thematically aligned documents, bridging generative modeling with natural design workflows.
• 摘要：我们提出了一个以文档为中心的背景生成框架，具有多页面编辑和主题连续性。为了确保文本区域保持可读，我们采用了 \emph{潜在掩蔽} 公式，该公式受到物理和数值优化中平滑势垒函数的启发，可以轻柔地衰减扩散空间中的更新。此外，我们还引入了 \emph{自动可读性优化 (ARO)}，它会自动将半透明的圆形背景形状放置在文本区域后面。 ARO 确定满足相对于底层背景的感知对比度标准 (WCAG 2.2) 所需的最小不透明度，确保可读性，同时在无需人工干预的情况下保持美学和谐。多页面一致性是通过摘要和指令过程来维护的，其中每个页面都被提炼成一个紧凑的表示，递归地指导后代。这种设计反映了人类如何通过保留先前的上下文来建立连续性，确保视觉主题在整个文档中连贯地演变。我们的方法进一步将文档视为结构化组合，其中文本、图形和背景被保留或重新生成为单独的层，从而允许有针对性的背景编辑而不影响可读性。最后，用户提供的提示允许对颜色和纹理进行风格调整，平衡自动一致性与灵活定制。我们的免培训框架可生成视觉连贯、文本保留且主题一致的文档，将生成建模与自然设计工作流程联系起来。

Title: PhysFire-WM: A Physics-Informed World Model for Emulating Fire Spread Dynamics

• Authors: Nan Zhou, Huandong Wang, Jiahao Li, Yang Li, Xiao-Ping Zhang, Yong Li, Xinlei Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17152
• Pdf URL: https://arxiv.org/pdf/2512.17152
• Copy Paste: [[2512.17152]] PhysFire-WM: A Physics-Informed World Model for Emulating Fire Spread Dynamics(https://arxiv.org/abs/2512.17152)
• Keywords: generation
• Abstract: Fine-grained fire prediction plays a crucial role in emergency response. Infrared images and fire masks provide complementary thermal and boundary information, yet current methods are predominantly limited to binary mask modeling with inherent signal sparsity, failing to capture the complex dynamics of fire. While world models show promise in video generation, their physical inconsistencies pose significant challenges for fire forecasting. This paper introduces PhysFire-WM, a Physics-informed World Model for emulating Fire spread dynamics. Our approach internalizes combustion dynamics by encoding structured priors from a Physical Simulator to rectify physical discrepancies, coupled with a Cross-task Collaborative Training strategy (CC-Train) that alleviates the issue of limited information in mask-based modeling. Through parameter sharing and gradient coordination, CC-Train effectively integrates thermal radiation dynamics and spatial boundary delineation, enhancing both physical realism and geometric accuracy. Extensive experiments on a fine-grained multimodal fire dataset demonstrate the superior accuracy of PhysFire-WM in fire spread prediction. Validation underscores the importance of physical priors and cross-task collaboration, providing new insights for applying physics-informed world models to disaster prediction.
• 摘要：细粒度的火灾预测在应急响应中发挥着至关重要的作用。红外图像和火灾掩模提供了互补的热和边界信息，但当前的方法主要限于具有固有信号稀疏性的二元掩模建模，无法捕获火灾的复杂动态。虽然世界模型在视频生成方面显示出良好的前景，但它们的物理不一致给火灾预报带来了重大挑战。本文介绍了 PhysFire-WM，这是一种用于模拟火灾蔓延动力学的物理世界模型。我们的方法通过对物理模拟器的结构化先验进行编码来纠正物理差异，并结合跨任务协作训练策略（CC-Train）来内部化燃烧动力学，从而缓解基于掩模的建模中信息有限的问题。通过参数共享和梯度协调，CC-Train有效地整合了热辐射动力学和空间边界描绘，增强了物理真实感和几何精度。对细粒度多模式火灾数据集的大量实验证明了 PhysFire-WM 在火灾蔓延预测方面的卓越准确性。验证强调了物理先验和跨任务协作的重要性，为将基于物理的世界模型应用于灾害预测提供了新的见解。

Title: Can Synthetic Images Serve as Effective and Efficient Class Prototypes?

• Authors: Dianxing Shi, Dingjie Fu, Yuqiao Liu, Jun Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17160
• Pdf URL: https://arxiv.org/pdf/2512.17160
• Copy Paste: [[2512.17160]] Can Synthetic Images Serve as Effective and Efficient Class Prototypes?(https://arxiv.org/abs/2512.17160)
• Keywords: generation
• Abstract: Vision-Language Models (VLMs) have shown strong performance in zero-shot image classification tasks. However, existing methods, including Contrastive Language-Image Pre-training (CLIP), all rely on annotated text-to-image pairs for aligning visual and textual modalities. This dependency introduces substantial cost and accuracy requirement in preparing high-quality datasets. At the same time, processing data from two modes also requires dual-tower encoders for most models, which also hinders their lightweight. To address these limitations, we introduce a ``Contrastive Language-Image Pre-training via Large-Language-Model-based Generation (LGCLIP)" framework. LGCLIP leverages a Large Language Model (LLM) to generate class-specific prompts that guide a diffusion model in synthesizing reference images. Afterwards these generated images serve as visual prototypes, and the visual features of real images are extracted and compared with the visual features of these prototypes to achieve comparative prediction. By optimizing prompt generation through the LLM and employing only a visual encoder, LGCLIP remains lightweight and efficient. Crucially, our framework requires only class labels as input during whole experimental procedure, eliminating the need for manually annotated image-text pairs and extra pre-processing. Experimental results validate the feasibility and efficiency of LGCLIP, demonstrating great performance in zero-shot classification tasks and establishing a novel paradigm for classification.
• 摘要：视觉语言模型（VLM）在零样本图像分类任务中表现出了强大的性能。然而，现有的方法，包括对比语言图像预训练（CLIP），都依赖于带注释的文本到图像对来对齐视觉和文本模态。这种依赖性在准备高质量数据集时引入了巨大的成本和准确性要求。同时，大多数模型处理两种模式的数据也需要双塔编码器，这也阻碍了它们的轻量化。为了解决这些限制，我们引入了“通过基于大语言模型的生成进行对比语言图像预训练（LGCLIP）”框架。LGCLIP利用大语言模型（LLM）来生成特定于类别的提示，指导扩散模型合成参考图像。然后，这些生成的图像作为视觉原型，提取真实图像的视觉特征并与这些原型的视觉特征进行比较，以实现比较预测。通过LLM优化提示生成并且仅使用视觉编码器，LGCLIP 保持轻量级和高效。至关重要的是，我们的框架在整个实验过程中只需要类标签作为输入，无需手动注释图像文本对和额外的预处理。实验结果验证了 LGCLIP 的可行性和效率，展示了零样本分类任务的出色性能，并建立了一种新颖的分类范例。

Title: Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs

• Authors: Rujiao Long, Yang Li, Xingyao Zhang, Weixun Wang, Tianqianjin Lin, Xi Zhao, Yuchi Xu, Wenbo Su, Junchi Yan, Bo Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17206
• Pdf URL: https://arxiv.org/pdf/2512.17206
• Copy Paste: [[2512.17206]] Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs(https://arxiv.org/abs/2512.17206)
• Keywords: generation
• Abstract: Exploration capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models, as stochastic sampling often yields redundant reasoning paths with little high-level diversity. This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a stochastic latent variable for strategic contextualization, guiding its internal planning prior to token generation. This latent context is inferred from the mean-pooled embedding of a question-answer pair via a variational autoencoder (VAE), where each sampled latent potentially encodes a distinct reasoning context. During inference, a sampled latent is decoded into learnable token prefixes and prepended to the input prompt, modulating the model's internal reasoning trajectory. In this way, the model performs internal sampling over reasoning strategies prior to output generation, which shapes the style and structure of the entire response sequence. A brief supervised fine-tuning (SFT) warm-up phase allows the model to adapt to this latent conditioning. Within RL optimization, Reasoning Palette facilitates structured exploration by enabling on-demand injection for diverse reasoning modes, significantly enhancing exploration efficiency and sustained learning capability. Experiments across multiple reasoning benchmarks demonstrate that our method enables interpretable and controllable control over the (vision-) language model's strategic behavior, thereby achieving consistent performance gains over standard RL methods.
• 摘要：探索能力决定了大型（视觉）语言模型的推理时间性能和强化学习（RL）训练，因为随机采样通常会产生几乎没有高级多样性的冗余推理路径。本文提出了 Reasoning Palette，这是一种新颖的潜在调制框架，赋予模型用于战略情​​境化的随机潜在变量，在代币生成之前指导其内部规划。该潜在上下文是通过变分自动编码器（VAE）从问答对的平均池嵌入推断出来的，其中每个采样的潜在上下文可能编码一个不同的推理上下文。在推理过程中，采样的潜在变量被解码为可学习的标记前缀，并添加到输入提示之前，从而调整模型的内部推理轨迹。通过这种方式，模型在输出生成之前对推理策略执行内部采样，从而塑造整个响应序列的风格和结构。短暂的监督微调 (SFT) 预热阶段使模型能够适应这种潜在的调节。在强化学习优化中，推理调色板通过按需注入多种推理模式来促进结构化探索，显着提高探索效率和持续学习能力。跨多个推理基准的实验表明，我们的方法能够对（视觉）语言模型的策略行为进行可解释和可控的控制，从而实现比标准 RL 方法一致的性能增益。

Title: Mitty: Diffusion-based Human-to-Robot Video Generation

• Authors: Yiren Song, Cheng Liu, Weijia Mao, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17253
• Pdf URL: https://arxiv.org/pdf/2512.17253
• Copy Paste: [[2512.17253]] Mitty: Diffusion-based Human-to-Robot Video Generation(https://arxiv.org/abs/2512.17253)
• Keywords: generation
• Abstract: Learning directly from human demonstration videos is a key milestone toward scalable and generalizable robot learning. Yet existing methods rely on intermediate representations such as keypoints or trajectories, introducing information loss and cumulative errors that harm temporal and visual consistency. We present Mitty, a Diffusion Transformer that enables video In-Context Learning for end-to-end Human2Robot video generation. Built on a pretrained video diffusion model, Mitty leverages strong visual-temporal priors to translate human demonstrations into robot-execution videos without action labels or intermediate abstractions. Demonstration videos are compressed into condition tokens and fused with robot denoising tokens through bidirectional attention during diffusion. To mitigate paired-data scarcity, we also develop an automatic synthesis pipeline that produces high-quality human-robot pairs from large egocentric datasets. Experiments on Human2Robot and EPIC-Kitchens show that Mitty delivers state-of-the-art results, strong generalization to unseen environments, and new insights for scalable robot learning from human observations.
• 摘要：直接从人类演示视频中学习是迈向可扩展和通用机器人学习的一个重要里程碑。然而，现有方法依赖于关键点或轨迹等中间表示，从而引入信息丢失和累积错误，从而损害时间和视觉一致性。我们推出了 Mitty，一种扩散变压器，可实现视频情境学习以实现端到端 Human2Robot 视频生成。 Mitty 基于预训练的视频扩散模型构建，利用强大的视觉时间先验将人类演示转换为机器人执行视频，而无需动作标签或中间抽象。演示视频被压缩为条件标记，并通过扩散过程中的双向注意力与机器人去噪标记融合。为了缓解配对数据的稀缺性，我们还开发了一种自动合成管道，可以从大型以自我为中心的数据集中生成高质量的人机对。 Human2Robot 和 EPIC-Kitchens 上的实验表明，Mitty 提供了最先进的结果、对未见过的环境的强大泛化能力，以及从人类观察中进行可扩展机器人学习的新见解。

Title: Understanding Generalization in Role-Playing Models via Information Theory

• Authors: Yongqi Li, Hao Lang, Fei Huang, Tieyun Qian, Yongbin Li
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2512.17270
• Pdf URL: https://arxiv.org/pdf/2512.17270
• Copy Paste: [[2512.17270]] Understanding Generalization in Role-Playing Models via Information Theory(https://arxiv.org/abs/2512.17270)
• Keywords: generation
• Abstract: Role-playing models (RPMs) are widely used in real-world applications but underperform when deployed in the wild. This degradation can be attributed to distribution shifts, including user, character, and dialogue compositional shifts. Existing methods like LLM-as-a-judge fall short in providing a fine-grained diagnosis of how these shifts affect RPM generalization, and thus there lack formal frameworks to characterize RPM generalization behaviors. To bridge these gaps, we introduce an information-theoretic metric, named reasoning-based effective mutual information difference (R-EMID), to measure RPM performance degradation in an interpretable way. We also derive an upper bound on R-EMID to predict the worst-case generalization performance of RPMs and theoretically reveal how various shifts contribute to the RPM performance degradation. Moreover, we propose a co-evolving reinforcement learning framework to adaptively model the connection among user, character, and dialogue context and thus enhance the estimation of dialogue response generation probability, which is critical for calculating R-EMID. Finally, we evaluate the generalization performance of various RPMs using R-EMID, finding that user shift poses the highest risk among all shifts and reinforcement learning is the most effective approach for enhancing RPM generalization.
• 摘要：角色扮演模型 (RPM) 广泛应用于现实世界的应用程序中，但在实际部署时表现不佳。这种降级可归因于分布变化，包括用户、角色和对话构成的变化。像 LLM-as-a-judge 这样的现有方法无法对这些变化如何影响 RPM 泛化提供细粒度的诊断，因此缺乏表征 RPM 泛化行为的正式框架。为了弥补这些差距，我们引入了一种信息论度量，称为基于推理的有效互信息差异（R-EMID），以可解释的方式测量 RPM 性能下降。我们还推导了 R-EMID 的上限，以预测 RPM 最坏情况的泛化性能，并从理论上揭示了各种变化如何导致 RPM 性能下降。此外，我们提出了一种共同进化的强化学习框架，可以自适应地对用户、角色和对话上下文之间的连接进行建模，从而增强对对话响应生成概率的估计，这对于计算 R-EMID 至关重要。最后，我们使用 R-EMID 评估各种 RPM 的泛化性能，发现用户转移在所有转移中风险最高，而强化学习是增强 RPM 泛化的最有效方法。

Title: Vision-Language Model Guided Image Restoration

• Authors: Cuixin Yang, Rongkang Dong, Kin-Man Lam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17292
• Pdf URL: https://arxiv.org/pdf/2512.17292
• Copy Paste: [[2512.17292]] Vision-Language Model Guided Image Restoration(https://arxiv.org/abs/2512.17292)
• Keywords: restoration
• Abstract: Many image restoration (IR) tasks require both pixel-level fidelity and high-level semantic understanding to recover realistic photos with fine-grained details. However, previous approaches often struggle to effectively leverage both the visual and linguistic knowledge. Recent efforts have attempted to incorporate Vision-language models (VLMs), which excel at aligning visual and textual features, into universal IR. Nevertheless, these methods fail to utilize the linguistic priors to ensure semantic coherence during the restoration process. To address this issue, in this paper, we propose the Vision-Language Model Guided Image Restoration (VLMIR) framework, which leverages the rich vision-language priors of VLMs, such as CLIP, to enhance IR performance through improved visual perception and semantic understanding. Our approach consists of two stages: VLM-based feature extraction and diffusion-based image restoration. In the first stage, we extract complementary visual and linguistic representations of input images by condensing the visual perception and high-level semantic priors through VLMs. Specifically, we align the embeddings of captions from low-quality and high-quality images using a cosine similarity loss with LoRA fine-tuning, and employ a degradation predictor to decompose degradation and clean image content embeddings. These complementary visual and textual embeddings are then integrated into a diffusion-based model via cross-attention mechanisms for enhanced restoration. Extensive experiments and ablation studies demonstrate that VLMIR achieves superior performance across both universal and degradation-specific IR tasks, underscoring the critical role of integrated visual and linguistic knowledge from VLMs in advancing image restoration capabilities.
• 摘要：许多图像恢复 (IR) 任务需要像素级保真度和高级语义理解，才能恢复具有细粒度细节的真实照片。然而，以前的方法常常难以有效地利用视觉和语言知识。最近的努力尝试将擅长协调视觉和文本特征的视觉语言模型 (VLM) 纳入通用 IR 中。然而，这些方法未能利用语言先验来确保恢复过程中的语义一致性。为了解决这个问题，在本文中，我们提出了视觉语言模型引导图像恢复（VLMIR）框架，该框架利用 VLM 丰富的视觉语言先验（例如 CLIP），通过改进视觉感知和语义理解来增强 IR 性能。我们的方法由两个阶段组成：基于 VLM 的特征提取和基于扩散的图像恢复。在第一阶段，我们通过 VLM 压缩视觉感知和高级语义先验，提取输入图像的互补视觉和语言表示。具体来说，我们使用余弦相似性损失和 LoRA 微调来对齐低质量和高质量图像的字幕嵌入，并采用退化预测器来分解退化和清理图像内容嵌入。然后，这些互补的视觉和文本嵌入通过交叉注意机制集成到基于扩散的模型中，以增强恢复。大量实验和消融研究表明，VLMIR 在通用和特定退化红外任务中均实现了卓越的性能，强调了 VLM 的集成视觉和语言知识在提高图像恢复能力方面的关键作用。

Title: ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration

• Authors: Fanpu Cao, Yaofo Chen, Zeng You, Wei Luo, Cen Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17298
• Pdf URL: https://arxiv.org/pdf/2512.17298
• Copy Paste: [[2512.17298]] ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration(https://arxiv.org/abs/2512.17298)
• Keywords: generative
• Abstract: Diffusion Transformers (DiTs) have achieved state-of-the-art performance in generative modeling, yet their high computational cost hinders real-time deployment. While feature caching offers a promising training-free acceleration solution by exploiting temporal redundancy, existing methods suffer from two key limitations: (1) uniform caching intervals fail to align with the non-uniform temporal dynamics of DiT, and (2) naive feature reuse with excessively large caching intervals can lead to severe error accumulation. In this work, we analyze the evolution of DiT features during denoising and reveal that both feature changes and error propagation are highly time- and depth-varying. Motivated by this, we propose ProCache, a training-free dynamic feature caching framework that addresses these issues via two core components: (i) a constraint-aware caching pattern search module that generates non-uniform activation schedules through offline constrained sampling, tailored to the model's temporal characteristics; and (ii) a selective computation module that selectively computes within deep blocks and high-importance tokens for cached segments to mitigate error accumulation with minimal overhead. Extensive experiments on PixArt-alpha and DiT demonstrate that ProCache achieves up to 1.96x and 2.90x acceleration with negligible quality degradation, significantly outperforming prior caching-based methods.
• 摘要：扩散变压器 (DiT) 在生成建模方面取得了最先进的性能，但其高计算成本阻碍了实时部署。虽然特征缓存通过利用时间冗余提供了一种有前途的免训练加速解决方案，但现有方法存在两个关键限制：（1）均匀缓存间隔无法与 DiT 的非均匀时间动态保持一致，（2）具有过大缓存间隔的简单特征重用可能导致严重的错误累积。在这项工作中，我们分析了去噪过程​​中 DiT 特征的演变，并揭示了特征变化和误差传播都具有高度的时间和深度变化。受此启发，我们提出了 ProCache，这是一种免训练的动态特征缓存框架，它通过两个核心组件解决这些问题：（i）约束感知缓存模式搜索模块，通过离线约束采样生成非均匀激活计划，根据模型的时间特征进行定制； (ii) 选择性计算模块，选择性地在深度块和缓存段的高重要性令牌内进行计算，以最小的开销减轻错误累积。在 PixArt-alpha 和 DiT 上进行的大量实验表明，ProCache 可实现高达 1.96 倍和 2.90 倍的加速，而质量下降可以忽略不计，显着优于先前基于缓存的方法。

Title: MatLat: Material Latent Space for PBR Texture Generation

• Authors: Kyeongmin Yeo, Yunhong Min, Jaihoon Kim, Minhyuk Sung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17302
• Pdf URL: https://arxiv.org/pdf/2512.17302
• Copy Paste: [[2512.17302]] MatLat: Material Latent Space for PBR Texture Generation(https://arxiv.org/abs/2512.17302)
• Keywords: generation, generative
• Abstract: We propose a generative framework for producing high-quality PBR textures on a given 3D mesh. As large-scale PBR texture datasets are scarce, our approach focuses on effectively leveraging the embedding space and diffusion priors of pretrained latent image generative models while learning a material latent space, MatLat, through targeted fine-tuning. Unlike prior methods that freeze the embedding network and thus lead to distribution shifts when encoding additional PBR channels and hinder subsequent diffusion training, we fine-tune the pretrained VAE so that new material channels can be incorporated with minimal latent distribution deviation. We further show that correspondence-aware attention alone is insufficient for cross-view consistency unless the latent-to-image mapping preserves locality. To enforce this locality, we introduce a regularization in the VAE fine-tuning that crops latent patches, decodes them, and aligns the corresponding image regions to maintain strong pixel-latent spatial correspondence. Ablation studies and comparison with previous baselines demonstrate that our framework improves PBR texture fidelity and that each component is critical for achieving state-of-the-art performance.
• 摘要：我们提出了一个生成框架，用于在给定的 3D 网格上生成高质量的 PBR 纹理。由于大规模 PBR 纹理数据集稀缺，我们的方法侧重于有效利用预训练潜在图像生成模型的嵌入空间和扩散先验，同时通过有针对性的微调来学习材料潜在空间 MatLat。与冻结嵌入网络并因此在编码额外 PBR 通道时导致分布变化并阻碍后续扩散训练的现有方法不同，我们对预训练的 VAE 进行微调，以便可以以最小的潜在分布偏差合并新的材料通道。我们进一步表明，除非潜在图像映射保留局部性，否则仅对应感知注意力不足以实现跨视图一致性。为了加强这种局部性，我们在 VAE 微调中引入了正则化，可以裁剪潜在补丁、对其进行解码并对齐相应的图像区域以保持强大的像素潜在空间对应性。消融研究以及与之前基线的比较表明，我们的框架提高了 PBR 纹理保真度，并且每个组件对于实现最先进的性能至关重要。

Title: EMAG: Self-Rectifying Diffusion Sampling with Exponential Moving Average Guidance

• Authors: Ankit Yadav, Ta Duc Huy, Lingqiao Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17303
• Pdf URL: https://arxiv.org/pdf/2512.17303
• Copy Paste: [[2512.17303]] EMAG: Self-Rectifying Diffusion Sampling with Exponential Moving Average Guidance(https://arxiv.org/abs/2512.17303)
• Keywords: generation, generative
• Abstract: In diffusion and flow-matching generative models, guidance techniques are widely used to improve sample quality and consistency. Classifier-free guidance (CFG) is the de facto choice in modern systems and achieves this by contrasting conditional and unconditional samples. Recent work explores contrasting negative samples at inference using a weaker model, via strong/weak model pairs, attention-based masking, stochastic block dropping, or perturbations to the self-attention energy landscape. While these strategies refine the generation quality, they still lack reliable control over the granularity or difficulty of the negative samples, and target-layer selection is often fixed. We propose Exponential Moving Average Guidance (EMAG), a training-free mechanism that modifies attention at inference time in diffusion transformers, with a statistics-based, adaptive layer-selection rule. Unlike prior methods, EMAG produces harder, semantically faithful negatives (fine-grained degradations), surfacing difficult failure modes, enabling the denoiser to refine subtle artifacts, boosting the quality and human preference score (HPS) by +0.46 over CFG. We further demonstrate that EMAG naturally composes with advanced guidance techniques, such as APG and CADS, further improving HPS.
• 摘要：在扩散和流量匹配生成模型中，引导技术被广泛用于提高样本质量和一致性。无分类器引导（CFG）是现代系统中事实上的选择，它通过对比条件样本和无条件样本来实现这一目标。最近的工作探索了使用较弱模型在推理时对比负样本，通过强/弱模型对、基于注意力的掩蔽、随机块丢弃或对自注意力能量景观的扰动。虽然这些策略提高了生成质量，但它们仍然缺乏对负样本的粒度或难度的可靠控制，并且目标层选择通常是固定的。我们提出了指数移动平均指导（EMAG），这是一种无需训练的机制，可以通过基于统计的自适应层选择规则来修改扩散变换器中推理时间的注意力。与之前的方法不同，EMAG 产生更难、语义上忠实的负片（细粒度降级），呈现困难的故障模式，使降噪器能够细化微妙的伪影，将质量和人类偏好评分 (HPS) 比 CFG 提高 +0.46。我们进一步证明，埃马克自然地与先进的制导技术（例如APG和CADS）相结合，进一步提高了HPS。

Title: EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categories

• Authors: Lu Wei, Yuta Nakashima, Noa Garcia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17320
• Pdf URL: https://arxiv.org/pdf/2512.17320
• Copy Paste: [[2512.17320]] EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categories(https://arxiv.org/abs/2512.17320)
• Keywords: generation
• Abstract: The widespread adoption of text-to-image (T2I) generation has raised concerns about privacy, bias, and copyright violations. Concept erasure techniques offer a promising solution by selectively removing undesired concepts from pre-trained models without requiring full retraining. However, these methods are often evaluated on a limited set of concepts, relying on overly simplistic and direct prompts. To test the boundaries of concept erasure techniques, and assess whether they truly remove targeted concepts from model representations, we introduce EMMA, a benchmark that evaluates five key dimensions of concept erasure over 12 metrics. EMMA goes beyond standard metrics like image quality and time efficiency, testing robustness under challenging conditions, including indirect descriptions, visually similar non-target concepts, and potential gender and ethnicity bias, providing a socially aware analysis of method behavior. Using EMMA, we analyze five concept erasure methods across five domains (objects, celebrities, art styles, NSFW, and copyright). Our results show that existing methods struggle with implicit prompts (i.e., generating the erased concept when it is indirectly referenced) and visually similar non-target concepts (i.e., failing to generate non-targeted concepts resembling the erased one), while some amplify gender and ethnicity bias compared to the original model.
• 摘要：文本到图像 (T2I) 生成技术的广泛采用引起了人们对隐私、偏见和侵犯版权的担忧。概念擦除技术提供了一种有前途的解决方案，它可以有选择地从预训练模型中删除不需要的概念，而无需完全重新训练。然而，这些方法通常是根据一组有限的概念进行评估，依赖于过于简单化和直接的提示。为了测试概念擦除技术的边界，并评估它们是否真正从模型表示中删除目标概念，我们引入了 EMMA，这是一个通过 12 个指标评估概念擦除的五个关键维度的基准。 EMMA 超越了图像质量和时间效率等标准指标，在具有挑战性的条件下测试稳健性，包括间接描述、视觉上相似的非目标概念以及潜在的性别和种族偏见，提供对方法行为的社会意识分析。使用 EMMA，我们分析了五个领域（物体、名人、艺术风格、NSFW 和版权）的五种概念擦除方法。我们的结果表明，现有的方法与隐式提示（即，间接引用时生成被删除的概念）和视觉上相似的非目标概念（即，未能生成类似于被删除的概念的非目标概念）作斗争，而与原始模型相比，有些方法放大了性别和种族偏见。

Title: Rotterdam artery-vein segmentation (RAV) dataset

• Authors: Jose Vargas Quiros, Bart Liefers, Karin van Garderen, Jeroen Vermeulen, Eyened Reading Center, Caroline Klaver
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17322
• Pdf URL: https://arxiv.org/pdf/2512.17322
• Copy Paste: [[2512.17322]] Rotterdam artery-vein segmentation (RAV) dataset(https://arxiv.org/abs/2512.17322)
• Keywords: quality assessment
• Abstract: Purpose: To provide a diverse, high-quality dataset of color fundus images (CFIs) with detailed artery-vein (A/V) segmentation annotations, supporting the development and evaluation of machine learning algorithms for vascular analysis in ophthalmology. Methods: CFIs were sampled from the longitudinal Rotterdam Study (RS), encompassing a wide range of ages, devices, and capture conditions. Images were annotated using a custom interface that allowed graders to label arteries, veins, and unknown vessels on separate layers, starting from an initial vessel segmentation mask. Connectivity was explicitly verified and corrected using connected component visualization tools. Results: The dataset includes 1024x1024-pixel PNG images in three modalities: original RGB fundus images, contrast-enhanced versions, and RGB-encoded A/V masks. Image quality varied widely, including challenging samples typically excluded by automated quality assessment systems, but judged to contain valuable vascular information. Conclusion: This dataset offers a rich and heterogeneous source of CFIs with high-quality segmentations. It supports robust benchmarking and training of machine learning models under real-world variability in image quality and acquisition settings. Translational Relevance: By including connectivity-validated A/V masks and diverse image conditions, this dataset enables the development of clinically applicable, generalizable machine learning tools for retinal vascular analysis, potentially improving automated screening and diagnosis of systemic and ocular diseases.
• 摘要：目的：提供具有详细动静脉 (A/V) 分割注释的多样化、高质量彩色眼底图像 (CFI) 数据集，支持眼科血管分析机器学习算法的开发和评估。方法：CFI 是从鹿特丹纵向研究 (RS) 中抽取的样本，涵盖各种年龄、设备和捕获条件。使用自定义界面对图像进行注释，该界面允许分级者从初始血管分割掩模开始，在不同的层上标记动脉、静脉和未知血管。使用连接组件可视化工具明确验证和纠正连接性。结果：数据集包括三种模式的 1024x1024 像素 PNG 图像：原始 RGB 眼底图像、对比度增强版本和 RGB 编码的 A/V 掩模。图像质量差异很大，包括通常被自动质量评估系统排除但被认为包含有价值的血管信息的挑战性样本。结论：该数据集提供了丰富且异构的 CFI 来源以及高质量的细分。它支持在图像质量和采集设置的真实变化下对机器学习模型进行稳健的基准测试和训练。转化相关性：通过包含经过连接验证的 A/V 掩模和不同的图像条件，该数据集能够开发用于视网膜血管分析的临床适用的、通用的机器学习工具，从而有可能改善系统和眼部疾病的自动筛查和诊断。

Title: Democratizing Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling

• Authors: Sander Moonemans, Sebastiaan Ram, Frédérique Meeuwsen, Carlijn Lems, Jeroen van der Laak, Geert Litjens, Francesco Ciompi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17326
• Pdf URL: https://arxiv.org/pdf/2512.17326
• Copy Paste: [[2512.17326]] Democratizing Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling(https://arxiv.org/abs/2512.17326)
• Keywords: generation
• Abstract: Vision-language models (VLMs) have the potential to become co-pilots for pathologists. However, most VLMs either focus on small regions of interest within whole-slide images, provide only static slide-level outputs, or rely on data that is not publicly available, limiting reproducibility. Furthermore, training data containing WSIs paired with detailed clinical reports is scarce, restricting progress toward transparent and generalisable VLMs. We address these limitations with three main contributions. First, we introduce Polysome, a standardised tool for synthetic instruction generation. Second, we apply Polysome to the public HISTAI dataset, generating HISTAI-Instruct, a large whole-slide instruction tuning dataset spanning 24,259 slides and over 1.1 million instruction-response pairs. Finally, we use HISTAI-Instruct to train ANTONI-{\alpha}, a VLM capable of visual-question answering (VQA). We show that ANTONI-{\alpha} outperforms MedGemma on WSI-level VQA tasks of tissue identification, neoplasm detection, and differential diagnosis. We also compare the performance of multiple incarnations of ANTONI-{\alpha} trained with different amounts of data. All methods, data, and code are publicly available.
• 摘要：视觉语言模型（VLM）有潜力成为病理学家的副驾驶。然而，大多数 VLM 要么专注于整个幻灯片图像中的小感兴趣区域，仅提供静态幻灯片级输出，要么依赖未公开提供的数据，从而限制了再现性。此外，包含 WSI 和详细临床报告的训练数据很少，限制了透明和通用 VLM 的进展。我们通过三个主要贡献来解决这些限制。首先，我们介绍 Polysome，一种用于合成指令生成的标准化工具。其次，我们将 Polysome 应用于公共 HISTAI 数据集，生成 HISTAI-Instruct，这是一个大型的整幻灯片指令调整数据集，涵盖 24,259 张幻灯片和超过 110 万个指令响应对。最后，我们使用 HISTAI-Instruct 来训练 ANTONI-{\alpha}，一个能够进行视觉问答（VQA）的 VLM。我们表明，ANTONI-{\alpha} 在组织识别、肿瘤检测和鉴别诊断的 WSI 级别 VQA 任务上优于 MedGemma。我们还比较了使用不同数据量训练的 ANTONI-{\alpha} 的多个化身的性能。所有方法、数据和代码都是公开的。

Title: Multi-level distortion-aware deformable network for omnidirectional image super-resolution

• Authors: Cuixin Yang, Rongkang Dong, Kin-Man Lam, Yuhang Zhang, Guoping Qiu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17343
• Pdf URL: https://arxiv.org/pdf/2512.17343
• Copy Paste: [[2512.17343]] Multi-level distortion-aware deformable network for omnidirectional image super-resolution(https://arxiv.org/abs/2512.17343)
• Keywords: super-resolution
• Abstract: As augmented reality and virtual reality applications gain popularity, image processing for OmniDirectional Images (ODIs) has attracted increasing attention. OmniDirectional Image Super-Resolution (ODISR) is a promising technique for enhancing the visual quality of ODIs. Before performing super-resolution, ODIs are typically projected from a spherical surface onto a plane using EquiRectangular Projection (ERP). This projection introduces latitude-dependent geometric distortion in ERP images: distortion is minimal near the equator but becomes severe toward the poles, where image content is stretched across a wider area. However, existing ODISR methods have limited sampling ranges and feature extraction capabilities, which hinder their ability to capture distorted patterns over large areas. To address this issue, we propose a novel Multi-level Distortion-aware Deformable Network (MDDN) for ODISR, designed to expand the sampling range and receptive field. Specifically, the feature extractor in MDDN comprises three parallel branches: a deformable attention mechanism (serving as the dilation=1 path) and two dilated deformable convolutions with dilation rates of 2 and 3. This architecture expands the sampling range to include more distorted patterns across wider areas, generating dense and comprehensive features that effectively capture geometric distortions in ERP images. The representations extracted from these deformable feature extractors are adaptively fused in a multi-level feature fusion module. Furthermore, to reduce computational cost, a low-rank decomposition strategy is applied to dilated deformable convolutions. Extensive experiments on publicly available datasets demonstrate that MDDN outperforms state-of-the-art methods, underscoring its effectiveness and superiority in ODISR.
• 摘要：随着增强现实和虚拟现实应用的普及，全向图像（ODI）的图像处理引起了越来越多的关注。全向图像超分辨率 (ODISR) 是一种很有前途的增强 ODI 视觉质量的技术。在执行超分辨率之前，通常使用等矩形投影 (ERP) 将 ODI 从球面投影到平面上。此投影在 ERP 图像中引入了与纬度相关的几何畸变：在赤道附近畸变最小，但在靠近两极时畸变变得严重，在两极处，图像内容被拉伸到更广泛的区域。然而，现有的 ODISR 方法的采样范围和特征提取能力有限，这阻碍了它们捕获大面积扭曲模式的能力。为了解决这个问题，我们为 ODISR 提出了一种新颖的多级失真感知变形网络（MDDN），旨在扩大采样范围和感受野。具体来说，MDDN 中的特征提取器包括三个并行分支：一个可变形注意机制（充当 dilation=1 路径）和两个扩张率分别为 2 和 3 的扩张变形卷积。该架构扩展了采样范围，以包含更广泛区域中的更多扭曲模式，生成密集且全面的特征，有效捕获 ERP 图像中的几何扭曲。从这些可变形特征提取器提取的表示在多级特征融合模块中自适应地融合。此外，为了降低计算成本，将低秩分解策略应用于扩张的可变形卷积。对公开数据集的大量实验表明，MDDN 的性能优于最先进的方法，强调了其在 ODISR 中的有效性和优越性。

Title: Beyond Semantic Features: Pixel-level Mapping for Generalized AI-Generated Image Detection

• Authors: Chenming Zhou, Jiaan Wang, Yu Li, Lei Li, Juan Cao, Sheng Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17350
• Pdf URL: https://arxiv.org/pdf/2512.17350
• Copy Paste: [[2512.17350]] Beyond Semantic Features: Pixel-level Mapping for Generalized AI-Generated Image Detection(https://arxiv.org/abs/2512.17350)
• Keywords: generation, generative
• Abstract: The rapid evolution of generative technologies necessitates reliable methods for detecting AI-generated images. A critical limitation of current detectors is their failure to generalize to images from unseen generative models, as they often overfit to source-specific semantic cues rather than learning universal generative artifacts. To overcome this, we introduce a simple yet remarkably effective pixel-level mapping pre-processing step to disrupt the pixel value distribution of images and break the fragile, non-essential semantic patterns that detectors commonly exploit as shortcuts. This forces the detector to focus on more fundamental and generalizable high-frequency traces inherent to the image generation process. Through comprehensive experiments on GAN and diffusion-based generators, we show that our approach significantly boosts the cross-generator performance of state-of-the-art detectors. Extensive analysis further verifies our hypothesis that the disruption of semantic cues is the key to generalization.
• 摘要：生成技术的快速发展需要可靠的方法来检测人工智能生成的图像。当前检测器的一个关键限制是它们无法推广到来自看不见的生成模型的图像，因为它们经常过度拟合特定于源的语义线索，而不是学习通用的生成工件。为了克服这个问题，我们引入了一种简单但非常有效的像素级映射预处理步骤，以破坏图像的像素值分布，并打破检测器通常利用的脆弱的、非必要的语义模式作为捷径。这迫使探测器专注于图像生成过程固有的更基本和更普遍的高频轨迹。通过对 GAN 和基于扩散的生成器的综合实验，我们表明我们的方法显着提高了最先进检测器的跨生成器性能。广泛的分析进一步验证了我们的假设，即语义线索的破坏是泛化的关键。

Title: Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach

• Authors: Yidong Chai, Yi Liu, Mohammadreza Ebrahimi, Weifeng Li, Balaji Padmanabhan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2512.17367
• Pdf URL: https://arxiv.org/pdf/2512.17367
• Copy Paste: [[2512.17367]] Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach(https://arxiv.org/abs/2512.17367)
• Keywords: generation
• Abstract: Social media platforms are plagued by harmful content such as hate speech, misinformation, and extremist rhetoric. Machine learning (ML) models are widely adopted to detect such content; however, they remain highly vulnerable to adversarial attacks, wherein malicious users subtly modify text to evade detection. Enhancing adversarial robustness is therefore essential, requiring detectors that can defend against diverse attacks (generalizability) while maintaining high overall accuracy. However, simultaneously achieving both optimal generalizability and accuracy is challenging. Following the computational design science paradigm, this study takes a sequential approach that first proposes a novel framework (Large Language Model-based Sample Generation and Aggregation, LLM-SGA) by identifying the key invariances of textual adversarial attacks and leveraging them to ensure that a detector instantiated within the framework has strong generalizability. Second, we instantiate our detector (Adversarially Robust Harmful Online Content Detector, ARHOCD) with three novel design components to improve detection accuracy: (1) an ensemble of multiple base detectors that exploits their complementary strengths; (2) a novel weight assignment method that dynamically adjusts weights based on each sample's predictability and each base detector's capability, with weights initialized using domain knowledge and updated via Bayesian inference; and (3) a novel adversarial training strategy that iteratively optimizes both the base detectors and the weight assignor. We addressed several limitations of existing adversarial robustness enhancement research and empirically evaluated ARHOCD across three datasets spanning hate speech, rumor, and extremist content. Results show that ARHOCD offers strong generalizability and improves detection accuracy under adversarial conditions.
• 摘要：社交媒体平台受到仇恨言论、错误信息和极端主义言论等有害内容的困扰。机器学习（ML）模型被广泛采用来检测此类内容；然而，它们仍然非常容易受到对抗性攻击，其中恶意用户巧妙地修改文本以逃避检测。因此，增强对抗鲁棒性至关重要，需要检测器能够防御多种攻击（通用性），同时保持较高的整体准确性。然而，同时实现最佳的普遍性和准确性是具有挑战性的。遵循计算设计科学范式，本研究采用顺序方法，首先通过识别文本对抗性攻击的关键不变性并利用它们来确保框架内实例化的检测器具有很强的通用性，提出一种新颖的框架（基于大语言模型的样本生成和聚合，LLM-SGA）。其次，我们用三个新颖的设计组件实例化我们的检测器（对抗性鲁棒有害在线内容检测器，ARHOCD）以提高检测精度：（1）多个基本检测器的集合，利用它们的互补优势； (2)一种新颖的权重分配方法，根据每个样本的可预测性和每个基础检测器的能力动态调整权重，权重使用领域知识初始化并通过贝叶斯推理更新； (3)一种新颖的对抗训练策略，迭代优化基础检测器和权重分配器。我们解决了现有对抗性鲁棒性增强研究的一些局限性，并在涵盖仇恨言论、谣言和极端主义内容的三个数据集上对 ARHOCD 进行了实证评估。结果表明，ARHOCD 具有很强的通用性，并提高了对抗条件下的检测精度。

Title: Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors

• Authors: Peixuan Zhang, Shuchen Weng, Jiajun Tang, Si Li, Boxin Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17376
• Pdf URL: https://arxiv.org/pdf/2512.17376
• Copy Paste: [[2512.17376]] Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors(https://arxiv.org/abs/2512.17376)
• Keywords: generative
• Abstract: Social media platforms enable users to express emotions by posting text with accompanying images. In this paper, we propose the Affective Image Filter (AIF) task, which aims to reflect visually-abstract emotions from text into visually-concrete images, thereby creating emotionally compelling results. We first introduce the AIF dataset and the formulation of the AIF models. Then, we present AIF-B as an initial attempt based on a multi-modal transformer architecture. After that, we propose AIF-D as an extension of AIF-B towards deeper emotional reflection, effectively leveraging generative priors from pre-trained large-scale diffusion models. Quantitative and qualitative experiments demonstrate that AIF models achieve superior performance for both content consistency and emotional fidelity compared to state-of-the-art methods. Extensive user study experiments demonstrate that AIF models are significantly more effective at evoking specific emotions. Based on the presented results, we comprehensively discuss the value and potential of AIF models.
• 摘要：社交媒体平台使用户能够通过发布带有图像的文本来表达情感。在本文中，我们提出了情感图像过滤器（AIF）任务，其目的是将文本中视觉抽象的情感反映到视觉具体的图像中，从而创建情感上引人注目的结果。我们首先介绍 AIF 数据集和 AIF 模型的制定。然后，我们将 AIF-B 作为基于多模态变压器架构的初步尝试。之后，我们提出 AIF-D 作为 AIF-B 的延伸，以实现更深入的情感反映，有效地利用预先训练的大规模扩散模型的生成先验。定量和定性实验表明，与最先进的方法相比，AIF 模型在内容一致性和情感保真度方面均取得了优异的性能。广泛的用户研究实验表明，AIF 模型在唤起特定情绪方面显着更有效。基于所呈现的结果，我们全面讨论了 AIF 模型的价值和潜力。

Title: Assessing Long-Term Electricity Market Design for Ambitious Decarbonization Targets using Multi-Agent Reinforcement Learning

• Authors: Javier Gonzalez-Ruiz, Carlos Rodriguez-Pardo, Iacopo Savelli, Alice Di Bella, Massimo Tavoni
• Subjects: cs.LG, cs.AI, cs.NE, econ.GN
• Abstract URL: https://arxiv.org/abs/2512.17444
• Pdf URL: https://arxiv.org/pdf/2512.17444
• Copy Paste: [[2512.17444]] Assessing Long-Term Electricity Market Design for Ambitious Decarbonization Targets using Multi-Agent Reinforcement Learning(https://arxiv.org/abs/2512.17444)
• Keywords: generation
• Abstract: Electricity systems are key to transforming today's society into a carbon-free economy. Long-term electricity market mechanisms, including auctions, support schemes, and other policy instruments, are critical in shaping the electricity generation mix. In light of the need for more advanced tools to support policymakers and other stakeholders in designing, testing, and evaluating long-term markets, this work presents a multi-agent reinforcement learning model capable of capturing the key features of decarbonizing energy systems. Profit-maximizing generation companies make investment decisions in the wholesale electricity market, responding to system needs, competitive dynamics, and policy signals. The model employs independent proximal policy optimization, which was selected for suitability to the decentralized and competitive environment. Nevertheless, given the inherent challenges of independent learning in multi-agent settings, an extensive hyperparameter search ensures that decentralized training yields market outcomes consistent with competitive behavior. The model is applied to a stylized version of the Italian electricity system and tested under varying levels of competition, market designs, and policy scenarios. Results highlight the critical role of market design for decarbonizing the electricity sector and avoiding price volatility. The proposed framework allows assessing long-term electricity markets in which multiple policy and market mechanisms interact simultaneously, with market participants responding and adapting to decarbonization pathways.
• 摘要：电力系统是将当今社会转变为无碳经济的关键。长期电力市场机制，包括拍卖、支持计划和其他政策工具，对于塑造发电结构至关重要。鉴于需要更先进的工具来支持政策制定者和其他利益相关者设计、测试和评估长期市场，这项工作提出了一种多智能体强化学习模型，能够捕获脱碳能源系统的关键特征。追求利润最大化的发电公司在批发电力市场上做出投资决策，响应系统需求、竞争动态和政策信号。该模型采用独立的近端策略优化，选择该优化是为了适应去中心化和竞争的环境。然而，考虑到多智能体环境中独立学习的固有挑战，广泛的超参数搜索可确保去中心化训练产生与竞争行为一致的市场结果。该模型应用于意大利电力系统的程式化版本，并在不同水平的竞争、市场设计和政策情景下进行测试。结果凸显了市场设计对于电力部门脱碳和避免价格波动的关键作用。拟议的框架允许评估长期电力市场，其中多种政策和市场机制同时相互作用，市场参与者响应并适应脱碳路径。

Title: 3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework

• Authors: Tobias Sautter, Jan-Niklas Dihlmann, Hendrik P.A. Lensch
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17459
• Pdf URL: https://arxiv.org/pdf/2512.17459
• Copy Paste: [[2512.17459]] 3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework(https://arxiv.org/abs/2512.17459)
• Keywords: generation, generative
• Abstract: Recent advances in 3D scene generation produce visually appealing output, but current representations hinder artists' workflows that require modifiable 3D textured mesh scenes for visual effects and game development. Despite significant advances, current textured mesh scene reconstruction methods are far from artist ready, suffering from incorrect object decomposition, inaccurate spatial relationships, and missing backgrounds. We present 3D-RE-GEN, a compositional framework that reconstructs a single image into textured 3D objects and a background. We show that combining state of the art models from specific domains achieves state of the art scene reconstruction performance, addressing artists' requirements. Our reconstruction pipeline integrates models for asset detection, reconstruction, and placement, pushing certain models beyond their originally intended domains. Obtaining occluded objects is treated as an image editing task with generative models to infer and reconstruct with scene level reasoning under consistent lighting and geometry. Unlike current methods, 3D-RE-GEN generates a comprehensive background that spatially constrains objects during optimization and provides a foundation for realistic lighting and simulation tasks in visual effects and games. To obtain physically realistic layouts, we employ a novel 4-DoF differentiable optimization that aligns reconstructed objects with the estimated ground plane. 3D-RE-GEN~achieves state of the art performance in single image 3D scene reconstruction, producing coherent, modifiable scenes through compositional generation guided by precise camera recovery and spatial optimization.
• 摘要：3D 场景生成方面的最新进展可产生具有视觉吸引力的输出，但当前的表示方式阻碍了艺术家的工作流程，因为艺术家需要可修改的 3D 纹理网格场景来实现视觉效果和游戏开发。尽管取得了显着的进步，但当前的纹理网格场景重建方法还远未为艺术家做好准备，存在对象分解不正确、空间关系不准确和背景缺失的问题。我们提出了 3D-RE-GEN，这是一个将单个图像重建为纹理 3D 对象和背景的合成框架。我们证明，结合特定领域的最先进模型可以实现最先进的场景重建性能，满足艺术家的要求。我们的重建管道集成了资产检测、重建和放置的模型，使某些模型超出了其最初的预期领域。获取被遮挡的对象被视为具有生成模型的图像编辑任务，以便在一致的光照和几何形状下通过场景级推理进行推断和重建。与当前方法不同，3D-RE-GEN 生成一个全面的背景，在优化过程中对对象进行空间约束，并为视觉效果和游戏中的真实光照和模拟任务提供基础。为了获得物理上真实的布局，我们采用了一种新颖的 4-DoF 可微优化，将重建对象与估计的地平面对齐。 3D-RE-GEN~在单图像 3D 场景重建方面实现了最先进的性能，通过精确相机恢复和空间优化指导的合成生成来生成连贯的、可修改的场景。

Title: Deep Learning-Based Surrogate Creep Modelling in Inconel 625: A High-Temperature Alloy Study

• Authors: Shubham Das, Kaushal Singhania, Amit Sadhu, Suprabhat Das, Arghya Nandi
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2512.17477
• Pdf URL: https://arxiv.org/pdf/2512.17477
• Copy Paste: [[2512.17477]] Deep Learning-Based Surrogate Creep Modelling in Inconel 625: A High-Temperature Alloy Study(https://arxiv.org/abs/2512.17477)
• Keywords: generative
• Abstract: Time-dependent deformation, particularly creep, in high-temperature alloys such as Inconel 625 is a key factor in the long-term reliability of components used in aerospace and energy systems. Although Inconel 625 shows excellent creep resistance, finite-element creep simulations in tools such as ANSYS remain computationally expensive, often requiring tens of minutes for a single 10,000-hour run. This work proposes deep learning based surrogate models to provide fast and accurate replacements for such simulations. Creep strain data was generated in ANSYS using the Norton law under uniaxial stresses of 50 to 150 MPa and temperatures of 700 to 1000 $^\circ$C, and this temporal dataset was used to train two architectures: a BiLSTM Variational Autoencoder for uncertainty-aware and generative predictions, and a BiLSTM Transformer hybrid that employs self-attention to capture long-range temporal behavior. Both models act as surrogate predictors, with the BiLSTM-VAE offering probabilistic output and the BiLSTM-Transformer delivering high deterministic accuracy. Performance is evaluated using RMSE, MAE, and $R^2$. Results show that the BiLSTM-VAE provides stable and reliable creep strain forecasts, while the BiLSTM-Transformer achieves strong accuracy across the full time range. Latency tests indicate substantial speedup: while each ANSYS simulation requires 30 to 40 minutes for a given stress-temperature condition, the surrogate models produce predictions within seconds. The proposed framework enables rapid creep assessment for design optimization and structural health monitoring, and provides a scalable solution for high-temperature alloy applications.
• 摘要：Inconel 625 等高温合金中随时间变化的变形（尤其是蠕变）是航空航天和能源系统中使用的组件长期可靠性的关键因素。尽管 Inconel 625 表现出优异的抗蠕变性，但在 ANSYS 等工具中进行有限元蠕变模拟的计算成本仍然很高，单次 10,000 小时的运行通常需要数十分钟。这项工作提出了基于深度学习的代理模型，为此类模拟提供快速、准确的替代。蠕变应变数据是在 50 至 150 MPa 的单轴应力和 700 至 1000 $^\circ$C 的温度下使用诺顿定律在 ANSYS 中生成的，并且该时间数据集用于训练两种架构：用于不确定性感知和生成预测的 BiLSTM 变分自编码器，以及采用自注意力来捕获长范围时间行为的 BiLSTM Transformer 混合体。这两个模型都充当代理预测器，BiLSTM-VAE 提供概率输出，BiLSTM-Transformer 提供高确定性精度。使用 RMSE、MAE 和 $R^2$ 评估性能。结果表明，BiLSTM-VAE 提供稳定可靠的蠕变应变预测，而 BiLSTM-Transformer 在整个时间范围内实现了很高的准确性。延迟测试表明显着的加速：虽然对于给定的应力-温度条件，每个 ANSYS 仿真需要 30 到 40 分钟，但替代模型在几秒钟内即可生成预测。所提出的框架能够对设计优化和结构健康监测进行快速蠕变评估，并为高温合金应用提供可扩展的解决方案。

Title: LumiCtrl : Learning Illuminant Prompts for Lighting Control in Personalized Text-to-Image Models

• Authors: Muhammad Atif Butt, Kai Wang, Javier Vazquez-Corral, Joost Van De Weijer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17489
• Pdf URL: https://arxiv.org/pdf/2512.17489
• Copy Paste: [[2512.17489]] LumiCtrl : Learning Illuminant Prompts for Lighting Control in Personalized Text-to-Image Models(https://arxiv.org/abs/2512.17489)
• Keywords: generation
• Abstract: Current text-to-image (T2I) models have demonstrated remarkable progress in creative image generation, yet they still lack precise control over scene illuminants, which is a crucial factor for content designers aiming to manipulate the mood, atmosphere, and visual aesthetics of generated images. In this paper, we present an illuminant personalization method named LumiCtrl that learns an illuminant prompt given a single image of an object. LumiCtrl consists of three basic components: given an image of the object, our method applies (a) physics-based illuminant augmentation along the Planckian locus to create fine-tuning variants under standard illuminants; (b) edge-guided prompt disentanglement using a frozen ControlNet to ensure prompts focus on illumination rather than structure; and (c) a masked reconstruction loss that focuses learning on the foreground object while allowing the background to adapt contextually, enabling what we call contextual light adaptation. We qualitatively and quantitatively compare LumiCtrl against other T2I customization methods. The results show that our method achieves significantly better illuminant fidelity, aesthetic quality, and scene coherence compared to existing personalization baselines. A human preference study further confirms strong user preference for LumiCtrl outputs. The code and data will be released upon publication.
• 摘要：当前的文本到图像（T2I）模型在创意图像生成方面取得了显着的进步，但它们仍然缺乏对场景光源的精确控制，而这对于旨在操纵生成图像的情绪、氛围和视觉美感的内容设计师来说是一个关键因素。在本文中，我们提出了一种名为 LumiCtrl 的光源个性化方法，该方法可以在给定物体的单个图像的情况下学习光源提示。 LumiCtrl 由三个基本组件组成：给定物体的图像，我们的方法应用（a）沿着普朗克轨迹的基于物理的光源增强，以在标准光源下创建微调变体； (b) 使用冻结的 ControlNet 进行边缘引导提示解开，以确保提示集中在照明而不是结构上； (c) 掩蔽重建损失，将学习重点放在前景物体上，同时允许背景根据上下文进行适应，从而实现我们所说的上下文光适应。我们对 LumiCtrl 与其他 T2I 定制方法进行定性和定量比较。结果表明，与现有的个性化基线相比，我们的方法实现了明显更好的光源保真度、美学质量和场景连贯性。人类偏好研究进一步证实了用户对 LumiCtrl 输出的强烈偏好。代码和数据将在发布后发布。

Title: GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation

• Authors: Rang Li, Lei Li, Shuhuai Ren, Hao Tian, Shuhao Gu, Shicheng Li, Zihao Yue, Yudong Wang, Wenhan Ma, Zhe Yang, Jingyuan Ma, Zhifang Sui, Fuli Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17495
• Pdf URL: https://arxiv.org/pdf/2512.17495
• Copy Paste: [[2512.17495]] GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation(https://arxiv.org/abs/2512.17495)
• Keywords: generation
• Abstract: Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing benchmarks, a fundamental question remains: can MLLMs truly ground language in vision with human-like sophistication, or are they merely pattern-matching on simplified datasets? Current benchmarks fail to capture real-world complexity where humans effortlessly navigate ambiguous references and recognize when grounding is impossible. To rigorously assess MLLMs' true capabilities, we introduce GroundingME, a benchmark that systematically challenges models across four critical dimensions: (1) Discriminative, distinguishing highly similar objects, (2) Spatial, understanding complex relational descriptions, (3) Limited, handling occlusions or tiny objects, and (4) Rejection, recognizing ungroundable queries. Through careful curation combining automated generation with human verification, we create 1,005 challenging examples mirroring real-world complexity. Evaluating 25 state-of-the-art MLLMs reveals a profound capability gap: the best model achieves only 45.1% accuracy, while most score 0% on rejection tasks, reflexively hallucinating objects rather than acknowledging their absence, raising critical safety concerns for deployment. We explore two strategies for improvements: (1) test-time scaling selects optimal response by thinking trajectory to improve complex grounding by up to 2.9%, and (2) data-mixture training teaches models to recognize ungroundable queries, boosting rejection accuracy from 0% to 27.9%. GroundingME thus serves as both a diagnostic tool revealing current limitations in MLLMs and a roadmap toward human-level visual grounding.
• 摘要：视觉基础，从自然语言描述中定位对象，代表了语言和视觉理解之间的重要桥梁。虽然多模态大语言模型 (MLLM) 在现有基准上取得了令人印象深刻的分数，但仍然存在一个基本问题：MLLM 能否真正以类人的复杂性在视觉中奠定基础语言，或者它们只是在简化数据集上进行模式匹配？当前的基准无法捕捉现实世界的复杂性，人类可以毫不费力地导航模糊的参考并识别何时不可能接地。为了严格评估 MLLM 的真实能力，我们引入了 GroundingME，这是一个在四个关键维度上系统地挑战模型的基准：(1) 判别性，区分高度相似的对象，(2) 空间，理解复杂的关系描述，(3) 有限，​​处理遮挡或微小对象，以及 (4) 拒绝，识别无法接地的查询。通过精心策划，将自动生成与人工验证相结合，我们创建了 1,005 个具有挑战性的示例，反映了现实世界的复杂性。对 25 个最先进的 MLLM 的评估揭示了巨大的能力差距：最好的模型仅达到 45.1% 的准确率，而大多数模型在拒绝任务上得分为 0%，反射性地产生幻觉而不是承认它们不存在，从而引发了部署的关键安全问题。我们探索了两种改进策略：(1) 测试时间缩放通过思维轨迹选择最佳响应，将复杂基础提高高达 2.9%，(2) 数据混合训练教会模型识别无法基础的查询，将拒绝准确率从 0% 提高到 27.9%。因此，GroundingME 既可以作为揭示 MLLM 当前局限性的诊断工具，又可以作为人类水平视觉基础的路线图。

Title: InsertAnywhere: Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion

• Authors: Hoiyeong Jin, Hyojin Jang, Jeongho Kim, Junha Hyung, Kinam Kim, Dongjin Kim, Huijin Choi, Hyeonji Kim, Jaegul Choo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.17504
• Pdf URL: https://arxiv.org/pdf/2512.17504
• Copy Paste: [[2512.17504]] InsertAnywhere: Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion(https://arxiv.org/abs/2512.17504)
• Keywords: generation
• Abstract: Recent advances in diffusion-based video generation have opened new possibilities for controllable video editing, yet realistic video object insertion (VOI) remains challenging due to limited 4D scene understanding and inadequate handling of occlusion and lighting effects. We present InsertAnywhere, a new VOI framework that achieves geometrically consistent object placement and appearance-faithful video synthesis. Our method begins with a 4D aware mask generation module that reconstructs the scene geometry and propagates user specified object placement across frames while maintaining temporal coherence and occlusion consistency. Building upon this spatial foundation, we extend a diffusion based video generation model to jointly synthesize the inserted object and its surrounding local variations such as illumination and shading. To enable supervised training, we introduce ROSE++, an illumination aware synthetic dataset constructed by transforming the ROSE object removal dataset into triplets of object removed video, object present video, and a VLM generated reference image. Through extensive experiments, we demonstrate that our framework produces geometrically plausible and visually coherent object insertions across diverse real world scenarios, significantly outperforming existing research and commercial models.
• 摘要：基于扩散的视频生成的最新进展为可控视频编辑开辟了新的可能性，但由于 4D 场景理解有限以及对遮挡和照明效果的处理不足，逼真的视频对象插入 (VOI) 仍然具有挑战性。我们推出了 InsertAnywhere，这是一种新的 VOI 框架，可实现几何一致的对象放置和外观忠实的视频合成。我们的方法从 4D 感知掩模生成模块开始，该模块重建场景几何形状并跨帧传播用户指定的对象放置，同时保持时间连贯性和遮挡一致性。在此空间基础上，我们扩展了基于扩散的视频生成模型，以联合合成插入的对象及其周围的局部变化，例如照明和阴影。为了实现监督训练，我们引入了 ROSE++，这是一种照明感知合成数据集，通过将 ROSE 对象删除数据集转换为对象删除视频、对象存在视频和 VLM 生成的参考图像的三元组而构建。通过大量的实验，我们证明我们的框架可以在不同的现实世界场景中产生几何上合理且视觉上连贯的对象插入，显着优于现有的研究和商业模型。

Title: 3One2: One-step Regression Plus One-step Diffusion for One-hot Modulation in Dual-path Video Snapshot Compressive Imaging

• Authors: Ge Wang, Xing Liu, Xin Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17578
• Pdf URL: https://arxiv.org/pdf/2512.17578
• Copy Paste: [[2512.17578]] 3One2: One-step Regression Plus One-step Diffusion for One-hot Modulation in Dual-path Video Snapshot Compressive Imaging(https://arxiv.org/abs/2512.17578)
• Keywords: generative
• Abstract: Video snapshot compressive imaging (SCI) captures dynamic scene sequences through a two-dimensional (2D) snapshot, fundamentally relying on optical modulation for hardware compression and the corresponding software reconstruction. While mainstream video SCI using random binary modulation has demonstrated success, it inevitably results in temporal aliasing during compression. One-hot modulation, activating only one sub-frame per pixel, provides a promising solution for achieving perfect temporal decoupling, thereby alleviating issues associated with aliasing. However, no algorithms currently exist to fully exploit this potential. To bridge this gap, we propose an algorithm specifically designed for one-hot masks. First, leveraging the decoupling properties of one-hot modulation, we transform the reconstruction task into a generative video inpainting problem and introduce a stochastic differential equation (SDE) of the forward process that aligns with the hardware compression process. Next, we identify limitations of the pure diffusion method for video SCI and propose a novel framework that combines one-step regression initialization with one-step diffusion refinement. Furthermore, to mitigate the spatial degradation caused by one-hot modulation, we implement a dual optical path at the hardware level, utilizing complementary information from another path to enhance the inpainted video. To our knowledge, this is the first work integrating diffusion into video SCI reconstruction. Experiments conducted on synthetic datasets and real scenes demonstrate the effectiveness of our method.
• 摘要：视频快照压缩成像（SCI）通过二维（2D）快照捕获动态场景序列，从根本上依赖光学调制进行硬件压缩和相应的软件重建。虽然使用随机二进制调制的主流视频 SCI 已被证明是成功的，但它不可避免地会在压缩过程中导致时间混叠。单热调制（每个像素仅激活一个子帧）为实现完美的时间解耦提供了一种有前途的解决方案，从而减轻了与混叠相关的问题。然而，目前还没有算法可以充分利用这种潜力。为了弥补这一差距，我们提出了一种专门为 one-hot mask 设计的算法。首先，利用单热调制的解耦特性，我们将重建任务转化为生成视频修复问题，并引入与硬件压缩过程一致的前向过程的随机微分方程（SDE）。接下来，我们确定了视频 SCI 纯扩散方法的局限性，并提出了一种将一步回归初始化与一步扩散细化相结合的新颖框架。此外，为了减轻由单热调制引起的空间退化，我们在硬件级别实现了双光路，利用来自另一条路径的补充信息来增强修复视频。据我们所知，这是第一个将扩散融入视频 SCI 重建的工作。在合成数据集和真实场景上进行的实验证明了我们方法的有效性。

Title: HeadHunt-VAD: Hunting Robust Anomaly-Sensitive Heads in MLLM for Tuning-Free Video Anomaly Detection

• Authors: Zhaolin Cai, Fan Li, Ziwei Zheng, Haixia Bi, Lijun He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17601
• Pdf URL: https://arxiv.org/pdf/2512.17601
• Copy Paste: [[2512.17601]] HeadHunt-VAD: Hunting Robust Anomaly-Sensitive Heads in MLLM for Tuning-Free Video Anomaly Detection(https://arxiv.org/abs/2512.17601)
• Keywords: generation
• Abstract: Video Anomaly Detection (VAD) aims to locate events that deviate from normal patterns in videos. Traditional approaches often rely on extensive labeled data and incur high computational costs. Recent tuning-free methods based on Multimodal Large Language Models (MLLMs) offer a promising alternative by leveraging their rich world knowledge. However, these methods typically rely on textual outputs, which introduces information loss, exhibits normalcy bias, and suffers from prompt sensitivity, making them insufficient for capturing subtle anomalous cues. To address these constraints, we propose HeadHunt-VAD, a novel tuning-free VAD paradigm that bypasses textual generation by directly hunting robust anomaly-sensitive internal attention heads within the frozen MLLM. Central to our method is a Robust Head Identification module that systematically evaluates all attention heads using a multi-criteria analysis of saliency and stability, identifying a sparse subset of heads that are consistently discriminative across diverse prompts. Features from these expert heads are then fed into a lightweight anomaly scorer and a temporal locator, enabling efficient and accurate anomaly detection with interpretable outputs. Extensive experiments show that HeadHunt-VAD achieves state-of-the-art performance among tuning-free methods on two major VAD benchmarks while maintaining high efficiency, validating head-level probing in MLLMs as a powerful and practical solution for real-world anomaly detection.
• 摘要：视频异常检测 (VAD) 旨在定位视频中偏离正常模式的事件。传统方法通常依赖于大量标记数据并产生高昂的计算成本。最近基于多模态大语言模型（MLLM）的免调优方法通过利用其丰富的世界知识提供了一种有前途的替代方案。然而，这些方法通常依赖于文本输出，这会导致信息丢失，表现出常态偏差，并且会受到即时敏感性的影响，使得它们不足以捕获微妙的异常线索。为了解决这些限制，我们提出了 HeadHunt-VAD，这是一种新颖的免调优 VAD 范式，它通过直接在冻结的 MLLM 中寻找强大的异常敏感内部注意力头来绕过文本生成。我们方法的核心是一个鲁棒的头部识别模块，它使用显着性和稳定性的多标准分析来系统地评估所有注意力头部，识别出在不同提示下始终具有区分性的头部的稀疏子集。然后，这些专家头的特征被输入到轻量级异常评分器和时间定位器中，从而实现高效、准确的异常检测和可解释的输出。大量实验表明，HeadHunt-VAD 在两个主要 VAD 基准上实现了免调优方法中最先进的性能，同时保持了高效率，验证了 MLLM 中的头部探测作为现实世界异常检测的强大且实用的解决方案。

Title: Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution

• Authors: Alireza Samadifardheris, Dirk H.J. Poot, Florian Wiesinger, Stefan Klein, Juan A. Hernandez-Tamames
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17612
• Pdf URL: https://arxiv.org/pdf/2512.17612
• Copy Paste: [[2512.17612]] Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution(https://arxiv.org/abs/2512.17612)
• Keywords: super-resolution
• Abstract: High-resolution (HR) quantitative MRI (qMRI) relaxometry provides objective tissue characterization but remains clinically underutilized due to lengthy acquisition times. We propose a physics-informed, self-supervised framework for qMRI super-resolution that uses routinely acquired HR weighted MRI (wMRI) scans as guidance, thus, removing the necessity for HR qMRI ground truth during training. We formulate super-resolution as Bayesian maximum a posteriori inference, minimizing two discrepancies: (1) between HR images synthesized from super-resolved qMRI maps and acquired wMRI guides via forward signal models, and (2) between acquired LR qMRI and downsampled predictions. This physics-informed objective allows the models to learn from clinical wMRI without HR qMRI supervision. To validate the concept, we generate training data by synthesizing wMRI guides from HR qMRI using signal equations, then degrading qMRI resolution via k-space truncation. A deep neural network learns the super-resolution mapping. Ablation experiments demonstrate that T1-weighted images primarily enhance T1 maps, T2-weighted images improve T2 maps, and combined guidance optimally enhances all parameters simultaneously. Validation on independently acquired in-vivo data from a different qMRI sequence confirms cross-qMRI sequence generalizability. Models trained on synthetic data can produce super-resolved maps from a 1-minute acquisition with quality comparable to a 5-minute reference scan, leveraging the scanner-independent nature of relaxometry parameters. By decoupling training from HR qMRI requirement, our framework enables fast qMRI acquisitions enhanced via routine clinical images, offering a practical pathway for integrating quantitative relaxometry into clinical workflows with acceptable additional scan time.
• 摘要：高分辨率 (HR) 定量 MRI (qMRI) 弛豫测量可提供客观的组织表征，但由于采集时间较长，在临床上仍未得到充分利用。我们提出了一种基于物理的、自我监督的 qMRI 超分辨率框架，该框架使用常规采集的 HR 加权 MRI (wMRI) 扫描作为指导，从而消除了训练期间使用 HR qMRI 地面实况的必要性。我们将超分辨率表述为贝叶斯最大后验推理，最大限度地减少两个差异：（1）从超分辨率 qMRI 图合成的 HR 图像与通过前向信号模型获取的 wMRI 引导之间的差异，以及（2）获取的 LR qMRI 与下采样预测之间的差异。这种基于物理的目标允许模型在没有 HR qMRI 监督的情况下从临床 wMRI 中学习。为了验证这个概念，我们通过使用信号方程从 HR qMRI 合成 wMRI 引导来生成训练数据，然后通过 k 空间截断降低 qMRI 分辨率。深度神经网络学习超分辨率映射。消融实验表明，T1 加权图像主要增强 T1 地图，T2 加权图像改进 T2 地图，而组合制导同时优化地增强所有参数。对来自不同 qMRI 序列的独立采集的体内数据进行验证证实了跨 qMRI 序列的普遍性。利用合成数据训练的模型可以通过 1 分钟采集生成超分辨率地图，其质量可与 5 分钟参考扫描相媲美，利用松弛测量参数的独立于扫描仪的性质。通过将训练与 HR qMRI 要求分离，我们的框架能够通过常规临床图像增强快速 qMRI 采集，为将定量松弛测量法集成到临床工作流程中提供了一条实用的途径，并具有可接受的额外扫描时间。

Title: StereoMV2D: A Sparse Temporal Stereo-Enhanced Framework for Robust Multi-View 3D Object Detection

• Authors: Di Wu, Feng Yang, Wenhui Zhao, Jinwen Yu, Pan Liao, Benlian Xu, Dingwen Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17620
• Pdf URL: https://arxiv.org/pdf/2512.17620
• Copy Paste: [[2512.17620]] StereoMV2D: A Sparse Temporal Stereo-Enhanced Framework for Robust Multi-View 3D Object Detection(https://arxiv.org/abs/2512.17620)
• Keywords: generation
• Abstract: Multi-view 3D object detection is a fundamental task in autonomous driving perception, where achieving a balance between detection accuracy and computational efficiency remains crucial. Sparse query-based 3D detectors efficiently aggregate object-relevant features from multi-view images through a set of learnable queries, offering a concise and end-to-end detection paradigm. Building on this foundation, MV2D leverages 2D detection results to provide high-quality object priors for query initialization, enabling higher precision and recall. However, the inherent depth ambiguity in single-frame 2D detections still limits the accuracy of 3D query generation. To address this issue, we propose StereoMV2D, a unified framework that integrates temporal stereo modeling into the 2D detection-guided multi-view 3D detector. By exploiting cross-temporal disparities of the same object across adjacent frames, StereoMV2D enhances depth perception and refines the query priors, while performing all computations efficiently within 2D regions of interest (RoIs). Furthermore, a dynamic confidence gating mechanism adaptively evaluates the reliability of temporal stereo cues through learning statistical patterns derived from the inter-frame matching matrix together with appearance consistency, ensuring robust detection under object appearance and occlusion. Extensive experiments on the nuScenes and Argoverse 2 datasets demonstrate that StereoMV2D achieves superior detection performance without incurring significant computational overhead. Code will be available at this https URL.
• 摘要：多视角 3D 物体检测是自动驾驶感知中的一项基本任务，其中实现检测精度和计算效率之间的平衡仍然至关重要。基于稀疏查询的 3D 检测器通过一组可学习的查询有效地聚合来自多视图图像的对象相关特征，提供简洁的端到端检测范例。在此基础上，MV2D 利用 2D 检测结果为查询初始化提供高质量的对象先验，从而实现更高的精度和召回率。然而，单帧 2D 检测中固有的深度模糊性仍然限制了 3D 查询生成的准确性。为了解决这个问题，我们提出了 StereoMV2D，这是一个将时间立体建模集成到 2D 检测引导的多视图 3D 检测器中的统一框架。通过利用相邻帧中同一对象的跨时间差异，StereoMV2D 增强了深度感知并细化了查询先验，同时在 2D 感兴趣区域 (RoIs) 内高效执行所有计算。此外，动态置信门机制通过学习从帧间匹配矩阵导出的统计模式以及外观一致性，自适应地评估时间立体线索的可靠性，确保在对象外观和遮挡下进行鲁棒检测。在 nuScenes 和 Argoverse 2 数据集上进行的大量实验表明，StereoMV2D 在不产生大量计算开销的情况下实现了卓越的检测性能。代码将在此 https URL 中提供。

Title: Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMs

• Authors: Zhaolin Cai, Huiyu Duan, Zitong Xu, Fan Li, Zhi Liu, Jing Liu, Wei Shen, Xiongkuo Min, Guangtao Zhai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17640
• Pdf URL: https://arxiv.org/pdf/2512.17640
• Copy Paste: [[2512.17640]] Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMs(https://arxiv.org/abs/2512.17640)
• Keywords: generation, generative
• Abstract: Human-object interaction (HOI) detection aims to localize human-object pairs and the interactions between them. Existing methods operate under a closed-world assumption, treating the task as a classification problem over a small, predefined verb set, which struggles to generalize to the long-tail of unseen or ambiguous interactions in the wild. While recent multi-modal large language models (MLLMs) possess the rich world knowledge required for open-vocabulary understanding, they remain decoupled from existing HOI detectors since fine-tuning them is computationally prohibitive. To address these constraints, we propose \GRASP-HO}, a novel Generative Reasoning And Steerable Perception framework that reformulates HOI detection from the closed-set classification task to the open-vocabulary generation problem. To bridge the vision and cognitive, we first extract hybrid interaction representations, then design a lightweight learnable cognitive steering conduit (CSC) module to inject the fine-grained visual evidence into a frozen MLLM for effective reasoning. To address the supervision mismatch between classification-based HOI datasets and open-vocabulary generative models, we introduce a hybrid guidance strategy that coupling the language modeling loss and auxiliary classification loss, enabling discriminative grounding without sacrificing generative flexibility. Experiments demonstrate state-of-the-art closed-set performance and strong zero-shot generalization, achieving a unified paradigm that seamlessly bridges discriminative perception and generative reasoning for open-world HOI detection.
• 摘要：人与物体交互（HOI）检测旨在定位人与物体对以及它们之间的交互。现有的方法在封闭世界的假设下运行，将任务视为一个小的、预定义的动词集的分类问题，该问题很难推广到野外看不见的或模糊的交互的长尾。虽然最近的多模态大语言模型（MLLM）拥有开放词汇理解所需的丰富的世界知识，但它们仍然与现有的 HOI 检测器脱钩，因为对其进行微调在计算上是令人望而却步的。为了解决这些限制，我们提出了 \GRASP-HO}，一种新颖的生成推理和可引导感知框架，它将 HOI 检测从封闭集分类任务重新表述为开放词汇生成问题。为了弥合视觉和认知，我们首先提取混合交互表示，然后设计一个轻量级可学习认知引导管道（CSC）模块，将细粒度视觉证据注入冻结的 MLLM 中以进行有效推理。为了解决基于分类的 HOI 数据集和开放词汇生成模型之间的监督不匹配问题，我们引入了一种混合指导策略，将语言建模损失和辅助分类损失耦合起来，在不牺牲生成灵活性的情况下实现判别性基础。实验展示了最先进的闭集性能和强大的零样本泛化能力，实现了无缝连接开放世界 HOI 检测的判别感知和生成推理的统一范式。

Title: Region-Constraint In-Context Generation for Instructional Video Editing

• Authors: Zhongwei Zhang, Fuchen Long, Wei Li, Zhaofan Qiu, Wu Liu, Ting Yao, Tao Mei
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2512.17650
• Pdf URL: https://arxiv.org/pdf/2512.17650
• Copy Paste: [[2512.17650]] Region-Constraint In-Context Generation for Instructional Video Editing(https://arxiv.org/abs/2512.17650)
• Keywords: generation
• Abstract: The In-context generation paradigm recently has demonstrated strong power in instructional image editing with both data efficiency and synthesis quality. Nevertheless, shaping such in-context learning for instruction-based video editing is not trivial. Without specifying editing regions, the results can suffer from the problem of inaccurate editing regions and the token interference between editing and non-editing areas during denoising. To address these, we present ReCo, a new instructional video editing paradigm that novelly delves into constraint modeling between editing and non-editing regions during in-context generation. Technically, ReCo width-wise concatenates source and target video for joint denoising. To calibrate video diffusion learning, ReCo capitalizes on two regularization terms, i.e., latent and attention regularization, conducting on one-step backward denoised latents and attention maps, respectively. The former increases the latent discrepancy of the editing region between source and target videos while reducing that of non-editing areas, emphasizing the modification on editing area and alleviating outside unexpected content generation. The latter suppresses the attention of tokens in the editing region to the tokens in counterpart of the source video, thereby mitigating their interference during novel object generation in target video. Furthermore, we propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training. Extensive experiments conducted on four major instruction-based video editing tasks demonstrate the superiority of our proposal.
• 摘要：最近，上下文生成范式在教学图像编辑方面在数据效率和合成质量方面表现出了强大的力量。然而，为基于指令的视频编辑塑造这种情境学习并非易事。在不指定编辑区域的情况下，去噪时结果会出现编辑区域不准确、编辑区域与非编辑区域之间的标记干扰等问题。为了解决这些问题，我们提出了 ReCo，一种新的教学视频编辑范例，它新颖地深入研究了上下文生成过程中编辑和非编辑区域之间的约束建模。从技术上讲，ReCo 宽度方向连接源视频和目标视频以进行联合去噪。为了校准视频扩散学习，ReCo 利用两个正则化项，即潜在正则化和注意力正则化，分别在一步后向去噪潜伏和注意力图上进行。前者增加了源视频和目标视频之间编辑区域的潜在差异，同时减少了非编辑区域的潜在差异，强调了编辑区域的修改，减轻了外部意外内容的生成。后者抑制了编辑区域中的标记对源视频对应部分中的标记的注意，从而减轻了它们在目标视频中的新对象生成期间的干扰。此外，我们提出了一个大规模、高质量的视频编辑数据集，即 ReCo-Data，包含 50 万个指令视频对，有利于模型训练。对四个主要的基于指令的视频编辑任务进行的广泛实验证明了我们建议的优越性。

Title: An Empirical Study of Sampling Hyperparameters in Diffusion-Based Super-Resolution

• Authors: Yudhistira Arief Wibowo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.17675
• Pdf URL: https://arxiv.org/pdf/2512.17675
• Copy Paste: [[2512.17675]] An Empirical Study of Sampling Hyperparameters in Diffusion-Based Super-Resolution(https://arxiv.org/abs/2512.17675)
• Keywords: super-resolution
• Abstract: Diffusion models have shown strong potential for solving inverse problems such as single-image super-resolution, where a high-resolution image is recovered from a low-resolution observation using a pretrained unconditional prior. Conditioning methods, including Diffusion Posterior Sampling (DPS) and Manifold Constrained Gradient (MCG), can substantially improve reconstruction quality, but they introduce additional hyperparameters that require careful tuning. In this work, we conduct an empirical ablation study on FFHQ super-resolution to identify the dominant factors affecting performance when applying conditioning to pretrained diffusion models, and show that the conditioning step size has a significantly greater impact than the diffusion step count, with step sizes in the range of [2.0, 3.0] yielding the best overall performance in our experiments.
• 摘要：扩散模型在解决逆问题（例如单图像超分辨率）方面表现出了强大的潜力，其中使用预训练的无条件先验从低分辨率观察中恢复高分辨率图像。调节方法，包括扩散后采样（DPS）和流形约束梯度（MCG），可以显着提高重建质量，但它们引入了需要仔细调整的额外超参数。在这项工作中，我们对 FFHQ 超分辨率进行了实证消融研究，以确定将条件应用于预训练扩散模型时影响性能的主导因素，并表明条件步长比扩散步数具有更大的影响，步长在 [2.0, 3.0] 范围内在我们的实验中产生最佳的整体性能。

Title: AdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection

• Authors: Yichen Jiang, Mohammed Talha Alam, Sohail Ahmed Khan, Duc-Tien Dang-Nguyen, Fakhri Karray
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17730
• Pdf URL: https://arxiv.org/pdf/2512.17730
• Copy Paste: [[2512.17730]] AdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection(https://arxiv.org/abs/2512.17730)
• Keywords: generation, generative
• Abstract: Recent advances in image generation have led to the widespread availability of highly realistic synthetic media, increasing the difficulty of reliable deepfake detection. A key challenge is generalization, as detectors trained on a narrow class of generators often fail when confronted with unseen models. In this work, we address the pressing need for generalizable detection by leveraging large vision-language models, specifically CLIP, to identify synthetic content across diverse generative techniques. First, we introduce Diff-Gen, a large-scale benchmark dataset comprising 100k diffusion-generated fakes that capture broad spectral artifacts unlike traditional GAN datasets. Models trained on Diff-Gen demonstrate stronger cross-domain generalization, particularly on previously unseen image generators. Second, we propose AdaptPrompt, a parameter-efficient transfer learning framework that jointly learns task-specific textual prompts and visual adapters while keeping the CLIP backbone frozen. We further show via layer ablation that pruning the final transformer block of the vision encoder enhances the retention of high-frequency generative artifacts, significantly boosting detection accuracy. Our evaluation spans 25 challenging test sets, covering synthetic content generated by GANs, diffusion models, and commercial tools, establishing a new state-of-the-art in both standard and cross-domain scenarios. We further demonstrate the framework's versatility through few-shot generalization (using as few as 320 images) and source attribution, enabling the precise identification of generator architectures in closed-set settings.
• 摘要：图像生成的最新进展导致高度逼真的合成媒体的广泛使用，增加了可靠的深度伪造检测的难度。一个关键的挑战是泛化，因为在一小类生成器上训练的检测器在遇到看不见的模型时经常会失败。在这项工作中，我们通过利用大型视觉语言模型（特别是 CLIP）来识别跨不同生成技术的合成内容，从而满足了通用检测的迫切需求。首先，我们介绍 Diff-Gen，这是一个大规模基准数据集，包含 10 万个扩散生成的假数据，与传统 GAN 数据集不同，它捕获广泛的光谱伪影。在 Diff-Gen 上训练的模型表现出更强的跨域泛化能力，特别是在以前未见过的图像生成器上。其次，我们提出了 AdaptPrompt，这是一种参数高效的迁移学习框架，可以联合学习特定于任务的文本提示和视觉适配器，同时保持 CLIP 主干冻结。我们通过层消融进一步表明，修剪视觉编码器的最终变压器块可以增强高频生成伪影的保留，从而显着提高检测精度。我们的评估涵盖 25 个具有挑战性的测试集，涵盖 GAN、扩散模型和商业工具生成的合成内容，在标准和跨域场景中建立了新的最先进技术。我们通过少样本泛化（使用少至 320 个图像）和源归因进一步展示了该框架的多功能性，从而能够在封闭集设置中精确识别生成器架构。

Title: Animate Any Character in Any World

• Authors: Yitong Wang, Fangyun Wei, Hongyang Zhang, Bo Dai, Yan Lu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.17796
• Pdf URL: https://arxiv.org/pdf/2512.17796
• Copy Paste: [[2512.17796]] Animate Any Character in Any World(https://arxiv.org/abs/2512.17796)
• Keywords: generation
• Abstract: Recent advances in world models have greatly enhanced interactive environment simulation. Existing methods mainly fall into two categories: (1) static world generation models, which construct 3D environments without active agents, and (2) controllable-entity models, which allow a single entity to perform limited actions in an otherwise uncontrollable environment. In this work, we introduce AniX, leveraging the realism and structural grounding of static world generation while extending controllable-entity models to support user-specified characters capable of performing open-ended actions. Users can provide a 3DGS scene and a character, then direct the character through natural language to perform diverse behaviors from basic locomotion to object-centric interactions while freely exploring the environment. AniX synthesizes temporally coherent video clips that preserve visual fidelity with the provided scene and character, formulated as a conditional autoregressive video generation problem. Built upon a pre-trained video generator, our training strategy significantly enhances motion dynamics while maintaining generalization across actions and characters. Our evaluation covers a broad range of aspects, including visual quality, character consistency, action controllability, and long-horizon coherence.
• 摘要：世界模型的最新进展极大地增强了交互式环境模拟。现有方法主要分为两类：（1）静态世界生成模型，无需主动代理即可构建 3D 环境；（2）可控实体模型，允许单个实体在不可控的环境中执行有限的操作。在这项工作中，我们引入了 AniX，利用静态世界生成的现实性和结构基础，同时扩展可控实体模型以支持能够执行开放式动作的用户指定角色。用户可以提供 3DGS 场景和角色，然后通过自然语言引导角色执行从基本运动到以对象为中心的交互的各种行为，同时自由探索环境。 AniX 合成时间连贯的视频剪辑，保持所提供场景和角色的视觉保真度，将其表述为条件自回归视频生成问题。我们的训练策略建立在预先训练的视频生成器的基础上，显着增强了运动动态，同时保持了动作和角色的泛化。我们的评估涵盖了广泛的方面，包括视觉质量、角色一致性、动作可控性和长视野连贯性。

Title: InSPECT: Invariant Spectral Features Preservation of Diffusion Models

• Authors: Baohua Yan, Qingyuan Liu, Jennifer Kava, Xuan Di
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17873
• Pdf URL: https://arxiv.org/pdf/2512.17873
• Copy Paste: [[2512.17873]] InSPECT: Invariant Spectral Features Preservation of Diffusion Models(https://arxiv.org/abs/2512.17873)
• Keywords: generation
• Abstract: Modern diffusion models (DMs) have achieved state-of-the-art image generation. However, the fundamental design choice of diffusing data all the way to white noise and then reconstructing it leads to an extremely difficult and computationally intractable prediction task. To overcome this limitation, we propose InSPECT (Invariant Spectral Feature-Preserving Diffusion Model), a novel diffusion model that keeps invariant spectral features during both the forward and backward processes. At the end of the forward process, the Fourier coefficients smoothly converge to a specified random noise, enabling features preservation while maintaining diversity and randomness. By preserving invariant features, InSPECT demonstrates enhanced visual diversity, faster convergence rate, and a smoother diffusion process. Experiments on CIFAR-10, Celeb-A, and LSUN demonstrate that InSPECT achieves on average a 39.23% reduction in FID and 45.80% improvement in IS against DDPM for 10K iterations under specified parameter settings, which demonstrates the significant advantages of preserving invariant features: achieving superior generation quality and diversity, while enhancing computational efficiency and enabling faster convergence rate. To the best of our knowledge, this is the first attempt to analyze and preserve invariant spectral features in diffusion models.
• 摘要：现代扩散模型（DM）已经实现了最先进的图像生成。然而，将数据一直扩散到白噪声然后重建它的基本设计选择导致了极其困难且计算上难以处理的预测任务。为了克服这个限制，我们提出了InSPECT（不变光谱特征保留扩散模型），这是一种新颖的扩散模型，可以在前向和后向过程中保持不变的光谱特征。在前向过程结束时，傅里叶系数平滑地收敛到指定的随机噪声，从而在保持多样性和随机性的同时保留特征。通过保留不变的特征，InSPECT 展示了增强的视觉多样性、更快的收敛速度和更平滑的扩散过程。在 CIFAR-10、Celeb-A 和 LSUN 上的实验表明，在指定参数设置下，InSPECT 相对于 DDPM 10K 次迭代，FID 平均降低 39.23%，IS 提高 45.80%，这证明了保留不变特征的显着优势：实现卓越的生成质量和多样性，同时提高计算效率并实现更快的收敛速度。据我们所知，这是分析和保留扩散模型中不变光谱特征的首次尝试。

Title: Weighted Stochastic Differential Equation to Implement Wasserstein-Fisher-Rao Gradient Flow

• Authors: Herlock Rahimi
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2512.17878
• Pdf URL: https://arxiv.org/pdf/2512.17878
• Copy Paste: [[2512.17878]] Weighted Stochastic Differential Equation to Implement Wasserstein-Fisher-Rao Gradient Flow(https://arxiv.org/abs/2512.17878)
• Keywords: generative
• Abstract: Score-based diffusion models currently constitute the state of the art in continuous generative modeling. These methods are typically formulated via overdamped or underdamped Ornstein--Uhlenbeck-type stochastic differential equations, in which sampling is driven by a combination of deterministic drift and Brownian diffusion, resulting in continuous particle trajectories in the ambient space. While such dynamics enjoy exponential convergence guarantees for strongly log-concave target distributions, it is well known that their mixing rates deteriorate exponentially in the presence of nonconvex or multimodal landscapes, such as double-well potentials. Since many practical generative modeling tasks involve highly non-log-concave target distributions, considerable recent effort has been devoted to developing sampling schemes that improve exploration beyond classical diffusion dynamics. A promising line of work leverages tools from information geometry to augment diffusion-based samplers with controlled mass reweighting mechanisms. This perspective leads naturally to Wasserstein--Fisher--Rao (WFR) geometries, which couple transport in the sample space with vertical (reaction) dynamics on the space of probability measures. In this work, we formulate such reweighting mechanisms through the introduction of explicit correction terms and show how they can be implemented via weighted stochastic differential equations using the Feynman--Kac representation. Our study provides a preliminary but rigorous investigation of WFR-based sampling dynamics, and aims to clarify their geometric and operator-theoretic structure as a foundation for future theoretical and algorithmic developments.
• 摘要：基于分数的扩散模型目前构成了连续生成建模的最先进技术。这些方法通常通过过阻尼或欠阻尼 Ornstein-Uhlenbeck 型随机微分方程来表述，其中采样由确定性漂移和布朗扩散的组合驱动，从而在环境空间中产生连续的粒子轨迹。虽然这种动态对于强对数凹目标分布享有指数收敛保证，但众所周知，在非凸或多峰景观（例如双井势）存在的情况下，它们的混合率呈指数恶化。由于许多实际的生成建模任务涉及高度非对数凹目标分布，因此最近投入了大量精力来开发采样方案，以改进经典扩散动力学之外的探索。一项有前途的工作利用信息几何工具来通过受控质量重加权机制来增强基于扩散的采样器。这种观点自然导致了 Wasserstein-Fisher-Rao (WFR) 几何学，它将样本空间中的传输与概率测度空间上的垂直（反应）动力学耦合起来。在这项工作中，我们通过引入显式校正项来制定这种重新加权机制，并展示如何使用 Feynman--Kac 表示通过加权随机微分方程来实现它们。我们的研究对基于 WFR 的采样动力学进行了初步但严格的研究，旨在阐明其几何和算子理论结构，作为未来理论和算法发展的基础。

Title: RadarGen: Automotive Radar Point Cloud Generation from Cameras

• Authors: Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2512.17897
• Pdf URL: https://arxiv.org/pdf/2512.17897
• Copy Paste: [[2512.17897]] RadarGen: Automotive Radar Point Cloud Generation from Cameras(https://arxiv.org/abs/2512.17897)
• Keywords: generation, generative
• Abstract: We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery. RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form that encodes spatial structure together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Conditioning on images makes the approach broadly compatible, in principle, with existing visual datasets and simulation frameworks, offering a scalable direction for multimodal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step toward unified generative simulation across sensing modalities.
• 摘要：我们推出 RadarGen，这是一种扩散模型，用于从多视图相机图像合成逼真的汽车雷达点云。 RadarGen 通过以鸟瞰图形式表示雷达测量值，将空间结构与雷达截面 (RCS) 和多普勒属性一起编码，从而将有效的图像潜在扩散应用于雷达域。轻量级恢复步骤从生成的地图重建点云。为了更好地使生成与视觉场景保持一致，RadarGen 结合了从预训练基础模型中提取的 BEV 对齐深度、语义和运动线索，引导随机生成过程朝着物理上合理的雷达模式发展。原则上，对图像的调节使该方法与现有的视觉数据集和模拟框架广泛兼容，为多模式生成模拟提供了可扩展的方向。对大规模驾驶数据的评估表明，RadarGen 可以捕获特征雷达测量分布，并缩小与基于真实数据训练的感知模型的差距，标志着跨传感模式的统一生成模拟迈出了一步。

Title: Diffusion Forcing for Multi-Agent Interaction Sequence Modeling

• Authors: Vongani H. Maluleke, Kie Horiuchi, Lea Wilken, Evonne Ng, Jitendra Malik, Angjoo Kanazawa
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2512.17900
• Pdf URL: https://arxiv.org/pdf/2512.17900
• Copy Paste: [[2512.17900]] Diffusion Forcing for Multi-Agent Interaction Sequence Modeling(https://arxiv.org/abs/2512.17900)
• Keywords: generation
• Abstract: Understanding and generating multi-person interactions is a fundamental challenge with broad implications for robotics and social computing. While humans naturally coordinate in groups, modeling such interactions remains difficult due to long temporal horizons, strong inter-agent dependencies, and variable group sizes. Existing motion generation methods are largely task-specific and do not generalize to flexible multi-agent generation. We introduce MAGNet (Multi-Agent Diffusion Forcing Transformer), a unified autoregressive diffusion framework for multi-agent motion generation that supports a wide range of interaction tasks through flexible conditioning and sampling. MAGNet performs dyadic prediction, partner inpainting, and full multi-agent motion generation within a single model, and can autoregressively generate ultra-long sequences spanning hundreds of v. Building on Diffusion Forcing, we introduce key modifications that explicitly model inter-agent coupling during autoregressive denoising, enabling coherent coordination across agents. As a result, MAGNet captures both tightly synchronized activities (e.g, dancing, boxing) and loosely structured social interactions. Our approach performs on par with specialized methods on dyadic benchmarks while naturally extending to polyadic scenarios involving three or more interacting people, enabled by a scalable architecture that is agnostic to the number of agents. We refer readers to the supplemental video, where the temporal dynamics and spatial coordination of generated interactions are best appreciated. Project page: this https URL
• 摘要：理解和生成多人交互是一项基本挑战，对机器人和社交计算具有广泛的影响。虽然人类自然地在群体中进行协调，但由于较长的时间范围、强的代理间依赖性和可变的群体规模，对这种交互进行建模仍然很困难。现有的运动生成方法很大程度上是特定于任务的，不能推广到灵活的多智能体生成。我们引入了 MAGNet（多智能体扩散强制变压器），这是一种用于多智能体运动生成的统一自回归扩散框架，通过灵活的调节和采样支持广泛的交互任务。 MAGNet 在单个模型中执行二元预测、伙伴修复和完整的多智能体运动生成，并且可以自回归生成跨越数百个 v 的超长序列。在扩散强迫的基础上，我们引入了关键修改，可以在自回归降噪期间显式模拟智能体间耦合，从而实现智能体之间的连贯协调。因此，MAGNet 捕获了紧密同步的活动（例如跳舞、拳击）和松散结构的社交互动。我们的方法在二元基准上的性能与专用方法相当，同时自然地扩展到涉及三个或更多交互人员的多元场景，这是由与代理数量无关的可扩展架构实现的。我们建议读者观看补充视频，其中生成的交互的时间动态和空间协调得到了最好的理解。项目页面：此 https URL

Title: Dexterous World Models

• Authors: Byungjun Kim, Taeksoo Kim, Junyoung Lee, Hanbyul Joo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17907
• Pdf URL: https://arxiv.org/pdf/2512.17907
• Copy Paste: [[2512.17907]] Dexterous World Models(https://arxiv.org/abs/2512.17907)
• Keywords: generation
• Abstract: Recent progress in 3D reconstruction has made it easy to create realistic digital twins from everyday environments. However, current digital twins remain largely static and are limited to navigation and view synthesis without embodied interactivity. To bridge this gap, we introduce Dexterous World Model (DWM), a scene-action-conditioned video diffusion framework that models how dexterous human actions induce dynamic changes in static 3D scenes. Given a static 3D scene rendering and an egocentric hand motion sequence, DWM generates temporally coherent videos depicting plausible human-scene interactions. Our approach conditions video generation on (1) static scene renderings following a specified camera trajectory to ensure spatial consistency, and (2) egocentric hand mesh renderings that encode both geometry and motion cues to model action-conditioned dynamics directly. To train DWM, we construct a hybrid interaction video dataset. Synthetic egocentric interactions provide fully aligned supervision for joint locomotion and manipulation learning, while fixed-camera real-world videos contribute diverse and realistic object dynamics. Experiments demonstrate that DWM enables realistic and physically plausible interactions, such as grasping, opening, and moving objects, while maintaining camera and scene consistency. This framework represents a first step toward video diffusion-based interactive digital twins and enables embodied simulation from egocentric actions.
• 摘要：3D 重建领域的最新进展使得从日常环境中创建逼真的数字孪生变得容易。然而，当前的数字孪生在很大程度上仍然是静态的，并且仅限于导航和视图合成，没有具体的交互性。为了弥补这一差距，我们引入了灵巧世界模型 (DWM)，这是一种场景动作条件视频扩散框架，用于模拟灵巧的人类动作如何引起静态 3D 场景的动态变化。给定静态 3D 场景渲染和以自我为中心的手部运动序列，DWM 会生成时间连贯的视频，描绘合理的人景交互。我们的方法将视频生成条件限制为（1）遵循指定摄像机轨迹的静态场景渲染，以确保空间一致性，以及（2）以自我为中心的手部网格渲染，对几何和运动线索进行编码，以直接对动作条件动态进行建模。为了训练 DWM，我们构建了一个混合交互视频数据集。合成的以自我为中心的交互为关节运动和操作学习提供了完全一致的监督，而固定摄像头的真实世界视频则提供了多样化和真实的对象动态。实验表明，DWM 可实现真实且物理上合理的交互，例如抓取、打开和移动对象，同时保持相机和场景的一致性。该框架代表了迈向基于视频传播的交互式数字孪生的第一步，并实现了以自我为中心的行为的具体模拟。

Title: Re-Depth Anything: Test-Time Depth Refinement via Self-Supervised Re-lighting

• Authors: Ananta R. Bhattarai, Helge Rhodin
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.17908
• Pdf URL: https://arxiv.org/pdf/2512.17908
• Copy Paste: [[2512.17908]] Re-Depth Anything: Test-Time Depth Refinement via Self-Supervised Re-lighting(https://arxiv.org/abs/2512.17908)
• Keywords: generative
• Abstract: Monocular depth estimation remains challenging as recent foundation models, such as Depth Anything V2 (DA-V2), struggle with real-world images that are far from the training distribution. We introduce Re-Depth Anything, a test-time self-supervision framework that bridges this domain gap by fusing DA-V2 with the powerful priors of large-scale 2D diffusion models. Our method performs label-free refinement directly on the input image by re-lighting predicted depth maps and augmenting the input. This re-synthesis method replaces classical photometric reconstruction by leveraging shape from shading (SfS) cues in a new, generative context with Score Distillation Sampling (SDS). To prevent optimization collapse, our framework employs a targeted optimization strategy: rather than optimizing depth directly or fine-tuning the full model, we freeze the encoder and only update intermediate embeddings while also fine-tuning the decoder. Across diverse benchmarks, Re-Depth Anything yields substantial gains in depth accuracy and realism over the DA-V2, showcasing new avenues for self-supervision by augmenting geometric reasoning.
• 摘要：单目深度估计仍然具有挑战性，因为最近的基础模型（例如 Depth Anything V2 (DA-V2)）难以处理远离训练分布的现实世界图像。我们引入了 Re-Depth Anything，这是一种测试时的自我监督框架，它通过将 DA-V2 与大规模 2D 扩散模型的强大先验融合来弥补这一领域差距。我们的方法通过重新照亮预测深度图并增强输入来直接对输入图像执行无标签细化。这种重新合成方法通过在新的生成环境中利用分数蒸馏采样 (SDS) 来利用阴影形状 (SfS) 线索，取代了经典的光度重建。为了防止优化崩溃，我们的框架采用了有针对性的优化策略：我们冻结编码器并仅更新中间嵌入，同时微调解码器，而不是直接优化深度或微调整个模型。在不同的基准测试中，Re-Depth Anything 比 DA-V2 在深度准确性和真实性方面取得了显着的进步，通过增强几何推理展示了自我监督的新途径。

Title: Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing

• Authors: Shilong Zhang, He Zhang, Zhifei Zhang, Chongjian Ge, Shuchen Xue, Shaoteng Liu, Mengwei Ren, Soo Ye Kim, Yuqian Zhou, Qing Liu, Daniil Pakhomov, Kai Zhang, Zhe Lin, Ping Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2512.17909
• Pdf URL: https://arxiv.org/pdf/2512.17909
• Copy Paste: [[2512.17909]] Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing(https://arxiv.org/abs/2512.17909)
• Keywords: generation, generative
• Abstract: Modern Latent Diffusion Models (LDMs) typically operate in low-level Variational Autoencoder (VAE) latent spaces that are primarily optimized for pixel-level reconstruction. To unify vision generation and understanding, a burgeoning trend is to adopt high-dimensional features from representation encoders as generative latents. However, we empirically identify two fundamental obstacles in this paradigm: (1) the discriminative feature space lacks compact regularization, making diffusion models prone to off-manifold latents that lead to inaccurate object structures; and (2) the encoder's inherently weak pixel-level reconstruction hinders the generator from learning accurate fine-grained geometry and texture. In this paper, we propose a systematic framework to adapt understanding-oriented encoder features for generative tasks. We introduce a semantic-pixel reconstruction objective to regularize the latent space, enabling the compression of both semantic information and fine-grained details into a highly compact representation (96 channels with 16x16 spatial downsampling). This design ensures that the latent space remains semantically rich and achieves state-of-the-art image reconstruction, while remaining compact enough for accurate generation. Leveraging this representation, we design a unified Text-to-Image (T2I) and image editing model. Benchmarking against various feature spaces, we demonstrate that our approach achieves state-of-the-art reconstruction, faster convergence, and substantial performance gains in both T2I and editing tasks, validating that representation encoders can be effectively adapted into robust generative components.
• 摘要：现代潜在扩散模型 (LDM) 通常在低级变分自动编码器 (VAE) 潜在空间中运行，这些空间主要针对像素级重建进行优化。为了统一视觉生成和理解，一个新兴的趋势是采用表示编码器的高维特征作为生成潜伏。然而，我们凭经验确定了该范式中的两个基本障碍：（1）判别性特征空间缺乏紧凑的正则化，使得扩散模型容易出现偏离流形的潜在问题，从而导致对象结构不准确； (2)编码器本质上较弱的像素级重建阻碍了生成器学习精确的细粒度几何和纹理。在本文中，我们提出了一个系统框架，以适应面向理解的编码器特征来执行生成任务。我们引入了语义像素重建目标来规范潜在空间，从而将语义信息和细粒度细节压缩为高度紧凑的表示（96 个通道，16x16 空间下采样）。这种设计确保潜在空间在语义上保持丰富并实现最先进的图像重建，同时保持足够紧凑以进行准确生成。利用这种表示，我们设计了统一的文本到图像（T2I）和图像编辑模型。通过对各种特征空间进行基准测试，我们证明了我们的方法在 T2I 和编辑任务中实现了最先进的重建、更快的收敛和显着的性能提升，验证了表示编码器可以有效地适应强大的生成组件。