2025-11-27

Title: Pretraining Transformer-Based Models on Diffusion-Generated Synthetic Graphs for Alzheimer's Disease Prediction

• Authors: Abolfazl Moslemi, Hossein Peyvandi
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2511.20704
• Pdf URL: https://arxiv.org/pdf/2511.20704
• Copy Paste: [[2511.20704]] Pretraining Transformer-Based Models on Diffusion-Generated Synthetic Graphs for Alzheimer's Disease Prediction(https://arxiv.org/abs/2511.20704)
• Keywords: generation
• Abstract: Early and accurate detection of Alzheimer's disease (AD) is crucial for enabling timely intervention and improving outcomes. However, developing reliable machine learning (ML) models for AD diagnosis is challenging due to limited labeled data, multi-site heterogeneity, and class imbalance. We propose a Transformer-based diagnostic framework that combines diffusion-based synthetic data generation with graph representation learning and transfer learning. A class-conditional denoising diffusion probabilistic model (DDPM) is trained on the real-world NACC dataset to generate a large synthetic cohort that mirrors multimodal clinical and neuroimaging feature distributions while balancing diagnostic classes. Modality-specific Graph Transformer encoders are first pretrained on this synthetic data to learn robust, class-discriminative representations and are then frozen while a neural classifier is trained on embeddings from the original NACC data. We quantify distributional alignment between real and synthetic cohorts using metrics such as Maximum Mean Discrepancy (MMD), Frechet distance, and energy distance, and complement discrimination metrics with calibration and fixed-specificity sensitivity analyses. Empirically, our framework outperforms standard baselines, including early and late fusion deep neural networks and the multimodal graph-based model MaGNet, yielding higher AUC, accuracy, sensitivity, and specificity under subject-wise cross-validation on NACC. These results show that diffusion-based synthetic pretraining with Graph Transformers can improve generalization in low-sample, imbalanced clinical prediction settings.
• 摘要：早期准确检测阿尔茨海默病 (AD) 对于及时干预和改善治疗结果至关重要。然而，由于标记数据有限、多站点异质性和类别不平衡，开发用于 AD 诊断的可靠机器学习 (ML) 模型具有挑战性。我们提出了一个基于 Transformer 的诊断框架，它将基于扩散的合成数据生成与图表示学习和迁移学习相结合。在现实世界的 NACC 数据集上训练类别条件去噪扩散概率模型 (DDPM)，以生成一个大型合成队列，该队列反映多模式临床和神经影像特征分布，同时平衡诊断类别。特定于模态的 Graph Transformer 编码器首先在此合成数据上进行预训练，以学习稳健的、有类别区分性的表示，然后在神经分类器在原始 NACC 数据的嵌入上进行训练时被冻结。我们使用最大平均差异（MMD）、弗雷切特距离和能量距离等指标来量化真实队列和合成队列之间的分布一致性，并通过校准和固定特异性敏感性分析来补充歧视指标。根据经验，我们的框架优于标准基线，包括早期和晚期融合深度神经网络和基于多模态图的模型 MaGNet，在 NACC 上的主题交叉验证下产生更高的 AUC、准确性、敏感性和特异性。这些结果表明，使用 Graph Transformer 进行基于扩散的综合预训练可以提高低样本、不平衡临床预测设置中的泛化能力。

Title: Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation

• Authors: Inferix Team: Tianyu Feng, Yizeng Han, Jiahao He, Yuanyu He, Xi Lin, Teng Liu, Hanfeng Lu, Jiasheng Tang, Wei Wang, Zhiyuan Wang, Jichao Wu, Mingyang Yang, Yinghao Yu, Zeyu Zhang, Bohan Zhuang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.20714
• Pdf URL: https://arxiv.org/pdf/2511.20714
• Copy Paste: [[2511.20714]] Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation(https://arxiv.org/abs/2511.20714)
• Keywords: generation
• Abstract: World models serve as core simulators for fields such as agentic AI, embodied AI, and gaming, capable of generating long, physically realistic, and interactive high-quality videos. Moreover, scaling these models could unlock emergent capabilities in visual perception, understanding, and reasoning, paving the way for a new paradigm that moves beyond current LLM-centric vision foundation models. A key breakthrough empowering them is the semi-autoregressive (block-diffusion) decoding paradigm, which merges the strengths of diffusion and autoregressive methods by generating video tokens in block-applying diffusion within each block while conditioning on previous ones, resulting in more coherent and stable video sequences. Crucially, it overcomes limitations of standard video diffusion by reintroducing LLM-style KV Cache management, enabling efficient, variable-length, and high-quality generation. Therefore, Inferix is specifically designed as a next-generation inference engine to enable immersive world synthesis through optimized semi-autoregressive decoding processes. This dedicated focus on world simulation distinctly sets it apart from systems engineered for high-concurrency scenarios (like vLLM or SGLang) and from classic video diffusion models (such as xDiTs). Inferix further enhances its offering with interactive video streaming and profiling, enabling real-time interaction and realistic simulation to accurately model world dynamics. Additionally, it supports efficient benchmarking through seamless integration of LV-Bench, a new fine-grained evaluation benchmark tailored for minute-long video generation scenarios. We hope the community will work together to advance Inferix and foster world model exploration.
• 摘要：世界模型作为代理人工智能、实体人工智能和游戏等领域的核心模拟器，能够生成长的、物理真实的、交互式的高质量视频。此外，扩展这些模型可以释放视觉感知、理解和推理方面的新兴能力，为超越当前以法学硕士为中心的视觉基础模型的新范式铺平道路。赋予它们能力的一个关键突破是半自回归（块扩散）解码范例，它通过在每个块内应用块扩散生成视频令牌，同时以先前的令牌为条件，融合了扩散和自回归方法的优点，从而产生更加连贯和稳定的视频序列。至关重要的是，它通过重新引入 LLM 风格的 KV 缓存管理来克服标准视频扩散的限制，从而实现高效、可变长度和高质量的生成。因此，Inferix 被专门设计为下一代推理引擎，通过优化的半自回归解码过程实现沉浸式世界合成。这种对世界模拟的专注使其与专为高并发场景（如 vLLM 或 SGLang）设计的系统以及经典视频扩散模型（如 xDiT）截然不同。 Inferix 通过交互式视频流和分析进一步增强其产品，实现实时交互和真实模拟，以准确地模拟世界动态。此外，它还通过与 LV-Bench 的无缝集成来支持高效的基准测试，LV-Bench 是专为时长视频生成场景量身定制的全新细粒度评估基准。我们希望社区共同努力推进 Inferix 并促进世界模型探索。

Title: DeeAD: Dynamic Early Exit of Vision-Language Action for Efficient Autonomous Driving

• Authors: Haibo HU, Lianming Huang, Nan Guan, Chun Jason Xue
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2511.20720
• Pdf URL: https://arxiv.org/pdf/2511.20720
• Copy Paste: [[2511.20720]] DeeAD: Dynamic Early Exit of Vision-Language Action for Efficient Autonomous Driving(https://arxiv.org/abs/2511.20720)
• Keywords: generation
• Abstract: Vision-Language Action (VLA) models unify perception, reasoning, and trajectory generation for autonomous driving, but suffer from significant inference latency due to deep transformer stacks. We present DeeAD, a training-free, action-guided early-exit framework that accelerates VLA planning by evaluating the physical feasibility of intermediate trajectories. Instead of relying on confidence scores, DeeAD terminates inference when predicted trajectories align with lightweight planning priors (e.g., Navigation or Low-precision Planning) within a tolerable deviation (<2m). To improve efficiency, we introduce a multi-hop controller that adaptively skips redundant layers based on the change rate of scores. DeeAD integrates into existing VLA models, such as ORION, without requiring retraining. Experiments on the Bench2Drive benchmark demonstrate up to 28% transformer-layer sparsity and 29% latency reduction, while preserving planning quality and safety.
• 摘要：视觉语言动作 (VLA) 模型统一了自动驾驶的感知、推理和轨迹生成，但由于深度变压器堆栈而存在显着的推理延迟。我们提出了 DeeAD，这是一种免训练、以行动为导向的提前退出框架，可通过评估中间轨迹的物理可行性来加速 VLA 规划。当预测轨迹与轻量级规划先验（例如导航或低精度规划）在可容忍偏差（<2m）内一致时，DeeAD 不依赖置信度分数，而是终止推理。为了提高效率，我们引入了一个多跳控制器，它根据分数的变化率自适应地跳过冗余层。 DeeAD 集成到现有的 VLA 模型（例如 ORION）中，无需重新训练。 Bench2Drive 基准测试表明，变压器层稀疏性高达 28%，延迟减少了 29%，同时保持了规划质量和安全性。

Title: DinoLizer: Learning from the Best for Generative Inpainting Localization

• Authors: Minh Thong Doi (IMT Nord Europe, CRIStAL), Jan Butora (CRIStAL), Vincent Itier (IMT Nord Europe, CRIStAL), Jérémie Boulanger (CRIStAL), Patrick Bas (CRIStAL)
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.20722
• Pdf URL: https://arxiv.org/pdf/2511.20722
• Copy Paste: [[2511.20722]] DinoLizer: Learning from the Best for Generative Inpainting Localization(https://arxiv.org/abs/2511.20722)
• Keywords: generative
• Abstract: We introduce DinoLizer, a DINOv2-based model for localizing manipulated regions in generative inpainting. Our method builds on a DINOv2 model pretrained to detect synthetic images on the B-Free dataset. We add a linear classification head on top of the Vision Transformer's patch embeddings to predict manipulations at a $14\times 14$ patch resolution. The head is trained to focus on semantically altered regions, treating non-semantic edits as part of the original content. Because the ViT accepts only fixed-size inputs, we use a sliding-window strategy to aggregate predictions over larger images; the resulting heatmaps are post-processed to refine the estimated binary manipulation masks. Empirical results show that DinoLizer surpasses state-of-the-art local manipulation detectors on a range of inpainting datasets derived from different generative models. It remains robust to common post-processing operations such as resizing, noise addition, and JPEG (double) compression. On average, DinoLizer achieves a 12\% higher Intersection-over-Union (IoU) than the next best model, with even greater gains after post-processing. Our experiments with off-the-shelf DINOv2 demonstrate the strong representational power of Vision Transformers for this task. Finally, extensive ablation studies comparing DINOv2 and its successor, DINOv3, in deepfake localization confirm DinoLizer's superiority. The code will be publicly available upon acceptance of the paper.
• 摘要：我们引入了 DinoLizer，这是一种基于 DINOv2 的模型，用于在生成修复中定位操作区域。我们的方法建立在经过预训练的 DINOv2 模型之上，用于检测 B-Free 数据集上的合成图像。我们在 Vision Transformer 的补丁嵌入之上添加了一个线性分类头，以预测 $14\times 14$ 补丁分辨率的操作。头部经过训练，专注于语义改变的区域，将非语义编辑视为原始内容的一部分。由于 ViT 仅接受固定大小的输入，因此我们使用滑动窗口策略来聚合较大图像的预测；对生成的热图进行后处理，以细化估计的二进制操作掩模。实证结果表明，DinoLizer 在源自不同生成模型的一系列修复数据集上超越了最先进的局部操作检测器。它对于常见的后处理操作（例如调整大小、添加噪声和 JPEG（双）压缩）仍然具有鲁棒性。平均而言，DinoLizer 的交集比 (IoU) 比次优模型高出 12%，并且在后处理后获得的收益甚至更大。我们使用现成的 DINOv2 进行的实验证明了 Vision Transformers 对于这项任务的强大表征能力。最后，在 Deepfake 定位中比较 DINOv2 及其后继者 DINOv3 的广泛消融研究证实了 DinoLizer 的优越性。该代码将在论文被接受后公开。

Title: Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge

• Authors: Yuhang Wang, Heye Huang, Zhenhua Xu, Kailai Sun, Baoshen Guo, Jinhua Zhao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.20726
• Pdf URL: https://arxiv.org/pdf/2511.20726
• Copy Paste: [[2511.20726]] Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge(https://arxiv.org/abs/2511.20726)
• Keywords: generation
• Abstract: Autonomous driving faces critical challenges in rare long-tail events and complex multi-agent interactions, which are scarce in real-world data yet essential for robust safety validation. This paper presents a high-fidelity scenario generation framework that integrates a conditional variational autoencoder (CVAE) with a large language model (LLM). The CVAE encodes historical trajectories and map information from large-scale naturalistic datasets to learn latent traffic structures, enabling the generation of physically consistent base scenarios. Building on this, the LLM acts as an adversarial reasoning engine, parsing unstructured scene descriptions into domain-specific loss functions and dynamically guiding scenario generation across varying risk levels. This knowledge-driven optimization balances realism with controllability, ensuring that generated scenarios remain both plausible and risk-sensitive. Extensive experiments in CARLA and SMARTS demonstrate that our framework substantially increases the coverage of high-risk and long-tail events, improves consistency between simulated and real-world traffic distributions, and exposes autonomous driving systems to interactions that are significantly more challenging than those produced by existing rule- or data-driven methods. These results establish a new pathway for safety validation, enabling principled stress-testing of autonomous systems under rare but consequential events.
• 摘要：自动驾驶在罕见的长尾事件和复杂的多智能体交互中面临着严峻的挑战，这些数据在现实世界中很少见，但对于稳健的安全验证至关重要。本文提出了一种高保真场景生成框架，它将条件变分自动编码器（CVAE）与大型语言模型（LLM）集成在一起。 CVAE 对来自大规模自然数据集的历史轨迹和地图信息进行编码，以学习潜在的交通结构，从而能够生成物理上一致的基本场景。在此基础上，LLM 充当对抗性推理引擎，将非结构化场景描述解析为特定领域的损失函数，并动态指导不同风险级别的场景生成。这种知识驱动的优化平衡了现实性和可控性，确保生成的场景既合理又对风险敏感。 CARLA 和 SMARTS 中的大量实验表明，我们的框架大大增加了高风险和长尾事件的覆盖范围，提高了模拟和现实世界交通分布之间的一致性，并使自动驾驶系统面临的交互比现有规则或数据驱动方法产生的交互更具挑战性。这些结果建立了一条新的安全验证途径，能够在罕见但重大的事件下对自主系统进行原则性的压力测试。

Title: Layer-Aware Video Composition via Split-then-Merge

• Authors: Ozgur Kara, Yujia Chen, Ming-Hsuan Yang, James M. Rehg, Wen-Sheng Chu, Du Tran
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.20809
• Pdf URL: https://arxiv.org/pdf/2511.20809
• Copy Paste: [[2511.20809]] Layer-Aware Video Composition via Split-then-Merge(https://arxiv.org/abs/2511.20809)
• Keywords: generation, generative
• Abstract: We present Split-then-Merge (StM), a novel framework designed to enhance control in generative video composition and address its data scarcity problem. Unlike conventional methods relying on annotated datasets or handcrafted rules, StM splits a large corpus of unlabeled videos into dynamic foreground and background layers, then self-composes them to learn how dynamic subjects interact with diverse scenes. This process enables the model to learn the complex compositional dynamics required for realistic video generation. StM introduces a novel transformation-aware training pipeline that utilizes a multi-layer fusion and augmentation to achieve affordance-aware composition, alongside an identity-preservation loss that maintains foreground fidelity during blending. Experiments show StM outperforms SoTA methods in both quantitative benchmarks and in humans/VLLM-based qualitative evaluations. More details are available at our project page: this https URL
• 摘要：我们提出了分割然后合并（StM），这是一种新颖的框架，旨在增强生成视频合成的控制并解决其数据稀缺问题。与依赖注释数据集或手工规则的传统方法不同，StM 将大量未标记视频分割为动态前景层和背景层，然后自行组合它们以了解动态主体如何与不同场景交互。此过程使模型能够学习真实视频生成所需的复杂合成动态。 StM 引入了一种新颖的变换感知训练管道，该管道利用多层融合和增强来实现可供性感知的组合，以及在混合过程中保持前景保真度的身份保留损失。实验表明，StM 在定量基准和基于人类/VLLM 的定性评估方面均优于 SoTA 方法。更多详细信息请访问我们的项目页面：此 https URL

Title: Training-Free Diffusion Priors for Text-to-Image Generation via Optimization-based Visual Inversion

• Authors: Samuele Dell'Erba, Andrew D. Bagdanov
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.20821
• Pdf URL: https://arxiv.org/pdf/2511.20821
• Copy Paste: [[2511.20821]] Training-Free Diffusion Priors for Text-to-Image Generation via Optimization-based Visual Inversion(https://arxiv.org/abs/2511.20821)
• Keywords: generation
• Abstract: Diffusion models have established the state-of-the-art in text-to-image generation, but their performance often relies on a diffusion prior network to translate text embeddings into the visual manifold for easier decoding. These priors are computationally expensive and require extensive training on massive datasets. In this work, we challenge the necessity of a trained prior at all by employing Optimization-based Visual Inversion (OVI), a training-free and data-free alternative, to replace the need for a prior. OVI initializes a latent visual representation from random pseudo-tokens and iteratively optimizes it to maximize the cosine similarity with input textual prompt embedding. We further propose two novel constraints, a Mahalanobis-based and a Nearest-Neighbor loss, to regularize the OVI optimization process toward the distribution of realistic images. Our experiments, conducted on Kandinsky 2.2, show that OVI can serve as an alternative to traditional priors. More importantly, our analysis reveals a critical flaw in current evaluation benchmarks like T2I-CompBench++, where simply using the text embedding as a prior achieves surprisingly high scores, despite lower perceptual quality. Our constrained OVI methods improve visual fidelity over this baseline, with the Nearest-Neighbor approach proving particularly effective, achieving quantitative scores comparable to or higher than the state-of-the-art data-efficient prior, indicating that the idea merits further investigation. The code will be publicly available upon acceptance.
• 摘要：扩散模型已经建立了文本到图像生成的最先进技术，但其性能通常依赖于扩散先验网络将文本嵌入转换为视觉流形以便于解码。这些先验的计算成本很高，并且需要对大量数据集进行大量训练。在这项工作中，我们通过采用基于优化的视觉反演（OVI）（一种免训练和无数据的替代方案）来取代对先验的需要，从而挑战了先验训练的必要性。 OVI 从随机伪标记初始化潜在视觉表示，并迭代优化它以最大化与输入文本提示嵌入的余弦相似度。我们进一步提出了两个新颖的约束，即基于马哈拉诺比斯的损失和最近邻损失，以将 OVI 优化过程规范为真实图像的分布。我们在 Kandinsky 2.2 上进行的实验表明，OVI 可以作为传统先验的替代方案。更重要的是，我们的分析揭示了 T2I-CompBench++ 等当前评估基准的一个关键缺陷，即仅使用文本嵌入作为先验即可获得令人惊讶的高分，尽管感知质量较低。我们的约束 OVI 方法提高了该基线的视觉保真度，最近邻方法被证明特别有效，达到了与最先进的数据效率先验相当或更高的定量分数，表明该想法值得进一步研究。该代码将在接受后公开发布。

Title: RefTr: Recurrent Refinement of Confluent Trajectories for 3D Vascular Tree Centerline Graphs

• Authors: Roman Naeem, David Hagerman, Jennifer Alvén, Fredrik Kahl
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.20823
• Pdf URL: https://arxiv.org/pdf/2511.20823
• Copy Paste: [[2511.20823]] RefTr: Recurrent Refinement of Confluent Trajectories for 3D Vascular Tree Centerline Graphs(https://arxiv.org/abs/2511.20823)
• Keywords: generation
• Abstract: Tubular trees, such as blood vessels and lung airways, are essential for material transport within the human body. Accurately detecting their centerlines with correct tree topology is critical for clinical tasks such as diagnosis, treatment planning, and surgical navigation. In these applications, maintaining high recall is crucial, as missing small branches can result in fatal mistakes caused by incomplete assessments or undetected abnormalities. We present RefTr, a 3D image-to-graph model for centerline generation of vascular trees via recurrent refinement of confluent trajectories. RefTr uses a Producer-Refiner architecture based on a Transformer decoder, where the Producer proposes a set of initial confluent trajectories that are recurrently refined by the Refiner to produce final trajectories, which forms the centerline graph. The confluent trajectory representation enables refinement of complete trajectories while explicitly enforcing a valid tree topology. The recurrent refinement scheme improves precision and reuses the same Refiner block across multiple steps, yielding a 2.4x reduction in decoder parameters compared to previous SOTA. We also introduce an efficient non-maximum suppression algorithm for spatial tree graphs to merge duplicate branches and boost precision. Across multiple public centerline datasets, RefTr achieves superior recall and comparable precision to previous SOTA, while offering faster inference and substantially fewer parameters, demonstrating its potential as a new state-of-the-art framework for vascular tree analysis in 3D medical imaging.
• 摘要：管状树，例如血管和肺气道，对于人体内的物质运输至关重要。使用正确的树拓扑准确检测其中心线对于诊断、治疗计划和手术导航等临床任务至关重要。在这些应用中，保持高召回率至关重要，因为丢失小分支可能会因评估不完整或未检测到的异常而导致致命错误。我们提出了 RefTr，这是一种 3D 图像到图形模型，用于通过汇合轨迹的循环细化来生成血管树的中心线。 RefTr 使用基于 Transformer 解码器的 Producer-Refiner 架构，其中 Producer 提出一组初始汇合轨迹，这些轨迹由 Refiner 反复细化以产生最终轨迹，从而形成中心线图。汇合轨迹表示可以细化完整的轨迹，同时显式地强制执行有效的树拓扑。循环细化方案提高了精度，并在多个步骤中重复使用相同的 Refiner 块，与之前的 SOTA 相比，解码器参数减少了 2.4 倍。我们还引入了一种有效的空间树图非极大值抑制算法，以合并重复分支并提高精度。在多个公共中心线数据集上，RefTr 实现了与之前的 SOTA 相比更高的召回率和可比的精度，同时提供更快的推理和更少的参数，展示了其作为 3D 医学成像中血管树分析的新的最先进框架的潜力。

Title: Primal: A Unified Deterministic Framework for Quasi-Orthogonal Hashing and Manifold Learning

• Authors: Vladimer Khasia
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.20839
• Pdf URL: https://arxiv.org/pdf/2511.20839
• Copy Paste: [[2511.20839]] Primal: A Unified Deterministic Framework for Quasi-Orthogonal Hashing and Manifold Learning(https://arxiv.org/abs/2511.20839)
• Keywords: generation
• Abstract: We present Primal, a deterministic feature mapping framework that harnesses the number-theoretic independence of prime square roots to construct robust, tunable vector representations. Diverging from standard stochastic projections (e.g., Random Fourier Features), our method exploits the Besicovitch property to create irrational frequency modulations that guarantee infinite non-repeating phase trajectories. We formalize two distinct algorithmic variants: (1) StaticPrime, a sequence generation method that produces temporal position encodings empirically approaching the theoretical Welch bound for quasi-orthogonality; and (2) DynamicPrime, a tunable projection layer for input-dependent feature mapping. A central novelty of the dynamic framework is its ability to unify two disparate mathematical utility classes through a single scaling parameter {\sigma}. In the low-frequency regime, the method acts as an isometric kernel map, effectively linearizing non-convex geometries (e.g., spirals) to enable high-fidelity signal reconstruction and compressive sensing. Conversely, the high-frequency regime induces chaotic phase wrapping, transforming the projection into a maximum-entropy one-way hash suitable for Hyperdimensional Computing and privacy-preserving Split Learning. Empirical evaluations demonstrate that our framework yields superior orthogonality retention and distribution tightness compared to normalized Gaussian baselines, establishing it as a computationally efficient, mathematically rigorous alternative to random matrix projections. The code is available at this https URL
• 摘要：我们提出了 Primal，一个确定性的特征映射框架，它利用素数平方根的数论独立性来构建鲁棒、可调的向量表示。与标准随机投影（例如随机傅里叶特征）不同，我们的方法利用贝西科维奇属性来创建无理频率调制，以保证无限的非重复相位轨迹。我们形式化了两种不同的算法变体：（1）StaticPrime，一种序列生成方法，它产生时间位置编码，根据经验接近准正交性的理论韦尔奇界限； (2) DynamicPrime，用于依赖于输入的特征映射的可调投影层。动态框架的一个核心新颖之处在于它能够通过单个缩放参数 {\sigma} 统一两个不同的数学实用程序类。在低频状态下，该方法充当等距核图，有效地线性化非凸几何形状（例如螺旋），以实现高保真信号重建和压缩传感。相反，高频状态会引起混沌相位缠绕，将投影转换为适合超维计算和隐私保护分割学习的最大熵单向哈希。经验评估表明，与归一化高斯基线相比，我们的框架具有优异的正交性保留和分布紧密性，使其成为随机矩阵投影的计算高效、数学严格的替代方案。该代码可在此 https URL 获取

Title: Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries

• Authors: Sree Bhattacharyya, Yaman Kumar Singla, Sudhir Yarram, Somesh Kumar Singh, Harini S I, James Z. Wang
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2511.20854
• Pdf URL: https://arxiv.org/pdf/2511.20854
• Copy Paste: [[2511.20854]] Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries(https://arxiv.org/abs/2511.20854)
• Keywords: generation
• Abstract: Visual content memorability has intrigued the scientific community for decades, with applications ranging widely, from understanding nuanced aspects of human memory to enhancing content design. A significant challenge in progressing the field lies in the expensive process of collecting memorability annotations from humans. This limits the diversity and scalability of datasets for modeling visual content memorability. Most existing datasets are limited to collecting aggregate memorability scores for visual content, not capturing the nuanced memorability signals present in natural, open-ended recall descriptions. In this work, we introduce the first large-scale unsupervised dataset designed explicitly for modeling visual memorability signals, containing over 82,000 videos, accompanied by descriptive recall data. We leverage tip-of-the-tongue (ToT) retrieval queries from online platforms such as Reddit. We demonstrate that our unsupervised dataset provides rich signals for two memorability-related tasks: recall generation and ToT retrieval. Large vision-language models fine-tuned on our dataset outperform state-of-the-art models such as GPT-4o in generating open-ended memorability descriptions for visual content. We also employ a contrastive training strategy to create the first model capable of performing multimodal ToT retrieval. Our dataset and models present a novel direction, facilitating progress in visual content memorability research.
• 摘要：视觉内容的可记忆性几十年来一直引起科学界的兴趣，其应用范围广泛，从理解人类记忆的细微差别到增强内容设计。该领域发展的一个重大挑战在于从人类那里收集记忆注释的昂贵过程。这限制了用于建模视觉内容记忆性的数据集的多样性和可扩展性。大多数现有数据集仅限于收集视觉内容的总体记忆分数，而不是捕获自然的、开放式回忆描述中存在的细微的记忆信号。在这项工作中，我们引入了第一个专门为视觉记忆信号建模而设计的大规模无监督数据集，其中包含超过 82,000 个视频，并附有描述性回忆数据。我们利用来自 Reddit 等在线平台的舌尖 (ToT) 检索查询。我们证明，我们的无监督数据集为两个与记忆相关的任务提供了丰富的信号：回忆生成和 ToT 检索。在我们的数据集上进行微调的大型视觉语言模型在为视觉内容生成开放式记忆描述方面优于 GPT-4o 等最先进的模型。我们还采用对比训练策略来创建第一个能够执行多模式 ToT 检索的模型。我们的数据集和模型提出了一个新颖的方向，促进了视觉内容记忆性研究的进展。

Title: Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation

• Authors: Taehoon Kim, Henry Gouk, Timothy Hospedales
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.20889
• Pdf URL: https://arxiv.org/pdf/2511.20889
• Copy Paste: [[2511.20889]] Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation(https://arxiv.org/abs/2511.20889)
• Keywords: generative
• Abstract: Test-time alignment (TTA) aims to adapt models to specific rewards during inference. However, existing methods tend to either under-optimise or over-optimise (reward hack) the target reward function. We propose Null-Text Test-Time Alignment (Null-TTA), which aligns diffusion models by optimising the unconditional embedding in classifier-free guidance, rather than manipulating latent or noise variables. Due to the structured semantic nature of the text embedding space, this ensures alignment occurs on a semantically coherent manifold and prevents reward hacking (exploiting non-semantic noise patterns to improve the reward). Since the unconditional embedding in classifier-free guidance serves as the anchor for the model's generative distribution, Null-TTA directly steers model's generative distribution towards the target reward rather than just adjusting the samples, even without updating model parameters. Thanks to these desirable properties, we show that Null-TTA achieves state-of-the-art target test-time alignment while maintaining strong cross-reward generalisation. This establishes semantic-space optimisation as an effective and principled novel paradigm for TTA.
• 摘要：测试时间对齐（TTA）旨在使模型在推理过程中适应特定的奖励。然而，现有方法往往对目标奖励函数优化不足或过度优化（奖励黑客）。我们提出了空文本测试时间对齐（Null-TTA），它通过优化无分类器指导中的无条件嵌入来对齐扩散模型，而不是操纵潜在变量或噪声变量。由于文本嵌入空间的结构化语义性质，这确保了对齐发生在语义连贯的流形上，并防止奖励黑客（利用非语义噪声模式来提高奖励）。由于无分类器指导中的无条件嵌入作为模型生成分布的锚，因此 Null-TTA 直接引导模型的生成分布朝向目标奖励，而不是仅仅调整样本，甚至不更新模型参数。由于这些理想的特性，我们表明 Null-TTA 实现了最先进的目标测试时间对齐，同时保持强大的交叉奖励泛化。这将语义空间优化确立为 TTA 的有效且有原则的新颖范式。

Title: GaINeR: Geometry-Aware Implicit Network Representation

• Authors: Weronika Jakubowska, Mikołaj Zieliński, Rafał Tobiasz, Krzysztof Byrski, Maciej Zięba, Dominik Belter, Przemysław Spurek
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.20924
• Pdf URL: https://arxiv.org/pdf/2511.20924
• Copy Paste: [[2511.20924]] GaINeR: Geometry-Aware Implicit Network Representation(https://arxiv.org/abs/2511.20924)
• Keywords: super-resolution
• Abstract: Implicit Neural Representations (INRs) have become an essential tool for modeling continuous 2D images, enabling high-fidelity reconstruction, super-resolution, and compression. Popular architectures such as SIREN, WIRE, and FINER demonstrate the potential of INR for capturing fine-grained image details. However, traditional INRs often lack explicit geometric structure and have limited capabilities for local editing or integration with physical simulation, restricting their applicability in dynamic or interactive settings. To address these limitations, we propose GaINeR: Geometry-Aware Implicit Network Representation, a novel framework for 2D images that combines trainable Gaussian distributions with a neural network-based INR. For a given image coordinate, the model retrieves the K nearest Gaussians, aggregates distance-weighted embeddings, and predicts the RGB value via a neural network. This design enables continuous image representation, interpretable geometric structure, and flexible local editing, providing a foundation for physically aware and interactive image manipulation. The official implementation of our method is publicly available at this https URL.
• 摘要：隐式神经表示 (INR) 已成为连续 2D 图像建模的重要工具，可实现高保真重建、超分辨率和压缩。 SIREN、WIRE 和 FINER 等流行架构展示了 INR 在捕获细粒度图像细节方面的潜力。然而，传统的 INR 通常缺乏明确的几何结构，并且本地编辑或与物理模拟集成的能力有限，限制了它们在动态或交互式设置中的适用性。为了解决这些限制，我们提出了 GaINeR：几何感知隐式网络表示，这是一种新颖的 2D 图像框架，它将可训练的高斯分布与基于神经网络的 INR 相结合。对于给定的图像坐标，模型检索 K 个最近的高斯分布，聚合距离加权嵌入，并通过神经网络预测 RGB 值。这种设计实现了连续的图像表示、可解释的几何结构和灵活的本地编辑，为物理感知和交互式图像操作提供了基础。我们方法的官方实现可通过此 https URL 公开获得。

Title: BUSTR: Breast Ultrasound Text Reporting with a Descriptor-Aware Vision-Language Model

• Authors: Rawa Mohammed, Mina Attin, Bryar Shareef
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.20956
• Pdf URL: https://arxiv.org/pdf/2511.20956
• Copy Paste: [[2511.20956]] BUSTR: Breast Ultrasound Text Reporting with a Descriptor-Aware Vision-Language Model(https://arxiv.org/abs/2511.20956)
• Keywords: generation
• Abstract: Automated radiology report generation (RRG) for breast ultrasound (BUS) is limited by the lack of paired image-report datasets and the risk of hallucinations from large language models. We propose BUSTR, a multitask vision-language framework that generates BUS reports without requiring paired image-report supervision. BUSTR constructs reports from structured descriptors (e.g., BI-RADS, pathology, histology) and radiomics features, learns descriptor-aware visual representations with a multi-head Swin encoder trained using a multitask loss over dataset-specific descriptor sets, and aligns visual and textual tokens via a dual-level objective that combines token-level cross-entropy with a cosine-similarity alignment loss between input and output representations. We evaluate BUSTR on two public BUS datasets, BrEaST and BUS-BRA, which differ in size and available descriptors. Across both datasets, BUSTR consistently improves standard natural language generation metrics and clinical efficacy metrics, particularly for key targets such as BI-RADS category and pathology. Our results show that this descriptor-aware vision model, trained with a combined token-level and alignment loss, improves both automatic report metrics and clinical efficacy without requiring paired image-report data. The source code can be found at this https URL
• 摘要：乳腺超声 (BUS) 的自动放射学报告生成 (RRG) 由于缺乏配对图像报告数据集以及大型语言模型产生幻觉的风险而受到限制。我们提出了 BUSTR，这是一种多任务视觉语言框架，可以生成 BUS 报告，而不需要配对图像报告监督。 BUSTR 根据结构化描述符（例如 BI-RADS、病理学、组织学）和放射组学特征构建报告，通过使用数据集特定描述符集的多任务损失训练的多头 Swin 编码器学习描述符感知的视觉表示，并通过将令牌级交叉熵与输入和输出表示之间的余弦相似性对齐损失相结合的双层目标来对齐视觉和文本令牌。我们在两个公共 BUS 数据集 BrEaST 和 BUS-BRA 上评估 BUSTR，这两个数据集的大小和可用描述符不同。在这两个数据集中，BUSTR 持续改进标准自然语言生成指标和临床疗效指标，特别是针对 BI-RADS 类别和病理学等关键目标。我们的结果表明，这种描述符感知视觉模型经过组合标记级别和对齐损失的训练，可以提高自动报告指标和临床疗效，而无需配对图像报告数据。源代码可以在此 https URL 找到

Title: TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs

• Authors: Md Adnan Arefeen, Biplob Debnath, Srimat Chakradhar
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2511.20965
• Pdf URL: https://arxiv.org/pdf/2511.20965
• Copy Paste: [[2511.20965]] TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs(https://arxiv.org/abs/2511.20965)
• Keywords: generation
• Abstract: Traffic cameras are essential in urban areas, playing a crucial role in intelligent transportation systems. Multiple cameras at intersections enhance law enforcement capabilities, traffic management, and pedestrian safety. However, efficiently managing and analyzing multi-camera feeds poses challenges due to the vast amount of data. Analyzing such huge video data requires advanced analytical tools. While Large Language Models (LLMs) like ChatGPT, equipped with retrieval-augmented generation (RAG) systems, excel in text-based tasks, integrating them into traffic video analysis demands converting video data into text using a Vision-Language Model (VLM), which is time-consuming and delays the timely utilization of traffic videos for generating insights and investigating incidents. To address these challenges, we propose TrafficLens, a tailored algorithm for multi-camera traffic intersections. TrafficLens employs a sequential approach, utilizing overlapping coverage areas of cameras. It iteratively applies VLMs with varying token limits, using previous outputs as prompts for subsequent cameras, enabling rapid generation of detailed textual descriptions while reducing processing time. Additionally, TrafficLens intelligently bypasses redundant VLM invocations through an object-level similarity detector. Experimental results with real-world datasets demonstrate that TrafficLens reduces video-to-text conversion time by up to $4\times$ while maintaining information accuracy.
• 摘要：交通摄像头在城市地区必不可少，在智能交通系统中发挥着至关重要的作用。十字路口的多个摄像头增强了执法能力、交通管理和行人安全。然而，由于数据量巨大，有效管理和分析多摄像机源带来了挑战。分析如此庞大的视频数据需要先进的分析工具。虽然像 ChatGPT 这样的大型语言模型 (LLM) 配备了检索增强生成 (RAG) 系统，在基于文本的任务中表现出色，但将它们集成到交通视频分析中需要使用视觉语言模型 (VLM) 将视频数据转换为文本，这非常耗时，并且会延迟及时利用交通视频来生成见解和调查事件。为了应对这些挑战，我们提出了 TrafficLens，这是一种针对多摄像头交通路口量身定制的算法。 TrafficLens 采用顺序方法，利用摄像机的重叠覆盖区域。它迭代地应用具有不同令牌限制的 VLM，使用先前的输出作为后续摄像机的提示，从而能够快速生成详细的文本描述，同时减少处理时间。此外，TrafficLens 通过对象级相似性检测器智能地绕过冗余的 VLM 调用。真实数据集的实验结果表明，TrafficLens 在保持信息准确性的同时，可将视频到文本的转换时间减少高达 4\times$。

Title: Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learning

• Authors: Shanwei Fan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.20993
• Pdf URL: https://arxiv.org/pdf/2511.20993
• Copy Paste: [[2511.20993]] Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learning(https://arxiv.org/abs/2511.20993)
• Keywords: generation
• Abstract: Large language models (LLMs) offer strong high-level planning capabilities for reinforcement learning (RL) by decomposing tasks into subgoals. However, their practical utility is limited by poor planning-execution alignment, which reflects a critical gap between abstract plans and actionable, environment-compatible behaviors. This misalignment arises from two interrelated limitations: (1) LLMs often produce subgoals that are semantically plausible but infeasible or irrelevant in the target environment due to insufficient grounding in environment-specific knowledge, and (2) single-LLM planning conflates generation with self-verification, resulting in overconfident yet unreliable subgoals that frequently fail during execution. To address these challenges, we propose Subgoal Graph-Augmented Actor-Critic-Refiner (SGA-ACR), a framework that integrates an environment-specific subgoal graph and structured entity knowledge with a multi-LLM planning pipeline that explicitly separates generation, critique, and refinement to produce executable and verifiable subgoals. A subgoal tracker further monitors execution progress, provides auxiliary rewards, and adaptively updates the subgoal graph to maintain alignment between plans and actions. Experimental results on 22 diverse tasks in the open-world game "Crafter" demonstrate the effectiveness of our proposed method.
• 摘要：大型语言模型 (LLM) 通过将任务分解为子目标，为强化学习 (RL) 提供强大的高级规划能力。然而，它们的实际效用受到规划与执行不一致的限制，这反映了抽象计划与可操作、环境兼容的行为之间存在重大差距。这种不一致源于两个相互关联的限制：（1）法学硕士通常会产生语义上合理但在目标环境中不可行或不相关的子目标，因为特定环境知识的基础不足；（2）单一法学硕士规划将生成与自我验证混为一谈，导致过于自信但不可靠的子目标在执行过程中经常失败。为了应对这些挑战，我们提出了子目标图增强的 Actor-Critic-Refiner (SGA-ACR)，这是一个框架，它将特定于环境的子目标图和结构化实体知识与多 LLM 规划管道集成在一起，该管道明确地将生成、批判和细化分开，以生成可执行和可验证的子目标。子目标跟踪器进一步监控执行进度，提供辅助奖励，并自适应更新子目标图以保持计划和行动之间的一致性。开放世界游戏“Crafter”中 22 个不同任务的实验结果证明了我们提出的方法的有效性。

Title: From Inpainting to Layer Decomposition: Repurposing Generative Inpainting Models for Image Layer Decomposition

• Authors: Jingxi Chen, Yixiao Zhang, Xiaoye Qian, Zongxia Li, Cornelia Fermuller, Caren Chen, Yiannis Aloimonos
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.20996
• Pdf URL: https://arxiv.org/pdf/2511.20996
• Copy Paste: [[2511.20996]] From Inpainting to Layer Decomposition: Repurposing Generative Inpainting Models for Image Layer Decomposition(https://arxiv.org/abs/2511.20996)
• Keywords: generative
• Abstract: Images can be viewed as layered compositions, foreground objects over background, with potential occlusions. This layered representation enables independent editing of elements, offering greater flexibility for content creation. Despite the progress in large generative models, decomposing a single image into layers remains challenging due to limited methods and data. We observe a strong connection between layer decomposition and in/outpainting tasks, and propose adapting a diffusion-based inpainting model for layer decomposition using lightweight finetuning. To further preserve detail in the latent space, we introduce a novel multi-modal context fusion module with linear attention complexity. Our model is trained purely on a synthetic dataset constructed from open-source assets and achieves superior performance in object removal and occlusion recovery, unlocking new possibilities in downstream editing and creative applications.
• 摘要：图像可以被视为分层构图，前景物体位于背景之上，并且具有潜在的遮挡。这种分层表示可以实现元素的独立编辑，为内容创建提供更大的灵活性。尽管大型生成模型取得了进展，但由于方法和数据有限，将单个图像分解为层仍然具有挑战性。我们观察到层分解和内/外绘制任务之间的紧密联系，并提出使用轻量级微调采用基于扩散的修复模型来进行层分解。为了进一步保留潜在空间中的细节，我们引入了一种具有线性注意复杂度的新颖的多模态上下文融合模块。我们的模型纯粹是在由开源资产构建的合成数据集上进行训练，并在对象去除和遮挡恢复方面实现了卓越的性能，从而为下游编辑和创意应用程序释放了新的可能性。

Title: Knowledge Completes the Vision: A Multimodal Entity-aware Retrieval-Augmented Generation Framework for News Image Captioning

• Authors: Xiaoxing You, Qiang Huang, Lingyu Li, Chi Zhang, Xiaopeng Liu, Min Zhang, Jun Yu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21002
• Pdf URL: https://arxiv.org/pdf/2511.21002
• Copy Paste: [[2511.21002]] Knowledge Completes the Vision: A Multimodal Entity-aware Retrieval-Augmented Generation Framework for News Image Captioning(https://arxiv.org/abs/2511.21002)
• Keywords: generation
• Abstract: News image captioning aims to produce journalistically informative descriptions by combining visual content with contextual cues from associated articles. Despite recent advances, existing methods struggle with three key challenges: (1) incomplete information coverage, (2) weak cross-modal alignment, and (3) suboptimal visual-entity grounding. To address these issues, we introduce MERGE, the first Multimodal Entity-aware Retrieval-augmented GEneration framework for news image captioning. MERGE constructs an entity-centric multimodal knowledge base (EMKB) that integrates textual, visual, and structured knowledge, enabling enriched background retrieval. It improves cross-modal alignment through a multistage hypothesis-caption strategy and enhances visual-entity matching via dynamic retrieval guided by image content. Extensive experiments on GoodNews and NYTimes800k show that MERGE significantly outperforms state-of-the-art baselines, with CIDEr gains of +6.84 and +1.16 in caption quality, and F1-score improvements of +4.14 and +2.64 in named entity recognition. Notably, MERGE also generalizes well to the unseen Visual News dataset, achieving +20.17 in CIDEr and +6.22 in F1-score, demonstrating strong robustness and domain adaptability.
• 摘要：新闻图像字幕旨在通过将视觉内容与相关文章的上下文线索相结合来生成新闻信息描述。尽管最近取得了进展，但现有方法仍面临三个关键挑战：（1）信息覆盖不完整，（2）跨模式对齐较弱，以及（3）视觉实体基础不佳。为了解决这些问题，我们引入了 MERGE，这是第一个用于新闻图像字幕的多模式实体感知检索增强生成框架。 MERGE构建了一个以实体为中心的多模态知识库（EMKB），集成了文本、视觉和结构化知识，从而实现了丰富的背景检索。它通过多阶段假设标题策略改进跨模式对齐，并通过图像内容引导的动态检索增强视觉实体匹配。 GoodNews 和 NYTimes800k 上的大量实验表明，MERGE 的性能显着优于最先进的基线，CIDEr 在标题质量方面提高了 +6.84 和 +1.16，在命名实体识别方面 F1 分数提高了 +4.14 和 +2.64。值得注意的是，MERGE 还可以很好地泛化到未见过的 Visual News 数据集，在 CIDEr 中达到 +20.17，在 F1 分数中达到 +6.22，表现出强大的鲁棒性和领域适应性。

Title: ChatGpt Content detection: A new approach using xlm-roberta alignment

• Authors: Md Tasnin Tanvir, Dr Santanu Kumar Dash, Ishan Shahnan, Nafis Fuad, Tanvir Rahman, Abdullah Al Faisal, Asadullah Al Mamun
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21009
• Pdf URL: https://arxiv.org/pdf/2511.21009
• Copy Paste: [[2511.21009]] ChatGpt Content detection: A new approach using xlm-roberta alignment(https://arxiv.org/abs/2511.21009)
• Keywords: generative
• Abstract: The challenge of separating AI-generated text from human-authored content is becoming more urgent as generative AI technologies like ChatGPT become more widely available. In this work, we address this issue by looking at both the detection of content that has been entirely generated by AI and the identification of human text that has been reworded by AI. In our work, a comprehensive methodology to detect AI- generated text using XLM-RoBERTa, a state-of-the-art multilingual transformer model. Our approach includes rigorous preprocessing, and feature extraction involving perplexity, semantic, and readability features. We fine-tuned the XLM-RoBERTa model on a balanced dataset of human and AI-generated texts and evaluated its performance. The model demonstrated high accuracy and robust performance across various text genres. Additionally, we conducted feature analysis to understand the model's decision-making process, revealing that perplexity and attention-based features are critical in differentiating between human and AI-generated texts. Our findings offer a valuable tool for maintaining academic integrity and contribute to the broader field of AI ethics by promoting transparency and accountability in AI systems. Future research directions include exploring other advanced models and expanding the dataset to enhance the model's generalizability.
• 摘要：随着 ChatGPT 等生成式人工智能技术变得越来越广泛，将人工智能生成的文本与人类创作的内容分开的挑战变得更加紧迫。在这项工作中，我们通过研究完全由人工智能生成的内容的检测和人工智能改写的人类文本的识别来解决这个问题。在我们的工作中，使用最先进的多语言转换器模型 XLM-RoBERTa 来检测人工智能生成的文本的综合方法。我们的方法包括严格的预处理和涉及复杂性、语义和可读性特征的特征提取。我们在人类和人工智能生成文本的平衡数据集上对 XLM-RoBERTa 模型进行了微调，并评估了其性能。该模型在各种文本类型中都表现出了高精度和稳健的性能。此外，我们还进行了特征分析，以了解模型的决策过程，揭示了困惑度和基于注意力的特征对于区分人类和人工智能生成的文本至关重要。我们的研究结果为维护学术诚信提供了宝贵的工具，并通过促进人工智能系统的透明度和问责制为更广泛的人工智能道德领域做出贡献。未来的研究方向包括探索其他先进模型和扩展数据集以增强模型的通用性。

Title: Probabilistic Wildfire Spread Prediction Using an Autoregressive Conditional Generative Adversarial Network

• Authors: Taehoon Kang, Taeyong Kim
• Subjects: cs.LG, cs.AI, cs.CE, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21019
• Pdf URL: https://arxiv.org/pdf/2511.21019
• Copy Paste: [[2511.21019]] Probabilistic Wildfire Spread Prediction Using an Autoregressive Conditional Generative Adversarial Network(https://arxiv.org/abs/2511.21019)
• Keywords: generative
• Abstract: Climate change has intensified the frequency and severity of wildfires, making rapid and accurate prediction of fire spread essential for effective mitigation and response. Physics-based simulators such as FARSITE offer high-fidelity predictions but are computationally intensive, limiting their applicability in real-time decision-making, while existing deep learning models often yield overly smooth predictions that fail to capture the complex, nonlinear dynamics of wildfire propagation. This study proposes an autoregressive conditional generative adversarial network (CGAN) for probabilistic wildfire spread prediction. By formulating the prediction task as an autoregressive problem, the model learns sequential state transitions, ensuring long-term prediction stability. Experimental results demonstrate that the proposed CGAN-based model outperforms conventional deep learning models in both overall predictive accuracy and boundary delineation of fire perimeters. These results demonstrate that adversarial learning allows the model to capture the strong nonlinearity and uncertainty of wildfire spread, instead of simply fitting the pixel average. Furthermore, the autoregressive framework facilitates systematic temporal forecasting of wildfire evolution. The proposed CGAN-based autoregressive framework enhances both the accuracy and physical interpretability of wildfire spread prediction, offering a promising foundation for time-sensitive response and evacuation planning.
• 摘要：气候变化加剧了野火的发生频率和严重程度，因此快速准确地预测火势蔓延对于有效缓解和应对至关重要。 FARSITE 等基于物理的模拟器提供高保真度预测，但计算量大，限制了它们在实时决策中的适用性，而现有的深度学习模型通常会产生过于平滑的预测，无法捕捉野火传播的复杂、非线性动态。本研究提出了一种用于概率野火蔓延预测的自回归条件生成对抗网络（CGAN）。通过将预测任务表述为自回归问题，模型学习顺序状态转换，确保长期预测稳定性。实验结果表明，所提出的基于 CGAN 的模型在整体预测精度和火灾周界的边界描绘方面均优于传统的深度学习模型。这些结果表明，对抗性学习使模型能够捕获野火蔓延的强非线性和不确定性，而不是简单地拟合像素平均值。此外，自回归框架有助于对野火演变进行系统的时间预测。所提出的基于 CGAN 的自回归框架提高了野火蔓延预测的准确性和物理可解释性，为时间敏感的响应和疏散规划提供了有希望的基础。

Title: CameraMaster: Unified Camera Semantic-Parameter Control for Photography Retouching

• Authors: Qirui Yang, Yang Yang, Ying Zeng, Xiaobin Hu, Bo Li, Huanjing Yue, Jingyu Yang, Peng-Tao Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21024
• Pdf URL: https://arxiv.org/pdf/2511.21024
• Copy Paste: [[2511.21024]] CameraMaster: Unified Camera Semantic-Parameter Control for Photography Retouching(https://arxiv.org/abs/2511.21024)
• Keywords: generation
• Abstract: Text-guided diffusion models have greatly advanced image editing and generation. However, achieving physically consistent image retouching with precise parameter control (e.g., exposure, white balance, zoom) remains challenging. Existing methods either rely solely on ambiguous and entangled text prompts, which hinders precise camera control, or train separate heads/weights for parameter adjustment, which compromises scalability, multi-parameter composition, and sensitivity to subtle variations. To address these limitations, we propose CameraMaster, a unified camera-aware framework for image retouching. The key idea is to explicitly decouple the camera directive and then coherently integrate two critical information streams: a directive representation that captures the photographer's intent, and a parameter embedding that encodes precise camera settings. CameraMaster first uses the camera parameter embedding to modulate both the camera directive and the content semantics. The modulated directive is then injected into the content features via cross-attention, yielding a strongly camera-sensitive semantic context. In addition, the directive and camera embeddings are injected as conditioning and gating signals into the time embedding, enabling unified, layer-wise modulation throughout the denoising process and enforcing tight semantic-parameter alignment. To train and evaluate CameraMaster, we construct a large-scale dataset of 78K image-prompt pairs annotated with camera parameters. Extensive experiments show that CameraMaster produces monotonic and near-linear responses to parameter variations, supports seamless multi-parameter composition, and significantly outperforms existing methods.
• 摘要：文本引导的扩散模型极大地改进了图像编辑和生成。然而，通过精确的参数控制（例如曝光、白平衡、变焦）实现物理一致的图像修饰仍然具有挑战性。现有的方法要么仅仅依赖于模糊且纠缠的文本提示，这阻碍了精确的相机控制，要么训练单独的头/权重进行参数调整，这会损害可扩展性、多参数组合和对细微变化的敏感性。为了解决这些限制，我们提出了 CameraMaster，这是一个用于图像修饰的统一相机感知框架。关键思想是显式解耦相机指令，然后连贯地集成两个关键信息流：捕获摄影师意图的指令表示，以及编码精确相机设置的参数嵌入。 CameraMaster 首先使用相机参数嵌入来调制相机指令和内容语义。然后，通过交叉注意力将调制后的指令注入到内容特征中，从而产生强烈的相机敏感语义上下文。此外，指令嵌入和相机嵌入作为条件和门控信号注入到时间嵌入中，从而在整个去噪过程中实现统一的分层调制，并强制执行严格的语义参数对齐。为了训练和评估 CameraMaster，我们构建了一个由 78K 个图像提示对组成的大规模数据集，并标注了相机参数。大量实验表明，CameraMaster 对参数变化产生单调且近线性的响应，支持无缝多参数合成，并且显着优于现有方法。

Title: FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation

• Authors: Kaixing Yang, Xulong Tang, Ziqiao Peng, Xiangyue Zhang, Puwei Wang, Jun He, Hongyan Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21029
• Pdf URL: https://arxiv.org/pdf/2511.21029
• Copy Paste: [[2511.21029]] FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation(https://arxiv.org/abs/2511.21029)
• Keywords: generation
• Abstract: Music-to-dance generation aims to translate auditory signals into expressive human motion, with broad applications in virtual reality, choreography, and digital entertainment. Despite promising progress, the limited generation efficiency of existing methods leaves insufficient computational headroom for high-fidelity 3D rendering, thereby constraining the expressiveness of 3D characters during real-world applications. Thus, we propose FlowerDance, which not only generates refined motion with physical plausibility and artistic expressiveness, but also achieves significant generation efficiency on inference speed and memory utilization . Specifically, FlowerDance combines MeanFlow with Physical Consistency Constraints, which enables high-quality motion generation with only a few sampling steps. Moreover, FlowerDance leverages a simple but efficient model architecture with BiMamba-based backbone and Channel-Level Cross-Modal Fusion, which generates dance with efficient non-autoregressive manner. Meanwhile, FlowerDance supports motion editing, enabling users to interactively refine dance sequences. Extensive experiments on AIST++ and FineDance show that FlowerDance achieves state-of-the-art results in both motion quality and generation efficiency. Code will be released upon acceptance.
• 摘要：音乐到舞蹈的生成旨在将听觉信号转化为富有表现力的人体动作，在虚拟现实、编舞和数字娱乐领域有着广泛的应用。尽管取得了可喜的进展，但现有方法的生成效率有限，导致高保真 3D 渲染的计算空间不足，从而限制了现实应用中 3D 角色的表现力。因此，我们提出了FlowerDance，它不仅生成具有物理合理性和艺术表现力的精细运动，而且在推理速度和内存利用率上实现了显着的生成效率。具体来说，FlowerDance 将 MeanFlow 与物理一致性约束相结合，只需几个采样步骤即可生成高质量的运动。此外，FlowerDance 利用简单但高效的模型架构，以及基于 BiMamba 的主干和通道级跨模态融合，以高效的非自回归方式生成舞蹈。同时，FlowerDance 支持动作编辑，使用户能够交互式地完善舞蹈序列。在 AIST++ 和 FineDance 上进行的大量实验表明，FlowerDance 在运动质量和生成效率方面都取得了最先进的结果。代码将在接受后发布。

Title: PG-ControlNet: A Physics-Guided ControlNet for Generative Spatially Varying Image Deblurring

• Authors: Hakki Motorcu, Mujdat Cetin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21043
• Pdf URL: https://arxiv.org/pdf/2511.21043
• Copy Paste: [[2511.21043]] PG-ControlNet: A Physics-Guided ControlNet for Generative Spatially Varying Image Deblurring(https://arxiv.org/abs/2511.21043)
• Keywords: generative
• Abstract: Spatially varying image deblurring remains a fundamentally ill-posed problem, especially when degradations arise from complex mixtures of motion and other forms of blur under significant noise. State-of-the-art learning-based approaches generally fall into two paradigms: model-based deep unrolling methods that enforce physical constraints by modeling the degradations, but often produce over-smoothed, artifact-laden textures, and generative models that achieve superior perceptual quality yet hallucinate details due to weak physical constraints. In this paper, we propose a novel framework that uniquely reconciles these paradigms by taming a powerful generative prior with explicit, dense physical constraints. Rather than oversimplifying the degradation field, we model it as a dense continuum of high-dimensional compressed kernels, ensuring that minute variations in motion and other degradation patterns are captured. We leverage this rich descriptor field to condition a ControlNet architecture, strongly guiding the diffusion sampling process. Extensive experiments demonstrate that our method effectively bridges the gap between physical accuracy and perceptual realism, outperforming state-of-the-art model-based methods as well as generative baselines in challenging, severely blurred scenarios.
• 摘要：空间变化的图像去模糊仍然是一个根本上不适定的问题，特别是当运动和其他形式的模糊在显着噪声下的复杂混合引起退化时。最先进的基于学习的方法通常分为两种范式：基于模型的深度展开方法，通过对退化进行建模来强制物理约束，但通常会产生过度平滑、充满伪影的纹理；以及生成模型，该模型实现了卓越的感知质量，但由于物理约束较弱而产生了幻觉细节。在本文中，我们提出了一种新颖的框架，通过使用明确、密集的物理约束来驯服强大的生成先验，以独特的方式协调这些范式。我们没有过度简化退化场，而是将其建模为高维压缩内核的密集连续体，确保捕获运动和其他退化模式的微小变化。我们利用这个丰富的描述符字段来调节 ControlNet 架构，有力地指导扩散采样过程。大量的实验表明，我们的方法有效地弥合了物理准确性和感知真实性之间的差距，优于最先进的基于模型的方法以及在具有挑战性、严重模糊的场景中的生成基线。

Title: MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization

• Authors: Yingjie Xia, Xi Wang, Jinglei Shi, Vicky Kalogeiton, Jian Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21051
• Pdf URL: https://arxiv.org/pdf/2511.21051
• Copy Paste: [[2511.21051]] MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization(https://arxiv.org/abs/2511.21051)
• Keywords: generation
• Abstract: Images evoke emotions that profoundly influence perception, often prioritized over content. Current Image Emotional Synthesis (IES) approaches artificially separate generation and editing tasks, creating inefficiencies and limiting applications where these tasks naturally intertwine, such as therapeutic interventions or storytelling. In this work, we introduce MUSE, the first unified framework capable of both emotional generation and editing. By adopting a strategy conceptually aligned with Test-Time Scaling (TTS) that widely used in both LLM and diffusion model communities, it avoids the requirement for additional updating diffusion model and specialized emotional synthesis datasets. More specifically, MUSE addresses three key questions in emotional synthesis: (1) HOW to stably guide synthesis by leveraging an off-the-shelf emotion classifier with gradient-based optimization of emotional tokens; (2) WHEN to introduce emotional guidance by identifying the optimal timing using semantic similarity as a supervisory signal; and (3) WHICH emotion to guide synthesis through a multi-emotion loss that reduces interference from inherent and similar emotions. Experimental results show that MUSE performs favorably against all methods for both generation and editing, improving emotional accuracy and semantic diversity while maintaining an optimal balance between desired content, adherence to text prompts, and realistic emotional expression. It establishes a new paradigm for emotion synthesis.
• 摘要：图像唤起深刻影响感知的情感，通常优先于内容。当前的图像情感合成（IES）方法人为地分离生成和编辑任务，导致效率低下并限制了这些任务自然交织在一起的应用，例如治疗干预或讲故事。在这项工作中，我们引入了 MUSE，这是第一个能够同时进行情感生成和编辑的统一框架。通过采用概念上与 LLM 和扩散模型社区中广泛使用的测试时间缩放 (TTS) 一致的策略，它避免了额外更新扩散模型和专门的情感合成数据集的要求。更具体地说，MUSE 解决了情感合成中的三个关键问题：（1）如何利用现成的情感分类器和基于梯度的情感标记优化来稳定地指导合成； （2）何时通过使用语义相似性作为监督信号来确定最佳时机来引入情感引导； (3)哪种情绪通过多重情绪损失来指导合成，从而减少固有和相似情绪的干扰。实验结果表明，MUSE 在生成和编辑方面均优于所有方法，提高了情感准确性和语义多样性，同时在所需内容、遵守文本提示和真实情感表达之间保持了最佳平衡。它建立了情感合成的新范式。

Title: A Unified Understanding of Offline Data Selection and Online Self-refining Generation for Post-training LLMs

• Authors: Quan Xiao, Tianyi Chen
• Subjects: cs.LG, cs.CL, math.OC
• Abstract URL: https://arxiv.org/abs/2511.21056
• Pdf URL: https://arxiv.org/pdf/2511.21056
• Copy Paste: [[2511.21056]] A Unified Understanding of Offline Data Selection and Online Self-refining Generation for Post-training LLMs(https://arxiv.org/abs/2511.21056)
• Keywords: generation
• Abstract: Offline data selection and online self-refining generation, which enhance the data quality, are crucial steps in adapting large language models (LLMs) to specific downstream tasks. We tackle offline data selection and online self-refining generations through an optimization perspective. Specifically, bilevel data selection is used for offline data selection with respect to the validation dataset, and we treat online self-refining generation as a model adaptation step of selecting the model trained on current responses that best fits the validation data. Our framework offers a unified understanding of offline data selection and self-refining generation by assigning a learned data weight to each question and response, either explicitly or implicitly. For the first time, we theoretically demonstrate the effectiveness of the bilevel data selection framework and demonstrate its performance gains over unfiltered direct mixing baselines. By combining offline data with validation-weighted online generations, our method enhances fine-tuning performance. Experiments on quality enhancement and safety-aware LLM fine-tuning validate its effectiveness.
• 摘要：离线数据选择和在线自提炼生成可以提高数据质量，是使大型语言模型（LLM）适应特定下游任务的关键步骤。我们通过优化的角度来解决离线数据选择和在线自精炼生成问题。具体来说，双层数据选择用于针对验证数据集的离线数据选择，并且我们将在线自精炼生成视为模型适应步骤，选择最适合验证数据的当前响应训练的模型。我们的框架通过显式或隐式地为每个问题和响应分配学习数据权重，提供对离线数据选择和自我完善生成的统一理解。我们第一次从理论上证明了双层数据选择框架的有效性，并证明了其相对于未过滤的直接混合基线的性能增益。通过将离线数据与验证加权的在线生成相结合，我们的方法增强了微调性能。质量提升和安全意识LLM微调的实验验证了其有效性。

Title: Long-Term Alzheimers Disease Prediction: A Novel Image Generation Method Using Temporal Parameter Estimation with Normal Inverse Gamma Distribution on Uneven Time Series

• Authors: Xin Hong, Xinze Sun, Yinhao Li, Yen-Wei Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21057
• Pdf URL: https://arxiv.org/pdf/2511.21057
• Copy Paste: [[2511.21057]] Long-Term Alzheimers Disease Prediction: A Novel Image Generation Method Using Temporal Parameter Estimation with Normal Inverse Gamma Distribution on Uneven Time Series(https://arxiv.org/abs/2511.21057)
• Keywords: generation
• Abstract: Image generation can provide physicians with an imaging diagnosis basis in the prediction of Alzheimer's Disease (AD). Recent research has shown that long-term AD predictions by image generation often face difficulties maintaining disease-related characteristics when dealing with irregular time intervals in sequential data. Considering that the time-related aspects of the distribution can reflect changes in disease-related characteristics when images are distributed unevenly, this research proposes a model to estimate the temporal parameter within the Normal Inverse Gamma Distribution (T-NIG) to assist in generating images over the long term. The T-NIG model employs brain images from two different time points to create intermediate brain images, forecast future images, and predict the disease. T-NIG is designed by identifying features using coordinate neighborhoods. It incorporates a time parameter into the normal inverse gamma distribution to understand how features change in brain imaging sequences that have varying time intervals. Additionally, T-NIG utilizes uncertainty estimation to reduce both epistemic and aleatoric uncertainties in the model, which arise from insufficient temporal data. In particular, the T-NIG model demonstrates state-of-the-art performance in both short-term and long-term prediction tasks within the dataset. Experimental results indicate that T-NIG is proficient in forecasting disease progression while maintaining disease-related characteristics, even when faced with an irregular temporal data distribution.
• 摘要：图像生成可以为医生预测阿尔茨海默病（AD）提供影像诊断依据。最近的研究表明，在处理连续数据中的不规则时间间隔时，通过图像生成进行的长期 AD 预测通常面临维持疾病相关特征的困难。考虑到当图像分布不均匀时，分布的时间相关方面可以反映疾病相关特征的变化，本研究提出了一种模型来估计正态逆伽玛分布（T-NIG）内的时间参数，以协助生成长期图像。 T-NIG 模型利用两个不同时间点的大脑图像来创建中间大脑图像、预测未来图像并预测疾病。 T-NIG 是通过使用坐标邻域识别特征来设计的。它将时间参数纳入正态逆伽马分布中，以了解具有不同时间间隔的大脑成像序列中的特征如何变化。此外，T-NIG 利用不确定性估计来减少模型中因时间数据不足而产生的认知和任意不确定性。特别是，T-NIG 模型在数据集中的短期和长期预测任务中展示了最先进的性能。实验结果表明，即使面对不规则的时间数据分布，T-NIG 也能熟练预测疾病进展，同时保持疾病相关特征。

Title: Generative Early Stage Ranking

• Authors: Juhee Hong, Meng Liu, Shengzhi Wang, Xiaoheng Mao, Huihui Cheng, Leon Gao, Christopher Leung, Jin Zhou, Chandra Mouli Sekar, Zhao Zhu, Ruochen Liu, Tuan Trieu, Dawei Sun, Jeet Kanjani, Rui Li, Jing Qian, Xuan Cao, Minjie Fan, Mingze Gao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21095
• Pdf URL: https://arxiv.org/pdf/2511.21095
• Copy Paste: [[2511.21095]] Generative Early Stage Ranking(https://arxiv.org/abs/2511.21095)
• Keywords: generative
• Abstract: Large-scale recommendations commonly adopt a multi-stage cascading ranking system paradigm to balance effectiveness and efficiency. Early Stage Ranking (ESR) systems utilize the "user-item decoupling" approach, where independently learned user and item representations are only combined at the final layer. While efficient, this design is limited in effectiveness, as it struggles to capture fine-grained user-item affinities and cross-signals. To address these, we propose the Generative Early Stage Ranking (GESR) paradigm, introducing the Mixture of Attention (MoA) module which leverages diverse attention mechanisms to bridge the effectiveness gap: the Hard Matching Attention (HMA) module encodes explicit cross-signals by computing raw match counts between user and item features; the Target-Aware Self Attention module generates target-aware user representations conditioned on the item, enabling more personalized learning; and the Cross Attention modules facilitate early and more enriched interactions between user-item features. MoA's specialized attention encodings are further refined in the final layer through a Multi-Logit Parameterized Gating (MLPG) module, which integrates the newly learned embeddings via gating and produces secondary logits that are fused with the primary logit. To address the efficiency and latency challenges, we have introduced a comprehensive suite of optimization techniques. These span from custom kernels that maximize the capabilities of the latest hardware to efficient serving solutions powered by caching mechanisms. The proposed GESR paradigm has shown substantial improvements in topline metrics, engagement, and consumption tasks, as validated by both offline and online experiments. To the best of our knowledge, this marks the first successful deployment of full target-aware attention sequence modeling within an ESR stage at such a scale.
• 摘要：大规模推荐通常采用多阶段级联排名系统范式来平衡有效性和效率。早期排名（ESR）系统利用“用户-项目解耦”方法，其中独立学习的用户和项目表示仅在最后一层组合。虽然有效，但这种设计的有效性有限，因为它很难捕获细粒度的用户-项目亲和力和交叉信号。为了解决这些问题，我们提出了生成早期排名（GESR）范例，引入了混合注意力（MoA）模块，该模块利用不同的注意力机制来弥合有效性差距：硬匹配注意力（HMA）模块通过计算用户和项目特征之间的原始匹配计数来编码显式交叉信号；目标感知自注意力模块生成以项目为条件的目标感知用户表示，从而实现更加个性化的学习；交叉注意力模块促进用户-项目特征之间的早期和更丰富的交互。 MoA 的专门注意力编码在最后一层通过多 Logit 参数化门控 (MLPG) 模块进一步细化，该模块通过门控集成新学习的嵌入，并生成与主 Logit 融合的辅助 Logit。为了解决效率和延迟挑战，我们引入了一套全面的优化技术。这些范围从最大化最新硬件功能的定制内核到由缓存机制支持的高效服务解决方案。经离线和在线实验验证，所提出的 GESR 范式在顶线指标、参与度和消费任务方面显示出显着改进。据我们所知，这标志着首次在 ESR 阶段成功部署如此规模的完整目标感知注意力序列建模。

Title: From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models

• Authors: Hengyu Fu, Baihe Huang, Virginia Adams, Charles Wang, Venkat Srinivasan, Jiantao Jiao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21103
• Pdf URL: https://arxiv.org/pdf/2511.21103
• Copy Paste: [[2511.21103]] From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models(https://arxiv.org/abs/2511.21103)
• Keywords: generation
• Abstract: Diffusion Language Models (DLMs) have recently emerged as a strong alternative to autoregressive language models (LMs). DLMs offer comparable accuracy with faster inference speed via parallel decoding. However, standard DLM decoding strategies relying on high-confidence tokens encounter an inherent information-theoretic bottleneck that restricts decoding progress and ultimately slows generation. We demonstrate both theoretically and empirically that prioritizing high-confidence tokens is inherently inefficient. High-probability tokens carry negligible information and strictly relying on them limits the effective progress made in each decoding round. We prove that the number of decoding rounds must grow linearly with the sample's total information (negative log-likelihood) and inversely with the per-round information budget, establishing a bits-to-rounds principle. We also propose Explore-Then-Exploit (ETE), a training-free decoding strategy that maximizes information throughput and decoding efficiency. ETE combines cross-block decoding with targeted exploration of high-uncertainty tokens to reshape the conditional distribution and trigger cascades of confident predictions. Experiments verify our theoretical bounds and demonstrate that ETE consistently reduces the required number of decoding rounds compared to confidence-only baselines without compromising generation quality.
• 摘要：扩散语言模型 (DLM) 最近已成为自回归语言模型 (LM) 的强大替代方案。 DLM 通过并行解码提供相当的精度和更快的推理速度。然而，依赖于高置信度令牌的标准 DLM 解码策略遇到了固有的信息论瓶颈，限制了解码进度并最终减慢了生成速度。我们从理论上和经验上证明，优先考虑高置信度令牌本质上是低效的。高概率令牌携带的信息可以忽略不计，严格依赖它们会限制每轮解码的有效进展。我们证明解码轮数必须与样本的总信息（负对数似然）线性增长，并与每轮信息预算成反比，建立比特轮原则。我们还提出了 Explore-Then-Exploit (ETE)，这是一种无需训练的解码策略，可以最大限度地提高信息吞吐量和解码效率。 ETE 将跨块解码与高不确定性令牌的有针对性的探索相结合，以重塑条件分布并触发置信预测的级联。实验验证了我们的理论界限，并证明与仅置信基线相比，ETE 始终减少了所需的解码轮数，而不会影响生成质量。

Title: BRIDGE: Building Representations In Domain Guided Program Verification

• Authors: Robert Joseph George, Carson Eisenach, Udaya Ghai, Dominique Perrault-Joncas, Anima Anandkumar, Dean Foster
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21104
• Pdf URL: https://arxiv.org/pdf/2511.21104
• Copy Paste: [[2511.21104]] BRIDGE: Building Representations In Domain Guided Program Verification(https://arxiv.org/abs/2511.21104)
• Keywords: generation
• Abstract: Large language models (LLMs) have achieved impressive results in code generation, yet struggle with program verification, especially in interactive proof frameworks such as Lean4. A central challenge is scalability: verified synthesis requires not just code, but also precise specifications and correctness proofs, and existing approaches rarely span all three domains. We present BRIDGE, the first systematic study of structured prompting for scalable verified program generation. BRIDGE decomposes verification into three interconnected domains: Code (executable implementations), Specifications (formal intent statements), and Proofs (constructive correctness arguments). Our key idea is to elicit distinct reasoning behaviors functional, specification-driven, and proof-oriented as intermediate representations that preserve semantic structure and connect these domains. Through systematic ablations, we show that this approach substantially improves both accuracy and efficiency beyond standard error feedback methods. For example, functional reasoning improves correctness of code in formal languages (Lean4) by nearly 1.5x (pass@5) over direct baselines. In inference-time compute, functional reasoning is also 2x more efficient, achieving higher pass rates with fewer generations and lower total sampling budgets. Similarly, we find that specification-driven prompting boosts Python coding pass rates by up to 17.5%. These findings suggest that structured domain alignment is a promising direction for advancing verified synthesis. BRIDGE establishes a foundation for training via expert iteration or RLVR, enabling models to internalize these reasoning strategies across code, specifications, and proofs.
• 摘要：大型语言模型 (LLM) 在代码生成方面取得了令人印象深刻的成果，但在程序验证方面却遇到了困难，尤其是在 Lean4 等交互式证明框架中。一个核心挑战是可扩展性：经过验证的综合不仅需要代码，还需要精确的规范和正确性证明，而现有的方法很少跨越所有三个领域。我们推出了 BRIDGE，这是第一个针对可扩展的验证程序生成的结构化提示的系统研究。 BRIDGE 将验证分解为三个相互关联的域：代码（可执行的实现）、规范（正式意图陈述）和证明（构造性正确性论证）。我们的关键思想是引出功能性、规范驱动和面向证明的不同推理行为，作为保留语义结构并连接这些领域的中间表示。通过系统的消融，我们表明这种方法比标准错误反馈方法大大提高了准确性和效率。例如，与直接基线相比，函数推理将形式语言 (Lean4) 中代码的正确性提高了近 1.5 倍 (pass@5)。在推理时间计算中，函数推理的效率也提高了 2 倍，以更少的代数和更低的总采样预算实现了更高的通过率。同样，我们发现规范驱动的提示可将 Python 编码通过率提高高达 17.5%。这些发现表明，结构化域对齐是推进验证合成的一个有希望的方向。 BRIDGE 通过专家迭代或 RLVR 建立了训练基础，使模型能够跨代码、规范和证明内化这些推理策略。

Title: FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain

• Authors: YuAn Wang, Xiaofan Li, Chi Huang, Wenhao Zhang, Hao Li, Bosheng Wang, Xun Sun, Jun Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21113
• Pdf URL: https://arxiv.org/pdf/2511.21113
• Copy Paste: [[2511.21113]] FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain(https://arxiv.org/abs/2511.21113)
• Keywords: restoration, generation
• Abstract: In controllable driving-scene reconstruction and 3D scene generation, maintaining geometric fidelity while synthesizing visually plausible appearance under large viewpoint shifts is crucial. However, effective fusion of geometry-based 3DGS and appearance-driven diffusion models faces inherent challenges, as the absence of pixel-wise, 3D-consistent editing criteria often leads to over-restoration and geometric drift. To address these issues, we introduce \textbf{FaithFusion}, a 3DGS-diffusion fusion framework driven by pixel-wise Expected Information Gain (EIG). EIG acts as a unified policy for coherent spatio-temporal synthesis: it guides diffusion as a spatial prior to refine high-uncertainty regions, while its pixel-level weighting distills the edits back into 3DGS. The resulting plug-and-play system is free from extra prior conditions and structural this http URL experiments on the Waymo dataset demonstrate that our approach attains SOTA performance across NTA-IoU, NTL-IoU, and FID, maintaining an FID of 107.47 even at 6 meters lane shift. Our code is available at this https URL.
• 摘要：在可控驾驶场景重建和 3D 场景生成中，保持几何保真度，同时在大视点变化下合成视觉上合理的外观至关重要。然而，基于几何的 3DGS 和外观驱动的扩散模型的有效融合面临着固有的挑战，因为缺乏像素级、3D 一致的编辑标准通常会导致过度恢复和几何漂移。为了解决这些问题，我们引入了 \textbf{FaithFusion}，这是一种由逐像素预期信息增益 (EIG) 驱动的 3DGS 扩散融合框架。 EIG 充当连贯时空合成的统一策略：它引导扩散作为空间先验以细化高不确定性区域，而其像素级加权将编辑提炼回 3DGS。由此产生的即插即用系统不受额外先验条件的影响，并且在 Waymo 数据集上构造此 http URL 实验表明，我们的方法在 NTA-IoU、NTL-IoU 和 FID 上实现了 SOTA 性能，即使在 6 米车道偏移时也能保持 107.47 的 FID。我们的代码可以在这个 https URL 上找到。

Title: Deformation-aware Temporal Generation for Early Prediction of Alzheimers Disease

• Authors: Xin Honga, Jie Lin, Minghui Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21114
• Pdf URL: https://arxiv.org/pdf/2511.21114
• Copy Paste: [[2511.21114]] Deformation-aware Temporal Generation for Early Prediction of Alzheimers Disease(https://arxiv.org/abs/2511.21114)
• Keywords: generation, generative
• Abstract: Alzheimer's disease (AD), a degenerative brain condition, can benefit from early prediction to slow its progression. As the disease progresses, patients typically undergo brain atrophy. Current prediction methods for Alzheimers disease largely involve analyzing morphological changes in brain images through manual feature extraction. This paper proposes a novel method, the Deformation-Aware Temporal Generative Network (DATGN), to automate the learning of morphological changes in brain images about disease progression for early prediction. Given the common occurrence of missing data in the temporal sequences of MRI images, DATGN initially interpolates incomplete sequences. Subsequently, a bidirectional temporal deformation-aware module guides the network in generating future MRI images that adhere to the disease's progression, facilitating early prediction of Alzheimer's disease. DATGN was tested for the generation of temporal sequences of future MRI images using the ADNI dataset, and the experimental results are competitive in terms of PSNR and MMSE image quality metrics. Furthermore, when DATGN-generated synthetic data was integrated into the SVM vs. CNN vs. 3DCNN-based classification methods, significant improvements were achieved from 6. 21\% to 16\% in AD vs. NC classification accuracy and from 7. 34\% to 21. 25\% in AD vs. MCI vs. NC classification accuracy. The qualitative visualization results indicate that DATGN produces MRI images consistent with the brain atrophy trend in Alzheimer's disease, enabling early disease prediction.
• 摘要：阿尔茨海默病 (AD) 是一种大脑退行性疾病，可以从早期预测中受益，以减缓其进展。随着疾病的进展，患者通常会出现脑萎缩。目前阿尔茨海默病的预测方法主要涉及通过手动特征提取来分析大脑图像的形态变化。本文提出了一种新方法，即变形感知时间生成网络（DATGN），可以自动学习有关疾病进展的大脑图像形态变化，以进行早期预测。鉴于 MRI 图像的时间序列中丢失数据的常见情况，DATGN 最初会插入不完整的序列。随后，双向时间变形感知模块指导网络生成符合疾病进展的未来 MRI 图像，从而促进阿尔茨海默病的早期预测。使用 ADNI 数据集对 DATGN 生成未来 MRI 图像的时间序列进行了测试，实验结果在 PSNR 和 MMSE 图像质量指标方面具有竞争力。此外，当将 DATGN 生成的合成数据集成到基于 SVM、CNN、3DCNN 的分类方法中时，AD 与 NC 分类精度从 6. 21\% 显着提高到 16\%，AD、MCI 和 NC 分类精度从 7. 34\% 提高到 21. 25\%。定性可视化结果表明，DATGN 生成的 MRI 图像与阿尔茨海默病的脑萎缩趋势一致，从而能够进行早期疾病预测。

Title: Which Layer Causes Distribution Deviation? Entropy-Guided Adaptive Pruning for Diffusion and Flow Models

• Authors: Changlin Li, Jiawei Zhang, Zeyi Shi, Zongxin Yang, Zhihui Li, Xiaojun Chang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21122
• Pdf URL: https://arxiv.org/pdf/2511.21122
• Copy Paste: [[2511.21122]] Which Layer Causes Distribution Deviation? Entropy-Guided Adaptive Pruning for Diffusion and Flow Models(https://arxiv.org/abs/2511.21122)
• Keywords: generation, generative
• Abstract: Large-scale vision generative models, including diffusion and flow models, have demonstrated remarkable performance in visual generation tasks. However, transferring these pre-trained models to downstream tasks often results in significant parameter redundancy. In this paper, we propose EntPruner, an entropy-guided automatic progressive pruning framework for diffusion and flow models. First, we introduce entropy-guided pruning, a block-level importance assessment strategy specifically designed for generative models. Unlike discriminative models, generative models require preserving the diversity and condition-fidelity of the output distribution. As the importance of each module can vary significantly across downstream tasks, EntPruner prioritizes pruning of less important blocks using data-dependent Conditional Entropy Deviation (CED) as a guiding metric. CED quantifies how much the distribution diverges from the learned conditional data distribution after removing a block. Second, we propose a zero-shot adaptive pruning framework to automatically determine when and how much to prune during training. This dynamic strategy avoids the pitfalls of one-shot pruning, mitigating mode collapse, and preserving model performance. Extensive experiments on DiT and SiT models demonstrate the effectiveness of EntPruner, achieving up to 2.22$\times$ inference speedup while maintaining competitive generation quality on ImageNet and three downstream datasets.
• 摘要：大规模视觉生成模型，包括扩散模型和流动模型，在视觉生成任务中表现出了卓越的性能。然而，将这些预先训练的模型转移到下游任务通常会导致显着的参数冗余。在本文中，我们提出了 EntPruner，一种用于扩散和流动模型的熵引导自动渐进修剪框架。首先，我们介绍熵引导剪枝，这是一种专为生成模型设计的块级重要性评估策略。与判别模型不同，生成模型需要保留输出分布的多样性和条件保真度。由于每个模块的重要性在下游任务中可能存在显着差异，因此 EntPruner 使用数据相关的条件熵偏差 (CED) 作为指导指标，优先修剪不太重要的块。 CED 量化移除块后分布与学习的条件数据分布的偏离程度。其次，我们提出了一个零样本自适应剪枝框架，以自动确定训练期间剪枝的时间和剪枝量。这种动态策略避免了一次性剪枝、减轻模式崩溃和保持模型性能的陷阱。 DiT 和 SiT 模型的大量实验证明了 EntPruner 的有效性，实现了高达 2.22$\times$ 的推理加速，同时在 ImageNet 和三个下游数据集上保持有竞争力的生成质量。

Title: CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusion

• Authors: Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, Xi Qiu, Jialun Liu, Hao Pan, Yuchi Huo, Rui Wang, Haibin Huang, Chi Zhang, Xuelong Li
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2511.21129
• Pdf URL: https://arxiv.org/pdf/2511.21129
• Copy Paste: [[2511.21129]] CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusion(https://arxiv.org/abs/2511.21129)
• Keywords: generation
• Abstract: We tackle the dual challenges of video understanding and controllable video generation within a unified diffusion framework. Our key insights are two-fold: geometry-only cues (e.g., depth, edges) are insufficient: they specify layout but under-constrain appearance, materials, and illumination, limiting physically meaningful edits such as relighting or material swaps and often causing temporal drift. Enriching the model with additional graphics-based modalities (intrinsics and semantics) provides complementary constraints that both disambiguate understanding and enable precise, predictable control during generation. However, building a single model that uses many heterogeneous cues introduces two core difficulties. Architecturally, the model must accept any subset of modalities, remain robust to missing inputs, and inject control signals without sacrificing temporal consistency. Data-wise, training demands large-scale, temporally aligned supervision that ties real videos to per-pixel multimodal annotations. We then propose CtrlVDiff, a unified diffusion model trained with a Hybrid Modality Control Strategy (HMCS) that routes and fuses features from depth, normals, segmentation, edges, and graphics-based intrinsics (albedo, roughness, metallic), and re-renders videos from any chosen subset with strong temporal coherence. To enable this, we build MMVideo, a hybrid real-and-synthetic dataset aligned across modalities and captions. Across understanding and generation benchmarks, CtrlVDiff delivers superior controllability and fidelity, enabling layer-wise edits (relighting, material adjustment, object insertion) and surpassing state-of-the-art baselines while remaining robust when some modalities are unavailable.
• 摘要：我们在统一的扩散框架内解决视频理解和可控视频生成的双重挑战。我们的主要见解有两个方面：仅几何形状的线索（例如深度、边缘）是不够的：它们指定了布局，但对外观、材质和照明的约束不足，限制了物理上有意义的编辑，例如重新照明或材质交换，并且经常导致时间漂移。使用额外的基于图形的模态（内在和语义）丰富模型提供了补充约束，既可以消除理解歧义，又可以在生成过程中实现精确、可预测的控制。然而，构建使用许多异构线索的单一模型会带来两个核心困难。从架构上讲，模型必须接受模态的任何子集，对丢失的输入保持鲁棒性，并在不牺牲时间一致性的情况下注入控制信号。在数据方面，训练需要大规模、时间一致的监督，将真实视频与每像素多模态注释联系起来。然后，我们提出了 CtrlVDiff，这是一种使用混合模态控制策略 (HMCS) 进行训练的统一扩散模型，该模型路由和融合来自深度、法线、分割、边缘和基于图形的内在函数（反照率、粗糙度、金属）的特征，并重新渲染来自任何选定子集的具有强时间一致性的视频。为了实现这一点，我们构建了 MMVideo，这是一个跨模态和字幕对齐的真实和合成混合数据集。在理解和生成基准方面，CtrlVDiff 提供卓越的可控性和保真度，支持分层编辑（重新照明、材质调整、对象插入）并超越最先进的基准，同时在某些模式不可用时保持稳健。

Title: DeepRFTv2: Kernel-level Learning for Image Deblurring

• Authors: Xintian Mao, Haofei Song, Yin-Nian Liu, Qingli Li, Yan Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21132
• Pdf URL: https://arxiv.org/pdf/2511.21132
• Copy Paste: [[2511.21132]] DeepRFTv2: Kernel-level Learning for Image Deblurring(https://arxiv.org/abs/2511.21132)
• Keywords: restoration
• Abstract: It is well-known that if a network aims to learn how to deblur, it should understand the blur process. Blurring is naturally caused by the convolution of the sharp image with the blur kernel. Thus, allowing the network to learn the blur process in the kernel-level can significantly improve the image deblurring performance. But, current deep networks are still at the pixel-level learning stage, either performing end-to-end pixel-level restoration or stage-wise pseudo kernel-level restoration, failing to enable the deblur model to understand the essence of the blur. To this end, we propose Fourier Kernel Estimator (FKE), which considers the activation operation in Fourier space and converts the convolution problem in the spatial domain to a multiplication problem in Fourier space. Our FKE, jointly optimized with the deblur model, enables the network to learn the kernel-level blur process with low complexity and without any additional supervision. Furthermore, we change the convolution object of the kernel from ``image" to network extracted ``feature", whose rich semantic and structural information is more suitable to blur process learning. With the convolution of the feature and the estimated kernel, our model can learn the essence of blur in kernel-level. To further improve the efficiency of feature extraction, we design a decoupled multi-scale architecture with multiple hierarchical sub-unets with a reversible strategy, which allows better multi-scale encoding and decoding in low training memory. Extensive experiments indicate that our method achieves state-of-the-art motion deblurring results and show potential for handling other kernel-related problems. Analysis also shows our kernel estimator is able to learn physically meaningful kernels. The code will be available at this https URL.
• 摘要：众所周知，如果网络旨在学习如何去模糊，它应该了解模糊过程。模糊自然是由清晰图像与模糊核的卷积引起的。因此，允许网络在内核级学习模糊过程可以显着提高图像去模糊性能。但是，当前的深度网络仍处于像素级学习阶段，要么进行端到端的像素级恢复，要么进行阶段式伪内核级恢复，无法使去模糊模型理解模糊的本质。为此，我们提出了傅里叶核估计器（FKE），它考虑傅里叶空间中的激活操作，将空间域中的卷积问题转换为傅里叶空间中的乘法问题。我们的 FKE 与去模糊模型联合优化，使网络能够以低复杂度学习内核级模糊过程，并且无需任何额外的监督。此外，我们将内核的卷积对象从“图像”更改为网络提取的“特征”，其丰富的语义和结构信息更适合模糊过程学习。通过特征和估计内核的卷积，我们的模型可以在内核级别学习模糊的本质。为了进一步提高特征提取的效率，我们设计了一种具有多个分层子网的解耦多尺度架构，具有可逆策略，可以在低训练内存中实现更好的多尺度编码和解码。大量的实验表明，我们的方法实现了最先进的运动去模糊结果，并显示出处理其他内核相关问题的潜力。分析还表明我们的核估计器能够学习具有物理意义的核。该代码将在此 https URL 中提供。

Title: Efficient Training for Human Video Generation with Entropy-Guided Prioritized Progressive Learning

• Authors: Changlin Li, Jiawei Zhang, Shuhao Liu, Sihao Lin, Zeyi Shi, Zhihui Li, Xiaojun Chang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21136
• Pdf URL: https://arxiv.org/pdf/2511.21136
• Copy Paste: [[2511.21136]] Efficient Training for Human Video Generation with Entropy-Guided Prioritized Progressive Learning(https://arxiv.org/abs/2511.21136)
• Keywords: generation, generative
• Abstract: Human video generation has advanced rapidly with the development of diffusion models, but the high computational cost and substantial memory consumption associated with training these models on high-resolution, multi-frame data pose significant challenges. In this paper, we propose Entropy-Guided Prioritized Progressive Learning (Ent-Prog), an efficient training framework tailored for diffusion models on human video generation. First, we introduce Conditional Entropy Inflation (CEI) to assess the importance of different model components on the target conditional generation task, enabling prioritized training of the most critical components. Second, we introduce an adaptive progressive schedule that adaptively increases computational complexity during training by measuring the convergence efficiency. Ent-Prog reduces both training time and GPU memory consumption while maintaining model performance. Extensive experiments across three datasets, demonstrate the effectiveness of Ent-Prog, achieving up to 2.2$\times$ training speedup and 2.4$\times$ GPU memory reduction without compromising generative performance.
• 摘要：随着扩散模型的发展，人类视频生成迅速发展，但在高分辨率、多帧数据上训练这些模型所需的高计算成本和大量内存消耗带来了重大挑战。在本文中，我们提出了熵引导优先级渐进学习（Ent-Prog），这是一种专为人类视频生成扩散模型量身定制的高效训练框架。首先，我们引入条件熵膨胀（CEI）来评估不同模型组件对目标条件生成任务的重要性，从而实现对最关键组件的优先训练。其次，我们引入了一种自适应渐进式调度，它通过测量收敛效率来自适应地增加训练期间的计算复杂性。 Ent-Prog 减少了训练时间和 GPU 内存消耗，同时保持了模型性能。跨三个数据集的广泛实验证明了 Ent-Prog 的有效性，在不影响生成性能的情况下实现了高达 2.2$\times$ 的训练加速和 2.4$\times$ GPU 内存减少。

Title: TEAR: Temporal-aware Automated Red-teaming for Text-to-Video Models

• Authors: Jiaming He, Guanyu Hou, Hongwei Li, Zhicong Huang, Kangjie Chen, Yi Yu, Wenbo Jiang, Guowen Xu, Tianwei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21145
• Pdf URL: https://arxiv.org/pdf/2511.21145
• Copy Paste: [[2511.21145]] TEAR: Temporal-aware Automated Red-teaming for Text-to-Video Models(https://arxiv.org/abs/2511.21145)
• Keywords: generation
• Abstract: Text-to-Video (T2V) models are capable of synthesizing high-quality, temporally coherent dynamic video content, but the diverse generation also inherently introduces critical safety challenges. Existing safety evaluation methods,which focus on static image and text generation, are insufficient to capture the complex temporal dynamics in video generation. To address this, we propose a TEmporal-aware Automated Red-teaming framework, named TEAR, an automated framework designed to uncover safety risks specifically linked to the dynamic temporal sequencing of T2V models. TEAR employs a temporal-aware test generator optimized via a two-stage approach: initial generator training and temporal-aware online preference learning, to craft textually innocuous prompts that exploit temporal dynamics to elicit policy-violating video output. And a refine model is adopted to improve the prompt stealthiness and adversarial effectiveness cyclically. Extensive experimental evaluation demonstrates the effectiveness of TEAR across open-source and commercial T2V systems with over 80% attack success rate, a significant boost from prior best result of 57%.
• 摘要：文本到视频 (T2V) 模型能够合成高质量、时间连贯的动态视频内容，但多样化的生成也固有地带来了关键的安全挑战。现有的安全评估方法侧重于静态图像和文本生成，不足以捕获视频生成中复杂的时间动态。为了解决这个问题，我们提出了一个 TEmporal 感知的自动化红队框架，名为 TEAR，这是一个自动化框架，旨在发现与 T2V 模型的动态时间排序特别相关的安全风险。 TEAR 采用通过两阶段方法优化的时间感知测试生成器：初始生成器训练和时间感知在线偏好学习，以制作文本无害的提示，利用时间动态来引发违反策略的视频输出。并采用细化模型来循环提高即时隐身性和对抗有效性。广泛的实验评估证明了 TEAR 在开源和商业 T2V 系统中的有效性，攻击成功率超过 80%，比之前 57% 的最佳结果有了显着提升。

Title: Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation

• Authors: Joonhyung Park, Hyeongwon Jang, Joowon Kim, Eunho Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21185
• Pdf URL: https://arxiv.org/pdf/2511.21185
• Copy Paste: [[2511.21185]] Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation(https://arxiv.org/abs/2511.21185)
• Keywords: generation
• Abstract: Recent visual autoregressive (AR) models have shown promising capabilities in text-to-image generation, operating in a manner similar to large language models. While test-time computation scaling has brought remarkable success in enabling reasoning-enhanced outputs for challenging natural language tasks, its adaptation to visual AR models remains unexplored and poses unique challenges. Naively applying test-time scaling strategies such as Best-of-N can be suboptimal: they consume full-length computation on erroneous generation trajectories, while the raster-scan decoding scheme lacks a blueprint of the entire canvas, limiting scaling benefits as only a few prompt-aligned candidates are generated. To address these, we introduce GridAR, a test-time scaling framework designed to elicit the best possible results from visual AR models. GridAR employs a grid-partitioned progressive generation scheme in which multiple partial candidates for the same position are generated within a canvas, infeasible ones are pruned early, and viable ones are fixed as anchors to guide subsequent decoding. Coupled with this, we present a layout-specified prompt reformulation strategy that inspects partial views to infer a feasible layout for satisfying the prompt. The reformulated prompt then guides subsequent image generation to mitigate the blueprint deficiency. Together, GridAR achieves higher-quality results under limited test-time scaling: with N=4, it even outperforms Best-of-N (N=8) by 14.4% on T2I-CompBench++ while reducing cost by 25.6%. It also generalizes to autoregressive image editing, showing comparable edit quality and a 13.9% gain in semantic preservation on PIE-Bench over larger-N baselines.
• 摘要：最近的视觉自回归（AR）模型在文本到图像生成方面表现出了有前途的能力，其操作方式类似于大型语言模型。虽然测试时计算扩展在为具有挑战性的自然语言任务提供推理增强输出方面取得了显着的成功，但其对视觉 AR 模型的适应仍未得到探索，并带来了独特的挑战。天真地应用测试时缩放策略（例如 Best-of-N）可能不是最理想的：它们消耗了错误生成轨迹上的全长计算，而光栅扫描解码方案缺乏整个画布的蓝图，限制了缩放优势，因为只生成了一些提示对齐的候选者。为了解决这些问题，我们引入了 GridAR，这是一个测试时间缩放框架，旨在从视觉 AR 模型中得出最佳结果。 GridAR采用网格划分的渐进生成方案，其中在画布内生成同一位置的多个部分候选，不可行的被提前修剪，可行的被固定为锚点以指导后续解码。除此之外，我们提出了一种布局指定的提示重新制定策略，该策略检查部分视图以推断满足提示的可行布局。然后，重新制定的提示将指导后续图像生成，以减轻蓝图缺陷。总之，GridAR 在有限的测试时间扩展下实现了更高质量的结果：当 N=4 时，它在 T2I-CompBench++ 上的性能甚至比 Best-of-N (N=8) 高出 14.4%，同时成本降低了 25.6%。它还推广到自回归图像编辑，显示出可比较的编辑质量，并且在 PIE-Bench 上语义保留比较大 N 基线提高了 13.9%。

Title: From Diffusion to One-Step Generation: A Comparative Study of Flow-Based Models with Application to Image Inpainting

• Authors: Umang Agarwal, Rudraksh Sangore, Sumit Laddha
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21215
• Pdf URL: https://arxiv.org/pdf/2511.21215
• Copy Paste: [[2511.21215]] From Diffusion to One-Step Generation: A Comparative Study of Flow-Based Models with Application to Image Inpainting(https://arxiv.org/abs/2511.21215)
• Keywords: generation, generative
• Abstract: We present a comprehensive comparative study of three generative modeling paradigms: Denoising Diffusion Probabilistic Models (DDPM), Conditional Flow Matching (CFM), and MeanFlow. While DDPM and CFM require iterative sampling, MeanFlow enables direct one-step generation by modeling the average velocity over time intervals. We implement all three methods using a unified TinyUNet architecture (<1.5M parameters) on CIFAR-10, demonstrating that CFM achieves an FID of 24.15 with 50 steps, significantly outperforming DDPM (FID 402.98). MeanFlow achieves FID 29.15 with single-step sampling -- a 50X reduction in inference time. We further extend CFM to image inpainting, implementing mask-guided sampling with four mask types (center, random bbox, irregular, half). Our fine-tuned inpainting model achieves substantial improvements: PSNR increases from 4.95 to 8.57 dB on center masks (+73%), and SSIM improves from 0.289 to 0.418 (+45%), demonstrating the effectiveness of inpainting-aware training.
• 摘要：我们对三种生成建模范式进行了全面的比较研究：去噪扩散概率模型 (DDPM)、条件流匹配 (CFM) 和 MeanFlow。虽然 DDPM 和 CFM 需要迭代采样，但 MeanFlow 通过对时间间隔内的平均速度进行建模来实现直接一步生成。我们在 CIFAR-10 上使用统一的 TinyUNet 架构（<1.5M 参数）实现了所有三种方法，证明 CFM 通过 50 个步骤实现了 24.15 的 FID，显着优于 DDPM (FID 402.98)。 MeanFlow 通过单步采样实现了 FID 29.15，推理时间缩短了 50 倍。我们进一步将 CFM 扩展到图像修复，实现具有四种掩模类型（中心、随机 bbox、不规则、半）的掩模引导采样。我们经过微调的修复模型取得了显着的改进：中心掩模上的 PSNR 从 4.95 dB 增加到 8.57 dB (+73%)，SSIM 从 0.289 提高到 0.418 (+45%)，证明了修复感知训练的有效性。

Title: AVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMs

• Authors: Shuhan Xia, Peipei Li, Xuannan Liu, Dongsen Zhang, Xinyu Guo, Zekun Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21251
• Pdf URL: https://arxiv.org/pdf/2511.21251
• Copy Paste: [[2511.21251]] AVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMs(https://arxiv.org/abs/2511.21251)
• Keywords: generative
• Abstract: The threat of Audio-Video (AV) forgery is rapidly evolving beyond human-centric deepfakes to include more diverse manipulations across complex natural scenes. However, existing benchmarks are still confined to DeepFake-based forgeries and single-granularity annotations, thus failing to capture the diversity and complexity of real-world forgery scenarios. To address this, we introduce AVFakeBench, the first comprehensive audio-video forgery detection benchmark that spans rich forgery semantics across both human subject and general subject. AVFakeBench comprises 12K carefully curated audio-video questions, covering seven forgery types and four levels of annotations. To ensure high-quality and diverse forgeries, we propose a multi-stage hybrid forgery framework that integrates proprietary models for task planning with expert generative models for precise manipulation. The benchmark establishes a multi-task evaluation framework covering binary judgment, forgery types classification, forgery detail selection, and explanatory reasoning. We evaluate 11 Audio-Video Large Language Models (AV-LMMs) and 2 prevalent detection methods on AVFakeBench, demonstrating the potential of AV-LMMs as emerging forgery detectors while revealing their notable weaknesses in fine-grained perception and reasoning.
• 摘要：音频视频 (AV) 伪造的威胁正在迅速演变，超越以人类为中心的深度伪造，包括跨复杂自然场景的更多样化的操作。然而，现有的基准仍然局限于基于 DeepFake 的伪造和单粒度注释，因此无法捕捉现实世界伪造场景的多样性和复杂性。为了解决这个问题，我们推出了 AVFakeBench，这是第一个全面的音频-视频伪造检测基准，涵盖了人类主体和一般主体的丰富伪造语义。 AVFakeBench 包含 12K 个精心策划的音频视频问题，涵盖七种伪造类型和四个级别的注释。为了确保高质量和多样化的伪造，我们提出了一个多阶段混合伪造框架，该框架将用于任务规划的专有模型与用于精确操作的专家生成模型集成在一起。该基准建立了一个涵盖二元判断、伪造类型分类、伪造细节选择和解释推理的多任务评估框架。我们在 AVFakeBench 上评估了 11 种音频视频大语言模型 (AV-LMM) 和 2 种流行的检测方法，展示了 AV-LMM 作为新兴伪造检测器的潜力，同时揭示了它们在细粒度感知和推理方面的显着弱点。

Title: LaGen: Towards Autoregressive LiDAR Scene Generation

• Authors: Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang, Junjie Li, Juyong Zhang, Yukang Feng, Jianwen Sun, Songbur Wong, Junqi You, Junchi Yan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21256
• Pdf URL: https://arxiv.org/pdf/2511.21256
• Copy Paste: [[2511.21256]] LaGen: Towards Autoregressive LiDAR Scene Generation(https://arxiv.org/abs/2511.21256)
• Keywords: generation, generative
• Abstract: Generative world models for autonomous driving (AD) have become a trending topic. Unlike the widely studied image modality, in this work we explore generative world models for LiDAR data. Existing generation methods for LiDAR data only support single frame generation, while existing prediction approaches require multiple frames of historical input and can only deterministically predict multiple frames at once, lacking interactivity. Both paradigms fail to support long-horizon interactive generation. To this end, we introduce LaGen, which to the best of our knowledge is the first framework capable of frame-by-frame autoregressive generation of long-horizon LiDAR scenes. LaGen is able to take a single-frame LiDAR input as a starting point and effectively utilize bounding box information as conditions to generate high-fidelity 4D scene point clouds. In addition, we introduce a scene decoupling estimation module to enhance the model's interactive generation capability for object-level content, as well as a noise modulation module to mitigate error accumulation during long-horizon generation. We construct a protocol based on nuScenes for evaluating long-horizon LiDAR scene generation. Experimental results comprehensively demonstrate LaGen outperforms state-of-the-art LiDAR generation and prediction models, especially on the later frames.
• 摘要：自动驾驶（AD）的生成世界模型已成为热门话题。与广泛研究的图像模态不同，在这项工作中，我们探索 LiDAR 数据的生成世界模型。现有的激光雷达数据生成方法仅支持单帧生成，而现有的预测方法需要多帧历史输入，并且只能一次确定性预测多帧，缺乏交互性。这两种范式都无法支持长视野交互生成。为此，我们引入了LaGen，据我们所知，它是第一个能够逐帧自回归生成长视距激光雷达场景的框架。 LaGen能够以单帧LiDAR输入为起点，有效利用边界框信息作为条件，生成高保真4D场景点云。此外，我们引入了场景解耦估计模块来增强模型对对象级内容的交互生成能力，以及噪声调制模块来减轻长视野生成过程中的错误累积。我们构建了一个基于 nuScenes 的协议，用于评估长视距 LiDAR 场景生成。实验结果全面证明 LaGen 优于最先进的 LiDAR 生成和预测模型，尤其是在后面的帧上。

Title: Unlocking Zero-shot Potential of Semi-dense Image Matching via Gaussian Splatting

• Authors: Juncheng Chen, Chao Xu, Yanjun Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21265
• Pdf URL: https://arxiv.org/pdf/2511.21265
• Copy Paste: [[2511.21265]] Unlocking Zero-shot Potential of Semi-dense Image Matching via Gaussian Splatting(https://arxiv.org/abs/2511.21265)
• Keywords: generation
• Abstract: Learning-based image matching critically depends on large-scale, diverse, and geometrically accurate training data. 3D Gaussian Splatting (3DGS) enables photorealistic novel-view synthesis and thus is attractive for data generation. However, its geometric inaccuracies and biased depth rendering currently prevent robust correspondence labeling. To address this, we introduce MatchGS, the first framework designed to systematically correct and leverage 3DGS for robust, zero-shot image matching. Our approach is twofold: (1) a geometrically-faithful data generation pipeline that refines 3DGS geometry to produce highly precise correspondence labels, enabling the synthesis of a vast and diverse range of viewpoints without compromising rendering fidelity; and (2) a 2D-3D representation alignment strategy that infuses 3DGS' explicit 3D knowledge into the 2D matcher, guiding 2D semi-dense matchers to learn viewpoint-invariant 3D representations. Our generated ground-truth correspondences reduce the epipolar error by up to 40 times compared to existing datasets, enable supervision under extreme viewpoint changes, and provide self-supervisory signals through Gaussian attributes. Consequently, state-of-the-art matchers trained solely on our data achieve significant zero-shot performance gains on public benchmarks, with improvements of up to 17.7%. Our work demonstrates that with proper geometric refinement, 3DGS can serve as a scalable, high-fidelity, and structurally-rich data source, paving the way for a new generation of robust zero-shot image matchers.
• 摘要：基于学习的图像匹配关键取决于大规模、多样化且几何精确的训练数据。 3D 高斯溅射 (3DGS) 可以实现逼真的新颖视图合成，因此对于数据生成很有吸引力。然而，其几何不准确和有偏差的深度渲染目前阻碍了稳健的对应标记。为了解决这个问题，我们引入了 MatchGS，这是第一个旨在系统地校正和利用 3DGS 进行稳健、零样本图像匹配的框架。我们的方法是双重的：(1) 几何忠实的数据生成管道，可细化 3DGS 几何图形以生成高精度的对应标签，从而能够在不影响渲染保真度的情况下合成广泛多样的视点； (2) 2D-3D 表示对齐策略，将 3DGS 的显式 3D 知识注入 2D 匹配器中，引导 2D 半密集匹配器学习视点不变的 3D 表示。与现有数据集相比，我们生成的地面实况对应将极线误差减少了多达 40 倍，能够在极端视点变化下进行监督，并通过高斯属性提供自我监督信号。因此，仅根据我们的数据进行训练的最先进的匹配器在公共基准上实现了显着的零样本性能提升，提升幅度高达 17.7%。我们的工作表明，通过适当的几何细化，3DGS 可以作为可扩展、高保真且结构丰富的数据源，为新一代稳健的零样本图像匹配器铺平道路。

Title: CaliTex: Geometry-Calibrated Attention for View-Coherent 3D Texture Generation

• Authors: Chenyu Liu, Hongze Chen, Jingzhi Bao, Lingting Zhu, Runze Zhang, Weikai Chen, Zeyu Hu, Yingda Yin, Keyang Luo, Xin Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21309
• Pdf URL: https://arxiv.org/pdf/2511.21309
• Copy Paste: [[2511.21309]] CaliTex: Geometry-Calibrated Attention for View-Coherent 3D Texture Generation(https://arxiv.org/abs/2511.21309)
• Keywords: generation
• Abstract: Despite major advances brought by diffusion-based models, current 3D texture generation systems remain hindered by cross-view inconsistency -- textures that appear convincing from one viewpoint often fail to align across others. We find that this issue arises from attention ambiguity, where unstructured full attention is applied indiscriminately across tokens and modalities, causing geometric confusion and unstable appearance-structure coupling. To address this, we introduce CaliTex, a framework of geometry-calibrated attention that explicitly aligns attention with 3D structure. It introduces two modules: Part-Aligned Attention that enforces spatial alignment across semantically matched parts, and Condition-Routed Attention which routes appearance information through geometry-conditioned pathways to maintain spatial fidelity. Coupled with a two-stage diffusion transformer, CaliTex makes geometric coherence an inherent behavior of the network rather than a byproduct of optimization. Empirically, CaliTex produces seamless and view-consistent textures and outperforms both open-source and commercial baselines.
• 摘要：尽管基于扩散的模型带来了重大进步，但当前的 3D 纹理生成系统仍然受到跨视图不一致的阻碍 - 从一个角度看来令人信服的纹理通常无法在其他角度对齐。我们发现这个问题源于注意力模糊性，即非结构化的充分注意力不加区别地应用于令牌和模态，导致几何混乱和不稳定的外观结构耦合。为了解决这个问题，我们引入了 CaliTex，这是一个几何校准注意力框架，可以明确地将注意力与 3D 结构对齐。它引入了两个模块：部分对齐注意力（Part-Aligned Attention），强制跨语义匹配的部分进行空间对齐；条件路由注意力（Condition-routed Attention），通过几何条件路径路由外观信息，以保持空间保真度。与两级扩散变压器相结合，CaliTex 使几何一致性成为网络的固有行为，而不是优化的副产品。根据经验，CaliTex 可以生成无缝且视图一致的纹理，并且优于开源和商业基线。

Title: TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models

• Authors: Haksoo Lim, Jaehoon Lee, Sewon Park, Minjung Kim, Noseong Park
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21335
• Pdf URL: https://arxiv.org/pdf/2511.21335
• Copy Paste: [[2511.21335]] TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models(https://arxiv.org/abs/2511.21335)
• Keywords: generation, generative
• Abstract: Score-based generative models (SGMs) have demonstrated unparalleled sampling quality and diversity in numerous fields, such as image generation, voice synthesis, and tabular data synthesis, etc. Inspired by those outstanding results, we apply SGMs to synthesize time-series by learning its conditional score function. To this end, we present a conditional score network for time-series synthesis, deriving a denoising score matching loss tailored for our purposes. In particular, our presented denoising score matching loss is the conditional denoising score matching loss for time-series synthesis. In addition, our framework is such flexible that both regular and irregular time-series can be synthesized with minimal changes to our model design. Finally, we obtain exceptional synthesis performance on various time-series datasets, achieving state-of-the-art sampling diversity and quality.
• 摘要：基于分数的生成模型（SGM）在图像生成、语音合成和表格数据合成等众多领域表现出了无与伦比的采样质量和多样性。受到这些出色结果的启发，我们通过学习其条件分数函数来应用SGM来合成时间序列。为此，我们提出了一个用于时间序列合成的条件得分网络，得出适合我们目的的去噪得分匹配损失。特别是，我们提出的去噪分数匹配损失是时间序列合成的条件去噪分数匹配损失。此外，我们的框架非常灵活，只需对模型设计进行最小的更改即可合成规则和不规则的时间序列。最后，我们在各种时间序列数据集上获得了出色的综合性能，实现了最先进的采样多样性和质量。

Title: Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models

• Authors: Julianna Piskorz, Cristina Pinneri, Alvaro Correia, Motasem Alfarra, Risheek Garrepalli, Christos Louizos
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21338
• Pdf URL: https://arxiv.org/pdf/2511.21338
• Copy Paste: [[2511.21338]] Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models(https://arxiv.org/abs/2511.21338)
• Keywords: generation
• Abstract: Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in principle, should enable more uniform context utilisation. In this work, we examine the context comprehension abilities of MDLMs and uncover two key limitations. First, despite their more global training objective and bidirectional attention mechanism, similarly to ARLMS, MDLMs exhibit a strong locality bias: performance is highly sensitive to the position of relevant information within the input, favouring local over distant context. Second, we show that appending a large number of mask tokens--required for generation--can significantly degrade context comprehension. Through systematic ablations, we find that these masks act as distractors, reducing the model's ability to process relevant information. To address this, we introduce a mask-agnostic loss function that encourages predictions to remain invariant to the number of appended masks. Fine-tuning with this objective substantially mitigates the distracting effect of masks, improving robustness of MDLMs. Overall, our findings reveal critical limitations of the current MDLM training paradigm and provide actionable insights for building diffusion-based language models with stronger context comprehension.
• 摘要：掩蔽扩散语言模型 (MDLM) 最近成为自回归语言模型 (ARLM) 的一种有前景的替代方案，利用去噪目标，原则上应该能够实现更统一的上下文利用。在这项工作中，我们检查了 MDLM 的上下文理解能力并发现了两个关键限制。首先，尽管 MDLM 具有更全局的训练目标和双向注意机制，但与 ARLMS 类似，MDLM 表现出强烈的局部性偏差：性能对输入中相关信息的位置高度敏感，倾向于本地而不是远程上下文。其次，我们表明附加大量掩码标记（生成所需）会显着降低上下文理解能力。通过系统的消融，我们发现这些掩模充当干扰因素，降低了模型处理相关信息的能力。为了解决这个问题，我们引入了一个与掩模无关的损失函数，它鼓励预测对附加掩模的数量保持不变。以此为目标进行微调可显着减轻掩模的干扰效应，从而提高 MDLM 的稳健性。总的来说，我们的研究结果揭示了当前 MDLM 训练范式的关键局限性，并为构建具有更强上下文理解的基于扩散的语言模型提供了可行的见解。

Title: DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models

• Authors: Mingue Park, Prin Phunyaphibarn, Phillip Y. Lee, Minhyuk Sung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21415
• Pdf URL: https://arxiv.org/pdf/2511.21415
• Copy Paste: [[2511.21415]] DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models(https://arxiv.org/abs/2511.21415)
• Keywords: generation
• Abstract: We introduce DiverseVAR, a framework that enhances the diversity of text-conditioned visual autoregressive models (VAR) at test time without requiring retraining, fine-tuning, or substantial computational overhead. While VAR models have recently emerged as strong competitors to diffusion and flow models for image generation, they suffer from a critical limitation in diversity, often producing nearly identical images even for simple prompts. This issue has largely gone unnoticed amid the predominant focus on image quality. We address this limitation at test time in two stages. First, inspired by diversity enhancement techniques in diffusion models, we propose injecting noise into the text embedding. This introduces a trade-off between diversity and image quality: as diversity increases, the image quality sharply declines. To preserve quality, we propose scale-travel: a novel latent refinement technique inspired by time-travel strategies in diffusion models. Specifically, we use a multi-scale autoencoder to extract coarse-scale tokens that enable us to resume generation at intermediate stages. Extensive experiments show that combining text-embedding noise injection with our scale-travel refinement significantly enhances diversity while minimizing image-quality degradation, achieving a new Pareto frontier in the diversity-quality trade-off.
• 摘要：我们引入了 DiverseVAR，这是一个框架，可以在测试时增强文本条件视觉自回归模型 (VAR) 的多样性，而无需重新训练、微调或大量计算开销。虽然 VAR 模型最近已成为图像生成的扩散和流动模型的有力竞争对手，但它们在多样性方面受到严重限制，即使对于简单的提示，通常也会生成几乎相同的图像。由于人们普遍关注图像质量，这个问题在很大程度上被忽视了。我们在测试时分两个阶段解决这个限制。首先，受到扩散模型中多样性增强技术的启发，我们建议将噪声注入文本嵌入中。这就引入了多样性和图像质量之间的权衡：随着多样性的增加，图像质量急剧下降。为了保持质量，我们提出了尺度旅行：一种受扩散模型中的时间旅行策略启发的新型潜在细化技术。具体来说，我们使用多尺度自动编码器来提取粗尺度标记，使我们能够在中间阶段恢复生成。大量实验表明，将文本嵌入噪声注入与我们的尺度行程细化相结合，可以显着增强多样性，同时最大限度地减少图像质量下降，从而在多样性与质量权衡中实现新的帕累托前沿。

Title: MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices

• Authors: Shuai Zhang, Bao Tang, Siyuan Yu, Yueting Zhu, Jingfeng Yao, Ya Zou, Shanglin Yuan, Li Yu, Wenyu Liu, Xinggang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21475
• Pdf URL: https://arxiv.org/pdf/2511.21475
• Copy Paste: [[2511.21475]] MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices(https://arxiv.org/abs/2511.21475)
• Keywords: generation
• Abstract: Recently, video generation has witnessed rapid advancements, drawing increasing attention to image-to-video (I2V) synthesis on mobile devices. However, the substantial computational complexity and slow generation speed of diffusion models pose significant challenges for real-time, high-resolution video generation on resource-constrained mobile devices. In this work, we propose MobileI2V, a 270M lightweight diffusion model for real-time image-to-video generation on mobile devices. The core lies in: (1) We analyzed the performance of linear attention modules and softmax attention modules on mobile devices, and proposed a linear hybrid architecture denoiser that balances generation efficiency and quality. (2) We design a time-step distillation strategy that compresses the I2V sampling steps from more than 20 to only two without significant quality loss, resulting in a 10-fold increase in generation speed. (3) We apply mobile-specific attention optimizations that yield a 2-fold speed-up for attention operations during on-device inference. MobileI2V enables, for the first time, fast 720p image-to-video generation on mobile devices, with quality comparable to existing models. Under one-step conditions, the generation speed of each frame of 720p video is less than 100 ms. Our code is available at: this https URL.
• 摘要：最近，视频生成取得了快速发展，引起了人们对移动设备上图像到视频（I2V）合成的越来越多的关注。然而，扩散模型的巨大计算复杂性和缓慢的生成速度对资源受限的移动设备上的实时、高分辨率视频生成提出了重大挑战。在这项工作中，我们提出了 MobileI2V，这是一种 270M 轻量级扩散模型，用于在移动设备上实时生成图像到视频。核心在于：（1）我们分析了线性注意力模块和softmax注意力模块在移动设备上的性能，提出了一种平衡生成效率和质量的线性混合架构降噪器。 (2) 我们设计了一种时间步蒸馏策略，将 I2V 采样步骤从 20 多个压缩到只有 2 个，而没有显着的质量损失，从而使生成速度提高了 10 倍。 (3) 我们应用了针对移动设备的注意力优化，使设备上推理期间的注意力操作速度提高了 2 倍。 MobileI2V 首次能够在移动设备上快速生成 720p 图像到视频，其质量可与现有型号相媲美。在一步条件下，720p视频每一帧的生成速度小于100ms。我们的代码位于：此 https URL。

Title: The Age-specific Alzheimer 's Disease Prediction with Characteristic Constraints in Nonuniform Time Span

• Authors: Xin Hong, Kaifeng Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21530
• Pdf URL: https://arxiv.org/pdf/2511.21530
• Copy Paste: [[2511.21530]] The Age-specific Alzheimer 's Disease Prediction with Characteristic Constraints in Nonuniform Time Span(https://arxiv.org/abs/2511.21530)
• Keywords: generation
• Abstract: Alzheimer's disease is a debilitating disorder marked by a decline in cognitive function. Timely identification of the disease is essential for the development of personalized treatment strategies that aim to mitigate its progression. The application of generated images for the prediction of Alzheimer's disease poses challenges, particularly in accurately representing the disease's characteristics when input sequences are captured at irregular time intervals. This study presents an innovative methodology for sequential image generation, guided by quantitative metrics, to maintain the essential features indicative of disease progression. Furthermore, an age-scaling factor is integrated into the process to produce age-specific MRI images, facilitating the prediction of advanced stages of the disease. The results obtained from the ablation study suggest that the inclusion of quantitative metrics significantly improves the accuracy of MRI image synthesis. Furthermore, the application of age-scaled pixel loss contributed to the enhanced iterative generation of MRI images. In terms of long-term disease prognosis, the Structural Similarity Index reached a peak value of 0.882, indicating a substantial degree of similarity in the synthesized images.
• 摘要：阿尔茨海默病是一种使人衰弱的疾病，其特征是认知功能下降。及时识别疾病对于制定旨在减缓其进展的个性化治疗策略至关重要。应用生成的图像来预测阿尔茨海默病提出了挑战，特别是在以不规则的时间间隔捕获输入序列时准确表示疾病的特征。这项研究提出了一种以定量指标为指导的连续图像生成的创新方法，以保持指示疾病进展的基本特征。此外，年龄缩放因子被集成到生成特定年龄 MRI 图像的过程中，有助于预测疾病的晚期阶段。消融研究的结果表明，定量指标的纳入显着提高了 MRI 图像合成的准确性。此外，年龄尺度像素损失的应用有助于增强 MRI 图像的迭代生成。在长期疾病预后方面，结构相似性指数达到峰值0.882，表明合成图像具有很大程度的相似性。

Title: Video Generation Models Are Good Latent Reward Models

• Authors: Xiaoyue Mi, Wenqing Yu, Jiesong Lian, Shibo Jie, Ruizhe Zhong, Zijun Liu, Guozhen Zhang, Zixiang Zhou, Zhiyong Xu, Yuan Zhou, Qinglin Lu, Fan Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21541
• Pdf URL: https://arxiv.org/pdf/2511.21541
• Copy Paste: [[2511.21541]] Video Generation Models Are Good Latent Reward Models(https://arxiv.org/abs/2511.21541)
• Keywords: generation
• Abstract: Reward feedback learning (ReFL) has proven effective for aligning image generation with human preferences. However, its extension to video generation faces significant challenges. Existing video reward models rely on vision-language models designed for pixel-space inputs, confining ReFL optimization to near-complete denoising steps after computationally expensive VAE decoding. This pixel-space approach incurs substantial memory overhead and increased training time, and its late-stage optimization lacks early-stage supervision, refining only visual quality rather than fundamental motion dynamics and structural coherence. In this work, we show that pre-trained video generation models are naturally suited for reward modeling in the noisy latent space, as they are explicitly designed to process noisy latent representations at arbitrary timesteps and inherently preserve temporal information through their sequential modeling capabilities. Accordingly, we propose Process Reward Feedback Learning~(PRFL), a framework that conducts preference optimization entirely in latent space, enabling efficient gradient backpropagation throughout the full denoising chain without VAE decoding. Extensive experiments demonstrate that PRFL significantly improves alignment with human preferences, while achieving substantial reductions in memory consumption and training time compared to RGB ReFL.
• 摘要：事实证明，奖励反馈学习（ReFL）对于使图像生成与人类偏好保持一致是有效的。然而，其扩展到视频生成面临着重大挑战。现有的视频奖励模型依赖于为像素空间输入设计的视觉语言模型，将 ReFL 优化限制在计算成本高昂的 VAE 解码后接近完成的去噪步骤。这种像素空间方法会产生大量的内存开销和增加的训练时间，并且其后期优化缺乏早期监督，仅改进视觉质量，而不是基本的运动动力学和结构连贯性。在这项工作中，我们表明预训练的视频生成模型自然适合在噪声潜在空间中进行奖励建模，因为它们被明确设计为在任意时间步处理噪声潜在表示，并通过其顺序建模功能本质上保留时间信息。因此，我们提出了过程奖励反馈学习（PRFL），这是一个完全在潜在空间中进行偏好优化的框架，无需 VAE 解码即可在整个去噪链中实现高效的梯度反向传播。大量实验表明，与 RGB ReFL 相比，PRFL 显着提高了与人类偏好的一致性，同时显着减少了内存消耗和训练时间。

Title: Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy

• Authors: Teng Hu, Zhentao Yu, Guozhen Zhang, Zihan Su, Zhengguang Zhou, Youliang Zhang, Yuan Zhou, Qinglin Lu, Ran Yi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21579
• Pdf URL: https://arxiv.org/pdf/2511.21579
• Copy Paste: [[2511.21579]] Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy(https://arxiv.org/abs/2511.21579)
• Keywords: generation, generative
• Abstract: The synthesis of synchronized audio-visual content is a key challenge in generative AI, with open-source models facing challenges in robust audio-video alignment. Our analysis reveals that this issue is rooted in three fundamental challenges of the joint diffusion process: (1) Correspondence Drift, where concurrently evolving noisy latents impede stable learning of alignment; (2) inefficient global attention mechanisms that fail to capture fine-grained temporal cues; and (3) the intra-modal bias of conventional Classifier-Free Guidance (CFG), which enhances conditionality but not cross-modal synchronization. To overcome these challenges, we introduce Harmony, a novel framework that mechanistically enforces audio-visual synchronization. We first propose a Cross-Task Synergy training paradigm to mitigate drift by leveraging strong supervisory signals from audio-driven video and video-driven audio generation tasks. Then, we design a Global-Local Decoupled Interaction Module for efficient and precise temporal-style alignment. Finally, we present a novel Synchronization-Enhanced CFG (SyncCFG) that explicitly isolates and amplifies the alignment signal during inference. Extensive experiments demonstrate that Harmony establishes a new state-of-the-art, significantly outperforming existing methods in both generation fidelity and, critically, in achieving fine-grained audio-visual synchronization.
• 摘要：同步视听内容的合成是生成人工智能的一个关键挑战，开源模型面临着稳健的视听对齐的挑战。我们的分析表明，这个问题根源于联合扩散过程的三个基本挑战：（1）对应漂移，同时演化的噪声潜伏阻碍了对齐的稳定学习； (2) 低效的全局注意力机制，无法捕捉细粒度的时间线索； （3）传统无分类器指导（CFG）的模态内偏差，它增强了条件性，但没有增强跨模态同步。为了克服这些挑战，我们引入了 Harmony，这是一种新颖的框架，可以机械地强制执行视听同步。我们首先提出了一种跨任务协同训练范例，通过利用来自音频驱动视频和视频驱动音频生成任务的强大监督信号来减轻漂移。然后，我们设计了一个全局-局部解耦交互模块，以实现高效、精确的时间式对齐。最后，我们提出了一种新颖的同步增强型 CFG (SyncCFG)，它可以在推理过程中显式地隔离和放大对齐信号。大量实验表明，Harmony 建立了一种新的最先进技术，在生成保真度以及最重要的是实现细粒度视听同步方面都显着优于现有方法。

Title: MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training

• Authors: Haotian Xue, Qi Chen, Zhonghao Wang, Xun Huang, Eli Shechtman, Jinrong Xie, Yongxin Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21592
• Pdf URL: https://arxiv.org/pdf/2511.21592
• Copy Paste: [[2511.21592]] MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training(https://arxiv.org/abs/2511.21592)
• Keywords: generation
• Abstract: Video diffusion models achieve strong frame-level fidelity but still struggle with motion coherence, dynamics and realism, often producing jitter, ghosting, or implausible dynamics. A key limitation is that the standard denoising MSE objective provides no direct supervision on temporal consistency, allowing models to achieve low loss while still generating poor motion. We propose MoGAN, a motion-centric post-training framework that improves motion realism without reward models or human preference data. Built atop a 3-step distilled video diffusion model, we train a DiT-based optical-flow discriminator to differentiate real from generated motion, combined with a distribution-matching regularizer to preserve visual fidelity. With experiments on Wan2.1-T2V-1.3B, MoGAN substantially improves motion quality across benchmarks. On VBench, MoGAN boosts motion score by +7.3% over the 50-step teacher and +13.3% over the 3-step DMD model. On VideoJAM-Bench, MoGAN improves motion score by +7.4% over the teacher and +8.8% over DMD, while maintaining comparable or even better aesthetic and image-quality scores. A human study further confirms that MoGAN is preferred for motion quality (52% vs. 38% for the teacher; 56% vs. 29% for DMD). Overall, MoGAN delivers significantly more realistic motion without sacrificing visual fidelity or efficiency, offering a practical path toward fast, high-quality video generation. Project webpage is: this https URL.
• 摘要：视频扩散模型实现了强大的帧级保真度，但仍难以实现运动连贯性、动态性和真实性，经常产生抖动、重影或令人难以置信的动态。一个关键的限制是标准去噪 MSE 目标不提供对时间一致性的直接监督，从而允许模型实现低损失，同时仍然产生较差的运动。我们提出了 MoGAN，一种以运动为中心的后训练框架，无需奖励模型或人类偏好数据即可提高运动真实感。我们构建在三步蒸馏视频扩散模型之上，训练基于 DiT 的光流鉴别器来区分真实运动和生成运动，并结合分布匹配正则器来保持视觉保真度。通过在 Wan2.1-T2V-1.3B 上进行实验，MoGAN 显着提高了基准测试中的运动质量。在 VBench 上，MoGAN 的运动得分比 50 步教师模型提高了 7.3%，比 3 步 DMD 模型提高了 13.3%。在 VideoJAM-Bench 上，MoGAN 的运动得分比老师提高了 7.4%，比 DMD 提高了 8.8%，同时保持了相当甚至更好的美学和图像质量得分。一项人类研究进一步证实，MoGAN 在运动质量方面更受青睐（教师为 52% vs. 38%；DMD 为 56% vs. 29%）。总体而言，MoGAN 在不牺牲视觉保真度或效率的情况下提供了更加真实的运动，为快速、高质量视频生成提供了一条实用途径。项目网页是：这个https URL。

Title: Aligning LLMs Toward Multi-Turn Conversational Outcomes Using Iterative PPO

• Authors: Daniel R. Jiang, Jalaj Bhandari, Yukai Yang, Rémi Munos, Tyler Lu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21638
• Pdf URL: https://arxiv.org/pdf/2511.21638
• Copy Paste: [[2511.21638]] Aligning LLMs Toward Multi-Turn Conversational Outcomes Using Iterative PPO(https://arxiv.org/abs/2511.21638)
• Keywords: generation
• Abstract: Optimizing large language models (LLMs) for multi-turn conversational outcomes remains a significant challenge, especially in goal-oriented settings like AI marketing or sales agents who facilitate transactions via messaging platforms. The difficulty stems from sparse, long-horizon rewards and the discrepancy between response-level planning and token-level generation. In this technical note, we propose a formal reduction of the multi-turn RL problem into a sequence of single-turn RLHF-style problems. This is achieved by setting a learned multi-turn Q-function as the reward model for the single-turn problem. We demonstrate and prove a key insight: solving this single-turn RL problem with standard token-level PPO is equivalent to a policy improvement step within the multi-turn problem. This insight naturally leads to Iterative PPO, a batch online policy iteration algorithm that alternates between fitting Q-functions from logged conversation trajectories and improving the policy. A major practical advantage is that Iterative PPO directly leverages stable, off-the-shelf single-turn RLHF tools, making it straightforward to implement. Our method occupies a middle ground between fully online and fully offline approaches, retaining the adaptability of online updates while gaining the stability benefits of offline training.
• 摘要：优化大型语言模型 (LLM) 以实现多轮对话结果仍然是一项重大挑战，尤其是在人工智能营销或通过消息平台促进交易等目标导向的环境中。困难源于稀疏、长期的奖励以及响应级别规划和代币级别生成之间的差异。在本技术说明中，我们建议将多轮 RL 问题正式简化为一系列单轮 RLHF 型问题。这是通过将学习的多圈 Q 函数设置为单圈问题的奖励模型来实现的。我们展示并证明了一个关键见解：使用标准代币级 PPO 解决单轮 RL 问题相当于多轮问题中的策略改进步骤。这种洞察自然会导致迭代 PPO，这是一种批量在线策略迭代算法，它在根据记录的对话轨迹拟合 Q 函数和改进策略之间交替进行。一个主要的实际优势是迭代 PPO 直接利用稳定的、现成的单圈 RLHF 工具，使其易于实施。我们的方法介于完全在线和完全离线方法之间，保留在线更新的适应性，同时获得离线训练的稳定性优势。

Title: CaFlow: Enhancing Long-Term Action Quality Assessment with Causal Counterfactual Flow

• Authors: Ruisheng Han, Kanglei Zhou, Shuang Chen, Amir Atapour-Abarghouei, Hubert P. H. Shum
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21653
• Pdf URL: https://arxiv.org/pdf/2511.21653
• Copy Paste: [[2511.21653]] CaFlow: Enhancing Long-Term Action Quality Assessment with Causal Counterfactual Flow(https://arxiv.org/abs/2511.21653)
• Keywords: quality assessment
• Abstract: Action Quality Assessment (AQA) predicts fine-grained execution scores from action videos and is widely applied in sports, rehabilitation, and skill evaluation. Long-term AQA, as in figure skating or rhythmic gymnastics, is especially challenging since it requires modeling extended temporal dynamics while remaining robust to contextual confounders. Existing approaches either depend on costly annotations or rely on unidirectional temporal modeling, making them vulnerable to spurious correlations and unstable long-term representations. To this end, we propose CaFlow, a unified framework that integrates counterfactual de-confounding with bidirectional time-conditioned flow. The Causal Counterfactual Regularization (CCR) module disentangles causal and confounding features in a self-supervised manner and enforces causal robustness through counterfactual interventions, while the BiT-Flow module models forward and backward dynamics with a cycle-consistency constraint to produce smoother and more coherent representations. Extensive experiments on multiple long-term AQA benchmarks demonstrate that CaFlow achieves state-of-the-art performance. Code is available at this https URL
• 摘要：动作质量评估（AQA）从动作视频中预测细粒度的执行分数，广泛应用于运动、康复和技能评估。长期 AQA，如花样滑冰或艺术体操，尤其具有挑战性，因为它需要对扩展的时间动态进行建模，同时保持对上下文混杂因素的鲁棒性。现有的方法要么依赖于昂贵的注释，要么依赖于单向时间建模，这使得它们容易受到虚假相关性和不稳定的长期表示的影响。为此，我们提出了 CaFlow，一个将反事实去混杂与双向时间条件流集成在一起的统一框架。因果反事实正则化 (CCR) 模块以自我监督的方式解开因果和混杂特征，并通过反事实干预增强因果鲁棒性，而 BiT-Flow 模块则通过循环一致性约束对前向和后向动态进行建模，以产生更平滑、更连贯的表示。对多个长期 AQA 基准的大量实验表明，CaFlow 实现了最先进的性能。代码可在此 https URL 获取

Title: Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following

• Authors: Tianyi Xiong, Yi Ge, Ming Li, Zuolong Zhang, Pranav Kulkarni, Kaishen Wang, Qi He, Zeying Zhu, Chenxi Liu, Ruibo Chen, Tong Zheng, Yanshuo Chen, Xiyao Wang, Renrui Zhang, Wenhu Chen, Heng Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21662
• Pdf URL: https://arxiv.org/pdf/2511.21662
• Copy Paste: [[2511.21662]] Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following(https://arxiv.org/abs/2511.21662)
• Keywords: generation
• Abstract: Large multimodal models (LMMs) are increasingly adopted as judges in multimodal evaluation systems due to their strong instruction following and consistency with human preferences. However, their ability to follow diverse, fine-grained evaluation criteria remains underexplored. We develop Multi-Crit, a benchmark for evaluating multimodal judges on their capacity to follow pluralistic criteria and produce reliable criterion-level judgments. Covering both open-ended generation and verifiable reasoning tasks, Multi-Crit is built through a rigorous data curation pipeline that gathers challenging response pairs with multi-criterion human annotations. It further introduces three novel metrics for systematically assessing pluralistic adherence, criterion-switching flexibility, and the ability to recognize criterion-level preference conflicts. Comprehensive analysis of 25 LMMs reveals that 1) proprietary models still struggle to maintain consistent adherence to pluralistic criteria--especially in open-ended evaluation; 2) open-source models lag further behind in flexibly following diverse criteria; and 3) critic fine-tuning with holistic judgment signals enhances visual grounding but fails to generalize to pluralistic criterion-level judgment. Additional analyses on reasoning fine-tuning, test-time scaling, and boundary consistency between open-source and proprietary models further probe the limits of current multimodal judges. As a pioneering study, Multi-Crit lays the foundation for building reliable and steerable multimodal AI evaluation.
• 摘要：大型多模态模型（LMM）由于其强大的指令遵循性和与人类偏好的一致性，越来越多地被采用作为多模态评估系统的判断者。然而，他们遵循多样化、细粒度评估标准的能力仍未得到充分探索。我们开发了 Multi-Crit，这是评估多模式法官遵循多元标准并产生可靠的标准级别判断的能力的基准。 Multi-Crit 涵盖开放式生成和可验证推理任务，是通过严格的数据管理管道构建的，该管道通过多标准人工注释收集具有挑战性的响应对。它还引入了三个新颖的指标，用于系统地评估多元依从性、标准转换灵活性以及识别标准级别偏好冲突的能力。对 25 个 LMM 的综合分析表明，1）专有模型仍然难以保持对多元标准的一致遵守——尤其是在开放式评估中； 2）开源模型在灵活遵循多样化标准方面进一步落后； 3）评论家对整体判断信号的微调增强了视觉基础，但未能推广到多元标准水平的判断。对推理微调、测试时间缩放以及开源和专有模型之间的边界一致性的额外分析进一步探讨了当前多模态判断的局限性。作为一项开创性研究，Multi-Crit 为构建可靠且可引导的多模式人工智能评估奠定了基础。

Title: Through the telecom lens: Are all training samples important?

• Authors: Shruti Bothe, Illyyne Saffar, Aurelie Boisbunon, Hasan Farooq, Julien Forgeat, Md Moin Uddin Chowdhury
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21668
• Pdf URL: https://arxiv.org/pdf/2511.21668
• Copy Paste: [[2511.21668]] Through the telecom lens: Are all training samples important?(https://arxiv.org/abs/2511.21668)
• Keywords: generation
• Abstract: The rise of AI in telecommunications, from optimizing Radio Access Networks to managing user experience, has sharply increased data volumes and training demands. Telecom data is often noisy, high-dimensional, costly to store, process, and label. Despite Ai's critical role, standard workflows still assume all training samples contribute equally. On the other hand, next generation systems require AI models that are accurate, efficient, and this http URL paper questions the assumptions of equal importance by focusing on applying and analyzing the roles of individual samples in telecom training and assessing whether the proposed model optimizes computation and energy use. we perform sample-level gradient analysis across epochs to identify patterns of influence and redundancy in model learning. Based on this, we propose a sample importance framework thats electively prioritizes impactful data and reduces computation without compromising accuracy. Experiments on three real-world telecom datasets show that our method [reserves performance while reducing data needs and computational overhead while advancing the goals of sustainable AI in telecommunications.
• 摘要：人工智能在电信领域的兴起，从优化无线接入网络到管理用户体验，大大增加了数据量和培训需求。电信数据通常是嘈杂的、高维度的、存储、处理和标记成本高昂的。尽管人工智能发挥着关键作用，但标准工作流程仍然假设所有训练样本的贡献相同。另一方面，下一代系统需要准确、高效的 AI 模型，这篇 http URL 论文通过重点应用和分析单个样本在电信训练中的作用，并评估所提出的模型是否优化了计算和能源使用，对同样重要的假设提出了质疑。我们跨时期进行样本级梯度分析，以识别模型学习中的影响和冗余模式。基于此，我们提出了一个示例重要性框架，该框架选择性地优先考虑有影响力的数据并减少计算而不影响准确性。对三个现实世界电信数据集的实验表明，我们的方法[保留性能，同时减少数据需求和计算开销，同时推进电信领域可持续人工智能的目标。

Title: DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving

• Authors: Fengze Yu, Leshu Li, Brad McDanel, Saiqian Zhang
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2511.21669
• Pdf URL: https://arxiv.org/pdf/2511.21669
• Copy Paste: [[2511.21669]] DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving(https://arxiv.org/abs/2511.21669)
• Keywords: generation
• Abstract: Large language model (LLM) inference often suffers from high decoding latency and limited scalability across heterogeneous edge-cloud environments. Existing speculative decoding (SD) techniques accelerate token generation but remain confined to single-node execution. We propose DSD, a distributed speculative decoding framework that extends SD to multi-device deployments through coordinated draft-target execution. Given the lack of prior work on simulating this paradigm, we first introduce DSD-Sim, a discrete-event simulator that captures network, batching, and scheduling dynamics. Building on insights from DSD-Sim, we further design an Adaptive Window Control (AWC) policy that dynamically adjusts speculation window size to optimize throughput. Experiments across diverse workloads show that DSD achieves up to 1.1x speedup and 9.7% higher throughput over existing SD baselines, enabling agile and scalable LLM serving across edge and cloud.
• 摘要：大型语言模型 (LLM) 推理通常会遇到高解码延迟和跨异构边缘云环境的可扩展性有限的问题。现有的推测解码（SD）技术可加速令牌生成，但仍仅限于单节点执行。我们提出了 DSD，一种分布式推测解码框架，通过协调草案目标执行将 SD 扩展到多设备部署。鉴于之前缺乏模拟此范例的工作，我们首先介绍 DSD-Sim，这是一种离散事件模拟器，可以捕获网络、批处理和调度动态。基于 DSD-Sim 的见解，我们进一步设计了自适应窗口控制 (AWC) 策略，可动态调整推测窗口大小以优化吞吐量。跨不同工作负载的实验表明，与现有 SD 基准相比，DSD 实现了高达 1.1 倍的加速和 9.7% 的吞吐量提高，从而实现了跨边缘和云的敏捷且可扩展的 LLM 服务。

Title: Canvas-to-Image: Compositional Image Generation with Multimodal Controls

• Authors: Yusuf Dalva, Guocheng Gordon Qian, Maya Goldenberg, Tsai-Shien Chen, Kfir Aberman, Sergey Tulyakov, Pinar Yanardag, Kuan-Chieh Jackson Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21691
• Pdf URL: https://arxiv.org/pdf/2511.21691
• Copy Paste: [[2511.21691]] Canvas-to-Image: Compositional Image Generation with Multimodal Controls(https://arxiv.org/abs/2511.21691)
• Keywords: generation
• Abstract: While modern diffusion models excel at generating high-quality and diverse images, they still struggle with high-fidelity compositional and multimodal control, particularly when users simultaneously specify text prompts, subject references, spatial arrangements, pose constraints, and layout annotations. We introduce Canvas-to-Image, a unified framework that consolidates these heterogeneous controls into a single canvas interface, enabling users to generate images that faithfully reflect their intent. Our key idea is to encode diverse control signals into a single composite canvas image that the model can directly interpret for integrated visual-spatial reasoning. We further curate a suite of multi-task datasets and propose a Multi-Task Canvas Training strategy that optimizes the diffusion model to jointly understand and integrate heterogeneous controls into text-to-image generation within a unified learning paradigm. This joint training enables Canvas-to-Image to reason across multiple control modalities rather than relying on task-specific heuristics, and it generalizes well to multi-control scenarios during inference. Extensive experiments show that Canvas-to-Image significantly outperforms state-of-the-art methods in identity preservation and control adherence across challenging benchmarks, including multi-person composition, pose-controlled composition, layout-constrained generation, and multi-control generation.
• 摘要：虽然现代扩散模型擅长生成高质量和多样化的图像，但它们仍然难以实现高保真构图和多模态控制，特别是当用户同时指定文本提示、主题参考、空间安排、姿势约束和布局注释时。我们引入了 Canvas-to-Image，这是一个统一的框架，它将这些异构控件整合到一个画布界面中，使用户能够生成忠实反映其意图的图像。我们的关键思想是将不同的控制信号编码到单个复合画布图像中，模型可以直接解释该图像以进行集成的视觉空间推理。我们进一步策划了一套多任务数据集，并提出了一种多任务画布训练策略，该策略优化扩散模型，以在统一的学习范式中共同理解异构控制并将其集成到文本到图像的生成中。这种联合训练使 Canvas-to-Image 能够跨多种控制模式进行推理，而不是依赖于特定于任务的启发式方法，并且它可以在推理过程中很好地推广到多控制场景。大量实验表明，在具有挑战性的基准（包括多人合成、姿势控制合成、布局约束生成和多控制生成）中，Canvas-to-Image 在身份保存和控制遵守方面显着优于最先进的方法。