2025-11-12

Title: Counterfactual Forecasting of Human Behavior using Generative AI and Causal Graphs

• Authors: Dharmateja Priyadarshi Uddandarao, Ravi Kiran Vadlamani
• Subjects: cs.LG, cs.CE, stat.ME
• Abstract URL: https://arxiv.org/abs/2511.07484
• Pdf URL: https://arxiv.org/pdf/2511.07484
• Copy Paste: [[2511.07484]] Counterfactual Forecasting of Human Behavior using Generative AI and Causal Graphs(https://arxiv.org/abs/2511.07484)
• Keywords: generative
• Abstract: This study presents a novel framework for counterfactual user behavior forecasting that combines structural causal models with transformer-based generative artificial intelligence. To model fictitious situations, the method creates causal graphs that map the connections between user interactions, adoption metrics, and product features. The framework generates realistic behavioral trajectories under counterfactual conditions by using generative models that are conditioned on causal variables. Tested on datasets from web interactions, mobile applications, and e-commerce, the methodology outperforms conventional forecasting and uplift modeling techniques. Product teams can effectively simulate and assess possible interventions prior to deployment thanks to the framework improved interpretability through causal path visualization.
• 摘要：这项研究提出了一种用于反事实用户行为预测的新颖框架，该框架将结构因果模型与基于变压器的生成人工智能相结合。为了对虚构的情况进行建模，该方法创建因果图来映射用户交互、采用指标和产品功能之间的联系。该框架通过使用以因果变量为条件的生成模型，在反事实条件下生成现实的行为轨迹。该方法在网络交互、移动应用程序和电子商务的数据集上进行了测试，其性能优于传统的预测和提升建模技术。由于框架通过因果路径可视化提高了可解释性，产品团队可以在部署之前有效地模拟和评估可能的干预措施。

Title: Provably Efficient Sample Complexity for Robust CMDP

• Authors: Sourav Ganguly, Arnob Ghosh
• Subjects: cs.LG, eess.SY, stat.ML
• Abstract URL: https://arxiv.org/abs/2511.07486
• Pdf URL: https://arxiv.org/pdf/2511.07486
• Copy Paste: [[2511.07486]] Provably Efficient Sample Complexity for Robust CMDP(https://arxiv.org/abs/2511.07486)
• Keywords: generative
• Abstract: We study the problem of learning policies that maximize cumulative reward while satisfying safety constraints, even when the real environment differs from a simulator or nominal model. We focus on robust constrained Markov decision processes (RCMDPs), where the agent must maximize reward while ensuring cumulative utility exceeds a threshold under the worst-case dynamics within an uncertainty set. While recent works have established finite-time iteration complexity guarantees for RCMDPs using policy optimization, their sample complexity guarantees remain largely unexplored. In this paper, we first show that Markovian policies may fail to be optimal even under rectangular uncertainty sets unlike the {\em unconstrained} robust MDP. To address this, we introduce an augmented state space that incorporates the remaining utility budget into the state representation. Building on this formulation, we propose a novel Robust constrained Value iteration (RCVI) algorithm with a sample complexity of $\mathcal{\tilde{O}}(|S||A|H^5/\epsilon^2)$ achieving at most $\epsilon$ violation using a generative model where $|S|$ and $|A|$ denote the sizes of the state and action spaces, respectively, and $H$ is the episode length. To the best of our knowledge, this is the {\em first sample complexity guarantee} for RCMDP. Empirical results further validate the effectiveness of our approach.
• 摘要：我们研究学习策略的问题，即使真实环境与模拟器或名义模型不同，也能在满足安全约束的同时最大化累积奖励。我们专注于鲁棒约束马尔可夫决策过程（RCMDP），其中代理必须最大化奖励，同时确保累积效用超过不确定性集内最坏情况动态下的阈值。虽然最近的工作已经使用策略优化为 RCMDP 建立了有限时间迭代复杂性保证，但它们的样本复杂性保证在很大程度上仍未得到探索。在本文中，我们首先表明，即使在矩形不确定性集下，马尔可夫策略也可能不是最优的，这与{\em不受约束}的鲁棒 MDP 不同。为了解决这个问题，我们引入了一个增强的状态空间，将剩余的公用事业预算纳入状态表示中。在此基础上，我们提出了一种新颖的鲁棒约束值迭代（RCVI）算法，其样本复杂度为 $\mathcal{\tilde{O}}(|S||A|H^5/\epsilon^2)$，使用生成模型最多实现 $\epsilon$ 违规，其中 $|S|$ 和 $|A|$ 分别表示状态和动作空间的大小，$H$ 是情节长度。据我们所知，这是 RCMDP 的{\em第一个样本复杂性保证}。实证结果进一步验证了我们方法的有效性。

Title: Laplacian Score Sharpening for Mitigating Hallucination in Diffusion Models

• Authors: Barath Chandran.C, Srinivas Anumasa, Dianbo Liu
• Subjects: cs.CV, cs.AI, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2511.07496
• Pdf URL: https://arxiv.org/pdf/2511.07496
• Copy Paste: [[2511.07496]] Laplacian Score Sharpening for Mitigating Hallucination in Diffusion Models(https://arxiv.org/abs/2511.07496)
• Keywords: generation
• Abstract: Diffusion models, though successful, are known to suffer from hallucinations that create incoherent or unrealistic samples. Recent works have attributed this to the phenomenon of mode interpolation and score smoothening, but they lack a method to prevent their generation during sampling. In this paper, we propose a post-hoc adjustment to the score function during inference that leverages the Laplacian (or sharpness) of the score to reduce mode interpolation hallucination in unconditional diffusion models across 1D, 2D, and high-dimensional image data. We derive an efficient Laplacian approximation for higher dimensions using a finite-difference variant of the Hutchinson trace estimator. We show that this correction significantly reduces the rate of hallucinated samples across toy 1D/2D distributions and a high- dimensional image dataset. Furthermore, our analysis explores the relationship between the Laplacian and uncertainty in the score.
• 摘要：扩散模型虽然成功，但众所周知会产生幻觉，从而产生不连贯或不切实际的样本。最近的工作将其归因于模式插值和分数平滑的现象，但他们缺乏一种方法来防止它们在采样过程中产生。在本文中，我们提出了在推理过程中对分数函数进行事后调整，利用分数的拉普拉斯（或清晰度）来减少一维、二维和高维图像数据的无条件扩散模型中的模式插值幻觉。我们使用 Hutchinson 迹估计器的有限差分变体推导出更高维度的有效拉普拉斯近似。我们表明，这种校正显着降低了玩具 1D/2D 分布和高维图像数据集的幻觉样本率。此外，我们的分析探讨了拉普拉斯算子与分数不确定性之间的关系。

Title: Toward the Frontiers of Reliable Diffusion Sampling via Adversarial Sinkhorn Attention Guidance

• Authors: Kwanyoung Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07499
• Pdf URL: https://arxiv.org/pdf/2511.07499
• Copy Paste: [[2511.07499]] Toward the Frontiers of Reliable Diffusion Sampling via Adversarial Sinkhorn Attention Guidance(https://arxiv.org/abs/2511.07499)
• Keywords: generative
• Abstract: Diffusion models have demonstrated strong generative performance when using guidance methods such as classifier-free guidance (CFG), which enhance output quality by modifying the sampling trajectory. These methods typically improve a target output by intentionally degrading another, often the unconditional output, using heuristic perturbation functions such as identity mixing or blurred conditions. However, these approaches lack a principled foundation and rely on manually designed distortions. In this work, we propose Adversarial Sinkhorn Attention Guidance (ASAG), a novel method that reinterprets attention scores in diffusion models through the lens of optimal transport and intentionally disrupt the transport cost via Sinkhorn algorithm. Instead of naively corrupting the attention mechanism, ASAG injects an adversarial cost within self-attention layers to reduce pixel-wise similarity between queries and keys. This deliberate degradation weakens misleading attention alignments and leads to improved conditional and unconditional sample quality. ASAG shows consistent improvements in text-to-image diffusion, and enhances controllability and fidelity in downstream applications such as IP-Adapter and ControlNet. The method is lightweight, plug-and-play, and improves reliability without requiring any model retraining.
• 摘要：当使用无分类器引导（CFG）等引导方法时，扩散模型表现出了强大的生成性能，它通过修改采样轨迹来提高输出质量。这些方法通常通过使用启发式扰动函数（例如身份混合或模糊条件）故意降低另一个目标输出（通常是无条件输出）来改进目标输出。然而，这些方法缺乏原则基础，并且依赖于手动设计的扭曲。在这项工作中，我们提出了对抗性 Sinkhorn 注意力指导（ASAG），这是一种新方法，它通过最佳传输的视角重新解释扩散模型中的注意力分数，并通过 Sinkhorn 算法有意破坏传输成本。 ASAG 不会天真地破坏注意力机制，而是在自注意力层中注入对抗成本，以减少查询和键之间的像素级相似性。这种故意的降级削弱了误导性的注意力排列，并提高了条件和无条件样本的质量。 ASAG 在文本到图像扩散方面表现出一致的改进，并增强了 IP-Adapter 和 ControlNet 等下游应用的可控性和保真度。该方法轻量、即插即用，无需任何模型重新训练即可提高可靠性。

Title: Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private

• Authors: Ruihan Wu, Erchi Wang, Zhiyuan Zhang, Yu-Xiang Wang
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2511.07637
• Pdf URL: https://arxiv.org/pdf/2511.07637
• Copy Paste: [[2511.07637]] Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private(https://arxiv.org/abs/2511.07637)
• Keywords: generation
• Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving documents from an external corpus at inference time. When this corpus contains sensitive information, however, unprotected RAG systems are at risk of leaking private information. Prior work has introduced differential privacy (DP) guarantees for RAG, but only in single-query settings, which fall short of realistic usage. In this paper, we study the more practical multi-query setting and propose two DP-RAG algorithms. The first, MURAG, leverages an individual privacy filter so that the accumulated privacy loss only depends on how frequently each document is retrieved rather than the total number of queries. The second, MURAG-ADA, further improves utility by privately releasing query-specific thresholds, enabling more precise selection of relevant documents. Our experiments across multiple LLMs and datasets demonstrate that the proposed methods scale to hundreds of queries within a practical DP budget ($\varepsilon\approx10$), while preserving meaningful utility.
• 摘要：检索增强生成 (RAG) 通过在推理时从外部语料库检索文档来增强大型语言模型 (LLM)。然而，当该语料库包含敏感信息时，未受保护的 RAG 系统就有泄露私人信息的风险。之前的工作已经为 RAG 引入了差分隐私 (DP) 保证，但仅限于单查询设置，这达不到实际用途。在本文中，我们研究了更实用的多查询设置并提出了两种DP-RAG算法。第一个，MURAG，利用单独的隐私过滤器，因此累积的隐私损失仅取决于每个文档的检索频率，而不是查询总数。第二个是 MURAG-ADA，通过私下发布特定于查询的阈值进一步提高实用性，从而能够更精确地选择相关文档。我们在多个 LLM 和数据集上进行的实验表明，所提出的方法可以在实际 DP 预算 ($\varepsilon\approx10$) 内扩展到数百个查询，同时保留有意义的效用。

Title: ZeroSim: Zero-Shot Analog Circuit Evaluation with Unified Transformer Embeddings

• Authors: Xiaomeng Yang, Jian Gao, Yanzhi Wang, Xuan Zhang
• Subjects: cs.LG, cs.AR
• Abstract URL: https://arxiv.org/abs/2511.07658
• Pdf URL: https://arxiv.org/pdf/2511.07658
• Copy Paste: [[2511.07658]] ZeroSim: Zero-Shot Analog Circuit Evaluation with Unified Transformer Embeddings(https://arxiv.org/abs/2511.07658)
• Keywords: generation
• Abstract: Although recent advancements in learning-based analog circuit design automation have tackled tasks such as topology generation, device sizing, and layout synthesis, efficient performance evaluation remains a major bottleneck. Traditional SPICE simulations are time-consuming, while existing machine learning methods often require topology-specific retraining or manual substructure segmentation for fine-tuning, hindering scalability and adaptability. In this work, we propose ZeroSim, a transformer-based performance modeling framework designed to achieve robust in-distribution generalization across trained topologies under novel parameter configurations and zero-shot generalization to unseen topologies without any fine-tuning. We apply three key enabling strategies: (1) a diverse training corpus of 3.6 million instances covering over 60 amplifier topologies, (2) unified topology embeddings leveraging global-aware tokens and hierarchical attention to robustly generalize to novel circuits, and (3) a topology-conditioned parameter mapping approach that maintains consistent structural representations independent of parameter variations. Our experimental results demonstrate that ZeroSim significantly outperforms baseline models such as multilayer perceptrons, graph neural networks and transformers, delivering accurate zero-shot predictions across different amplifier topologies. Additionally, when integrated into a reinforcement learning-based parameter optimization pipeline, ZeroSim achieves a remarkable speedup (13x) compared to conventional SPICE simulations, underscoring its practical value for a wide range of analog circuit design automation tasks.
• 摘要：尽管基于学习的模拟电路设计自动化的最新进展已经解决了拓扑生成、器件尺寸确定和布局综合等任务，但高效的性能评估仍然是主要瓶颈。传统的 SPICE 模拟非常耗时，而现有的机器学习方法通​​常需要特定于拓扑的重新训练或手动子结构分割来进行微调，从而阻碍了可扩展性和适应性。在这项工作中，我们提出了 ZeroSim，一种基于 Transformer 的性能建模框架，旨在在新颖的参数配置下实现跨训练拓扑的鲁棒分布内泛化，以及在无需任何微调的情况下对未见拓扑的零样本泛化。我们应用了三种关键的支持策略：(1) 包含 360 万个实例的多样化训练语料库，涵盖 60 多种放大器拓扑；(2) 统一拓扑嵌入，利用全局感知令牌和分层注意力来稳健地推广到新颖的电路；(3) 拓扑条件参数映射方法，可保持独立于参数变化的一致结构表示。我们的实验结果表明，ZeroSim 的性能显着优于多层感知器、图神经网络和变压器等基线模型，可在不同的放大器拓扑中提供准确的零样本预测。此外，当集成到基于强化学习的参数优化管道中时，与传统 SPICE 仿真相比，ZeroSim 实现了显着的加速（13 倍），凸显了其对于各种模拟电路设计自动化任务的实用价值。

Title: From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training

• Authors: Donglai Xu, Hongzheng Yang, Yuzhi Zhao, Pingping Zhang, Jinpeng Chen, Wenao Ma, Zhijian Hou, Mengyang Wu, Xiaolei Li, Senkang Hu, Ziyi Guan, Jason Chun Lok Li, Lai Man Po
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07738
• Pdf URL: https://arxiv.org/pdf/2511.07738
• Copy Paste: [[2511.07738]] From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training(https://arxiv.org/abs/2511.07738)
• Keywords: generation
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) for Multimodal Large Language Models (MLLMs) is highly dependent on high-quality labeled data, which is often scarce and prone to substantial annotation noise in real-world scenarios. Existing unsupervised RLVR methods, including pure entropy minimization, can overfit to incorrect labels and limit the crucial reward ranking signal for Group-Relative Policy Optimization (GRPO). To address these challenges and enhance noise tolerance, we propose a novel two-stage, token-level entropy optimization method for RLVR. This approach dynamically guides the model from exploration to exploitation during training. In the initial exploration phase, token-level entropy maximization promotes diverse and stochastic output generation, serving as a strong regularizer that prevents premature convergence to noisy labels and ensures sufficient intra-group variation, which enables more reliable reward gradient estimation in GRPO. As training progresses, the method transitions into the exploitation phase, where token-level entropy minimization encourages the model to produce confident and deterministic outputs, thereby consolidating acquired knowledge and refining prediction accuracy. Empirically, across three MLLM backbones - Qwen2-VL-2B, Qwen2-VL-7B, and Qwen2.5-VL-3B - spanning diverse noise settings and multiple tasks, our phased strategy consistently outperforms prior approaches by unifying and enhancing external, internal, and entropy-based methods, delivering robust and superior performance across the board.
• 摘要：多模态大型语言模型 (MLLM) 的可验证奖励强化学习 (RLVR) 高度依赖于高质量的标记数据，而这些数据通常稀缺，并且在现实场景中容易产生大量注释噪声。现有的无监督 RLVR 方法（包括纯熵最小化）可能会过度拟合不正确的标签，并限制组相对策略优化 (GRPO) 的关键奖励排名信号。为了解决这些挑战并增强噪声容忍度，我们提出了一种新颖的 RLVR 两阶段令牌级熵优化方法。这种方法在训练期间动态引导模型从探索到利用。在初始探索阶段，令牌级熵最大化促进了多样化和随机的输出生成，作为强大的正则化器，可以防止过早收敛到噪声标签并确保足够的组内变化，从而在 GRPO 中实现更可靠的奖励梯度估计。随着训练的进行，该方法过渡到开发阶段，其中令牌级熵最小化鼓励模型产生可信且确定性的输出，从而巩固所获得的知识并提高预测准确性。根据经验，在三个 MLLM 主干（Qwen2-VL-2B、Qwen2-VL-7B 和 Qwen2.5-VL-3B）中，跨越不同的噪声设置和多个任务，我们的分阶段策略通过统一和增强外部、内部和基于熵的方法，始终优于先前的方法，全面提供稳健且卓越的性能。

Title: VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semantics

• Authors: Daniel Cher, Brian Wei, Srikumar Sastry, Nathan Jacobs
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07744
• Pdf URL: https://arxiv.org/pdf/2511.07744
• Copy Paste: [[2511.07744]] VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semantics(https://arxiv.org/abs/2511.07744)
• Keywords: generation
• Abstract: We introduce VectorSynth, a diffusion-based framework for pixel-accurate satellite image synthesis conditioned on polygonal geographic annotations with semantic attributes. Unlike prior text- or layout-conditioned models, VectorSynth learns dense cross-modal correspondences that align imagery and semantic vector geometry, enabling fine-grained, spatially grounded edits. A vision language alignment module produces pixel-level embeddings from polygon semantics; these embeddings guide a conditional image generation framework to respect both spatial extents and semantic cues. VectorSynth supports interactive workflows that mix language prompts with geometry-aware conditioning, allowing rapid what-if simulations, spatial edits, and map-informed content generation. For training and evaluation, we assemble a collection of satellite scenes paired with pixel-registered polygon annotations spanning diverse urban scenes with both built and natural features. We observe strong improvements over prior methods in semantic fidelity and structural realism, and show that our trained vision language model demonstrates fine-grained spatial grounding. The code and data are available at this https URL.
• 摘要：我们介绍了 VectorSynth，这是一种基于扩散的框架，用于以具有语义属性的多边形地理注释为条件进行像素精确的卫星图像合成。与之前的文本或布局条件模型不同，VectorSynth 可以学习密集的跨模式对应关系，从而对齐图像和语义向量几何形状，从而实现细粒度、基于空间的编辑。视觉语言对齐模块根据多边形语义生成像素级嵌入；这些嵌入指导条件图像生成框架尊重空间范围和语义线索。 VectorSynth 支持将语言提示与几何感知调节相结合的交互式工作流程，从而允许快速假设模拟、空间编辑和地图通知内容生成。为了进行训练和评估，我们组装了一系列卫星场景，并配有像素配准的多边形注释，涵盖具有建筑和自然特征的不同城市场景。我们观察到在语义保真度和结构真实性方面比先前的方法有了显着的改进，并表明我们训练的视觉语言模型展示了细粒度的空间基础。代码和数据可从此 https URL 获取。

Title: Beyond Randomness: Understand the Order of the Noise in Diffusion

• Authors: Song Yan, Min Li, Bi Xinliang, Jian Yang, Yusen Zhang, Guanye Xiong, Yunwei Lan, Tao Zhang, Wei Zhai, Zheng-Jun Zha
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07756
• Pdf URL: https://arxiv.org/pdf/2511.07756
• Copy Paste: [[2511.07756]] Beyond Randomness: Understand the Order of the Noise in Diffusion(https://arxiv.org/abs/2511.07756)
• Keywords: generation
• Abstract: In text-driven content generation (T2C) diffusion model, semantic of generated content is mostly attributed to the process of text embedding and attention mechanism interaction. The initial noise of the generation process is typically characterized as a random element that contributes to the diversity of the generated content. Contrary to this view, this paper reveals that beneath the random surface of noise lies strong analyzable patterns. Specifically, this paper first conducts a comprehensive analysis of the impact of random noise on the model's generation. We found that noise not only contains rich semantic information, but also allows for the erasure of unwanted semantics from it in an extremely simple way based on information theory, and using the equivalence between the generation process of diffusion model and semantic injection to inject semantics into the cleaned noise. Then, we mathematically decipher these observations and propose a simple but efficient training-free and universal two-step "Semantic Erasure-Injection" process to modulate the initial noise in T2C diffusion model. Experimental results demonstrate that our method is consistently effective across various T2C models based on both DiT and UNet architectures and presents a novel perspective for optimizing the generation of diffusion model, providing a universal tool for consistent generation.
• 摘要：在文本驱动的内容生成（T2C）扩散模型中，生成内容的语义主要归因于文本嵌入和注意机制交互的过程。生成过程的初始噪声通常被表征为有助于生成内容的多样性的随机元素。与这种观点相反，本文揭示了在噪声的随机表面之下隐藏着强大的可分析模式。具体来说，本文首先对随机噪声对模型生成的影响进行了全面分析。我们发现噪声不仅包含丰富的语义信息，而且还可以基于信息论以极其简单的方式从噪声中擦除不需要的语义，并利用扩散模型的生成过程和语义注入之间的等价性将语义注入到清理后的噪声中。然后，我们从数学上破译这些观察结果，并提出一种简单但有效的免训练且通用的两步“语义擦除注入”过程来调制 T2C 扩散模型中的初始噪声。实验结果表明，我们的方法在基于 DiT 和 UNet 架构的各种 T2C 模型中始终有效，并为优化扩散模型的生成提供了新颖的视角，为一致生成提供了通用工具。

Title: Physical Consistency of Aurora's Encoder: A Quantitative Study

• Authors: Benjamin Richards, Pushpa Kumar Balan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07787
• Pdf URL: https://arxiv.org/pdf/2511.07787
• Copy Paste: [[2511.07787]] Physical Consistency of Aurora's Encoder: A Quantitative Study(https://arxiv.org/abs/2511.07787)
• Keywords: generation
• Abstract: The high accuracy of large-scale weather forecasting models like Aurora is often accompanied by a lack of transparency, as their internal representations remain largely opaque. This "black box" nature hinders their adoption in high-stakes operational settings. In this work, we probe the physical consistency of Aurora's encoder by investigating whether its latent representations align with known physical and meteorological concepts. Using a large-scale dataset of embeddings, we train linear classifiers to identify three distinct concepts: the fundamental land-sea boundary, high-impact extreme temperature events, and atmospheric instability. Our findings provide quantitative evidence that Aurora learns physically consistent features, while also highlighting its limitations in capturing the rarest events. This work underscores the critical need for interpretability methods to validate and build trust in the next generation of Al-driven weather models.
• 摘要：像极光这样的大规模天气预报模型的高精度往往伴随着缺乏透明度，因为它们的内部表示在很大程度上仍然是不透明的。这种“黑匣子”性质阻碍了它们在高风险操作环境中的采用。在这项工作中，我们通过调查 Aurora 编码器的潜在表示是否与已知的物理和气象概念相符来探讨其物理一致性。使用大规模嵌入数据集，我们训练线性分类器来识别三个不同的概念：基本的陆地-海洋边界、高影响的极端温度事件和大气不稳定。我们的研究结果提供了定量证据，证明奥罗拉（Aurora）学习了物理上一致的特征，同时也强调了它在捕捉最罕见事件方面的局限性。这项工作强调了对可解释性方法的迫切需求，以验证和建立对下一代人工智能驱动的天气模型的信任。

Title: PC-Diffusion: Aligning Diffusion Models with Human Preferences via Preference Classifier

• Authors: Shaomeng Wang, He Wang, Xiaolu Wei, Longquan Dai, Jinhui Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07806
• Pdf URL: https://arxiv.org/pdf/2511.07806
• Copy Paste: [[2511.07806]] PC-Diffusion: Aligning Diffusion Models with Human Preferences via Preference Classifier(https://arxiv.org/abs/2511.07806)
• Keywords: generation, generative
• Abstract: Diffusion models have achieved remarkable success in conditional image generation, yet their outputs often remain misaligned with human preferences. To address this, recent work has applied Direct Preference Optimization (DPO) to diffusion models, yielding significant improvements.~However, DPO-like methods exhibit two key limitations: 1) High computational cost,due to the entire model fine-tuning; 2) Sensitivity to reference model quality}, due to its tendency to introduce instability and bias. To overcome these limitations, we propose a novel framework for human preference alignment in diffusion models (PC-Diffusion), using a lightweight, trainable Preference Classifier that directly models the relative preference between samples. By restricting preference learning to this classifier, PC-Diffusion decouples preference alignment from the generative model, eliminating the need for entire model fine-tuning and reference model reliance.~We further provide theoretical guarantees for PC-Diffusion:1) PC-Diffusion ensures that the preference-guided distributions are consistently propagated across timesteps. 2)The training objective of the preference classifier is equivalent to DPO, but does not require a reference model.3) The proposed preference-guided correction can progressively steer generation toward preference-aligned regions.~Empirical results show that PC-Diffusion achieves comparable preference consistency to DPO while significantly reducing training costs and enabling efficient and stable preference-guided generation.
• 摘要：扩散模型在条件图像生成方面取得了显着的成功，但它们的输出常常与人类的偏好不一致。为了解决这个问题，最近的工作将直接偏好优化（DPO）应用于扩散模型，取得了显着的改进。~然而，类似 DPO 的方法表现出两个关键的局限性：1）由于整个模型的微调，计算成本较高； 2) 对参考模型质量的敏感性}，因为它容易引入不稳定和偏差。为了克服这些限制，我们提出了一种新的扩散模型中的人类偏好对齐框架（PC-Diffusion），使用轻量级、可训练的偏好分类器来直接模拟样本之间的相对偏好。通过将偏好学习限制在该分类器上，PC-Diffusion 将偏好对齐与生成模型解耦，从而消除了整个模型微调和参考模型依赖的需要。〜我们进一步为 PC-Diffusion 提供理论保证：1）PC-Diffusion 确保偏好引导的分布在时间步上一致传播。 2）偏好分类器的训练目标与DPO等效，但不需要参考模型。3）所提出的偏好引导校正可以逐步将生成引导到偏好对齐区域。~经验结果表明，PC-Diffusion实现了与DPO相当的偏好一致性，同时显着降低了训练成本，并实现了高效稳定的偏好引导生成。

Title: Revisiting MLLM Based Image Quality Assessment: Errors and Remedy

• Authors: Zhenchen Tang, Songlin Yang, Bo Peng, Zichuan Wang, Jing Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07812
• Pdf URL: https://arxiv.org/pdf/2511.07812
• Copy Paste: [[2511.07812]] Revisiting MLLM Based Image Quality Assessment: Errors and Remedy(https://arxiv.org/abs/2511.07812)
• Keywords: quality assessment
• Abstract: The rapid progress of multi-modal large language models (MLLMs) has boosted the task of image quality assessment (IQA). However, a key challenge arises from the inherent mismatch between the discrete token outputs of MLLMs and the continuous nature of quality scores required by IQA tasks. This discrepancy significantly hinders the performance of MLLM-based IQA methods. Previous approaches that convert discrete token predictions into continuous scores often suffer from conversion errors. Moreover, the semantic confusion introduced by level tokens (e.g., ``good'') further constrains the performance of MLLMs on IQA tasks and degrades their original capabilities for related tasks. To tackle these problems, we provide a theoretical analysis of the errors inherent in previous approaches and, motivated by this analysis, propose a simple yet effective framework, Q-Scorer. This framework incorporates a lightweight regression module and IQA-specific score tokens into the MLLM pipeline. Extensive experiments demonstrate that Q-Scorer achieves state-of-the-art performance across multiple IQA benchmarks, generalizes well to mixed datasets, and further improves when combined with other methods.
• 摘要：多模态大语言模型（MLLM）的快速进展推动了图像质量评估（IQA）的任务。然而，一个关键的挑战来自于 MLLM 的离散令牌输出与 IQA 任务所需的质量分数的连续性之间固有的不匹配。这种差异极大地阻碍了基于 MLLM 的 IQA 方法的性能。以前将离散标记预测转换为连续分数的方法经常会遇到转换错误。此外，级别标记（例如“好”）引入的语义混乱进一步限制了 MLLM 在 IQA 任务上的性能，并降低了它们执行相关任务的原始能力。为了解决这些问题，我们对先前方法中固有的错误进行了理论分析，并受此分析的启发，提出了一个简单而有效的框架：Q-Scorer。该框架将轻量级回归模块和 IQA 特定的分数标记合并到 MLLM 管道中。大量实验表明，Q-Scorer 在多个 IQA 基准测试中实现了最先进的性能，可以很好地推广到混合数据集，并在与其他方法结合时进一步改进。

Title: MURPHY: Multi-Turn GRPO for Self Correcting Code Generation

• Authors: Chanakya Ekbote, Vijay Lingam, Behrooz Omidvar-Tehrani, Jun Huan, Sujay Sanghavi, Anoop Deoras, Stefano Soatto
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07833
• Pdf URL: https://arxiv.org/pdf/2511.07833
• Copy Paste: [[2511.07833]] MURPHY: Multi-Turn GRPO for Self Correcting Code Generation(https://arxiv.org/abs/2511.07833)
• Keywords: generation
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful framework for enhancing the reasoning capabilities of large language models (LLMs). However, existing approaches such as Group Relative Policy Optimization (GRPO) and its variants, while effective on reasoning benchmarks, struggle with agentic tasks that require iterative decision-making. We introduce Murphy, a multi-turn reflective optimization framework that extends GRPO by incorporating iterative self-correction during training. By leveraging both quantitative and qualitative execution feedback, Murphy enables models to progressively refine their reasoning across multiple turns. Evaluations on code generation benchmarks with model families such as Qwen and OLMo show that Murphy consistently improves performance, achieving up to a 8% relative gain in pass@1 over GRPO, on similar compute budgets.
• 摘要：具有可验证奖励的强化学习（RLVR）已成为增强大型语言模型（LLM）推理能力的强大框架。然而，组相对策略优化 (GRPO) 及其变体等现有方法虽然在推理基准上有效，但在处理需要迭代决策的代理任务时却遇到了困难。我们引入了 Murphy，一种多轮反射优化框架，它通过在训练期间合并迭代自我校正来扩展 GRPO。通过利用定量和定性执行反馈，墨菲使模型能够在多个回合中逐步完善其推理。对 Qwen 和 OLMo 等模型系列的代码生成基准进行的评估表明，Murphy 持续提高性能，在相似的计算预算下，pass@1 相对于 GRPO 实现了高达 8% 的相对增益。

Title: Visual Bridge: Universal Visual Perception Representations Generating

• Authors: Yilin Gao, Shuguang Dou, Junzhou Li, Zhiheng Yu, Yin Li, Dongsheng Jiang, Shugong Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07877
• Pdf URL: https://arxiv.org/pdf/2511.07877
• Copy Paste: [[2511.07877]] Visual Bridge: Universal Visual Perception Representations Generating(https://arxiv.org/abs/2511.07877)
• Keywords: generation
• Abstract: Recent advances in diffusion models have achieved remarkable success in isolated computer vision tasks such as text-to-image generation, depth estimation, and optical flow. However, these models are often restricted by a ``single-task-single-model'' paradigm, severely limiting their generalizability and scalability in multi-task scenarios. Motivated by the cross-domain generalization ability of large language models, we propose a universal visual perception framework based on flow matching that can generate diverse visual representations across multiple tasks. Our approach formulates the process as a universal flow-matching problem from image patch tokens to task-specific representations rather than an independent generation or regression problem. By leveraging a strong self-supervised foundation model as the anchor and introducing a multi-scale, circular task embedding mechanism, our method learns a universal velocity field to bridge the gap between heterogeneous tasks, supporting efficient and flexible representation transfer. Extensive experiments on classification, detection, segmentation, depth estimation, and image-text retrieval demonstrate that our model achieves competitive performance in both zero-shot and fine-tuned settings, outperforming prior generalist and several specialist models. Ablation studies further validate the robustness, scalability, and generalization of our framework. Our work marks a significant step towards general-purpose visual perception, providing a solid foundation for future research in universal vision modeling.
• 摘要：扩散模型的最新进展在文本到图像生成、深度估计和光流等孤立的计算机视觉任务中取得了显着的成功。然而，这些模型通常受到“单任务单模型”范式的限制，严重限制了它们在多任务场景中的通用性和可扩展性。受大型语言模型跨领域泛化能力的启发，我们提出了一种基于流匹配的通用视觉感知框架，可以跨多个任务生成不同的视觉表示。我们的方法将该过程表述为从图像补丁标记到特定于任务的表示的通用流程匹配问题，而不是独立的生成或回归问题。通过利用强大的自监督基础模型作为锚点并引入多尺度、循环任务嵌入机制，我们的方法学习通用速度场来弥合异构任务之间的差距，支持高效灵活的表示传输。关于分类、检测、分割、深度估计和图像文本检索的大量实验表明，我们的模型在零样本和微调设置中都实现了有竞争力的性能，优于先前的通才模型和几个专业模型。消融研究进一步验证了我们框架的稳健性、可扩展性和泛化性。我们的工作标志着朝着通用视觉感知迈出了重要一步，为通用视觉建模的未来研究奠定了坚实的基础。

Title: Generating Sketches in a Hierarchical Auto-Regressive Process for Flexible Sketch Drawing Manipulation at Stroke-Level

• Authors: Sicong Zang, Shuhui Gao, Zhijun Fang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07889
• Pdf URL: https://arxiv.org/pdf/2511.07889
• Copy Paste: [[2511.07889]] Generating Sketches in a Hierarchical Auto-Regressive Process for Flexible Sketch Drawing Manipulation at Stroke-Level(https://arxiv.org/abs/2511.07889)
• Keywords: generation
• Abstract: Generating sketches with specific patterns as expected, i.e., manipulating sketches in a controllable way, is a popular task. Recent studies control sketch features at stroke-level by editing values of stroke embeddings as conditions. However, in order to provide generator a global view about what a sketch is going to be drawn, all these edited conditions should be collected and fed into generator simultaneously before generation starts, i.e., no further manipulation is allowed during sketch generating process. In order to realize sketch drawing manipulation more flexibly, we propose a hierarchical auto-regressive sketch generating process. Instead of generating an entire sketch at once, each stroke in a sketch is generated in a three-staged hierarchy: 1) predicting a stroke embedding to represent which stroke is going to be drawn, and 2) anchoring the predicted stroke on the canvas, and 3) translating the embedding to a sequence of drawing actions to form the full sketch. Moreover, the stroke prediction, anchoring and translation are proceeded auto-regressively, i.e., both the recently generated strokes and their positions are considered to predict the current one, guiding model to produce an appropriate stroke at a suitable position to benefit the full sketch generation. It is flexible to manipulate stroke-level sketch drawing at any time during generation by adjusting the exposed editable stroke embeddings.
• 摘要：按照预期生成具有特定图案的草图，即以可控的方式操作草图，是一项流行的任务。最近的研究通过编辑笔划嵌入的值作为条件来控制笔划级别的草图特征。然而，为了向生成器提供有关将绘制草图的全局视图，应在生成开始之前收集所有这些编辑条件并同时将其输入生成器，即在草图生成过程中不允许进一步操作。为了更灵活地实现草图绘制操作，我们提出了一种分层自回归草图生成过程。草图中的每个笔划不是立即生成整个草图，而是按三阶段层次结构生成：1) 预测笔划嵌入以表示将要绘制哪个笔划，2) 将预测的笔划锚定在画布上，3) 将嵌入转换为一系列绘图操作以形成完整的草图。此外，笔画预测、锚定和平移是自回归进行的，即，最近生成的笔画及其位置都被考虑来预测当前的笔画，引导模型在合适的位置产生适当的笔画，以有利于完整的草图生成。通过调整公开的可编辑笔划嵌入，可以在生成过程中随时灵活地操作笔划级草图。

Title: Rectified Noise: A Generative Model Using Positive-incentive Noise

• Authors: Zhenyu Gu, Yanchen Xu, Sida Huang, Yubin Guo, Hongyuan Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07911
• Pdf URL: https://arxiv.org/pdf/2511.07911
• Copy Paste: [[2511.07911]] Rectified Noise: A Generative Model Using Positive-incentive Noise(https://arxiv.org/abs/2511.07911)
• Keywords: generative
• Abstract: Rectified Flow (RF) has been widely used as an effective generative model. Although RF is primarily based on probability flow Ordinary Differential Equations (ODE), recent studies have shown that injecting noise through reverse-time Stochastic Differential Equations (SDE) for sampling can achieve superior generative performance. Inspired by Positive-incentive Noise ($\pi$-noise), we propose an innovative generative algorithm to train $\pi$-noise generators, namely Rectified Noise ($\Delta$RN), which improves the generative performance by injecting $\pi$-noise into the velocity field of pre-trained RF models. After introducing the Rectified Noise pipeline, pre-trained RF models can be efficiently transformed into $\pi$-noise generators. We validate Rectified Noise by conducting extensive experiments across various model architectures on different datasets. Notably, we find that: (1) RF models using Rectified Noise reduce FID from \textbf{10.16 to 9.05} on ImageNet-1k. (2) The models of $\pi$-noise generators achieve improved performance with only \textbf{0.39\%} additional training parameters.
• 摘要：整流流（RF）作为一种有效的生成模型已被广泛使用。虽然 RF 主要基于概率流常微分方程 (ODE)，但最近的研究表明，通过反时随机微分方程 (SDE) 注入噪声进行采样可以实现卓越的生成性能。受正激励噪声（$\pi$-noise）的启发，我们提出了一种创新的生成算法来训练$\pi$-噪声生成器，即整流噪声（$\Delta$RN），它通过将$\pi$-噪声注入预训练的射频模型的速度场来提高生成性能。引入整流噪声管道后，预训练的射频模型可以有效地转换为 $\pi$-噪声发生器。我们通过在不同数据集上的各种模型架构上进行广泛的实验来验证修正噪声。值得注意的是，我们发现：（1）使用整流噪声的 RF 模型将 ImageNet-1k 上的 FID 从 \textbf{10.16 减少到 9.05}。 (2) $\pi$-噪声生成器的模型仅通过 \textbf{0.39\%} 额外的训练参数就获得了改进的性能。

Title: Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformers

• Authors: Sida Huang, Siqi Huang, Ping Luo, Hongyuan Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07934
• Pdf URL: https://arxiv.org/pdf/2511.07934
• Copy Paste: [[2511.07934]] Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformers(https://arxiv.org/abs/2511.07934)
• Keywords: generation, generative
• Abstract: With the development of diffusion models, enhancing spatial controllability in text-to-image generation has become a vital challenge. As a representative task for addressing this challenge, layout-to-image generation aims to generate images that are spatially consistent with the given layout condition. Existing layout-to-image methods typically introduce the layout condition by integrating adapter modules into the base generative model. However, the generated images often exhibit low visual quality and stylistic inconsistency with the base model, indicating a loss of pretrained knowledge. To alleviate this issue, we construct the Layout Synthesis (LaySyn) dataset, which leverages images synthesized by the base model itself to mitigate the distribution shift from the pretraining data. Moreover, we propose the Layout Control (Laytrol) Network, in which parameters are inherited from MM-DiT to preserve the pretrained knowledge of the base model. To effectively activate the copied parameters and avoid disturbance from unstable control conditions, we adopt a dedicated initialization scheme for Laytrol. In this scheme, the layout encoder is initialized as a pure text encoder to ensure that its output tokens remain within the data domain of MM-DiT. Meanwhile, the outputs of the layout control network are initialized to zero. In addition, we apply Object-level Rotary Position Embedding to the layout tokens to provide coarse positional information. Qualitative and quantitative experiments demonstrate the effectiveness of our method.
• 摘要：随着扩散模型的发展，增强文本到图像生成的空间可控性已成为一个重大挑战。作为解决这一挑战的代表性任务，布局到图像生成旨在生成在空间上与给定布局条件一致的图像。现有的布局到图像方法通常通过将适配器模块集成到基本生成模型中来引入布局条件。然而，生成的图像通常表现出较低的视觉质量并且与基础模型的风格不一致，这表明预训练知识的损失。为了缓解这个问题，我们构建了布局合成（LaySyn）数据集，它利用基础模型本身合成的图像来减轻预训练数据的分布变化。此外，我们提出了布局控制（Laytrol）网络，其中参数继承自 MM-DiT 以保留基础模型的预训练知识。为了有效激活复制的参数并避免不稳定控制条件的干扰，我们采用了Laytrol专用的初始化方案。在该方案中，布局编码器被初始化为纯文本编码器，以确保其输出标记保留在MM-DiT的数据域内。同时，布局控制网络的输出被初始化为零。此外，我们将对象级旋转位置嵌入应用于布局标记以提供粗略的位置信息。定性和定量实验证明了我们方法的有效性。

Title: Is It Truly Necessary to Process and Fit Minutes-Long Reference Videos for Personalized Talking Face Generation?

• Authors: Rui-Qing Sun, Ang Li, Zhijing Wu, Tian Lan, Qianyu Lu, Xingshan Yao, Chen Xu, Xian-Ling Mao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07940
• Pdf URL: https://arxiv.org/pdf/2511.07940
• Copy Paste: [[2511.07940]] Is It Truly Necessary to Process and Fit Minutes-Long Reference Videos for Personalized Talking Face Generation?(https://arxiv.org/abs/2511.07940)
• Keywords: generation
• Abstract: Talking Face Generation (TFG) aims to produce realistic and dynamic talking portraits, with broad applications in fields such as digital education, film and television production, e-commerce live streaming, and other related areas. Currently, TFG methods based on Neural Radiated Field (NeRF) or 3D Gaussian sputtering (3DGS) are received widespread attention. They learn and store personalized features from reference videos of each target individual to generate realistic speaking videos. To ensure models can capture sufficient 3D information and successfully learns the lip-audio mapping, previous studies usually require meticulous processing and fitting several minutes of reference video, which always takes hours. The computational burden of processing and fitting long reference videos severely limits the practical application value of these this http URL, is it really necessary to fit such minutes of reference video? Our exploratory case studies show that using some informative reference video segments of just a few seconds can achieve performance comparable to or even better than the full reference video. This indicates that video informative quality is much more important than its length. Inspired by this observation, we propose the ISExplore (short for Informative Segment Explore), a simple-yet-effective segment selection strategy that automatically identifies the informative 5-second reference video segment based on three key data quality dimensions: audio feature diversity, lip movement amplitude, and number of camera views. Extensive experiments demonstrate that our approach increases data processing and training speed by more than 5x for NeRF and 3DGS methods, while maintaining high-fidelity output. Project resources are available at xx.
• 摘要：Talking Face Generation（TFG）旨在生成逼真、动态的说话肖像，在数字教育、影视制作、电商直播等相关领域有着广泛的应用。目前，基于神经辐射场（NeRF）或3D高斯溅射（3DGS）的TFG方法受到广泛关注。他们从每个目标个体的参考视频中学习并存储个性化特征，以生成逼真的演讲视频。为了确保模型能够捕获足够的 3D 信息并成功学习唇音映射，以前的研究通常需要仔细处理和拟合几分钟的参考视频，而这通常需要几个小时。处理和拟合长参考视频的计算负担严重限制了这些http URL的实际应用价值，真的有必要拟合这么分钟的参考视频吗？我们的探索性案例研究表明，使用一些仅几秒钟的信息丰富的参考视频片段可以获得与完整参考视频相当甚至更好的性能。这表明视频信息质量比其长度重要得多。受这一观察的启发，我们提出了 ISExplore（Informative Segment Explore 的缩写），这是一种简单而有效的片段选择策略，可根据三个关键数据质量维度自动识别信息丰富的 5 秒参考视频片段：音频特征多样性、嘴唇运动幅度和摄像机视图数量。大量实验表明，我们的方法将 NeRF 和 3DGS 方法的数据处理和训练速度提高了 5 倍以上，同时保持高保真输出。项目资源可在 xx 处获得。

Title: Burst Image Quality Assessment: A New Benchmark and Unified Framework for Multiple Downstream Tasks

• Authors: Xiaoye Liang, Lai Jiang, Minglang Qiao, Yichen Guo, Yue Zhang, Xin Deng, Shengxi Li, Yufan Liu, Mai Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07958
• Pdf URL: https://arxiv.org/pdf/2511.07958
• Copy Paste: [[2511.07958]] Burst Image Quality Assessment: A New Benchmark and Unified Framework for Multiple Downstream Tasks(https://arxiv.org/abs/2511.07958)
• Keywords: super-resolution, generation, quality assessment
• Abstract: In recent years, the development of burst imaging technology has improved the capture and processing capabilities of visual data, enabling a wide range of applications. However, the redundancy in burst images leads to the increased storage and transmission demands, as well as reduced efficiency of downstream tasks. To address this, we propose a new task of Burst Image Quality Assessment (BuIQA), to evaluate the task-driven quality of each frame within a burst sequence, providing reasonable cues for burst image selection. Specifically, we establish the first benchmark dataset for BuIQA, consisting of $7,346$ burst sequences with $45,827$ images and $191,572$ annotated quality scores for multiple downstream scenarios. Inspired by the data analysis, a unified BuIQA framework is proposed to achieve an efficient adaption for BuIQA under diverse downstream scenarios. Specifically, a task-driven prompt generation network is developed with heterogeneous knowledge distillation, to learn the priors of the downstream task. Then, the task-aware quality assessment network is introduced to assess the burst image quality based on the task prompt. Extensive experiments across 10 downstream scenarios demonstrate the impressive BuIQA performance of the proposed approach, outperforming the state-of-the-art. Furthermore, it can achieve $0.33$ dB PSNR improvement in the downstream tasks of denoising and super-resolution, by applying our approach to select the high-quality burst frames.
• 摘要：近年来，突发成像技术的发展提高了视觉数据的捕获和处理能力，实现了广泛的应用。然而，突发图像的冗余导致存储和传输需求的增加，以及下游任务效率的降低。为了解决这个问题，我们提出了突发图像质量评估（BuIQA）的新任务，以评估突发序列中每帧的任务驱动质量，为突发图像选择提供合理的线索。具体来说，我们为 BuIQA 建立了第一个基准数据集，其中包含 7,346 美元的突发序列、45,827 美元的图像和 191,572 美元的多个下游场景的注释质量分数。受数据分析的启发，提出了统一的BuIQA框架，以实现BuIQA在不同下游场景下的高效适应。具体来说，通过异构知识蒸馏开发任务驱动的提示生成网络，以学习下游任务的先验。然后，引入任务感知质量评估网络，根据任务提示评估突发图像质量。跨 10 个下游场景的广泛实验证明了所提出方法令人印象深刻的 BuIQA 性能，优于最先进的方法。此外，通过应用我们的方法选择高质量突发帧，它可以在去噪和超分辨率的下游任务中实现 $0.33$ dB PSNR 改善。

Title: Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective

• Authors: Justin Lee, Zheda Mai, Jinsu Yoo, Chongyu Fan, Cheng Zhang, Wei-Lun Chao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07970
• Pdf URL: https://arxiv.org/pdf/2511.07970
• Copy Paste: [[2511.07970]] Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective(https://arxiv.org/abs/2511.07970)
• Keywords: generation, generative
• Abstract: Machine unlearning--the ability to remove designated concepts from a pre-trained model--has advanced rapidly, particularly for text-to-image diffusion models. However, existing methods typically assume that unlearning requests arrive all at once, whereas in practice they often arrive sequentially. We present the first systematic study of continual unlearning in text-to-image diffusion models and show that popular unlearning methods suffer from rapid utility collapse: after only a few requests, models forget retained knowledge and generate degraded images. We trace this failure to cumulative parameter drift from the pre-training weights and argue that regularization is crucial to addressing it. To this end, we study a suite of add-on regularizers that (1) mitigate drift and (2) remain compatible with existing unlearning methods. Beyond generic regularizers, we show that semantic awareness is essential for preserving concepts close to the unlearning target, and propose a gradient-projection method that constrains parameter drift orthogonal to their subspace. This substantially improves continual unlearning performance and is complementary to other regularizers for further gains. Taken together, our study establishes continual unlearning as a fundamental challenge in text-to-image generation and provides insights, baselines, and open directions for advancing safe and accountable generative AI.
• 摘要：机器取消学习（从预先训练的模型中删除指定概念的能力）已经迅速发展，特别是对于文本到图像的扩散模型。然而，现有的方法通常假设取消学习请求同时到达，而实际上它们通常是按顺序到达的。我们首次对文本到图像扩散模型中的持续忘却进行了系统研究，并表明流行的忘却方法会遭受效用的快速崩溃：仅在几次请求后，模型就会忘记保留的知识并生成质量下降的图像。我们将这种失败归因于预训练权重的累积参数漂移，并认为正则化对于解决该问题至关重要。为此，我们研究了一套附加正则化器，它们（1）减轻漂移，（2）与现有的遗忘方法保持兼容。除了通用正则化器之外，我们还表明语义意识对于保留接近未学习目标的概念至关重要，并提出了一种梯度投影方法，该方法限制与其子空间正交的参数漂移。这大大提高了持续忘却的性能，并且与其他正则化器相辅相成，以获得进一步的收益。总而言之，我们的研究将持续遗忘视为文本到图像生成的基本挑战，并为推进安全和负责任的生成人工智能提供了见解、基线和开放方向。

Title: ChexFract: From General to Specialized - Enhancing Fracture Description Generation

• Authors: Nikolay Nechaev, Evgeniia Przhezdzetskaia, Dmitry Umerenkov, Dmitry V. Dylov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.07983
• Pdf URL: https://arxiv.org/pdf/2511.07983
• Copy Paste: [[2511.07983]] ChexFract: From General to Specialized - Enhancing Fracture Description Generation(https://arxiv.org/abs/2511.07983)
• Keywords: generation
• Abstract: Generating accurate and clinically meaningful radiology reports from chest X-ray images remains a significant challenge in medical AI. While recent vision-language models achieve strong results in general radiology report generation, they often fail to adequately describe rare but clinically important pathologies like fractures. This work addresses this gap by developing specialized models for fracture pathology detection and description. We train fracture-specific vision-language models with encoders from MAIRA-2 and CheXagent, demonstrating significant improvements over general-purpose models in generating accurate fracture descriptions. Analysis of model outputs by fracture type, location, and age reveals distinct strengths and limitations of current vision-language model architectures. We publicly release our best-performing fracture-reporting model, facilitating future research in accurate reporting of rare pathologies.
• 摘要：从胸部 X 射线图像生成准确且具有临床意义的放射学报告仍然是医疗人工智能领域的一项重大挑战。虽然最近的视觉语言模型在一般放射学报告生成方面取得了很好的成果，但它们往往无法充分描述罕见但临床上重要的病理，例如骨折。这项工作通过开发用于骨折病理学检测和描述的专用模型来解决这一差距。我们使用 MAIRA-2 和 CheXagent 的编码器训练骨折特定的视觉语言模型，证明在生成准确的骨折描述方面比通用模型有显着改进。按骨折类型、位置和年龄对模型输出进行分析，揭示了当前视觉语言模型架构的独特优势和局限性。我们公开发布性能最佳的骨折报告模型，促进未来对罕见病理的准确报告的研究。

Title: Invisible Triggers, Visible Threats! Road-Style Adversarial Creation Attack for Visual 3D Detection in Autonomous Driving

• Authors: Jian Wang, Lijun He, Yixing Yong, Haixia Bi, Fan Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08015
• Pdf URL: https://arxiv.org/pdf/2511.08015
• Copy Paste: [[2511.08015]] Invisible Triggers, Visible Threats! Road-Style Adversarial Creation Attack for Visual 3D Detection in Autonomous Driving(https://arxiv.org/abs/2511.08015)
• Keywords: generation
• Abstract: Modern autonomous driving (AD) systems leverage 3D object detection to perceive foreground objects in 3D environments for subsequent prediction and planning. Visual 3D detection based on RGB cameras provides a cost-effective solution compared to the LiDAR paradigm. While achieving promising detection accuracy, current deep neural network-based models remain highly susceptible to adversarial examples. The underlying safety concerns motivate us to investigate realistic adversarial attacks in AD scenarios. Previous work has demonstrated the feasibility of placing adversarial posters on the road surface to induce hallucinations in the detector. However, the unnatural appearance of the posters makes them easily noticeable by humans, and their fixed content can be readily targeted and defended. To address these limitations, we propose the AdvRoad to generate diverse road-style adversarial posters. The adversaries have naturalistic appearances resembling the road surface while compromising the detector to perceive non-existent objects at the attack locations. We employ a two-stage approach, termed Road-Style Adversary Generation and Scenario-Associated Adaptation, to maximize the attack effectiveness on the input scene while ensuring the natural appearance of the poster, allowing the attack to be carried out stealthily without drawing human attention. Extensive experiments show that AdvRoad generalizes well to different detectors, scenes, and spoofing locations. Moreover, physical attacks further demonstrate the practical threats in real-world environments.
• 摘要：现代自动驾驶 (AD) 系统利用 3D 对象检测来感知 3D 环境中的前景对象，以便进行后续预测和规划。与 LiDAR 范例相比，基于 RGB 相机的视觉 3D 检测提供了一种经济高效的解决方案。虽然实现了有希望的检测精度，但当前基于深度神经网络的模型仍然非常容易受到对抗性示例的影响。潜在的安全问题促使我们研究 AD 场景中的现实对抗性攻击。之前的工作已经证明了在路面上放置对抗性海报以诱发探测器产生幻觉的可行性。然而，海报的不自然外观使其很容易被人类注意到，并且其固定内容很容易被瞄准和防御。为了解决这些限制，我们建议 AdvRoad 生成多种道路风格的对抗性海报。对手具有类似于路面的自然外观，同时使探测器感知到攻击位置上不存在的物体。我们采用两阶段方法，即道路式对抗生成和场景相关适应，以最大限度地提高输入场景的攻击效果，同时确保海报的自然外观，从而使攻击能够在不引起人类注意的情况下秘密进行。大量实验表明，AdvRoad 可以很好地推广到不同的探测器、场景和欺骗位置。此外，物理攻击进一步展示了现实环境中的实际威胁。

Title: Multi-modal Deepfake Detection and Localization with FPN-Transformer

• Authors: Chende Zheng, Ruiqi Suo, Zhoulin Ji, Jingyi Deng, Fangbin Yi, Chenhao Lin, Chao Shen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08031
• Pdf URL: https://arxiv.org/pdf/2511.08031
• Copy Paste: [[2511.08031]] Multi-modal Deepfake Detection and Localization with FPN-Transformer(https://arxiv.org/abs/2511.08031)
• Keywords: generative
• Abstract: The rapid advancement of generative adversarial networks (GANs) and diffusion models has enabled the creation of highly realistic deepfake content, posing significant threats to digital trust across audio-visual domains. While unimodal detection methods have shown progress in identifying synthetic media, their inability to leverage cross-modal correlations and precisely localize forged segments limits their practicality against sophisticated, fine-grained manipulations. To address this, we introduce a multi-modal deepfake detection and localization framework based on a Feature Pyramid-Transformer (FPN-Transformer), addressing critical gaps in cross-modal generalization and temporal boundary regression. The proposed approach utilizes pre-trained self-supervised models (WavLM for audio, CLIP for video) to extract hierarchical temporal features. A multi-scale feature pyramid is constructed through R-TLM blocks with localized attention mechanisms, enabling joint analysis of cross-context temporal dependencies. The dual-branch prediction head simultaneously predicts forgery probabilities and refines temporal offsets of manipulated segments, achieving frame-level localization precision. We evaluate our approach on the test set of the IJCAI'25 DDL-AV benchmark, showing a good performance with a final score of 0.7535 for cross-modal deepfake detection and localization in challenging environments. Experimental results confirm the effectiveness of our approach and provide a novel way for generalized deepfake detection. Our code is available at this https URL
• 摘要：生成对抗网络（GAN）和扩散模型的快速发展使得高度逼真的深度伪造内容的创建成为可能，对跨视听领域的数字信任构成了重大威胁。虽然单模态检测方法在识别合成介质方面取得了进展，但它们无法利用跨模态相关性和精确定位伪造片段，限制了它们针对复杂、细粒度操作的实用性。为了解决这个问题，我们引入了一种基于特征金字塔变换器（FPN-Transformer）的多模态深度伪造检测和定位框架，解决了跨模态泛化和时间边界回归中的关键差距。所提出的方法利用预先训练的自监督模型（用于音频的 WavLM，用于视频的 CLIP）来提取分层时间特征。通过具有局部注意机制的 R-TLM 块构建多尺度特征金字塔，从而能够联合分析跨上下文时间依赖性。双分支预测头同时预测伪造概率并细化操作片段的时间偏移，实现帧级定位精度。我们在 IJCAI'25 DDL-AV 基准测试集上评估了我们的方法，在具有挑战性的环境中，跨模式深度伪造检测和定位显示出良好的性能，最终得分为 0.7535。实验结果证实了我们方法的有效性，并为广义的深度伪造检测提供了一种新方法。我们的代码可在此 https URL 获取

Title: Perceptual Quality Assessment of 3D Gaussian Splatting: A Subjective Dataset and Prediction Metric

• Authors: Zhaolin Wan, Yining Diao, Jingqi Xu, Hao Wang, Zhiyang Li, Xiaopeng Fan, Wangmeng Zuo, Debin Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08032
• Pdf URL: https://arxiv.org/pdf/2511.08032
• Copy Paste: [[2511.08032]] Perceptual Quality Assessment of 3D Gaussian Splatting: A Subjective Dataset and Prediction Metric(https://arxiv.org/abs/2511.08032)
• Keywords: quality assessment
• Abstract: With the rapid advancement of 3D visualization, 3D Gaussian Splatting (3DGS) has emerged as a leading technique for real-time, high-fidelity rendering. While prior research has emphasized algorithmic performance and visual fidelity, the perceptual quality of 3DGS-rendered content, especially under varying reconstruction conditions, remains largely underexplored. In practice, factors such as viewpoint sparsity, limited training iterations, point downsampling, noise, and color distortions can significantly degrade visual quality, yet their perceptual impact has not been systematically studied. To bridge this gap, we present 3DGS-QA, the first subjective quality assessment dataset for 3DGS. It comprises 225 degraded reconstructions across 15 object types, enabling a controlled investigation of common distortion factors. Based on this dataset, we introduce a no-reference quality prediction model that directly operates on native 3D Gaussian primitives, without requiring rendered images or ground-truth references. Our model extracts spatial and photometric cues from the Gaussian representation to estimate perceived quality in a structure-aware manner. We further benchmark existing quality assessment methods, spanning both traditional and learning-based approaches. Experimental results show that our method consistently achieves superior performance, highlighting its robustness and effectiveness for 3DGS content evaluation. The dataset and code are made publicly available at this https URL to facilitate future research in 3DGS quality assessment.
• 摘要：随着 3D 可视化的快速发展，3D 高斯分布 (3DGS) 已成为实时、高保真渲染的领先技术。虽然之前的研究强调算法性能和视觉保真度，但 3DGS 渲染内容的感知质量，尤其是在不同的重建条件下，仍然很大程度上没有得到充分探索。在实践中，视点稀疏、有限的训练迭代、点下采样、噪声和颜色失真等因素会显着降低视觉质量，但它们的感知影响尚未得到系统研究。为了弥补这一差距，我们提出了 3DGS-QA，这是第一个 3DGS 主观质量评估数据集。它包含 15 种对象类型的 225 个降级重建，从而能够对常见失真因素进行受控调查。基于该数据集，我们引入了一种无参考质量预测模型，该模型直接对本机 3D 高斯基元进行操作，无需渲染图像或地面实况参考。我们的模型从高斯表示中提取空间和光度线索，以结构感知的方式估计感知质量。我们进一步对现有的质量评估方法进行基准测试，涵盖传统方法和基于学习的方法。实验结果表明，我们的方法始终实现卓越的性能，突出了其对于 3DGS 内容评估的鲁棒性和有效性。数据集和代码在此 https URL 上公开提供，以促进 3DGS 质量评估的未来研究。

Title: Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matching

• Authors: Aditi Singhania, Arushi Jain, Krutik Malani, Riddhi Dhawan, Souymodip Chakraborty, Vineet Batra, Ankit Phogat
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08061
• Pdf URL: https://arxiv.org/pdf/2511.08061
• Copy Paste: [[2511.08061]] Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matching(https://arxiv.org/abs/2511.08061)
• Keywords: restoration, generation, quality assessment
• Abstract: Subject-driven image generation aims to synthesize novel depictions of a specific subject across diverse contexts while preserving its core identity features. Achieving both strong identity consistency and high prompt diversity presents a fundamental trade-off. We propose a LoRA fine-tuned diffusion model employing a latent concatenation strategy, which jointly processes reference and target images, combined with a masked Conditional Flow Matching (CFM) objective. This approach enables robust identity preservation without architectural modifications. To facilitate large-scale training, we introduce a two-stage Distilled Data Curation Framework: the first stage leverages data restoration and VLM-based filtering to create a compact, high-quality seed dataset from diverse sources; the second stage utilizes these curated examples for parameter-efficient fine-tuning, thus scaling the generation capability across various subjects and contexts. Finally, for filtering and quality assessment, we present CHARIS, a fine-grained evaluation framework that performs attribute-level comparisons along five key axes: identity consistency, prompt adherence, region-wise color fidelity, visual quality, and transformation diversity.
• 摘要：主题驱动的图像生成旨在综合不同背景下特定主题的新颖描述，同时保留其核心身份特征。实现强身份一致性和高度即时多样性需要一个基本的权衡。我们提出了一种 LoRA 微调扩散模型，采用潜在串联策略，联合处理参考图像和目标图像，并结合屏蔽条件流匹配 (CFM) 目标。这种方法无需进行架构修改即可实现强大的身份保存。为了促进大规模训练，我们引入了一个两阶段的蒸馏数据管理框架：第一阶段利用数据恢复和基于 VLM 的过滤来创建来自不同来源的紧凑、高质量的种子数据集；第二阶段利用这些精选的示例进行参数有效的微调，从而扩展跨不同主题和上下文的生成能力。最后，对于过滤和质量评估，我们提出了 CHARIS，这是一个细粒度的评估框架，它沿着五个关键轴执行属性级别比较：身份一致性、提示依从性、区域颜色保真度、视觉质量和转换多样性。

Title: CLIP is All You Need for Human-like Semantic Representations in Stable Diffusion

• Authors: Cameron Braunstein, Mariya Toneva, Eddy Ilg
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08075
• Pdf URL: https://arxiv.org/pdf/2511.08075
• Copy Paste: [[2511.08075]] CLIP is All You Need for Human-like Semantic Representations in Stable Diffusion(https://arxiv.org/abs/2511.08075)
• Keywords: generation
• Abstract: Latent diffusion models such as Stable Diffusion achieve state-of-the-art results on text-to-image generation tasks. However, the extent to which these models have a semantic understanding of the images they generate is not well understood. In this work, we investigate whether the internal representations used by these models during text-to-image generation contain semantic information that is meaningful to humans. To do so, we perform probing on Stable Diffusion with simple regression layers that predict semantic attributes for objects and evaluate these predictions against human annotations. Surprisingly, we find that this success can actually be attributed to the text encoding occurring in CLIP rather than the reverse diffusion process. We demonstrate that groups of specific semantic attributes have markedly different decoding accuracy than the average, and are thus represented to different degrees. Finally, we show that attributes become more difficult to disambiguate from one another during the inverse diffusion process, further demonstrating the strongest semantic representation of object attributes in CLIP. We conclude that the separately trained CLIP vision-language model is what determines the human-like semantic representation, and that the diffusion process instead takes the role of a visual decoder.
• 摘要：稳定扩散等潜在扩散模型在文本到图像生成任务上取得了最先进的结果。然而，这些模型对其生成的图像的语义理解程度尚不清楚。在这项工作中，我们研究了这些模型在文本到图像生成过程中使用的内部表示是否包含对人类有意义的语义信息。为此，我们使用简单的回归层对稳定扩散进行探测，预测对象的语义属性并根据人类注释评估这些预测。令人惊讶的是，我们发现这种成功实际上可以归因于 CLIP 中发生的文本编码，而不是反向扩散过程。我们证明，特定语义属性组的解码精度与平均值明显不同，因此表示程度不同。最后，我们表明，在逆扩散过程中，属性变得更加难以消除彼此的歧义，进一步证明了 CLIP 中对象属性的最强语义表示。我们得出的结论是，单独训练的 CLIP 视觉语言模型决定了类人语义表示，而扩散过程则扮演了视觉解码器的角色。

Title: Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editing

• Authors: Ziyu Fan, Zhijian Huang, Yahan Li, Xiaowen Hu, Siyuan Shen, Yunliang Wang, Zeyu Zhong, Shuhong Liu, Shuning Yang, Shangqian Wu, Min Wu, Lei Deng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08080
• Pdf URL: https://arxiv.org/pdf/2511.08080
• Copy Paste: [[2511.08080]] Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editing(https://arxiv.org/abs/2511.08080)
• Keywords: generation
• Abstract: Property-constrained molecular generation and editing are crucial in AI-driven drug discovery but remain hindered by two factors: (i) capturing the complex relationships between molecular structures and multiple properties remains challenging, and (ii) the narrow coverage and incomplete annotations of molecular properties weaken the effectiveness of property-based models. To tackle these limitations, we propose HSPAG, a data-efficient framework featuring hierarchical structure-property alignment. By treating SMILES and molecular properties as complementary modalities, the model learns their relationships at atom, substructure, and whole-molecule levels. Moreover, we select representative samples through scaffold clustering and hard samples via an auxiliary variational auto-encoder (VAE), substantially reducing the required pre-training data. In addition, we incorporate a property relevance-aware masking mechanism and diversified perturbation strategies to enhance generation quality under sparse annotations. Experiments demonstrate that HSPAG captures fine-grained structure-property relationships and supports controllable generation under multiple property constraints. Two real-world case studies further validate the editing capabilities of HSPAG.
• 摘要：属性受限的分子生成和编辑对于人工智能驱动的药物发现至关重要，但仍然受到两个因素的阻碍：（i）捕获分子结构和多种属性之间的复杂关系仍然具有挑战性，（ii）分子属性的狭窄覆盖范围和不完整的注释削弱了基于属性的模型的有效性。为了解决这些限制，我们提出了 HSPAG，一种具有分层结构-属性对齐的数据高效框架。通过将 SMILES 和分子特性视为互补模式，该模型可以学习它们在原子、子结构和整个分子水平上的关系。此外，我们通过支架聚类选择代表性样本，并通过辅助变分自动编码器（VAE）选择硬样本，大大减少了所需的预训练数据。此外，我们结合了属性相关性感知屏蔽机制和多样化的扰动策略，以提高稀疏注释下的生成质量。实验表明，HSPAG 捕获了细粒度的结构-性质关系，并支持多种性质约束下的可控生成。两个真实案例研究进一步验证了 HSPAG 的编辑能力。

Title: Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis

• Authors: Aditi Singhania, Krutik Malani, Riddhi Dhawan, Arushi Jain, Garv Tandon, Nippun Sharma, Souymodip Chakraborty, Vineet Batra, Ankit Phogat
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08087
• Pdf URL: https://arxiv.org/pdf/2511.08087
• Copy Paste: [[2511.08087]] Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis(https://arxiv.org/abs/2511.08087)
• Keywords: generative
• Abstract: Evaluating identity preservation in generative models remains a critical yet unresolved challenge. Existing metrics rely on global embeddings or coarse VLM prompting, failing to capture fine-grained identity changes and providing limited diagnostic insight. We introduce Beyond the Pixels, a hierarchical evaluation framework that decomposes identity assessment into feature-level transformations. Our approach guides VLMs through structured reasoning by (1) hierarchically decomposing subjects into (type, style) -> attribute -> feature decision tree, and (2) prompting for concrete transformations rather than abstract similarity scores. This decomposition grounds VLM analysis in verifiable visual evidence, reducing hallucinations and improving consistency. We validate our framework across four state-of-the-art generative models, demonstrating strong alignment with human judgments in measuring identity consistency. Additionally, we introduce a new benchmark specifically designed to stress-test generative models. It comprises 1,078 image-prompt pairs spanning diverse subject types, including underrepresented categories such as anthropomorphic and animated characters, and captures an average of six to seven transformation axes per prompt.
• 摘要：评估生成模型中的身份保存仍然是一个关键但尚未解决的挑战。现有指标依赖于全局嵌入或粗略的 VLM 提示，无法捕获细粒度的身份更改并提供有限的诊断洞察力。我们引入 Beyond the Pixels，这是一个分层评估框架，可将身份评估分解为特征级转换。我们的方法通过以下方式引导 VLM 进行结构化推理：（1）将主题分层分解为（类型，风格）-> 属性 -> 特征决策树，以及（2）提示具体的转换而不是抽象的相似性分数。这种分解使 VLM 分析基于可验证的视觉证据，减少幻觉并提高一致性。我们在四种最先进的生成模型中验证了我们的框架，证明了在衡量身份一致性方面与人类判断的强烈一致性。此外，我们还引入了一个专门用于对生成模型进行压力测试的新基准。它包含 1,078 个图像提示对，涵盖不同的主题类型，包括拟人化和动画角色等代表性不足的类别，每个提示平均捕获六到七个变换轴。

Title: StableMorph: High-Quality Face Morph Generation with Stable Diffusion

• Authors: Wassim Kabbani, Kiran Raja, Raghavendra Ramachandra, Christoph Busch
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08090
• Pdf URL: https://arxiv.org/pdf/2511.08090
• Copy Paste: [[2511.08090]] StableMorph: High-Quality Face Morph Generation with Stable Diffusion(https://arxiv.org/abs/2511.08090)
• Keywords: generation
• Abstract: Face morphing attacks threaten the integrity of biometric identity systems by enabling multiple individuals to share a single identity. To develop and evaluate effective morphing attack detection (MAD) systems, we need access to high-quality, realistic morphed images that reflect the challenges posed in real-world scenarios. However, existing morph generation methods often produce images that are blurry, riddled with artifacts, or poorly constructed making them easy to detect and not representative of the most dangerous attacks. In this work, we introduce StableMorph, a novel approach that generates highly realistic, artifact-free morphed face images using modern diffusion-based image synthesis. Unlike prior methods, StableMorph produces full-head images with sharp details, avoids common visual flaws, and offers unmatched control over visual attributes. Through extensive evaluation, we show that StableMorph images not only rival or exceed the quality of genuine face images but also maintain a strong ability to fool face recognition systems posing a greater challenge to existing MAD solutions and setting a new standard for morph quality in research and operational testing. StableMorph improves the evaluation of biometric security by creating more realistic and effective attacks and supports the development of more robust detection systems.
• 摘要：面部变形攻击使多个人共享一个身份，从而威胁生物识别系统的完整性。为了开发和评估有效的变形攻击检测 (MAD) 系统，我们需要获得高质量、真实的变形图像，以反映现实场景中提出的挑战。然而，现有的变形生成方法通常会生成模糊、充满伪影或结构不良的图像，使得它们易于检测并且不能代表最危险的攻击。在这项工作中，我们介绍了 StableMorph，这是一种使用现代基于扩散的图像合成生成高度逼真、无伪影的变形面部图像的新颖方法。与之前的方法不同，StableMorph 可以生成具有清晰细节的全头图像，避免常见的视觉缺陷，并提供对视觉属性的无与伦比的控制。通过广泛的评估，我们表明，StableMorph 图像不仅可以媲美或超过真实人脸图像的质量，而且还保持了强大的欺骗人脸识别系统的能力，这对现有的 MAD 解决方案构成了更大的挑战，并为研究和操作测试中的变形质量设定了新的标准。 StableMorph 通过创建更真实、更有效的攻击来改进生物识别安全性的评估，并支持开发更强大的检测系统。

Title: Non-Aligned Reference Image Quality Assessment for Novel View Synthesis

• Authors: Abhijay Ghildyal, Rajesh Sureddi, Nabajeet Barman, Saman Zadtootaghaj, Alan Bovik
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08155
• Pdf URL: https://arxiv.org/pdf/2511.08155
• Copy Paste: [[2511.08155]] Non-Aligned Reference Image Quality Assessment for Novel View Synthesis(https://arxiv.org/abs/2511.08155)
• Keywords: quality assessment
• Abstract: Evaluating the perceptual quality of Novel View Synthesis (NVS) images remains a key challenge, particularly in the absence of pixel-aligned ground truth references. Full-Reference Image Quality Assessment (FR-IQA) methods fail under misalignment, while No-Reference (NR-IQA) methods struggle with generalization. In this work, we introduce a Non-Aligned Reference (NAR-IQA) framework tailored for NVS, where it is assumed that the reference view shares partial scene content but lacks pixel-level alignment. We constructed a large-scale image dataset containing synthetic distortions targeting Temporal Regions of Interest (TROI) to train our NAR-IQA model. Our model is built on a contrastive learning framework that incorporates LoRA-enhanced DINOv2 embeddings and is guided by supervision from existing IQA methods. We train exclusively on synthetically generated distortions, deliberately avoiding overfitting to specific real NVS samples and thereby enhancing the model's generalization capability. Our model outperforms state-of-the-art FR-IQA, NR-IQA, and NAR-IQA methods, achieving robust performance on both aligned and non-aligned references. We also conducted a novel user study to gather data on human preferences when viewing non-aligned references in NVS. We find strong correlation between our proposed quality prediction model and the collected subjective ratings. For dataset and code, please visit our project page: this https URL
• 摘要：评估新视图合成（NVS）图像的感知质量仍然是一个关键挑战，特别是在缺乏像素对齐的地面实况参考的情况下。全参考图像质量评估 (FR-IQA) 方法在未对准的情况下会失败，而无参考 (NR-IQA) 方法则难以泛化。在这项工作中，我们引入了一种专为 NVS 定制的非对齐参考 (NAR-IQA) 框架，其中假设参考视图共享部分场景内容，但缺乏像素级对齐。我们构建了一个包含针对时间感兴趣区域 (TROI) 的合成失真的大规模图像数据集来训练我们的 NAR-IQA 模型。我们的模型建立在对比学习框架的基础上，该框架结合了 LoRA 增强的 DINOv2 嵌入，并以现有 IQA 方法的监督为指导。我们专门针对合成生成的失真进行训练，刻意避免过度拟合特定的真实 NVS 样本，从而增强模型的泛化能力。我们的模型优于最先进的 FR-IQA、NR-IQA 和 NAR-IQA 方法，在对齐和非对齐参考上均实现了稳健的性能。我们还进行了一项新颖的用户研究，以收集人类在 NVS 中查看非对齐参考时的偏好数据。我们发现我们提出的质量预测模型与收集的主观评分之间存在很强的相关性。有关数据集和代码，请访问我们的项目页面：此 https URL

Title: KPLM-STA: Physically-Accurate Shadow Synthesis for Human Relighting via Keypoint-Based Light Modeling

• Authors: Xinhui Yin, Qifei Li, Yilin Guo, Hongxia Xie, Xiaoli Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08169
• Pdf URL: https://arxiv.org/pdf/2511.08169
• Copy Paste: [[2511.08169]] KPLM-STA: Physically-Accurate Shadow Synthesis for Human Relighting via Keypoint-Based Light Modeling(https://arxiv.org/abs/2511.08169)
• Keywords: generation
• Abstract: Image composition aims to seamlessly integrate a foreground object into a background, where generating realistic and geometrically accurate shadows remains a persistent challenge. While recent diffusion-based methods have outperformed GAN-based approaches, existing techniques, such as the diffusion-based relighting framework IC-Light, still fall short in producing shadows with both high appearance realism and geometric precision, especially in composite images. To address these limitations, we propose a novel shadow generation framework based on a Keypoints Linear Model (KPLM) and a Shadow Triangle Algorithm (STA). KPLM models articulated human bodies using nine keypoints and one bounding block, enabling physically plausible shadow projection and dynamic shading across joints, thereby enhancing visual realism. STA further improves geometric accuracy by computing shadow angles, lengths, and spatial positions through explicit geometric formulations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on shadow realism benchmarks, particularly under complex human poses, and generalizes effectively to multi-directional relighting scenarios such as those supported by IC-Light.
• 摘要：图像合成的目的是将前景对象无缝地集成到背景中，其中生成逼真且几何精确的阴影仍然是一个持续的挑战。虽然最近基于扩散的方法优于基于 GAN 的方法，但现有技术（例如基于扩散的重新照明框架 IC-Light）在产生具有高外观真实性和几何精度的阴影方面仍然存在不足，尤其是在合成图像中。为了解决这些限制，我们提出了一种基于关键点线性模型（KPLM）和阴影三角形算法（STA）的新型阴影生成框架。 KPLM 使用九个关键点和一个边界块对铰接人体进行建模，从而实现物理上合理的阴影投影和跨关节的动态着色，从而增强视觉真实感。 STA 通过显式几何公式计算阴影角度、长度和空间位置，进一步提高几何精度。大量的实验表明，我们的方法在阴影真实感基准上实现了最先进的性能，特别是在复杂的人体姿势下，并有效地推广到多方向重新照明场景，例如 IC-Light 支持的场景。

Title: VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion

• Authors: Samet Hicsonmez, Abd El Rahman Shabayek, Djamila Aouada
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08173
• Pdf URL: https://arxiv.org/pdf/2511.08173
• Copy Paste: [[2511.08173]] VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion(https://arxiv.org/abs/2511.08173)
• Keywords: generation
• Abstract: Detecting visual anomalies in diverse, multi-class real-world images is a significant challenge. We introduce \ours, a novel unsupervised multi-class visual anomaly detection framework. It integrates a Latent Diffusion Model (LDM) with a Vision-Language Model (VLM) for enhanced anomaly localization and detection. Specifically, a pre-trained VLM with a simple prompt extracts detailed image descriptions, serving as additional conditioning for LDM training. Current diffusion-based methods rely on synthetic noise generation, limiting their generalization and requiring per-class model training, which hinders scalability. \ours, however, leverages VLMs to obtain normal captions without manual annotations or additional training. These descriptions condition the diffusion model, learning a robust normal image feature representation for multi-class anomaly detection. Our method achieves competitive performance, improving the pixel-level Per-Region-Overlap (PRO) metric by up to 25 points on the Real-IAD dataset and 8 points on the COCO-AD dataset, outperforming state-of-the-art diffusion-based approaches. Code is available at this https URL.
• 摘要：检测多样化、多类别的现实世界图像中的视觉异常是一项重大挑战。我们介绍我们的，一种新颖的无监督多类视觉异常检测框架。它将潜在扩散模型 (LDM) 与视觉语言模型 (VLM) 集成，以增强异常定位和检测。具体来说，经过简单提示的预训练 VLM 会提取详细的图像描述，作为 LDM 训练的附加条件。当前基于扩散的方法依赖于合成噪声的生成，限制了它们的泛化并且需要每类模型训练，这阻碍了可扩展性。然而，我们利用 VLM 来获取正常的字幕，无需手动注释或额外培训。这些描述调节扩散模型，学习用于多类异常检测的鲁棒正常图像特征表示。我们的方法实现了具有竞争力的性能，在 Real-IAD 数据集上将像素级每区域重叠 (PRO) 指标提高了多达 25 个点，在 COCO-AD 数据集上提高了 8 个点，优于最先进的基于扩散的方法。代码可从此 https URL 获取。

Title: WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting

• Authors: Kaitao Huang, Yan Yan, Jing-Hao Xue, Hanzi Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08178
• Pdf URL: https://arxiv.org/pdf/2511.08178
• Copy Paste: [[2511.08178]] WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting(https://arxiv.org/abs/2511.08178)
• Keywords: generation, generative
• Abstract: 3D GAN inversion projects a single image into the latent space of a pre-trained 3D GAN to achieve single-shot novel view synthesis, which requires visible regions with high fidelity and occluded regions with realism and multi-view consistency. However, existing methods focus on the reconstruction of visible regions, while the generation of occluded regions relies only on the generative prior of 3D GAN. As a result, the generated occluded regions often exhibit poor quality due to the information loss caused by the low bit-rate latent code. To address this, we introduce the warping-and-inpainting strategy to incorporate image inpainting into 3D GAN inversion and propose a novel 3D GAN inversion method, WarpGAN. Specifically, we first employ a 3D GAN inversion encoder to project the single-view image into a latent code that serves as the input to 3D GAN. Then, we perform warping to a novel view using the depth map generated by 3D GAN. Finally, we develop a novel SVINet, which leverages the symmetry prior and multi-view image correspondence w.r.t. the same latent code to perform inpainting of occluded regions in the warped image. Quantitative and qualitative experiments demonstrate that our method consistently outperforms several state-of-the-art methods.
• 摘要：3D GAN 反转将单个图像投影到预训练的 3D GAN 的潜在空间中，以实现单次新颖的视图合成，这需要具有高保真度的可见区域和具有真实性和多视图一致性的遮挡区域。然而，现有方法侧重于可见区域的重建，而遮挡区域的生成仅依赖于3D GAN的生成先验。结果，由于低比特率潜在码引起的信息丢失，生成的遮挡区域通常表现出较差的质量。为了解决这个问题，我们引入了变形和修复策略，将图像修复纳入 3D GAN 反转，并提出了一种新颖的 3D GAN 反转方法 WarpGAN。具体来说，我们首先采用 3D GAN 反转编码器将单视图图像投影到潜在代码中，作为 3D GAN 的输入。然后，我们使用 3D GAN 生成的深度图对新视图进行变形。最后，我们开发了一种新颖的 SVINet，它利用了对称先验和多视图图像对应。使用相同的潜在代码对扭曲图像中的遮挡区域进行修复。定量和定性实验表明，我们的方法始终优于几种最先进的方法。

Title: Pixel-level Quality Assessment for Oriented Object Detection

• Authors: Yunhui Zhu, Buliao Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08186
• Pdf URL: https://arxiv.org/pdf/2511.08186
• Copy Paste: [[2511.08186]] Pixel-level Quality Assessment for Oriented Object Detection(https://arxiv.org/abs/2511.08186)
• Keywords: quality assessment
• Abstract: Modern oriented object detectors typically predict a set of bounding boxes and select the top-ranked ones based on estimated localization quality. Achieving high detection performance requires that the estimated quality closely aligns with the actual localization accuracy. To this end, existing approaches predict the Intersection over Union (IoU) between the predicted and ground-truth (GT) boxes as a proxy for localization quality. However, box-level IoU prediction suffers from a structural coupling issue: since the predicted box is derived from the detector's internal estimation of the GT box, the predicted IoU--based on their similarity--can be overestimated for poorly localized boxes. To overcome this limitation, we propose a novel Pixel-level Quality Assessment (PQA) framework, which replaces box-level IoU prediction with the integration of pixel-level spatial consistency. PQA measures the alignment between each pixel's relative position to the predicted box and its corresponding position to the GT box. By operating at the pixel level, PQA avoids directly comparing the predicted box with the estimated GT box, thereby eliminating the inherent similarity bias in box-level IoU prediction. Furthermore, we introduce a new integration metric that aggregates pixel-level spatial consistency into a unified quality score, yielding a more accurate approximation of the actual localization quality. Extensive experiments on HRSC2016 and DOTA demonstrate that PQA can be seamlessly integrated into various oriented object detectors, consistently improving performance (e.g., +5.96% AP$_{50:95}$ on Rotated RetinaNet and +2.32% on STD).
• 摘要：现代面向对象检测器通常会预测一组边界框，并根据估计的定位质量选择排名靠前的边界框。实现高检测性能需要估计的质量与实际的定位精度紧密结合。为此，现有方法预测预测框和真实框 (GT) 之间的交集 (IoU)，作为定位质量的代理。然而，框级 IoU 预测存在结构耦合问题：由于预测框是从检测器对 GT 框的内部估计得出的，因此基于相似性的预测 IoU 对于定位不良的框可能会被高估。为了克服这一限制，我们提出了一种新颖的像素级质量评估（PQA）框架，该框架通过集成像素级空间一致性来取代框级 IoU 预测。 PQA 测量每个像素相对于预测框的相对位置与其相对于 GT 框的对应位置之间的对齐情况。通过在像素级操作，PQA避免了直接将预测框与估计的GT框进行比较，从而消除了框级IoU预测中固有的相似性偏差。此外，我们引入了一种新的集成指标，将像素级空间一致性聚合为统一的质量分数，从而产生更准确的实际定位质量近似值。 HRSC2016 和 DOTA 上的大量实验表明，PQA 可以无缝集成到各种定向目标检测器中，从而持续提高性能（例如，Rotated RetinaNet 上 +5.96% AP$_{50:95}$，STD 上 +2.32%）。

Title: UI2Code$^\text{N}$: A Visual Language Model for Test-Time Scalable Interactive UI-to-Code Generation

• Authors: Zhen Yang, Wenyi Hong, Mingde Xu, Xinyue Fan, Weihan Wang, Jiele Cheng, Xiaotao Gu, Jie Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08195
• Pdf URL: https://arxiv.org/pdf/2511.08195
• Copy Paste: [[2511.08195]] UI2Code$^\text{N}$: A Visual Language Model for Test-Time Scalable Interactive UI-to-Code Generation(https://arxiv.org/abs/2511.08195)
• Keywords: generation
• Abstract: User interface (UI) programming is a core yet highly complex part of modern software development. Recent advances in visual language models (VLMs) highlight the potential of automatic UI coding, but current approaches face two key limitations: multimodal coding capabilities remain underdeveloped, and single-turn paradigms make little use of iterative visual feedback. We address these challenges with an interactive UI-to-code paradigm that better reflects real-world workflows and raises the upper bound of achievable performance. Under this paradigm, we present UI2Code$^\text{N}$, a visual language model trained through staged pretraining, fine-tuning, and reinforcement learning to achieve foundational improvements in multimodal coding. The model unifies three key capabilities: UI-to-code generation, UI editing, and UI polishing. We further explore test-time scaling for interactive generation, enabling systematic use of multi-turn feedback. Experiments on UI-to-code and UI polishing benchmarks show that UI2Code$^\text{N}$ establishes a new state of the art among open-source models and achieves performance comparable to leading closed-source models such as Claude-4-Sonnet and GPT-5. Our code and models are available at this https URL.
• 摘要：用户界面（UI）编程是现代软件开发的核心但高度复杂的部分。视觉语言模型 (VLM) 的最新进展凸显了自动 UI 编码的潜力，但当前的方法面临两个关键限制：多模式编码功能仍然不发达，单轮范式很少使用迭代视觉反馈。我们通过交互式 UI 到代码范例来应对这些挑战，该范例可以更好地反映现实世界的工作流程并提高可实现性能的上限。在此范例下，我们提出了 UI2Code$^\text{N}$，这是一种通过分阶段预训练、微调和强化学习进行训练的视觉语言模型，以实现多模态编码的基础性改进。该模型统一了三个关键功能：UI 到代码生成、UI 编辑和 UI 完善。我们进一步探索交互式生成的测试时间扩展，从而能够系统地使用多轮反馈。 UI 到代码和 UI 抛光基准的实验表明，UI2Code$^\text{N}$ 在开源模型中建立了新的最先进水平，并实现了与领先的闭源模型（如 Claude-4-Sonnet 和 GPT-5）相当的性能。我们的代码和模型可从此 https URL 获取。

Title: Twist and Compute: The Cost of Pose in 3D Generative Diffusion

• Authors: Kyle Fogarty, Jack Foster, Boqiao Zhang, Jing Yang, Cengiz Öztireli
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08203
• Pdf URL: https://arxiv.org/pdf/2511.08203
• Copy Paste: [[2511.08203]] Twist and Compute: The Cost of Pose in 3D Generative Diffusion(https://arxiv.org/abs/2511.08203)
• Keywords: generative
• Abstract: Despite their impressive results, large-scale image-to-3D generative models remain opaque in their inductive biases. We identify a significant limitation in image-conditioned 3D generative models: a strong canonical view bias. Through controlled experiments using simple 2D rotations, we show that the state-of-the-art Hunyuan3D 2.0 model can struggle to generalize across viewpoints, with performance degrading under rotated inputs. We show that this failure can be mitigated by a lightweight CNN that detects and corrects input orientation, restoring model performance without modifying the generative backbone. Our findings raise an important open question: Is scale enough, or should we pursue modular, symmetry-aware designs?
• 摘要：尽管取得了令人印象深刻的结果，大规模图像到 3D 生成模型的归纳偏差仍然不透明。我们发现图像条件 3D 生成模型的一个重大限制：强烈的规范视图偏差。通过使用简单 2D 旋转的受控实验，我们表明最先进的 Hunyuan3D 2.0 模型很难跨视点进行泛化，并且在旋转输入下性能会下降。我们证明，可以通过轻量级 CNN 来缓解这种故障，该 CNN 可以检测并纠正输入方向，从而在不修改生成主干的情况下恢复模型性能。我们的研究结果提出了一个重要的开放性问题：规模是否足够，或者我们应该追求模块化、对称感知的设计？

Title: Evaluating Gemini LLM in Food Image-Based Recipe and Nutrition Description with EfficientNet-B4 Visual Backbone

• Authors: Rizal Khoirul Anam
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.08215
• Pdf URL: https://arxiv.org/pdf/2511.08215
• Copy Paste: [[2511.08215]] Evaluating Gemini LLM in Food Image-Based Recipe and Nutrition Description with EfficientNet-B4 Visual Backbone(https://arxiv.org/abs/2511.08215)
• Keywords: generative
• Abstract: The proliferation of digital food applications necessitates robust methods for automated nutritional analysis and culinary guidance. This paper presents a comprehensive comparative evaluation of a decoupled, multimodal pipeline for food recognition. We evaluate a system integrating a specialized visual backbone (EfficientNet-B4) with a powerful generative large language model (Google's Gemini LLM). The core objective is to evaluate the trade-offs between visual classification accuracy, model efficiency, and the quality of generative output (nutritional data and recipes). We benchmark this pipeline against alternative vision backbones (VGG-16, ResNet-50, YOLOv8) and a lightweight LLM (Gemma). We introduce a formalization for "Semantic Error Propagation" (SEP) to analyze how classification inaccuracies from the visual module cascade into the generative output. Our analysis is grounded in a new Custom Chinese Food Dataset (CCFD) developed to address cultural bias in public datasets. Experimental results demonstrate that while EfficientNet-B4 (89.0\% Top-1 Acc.) provides the best balance of accuracy and efficiency, and Gemini (9.2/10 Factual Accuracy) provides superior generative quality, the system's overall utility is fundamentally bottlenecked by the visual front-end's perceptive accuracy. We conduct a detailed per-class analysis, identifying high semantic similarity as the most critical failure mode.
• 摘要：数字食品应用的激增需要强大的方法来进行自动营养分析和烹饪指导。本文对食品识别的解耦、多模式管道进行了全面的比较评估。我们评估了一个将专门的视觉主干（EfficientNet-B4）与强大的生成大语言模型（Google 的 Gemini LLM）集成的系统。核心目标是评估视觉分类准确性、模型效率和生成输出质量（营养数据和食谱）之间的权衡。我们针对替代视觉主干（VGG-16、ResNet-50、YOLOv8）和轻量级 LLM（Gemma）对该流程进行基准测试。我们引入了“语义错误传播”（SEP）的形式化来分析视觉模块中的分类不准确如何级联到生成输出中。我们的分析基于一个新的中国定制食品数据集（CCFD），该数据集是为了解决公共数据集中的文化偏见而开发的。实验结果表明，虽然 EfficientNet-B4 (89.0\% Top-1 Acc.) 提供了准确性和效率的最佳平衡，而 Gemini (9.2/10 Factual Accuracy) 提供了卓越的生成质量，但系统的整体效用从根本上受到视觉前端感知准确性的瓶颈。我们进行了详细的每类分析，将高语义相似性确定为最关键的故障模式。

Title: 2D Representation for Unguided Single-View 3D Super-Resolution in Real-Time

• Authors: Ignasi Mas, Ivan Huerta, Ramon Morros, Javier Ruiz-Hidalgo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08224
• Pdf URL: https://arxiv.org/pdf/2511.08224
• Copy Paste: [[2511.08224]] 2D Representation for Unguided Single-View 3D Super-Resolution in Real-Time(https://arxiv.org/abs/2511.08224)
• Keywords: super-resolution
• Abstract: We introduce 2Dto3D-SR, a versatile framework for real-time single-view 3D super-resolution that eliminates the need for high-resolution RGB guidance. Our framework encodes 3D data from a single viewpoint into a structured 2D representation, enabling the direct application of existing 2D image super-resolution architectures. We utilize the Projected Normalized Coordinate Code (PNCC) to represent 3D geometry from a visible surface as a regular image, thereby circumventing the complexities of 3D point-based or RGB-guided methods. This design supports lightweight and fast models adaptable to various deployment environments. We evaluate 2Dto3D-SR with two implementations: one using Swin Transformers for high accuracy, and another using Vision Mamba for high efficiency. Experiments show the Swin Transformer model achieves state-of-the-art accuracy on standard benchmarks, while the Vision Mamba model delivers competitive results at real-time speeds. This establishes our geometry-guided pipeline as a surprisingly simple yet viable and practical solution for real-world scenarios, especially where high-resolution RGB data is inaccessible.
• 摘要：我们推出了 2Dto3D-SR，这是一种用于实时单视图 3D 超分辨率的多功能框架，无需高分辨率 RGB 引导。我们的框架将来自单一视点的 3D 数据编码为结构化 2D 表示，从而能够直接应用现有的 2D 图像超分辨率架构。我们利用投影归一化坐标代码 (PNCC) 将可见表面的 3D 几何图形表示为规则图像，从而避免了基于 3D 点或 RGB 引导方法的复杂性。该设计支持适应各种部署环境的轻量级快速模型。我们通过两种实现方式评估 2Dto3D-SR：一种使用 Swin Transformers 实现高精度，另一种使用 Vision Mamba 实现高效率。实验表明，Swin Transformer 模型在标准基准测试中实现了最先进的精度，而 Vision Mamba 模型则以实时速度提供具有竞争力的结果。这使我们的几何引导管道成为现实场景中极其简单但可行且实用的解决方案，特别是在无法访问高分辨率 RGB 数据的情况下。

Title: Top2Ground: A Height-Aware Dual Conditioning Diffusion Model for Robust Aerial-to-Ground View Generation

• Authors: Jae Joong Lee, Bedrich Benes
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08258
• Pdf URL: https://arxiv.org/pdf/2511.08258
• Copy Paste: [[2511.08258]] Top2Ground: A Height-Aware Dual Conditioning Diffusion Model for Robust Aerial-to-Ground View Generation(https://arxiv.org/abs/2511.08258)
• Keywords: generation
• Abstract: Generating ground-level images from aerial views is a challenging task due to extreme viewpoint disparity, occlusions, and a limited field of view. We introduce Top2Ground, a novel diffusion-based method that directly generates photorealistic ground-view images from aerial input images without relying on intermediate representations such as depth maps or 3D voxels. Specifically, we condition the denoising process on a joint representation of VAE-encoded spatial features (derived from aerial RGB images and an estimated height map) and CLIP-based semantic embeddings. This design ensures the generation is both geometrically constrained by the scene's 3D structure and semantically consistent with its content. We evaluate Top2Ground on three diverse datasets: CVUSA, CVACT, and the Auto Arborist. Our approach shows 7.3% average improvement in SSIM across three benchmark datasets, showing Top2Ground can robustly handle both wide and narrow fields of view, highlighting its strong generalization capabilities.
• 摘要：由于极端的视点差异、遮挡和有限的视野，从鸟瞰图生成地面图像是一项具有挑战性的任务。我们介绍了 Top2Ground，这是一种基于扩散的新颖方法，可以直接从航空输入图像生成逼真的地面视图图像，而不依赖于深度图或 3D 体素等中间表示。具体来说，我们将去噪过程基于 VAE 编码的空间特征（源自航空 RGB 图像和估计的高度图）和基于 CLIP 的语义嵌入的联合表示。这种设计确保生成在几何上受到场景 3D 结构的约束，并且在语义上与其内容一致。我们在三个不同的数据集上评估 Top2Ground：CVUSA、CVACT 和 Auto Arborist。我们的方法显示，三个基准数据集的 SSIM 平均提高了 7.3%，这表明 Top2Ground 可以稳健地处理宽视场和窄视场，突出了其强大的泛化能力。

Title: Test-time Diverse Reasoning by Riemannian Activation Steering

• Authors: Ly Tran Ho Khanh, Dongxuan Zhu, Man-Chung Yue, Viet Anh Nguyen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08305
• Pdf URL: https://arxiv.org/pdf/2511.08305
• Copy Paste: [[2511.08305]] Test-time Diverse Reasoning by Riemannian Activation Steering(https://arxiv.org/abs/2511.08305)
• Keywords: generation, generative
• Abstract: Best-of-$N$ reasoning improves the accuracy of language models in solving complex tasks by sampling multiple candidate solutions and then selecting the best one based on some criteria. A critical bottleneck for this strategy is the output diversity limit, which occurs when the model generates similar outputs despite stochastic sampling, and hence recites the same error. To address this lack of variance in reasoning paths, we propose a novel unsupervised activation steering strategy that simultaneously optimizes the steering vectors for multiple reasoning trajectories at test time. At any synchronization anchor along the batch generation process, we find the steering vectors that maximize the total volume spanned by all possible intervened activation subsets. We demonstrate that these steering vectors can be determined by solving a Riemannian optimization problem over the product of spheres with a log-determinant objective function. We then use a Riemannian block-coordinate descent algorithm with a well-tuned learning rate to obtain a stationary point of the problem, and we apply these steering vectors until the generation process reaches the subsequent synchronization anchor. Empirical evaluations on popular mathematical benchmarks demonstrate that our test-time Riemannian activation steering strategy outperforms vanilla sampling techniques in terms of generative diversity and solution accuracy.
• 摘要：Best-of-$N$ 推理通过对多个候选解决方案进行采样，然后根据某些标准选择最佳解决方案，提高了语言模型解决复杂任务的准确性。该策略的一个关键瓶颈是输出多样性限制，当模型尽管随机采样但仍生成相似的输出并因此出现相同的错误时，就会出现这种情况。为了解决推理路径缺乏方差的问题，我们提出了一种新颖的无监督激活转向策略，该策略在测试时同时优化多个推理轨迹的转向向量。在批量生成过程中的任何同步锚点，我们找到使所有可能的干预激活子集跨越的总体积最大化的转向向量。我们证明，这些转向向量可以通过求解具有对数行列式目标函数的球体乘积的黎曼优化问题来确定。然后，我们使用具有经过良好调整的学习率的黎曼块坐标下降算法来获得问题的驻点，并应用这些引导向量，直到生成过程到达后续同步锚点。对流行数学基准的实证评估表明，我们的测试时黎曼激活引导策略在生成多样性和解决方案准确性方面优于普通采样技术。

Title: HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting

• Authors: Andrey Savchenko, Oleg Kachan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08340
• Pdf URL: https://arxiv.org/pdf/2511.08340
• Copy Paste: [[2511.08340]] HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting(https://arxiv.org/abs/2511.08340)
• Keywords: generative
• Abstract: Accurate forecasting of multivariate time series data remains a formidable challenge, particularly due to the growing complexity of temporal dependencies in real-world scenarios. While neural network-based models have achieved notable success in this domain, complex channel-dependent models often suffer from performance degradation compared to channel-independent models that do not consider the relationship between components but provide high robustness due to small capacity. In this work, we propose HN-MVTS, a novel architecture that integrates a hypernetwork-based generative prior with an arbitrary neural network forecasting model. The input of this hypernetwork is a learnable embedding matrix of time series components. To restrict the number of new parameters, the hypernetwork learns to generate the weights of the last layer of the target forecasting networks, serving as a data-adaptive regularizer that improves generalization and long-range predictive accuracy. The hypernetwork is used only during the training, so it does not increase the inference time compared to the base forecasting model. Extensive experiments on eight benchmark datasets demonstrate that application of HN-MVTS to the state-of-the-art models (DLinear, PatchTST, TSMixer, etc.) typically improves their performance. Our findings suggest that hypernetwork-driven parameterization offers a promising direction for enhancing existing forecasting techniques in complex scenarios.
• 摘要：多元时间序列数据的准确预测仍然是一个艰巨的挑战，特别是由于现实场景中时间依赖性日益复杂。虽然基于神经网络的模型在该领域取得了显着的成功，但与通道无关模型相比，复杂的通道相关模型通常会遭受性能下降，通道无关模型不考虑组件之间的关系，但由于容量小而提供高鲁棒性。在这项工作中，我们提出了 HN-MVTS，这是一种新颖的架构，它将基于超网络的生成先验与任意神经网络预测模型相集成。该超网络的输入是时间序列分量的可学习嵌入矩阵。为了限制新参数的数量，超网络学习生成目标预测网络最后一层的权重，充当数据自适应正则化器，提高泛化性和长期预测准确性。超网络仅在训练期间使用，因此与基本预测模型相比，它不会增加推理时间。对八个基准数据集的大量实验表明，将 HN-MVTS 应用于最先进的模型（DLinear、PatchTST、TSMixer 等）通常可以提高其性能。我们的研究结果表明，超网络驱动的参数化为增强复杂场景中的现有预测技术提供了一个有前途的方向。

Title: Towards Open-Set Myoelectric Gesture Recognition via Dual-Perspective Inconsistency Learning

• Authors: Chen Liu, Can Han, Weishi Xu, Yaqi Wang, Dahong Qian
• Subjects: cs.CV, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2511.08344
• Pdf URL: https://arxiv.org/pdf/2511.08344
• Copy Paste: [[2511.08344]] Towards Open-Set Myoelectric Gesture Recognition via Dual-Perspective Inconsistency Learning(https://arxiv.org/abs/2511.08344)
• Keywords: generation
• Abstract: Surface electromyography (sEMG)-based gesture recognition plays a critical role in human-machine interaction (HMI), particularly for rehabilitation and prosthetic control. However, sEMG-based systems often suffer from the scarcity of informative training data, leading to overfitting and poor generalization in deep learning models. Data augmentation offers a promising approach to increasing the size and diversity of training data, where faithfulness and diversity are two critical factors to effectiveness. However, promoting untargeted diversity can result in redundant samples with limited utility. To address these challenges, we propose a novel diffusion-based data augmentation approach, Sparse-Aware Semantic-Guided Diffusion Augmentation (SASG-DA). To enhance generation faithfulness, we introduce the Semantic Representation Guidance (SRG) mechanism by leveraging fine-grained, task-aware semantic representations as generation conditions. To enable flexible and diverse sample generation, we propose a Gaussian Modeling Semantic Modeling (GMSS) strategy, which models the semantic representation distribution and allows stochastic sampling to produce both faithful and diverse samples. To enhance targeted diversity, we further introduce a Sparse-Aware Semantic Sampling strategy to explicitly explore underrepresented regions, improving distribution coverage and sample utility. Extensive experiments on benchmark sEMG datasets, Ninapro DB2, DB4, and DB7, demonstrate that SASG-DA significantly outperforms existing augmentation methods. Overall, our proposed data augmentation approach effectively mitigates overfitting and improves recognition performance and generalization by offering both faithful and diverse samples.
• 摘要：基于表面肌电图 (sEMG) 的手势识别在人机交互 (HMI) 中发挥着至关重要的作用，特别是在康复和假肢控制方面。然而，基于表面肌电图的系统往往缺乏信息丰富的训练数据，导致深度学习模型过度拟合和泛化能力差。数据增强提供了一种有前途的方法来增加训练数据的规模和多样性，其中忠实性和多样性是有效性的两个关键因素。然而，促进无针对性的多样性可能会导致效用有限的冗余样本。为了应对这些挑战，我们提出了一种新颖的基于扩散的数据增强方法，即稀疏感知语义引导扩散增强（SASG-DA）。为了提高生成的忠实度，我们通过利用细粒度、任务感知的语义表示作为生成条件，引入了语义表示指导（SRG）机制。为了实现灵活多样的样本生成，我们提出了高斯建模语义建模（GMSS）策略，该策略对语义表示分布进行建模，并允许随机采样产生忠实且多样化的样本。为了增强目标多样性，我们进一步引入了稀疏感知语义采样策略，以明确探索代表性不足的区域，提高分布覆盖范围和样本效用。对基准 sEMG 数据集、Ninapro DB2、DB4 和 DB7 进行的大量实验表明，SASG-DA 的性能显着优于现有的增强方法。总的来说，我们提出的数据增强方法通过提供忠实且多样化的样本，有效地减轻了过度拟合并提高了识别性能和泛化能力。

Title: VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation

• Authors: Arpan Phukan, Anupam Pandey, Deepjyoti Bodo, Asif Ekbal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08348
• Pdf URL: https://arxiv.org/pdf/2511.08348
• Copy Paste: [[2511.08348]] VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation(https://arxiv.org/abs/2511.08348)
• Keywords: generation
• Abstract: Multi-hop Question Generation (QG) effectively evaluates reasoning but remains confined to text; Video Question Generation (VideoQG) is limited to zero-hop questions over single segments. To address this, we introduce VideoChain, a novel Multi-hop Video Question Generation (MVQG) framework designed to generate questions that require reasoning across multiple, temporally separated video segments. VideoChain features a modular architecture built on a modified BART backbone enhanced with video embeddings, capturing textual and visual dependencies. Using the TVQA+ dataset, we automatically construct the large-scale MVQ-60 dataset by merging zero-hop QA pairs, ensuring scalability and diversity. Evaluations show VideoChain's strong performance across standard generation metrics: ROUGE-L (0.6454), ROUGE-1 (0.6854), BLEU-1 (0.6711), BERTScore-F1 (0.7967), and semantic similarity (0.8110). These results highlight the model's ability to generate coherent, contextually grounded, and reasoning-intensive questions.
• 摘要：多跳问题生成（QG）有效评估推理，但仍局限于文本；视频问题生成 (VideoQG) 仅限于单个片段上的零跳问题。为了解决这个问题，我们引入了 VideoChain，这是一种新颖的多跳视频问题生成 (MVQG) 框架，旨在生成需要跨多个时间分离的视频片段进行推理的问题。 VideoChain 采用模块化架构，构建在经过修改的 BART 主干上，并通过视频嵌入增强，捕获文本和视觉依赖性。使用 TVQA+ 数据集，我们通过合并零跳 QA 对自动构建大规模 MVQ-60 数据集，确保可扩展性和多样性。评估显示 VideoChain 在标准生成指标上表现强劲：ROUGE-L (0.6454)、ROUGE-1 (0.6854)、BLEU-1 (0.6711)、BERTScore-F1 (0.7967) 和语义相似度 (0.8110)。这些结果凸显了该模型生成连贯、基于上下文和推理密集型问题的能力。

Title: Text-based Aerial-Ground Person Retrieval

• Authors: Xinyu Zhou, Yu Wu, Jiayao Ma, Wenhao Wang, Min Cao, Mang Ye
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.08369
• Pdf URL: https://arxiv.org/pdf/2511.08369
• Copy Paste: [[2511.08369]] Text-based Aerial-Ground Person Retrieval(https://arxiv.org/abs/2511.08369)
• Keywords: generation
• Abstract: This work introduces Text-based Aerial-Ground Person Retrieval (TAG-PR), which aims to retrieve person images from heterogeneous aerial and ground views with textual descriptions. Unlike traditional Text-based Person Retrieval (T-PR), which focuses solely on ground-view images, TAG-PR introduces greater practical significance and presents unique challenges due to the large viewpoint discrepancy across images. To support this task, we contribute: (1) TAG-PEDES dataset, constructed from public benchmarks with automatically generated textual descriptions, enhanced by a diversified text generation paradigm to ensure robustness under view heterogeneity; and (2) TAG-CLIP, a novel retrieval framework that addresses view heterogeneity through a hierarchically-routed mixture of experts module to learn view-specific and view-agnostic features and a viewpoint decoupling strategy to decouple view-specific features for better cross-modal alignment. We evaluate the effectiveness of TAG-CLIP on both the proposed TAG-PEDES dataset and existing T-PR benchmarks. The dataset and code are available at this https URL.
• 摘要：这项工作介绍了基于文本的空中-地面人物检索（TAG-PR），其目的是从带有文本描述的异构空中和地面视图中检索人物图像。与仅关注地面视图图像的传统基于文本的人物检索（T-PR）不同，TAG-PR 引入了更大的实际意义，并且由于图像之间存在较大的视点差异而提出了独特的挑战。为了支持这项任务，我们贡献：（1）TAG-PEDES数据集，根据公共基准构建并自动生成文本描述，并通过多样化的文本生成范例进行增强，以确保视图异质性下的鲁棒性； （2）TAG-CLIP，一种新颖的检索框架，通过专家模块的分层路由混合来解决视图异质性，以学习视图特定和视图不可知的特征，以及观点解耦策略来解耦视图特定的特征以实现更好的跨模式对齐。我们评估了 TAG-CLIP 在提议的 TAG-PEDES 数据集和现有 T-PR 基准上的有效性。数据集和代码可从此 https URL 获取。

Title: OmniAID: Decoupling Semantic and Artifacts for Universal AI-Generated Image Detection in the Wild

• Authors: Yuncheng Guo, Junyan Ye, Chenjue Zhang, Hengrui Kang, Haohuan Fu, Conghui He, Weijia Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08423
• Pdf URL: https://arxiv.org/pdf/2511.08423
• Copy Paste: [[2511.08423]] OmniAID: Decoupling Semantic and Artifacts for Universal AI-Generated Image Detection in the Wild(https://arxiv.org/abs/2511.08423)
• Keywords: generative
• Abstract: A truly universal AI-Generated Image (AIGI) detector must simultaneously generalize across diverse generative models and varied semantic content. Current state-of-the-art methods learn a single, entangled forgery representation--conflating content-dependent flaws with content-agnostic artifacts--and are further constrained by outdated benchmarks. To overcome these limitations, we propose OmniAID, a novel framework centered on a decoupled Mixture-of-Experts (MoE) architecture. The core of our method is a hybrid expert system engineered to decouple: (1) semantic flaws across distinct content domains, and (2) these content-dependent flaws from content-agnostic universal artifacts. This system employs a set of Routable Specialized Semantic Experts, each for a distinct domain (e.g., human, animal), complemented by a Fixed Universal Artifact Expert. This architecture is trained using a bespoke two-stage strategy: we first train the experts independently with domain-specific hard-sampling to ensure specialization, and subsequently train a lightweight gating network for effective input routing. By explicitly decoupling "what is generated" (content-specific flaws) from "how it is generated" (universal artifacts), OmniAID achieves robust generalization. To address outdated benchmarks and validate real-world applicability, we introduce Mirage, a new large-scale, contemporary dataset. Extensive experiments, using both traditional benchmarks and our Mirage dataset, demonstrate our model surpasses existing monolithic detectors, establishing a new, robust standard for AIGI authentication against modern, in-the-wild threats.
• 摘要：真正通用的人工智能生成图像（AIGI）检测器必须同时泛化不同的生成模型和不同的语义内容。当前最先进的方法学习单一的、纠缠的伪造表示——将与内容相关的缺陷与与内容无关的工件混为一谈——并且进一步受到过时基准的限制。为了克服这些限制，我们提出了 OmniAID，这是一种以解耦专家混合 (MoE) 架构为中心的新颖框架。我们方法的核心是一个混合专家系统，旨在解耦：（1）不同内容域之间的语义缺陷，以及（2）这些与内容无关的通用工件中的内容相关缺陷。该系统采用一组可路由的专业语义专家，每个专家针对一个不同的领域（例如，人类、动物），并由固定的通用工件专家进行补充。该架构使用定制的两阶段策略进行训练：我们首先通过特定领域的硬采样独立训练专家以确保专业化，然后训练轻量级门网络以实现有效的输入路由。通过明确地将“生成的内容”（特定于内容的缺陷）与“生成的方式”（通用工件）分离，OmniAID 实现了强大的泛化。为了解决过时的基准并验证现实世界的适用性，我们引入了 Mirage，一个新的大规模当代数据集。使用传统基准和我们的 Mirage 数据集进行的大量实验表明，我们的模型超越了现有的整体检测器，为针对现代野外威胁的 AIGI 身份验证建立了新的、强大的标准。

Title: HardFlow: Hard-Constrained Sampling for Flow-Matching Models via Trajectory Optimization

• Authors: Zeyang Li, Kaveh Alim, Navid Azizan
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2511.08425
• Pdf URL: https://arxiv.org/pdf/2511.08425
• Copy Paste: [[2511.08425]] HardFlow: Hard-Constrained Sampling for Flow-Matching Models via Trajectory Optimization(https://arxiv.org/abs/2511.08425)
• Keywords: generative
• Abstract: Diffusion and flow-matching have emerged as powerful methodologies for generative modeling, with remarkable success in capturing complex data distributions and enabling flexible guidance at inference time. Many downstream applications, however, demand enforcing hard constraints on generated samples (for example, robot trajectories must avoid obstacles), a requirement that goes beyond simple guidance. Prevailing projection-based approaches constrain the entire sampling path to the constraint manifold, which is overly restrictive and degrades sample quality. In this paper, we introduce a novel framework that reformulates hard-constrained sampling as a trajectory optimization problem. Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time. By exploiting the underlying structure of flow-matching models and adopting techniques from model predictive control, we transform this otherwise complex constrained optimization problem into a tractable surrogate that can be solved efficiently and effectively. Furthermore, this trajectory optimization perspective offers significant flexibility beyond mere constraint satisfaction, allowing for the inclusion of integral costs to minimize distribution shift and terminal objectives to further enhance sample quality, all within a unified framework. We provide a control-theoretic analysis of our method, establishing bounds on the approximation error between our tractable surrogate and the ideal formulation. Extensive experiments across diverse domains, including robotics (planning), partial differential equations (boundary control), and vision (text-guided image editing), demonstrate that our algorithm, which we name $\textit{HardFlow}$, substantially outperforms existing methods in both constraint satisfaction and sample quality.
• 摘要：扩散和流匹配已成为生成建模的强大方法，在捕获复杂的数据分布和在推理时实现灵活的指导方面取得了显着的成功。然而，许多下游应用程序要求对生成的样本实施硬约束（例如，机器人轨迹必须避开障碍物），这超出了简单指导的要求。流行的基于投影的方法将整个采样路径限制到约束流形，这限制性太大并降低了样本质量。在本文中，我们介绍了一种新颖的框架，它将硬约束采样重新表述为轨迹优化问题。我们的主要见解是利用数值最优控制来引导采样轨迹，以便在最终时间精确满足约束。通过利用流匹配模型的底层结构并采用模型预测控制技术，我们将这个原本复杂的约束优化问题转化为可以高效解决的易于处理的替代问题。此外，这种轨迹优化视角提供了超越单纯约束满足的显着灵活性，允许包含积分成本以最小化分布变化和终端目标以进一步提高样本质量，所有这些都在统一的框架内。我们对我们的方法进行了控制理论分析，建立了我们易于处理的替代方案和理想公式之间的近似误差界限。跨不同领域（包括机器人（规划）、偏微分方程（边界控制）和视觉（文本引导图像编辑））的广泛实验表明，我们的算法（我们将其命名为$\textit{HardFlow}$）在约束满足和样本质量方面都远远优于现有方法。

Title: UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist

• Authors: Zhengyang Liang, Daoan Zhang, Huichi Zhou, Rui Huang, Bobo Li, Yuechen Zhang, Shengqiong Wu, Xiaohan Wang, Jiebo Luo, Lizi Liao, Hao Fei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08521
• Pdf URL: https://arxiv.org/pdf/2511.08521
• Copy Paste: [[2511.08521]] UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist(https://arxiv.org/abs/2511.08521)
• Keywords: generation
• Abstract: While specialized AI models excel at isolated video tasks like generation or understanding, real-world applications demand complex, iterative workflows that combine these capabilities. To bridge this gap, we introduce UniVA, an open-source, omni-capable multi-agent framework for next-generation video generalists that unifies video understanding, segmentation, editing, and generation into cohesive workflows. UniVA employs a Plan-and-Act dual-agent architecture that drives a highly automated and proactive workflow: a planner agent interprets user intentions and decomposes them into structured video-processing steps, while executor agents execute these through modular, MCP-based tool servers (for analysis, generation, editing, tracking, etc.). Through a hierarchical multi-level memory (global knowledge, task context, and user-specific preferences), UniVA sustains long-horizon reasoning, contextual continuity, and inter-agent communication, enabling interactive and self-reflective video creation with full traceability. This design enables iterative and any-conditioned video workflows (e.g., text/image/video-conditioned generation $\rightarrow$ multi-round editing $\rightarrow$ object segmentation $\rightarrow$ compositional synthesis) that were previously cumbersome to achieve with single-purpose models or monolithic video-language models. We also introduce UniVA-Bench, a benchmark suite of multi-step video tasks spanning understanding, editing, segmentation, and generation, to rigorously evaluate such agentic video systems. Both UniVA and UniVA-Bench are fully open-sourced, aiming to catalyze research on interactive, agentic, and general-purpose video intelligence for the next generation of multimodal AI systems. (this https URL)
• 摘要：虽然专门的人工智能模型擅长生成或理解等独立视频任务，但现实世界的应用程序需要结合这些功能的复杂、迭代的工作流程。为了弥补这一差距，我们引入了 UniVA，这是一个面向下一代视频通才的开源、全能的多代理框架，它将视频理解、分割、编辑和生成统一到有凝聚力的工作流程中。 UniVA 采用计划与行动双代理架构，可驱动高度自动化和主动的工作流程：计划代理解释用户意图并将其分解为结构化视频处理步骤，而执行代理通过模块化、基于 MCP 的工具服务器执行这些步骤（用于分析、生成、编辑、跟踪等）。通过分层多级记忆（全局知识、任务上下文和用户特定偏好），UniVA 维持长视野推理、上下文连续性和代理间通信，从而实现具有完全可追溯性的交互式和自我反思视频创建。这种设计支持迭代和任何条件视频工作流程（例如，文本/图像/视频条件生成$\rightarrow$多轮编辑$\rightarrow$对象分割$\rightarrow$合成合成），而这些工作流程以前用单一用途模型或整体视频语言模型实现起来很麻烦。我们还引入了 UniVA-Bench，这是一个涵盖理解、编辑、分割和生成的多步骤视频任务的基准套件，用于严格评估此类代理视频系统。 UniVA 和 UniVA-Bench 都是完全开源的，旨在促进下一代多模式人工智能系统的交互式、代理和通用视频智能的研究。 （此 https 网址）

Title: 3D4D: An Interactive, Editable, 4D World Model via 3D Video Generation

• Authors: Yunhong He, Zhengqing Yuan, Zhengzhong Tu, Yanfang Ye, Lichao Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.08536
• Pdf URL: https://arxiv.org/pdf/2511.08536
• Copy Paste: [[2511.08536]] 3D4D: An Interactive, Editable, 4D World Model via 3D Video Generation(https://arxiv.org/abs/2511.08536)
• Keywords: generation
• Abstract: We introduce 3D4D, an interactive 4D visualization framework that integrates WebGL with Supersplat rendering. It transforms static images and text into coherent 4D scenes through four core modules and employs a foveated rendering strategy for efficient, real-time multi-modal interaction. This framework enables adaptive, user-driven exploration of complex 4D environments. The project page and code are available at this https URL.
• 摘要：我们介绍 3D4D，这是一种将 WebGL 与 Supersplat 渲染集成的交互式 4D 可视化框架。它通过四个核心模块将静态图像和文本转换为连贯的 4D 场景，并采用注视点渲染策略来实现高效、实时的多模态交互。该框架支持对复杂 4D 环境进行自适应、用户驱动的探索。项目页面和代码可从此 https URL 获取。