2025-11-17

Title: Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning

• Authors: Sirui Liang, Pengfei Cao, Jian Zhao, Cong Huang, Jun Zhao, Kang Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.10707
• Pdf URL: https://arxiv.org/pdf/2511.10707
• Copy Paste: [[2511.10707]] Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning(https://arxiv.org/abs/2511.10707)
• Keywords: generation
• Abstract: Parameter-Efficient finetuning (PEFT) enhances model performance on downstream tasks by updating a minimal subset of parameters. Representation finetuning (ReFT) methods further improve efficiency by freezing model weights and optimizing internal representations with fewer parameters than PEFT, outperforming PEFT on several tasks. However, ReFT exhibits a significant performance decline on mathematical reasoning tasks. To address this problem, the paper demonstrates that ReFT's poor performance on mathematical tasks primarily stems from its struggle to generate effective reasoning prefixes during the early inference phase. Moreover, ReFT disturbs the numerical encoding and the error accumulats during the CoT stage. Based on these observations, this paper proposes Bias-REstrained Prefix Representation FineTuning (BREP ReFT), which enhances ReFT's mathematical reasoning capability by truncating training data to optimize the generation of initial reasoning prefixes, intervening on the early inference stage to prevent error accumulation, and constraining the intervention vectors' magnitude to avoid disturbing numerical encoding. Extensive experiments across diverse model architectures demonstrate BREP's superior effectiveness, efficiency, and robust generalization capability, outperforming both standard ReFT and weight-based PEFT methods on the task of mathematical reasoning. The source code is available at this https URL.
• 摘要：参数高效微调 (PEFT) 通过更新参数的最小子集来增强下游任务的模型性能。表示微调 (ReFT) 方法通过冻结模型权重并使用比 PEFT 更少的参数优化内部表示，进一步提高了效率，在多项任务上优于 PEFT。然而，ReFT 在数学推理任务上表现出显着的性能下降。为了解决这个问题，论文证明 ReFT 在数学任务上表现不佳主要源于其在早期推理阶段难以生成有效的推理前缀。此外，ReFT 会干扰数值编码，并且在 CoT 阶段误差会累积。基于这些观察，本文提出了 Bias-REstrained Prefix Representation FineTuning (BREP ReFT)，它通过截断训练数据来优化初始推理前缀的生成、在早期推理阶段进行干预以防止错误累积、以及限制干预向量的幅度以避免干扰数值编码来增强 ReFT 的数学推理能力。跨不同模型架构的大量实验证明了 BREP 卓越的有效性、效率和强大的泛化能力，在数学推理任务上优于标准 ReFT 和基于权重的 PEFT 方法。源代码可从此 https URL 获取。

Title: Towards Uncertainty Quantification in Generative Model Learning

• Authors: Giorgio Morales, Frederic Jurie, Jalal Fadili
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.10710
• Pdf URL: https://arxiv.org/pdf/2511.10710
• Copy Paste: [[2511.10710]] Towards Uncertainty Quantification in Generative Model Learning(https://arxiv.org/abs/2511.10710)
• Keywords: generative
• Abstract: While generative models have become increasingly prevalent across various domains, fundamental concerns regarding their reliability persist. A crucial yet understudied aspect of these models is the uncertainty quantification surrounding their distribution approximation capabilities. Current evaluation methodologies focus predominantly on measuring the closeness between the learned and the target distributions, neglecting the inherent uncertainty in these measurements. In this position paper, we formalize the problem of uncertainty quantification in generative model learning. We discuss potential research directions, including the use of ensemble-based precision-recall curves. Our preliminary experiments on synthetic datasets demonstrate the effectiveness of aggregated precision-recall curves in capturing model approximation uncertainty, enabling systematic comparison among different model architectures based on their uncertainty characteristics.
• 摘要：尽管生成模型在各个领域变得越来越普遍，但对其可靠性的基本担忧仍然存在。这些模型的一个重要但尚未充分研究的方面是围绕其分布近似能力的不确定性量化。当前的评估方法主要侧重于测量学习分布和目标分布之间的接近程度，忽略了这些测量中固有的不确定性。在这篇立场文件中，我们形式化了生成模型学习中的不确定性量化问题。我们讨论了潜在的研究方向，包括使用基于集成的精确召回曲线。我们对合成数据集的初步实验证明了聚合精确回忆曲线在捕获模型近似不确定性方面的有效性，从而能够根据不同模型架构的不确定性特征对它们进行系统比较。

Title: ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries

• Authors: Tom Yuviler, Dana Drachsler-Cohen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.10855
• Pdf URL: https://arxiv.org/pdf/2511.10855
• Copy Paste: [[2511.10855]] ExPairT-LLM: Exact Learning for LLM Code Selection by Pairwise Queries(https://arxiv.org/abs/2511.10855)
• Keywords: generation
• Abstract: Despite recent advances in LLMs, the task of code generation is still challenging. To cope, code selection algorithms select the best program from multiple programs generated by an LLM. However, existing algorithms can fail to identify the correct program, either because they can misidentify nonequivalent programs or because they rely on an LLM and assume it always correctly determines the output for every input. We present ExPairT-LLM, an exact learning algorithm for code selection that selects a program by posing to an LLM oracle two new types of queries: pairwise membership and pairwise equivalence. These queries are simpler for LLMs and enable ExPairT-LLM to identify the correct program through a tournament, which is robust to some LLM mistakes. We evaluate ExPairT-LLM on four popular code datasets. Its pass@1 (success rate) outperforms the state-of-the-art code selection algorithm on average by +13.0% and up to +27.1%. It also improves the pass@1 of LLMs performing complex reasoning by +24.0%.
• 摘要：尽管法学硕士最近取得了进展，但代码生成的任务仍然具有挑战性。为了应对这种情况，代码选择算法从法学硕士生成的多个程序中选择最佳程序。然而，现有的算法可能无法识别正确的程序，要么是因为它们可能会错误识别不等效的程序，要么是因为它们依赖于 LLM 并假设它总是正确地确定每个输入的输出。我们提出了 ExPairT-LLM，一种用于代码选择的精确学习算法，它通过向 LLM 预言机提出两种新类型的查询来选择程序：成对成员资格和成对等价。这些查询对于 LLM 来说更简单，并且使 ExPairT-LLM 能够通过锦标赛识别正确的程序，这对某些 LLM 错误具有鲁棒性。我们在四个流行的代码数据集上评估 ExPairT-LLM。其 pass@1（成功率）平均优于最先进的代码选择算法 +13.0%，最高可达 +27.1%。它还将执行复杂推理的法学硕士的 pass@1 提高了 24.0%。

Title: Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go

• Authors: Yashshi Pipalani, Hritik Raj, Rajat Ghosh, Vaishnavi Bhargava, Debojyoti Dutta
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.10868
• Pdf URL: https://arxiv.org/pdf/2511.10868
• Copy Paste: [[2511.10868]] Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go(https://arxiv.org/abs/2511.10868)
• Keywords: generation
• Abstract: Training data imbalance poses a major challenge for code LLMs. Most available data heavily over represents raw opensource code while underrepresenting broader software engineering tasks, especially in low resource languages like Golang. As a result, models excel at code autocompletion but struggle with real world developer workflows such as unit test generation. To address this gap, we introduce GO UT Bench, a benchmark dataset of 5264 pairs of code and unit tests, drawn from 10 permissively licensed Golang repositories spanning diverse domain. We evaluate its effectiveness as a fine tuning dataset across two LLM families i.e. mixture of experts and dense decoders. Our results show that finetuned models outperform their base counterparts on more than 75% of benchmark tasks.
• 摘要：训练数据不平衡给代码法学硕士带来了重大挑战。大多数可用数据过多地代表了原始开源代码，而低估了更广泛的软件工程任务，尤其是在像 Golang 这样的低资源语言中。因此，模型在代码自动完成方面表现出色，但在现实世界的开发人员工作流程（例如单元测试生成）中遇到困难。为了解决这一差距，我们引入了 GO UT Bench，这是一个包含 5264 对代码和单元测试的基准数据集，来自跨不同领域的 10 个获得许可的 Golang 存储库。我们评估其作为跨两个 LLM 系列（即专家和密集解码器的混合）的微调数据集的有效性。我们的结果表明，经过微调的模型在超过 75% 的基准任务中表现优于基础模型。

Title: Incorporating Spatial Information into Goal-Conditioned Hierarchical Reinforcement Learning via Graph Representations

• Authors: Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang, Doina Precup
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.10872
• Pdf URL: https://arxiv.org/pdf/2511.10872
• Copy Paste: [[2511.10872]] Incorporating Spatial Information into Goal-Conditioned Hierarchical Reinforcement Learning via Graph Representations(https://arxiv.org/abs/2511.10872)
• Keywords: generation
• Abstract: The integration of graphs with Goal-conditioned Hierarchical Reinforcement Learning (GCHRL) has recently gained attention, as intermediate goals (subgoals) can be effectively sampled from graphs that naturally represent the overall task structure in most RL tasks. However, existing approaches typically rely on domain-specific knowledge to construct these graphs, limiting their applicability to new tasks. Other graph-based approaches create graphs dynamically during exploration but struggle to fully utilize them, because they have problems passing the information in the graphs to newly visited states. Additionally, current GCHRL methods face challenges such as sample inefficiency and poor subgoal representation. This paper proposes a solution to these issues by developing a graph encoder-decoder to evaluate unseen states. Our proposed method, Graph-Guided sub-Goal representation Generation RL (G4RL), can be incorporated into any existing GCHRL method when operating in environments with primarily symmetric and reversible transitions to enhance performance across this class of problems. We show that the graph encoder-decoder can be effectively implemented using a network trained on the state graph generated during exploration. Empirical results indicate that leveraging high and low-level intrinsic rewards from the graph encoder-decoder significantly enhances the performance of state-of-the-art GCHRL approaches with an extra small computational cost in dense and sparse reward environments.
• 摘要：图与目标条件分层强化学习 (GCHRL) 的集成最近引起了人们的关注，因为可以从自然代表大多数 RL 任务中整体任务结构的图中有效地采样中间目标（子目标）。然而，现有方法通常依赖于特定领域的知识来构建这些图，限制了它们对新任务的适用性。其他基于图的方法在探索过程中动态创建图，但很难充分利用它们，因为它们在将图中的信息传递到新访问的状态时存在问题。此外，当前的 GCHRL 方法还面临样本效率低和子目标表示不佳等挑战。本文通过开发图编码器-解码器来评估未见过的状态，提出了解决这些问题的方法。我们提出的方法，图引导子目标表示生成 RL (G4RL)，在主要对称和可逆转换的环境中运行时，可以合并到任何现有的 GCHRL 方法中，以提高此类问题的性能。我们表明，可以使用在探索过程中生成的状态图上训练的网络来有效地实现图编码器-解码器。经验结果表明，利用图编码器-解码器的高水平和低水平内在奖励可以显着提高最先进的 GCHRL 方法的性能，并且在密集和稀疏奖励环境中计算成本极小。

Title: Flow matching-based generative models for MIMO channel estimation

• Authors: Wenkai Liu, Nan Ma, Jianqiao Chen, Xiaoxuan Qi, Yuhang Ma
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.10941
• Pdf URL: https://arxiv.org/pdf/2511.10941
• Copy Paste: [[2511.10941]] Flow matching-based generative models for MIMO channel estimation(https://arxiv.org/abs/2511.10941)
• Keywords: generative
• Abstract: Diffusion model (DM)-based channel estimation, which generates channel samples via a posteriori sampling stepwise with denoising process, has shown potential in high-precision channel state information (CSI) acquisition. However, slow sampling speed is an essential challenge for recent developed DM-based schemes. To alleviate this problem, we propose a novel flow matching (FM)-based generative model for multiple-input multiple-output (MIMO) channel estimation. We first formulate the channel estimation problem within FM framework, where the conditional probability path is constructed from the noisy channel distribution to the true channel distribution. In this case, the path evolves along the straight-line trajectory at a constant speed. Then, guided by this, we derive the velocity field that depends solely on the noise statistics to guide generative models training. Furthermore, during the sampling phase, we utilize the trained velocity field as prior information for channel estimation, which allows for quick and reliable noise channel enhancement via ordinary differential equation (ODE) Euler solver. Finally, numerical results demonstrate that the proposed FM-based channel estimation scheme can significantly reduce the sampling overhead compared to other popular DM-based schemes, such as the score matching (SM)-based scheme. Meanwhile, it achieves superior channel estimation accuracy under different channel conditions.
• 摘要：基于扩散模型（DM）的信道估计通过后验采样和去噪过程逐步生成信道样本，在高精度信道状态信息（CSI）获取方面显示出了潜力。然而，采样速度慢是最近开发的基于 DM 的方案面临的一个重要挑战。为了缓解这个问题，我们提出了一种新颖的基于流匹配（FM）的多输入多输出（MIMO）信道估计生成模型。我们首先在 FM 框架内制定信道估计问题，其中条件概率路径是从噪声信道分布到真实信道分布构建的。在这种情况下，路径以恒定速度沿着直线轨迹演化。然后，以此为指导，我们推导出仅依赖于噪声统计的速度场来指导生成模型训练。此外，在采样阶段，我们利用经过训练的速度场作为信道估计的先验信息，这允许通过常微分方程（ODE）欧拉求解器快速可靠地增强噪声信道。最后，数值结果表明，与其他流行的基于 DM 的方案（例如基于分数匹配（SM）的方案）相比，所提出的基于 FM 的信道估计方案可以显着减少采样开销。同时，在不同信道条件下均实现了优异的信道估计精度。

Title: From Parameter to Representation: A Closed-Form Approach for Controllable Model Merging

• Authors: Jialin Wu, Jian Yang, Handing Wang, Jiajun Wen, Zhiyong Yu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.10943
• Pdf URL: https://arxiv.org/pdf/2511.10943
• Copy Paste: [[2511.10943]] From Parameter to Representation: A Closed-Form Approach for Controllable Model Merging(https://arxiv.org/abs/2511.10943)
• Keywords: generation
• Abstract: Model merging combines expert models for multitask performance but faces challenges from parameter interference. This has sparked recent interest in controllable model merging, giving users the ability to explicitly balance performance trade-offs. Existing approaches employ a compile-then-query paradigm, performing a costly offline multi-objective optimization to enable fast, preference-aware model generation. This offline stage typically involves iterative search or dedicated training, with complexity that grows exponentially with the number of tasks. To overcome these limitations, we shift the perspective from parameter-space optimization to a direct correction of the model's final representation. Our approach models this correction as an optimal linear transformation, yielding a closed-form solution that replaces the entire offline optimization process with a single-step, architecture-agnostic computation. This solution directly incorporates user preferences, allowing a Pareto-optimal model to be generated on-the-fly with complexity that scales linearly with the number of tasks. Experimental results show our method generates a superior Pareto front with more precise preference alignment and drastically reduced computational cost.
• 摘要：模型合并结合了多任务性能的专家模型，但面临参数干扰的挑战。这引发了人们最近对可控模型合并的兴趣，使用户能够明确地平衡性能权衡。现有方法采用先编译后查询的范例，执行成本高昂的离线多目标优化，以实现快速、偏好感知的模型生成。这个离线阶段通常涉及迭代搜索或专门训练，其复杂性随着任务数量呈指数级增长。为了克服这些限制，我们将视角从参数空间优化转向直接修正模型的最终表示。我们的方法将这种校正建模为最佳线性变换，产生封闭式解决方案，用单步、与架构无关的计算取代整个离线优化过程。该解决方案直接结合了用户偏好，允许动态生成帕累托最优模型，其复杂性随任务数量线性扩展。实验结果表明，我们的方法生成了一个优越的帕累托前沿，具有更精确的偏好对齐并大大降低了计算成本。

Title: Language-Guided Graph Representation Learning for Video Summarization

• Authors: Wenrui Li, Wei Han, Hengyu Man, Wangmeng Zuo, Xiaopeng Fan, Yonghong Tian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.10953
• Pdf URL: https://arxiv.org/pdf/2511.10953
• Copy Paste: [[2511.10953]] Language-Guided Graph Representation Learning for Video Summarization(https://arxiv.org/abs/2511.10953)
• Keywords: generation
• Abstract: With the rapid growth of video content on social media, video summarization has become a crucial task in multimedia processing. However, existing methods face challenges in capturing global dependencies in video content and accommodating multimodal user customization. Moreover, temporal proximity between video frames does not always correspond to semantic proximity. To tackle these challenges, we propose a novel Language-guided Graph Representation Learning Network (LGRLN) for video summarization. Specifically, we introduce a video graph generator that converts video frames into a structured graph to preserve temporal order and contextual dependencies. By constructing forward, backward and undirected graphs, the video graph generator effectively preserves the sequentiality and contextual relationships of video content. We designed an intra-graph relational reasoning module with a dual-threshold graph convolution mechanism, which distinguishes semantically relevant frames from irrelevant ones between nodes. Additionally, our proposed language-guided cross-modal embedding module generates video summaries with specific textual descriptions. We model the summary generation output as a mixture of Bernoulli distribution and solve it with the EM algorithm. Experimental results show that our method outperforms existing approaches across multiple benchmarks. Moreover, we proposed LGRLN reduces inference time and model parameters by 87.8% and 91.7%, respectively. Our codes and pre-trained models are available at this https URL.
• 摘要：随着社交媒体上视频内容的快速增长，视频摘要已成为多媒体处理中的关键任务。然而，现有方法在捕获视频内容的全局依赖性和适应多模式用户定制方面面临挑战。此外，视频帧之间的时间接近度并不总是对应于语义接近度。为了应对这些挑战，我们提出了一种新颖的语言引导图表示学习网络（LGRLN）用于视频摘要。具体来说，我们引入了一个视频图生成器，它将视频帧转换为结构化图，以保留时间顺序和上下文依赖性。通过构建前向图、后向图和无向图，视频图生成器有效地保留了视频内容的顺序性和上下文关系。我们设计了一个具有双阈值图卷积机制的图内关系推理模块，该模块可以区分节点之间语义相关的帧和不相关的帧。此外，我们提出的语言引导的跨模式嵌入模块生成带有特定文本描述的视频摘要。我们将摘要生成输出建模为伯努利分布的混合，并使用 EM 算法对其进行求解。实验结果表明，我们的方法在多个基准测试中优于现有方法。此外，我们提出 LGRLN 分别减少了 87.8% 和 91.7% 的推理时间和模型参数。我们的代码和预训练模型可从此 https URL 获取。

Title: Preserving Cross-Modal Consistency for CLIP-based Class-Incremental Learning

• Authors: Haoran Chen, Houze Xu, Micah Goldblum, Daoguo Dong, Zuxuan Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.10974
• Pdf URL: https://arxiv.org/pdf/2511.10974
• Copy Paste: [[2511.10974]] Preserving Cross-Modal Consistency for CLIP-based Class-Incremental Learning(https://arxiv.org/abs/2511.10974)
• Keywords: generative
• Abstract: Class-incremental learning (CIL) enables models to continuously learn new categories from sequential tasks without forgetting previously acquired knowledge. While recent advances in vision-language models such as CLIP have demonstrated strong generalization across domains, extending them to continual settings remains challenging. In particular, learning task-specific soft prompts for newly introduced classes often leads to severe classifier bias, as the text prototypes overfit to recent categories when prior data are unavailable. In this paper, we propose DMC, a simple yet effective two-stage framework for CLIP-based CIL that decouples the adaptation of the vision encoder and the optimization of textual soft prompts. Each stage is trained with the other frozen, allowing one modality to act as a stable semantic anchor for the other to preserve cross-modal alignment. Furthermore, current CLIP-based CIL approaches typically store class-wise Gaussian statistics for generative replay, yet they overlook the distributional drift that arises when the vision encoder is updated over time. To address this issue, we introduce DMC-OT, an enhanced version of DMC that incorporates an optimal-transport guided calibration strategy to align memory statistics across evolving encoders, along with a task-specific prompting design that enhances inter-task separability. Extensive experiments on CIFAR-100, Imagenet-R, CUB-200, and UCF-101 demonstrate that both DMC and DMC-OT achieve state-of-the-art performance, with DMC-OT further improving accuracy by an average of 1.80%.
• 摘要：类增量学习 (CIL) 使模型能够从连续任务中不断学习新类别，而不会忘记以前获得的知识。虽然 CLIP 等视觉语言模型的最新进展已经证明了跨领域的强大泛化性，但将它们扩展到连续设置仍然具有挑战性。特别是，学习新引入的类的特定任务的软提示通常会导致严重的分类器偏差，因为当先前的数据不可用时，文本原型会过度拟合最近的类别。在本文中，我们提出了 DMC，这是一种简单而有效的基于 CLIP 的 CIL 的两阶段框架，它将视觉编码器的适配和文本软提示的优化解耦。每个阶段都与另一个阶段一起进行训练，从而允许一种模态充当另一种模态的稳定语义锚，以保持跨模态对齐。此外，当前基于 CLIP 的 CIL 方法通常存储分类高斯统计数据以进行生成重放，但它们忽略了视觉编码器随时间更新时出现的分布漂移。为了解决这个问题，我们引入了 DMC-OT，这是 DMC 的增强版本，它结合了最佳传输引导校准策略，以跨不断发展的编码器调整内存统计数据，以及增强任务间可分离性的特定于任务的提示设计。在 CIFAR-100、Imagenet-R、CUB-200 和 UCF-101 上进行的大量实验表明，DMC 和 DMC-OT 均实现了最先进的性能，其中 DMC-OT 进一步将准确率平均提高了 1.80%。

Title: CLUE: Controllable Latent space of Unprompted Embeddings for Diversity Management in Text-to-Image Synthesis

• Authors: Keunwoo Park, Jihye Chae, Joong Ho Ahn, Jihoon Kweon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.10993
• Pdf URL: https://arxiv.org/pdf/2511.10993
• Copy Paste: [[2511.10993]] CLUE: Controllable Latent space of Unprompted Embeddings for Diversity Management in Text-to-Image Synthesis(https://arxiv.org/abs/2511.10993)
• Keywords: generation, generative
• Abstract: Text-to-image synthesis models require the ability to generate diverse images while maintaining stability. To overcome this challenge, a number of methods have been proposed, including the collection of prompt-image datasets and the integration of additional data modalities during training. Although these methods have shown promising results in general domains, they face limitations when applied to specialized fields such as medicine, where only limited types and insufficient amounts of data are available. We present CLUE (Controllable Latent space of Unprompted Embeddings), a generative model framework that achieves diverse generation while maintaining stability through fixed-format prompts without requiring any additional data. Based on the Stable Diffusion architecture, CLUE employs a Style Encoder that processes images and prompts to generate style embeddings, which are subsequently fed into a new second attention layer of the U-Net architecture. Through Kullback-Leibler divergence, the latent space achieves continuous representation of image features within Gaussian regions, independent of prompts. Performance was assessed on otitis media dataset. CLUE reduced FID to 9.30 (vs. 46.81) and improved recall to 70.29% (vs. 49.60%). A classifier trained on synthetic-only data at 1000% scale achieved an F1 score of 83.21% (vs. 73.83%). Combining synthetic data with equal amounts of real data achieved an F1 score of 94.76%, higher than when using only real data. On an external dataset, synthetic-only training achieved an F1 score of 76.77% (vs. 60.61%) at 1000% scale. The combined approach achieved an F1 score of 85.78%, higher than when using only the internal dataset. These results demonstrate that CLUE enables diverse yet stable image generation from limited datasets and serves as an effective data augmentation method for domain-specific applications.
• 摘要：文本到图像的合成模型需要能够生成不同的图像，同时保持稳定性。为了克服这一挑战，人们提出了多种方法，包括收集即时图像数据集以及在训练期间集成其他数据模式。尽管这些方法在一般领域显示出了有希望的结果，但当应用于医学等专业领域时，它们面临着局限性，因为在这些领域中，可用的类型有限且数据量不足。我们提出了 CLUE（无提示嵌入的可控潜在空间），这是一种生成模型框架，可以实现多样化生成，同时通过固定格式提示保持稳定性，而不需要任何额外的数据。基于稳定扩散架构，CLUE 采用风格编码器来处理图像并提示生成风格嵌入，随后将其输入到 U-Net 架构的新第二注意力层中。通过 Kullback-Leibler 散度，潜在空间实现了高斯区域内图像特征的连续表示，而与提示无关。在中耳炎数据集上评估性能。 CLUE 将 FID 降低至 9.30（对比 46.81），并将召回率提高至 70.29%（对比 49.60%）。在 1000% 规模的纯合成数据上训练的分类器取得了 83.21% 的 F1 分数（对比 73.83%）。将合成数据与等量的真实数据相结合，F1 分数达到 94.76%，高于仅使用真实数据时的分数。在外部数据集上，仅合成训练在 1000% 规模下的 F1 分数为 76.77%（对比 60.61%）。组合方法的 F1 分数为 85.78%，高于仅使用内部数据集时的分数。这些结果表明，CLUE 能够从有限的数据集中生成多样化但稳定的图像，并可作为特定领域应用的有效数据增强方法。

Title: PROMISE: Prompt-Attentive Hierarchical Contrastive Learning for Robust Cross-Modal Representation with Missing Modalities

• Authors: Jiajun Chen, Sai Cheng, Yutao Yuan, Yirui Zhang, Haitao Yuan, Peng Peng, Yi Zhong
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.10997
• Pdf URL: https://arxiv.org/pdf/2511.10997
• Copy Paste: [[2511.10997]] PROMISE: Prompt-Attentive Hierarchical Contrastive Learning for Robust Cross-Modal Representation with Missing Modalities(https://arxiv.org/abs/2511.10997)
• Keywords: generation
• Abstract: Multimodal models integrating natural language and visual information have substantially improved generalization of representation models. However, their effectiveness significantly declines in real-world situations where certain modalities are missing or unavailable. This degradation primarily stems from inconsistent representation learning between complete multimodal data and incomplete modality scenarios. Existing approaches typically address missing modalities through relatively simplistic generation methods, yet these approaches fail to adequately preserve cross-modal consistency, leading to suboptimal performance. To overcome this limitation, we propose a novel multimodal framework named PROMISE, a PROMpting-Attentive HIerarchical ContraStive LEarning approach designed explicitly for robust cross-modal representation under conditions of missing modalities. Specifically, PROMISE innovatively incorporates multimodal prompt learning into a hierarchical contrastive learning framework, equipped with a specially designed prompt-attention mechanism. This mechanism dynamically generates robust and consistent representations for scenarios where particular modalities are absent, thereby effectively bridging the representational gap between complete and incomplete data. Extensive experiments conducted on benchmark datasets, along with comprehensive ablation studies, clearly demonstrate the superior performance of PROMISE compared to current state-of-the-art multimodal methods.
• 摘要：集成自然语言和视觉信息的多模态模型极大地提高了表示模型​​的泛化能力。然而，在某些模式缺失或不可用的现实情况下，它们的有效性会显着下降。这种退化主要源于完整的多模态数据和不完整的模态场景之间不一致的表示学习。现有方法通常通过相对简单的生成方法来解决缺失的模态，但这些方法无法充分保持跨模态一致性，从而导致性能不佳。为了克服这一限制，我们提出了一种名为 PROMISE 的新型多模态框架，这是一种促进注意的分层对比学习方法，专为在模态缺失的情况下稳健的跨模态表示而设计。具体来说，PROMISE创新地将多模态提示学习融入分层对比学习框架中，并配备了专门设计的提示注意机制。该机制为缺乏特定模态的场景动态生成稳健且一致的表示，从而有效地弥合完整数据和不完整数据之间的表示差距。在基准数据集上进行的大量实验以及全面的消融研究清楚地证明了 PROMISE 与当前最先进的多模态方法相比的优越性能。

Title: EmoVid: A Multimodal Emotion Video Dataset for Emotion-Centric Video Understanding and Generation

• Authors: Zongyang Qiu, Bingyuan Wang, Xingbei Chen, Yingqing He, Zeyu Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11002
• Pdf URL: https://arxiv.org/pdf/2511.11002
• Copy Paste: [[2511.11002]] EmoVid: A Multimodal Emotion Video Dataset for Emotion-Centric Video Understanding and Generation(https://arxiv.org/abs/2511.11002)
• Keywords: generation, generative
• Abstract: Emotion plays a pivotal role in video-based expression, but existing video generation systems predominantly focus on low-level visual metrics while neglecting affective dimensions. Although emotion analysis has made progress in the visual domain, the video community lacks dedicated resources to bridge emotion understanding with generative tasks, particularly for stylized and non-realistic contexts. To address this gap, we introduce EmoVid, the first multimodal, emotion-annotated video dataset specifically designed for creative media, which includes cartoon animations, movie clips, and animated stickers. Each video is annotated with emotion labels, visual attributes (brightness, colorfulness, hue), and text captions. Through systematic analysis, we uncover spatial and temporal patterns linking visual features to emotional perceptions across diverse video forms. Building on these insights, we develop an emotion-conditioned video generation technique by fine-tuning the Wan2.1 model. The results show a significant improvement in both quantitative metrics and the visual quality of generated videos for text-to-video and image-to-video tasks. EmoVid establishes a new benchmark for affective video computing. Our work not only offers valuable insights into visual emotion analysis in artistically styled videos, but also provides practical methods for enhancing emotional expression in video generation.
• 摘要：情感在基于视频的表达中起着关键作用，但现有的视频生成系统主要关注低级视觉指标，而忽略了情感维度。尽管情感分析在视觉领域取得了进展，但视频社区缺乏专门的资源来将情感理解与生成任务联系起来，特别是对于程式化和非现实的环境。为了解决这一差距，我们推出了 EmoVid，这是第一个专为创意媒体设计的多模式、情感注释视频数据集，其中包括卡通动画、影片剪辑和动画贴纸。每个视频都带有情感标签、视觉属性（亮度、色彩、色调）和文本说明。通过系统分析，我们发现了将不同视频形式的视觉特征与情感感知联系起来的空间和时间模式。基于这些见解，我们通过微调 Wan2.1 模型开发了一种情绪调节视频生成技术。结果表明，文本到视频和图像到视频任务的生成视频的定量指标和视觉质量都有显着改善。 EmoVid 为情感视频计算建立了新的基准。我们的工作不仅为艺术风格视频中的视觉情感分析提供了宝贵的见解，而且还提供了增强视频生成中情感表达的实用方法。

Title: VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models

• Authors: Xinlei Yu, Chengming Xu, Guibin Zhang, Zhangquan Chen, Yudong Zhang, Yongbo He, Peng-Tao Jiang, Jiangning Zhang, Xiaobin Hu, Shuicheng Yan
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.11007
• Pdf URL: https://arxiv.org/pdf/2511.11007
• Copy Paste: [[2511.11007]] VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models(https://arxiv.org/abs/2511.11007)
• Keywords: generation
• Abstract: Despite the remarkable success of Vision-Language Models (VLMs), their performance on a range of complex visual tasks is often hindered by a "visual processing bottleneck": a propensity to lose grounding in visual evidence and exhibit a deficit in contextualized visual experience during prolonged generation. Drawing inspiration from human cognitive memory theory, which distinguishes short-term visually-dominant memory and long-term semantically-dominant memory, we propose VisMem, a cognitively-aligned framework that equips VLMs with dynamic latent vision memories, a short-term module for fine-grained perceptual retention and a long-term module for abstract semantic consolidation. These memories are seamlessly invoked during inference, allowing VLMs to maintain both perceptual fidelity and semantic consistency across thinking and generation. Extensive experiments across diverse visual benchmarks for understanding, reasoning, and generation reveal that VisMem delivers a significant average performance boost of 11.8% relative to the vanilla model and outperforms all counterparts, establishing a new paradigm for latent-space memory enhancement. The code will be available: this https URL.
• 摘要：尽管视觉语言模型（VLM）取得了显着的成功，但它们在一系列复杂视觉任务上的表现常常受到“视觉处理瓶颈”的阻碍：在长时间的生成过程中，倾向于失去视觉证据的基础并表现出情境化视觉体验的缺陷。受人类认知记忆理论的启发，该理论区分了短期视觉主导记忆和长期语义主导记忆，我们提出了 VisMem，一种认知对齐框架，为 VLM 配备动态潜在视觉记忆、细粒度感知保留的短期模块和抽象语义巩固的长期模块。这些记忆在推理过程中被无缝调用，使 VLM 能够在思维和生成过程中保持感知保真度和语义一致性。针对理解、推理和生成的各种视觉基准的广泛实验表明，相对于普通模型，VisMem 的平均性能显着提升了 11.8%，并且优于所有同类模型，为潜在空间记忆增强建立了新的范式。代码将可用：此 https URL。

Title: SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generation

• Authors: Sumin Yu, Taesup Moon
• Subjects: cs.CV, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.11014
• Pdf URL: https://arxiv.org/pdf/2511.11014
• Copy Paste: [[2511.11014]] SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generation(https://arxiv.org/abs/2511.11014)
• Keywords: generation
• Abstract: While diffusion-based T2I models have achieved remarkable image generation quality, they also enable easy creation of harmful content, raising social concerns and highlighting the need for safer generation. Existing inference-time guiding methods lack both adaptivity--adjusting guidance strength based on the prompt--and selectivity--targeting only unsafe regions of the image. Our method, SP-Guard, addresses these limitations by estimating prompt harmfulness and applying a selective guidance mask to guide only unsafe areas. Experiments show that SP-Guard generates safer images than existing methods while minimizing unintended content alteration. Beyond improving safety, our findings highlight the importance of transparency and controllability in image generation.
• 摘要：虽然基于扩散的 T2I 模型已经实现了卓越的图像生成质量，但它们也可以轻松创建有害内容，引起社会关注并强调对更安全生成的需求。现有的推理时间引导方法缺乏适应性（根据提示调整引导强度）和选择性（仅针对图像的不安全区域）。我们的方法 SP-Guard 通过估计即时危害并应用选择性引导掩模仅引导不安全区域来解决这些限制。实验表明，SP-Guard 生成比现有方法更安全的图像，同时最大限度地减少意外的内容更改。除了提高安全性之外，我们的研究结果还强调了图像生成中透明度和可控性的重要性。

Title: Accelerating Controllable Generation via Hybrid-grained Cache

• Authors: Lin Liu, Huixia Ben, Shuo Wang, Jinda Lu, Junxiang Qiu, Shengeng Tang, Yanbin Hao
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2511.11031
• Pdf URL: https://arxiv.org/pdf/2511.11031
• Copy Paste: [[2511.11031]] Accelerating Controllable Generation via Hybrid-grained Cache(https://arxiv.org/abs/2511.11031)
• Keywords: generation, generative
• Abstract: Controllable generative models have been widely used to improve the realism of synthetic visual content. However, such models must handle control conditions and content generation computational requirements, resulting in generally low generation efficiency. To address this issue, we propose a Hybrid-Grained Cache (HGC) approach that reduces computational overhead by adopting cache strategies with different granularities at different computational stages. Specifically, (1) we use a coarse-grained cache (block-level) based on feature reuse to dynamically bypass redundant computations in encoder-decoder blocks between each step of model reasoning. (2) We design a fine-grained cache (prompt-level) that acts within a module, where the fine-grained cache reuses cross-attention maps within consecutive reasoning steps and extends them to the corresponding module computations of adjacent steps. These caches of different granularities can be seamlessly integrated into each computational link of the controllable generation process. We verify the effectiveness of HGC on four benchmark datasets, especially its advantages in balancing generation efficiency and visual quality. For example, on the COCO-Stuff segmentation benchmark, our HGC significantly reduces the computational cost (MACs) by 63% (from 18.22T to 6.70T), while keeping the loss of semantic fidelity (quantized performance degradation) within 1.5%.
• 摘要：可控生成模型已被广泛用于提高合成视觉内容的真实感。然而，此类模型必须处理控制条件和内容生成计算要求，导致生成效率普遍较低。为了解决这个问题，我们提出了一种混合粒度缓存（HGC）方法，通过在不同计算阶段采用不同粒度的缓存策略来减少计算开销。具体来说，（1）我们使用基于特征重用的粗粒度缓存（块级）来动态绕过模型推理的每个步骤之间编码器-解码器块中的冗余计算。 （2）我们设计了一个在模块内起作用的细粒度缓存（提示级），其中细粒度缓存在连续推理步骤中重用交叉注意力图，并将它们扩展到相邻步骤的相应模块计算。这些不同粒度的缓存可以无缝集成到可控生成过程的各个计算环节中。我们在四个基准数据集上验证了 HGC 的有效性，特别是它在平衡生成效率和视觉质量方面的优势。例如，在 COCO-Stuff 分割基准上，我们的 HGC 将计算成本 (MAC) 显着降低了 63%（从 18.22T 到 6.70T），同时将语义保真度损失（量化性能下降）控制在 1.5% 以内。

Title: PINGS-X: Physics-Informed Normalized Gaussian Splatting with Axes Alignment for Efficient Super-Resolution of 4D Flow MRI

• Authors: Sun Jo, Seok Young Hong, JinHyun Kim, Seungmin Kang, Ahjin Choi, Don-Gwan An, Simon Song, Je Hyeong Hong
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.11048
• Pdf URL: https://arxiv.org/pdf/2511.11048
• Copy Paste: [[2511.11048]] PINGS-X: Physics-Informed Normalized Gaussian Splatting with Axes Alignment for Efficient Super-Resolution of 4D Flow MRI(https://arxiv.org/abs/2511.11048)
• Keywords: super-resolution
• Abstract: 4D flow magnetic resonance imaging (MRI) is a reliable, non-invasive approach for estimating blood flow velocities, vital for cardiovascular diagnostics. Unlike conventional MRI focused on anatomical structures, 4D flow MRI requires high spatiotemporal resolution for early detection of critical conditions such as stenosis or aneurysms. However, achieving such resolution typically results in prolonged scan times, creating a trade-off between acquisition speed and prediction accuracy. Recent studies have leveraged physics-informed neural networks (PINNs) for super-resolution of MRI data, but their practical applicability is limited as the prohibitively slow training process must be performed for each patient. To overcome this limitation, we propose PINGS-X, a novel framework modeling high-resolution flow velocities using axes-aligned spatiotemporal Gaussian representations. Inspired by the effectiveness of 3D Gaussian splatting (3DGS) in novel view synthesis, PINGS-X extends this concept through several non-trivial novel innovations: (i) normalized Gaussian splatting with a formal convergence guarantee, (ii) axes-aligned Gaussians that simplify training for high-dimensional data while preserving accuracy and the convergence guarantee, and (iii) a Gaussian merging procedure to prevent degenerate solutions and boost computational efficiency. Experimental results on computational fluid dynamics (CFD) and real 4D flow MRI datasets demonstrate that PINGS-X substantially reduces training time while achieving superior super-resolution accuracy. Our code and datasets are available at this https URL.
• 摘要：4D 血流磁共振成像 (MRI) 是一种可靠的非侵入性方法，用于估计血流速度，对于心血管诊断至关重要。与专注于解剖结构的传统 MRI 不同，4D 血流 MRI 需要高时空分辨率，以便及早发现狭窄或动脉瘤等危急情况。然而，实现这种分辨率通常会导致扫描时间延长，从而在采集速度和预测精度之间产生权衡。最近的研究利用物理信息神经网络 (PINN) 来实现 MRI 数据的超分辨率，但其实际适用性受到限制，因为必须对每个患者进行极其缓慢的训练过程。为了克服这一限制，我们提出了 PINGS-X，这是一种使用轴对齐时空高斯表示来建模高分辨率流速的新颖框架。受到 3D 高斯分布 (3DGS) 在新颖视图合成中的有效性的启发，PINGS-X 通过几项重要的新颖创新扩展了这一概念：(i) 具有形式收敛保证的归一化高斯分布，(ii) 轴对齐高斯，可简化高维数据的训练，同时保持准确性和收敛保证，以及 (iii) 高斯合并程序，以防止退化解决方案并提高计算效率。计算流体动力学 (CFD) 和真实 4D 流 MRI 数据集的实验结果表明，PINGS-X 大大减少了训练时间，同时实现了卓越的超分辨率精度。我们的代码和数据集可在此 https URL 获取。

Title: NP-LoRA: Null Space Projection Unifies Subject and Style in LoRA Fusion

• Authors: Chuheng Chen, Xiaofei Zhou, Geyuan Zhang, Yong Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11051
• Pdf URL: https://arxiv.org/pdf/2511.11051
• Copy Paste: [[2511.11051]] NP-LoRA: Null Space Projection Unifies Subject and Style in LoRA Fusion(https://arxiv.org/abs/2511.11051)
• Keywords: generation, generative
• Abstract: Low-Rank Adaptation (LoRA) fusion has emerged as a key technique for reusing and composing learned subject and style representations for controllable generation without costly retraining. However, existing methods rely on weight-based merging, where one LoRA often dominates the other, leading to interference and degraded fidelity. This interference is structural: separately trained LoRAs occupy low-rank high-dimensional subspaces, leading to non-orthogonal and overlapping representations. In this work, we analyze the internal structure of LoRAs and find their generative behavior is dominated by a few principal directions in the low-rank subspace, which should remain free from interference during fusion. To achieve this, we propose Null Space Projection LoRA (NP-LoRA), a projection-based framework for LoRA fusion that enforces subspace separation to prevent structural interference among principal directions. Specifically, we first extract principal style directions via singular value decomposition (SVD) and then project the subject LoRA into its orthogonal null space. Furthermore, we introduce a soft projection mechanism that enables smooth control over the trade-off between subject fidelity and style consistency. Experiments show NP-LoRA consistently improves fusion quality over strong baselines (e.g., DINO and CLIP-based metrics, with human and LLM preference scores), and applies broadly across backbones and LoRA pairs without retraining.
• 摘要：低秩适应（LoRA）融合已成为重用和组合学习的主题和风格表示的关键技术，以实现可控生成，而无需昂贵的再训练。然而，现有方法依赖于基于权重的合并，其中一种 LoRA 常常支配另一种，导致干扰和保真度下降。这种干扰是结构性的：单独训练的 LoRA 占据低秩高维子空间，导致非正交和重叠表示。在这项工作中，我们分析了 LoRA 的内部结构，发现它们的生成行为由低秩子空间中的几个主要方向主导，这些方向在融合过程中应该不受干扰。为了实现这一目标，我们提出了零空间投影 LoRA (NP-LoRA)，这是一种基于投影的 LoRA 融合框架，可强制子空间分离以防止主方向之间的结构干扰。具体来说，我们首先通过奇异值分解（SVD）提取主要风格方向，然后将主题 LoRA 投影到其正交零空间中。此外，我们引入了一种软投影机制，可以平滑控制主题保真度和风格一致性之间的权衡。实验表明，NP-LoRA 在强大的基线（例如，基于 DINO 和 CLIP 的指标，以及人类和 LLM 偏好分数）上持续提高融合质量，并且广泛应用于骨干网和 LoRA 对，无需重新训练。

Title: CareCom: Generative Image Composition with Calibrated Reference Features

• Authors: Jiaxuan Chen, Bo Zhang, Qingdong He, Jinlong Peng, Li Niu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11060
• Pdf URL: https://arxiv.org/pdf/2511.11060
• Copy Paste: [[2511.11060]] CareCom: Generative Image Composition with Calibrated Reference Features(https://arxiv.org/abs/2511.11060)
• Keywords: generative
• Abstract: Image composition aims to seamlessly insert foreground object into background. Despite the huge progress in generative image composition, the existing methods are still struggling with simultaneous detail preservation and foreground pose/view adjustment. To address this issue, we extend the existing generative composition model to multi-reference version, which allows using arbitrary number of foreground reference images. Furthermore, we propose to calibrate the global and local features of foreground reference images to make them compatible with the background information. The calibrated reference features can supplement the original reference features with useful global and local information of proper pose/view. Extensive experiments on MVImgNet and MureCom demonstrate that the generative model can greatly benefit from the calibrated reference features.
• 摘要：图像合成的目的是将前景对象无缝插入背景中。尽管生成图像合成取得了巨大进步，但现有方法仍在同时处理细节保留和前景姿势/视图调整方面遇到困难。为了解决这个问题，我们将现有的生成合成模型扩展到多参考版本，它允许使用任意数量的前景参考图像。此外，我们建议校准前景参考图像的全局和局部特征，以使它们与背景信息兼容。校准的参考特征可以用正确姿势/视图的有用的全局和局部信息来补充原始参考特征。 MVImgNet 和 MureCom 上的大量实验表明，生成模型可以极大地受益于校准的参考特征。

Title: LiteAttention: A Temporal Sparse Attention for Diffusion Transformers

• Authors: Dor Shmilovich, Tony Wu, Aviad Dahan, Yuval Domb
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.11062
• Pdf URL: https://arxiv.org/pdf/2511.11062
• Copy Paste: [[2511.11062]] LiteAttention: A Temporal Sparse Attention for Diffusion Transformers(https://arxiv.org/abs/2511.11062)
• Keywords: generation
• Abstract: Diffusion Transformers, particularly for video generation, achieve remarkable quality but suffer from quadratic attention complexity, leading to prohibitive latency. Existing acceleration methods face a fundamental trade-off: dynamically estimating sparse attention patterns at each denoising step incurs high computational overhead and estimation errors, while static sparsity patterns remain fixed and often suboptimal throughout denoising. We identify a key structural property of diffusion attention, namely, its sparsity patterns exhibit strong temporal coherence across denoising steps. Tiles deemed non-essential at step $t$ typically remain so at step $t+\delta$. Leveraging this observation, we introduce LiteAttention, a method that exploits temporal coherence to enable evolutionary computation skips across the denoising sequence. By marking non-essential tiles early and propagating skip decisions forward, LiteAttention eliminates redundant attention computations without repeated profiling overheads, combining the adaptivity of dynamic methods with the efficiency of static ones. We implement a highly optimized LiteAttention kernel on top of FlashAttention and demonstrate substantial speedups on production video diffusion models, with no degradation in quality. The code and implementation details will be publicly released.
• 摘要：Diffusion Transformer，特别是用于视频生成的，实现了卓越的质量，但受到二次注意力复杂性的影响，导致令人望而却步的延迟。现有的加速方法面临着一个基本的权衡：在每个去噪步骤中动态估计稀疏注意力模式会产生高计算开销和估计误差，而静态稀疏模式在整个去噪过程中保持固定并且通常不是最优的。我们确定了扩散注意力的一个关键结构特性，即其稀疏模式在去噪步骤中表现出很强的时间一致性。在步骤 $t$ 中被视为非必需的图块通常在步骤 $t+\delta$ 中仍然如此。利用这一观察结果，我们引入了 LiteAttention，一种利用时间相干性使进化计算能够跳过去噪序列的方法。通过尽早标记非必要的图块并向前传播跳过决策，LiteAttention 消除了冗余的注意力计算，而无需重复的分析开销，将动态方法的适应性与静态方法的效率结合起来。我们在 FlashAttention 之上实现了高度优化的 LiteAttention 内核，并展示了生产视频扩散模型的显着加速，而质量没有下降。代码和实施细节将公开发布。

Title: S2D-ALIGN: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation

• Authors: Jiechao Gao, Chang Liu, Yuangang Li
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2511.11066
• Pdf URL: https://arxiv.org/pdf/2511.11066
• Copy Paste: [[2511.11066]] S2D-ALIGN: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation(https://arxiv.org/abs/2511.11066)
• Keywords: generation
• Abstract: Radiology Report Generation (RRG) aims to automatically generate diagnostic reports from radiology images. To achieve this, existing methods have leveraged the powerful cross-modal generation capabilities of Multimodal Large Language Models (MLLMs), primarily focusing on optimizing cross-modal alignment between radiographs and reports through Supervised Fine-Tuning (SFT). However, by only performing instance-level alignment with the image-text pairs, the standard SFT paradigm fails to establish anatomically-grounded alignment, where the templated nature of reports often leads to sub-optimal generation quality. To address this, we propose \textsc{S2D-Align}, a novel SFT paradigm that establishes anatomically-grounded alignment by leveraging auxiliary signals of varying granularities. \textsc{S2D-Align} implements a shallow-to-deep strategy, progressively enriching the alignment process: it begins with the coarse radiograph-report pairing, then introduces reference reports for instance-level guidance, and ultimately utilizes key phrases to ground the generation in specific anatomical details. To bridge the different alignment stages, we introduce a memory-based adapter that empowers feature sharing, thereby integrating coarse and fine-grained guidance. For evaluation, we conduct experiments on the public \textsc{MIMIC-CXR} and \textsc{IU X-Ray} benchmarks, where \textsc{S2D-Align} achieves state-of-the-art performance compared to existing methods. Ablation studies validate the effectiveness of our multi-stage, auxiliary-guided approach, highlighting a promising direction for enhancing grounding capabilities in complex, multi-modal generation tasks.
• 摘要：放射学报告生成（RRG）旨在从放射学图像自动生成诊断报告。为了实现这一目标，现有方法利用了多模态大语言模型（MLLM）强大的跨模态生成功能，主要侧重于通过监督微调（SFT）来优化射线照片和报告之间的跨模态对齐。然而，通过仅使用图像-文本对执行实例级对齐，标准 SFT 范例无法建立基于解剖学的对齐，其中报告的模板化性质通常会导致次优的生成质量。为了解决这个问题，我们提出了 \textsc{S2D-Align}，这是一种新颖的 SFT 范例，它通过利用不同粒度的辅助信号来建立基于解剖学的对齐。 \textsc{S2D-Align} 实现了由浅到深的策略，逐步丰富对齐过程：它从粗略的放射线照片-报告配对开始，然后引入实例级指导的参考报告，并最终利用关键短语将生成基于特定的解剖细节。为了桥接不同的对齐阶段，我们引入了基于内存的适配器，该适配器支持功能共享，从而集成粗粒度和细粒度的指导。为了进行评估，我们在公共 \textsc{MIMIC-CXR} 和 \textsc{IU X-Ray} 基准上进行了实验，其中 \textsc{S2D-Align} 与现有方法相比实现了最先进的性能。消融研究验证了我们的多阶段辅助引导方法的有效性，突出了在复杂的多模式生成任务中增强接地能力的有希望的方向。

Title: Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image

• Authors: Matthias Humt, Ulrich Hillenbrand, Rudolph Triebel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11074
• Pdf URL: https://arxiv.org/pdf/2511.11074
• Copy Paste: [[2511.11074]] Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image(https://arxiv.org/abs/2511.11074)
• Keywords: generation, generative
• Abstract: While generative models have seen significant adoption across a wide range of data modalities, including 3D data, a consensus on which model is best suited for which task has yet to be reached. Further, conditional information such as text and images to steer the generation process are frequently employed, whereas others, like partial 3D data, have not been thoroughly evaluated. In this work, we compare two of the most promising generative models--Denoising Diffusion Probabilistic Models and Autoregressive Causal Transformers--which we adapt for the tasks of generative shape modeling and completion. We conduct a thorough quantitative evaluation and comparison of both tasks, including a baseline discriminative model and an extensive ablation study. Our results show that (1) the diffusion model with continuous latents outperforms both the discriminative model and the autoregressive approach and delivers state-of-the-art performance on multi-modal shape completion from a single, noisy depth image under realistic conditions and (2) when compared on the same discrete latent space, the autoregressive model can match or exceed diffusion performance on these tasks.
• 摘要：虽然生成模型已在包括 3D 数据在内的各种数据模式中得到广泛采用，但尚未就哪种模型最适合哪种任务达成共识。此外，经常使用文本和图像等条件信息来引导生成过程，而其他信息（例如部分 3D 数据）尚未得到彻底评估。在这项工作中，我们比较了两种最有前途的生成模型——去噪扩散概率模型和自回归因果变换器——我们将其适应生成形状建模和完成的任务。我们对这两项任务进行了彻底的定量评估和比较，包括基线判别模型和广泛的消融研究。我们的结果表明，（1）具有连续潜伏的扩散模型优于判别模型和自回归方法，并且在现实条件下从单个噪声深度图像完成多模态形状方面提供了最先进的性能；（2）在相同的离散潜伏空间上进行比较时，自回归模型可以匹配或超过这些任务的扩散性能。

Title: Phys-Liquid: A Physics-Informed Dataset for Estimating 3D Geometry and Volume of Transparent Deformable Liquids

• Authors: Ke Ma, Yizhou Fang, Jean-Baptiste Weibel, Shuai Tan, Xinggang Wang, Yang Xiao, Yi Fang, Tian Xia
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2511.11077
• Pdf URL: https://arxiv.org/pdf/2511.11077
• Copy Paste: [[2511.11077]] Phys-Liquid: A Physics-Informed Dataset for Estimating 3D Geometry and Volume of Transparent Deformable Liquids(https://arxiv.org/abs/2511.11077)
• Keywords: generation
• Abstract: Estimating the geometric and volumetric properties of transparent deformable liquids is challenging due to optical complexities and dynamic surface deformations induced by container movements. Autonomous robots performing precise liquid manipulation tasks, such as dispensing, aspiration, and mixing, must handle containers in ways that inevitably induce these deformations, complicating accurate liquid state assessment. Current datasets lack comprehensive physics-informed simulation data representing realistic liquid behaviors under diverse dynamic scenarios. To bridge this gap, we introduce Phys-Liquid, a physics-informed dataset comprising 97,200 simulation images and corresponding 3D meshes, capturing liquid dynamics across multiple laboratory scenes, lighting conditions, liquid colors, and container rotations. To validate the realism and effectiveness of Phys-Liquid, we propose a four-stage reconstruction and estimation pipeline involving liquid segmentation, multi-view mask generation, 3D mesh reconstruction, and real-world scaling. Experimental results demonstrate improved accuracy and consistency in reconstructing liquid geometry and volume, outperforming existing benchmarks. The dataset and associated validation methods facilitate future advancements in transparent liquid perception tasks. The dataset and code are available at this https URL.
• 摘要：由于光学复杂性和容器运动引起的动态表面变形，估计透明可变形液体的几何和体积特性具有挑战性。执行精确液体操作任务（例如分配、抽吸和混合）的自主机器人必须以不可避免地引起这些变形的方式处理容器，从而使准确的液体状态评估复杂化。当前的数据集缺乏全面的物理模拟数据，无法代表不同动态场景下的真实液体行为。为了弥补这一差距，我们引入了 Phys-Liquid，这是一个物理信息数据集，包含 97,200 个模拟图像和相应的 3D 网格，捕获多个实验室场景、照明条件、液体颜色和容器旋转的液体动力学。为了验证 Phys-Liquid 的真实性和有效性，我们提出了一个四阶段重建和估计管道，涉及液体分割、多视图掩模生成、3D 网格重建和真实世界缩放。实验结果表明，重建液体几何形状和体积的准确性和一致性有所提高，优于现有基准。该数据集和相关的验证方法有助于透明液体感知任务的未来进步。数据集和代码可从此 https URL 获取。

Title: Machine-Learning Based Detection of Coronary Artery Calcification Using Synthetic Chest X-Rays

• Authors: Dylan Saeed, Ramtin Gharleghi, Susann Bier, Sonit Singh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11093
• Pdf URL: https://arxiv.org/pdf/2511.11093
• Copy Paste: [[2511.11093]] Machine-Learning Based Detection of Coronary Artery Calcification Using Synthetic Chest X-Rays(https://arxiv.org/abs/2511.11093)
• Keywords: super-resolution
• Abstract: Coronary artery calcification (CAC) is a strong predictor of cardiovascular events, with CT-based Agatston scoring widely regarded as the clinical gold standard. However, CT is costly and impractical for large-scale screening, while chest X-rays (CXRs) are inexpensive but lack reliable ground truth labels, constraining deep learning development. Digitally reconstructed radiographs (DRRs) offer a scalable alternative by projecting CT volumes into CXR-like images while inheriting precise labels. In this work, we provide the first systematic evaluation of DRRs as a surrogate training domain for CAC detection. Using 667 CT scans from the COCA dataset, we generate synthetic DRRs and assess model capacity, super-resolution fidelity enhancement, preprocessing, and training strategies. Lightweight CNNs trained from scratch outperform large pretrained networks; pairing super-resolution with contrast enhancement yields significant gains; and curriculum learning stabilises training under weak supervision. Our best configuration achieves a mean AUC of 0.754, comparable to or exceeding prior CXR-based studies. These results establish DRRs as a scalable, label-rich foundation for CAC detection, while laying the foundation for future transfer learning and domain adaptation to real CXRs.
• 摘要：冠状动脉钙化 (CAC) 是心血管事件的有力预测因子，基于 CT 的 Agatston 评分被广泛视为临床金标准。然而，CT 成本高昂且不适合大规模筛查，而胸部 X 光检查 (CXR) 价格低廉但缺乏可靠的地面真实标签，限制了深度学习的发展。数字重建放射线照片 (DRR) 通过将 CT 体积投影到类似 CXR 的图像中，同时继承精确的标签，提供了一种可扩展的替代方案。在这项工作中，我们首次对 DRR 作为 CAC 检测的替代训练域进行系统评估。使用 COCA 数据集中的 667 个 CT 扫描，我们生成合成 DRR 并评估模型容量、超分辨率保真度增强、预处理和训练策略。从头开始训练的轻量级 CNN 优于大型预训练网络；将超分辨率与对比度增强相结合可产生显着的效果；课程学习在弱监督下稳定训练。我们的最佳配置实现了 0.754 的平均 AUC，相当于或超过了之前基于 CXR 的研究。这些结果将 DRR 确立为 CAC 检测的可扩展、标签丰富的基础，同时为未来的迁移学习和领域适应真实 CXR 奠定了基础。

Title: Toward Generalized Detection of Synthetic Media: Limitations, Challenges, and the Path to Multimodal Solutions

• Authors: Redwan Hussain, Mizanur Rahman, Prithwiraj Bhattacharjee
• Subjects: cs.CV, cs.NE
• Abstract URL: https://arxiv.org/abs/2511.11116
• Pdf URL: https://arxiv.org/pdf/2511.11116
• Copy Paste: [[2511.11116]] Toward Generalized Detection of Synthetic Media: Limitations, Challenges, and the Path to Multimodal Solutions(https://arxiv.org/abs/2511.11116)
• Keywords: generation, generative
• Abstract: Artificial intelligence (AI) in media has advanced rapidly over the last decade. The introduction of Generative Adversarial Networks (GANs) improved the quality of photorealistic image generation. Diffusion models later brought a new era of generative media. These advances made it difficult to separate real and synthetic content. The rise of deepfakes demonstrated how these tools could be misused to spread misinformation, political conspiracies, privacy violations, and fraud. For this reason, many detection models have been developed. They often use deep learning methods such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). These models search for visual, spatial, or temporal anomalies. However, such approaches often fail to generalize across unseen data and struggle with content from different models. In addition, existing approaches are ineffective in multimodal data and highly modified content. This study reviews twenty-four recent works on AI-generated media detection. Each study was examined individually to identify its contributions and weaknesses, respectively. The review then summarizes the common limitations and key challenges faced by current approaches. Based on this analysis, a research direction is suggested with a focus on multimodal deep learning models. Such models have the potential to provide more robust and generalized detection. It offers future researchers a clear starting point for building stronger defenses against harmful synthetic media.
• 摘要：媒体领域的人工智能 (AI) 在过去十年中发展迅速。生成对抗网络（GAN）的引入提高了逼真图像生成的质量。扩散模型后来带来了生成媒体的新时代。这些进步使得区分真实内容和合成内容变得困难。深度造假的兴起表明，这些工具可能被滥用来传播错误信息、政治阴谋、侵犯隐私和欺诈。为此，开发了许多检测模型。他们经常使用深度学习方法，例如卷积神经网络（CNN）和视觉变换器（ViT）。这些模型搜索视觉、空间或时间异常。然而，此类方法通常无法泛化看不见的数据，并且难以处理来自不同模型的内容。此外，现有方法对于多模式数据和高度修改的内容无效。本研究回顾了最近二十四篇关于人工智能生成媒体检测的工作。每项研究都经过单独审查，分别确定其贡献和弱点。然后，该评论总结了当前方法面临的常见局限性和主要挑战。基于此分析，提出了一个以多模态深度学习模型为重点的研究方向。此类模型有可能提供更稳健和更通用的检测。它为未来的研究人员提供了一个明确的起点，以建立针对有害合成介质的更强大的防御措施。

Title: Stroke Modeling Enables Vectorized Character Generation with Large Vectorized Glyph Model

• Authors: Xinyue Zhang, Haolong Li, Jiawei Ma, Chen Ye
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11119
• Pdf URL: https://arxiv.org/pdf/2511.11119
• Copy Paste: [[2511.11119]] Stroke Modeling Enables Vectorized Character Generation with Large Vectorized Glyph Model(https://arxiv.org/abs/2511.11119)
• Keywords: generation
• Abstract: Vectorized glyphs are widely used in poster design, network animation, art display, and various other fields due to their scalability and flexibility. In typography, they are often seen as special sequences composed of ordered strokes. This concept extends to the token sequence prediction abilities of large language models (LLMs), enabling vectorized character generation through stroke modeling. In this paper, we propose a novel Large Vectorized Glyph Model (LVGM) designed to generate vectorized Chinese glyphs by predicting the next stroke. Initially, we encode strokes into discrete latent variables called stroke embeddings. Subsequently, we train our LVGM via fine-tuning DeepSeek LLM by predicting the next stroke embedding. With limited strokes given, it can generate complete characters, semantically elegant words, and even unseen verses in vectorized form. Moreover, we release a new large-scale Chinese SVG dataset containing 907,267 samples based on strokes for dynamically vectorized glyph generation. Experimental results show that our model has scaling behaviors on data scales. Our generated vectorized glyphs have been validated by experts and relevant individuals.
• 摘要：矢量字形因其可扩展性和灵活性而被广泛应用于海报设计、网络动画、艺术展示等各个领域。在印刷术中，它们通常被视为由有序笔画组成的特殊序列。这一概念扩展到大型语言模型 (LLM) 的标记序列预测能力，从而通过笔画建模实现矢量化字符生成。在本文中，我们提出了一种新颖的大型矢量化字形模型（LVGM），旨在通过预测下一个笔画来生成矢量化中文字形。最初，我们将笔画编码为离散的潜在变量，称为笔画嵌入。随后，我们通过预测下一个笔画嵌入来微调 DeepSeek LLM 来训练 LVGM。给定有限的笔画，它可以生成完整的汉字、语义优美的单词，甚至矢量化形式的看不见的诗句。此外，我们还发布了一个新的大规模中文 SVG 数据集，其中包含基于笔划的 907,267 个样本，用于动态矢量化字形生成。实验结果表明我们的模型在数据尺度上具有缩放行为。我们生成的矢量化字形已经过专家和相关人员的验证。

Title: Hindsight Distillation Reasoning with Knowledge Encouragement Preference for Knowledge-based Visual Question Answering

• Authors: Yu Zhao, Ying Zhang, Xuhui Sui, Baohang Zhou, Li Shen, Dacheng Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11132
• Pdf URL: https://arxiv.org/pdf/2511.11132
• Copy Paste: [[2511.11132]] Hindsight Distillation Reasoning with Knowledge Encouragement Preference for Knowledge-based Visual Question Answering(https://arxiv.org/abs/2511.11132)
• Keywords: generation
• Abstract: Knowledge-based Visual Question Answering (KBVQA) necessitates external knowledge incorporation beyond cross-modal understanding. Existing KBVQA methods either utilize implicit knowledge in multimodal large language models (MLLMs) via in-context learning or explicit knowledge via retrieval augmented generation. However, their reasoning processes remain implicit, without explicit multi-step trajectories from MLLMs. To address this gap, we provide a Hindsight Distilled Reasoning (HinD) framework with Knowledge Encouragement Preference Optimization (KEPO), designed to elicit and harness internal knowledge reasoning ability in MLLMs. First, to tackle the reasoning supervision problem, we propose to emphasize the hindsight wisdom of MLLM by prompting a frozen 7B-size MLLM to complete the reasoning process between the question and its ground truth answer, constructing Hindsight-Zero training data. Then we self-distill Hindsight-Zero into Chain-of-Thought (CoT) Generator and Knowledge Generator, enabling the generation of sequential steps and discrete facts. Secondly, to tackle the misalignment between knowledge correctness and confidence, we optimize the Knowledge Generator with KEPO, preferring under-confident but helpful knowledge over the over-confident but unhelpful one. The generated CoT and sampled knowledge are then exploited for answer prediction. Experiments on OK-VQA and A-OKVQA validate the effectiveness of HinD, showing that HinD with elicited reasoning from 7B-size MLLM achieves superior performance without commercial model APIs or outside knowledge.
• 摘要：基于知识的视觉问答（KBVQA）需要超越跨模式理解的外部知识整合。现有的 KBVQA 方法要么通过上下文学习利用多模态大语言模型 (MLLM) 中的隐性知识，要么通过检索增强生成利用显性知识。然而，它们的推理过程仍然是隐式的，没有来自 MLLM 的显式多步骤轨迹。为了解决这一差距，我们提供了一个具有知识鼓励偏好优化 (KEPO) 的事后蒸馏推理 (HinD) 框架，旨在激发和利用 MLLM 的内部知识推理能力。首先，为了解决推理监督问题，我们建议通过促使冻结的 7B 大小 MLLM 完成问题与其真实答案之间的推理过程，构建后见之零训练数据来强调 MLLM 的后见之明。然后，我们将 Hindsight-Zero 自我提炼为思想链 (CoT) 生成器和知识生成器，从而能够生成连续步骤和离散事实。其次，为了解决知识正确性和置信度之间的不一致问题，我们用 KEPO 优化知识生成器，优先选择信心不足但有用的知识，而不是过度自信但无用的知识。然后利用生成的 CoT 和采样知识进行答案预测。 OK-VQA 和 A-OKVQA 的实验验证了 HinD 的有效性，表明 HinD 通过 7B 大小的 MLLM 进行推理，无需商业模型 API 或外部知识即可实现卓越的性能。

Title: Adaptive Symmetrization of the KL Divergence

• Authors: Omri Ben-Dov, Luiz F.O. Chamon
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.11159
• Pdf URL: https://arxiv.org/pdf/2511.11159
• Copy Paste: [[2511.11159]] Adaptive Symmetrization of the KL Divergence(https://arxiv.org/abs/2511.11159)
• Keywords: generation, generative
• Abstract: Many tasks in machine learning can be described as or reduced to learning a probability distribution given a finite set of samples. A common approach is to minimize a statistical divergence between the (empirical) data distribution and a parameterized distribution, e.g., a normalizing flow (NF) or an energy-based model (EBM). In this context, the forward KL divergence is a ubiquitous due to its tractability, though its asymmetry may prevent capturing some properties of the target distribution. Symmetric alternatives involve brittle min-max formulations and adversarial training (e.g., generative adversarial networks) or evaluating the reverse KL divergence, as is the case for the symmetric Jeffreys divergence, which is challenging to compute from samples. This work sets out to develop a new approach to minimize the Jeffreys divergence. To do so, it uses a proxy model whose goal is not only to fit the data, but also to assist in optimizing the Jeffreys divergence of the main model. This joint training task is formulated as a constrained optimization problem to obtain a practical algorithm that adapts the models priorities throughout training. We illustrate how this framework can be used to combine the advantages of NFs and EBMs in tasks such as density estimation, image generation, and simulation-based inference.
• 摘要：机器学习中的许多任务可以被描述为或简化为学习给定有限样本集的概率分布。一种常见的方法是最小化（经验）数据分布和参数化分布（例如归一化流（NF）或基于能量的模型（EBM））之间的统计差异。在这种情况下，前向 KL 散度由于其易处理性而普遍存在，尽管其不对称性可能会妨碍捕获目标分布的某些属性。对称替代方案涉及脆弱的最小-最大公式和对抗性训练（例如生成对抗网络）或评估反向 KL 散度，就像对称 Jeffreys 散度的情况一样，从样本中计算这是一个挑战。这项工作旨在开发一种新方法来最小化杰弗里斯分歧。为此，它使用代理模型，其目标不仅是拟合数据，而且还协助优化主模型的 Jeffreys 散度。这种联合训练任务被表述为一个约束优化问题，以获得在整个训练过程中适应模型优先级的实用算法。我们说明了如何使用该框架在密度估计、图像生成和基于模拟的推理等任务中结合 NF 和 EBM 的优势。

Title: Power Ensemble Aggregation for Improved Extreme Event AI Prediction

• Authors: Julien Collard, Pierre Gentine, Tian Zheng
• Subjects: cs.LG, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2511.11170
• Pdf URL: https://arxiv.org/pdf/2511.11170
• Copy Paste: [[2511.11170]] Power Ensemble Aggregation for Improved Extreme Event AI Prediction(https://arxiv.org/abs/2511.11170)
• Keywords: generative
• Abstract: This paper addresses the critical challenge of improving predictions of climate extreme events, specifically heat waves, using machine learning methods. Our work is framed as a classification problem in which we try to predict whether surface air temperature will exceed its q-th local quantile within a specified timeframe. Our key finding is that aggregating ensemble predictions using a power mean significantly enhances the classifier's performance. By making a machine-learning based weather forecasting model generative and applying this non-linear aggregation method, we achieve better accuracy in predicting extreme heat events than with the typical mean prediction from the same model. Our power aggregation method shows promise and adaptability, as its optimal performance varies with the quantile threshold chosen, demonstrating increased effectiveness for higher extremes prediction.
• 摘要：本文解决了使用机器学习方法改进气候极端事件（特别是热浪）预测的关键挑战。我们的工作被定义为一个分类问题，其中我们尝试预测地表空气温度是否会在指定时间范围内超过其第 q 个局部分位数。我们的主要发现是，使用幂均值聚合集成预测可以显着提高分类器的性能。通过生成基于机器学习的天气预报模型并应用这种非线性聚合方法，我们在预测极端高温事件方面取得了比同一模型的典型平均预测更高的准确性。我们的功率聚合方法显示出前景和适应性，因为其最佳性能随所选分位数阈值的变化而变化，证明了更高的极端预测的有效性。

Title: When to Stop Federated Learning: Zero-Shot Generation of Synthetic Validation Data with Generative AI for Early Stopping

• Authors: Youngjoon Lee, Hyukjoon Lee, Jinu Gong, Yang Cao, Joonhyuk Kang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.11208
• Pdf URL: https://arxiv.org/pdf/2511.11208
• Copy Paste: [[2511.11208]] When to Stop Federated Learning: Zero-Shot Generation of Synthetic Validation Data with Generative AI for Early Stopping(https://arxiv.org/abs/2511.11208)
• Keywords: generation, generative
• Abstract: Federated Learning (FL) enables collaborative model training across decentralized devices while preserving data privacy. However, FL methods typically run for a predefined number of global rounds, often leading to unnecessary computation when optimal performance is reached earlier. In addition, training may continue even when the model fails to achieve meaningful performance. To address this inefficiency, we introduce a zero-shot synthetic validation framework that leverages generative AI to monitor model performance and determine early stopping points. Our approach adaptively stops training near the optimal round, thereby conserving computational resources and enabling rapid hyperparameter adjustments. Numerical results on multi-label chest X-ray classification demonstrate that our method reduces training rounds by up to 74% while maintaining accuracy within 1% of the optimal.
• 摘要：联邦学习 (FL) 支持跨分散设备的协作模型训练，同时保护数据隐私。然而，FL 方法通常运行预定义的全局轮数，当较早达到最佳性能时，通常会导致不必要的计算。此外，即使模型未能实现有意义的性能，训练也可能会继续。为了解决这种低效率问题，我们引入了一个零样本综合验证框架，该框架利用生成式人工智能来监控模型性能并确定早期停止点。我们的方法自适应地停止接近最佳轮次的训练，从而节省计算资源并实现快速的超参数调整。多标签胸部 X 射线分类的数值结果表明，我们的方法将训练次数减少了 74%，同时将准确度保持在最佳值的 1% 以内。

Title: RealisticDreamer: Guidance Score Distillation for Few-shot Gaussian Splatting

• Authors: Ruocheng Wu, Haolan He, Yufei Wang, Zhihao Li, Bihan Wen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11213
• Pdf URL: https://arxiv.org/pdf/2511.11213
• Copy Paste: [[2511.11213]] RealisticDreamer: Guidance Score Distillation for Few-shot Gaussian Splatting(https://arxiv.org/abs/2511.11213)
• Keywords: generative
• Abstract: 3D Gaussian Splatting (3DGS) has recently gained great attention in the 3D scene representation for its high-quality real-time rendering capabilities. However, when the input comprises sparse training views, 3DGS is prone to overfitting, primarily due to the lack of intermediate-view supervision. Inspired by the recent success of Video Diffusion Models (VDM), we propose a framework called Guidance Score Distillation (GSD) to extract the rich multi-view consistency priors from pretrained VDMs. Building on the insights from Score Distillation Sampling (SDS), GSD supervises rendered images from multiple neighboring views, guiding the Gaussian splatting representation towards the generative direction of VDM. However, the generative direction often involves object motion and random camera trajectories, making it challenging for direct supervision in the optimization process. To address this problem, we introduce an unified guidance form to correct the noise prediction result of VDM. Specifically, we incorporate both a depth warp guidance based on real depth maps and a guidance based on semantic image features, ensuring that the score update direction from VDM aligns with the correct camera pose and accurate geometry. Experimental results show that our method outperforms existing approaches across multiple datasets.
• 摘要：3D高斯溅射（3DGS）最近因其高质量的实时渲染能力在3D场景表示中受到了极大的关注。然而，当输入包含稀疏训练视图时，3DGS 很容易出现过度拟合，这主要是由于缺乏中间视图监督。受视频扩散模型 (VDM) 最近成功的启发，我们提出了一个称为指导分数蒸馏 (GSD) 的框架，用于从预训练的 VDM 中提取丰富的多视图一致性先验。基于分数蒸馏采样 (SDS) 的见解，GSD 监督来自多个相邻视图的渲染图像，引导高斯泼溅表示朝着 VDM 的生成方向发展。然而，生成方向通常涉及对象运动和随机相机轨迹，这使得优化过程中的直接监督具有挑战性。为了解决这个问题，我们引入了统一的指导形式来纠正VDM的噪声预测结果。具体来说，我们结合了基于真实深度图的深度扭曲指导和基于语义图像特征的指导，确保 VDM 的分数更新方向与正确的相机姿态和准确的几何形状保持一致。实验结果表明，我们的方法在多个数据集上优于现有方法。

Title: Positional Bias in Multimodal Embedding Models: Do They Favor the Beginning, the Middle, or the End?

• Authors: Kebin Wu, Fatima Albreiki
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11216
• Pdf URL: https://arxiv.org/pdf/2511.11216
• Copy Paste: [[2511.11216]] Positional Bias in Multimodal Embedding Models: Do They Favor the Beginning, the Middle, or the End?(https://arxiv.org/abs/2511.11216)
• Keywords: generation
• Abstract: Positional bias - where models overemphasize certain positions regardless of content - has been shown to negatively impact model performance across various tasks. While recent research has extensively examined positional bias in text generation models, its presence and effects in representation models remain underexplored. Even less is known about such biases in multimodal models. In this work, we investigate positional bias in multimodal representation models, specifically in the context of image-text retrieval. We begin by distinguishing between context importance and positional bias, and then assess the presence and extent of positional bias across different models and datasets. Our experiments demonstrate that positional bias is prevalent in multimodal models, but manifests differently across modalities: text encoders tend to exhibit bias toward the beginning of the input, whereas image encoders show bias at both the beginning and end. Furthermore, we find that this bias arises from, or is amplified by, a combination of factors, including the positional encoding scheme, training loss, context importance, and the nature of using image-text pairs in multimodal training.
• 摘要：位置偏差（模型过分强调某些位置而不管内容如何）已被证明会对各种任务的模型性能产生负面影响。尽管最近的研究广泛研究了文本生成模型中的位置偏差，但其在表示模型中的存在和影响仍未得到充分探索。对于多模式模型中的这种偏差我们知之甚少。在这项工作中，我们研究了多模态表示模型中的位置偏差，特别是在图像文本检索的背景下。我们首先区分上下文重要性和位置偏差，然后评估不同模型和数据集中位置偏差的存在和程度。我们的实验表明，位置偏差在多模态模型中普遍存在，但在不同模态中表现不同：文本编码器倾向于在输入的开头表现出偏差，而图像编码器在开始和结束时都表现出偏差。此外，我们发现这种偏差是由多种因素组合产生或放大的，包括位置编码方案、训练损失、上下文重要性以及在多模式训练中使用图像文本对的性质。

Title: Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regression

• Authors: Zhongbin Guo, Jiahe Liu, Yushan Li, Wenyu Gao, Zhen Yang, Chenzhi Li, Xinyue Zhang, Ping Jian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11239
• Pdf URL: https://arxiv.org/pdf/2511.11239
• Copy Paste: [[2511.11239]] Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regression(https://arxiv.org/abs/2511.11239)
• Keywords: generation
• Abstract: Existing Vision Language Models (VLMs) architecturally rooted in "flatland" perception, fundamentally struggle to comprehend real-world 3D spatial intelligence. This failure stems from a dual-bottleneck: input-stage conflict between computationally exorbitant geometric-aware encoders and superficial 2D-only features, and output-stage misalignment where discrete tokenizers are structurally incapable of producing precise, continuous numerical values. To break this impasse, we introduce GEODE (Geometric-Output and Decoupled-Input Engine), a novel architecture that resolves this dual-bottleneck by decoupling 3D reasoning from numerical generation. GEODE augments main VLM with two specialized, plug-and-play modules: Decoupled Rationale Module (DRM) that acts as spatial co-processor, aligning explicit 3D data with 2D visual features via cross-attention and distilling spatial Chain-of-Thought (CoT) logic into injectable Rationale Tokens; and Direct Regression Head (DRH), an "Embedding-as-Value" paradigm which routes specialized control tokens to a lightweight MLP for precise, continuous regression of scalars and 3D bounding boxes. The synergy of these modules allows our 1.5B parameter model to function as a high-level semantic dispatcher, achieving state-of-the-art spatial reasoning performance that rivals 7B+ models.
• 摘要：现有的视觉语言模型 (VLM) 在架构上植根于“平地”感知，从根本上难以理解现实世界的 3D 空间智能。这种失败源于双重瓶颈：计算量过高的几何感知编码器和表面的 2D 特征之间的输入阶段冲突，以及离散分词器在结构上无法生成精确、连续数值的输出阶段未对准。为了打破这一僵局，我们引入了 GEODE（几何输出和解耦输入引擎），这是一种新颖的架构，通过将 3D 推理与数值生成解耦来解决这一双瓶颈。 GEODE ​​通过两个专用的即插即用模块增强了主 VLM：解耦原理模块 (DRM)，充当空间协处理器，通过交叉注意力将显式 3D 数据与 2D 视觉特征对齐，并将空间思想链 (CoT) 逻辑提炼为可注入的原理令牌；直接回归头 (DRH)，一种“嵌入即值”范式，它将专门的控制令牌路由到轻量级 MLP，以实现标量和 3D 边界框的精确、连续回归。这些模块的协同作用使我们的 1.5B 参数模型能够充当高级语义调度程序，实现可与 7B+ 模型相媲美的最先进的空间推理性能。

Title: HealSplit: Towards Self-Healing through Adversarial Distillation in Split Federated Learning

• Authors: Yuhan Xie, Chen Lyu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.11240
• Pdf URL: https://arxiv.org/pdf/2511.11240
• Copy Paste: [[2511.11240]] HealSplit: Towards Self-Healing through Adversarial Distillation in Split Federated Learning(https://arxiv.org/abs/2511.11240)
• Keywords: generative
• Abstract: Split Federated Learning (SFL) is an emerging paradigm for privacy-preserving distributed learning. However, it remains vulnerable to sophisticated data poisoning attacks targeting local features, labels, smashed data, and model weights. Existing defenses, primarily adapted from traditional Federated Learning (FL), are less effective under SFL due to limited access to complete model updates. This paper presents HealSplit, the first unified defense framework tailored for SFL, offering end-to-end detection and recovery against five sophisticated types of poisoning attacks. HealSplit comprises three key components: (1) a topology-aware detection module that constructs graphs over smashed data to identify poisoned samples via topological anomaly scoring (TAS); (2) a generative recovery pipeline that synthesizes semantically consistent substitutes for detected anomalies, validated by a consistency validation student; and (3) an adversarial multi-teacher distillation framework trains the student using semantic supervision from a Vanilla Teacher and anomaly-aware signals from an Anomaly-Influence Debiasing (AD) Teacher, guided by the alignment between topological and gradient-based interaction matrices. Extensive experiments on four benchmark datasets demonstrate that HealSplit consistently outperforms ten state-of-the-art defenses, achieving superior robustness and defense effectiveness across diverse attack scenarios.
• 摘要：分裂联合学习（SFL）是一种新兴的隐私保护分布式学习范例。然而，它仍然容易受到针对局部特征、标签、破坏数据和模型权重的复杂数据中毒攻击。现有的防御主要改编自传统的联邦学习 (FL)，但由于对完整模型更新的访问有限，在 SFL 下效果较差。本文介绍了 HealSplit，这是第一个专为 SFL 定制的统一防御框架，提供针对五种复杂类型的中毒攻击的端到端检测和恢复。 HealSplit 包含三个关键组件：(1) 拓扑感知检测模块，在破坏的数据上构建图表，通过拓扑异常评分 (TAS) 识别中毒样本； (2) 生成恢复管道，用于合成检测到的异常的语义一致替代品，并由一致性验证学生进行验证； （3）对抗性多教师蒸馏框架使用来自普通教师的语义监督和来自异常影响去偏（AD）教师的异常感知信号来训练学生，并以拓扑和基于梯度的交互矩阵之间的对齐为指导。对四个基准数据集的大量实验表明，HealSplit 始终优于十种最先进的防御措施，在不同的攻击场景中实现了卓越的鲁棒性和防御有效性。

Title: Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs

• Authors: Jitesh Chavan, Rohit Lal, Anand Kamat, Mengjia Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11243
• Pdf URL: https://arxiv.org/pdf/2511.11243
• Copy Paste: [[2511.11243]] Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs(https://arxiv.org/abs/2511.11243)
• Keywords: generation, generative
• Abstract: State-space models (SSMs), Mamba in particular, are increasingly adopted for long-context sequence modeling, providing linear-time aggregation via an input-dependent, causal selective-scan operation. Along this line, recent "Mamba-for-vision" variants largely explore multiple scan orders to relax strict causality for non-sequential signals (e.g., images). Rather than preserving cross-block memory, the conventional formulation of the selective-scan operation in Mamba reinitializes each block's state-space dynamics from zero, discarding the terminal state-space representation (SSR) from the previous block. Arcee, a cross-block recurrent state chain, reuses each block's terminal state-space representation as the initial condition for the next block. Handoff across blocks is constructed as a differentiable boundary map whose Jacobian enables end-to-end gradient flow across terminal boundaries. Key to practicality, Arcee is compatible with all prior "vision-mamba" variants, parameter-free, and incurs constant, negligible cost. As a modeling perspective, we view terminal SSR as a mild directional prior induced by a causal pass over the input, rather than an estimator of the non-sequential signal itself. To quantify the impact, for unconditional generation on CelebA-HQ (256$\times$256) with Flow Matching, Arcee reduces FID$\downarrow$ from $82.81$ to $15.33$ ($5.4\times$ lower) on a single scan-order Zigzag Mamba baseline. Efficient CUDA kernels and training code will be released to support rigorous and reproducible research.
• 摘要：状态空间模型 (SSM)，尤其是 Mamba，越来越多地用于长上下文序列建模，通过依赖于输入的因果选择性扫描操作提供线性时间聚合。沿着这条线，最近的“Mamba-for-vision”变体主要探索多个扫描顺序，以放松非顺序信号（例如图像）的严格因果关系。 Mamba 中选择性扫描操作的传统公式不是保留跨块内存，而是从零重新初始化每个块的状态空间动态，丢弃前一个块的终端状态空间表示 (SSR)。 Arcee 是一个跨块循环状态链，它重用每个块的终端状态空间表示作为下一个块的初始条件。跨块的切换被构造为可微的边界图，其雅可比行列式使得能够跨终端边界的端到端梯度流。实用性的关键是，Arcee 与所有先前的“vision-mamba”变体兼容，无参数，并且产生恒定的、可以忽略不计的成本。从建模的角度来看，我们将终端 SSR 视为由输入的因果传递引起的温和定向先验，而不是非序列信号本身的估计器。为了量化影响，对于使用流匹配在 CelebA-HQ 上进行无条件生成（256$\times$256），Arcee 在单个扫描顺序 Zigzag Mamba 基线上将 FID$\downarrow$ 从 $82.81$ 减少到 $15.33$（降低了 $5.4\times$）。将发布高效的 CUDA 内核和训练代码，以支持严格且可重复的研究。

Title: CountSteer: Steering Attention for Object Counting in Diffusion Models

• Authors: Hyemin Boo, Hyoryung Kim, Myungjin Lee, Seunghyeon Lee, Jiyoung Lee, Jang-Hwan Choi, Hyunsoo Cho
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11253
• Pdf URL: https://arxiv.org/pdf/2511.11253
• Copy Paste: [[2511.11253]] CountSteer: Steering Attention for Object Counting in Diffusion Models(https://arxiv.org/abs/2511.11253)
• Keywords: generation
• Abstract: Text-to-image diffusion models generate realistic and coherent images but often fail to follow numerical instructions in text, revealing a gap between language and visual representation. Interestingly, we found that these models are not entirely blind to numbers-they are implicitly aware of their own counting accuracy, as their internal signals shift in consistent ways depending on whether the output meets the specified count. This observation suggests that the model already encodes a latent notion of numerical correctness, which can be harnessed to guide generation more precisely. Building on this intuition, we introduce CountSteer, a training-free method that improves generation of specified object counts by steering the model's cross-attention hidden states during inference. In our experiments, CountSteer improved object-count accuracy by about 4% without compromising visual quality, demonstrating a simple yet effective step toward more controllable and semantically reliable text-to-image generation.
• 摘要：文本到图像的扩散模型生成真实且连贯的图像，但通常无法遵循文本中的数字指令，从而揭示了语言和视觉表示之间的差距。有趣的是，我们发现这些模型并非完全对数字视而不见，它们隐含地意识到自己的计数精度，因为它们的内部信号根据输出是否满足指定计数以一致的方式变化。这一观察结果表明，该模型已经编码了数字正确性的潜在概念，可以利用它来更精确地指导生成。基于这种直觉，我们引入了 CountSteer，这是一种免训练方法，可通过在推理过程中控制模型的交叉注意力隐藏状态来改进指定对象计数的生成。在我们的实验中，CountSteer 在不影响视觉质量的情况下将对象计数准确性提高了约 4%，展示了朝着更可控和语义可靠的文本到图像生成迈出的简单而有效的一步。

Title: RTGaze: Real-Time 3D-Aware Gaze Redirection from a Single Image

• Authors: Hengfei Wang, Zhongqun Zhang, Yihua Cheng, Hyung Jin Chang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11289
• Pdf URL: https://arxiv.org/pdf/2511.11289
• Copy Paste: [[2511.11289]] RTGaze: Real-Time 3D-Aware Gaze Redirection from a Single Image(https://arxiv.org/abs/2511.11289)
• Keywords: generation
• Abstract: Gaze redirection methods aim to generate realistic human face images with controllable eye movement. However, recent methods often struggle with 3D consistency, efficiency, or quality, limiting their practical applications. In this work, we propose RTGaze, a real-time and high-quality gaze redirection method. Our approach learns a gaze-controllable facial representation from face images and gaze prompts, then decodes this representation via neural rendering for gaze redirection. Additionally, we distill face geometric priors from a pretrained 3D portrait generator to enhance generation quality. We evaluate RTGaze both qualitatively and quantitatively, demonstrating state-of-the-art performance in efficiency, redirection accuracy, and image quality across multiple datasets. Our system achieves real-time, 3D-aware gaze redirection with a feedforward network (~0.06 sec/image), making it 800x faster than the previous state-of-the-art 3D-aware methods.
• 摘要：视线重定向方法旨在生成具有可控眼球运动的逼真人脸图像。然而，最近的方法经常在 3D 一致性、效率或质量方面遇到困难，限制了它们的实际应用。在这项工作中，我们提出了 RTGaze，一种实时、高质量的注视重定向方法。我们的方法从面部图像和注视提示中学习注视可控的面部表征，然后通过神经渲染解码该表征以进行注视重定向。此外，我们从预训练的 3D 肖像生成器中提取面部几何先验，以提高生成质量。我们对 RTGaze 进行定性和定量评估，展示了跨多个数据集在效率、重定向准确性和图像质量方面最先进的性能。我们的系统通过前馈网络实现实时、3D 感知注视重定向（~0.06 秒/图像），使其比之前最先进的 3D 感知方法快 800 倍。

Title: YCB-Ev SD: Synthetic event-vision dataset for 6DoF object pose estimation

• Authors: Pavel Rojtberg, Julius Kühn
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11344
• Pdf URL: https://arxiv.org/pdf/2511.11344
• Copy Paste: [[2511.11344]] YCB-Ev SD: Synthetic event-vision dataset for 6DoF object pose estimation(https://arxiv.org/abs/2511.11344)
• Keywords: generation
• Abstract: We introduce YCB-Ev SD, a synthetic dataset of event-camera data at standard definition (SD) resolution for 6DoF object pose estimation. While synthetic data has become fundamental in frame-based computer vision, event-based vision lacks comparable comprehensive resources. Addressing this gap, we present 50,000 event sequences of 34 ms duration each, synthesized from Physically Based Rendering (PBR) scenes of YCB-Video objects following the Benchmark for 6D Object Pose (BOP) methodology. Our generation framework employs simulated linear camera motion to ensure complete scene coverage, including background activity. Through systematic evaluation of event representations for CNN-based inference, we demonstrate that time-surfaces with linear decay and dual-channel polarity encoding achieve superior pose estimation performance, outperforming exponential decay and single-channel alternatives by significant margins. Our analysis reveals that polarity information contributes most substantially to performance gains, while linear temporal encoding preserves critical motion information more effectively than exponential decay. The dataset is provided in a structured format with both raw event streams and precomputed optimal representations to facilitate immediate research use and reproducible benchmarking. The dataset is publicly available at this https URL.
• 摘要：我们引入了 YCB-Ev SD，这是一个标准定义 (SD) 分辨率的事件相机数据的合成数据集，用于 6DoF 对象姿态估计。虽然合成数据已成为基于框架的计算机视觉的基础，但基于事件的视觉缺乏可比较的综合资源。为了解决这一差距，我们提出了 50,000 个事件序列，每个事件序列持续时间为 34 毫秒，这些事件序列是根据 6D 对象姿势 (BOP) 方法基准从 YCB-Video 对象的基于物理的渲染 (PBR) 场景合成的。我们的生成框架采用模拟线性相机运动来确保完整的场景覆盖，包括背景活动。通过对基于 CNN 的推理的事件表示进行系统评估，我们证明具有线性衰减和双通道极性编码的时间表面可实现卓越的姿态估计性能，显着优于指数衰减和单通道替代方案。我们的分析表明，极性信息对性能增益的贡献最大，而线性时间编码比指数衰减更有效地保留关键运动信息。该数据集以结构化格式提供，包含原始事件流和预先计算的最佳表示，以促进即时研究使用和可重复的基准测试。该数据集可通过此 https URL 公开获取。

Title: Fast and Expressive Multi-Token Prediction with Probabilistic Circuits

• Authors: Andreas Grivas, Lorenzo Loconte, Emile van Krieken, Piotr Nawrot, Yu Zhao, Euan Wielewski, Pasquale Minervini, Edoardo Ponti, Antonio Vergari
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2511.11346
• Pdf URL: https://arxiv.org/pdf/2511.11346
• Copy Paste: [[2511.11346]] Fast and Expressive Multi-Token Prediction with Probabilistic Circuits(https://arxiv.org/abs/2511.11346)
• Keywords: generation
• Abstract: Multi-token prediction (MTP) is a prominent strategy to significantly speed up generation in large language models (LLMs), including byte-level LLMs, which are tokeniser-free but prohibitively slow. However, existing MTP methods often sacrifice expressiveness by assuming independence between future tokens. In this work, we investigate the trade-off between expressiveness and latency in MTP within the framework of probabilistic circuits (PCs). Our framework, named MTPC, allows one to explore different ways to encode the joint distributions over future tokens by selecting different circuit architectures, generalising classical models such as (hierarchical) mixture models, hidden Markov models and tensor networks. We show the efficacy of MTPC by retrofitting existing byte-level LLMs, such as EvaByte. Our experiments show that, when combined with speculative decoding, MTPC significantly speeds up generation compared to MTP with independence assumptions, while guaranteeing to retain the performance of the original verifier LLM. We also rigorously study the optimal trade-off between expressiveness and latency when exploring the possible parameterisations of MTPC, such as PC architectures and partial layer sharing between the verifier and draft LLMs.
• 摘要：多标记预测 (MTP) 是一种显着加快大型语言模型 (LLM) 生成速度的重要策略，其中包括字节级 LLM，这些模型无需标记器，但速度非常慢。然而，现有的 MTP 方法经常通过假设未来令牌之间的独立性来牺牲表达能力。在这项工作中，我们研究了概率电路 (PC) 框架内 MTP 的表现力和延迟之间的权衡。我们的框架名为 MTPC，它允许人们通过选择不同的电路架构、概括经典模型（例如（分层）混合模型、隐马尔可夫模型和张量网络）来探索对未来代币的联合分布进行编码的不同方法。我们通过改造现有的字节级 LLM（例如 EvaByte）来展示 MTPC 的功效。我们的实验表明，与推测性解码相结合时，与具有独立性假设的 MTP 相比，MTPC 显着加快了生成速度，同时保证保留原始验证器 LLM 的性能。在探索 MTPC 的可能参数化（例如 PC 架构以及验证者和草案 LLM 之间的部分层共享）时，我们还严格研究了表现力和延迟之间的最佳权衡。

Title: Free3D: 3D Human Motion Emerges from Single-View 2D Supervision

• Authors: Sheng Liu, Yuanzhi Liang, Sidan Du
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11368
• Pdf URL: https://arxiv.org/pdf/2511.11368
• Copy Paste: [[2511.11368]] Free3D: 3D Human Motion Emerges from Single-View 2D Supervision(https://arxiv.org/abs/2511.11368)
• Keywords: generation
• Abstract: Recent 3D human motion generation models demonstrate remarkable reconstruction accuracy yet struggle to generalize beyond training distributions. This limitation arises partly from the use of precise 3D supervision, which encourages models to fit fixed coordinate patterns instead of learning the essential 3D structure and motion semantic cues required for robust this http URL overcome this limitation, we propose Free3D, a framework that synthesizes realistic 3D motions without any 3D motion annotations. Free3D introduces a Motion-Lifting Residual Quantized VAE (ML-RQ) that maps 2D motion sequences into 3D-consistent latent spaces, and a suite of 3D-free regularization objectives enforcing view consistency, orientation coherence, and physical plausibility. Trained entirely on 2D motion data, Free3D generates diverse, temporally coherent, and semantically aligned 3D motions, achieving performance comparable to or even surpassing fully 3D-supervised counterparts. These results suggest that relaxing explicit 3D supervision encourages stronger structural reasoning and generalization, offering a scalable and data-efficient paradigm for 3D motion generation.
• 摘要：最近的 3D 人体运动生成模型展示了出色的重建精度，但很难推广到训练分布之外。这一限制部分源于精确 3D 监督的使用，它鼓励模型适应固定的坐标模式，而不是学习鲁棒所需的基本 3D 结构和运动语义线索。为了克服这一限制，我们提出了 Free3D，这是一个无需任何 3D 运动注释即可合成真实 3D 运动的框架。 Free3D 引入了运动提升残差量化 VAE (ML-RQ)，可将 2D 运动序列映射到 3D 一致的潜在空间，并引入了一套无 3D 正则化目标，以增强视图一致性、方向一致性和物理合理性。 Free3D 完全基于 2D 运动数据进行训练，生成多样化、时间连贯且语义对齐的 3D 运动，其性能可与完全 3D 监督的同类产品相媲美甚至超越。这些结果表明，放松显式 3D 监督可以鼓励更强的结构推理和泛化，为 3D 运动生成提供可扩展且数据高效的范例。

Title: Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models

• Authors: Jiaxi Huang, Dongxu Wu, Hanwei Zhu, Lingyu Zhu, Jun Xing, Xu Wang, Baoliang Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11410
• Pdf URL: https://arxiv.org/pdf/2511.11410
• Copy Paste: [[2511.11410]] Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models(https://arxiv.org/abs/2511.11410)
• Keywords: quality assessment
• Abstract: The rapid advancement of Multi-modal Large Language Models (MLLMs) has expanded their capabilities beyond high-level vision tasks. Nevertheless, their potential for Document Image Quality Assessment (DIQA) remains underexplored. To bridge this gap, we propose Q-Doc, a three-tiered evaluation framework for systematically probing DIQA capabilities of MLLMs at coarse, middle, and fine granularity levels. a) At the coarse level, we instruct MLLMs to assign quality scores to document images and analyze their correlation with Quality Annotations. b) At the middle level, we design distortion-type identification tasks, including single-choice and multi-choice tests for multi-distortion scenarios. c) At the fine level, we introduce distortion-severity assessment where MLLMs classify distortion intensity against human-annotated references. Our evaluation demonstrates that while MLLMs possess nascent DIQA abilities, they exhibit critical limitations: inconsistent scoring, distortion misidentification, and severity misjudgment. Significantly, we show that Chain-of-Thought (CoT) prompting substantially enhances performance across all levels. Our work provides a benchmark for DIQA capabilities in MLLMs, revealing pronounced deficiencies in their quality perception and promising pathways for enhancement. The benchmark and code are publicly available at: this https URL.
• 摘要：多模态大型语言模型（MLLM）的快速发展已将其功能扩展到高级视觉任务之外。然而，它们在文档图像质量评估 (DIQA) 方面的潜力仍未得到充分开发。为了弥补这一差距，我们提出了 Q-Doc，这是一个三层评估框架，用于在粗、中和细粒度级别系统地探索 MLLM 的 DIQA 功能。 a) 在粗略层面上，我们指示 MLLM 为文档图像分配质量分数，并分析它们与质量注释的相关性。 b）在中间层，我们设计了失真类型识别任务，包括针对多失真场景的单选和多选测试。 c) 在精细层面，我们引入了失真严重性评估，其中 MLLM 根据人工注释的参考对失真强度进行分类。我们的评估表明，虽然 MLLM 具有新生的 DIQA 能力，但它们表现出严重的局限性：评分不一致、失真错误识别和严重性误判。值得注意的是，我们表明思想链（CoT）提示可以显着提高各个级别的绩效。我们的工作为 MLLM 中的 DIQA 能力提供了基准，揭示了其质量认知中的明显缺陷以及有希望的增强途径。基准测试和代码可在以下位置公开获取：此 https URL。

Title: Low-Bit, High-Fidelity: Optimal Transport Quantization for Flow Matching

• Authors: Dara Varam, Diaa A. Abuhani, Imran Zualkernan, Raghad AlDamani, Lujain Khalil
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11418
• Pdf URL: https://arxiv.org/pdf/2511.11418
• Copy Paste: [[2511.11418]] Low-Bit, High-Fidelity: Optimal Transport Quantization for Flow Matching(https://arxiv.org/abs/2511.11418)
• Keywords: generation, generative
• Abstract: Flow Matching (FM) generative models offer efficient simulation-free training and deterministic sampling, but their practical deployment is challenged by high-precision parameter requirements. We adapt optimal transport (OT)-based post-training quantization to FM models, minimizing the 2-Wasserstein distance between quantized and original weights, and systematically compare its effectiveness against uniform, piecewise, and logarithmic quantization schemes. Our theoretical analysis provides upper bounds on generative degradation under quantization, and empirical results across five benchmark datasets of varying complexity show that OT-based quantization preserves both visual generation quality and latent space stability down to 2-3 bits per parameter, where alternative methods fail. This establishes OT-based quantization as a principled, effective approach to compress FM generative models for edge and embedded AI applications.
• 摘要：流匹配（FM）生成模型提供高效的免模拟训练和确定性采样，但其实际部署受到高精度参数要求的挑战。我们将基于最佳传输 (OT) 的训练后量化应用于 FM 模型，最小化量化权重和原始权重之间的 2-Wasserstein 距离，并系统地将其有效性与均匀、分段和对数量化方案进行比较。我们的理论分析提供了量化下生成退化的上限，并且五个不同复杂度的基准数据集的经验结果表明，基于 OT 的量化将视觉生成质量和潜在空间稳定性保持在每个参数 2-3 位，而其他方法则失败了。这使得基于 OT 的量化成为一种有原则的、有效的方法，用于压缩边缘和嵌入式 AI 应用的 FM 生成模型。

Title: WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation

• Authors: Wei Chow, Jiachun Pan, Yongyuan Liang, Mingze Zhou, Xue Song, Liyu Jia, Saining Zhang, Siliang Tang, Juncheng Li, Fengda Zhang, Weijia Wu, Hanwang Zhang, Tat-Seng Chua
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11434
• Pdf URL: https://arxiv.org/pdf/2511.11434
• Copy Paste: [[2511.11434]] WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation(https://arxiv.org/abs/2511.11434)
• Keywords: generation
• Abstract: Recent advances in unified multimodal models (UMMs) have enabled impressive progress in visual comprehension and generation. However, existing datasets and benchmarks focus primarily on single-turn interactions, failing to capture the multi-turn, context-dependent nature of real-world image creation and editing. To address this gap, we present WEAVE, the first suite for in-context interleaved cross-modality comprehension and generation. Our suite consists of two complementary parts. WEAVE-100k is a large-scale dataset of 100K interleaved samples spanning over 370K dialogue turns and 500K images, covering comprehension, editing, and generation tasks that require reasoning over historical context. WEAVEBench is a human-annotated benchmark with 100 tasks based on 480 images, featuring a hybrid VLM judger evaluation framework based on both the reference image and the combination of the original image with editing instructions that assesses models' abilities in multi-turn generation, visual memory, and world-knowledge reasoning across diverse domains. Experiments demonstrate that training on WEAVE-100k enables vision comprehension, image editing, and comprehension-generation collaboration capabilities. Furthermore, it facilitates UMMs to develop emergent visual-memory capabilities, while extensive evaluations on WEAVEBench expose the persistent limitations and challenges of current approaches in multi-turn, context-aware image generation and editing. We believe WEAVE provides a view and foundation for studying in-context interleaved comprehension and generation for multi-modal community.
• 摘要：统一多模态模型（UMM）的最新进展在视觉理解和生成方面取得了令人瞩目的进展。然而，现有的数据集和基准主要关注单轮交互，未能捕捉现实世界图像创建和编辑的多轮、上下文相关的性质。为了解决这一差距，我们推出了 WEAVE，这是第一个用于上下文交错的跨模态理解和生成的套件。我们的套件由两个互补的部分组成。 WEAVE-100k 是一个包含 100K 交错样本的大型数据集，涵盖超过 370K 对话回合和 500K 图像，涵盖需要对历史背景进行推理的理解、编辑和生成任务。 WEAVEBench 是一个人工注释的基准测试，包含基于 480 张图像的 100 个任务，具有基于参考图像以及原始图像与编辑指令相结合的混合 VLM 判断器评估框架，可评估模型在多轮生成、视觉记忆和跨不同领域的世界知识推理方面的能力。实验表明，在 WEAVE-100k 上进行训练可以实现视觉理解、图像编辑和理解生成协作功能。此外，它有助于 UMM 开发新兴的视觉记忆功能，而对 WEAVEBench 的广泛评估暴露了当前多轮、上下文感知图像生成和编辑方法的持续局限性和挑战。我们相信 WEAVE 为研究多模态社区的上下文交错理解和生成提供了一个观点和基础。

Title: From Synthetic Scenes to Real Performance: Enhancing Spatial Reasoning in VLMs

• Authors: Massimo Rizzoli, Simone Alghisi, Seyed Mahed Mousavi, Giuseppe Riccardi
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2511.11440
• Pdf URL: https://arxiv.org/pdf/2511.11440
• Copy Paste: [[2511.11440]] From Synthetic Scenes to Real Performance: Enhancing Spatial Reasoning in VLMs(https://arxiv.org/abs/2511.11440)
• Keywords: generation
• Abstract: Fine-tuning Vision-Language Models (VLMs) is a common strategy to improve performance following an ad-hoc data collection and annotation of real-world scenes. However, this process is often prone to biases, errors, and distribution imbalance, resulting in overfitting and imbalanced performance. Although a few studies have tried to address this problem by generating synthetic data, they lacked control over distribution bias and annotation quality. To address these challenges, we redesign the fine-tuning process in two ways. First, we control the generation of data and its annotations, ensuring it is free from bias, distribution imbalance, and annotation errors. We automatically construct the dataset by comprehensively sampling objects' attributes, including color, shape, size, and position within the scene. Secondly, using this annotated dataset, we fine-tune state-of-the-art VLMs and assess performance transferability to real-world data on the absolute position task. We conduct exhaustive evaluations on both synthetic and real-world benchmarks. Our experiments reveal two key findings: 1) fine-tuning on balanced synthetic data yields uniform performance across the visual scene and mitigates common biases; and 2) fine-tuning on synthetic stimuli significantly improves performance on real-world data (COCO), outperforming models fine-tuned in the matched setting.
• 摘要：微调视觉语言模型 (VLM) 是在临时数据收集和现实场景注释后提高性能的常见策略。然而，这个过程往往容易出现偏差、错误和分布不平衡，导致过度拟合和性能不平衡。尽管一些研究试图通过生成合成数据来解决这个问题，但它们缺乏对分布偏差和注释质量的控制。为了应对这些挑战，我们通过两种方式重新设计了微调过程。首先，我们控制数据的生成及其注释，确保其不存在偏差、分布不平衡和注释错误。我们通过对场景中对象的属性（包括颜色、形状、大小和位置）进行全面采样来自动构建数据集。其次，使用这个带注释的数据集，我们对最先进的 VLM 进行微调，并评估绝对位置任务上真实世界数据的性能可转移性。我们对综合基准和现实基准进行详尽的评估。我们的实验揭示了两个关键发现：1）对平衡的合成数据进行微调可以在整个视觉场景中产生一致的性能并减轻常见偏差； 2）对合成刺激的微调显着提高了现实世界数据（COCO）的性能，优于在匹配设置中微调的模型。

Title: Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery

• Authors: Yijie Kang, Xinliang Wang, Zhenyu Wu, Yifeng Shi, Hailong Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11470
• Pdf URL: https://arxiv.org/pdf/2511.11470
• Copy Paste: [[2511.11470]] Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery(https://arxiv.org/abs/2511.11470)
• Keywords: generation, generative
• Abstract: Recent advances in generative modeling have substantially enhanced 3D urban generation, enabling applications in digital twins, virtual cities, and large-scale simulations. However, existing methods face two key challenges: (1) the need for large-scale 3D city assets for supervised training, which are difficult and costly to obtain, and (2) reliance on semantic or height maps, which are used exclusively for generating buildings in virtual worlds and lack connection to real-world appearance, limiting the realism and generalizability of generated cities. To address these limitations, we propose Sat2RealCity, a geometry-aware and appearance-controllable framework for 3D urban generation from real-world satellite imagery. Unlike previous city-level generation methods, Sat2RealCity builds generation upon individual building entities, enabling the use of rich priors and pretrained knowledge from 3D object generation while substantially reducing dependence on large-scale 3D city assets. Specifically, (1) we introduce the OSM-based spatial priors strategy to achieve interpretable geometric generation from spatial topology to building instances; (2) we design an appearance-guided controllable modeling mechanism for fine-grained appearance realism and style control; and (3) we construct an MLLM-powered semantic-guided generation pipeline, bridging semantic interpretation and geometric reconstruction. Extensive quantitative and qualitative experiments demonstrate that Sat2RealCity significantly surpasses existing baselines in structural consistency and appearance realism, establishing a strong foundation for real-world aligned 3D urban content creation. The code will be released soon.
• 摘要：生成建模的最新进展极大地增强了 3D 城市生成，从而实现了数字孪生、虚拟城市和大规模模拟中的应用。然而，现有方法面临两个关键挑战：(1) 需要大规模 3D 城市资产进行监督训练，获得这些资产既困难又昂贵，(2) 依赖语义或高度图，这些地图专门用于在虚拟世界中生成建筑物，与现实世界外观缺乏联系，限制了生成城市的真实性和普遍性。为了解决这些限制，我们提出了 Sat2RealCity，这是一种几何感知和外观可控的框架，用于根据现实世界的卫星图像生成 3D 城市。与以前的城市级生成方法不同，Sat2RealCity 在各个建筑实体的基础上构建生成，从而能够使用来自 3D 对象生成的丰富先验和预训练知识，同时大大减少对大型 3D 城市资产的依赖。具体来说，（1）我们引入基于OSM的空间先​​验策略来实现从空间拓扑到建筑实例的可解释的几何生成； （2）我们设计了一种外观引导的可控建模机制，用于细粒度的外观真实性和风格控制； (3) 我们构建了一个由 MLLM 驱动的语义引导生成管道，连接语义解释和几何重建。广泛的定量和定性实验表明，Sat2RealCity 在结构一致性和外观真实性方面显着超越了现有基线，为现实世界对齐的 3D 城市内容创建奠定了坚实的基础。该代码即将发布。

Title: ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation

• Authors: Kaishen Wang, Ruibo Chen, Tong Zheng, Heng Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.11483
• Pdf URL: https://arxiv.org/pdf/2511.11483
• Copy Paste: [[2511.11483]] ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation(https://arxiv.org/abs/2511.11483)
• Keywords: generation
• Abstract: Recent text-to-image (T2I) models have made remarkable progress in generating visually realistic and semantically coherent images. However, they still suffer from randomness and inconsistency with the given prompts, particularly when textual descriptions are vague or underspecified. Existing approaches, such as prompt rewriting, best-of-N sampling, and self-refinement, can mitigate these issues but usually require additional modules and operate independently, hindering test-time scaling efficiency and increasing computational overhead. In this paper, we introduce ImAgent, a training-free unified multimodal agent that integrates reasoning, generation, and self-evaluation within a single framework for efficient test-time scaling. Guided by a policy controller, multiple generation actions dynamically interact and self-organize to enhance image fidelity and semantic alignment without relying on external models. Extensive experiments on image generation and editing tasks demonstrate that ImAgent consistently improves over the backbone and even surpasses other strong baselines where the backbone model fails, highlighting the potential of unified multimodal agents for adaptive and efficient image generation under test-time scaling.
• 摘要：最近的文本到图像（T2I）模型在生成视觉逼真和语义连贯的图像方面取得了显着进展。然而，它们仍然存在随机性和与给定提示不一致的问题，特别是当文本描述模糊或不明确时。现有的方法，例如即时重写、N 次最佳采样和自我改进，可以缓解这些问题，但通常需要额外的模块并独立运行，从而阻碍了测试时间的扩展效率并增加了计算开销。在本文中，我们介绍了 ImAgent，这是一种免训练的统一多模式代理，它将推理、生成和自我评估集成在一个框架内，以实现有效的测试时间扩展。在策略控制器的指导下，多个生成动作动态交互和自组织，以增强图像保真度和语义对齐，而无需依赖外部模型。关于图像生成和编辑任务的大量实验表明，ImAgent 在主干上持续改进，甚至超越了主干模型失败的其他强基线，凸显了统一多模态代理在测试时间缩放下自适应和高效图像生成的潜力。

Title: Bridging Hidden States in Vision-Language Models

• Authors: Benjamin Fein-Ashley, Jacob Fein-Ashley
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2511.11526
• Pdf URL: https://arxiv.org/pdf/2511.11526
• Copy Paste: [[2511.11526]] Bridging Hidden States in Vision-Language Models(https://arxiv.org/abs/2511.11526)
• Keywords: generation
• Abstract: Vision-Language Models (VLMs) are a new family of models that align image content with natural language. Existing approaches typically fuse either (a) early: by mixing tokens/features inside the encoders, or (b) late: by comparing pooled embeddings. Many methods also tie fusion to an autoregressive decoder. However, the hidden states of both modalities already carry rich, modality-specific structure (spatial layout in vision; syntax and semantics in text), so directly aligning these states is a natural way to match what the two modalities "think". We propose a lightweight fusion module: a few cross-only, bidirectional attention layers placed near the top of both encoders. Each layer projects the vision and text encoder hidden-state sequences into a shared space, attends across modalities, and sends gated residual updates back, with simple stabilizers to improve alignment. The encoders remain non-causal and strong for understanding, while generation stays cleanly decoupled via an optional decoder. Across standard retrieval, VQA, and visual reasoning benchmarks, BRIDGE outperforms comparable VLMs while preserving the bi-encoder efficiency of contrastive models. We make our code publicly available at this https URL.
• 摘要：视觉语言模型 (VLM) 是一个新的模型系列，可将图像内容与自然语言保持一致。现有方法通常会（a）早期融合：通过在编码器内混合令牌/特征，或（b）晚期融合：通过比较池化嵌入。许多方法还将融合与自回归解码器联系起来。然而，两种模态的隐藏状态已经承载了丰富的、模态特定的结构（视觉中的空间布局；文本中的语法和语义），因此直接对齐这些状态是匹配两种模态“所想”的自然方式。我们提出了一个轻量级融合模块：一些仅交叉的双向注意层放置在两个编码器的顶部附近。每层将视觉和文本编码器隐藏状态序列投影到共享空间中，参与跨模态，并将门控残差更新发回，并使用简单的稳定器来改善对齐。编码器保持非因果性且易于理解，而生成则通过可选解码器保持干净的解耦。在标准检索、VQA 和视觉推理基准测试中，BRIDGE 的性能优于同类 VLM，同时保留了对比模型的双编码器效率。我们通过此 https URL 公开提供我们的代码。