2025-06-17

Title: FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization

• Authors: Fangxin Liu, Zongwu Wang, JinHong Xia, Junping Zhao, Jian Liu, Haibing Guan, Li Jiang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.12024
• Pdf URL: https://arxiv.org/pdf/2506.12024
• Copy Paste: [[2506.12024]] FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization(https://arxiv.org/abs/2506.12024)
• Keywords: generation
• Abstract: The rapid advancement of large language models (LLMs) has exacerbated the memory bottleneck due to the widening gap between model parameter scaling and hardware capabilities. While post-training quantization (PTQ) techniques effectively reduce memory overhead, existing methods predominantly rely on static quantization strategies, which struggle to adapt to dynamic workloads. To address this, we propose FlexQuant, a dynamic precision-switching framework that optimizes the trade-off between inference speed and accuracy. Leveraging model perplexity entropy and Kullback-Leibler (KL) divergence, FlexQuant enables fine-grained, layer-wise mixed-precision quantization and dynamically adjusts bit-widths during each token generation. Our work provides a comprehensive analysis of quantization strategies, introduces a precision requirement model for optimal switching, and implements efficient fine-grained precision management. Experimental results demonstrate that FlexQuant achieves a 1.3x end-to-end speedup across diverse language tasks with negligible accuracy loss introduced. This framework offers a flexible and adaptive solution for efficient LLM deployment.
• 摘要：大型语言模型（LLMS）的快速进步加剧了由于模型参数缩放和硬件功能之间的差距的扩大而加剧了内存瓶颈。虽然训练后量化（PTQ）技术有效地减少了内存开销，但现有方法主要依赖于静态量化策略，这些策略难以适应动态工作负载。为了解决这个问题，我们提出了Flexquant，这是一个动态的精确开关框架，可优化推理速度和准确性之间的权衡。利用模型的困惑熵和kullback-leibler（KL）差异，flexquant可以实现细粒度，层的混合精液量化，并在每次代币生成过程中动态调整位宽度。我们的工作提供了对量化策略的全面分析，引入了最佳切换的精确要求模型，并实施了有效的细粒精度管理。实验结果表明，弹性在跨不同语言任务的端到端速度达到了1.3倍，而准确的精度损失却可以忽略不计。该框架为有效的LLM部署提供了灵活和自适应的解决方案。

Title: Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset

• Authors: Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12031
• Pdf URL: https://arxiv.org/pdf/2506.12031
• Copy Paste: [[2506.12031]] Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset(https://arxiv.org/abs/2506.12031)
• Keywords: generative
• Abstract: Federated Continual Learning (FCL) has recently emerged as a crucial research area, as data from distributed clients typically arrives as a stream, requiring sequential learning. This paper explores a more practical and challenging FCL setting, where clients may have unrelated or even conflicting data and tasks. In this scenario, statistical heterogeneity and data noise can create spurious correlations, leading to biased feature learning and catastrophic forgetting. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. To address these challenges, we propose a novel approach called Spatio-Temporal grAdient Matching with network-free Prototype (STAMP). Our contributions are threefold: 1) We develop a model-agnostic method to determine subset of samples that effectively form prototypes when using a prototypical network, making it resilient to continual learning challenges; 2) We introduce a spatio-temporal gradient matching approach, applied at both the client-side (temporal) and server-side (spatial), to mitigate catastrophic forgetting and data heterogeneity; 3) We leverage prototypes to approximate task-wise gradients, improving gradient matching on the client-side. Extensive experiments demonstrate our method's superiority over existing baselines.
• 摘要：联邦持续学习（FCL）最近成为一个关键的研究领域，因为来自分布式客户的数据通常以流为流，需要顺序学习。本文探讨了更实用和具有挑战性的FCL设置，客户可能拥有无关甚至相互冲突的数据和任务。在这种情况下，统计异质性和数据噪声会产生虚假的相关性，从而导致特征学习和灾难性遗忘。现有的FCL方法通常使用生成重播来创建以前任务的伪数据。但是，生成的重播本身遭受了客户之间灾难性的遗忘和任务差异，导致FCL过度拟合。现有的FCL方法通常使用生成重播来创建以前任务的伪数据。但是，生成的重播本身遭受了客户之间灾难性的遗忘和任务差异，导致FCL过度拟合。为了应对这些挑战，我们提出了一种新型方法，称为时空梯度与无网络原型（邮票）匹配。我们的贡献是三个方面：1）我们开发了一种模型不可吻合的方法来确定样品的子集，这些样品在使用原型网络时有效地形成原型，使其对持续学习的挑战具有弹性； 2）我们引入了一种时空梯度匹配方法，该方法在客户端（时间）和服务器端（空间）上都应用，以减轻灾难性遗忘和数据异质性； 3）我们利用原型来近似任务梯度，从而改善了客户端的梯度匹配。广泛的实验证明了我们的方法优于现有基线。

Title: EMERGENT: Efficient and Manipulation-resistant Matching using GFlowNets

• Authors: Mayesha Tasnim, Erman Acar, Sennay Ghebreab
• Subjects: cs.LG, cs.AI, cs.GT
• Abstract URL: https://arxiv.org/abs/2506.12033
• Pdf URL: https://arxiv.org/pdf/2506.12033
• Copy Paste: [[2506.12033]] EMERGENT: Efficient and Manipulation-resistant Matching using GFlowNets(https://arxiv.org/abs/2506.12033)
• Keywords: generative
• Abstract: The design of fair and efficient algorithms for allocating public resources, such as school admissions, housing, or medical residency, has a profound social impact. In one-sided matching problems, where individuals are assigned to items based on ranked preferences, a fundamental trade-off exists between efficiency and strategyproofness. Existing algorithms like Random Serial Dictatorship (RSD), Probabilistic Serial (PS), and Rank Minimization (RM) capture only one side of this trade-off: RSD is strategyproof but inefficient, while PS and RM are efficient but incentivize manipulation. We propose EMERGENT, a novel application of Generative Flow Networks (GFlowNets) to one-sided matching, leveraging its ability to sample diverse, high-reward solutions. In our approach, efficient and manipulation-resistant matches emerge naturally: high-reward solutions yield efficient matches, while the stochasticity of GFlowNets-based outputs reduces incentives for manipulation. Experiments show that EMERGENT outperforms RSD in rank efficiency while significantly reducing strategic vulnerability compared to matches produced by RM and PS. Our work highlights the potential of GFlowNets for applications involving social choice mechanisms, where it is crucial to balance efficiency and manipulability.
• 摘要：为分配公共资源的公平有效算法的设计，例如学校招生，住房或医疗居住，具有深远的社会影响。在单方面的匹配问题中，根据排名偏好将个人分配给项目，效率和防策略性之间存在基本的权衡。现有的算法，例如随机串行独裁统治（RSD），概率序列（PS）和等级最小化（RM）（RM）仅捕获此权衡的一侧：RSD是策略性的，但效率低下，而PS和RM则有效，但激励操纵。我们建议新兴的生成流网络（GFLOWNETS）在单方面匹配中的新颖应用，以利用其品尝多样化，高奖励解决方案的能力。在我们的方法中，自然而然地出现了高效和耐操作的匹配：高回报的解决方案产生有效的匹配，而基于Gflownets的输出的随机性减少了对操纵的激励措施。实验表明，与RM和PS产生的匹配相比，紧急情况的排名效率优于RSD，同时显着降低了战略脆弱性。我们的工作突出了Gflownets在涉及社会选择机制的应用中的潜力，这对于平衡效率和可操作性至关重要。

Title: MARché: Fast Masked Autoregressive Image Generation with Cache-Aware Attention

• Authors: Chaoyi Jiang, Sungwoo Kim, Lei Gao, Hossein Entezari Zarch, Won Woo Ro, Murali Annavaram
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12035
• Pdf URL: https://arxiv.org/pdf/2506.12035
• Copy Paste: [[2506.12035]] MARché: Fast Masked Autoregressive Image Generation with Cache-Aware Attention(https://arxiv.org/abs/2506.12035)
• Keywords: generation
• Abstract: Masked autoregressive (MAR) models unify the strengths of masked and autoregressive generation by predicting tokens in a fixed order using bidirectional attention for image generation. While effective, MAR models suffer from significant computational overhead, as they recompute attention and feed-forward representations for all tokens at every decoding step, despite most tokens remaining semantically stable across steps. We propose a training-free generation framework MARché to address this inefficiency through two key components: cache-aware attention and selective KV refresh. Cache-aware attention partitions tokens into active and cached sets, enabling separate computation paths that allow efficient reuse of previously computed key/value projections without compromising full-context modeling. But a cached token cannot be used indefinitely without recomputation due to the changing contextual information over multiple steps. MARché recognizes this challenge and applies a technique called selective KV refresh. Selective KV refresh identifies contextually relevant tokens based on attention scores from newly generated tokens and updates only those tokens that require recomputation, while preserving image generation quality. MARché significantly reduces redundant computation in MAR without modifying the underlying architecture. Empirically, MARché achieves up to 1.7x speedup with negligible impact on image quality, offering a scalable and broadly applicable solution for efficient masked transformer generation.
• 摘要：掩盖自回旋（MAR）模型通过使用双向关注图像生成来预测固定顺序的令牌，从而统一了掩盖和自回归产生的优势。尽管有效，但MAR模型仍遭受了大量的计算开销，尽管它们在每个解码步骤中都重新计算所有令牌的注意力和前馈表示，尽管大多数令牌跨步骤保持语义稳定。我们提出了一个无培训的生成框架Marché，通过两个关键组成部分来解决此效率低下的效率：Cache-AwawawawawawawawawawaweawawaweawawawawawawawawawawawawawaPas所和选择性的KV刷新。缓存感知的注意分区将图表显示为主动和缓存集，从而实现了单独的计算路径，从而可以有效地重复使用先前计算的密钥/值投影，而不会损害全文建模。但是，由于多个步骤的上下文信息的变化，因此没有重新计算，无法无限地使用缓存的令牌。 Marché意识到了这一挑战，并采用了一种称为选择性KV刷新的技术。选择性KV刷新基于新生成的令牌的注意分数标识上下文相关的令牌，并且仅更新那些需要重新计算的令牌，同时保留图像生成质量。 Marché大大减少了MAR中的冗余计算，而无需修改基础体系结构。从经验上讲，Marché可实现高达1.7倍的加速，对图像质量的影响微不足道，为有效的掩盖变压器生成提供了可扩展且广泛的解决方案。

Title: GUST: Quantifying Free-Form Geometric Uncertainty of Metamaterials Using Small Data

• Authors: Jiahui Zheng, Cole Jahnke, Wei "Wayne" Chen
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2506.12051
• Pdf URL: https://arxiv.org/pdf/2506.12051
• Copy Paste: [[2506.12051]] GUST: Quantifying Free-Form Geometric Uncertainty of Metamaterials Using Small Data(https://arxiv.org/abs/2506.12051)
• Keywords: generative
• Abstract: This paper introduces GUST (Generative Uncertainty learning via Self-supervised pretraining and Transfer learning), a framework for quantifying free-form geometric uncertainties inherent in the manufacturing of metamaterials. GUST leverages the representational power of deep generative models to learn a high-dimensional conditional distribution of as-fabricated unit cell geometries given nominal designs, thereby enabling uncertainty quantification. To address the scarcity of real-world manufacturing data, GUST employs a two-stage learning process. First, it leverages self-supervised pretraining on a large-scale synthetic dataset to capture the structure variability inherent in metamaterial geometries and an approximated distribution of as-fabricated geometries given nominal designs. Subsequently, GUST employs transfer learning by fine-tuning the pretrained model on limited real-world manufacturing data, allowing it to adapt to specific manufacturing processes and nominal designs. With only 960 unit cells additively manufactured in only two passes, GUST can capture the variability in geometry and effective material properties. In contrast, directly training a generative model on the same amount of real-world data proves insufficient, as demonstrated through both qualitative and quantitative comparisons. This scalable and cost-effective approach significantly reduces data requirements while maintaining the effectiveness in learning complex, real-world geometric uncertainties, offering an affordable method for free-form geometric uncertainty quantification in the manufacturing of metamaterials. The capabilities of GUST hold significant promise for high-precision industries such as aerospace and biomedical engineering, where understanding and mitigating manufacturing uncertainties are critical.
• 摘要：本文介绍了阵风（通过自我监督预处理和转移学习通过生成不确定性学习），这是量化自由形式的几何不确定性固有的框架。阵风利用深生成模型的代表力来学习鉴于名义设计的高维单位细胞几何形状的高维条件分布，从而实现了不确定性定量。为了解决现实世界制造数据的稀缺性，Gust采用了​​两个阶段的学习过程。首先，它利用在大规模合成数据集上进行自制的预处理，以捕获超材料几何形状固有的结构可变性，并在标称设计给定的几何形状近似分布中。随后，阵风通过在有限的实际制造数据上微调验证的模型来采用转移学习，从而使其适应特定的制造过程和名义设计。只有960个单位单元仅在两次通过中加上生产，阵风就可以捕获几何形状和有效材料特性的变异性。相比之下，直接对相同数量的现实数据训练生成模型被证明不足，如定性和定量比较所证明的那样。这种可扩展且具有成本效益的方法可显着降低数据需求，同时保持学习复杂，现实世界的几何不确定性的有效性，为自由形式的自由形式的不确定性定量提供了一种负担得起的方法。阵风的能力对航空航天和生物医学工程等高精度行业具有巨大的希望，在这种行业中，理解和缓解制造业不确定性至关重要。

Title: Generative or Discriminative? Revisiting Text Classification in the Era of Transformers

• Authors: Siva Rajesh Kasa, Karan Gupta, Sumegh Roychowdhury, Ashutosh Kumar, Yaswanth Biruduraju, Santhosh Kumar Kasa, Nikhil Priyatam Pattisapu, Arindam Bhattacharya, Shailendra Agarwal, Vijay huddar
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.12181
• Pdf URL: https://arxiv.org/pdf/2506.12181
• Copy Paste: [[2506.12181]] Generative or Discriminative? Revisiting Text Classification in the Era of Transformers(https://arxiv.org/abs/2506.12181)
• Keywords: generative
• Abstract: The comparison between discriminative and generative classifiers has intrigued researchers since Efron's seminal analysis of logistic regression versus discriminant analysis. While early theoretical work established that generative classifiers exhibit lower sample complexity but higher asymptotic error in simple linear settings, these trade-offs remain unexplored in the transformer era. We present the first comprehensive evaluation of modern generative and discriminative architectures - Auto-regressive modeling, Masked Language Modeling, Discrete Diffusion, and Encoders for text classification. Our study reveals that the classical 'two regimes' phenomenon manifests distinctly across different architectures and training paradigms. Beyond accuracy, we analyze sample efficiency, calibration, noise robustness, and ordinality across diverse scenarios. Our findings offer practical guidance for selecting the most suitable modeling approach based on real-world constraints such as latency and data limitations.
• 摘要：自EFRON对逻辑回归与判别分析的开创性分析以来，判别和生成分类器之间的比较引起了研究人员的兴趣。尽管早期的理论工作表明，生成分类器在简单的线性设置中表现出较低的样本复杂性，但渐近误差较高，但在变压器时代，这些权衡仍未得到探索。我们介绍了对现代生成和判别架构的首次全面评估 - 自动回归建模，掩盖语言建模，离散扩散以及用于文本分类的编码器。我们的研究表明，经典的“两个制度”现象在不同的​​体系结构和训练范式中明显表现出来。除了准确性之外，我们还分析了各种情况下的样本效率，校准，稳健性和法规性。我们的发现提供了基于实际限制（例如延迟和数据限制）选择最合适的建模方法的实用指导。

Title: ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Models

• Authors: Sibo Dong, Ismail Shaheen, Maggie Shen, Rupayan Mallick, Sarah Adel Bargal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12198
• Pdf URL: https://arxiv.org/pdf/2506.12198
• Copy Paste: [[2506.12198]] ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Models(https://arxiv.org/abs/2506.12198)
• Keywords: generation
• Abstract: Text-to-image diffusion models have achieved remarkable success, yet generating coherent image sequences for visual storytelling remains challenging. A key challenge is effectively leveraging all previous text-image pairs, referred to as history text-image pairs, which provide contextual information for maintaining consistency across frames. Existing auto-regressive methods condition on all past image-text pairs but require extensive training, while training-free subject-specific approaches ensure consistency but lack adaptability to narrative prompts. To address these limitations, we propose a multi-modal history adapter for text-to-image diffusion models, \textbf{ViSTA}. It consists of (1) a multi-modal history fusion module to extract relevant history features and (2) a history adapter to condition the generation on the extracted relevant features. We also introduce a salient history selection strategy during inference, where the most salient history text-image pair is selected, improving the quality of the conditioning. Furthermore, we propose to employ a Visual Question Answering-based metric TIFA to assess text-image alignment in visual storytelling, providing a more targeted and interpretable assessment of generated images. Evaluated on the StorySalon and FlintStonesSV dataset, our proposed ViSTA model is not only consistent across different frames, but also well-aligned with the narrative text descriptions.
• 摘要：文本对图像扩散模型取得了巨大的成功，但为视觉讲故事产生了连贯的图像序列仍然具有挑战性。一个关键的挑战是有效利用所有以前的文本图像对，称为历史文本图像对，该对提供了上下文信息，以维持跨帧的一致性。现有的自动回归方法条件在过去的所有图像文本对上，但需要进行广泛的培训，而无训练的主体特定方法可确保一致性，但缺乏对叙事提示的适应性。为了解决这些限制，我们为文本到图像扩散模型提出了一个多模式历史记录适配器，\ textbf {vista}。它由（1）一个多模式历史融合模块组成，用于提取相关的历史记录特征和（2）历史适配器，以在提取的相关特征上调节生成。我们还在推理期间引入了一个显着的历史选择策略，其中选择了最显着的文本图像对，从而提高了条件的质量。此外，我们建议采用一个视觉问题，以回答基于公制的指标来评估视觉故事讲述中的文本图像对齐，从而对生成的图像进行了更具针对性和可解释的评估。在Storysalon和Flintstonessv数据集上进行了评估，我们提出的Vista模型不仅在不同的框架上保持一致，而且与叙事文本描述相吻合。

Title: Private Continuous-Time Synthetic Trajectory Generation via Mean-Field Langevin Dynamics

• Authors: Anming Gu, Edward Chien, Kristjan Greenewald
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.12203
• Pdf URL: https://arxiv.org/pdf/2506.12203
• Copy Paste: [[2506.12203]] Private Continuous-Time Synthetic Trajectory Generation via Mean-Field Langevin Dynamics(https://arxiv.org/abs/2506.12203)
• Keywords: generation
• Abstract: We provide an algorithm to privately generate continuous-time data (e.g. marginals from stochastic differential equations), which has applications in highly sensitive domains involving time-series data such as healthcare. We leverage the connections between trajectory inference and continuous-time synthetic data generation, along with a computational method based on mean-field Langevin dynamics. As discretized mean-field Langevin dynamics and noisy particle gradient descent are equivalent, DP results for noisy SGD can be applied to our setting. We provide experiments that generate realistic trajectories on a synthesized variation of hand-drawn MNIST data while maintaining meaningful privacy guarantees. Crucially, our method has strong utility guarantees under the setting where each person contributes data for \emph{only one time point}, while prior methods require each person to contribute their \emph{entire temporal trajectory}--directly improving the privacy characteristics by construction.
• 摘要：我们提供了一种算法来私下生成连续时间数据（例如，从随机微分方程中的边际数据），该数据在涉及时间序列数据（例如医疗保健）的高度敏感域中应用。我们利用轨迹推理与连续时间合成数据生成之间的连接，以及基于平均场Langevin动力学的计算方法。由于离散的平均范围langevin动力学和嘈杂的粒子梯度下降等效，因此可以将噪声SGD的DP结果应用于我们的设置。我们提供的实验可以在维持有意义的隐私保证的同时，在手绘MNIST数据的合成变化上产生逼真的轨迹。至关重要的是，我们的方法在每个人贡献\ emph {仅一个时间点}的数据下具有强大的效用保证，而先前的方法要求每个人贡献其\ emph {整个时间轨迹} - 直接通过构造提高隐私特征。

Title: Zero-Shot Scene Understanding with Multimodal Large Language Models for Automated Vehicles

• Authors: Mohammed Elhenawy, Shadi Jaradat, Taqwa I. Alhadidi, Huthaifa I. Ashqar, Ahmed Jaber, Andry Rakotonirainy, Mohammad Abu Tami
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.12232
• Pdf URL: https://arxiv.org/pdf/2506.12232
• Copy Paste: [[2506.12232]] Zero-Shot Scene Understanding with Multimodal Large Language Models for Automated Vehicles(https://arxiv.org/abs/2506.12232)
• Keywords: generation
• Abstract: Scene understanding is critical for various downstream tasks in autonomous driving, including facilitating driver-agent communication and enhancing human-centered explainability of autonomous vehicle (AV) decisions. This paper evaluates the capability of four multimodal large language models (MLLMs), including relatively small models, to understand scenes in a zero-shot, in-context learning setting. Additionally, we explore whether combining these models using an ensemble approach with majority voting can enhance scene understanding performance. Our experiments demonstrate that GPT-4o, the largest model, outperforms the others in scene understanding. However, the performance gap between GPT-4o and the smaller models is relatively modest, suggesting that advanced techniques such as improved in-context learning, retrieval-augmented generation (RAG), or fine-tuning could further optimize the smaller models' performance. We also observe mixed results with the ensemble approach: while some scene attributes show improvement in performance metrics such as F1-score, others experience a decline. These findings highlight the need for more sophisticated ensemble techniques to achieve consistent gains across all scene attributes. This study underscores the potential of leveraging MLLMs for scene understanding and provides insights into optimizing their performance for autonomous driving applications.
• 摘要：场景的理解对于自动驾驶中的各种下游任务至关重要，包括促进驾驶员通讯和增强以人为中心的自动驾驶汽车（AV）决策的解释性。本文评估了四种多模式大型语言模型（包括相对较小的模型）的能力，以了解零拍的场景，即封闭式学习设置。此外，我们探索使用合奏方法与多数投票结合这些模型是否可以增强场景的理解性能。我们的实验表明，最大的模型GPT-4O在场景理解中的表现都优于其他模型。但是，GPT-4O与较小模型之间的性能差距相对谦虚，这表明诸如改进的秘密学习，检索效果生成（RAG）或微调等高级技术可以进一步优化较小的模型的性能。我们还通过合奏方法观察到混合的结果：虽然某些场景属性在诸如F1得分之类的性能指标上显示出改善，但另一些场景的属性则有所下降。这些发现凸显了需要更复杂的整体技术，以在所有场景属性中获得一致的收益。这项研究强调了利用MLLM进行场景理解的潜力，并为优化其自主驾驶应用程序的性能提供了见解。

Title: MatchPlant: An Open-Source Pipeline for UAV-Based Single-Plant Detection and Data Extraction

• Authors: Worasit Sangjan, Piyush Pandey, Norman B. Best, Jacob D. Washburn
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12295
• Pdf URL: https://arxiv.org/pdf/2506.12295
• Copy Paste: [[2506.12295]] MatchPlant: An Open-Source Pipeline for UAV-Based Single-Plant Detection and Data Extraction(https://arxiv.org/abs/2506.12295)
• Keywords: generation
• Abstract: Accurate identification of individual plants from unmanned aerial vehicle (UAV) images is essential for advancing high-throughput phenotyping and supporting data-driven decision-making in plant breeding. This study presents MatchPlant, a modular, graphical user interface-supported, open-source Python pipeline for UAV-based single-plant detection and geospatial trait extraction. MatchPlant enables end-to-end workflows by integrating UAV image processing, user-guided annotation, Convolutional Neural Network model training for object detection, forward projection of bounding boxes onto an orthomosaic, and shapefile generation for spatial phenotypic analysis. In an early-season maize case study, MatchPlant achieved reliable detection performance (validation AP: 89.6%, test AP: 85.9%) and effectively projected bounding boxes, covering 89.8% of manually annotated boxes with 87.5% of projections achieving an Intersection over Union (IoU) greater than 0.5. Trait values extracted from predicted bounding instances showed high agreement with manual annotations (r = 0.87-0.97, IoU >= 0.4). Detection outputs were reused across time points to extract plant height and Normalized Difference Vegetation Index with minimal additional annotation, facilitating efficient temporal phenotyping. By combining modular design, reproducibility, and geospatial precision, MatchPlant offers a scalable framework for UAV-based plant-level analysis with broad applicability in agricultural and environmental monitoring.
• 摘要：从无人驾驶汽车（UAV）图像中准确鉴定单个植物对于推进高通量表型和支持数据驱动的植物育种决策至关重要。这项研究介绍了Matchplant，这是一种模块化的图形用户界面支持的开源Python管道，用于基于无人机的单植物检测和地理空间性状提取。 Matchplant通过集成无人机图像处理，用户引导的注释，卷积神经网络模型训练，用于对象检测，将边界盒的正向投影到正交原状体上以及用于空间表型分析的ShapeFile生成来实现端到端的工作流。在一项早期的玉米案例研究中，Matchplant实现了可靠的检测性能（验证AP：89.6％，测试AP：85.9％）并有效地投影了边界盒，覆盖了87.8％的手动注释盒子，其中87.5％的投影实现了超过0.5的投影。从预测的边界实例中提取的性状值与手动注释相一致（r = 0.87-0.97，iou> = 0.4）。在跨时间点重新使用检测输出，以提取植物高度和归一化差异指数，并以最小的额外注释，从而促进有效的时间表型。通过将模块化设计，可重复性和地理空间精度相结合，Matchplant为基于无人机的植物水平分析提供了可扩展的框架，并在农业和环境监测中广泛适用。

Title: Unveiling Confirmation Bias in Chain-of-Thought Reasoning

• Authors: Yue Wan, Xiaowei Jia, Xiang Lorraine Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12301
• Pdf URL: https://arxiv.org/pdf/2506.12301
• Copy Paste: [[2506.12301]] Unveiling Confirmation Bias in Chain-of-Thought Reasoning(https://arxiv.org/abs/2506.12301)
• Keywords: generation
• Abstract: Chain-of-thought (CoT) prompting has been widely adopted to enhance the reasoning capabilities of large language models (LLMs). However, the effectiveness of CoT reasoning is inconsistent across tasks with different reasoning types. This work presents a novel perspective to understand CoT behavior through the lens of \textit{confirmation bias} in cognitive psychology. Specifically, we examine how model internal beliefs, approximated by direct question-answering probabilities, affect both reasoning generation ($Q \to R$) and reasoning-guided answer prediction ($QR \to A$) in CoT. By decomposing CoT into a two-stage process, we conduct a thorough correlation analysis in model beliefs, rationale attributes, and stage-wise performance. Our results provide strong evidence of confirmation bias in LLMs, such that model beliefs not only skew the reasoning process but also influence how rationales are utilized for answer prediction. Furthermore, the interplay between task vulnerability to confirmation bias and the strength of beliefs also provides explanations for CoT effectiveness across reasoning tasks and models. Overall, this study provides a valuable insight for the needs of better prompting strategies that mitigate confirmation bias to enhance reasoning performance. Code is available at \textit{this https URL}.
• 摘要：经过思考链（COT）提示已被广泛采用，以增强大语言模型（LLMS）的推理能力。但是，在具有不同推理类型的任务之间，COT推理的有效性并不一致。这项工作提出了一种新颖的观点，可以通过认知心理学中的\ textit {确认偏见}的角度来理解婴儿床行为。具体而言，我们研究了通过直接提问概率近似的模型内部信念会影响推理生成（$ q \ to r $）和推理引导的答案预测（$ qr \ to $ $）。通过将COT分解为两个阶段的过程，我们在模型信念，基本原理属性和阶段的性能中进行了彻底的相关分析。我们的结果提供了有力的LLM中确认偏见的有力证据，因此模型信念不仅偏向推理过程，而且还影响了如何利用理由来回答预测。此外，任务脆弱性与确认偏见的相互作用与信念的力量之间的相互作用也为跨推理任务和模型的COT有效性提供了解释。总体而言，这项研究为更好地提示策略的需求提供了宝贵的见解，以减轻确认偏见以提高推理绩效。代码可在\ textit {this https url}上获得。

Title: SPIRE: Conditional Personalization for Federated Diffusion Generative Models

• Authors: Kaan Ozkara, Ruida Zhou, Suhas Diggavi
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.12303
• Pdf URL: https://arxiv.org/pdf/2506.12303
• Copy Paste: [[2506.12303]] SPIRE: Conditional Personalization for Federated Diffusion Generative Models(https://arxiv.org/abs/2506.12303)
• Keywords: generation, generative
• Abstract: Recent advances in diffusion models have revolutionized generative AI, but their sheer size makes on device personalization, and thus effective federated learning (FL), infeasible. We propose Shared Backbone Personal Identity Representation Embeddings (SPIRE), a framework that casts per client diffusion based generation as conditional generation in FL. SPIRE factorizes the network into (i) a high capacity global backbone that learns a population level score function and (ii) lightweight, learnable client embeddings that encode local data statistics. This separation enables parameter efficient finetuning that touches $\leq 0.01\%$ of weights. We provide the first theoretical bridge between conditional diffusion training and maximum likelihood estimation in Gaussian mixture models. For a two component mixture we prove that gradient descent on the DDPM with respect to mixing weights loss recovers the optimal mixing weights and enjoys dimension free error bounds. Our analysis also hints at how client embeddings act as biases that steer a shared score network toward personalized distributions. Empirically, SPIRE matches or surpasses strong baselines during collaborative pretraining, and vastly outperforms them when adapting to unseen clients, reducing Kernel Inception Distance while updating only hundreds of parameters. SPIRE further mitigates catastrophic forgetting and remains robust across finetuning learning rate and epoch choices.
• 摘要：扩散模型的最新进展已彻底改变了生成的AI，但它们的庞大规模在设备个性化方面，因此有效的联合学习（FL）是不可行的。我们提出了共享的骨干个人身份表示嵌入（SPIRE），该框架是在佛罗里达州的有条件产生的每个客户扩散生成的框架。 SPIRE将网络分配到（i）学习人口水平分数功能的高容量全球主链中，以及（ii）编码本地数据统计信息的轻质，可学习的客户嵌入。此分隔启用了触摸$ \ leq 0.01 \％$权重的参数有效的固定。我们在高斯混合模型中提供了条件扩散训练和最大似然估计之间的第一个理论桥。对于两个组件混合物，我们证明DDPM上的梯度下降相对于混合重量损失会恢复最佳混合权重，并享受尺寸自由误差界。我们的分析还暗示了客户嵌入方式是将共享分数网络转向个性化分布的偏见。从经验上讲，在协作预处理期间，尖顶匹配或超过了强大的基线，并且在适应看不见的客户时大大胜过它们，减少了内核成立距离，同时仅更新了数百个参数。尖顶进一步减轻了灾难性的遗忘，并在填充学习率和时期选择方面保持了强大的稳定。

Title: Extending Memorization Dynamics in Pythia Models from Instance-Level Insights

• Authors: Jie Zhang, Qinghua Zhao, Lei Li, Chi-ho Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12321
• Pdf URL: https://arxiv.org/pdf/2506.12321
• Copy Paste: [[2506.12321]] Extending Memorization Dynamics in Pythia Models from Instance-Level Insights(https://arxiv.org/abs/2506.12321)
• Keywords: generation
• Abstract: Large language models have demonstrated a remarkable ability for verbatim memorization. While numerous works have explored factors influencing model memorization, the dynamic evolution memorization patterns remains underexplored. This paper presents a detailed analysis of memorization in the Pythia model family across varying scales and training steps under prefix perturbations. Using granular metrics, we examine how model architecture, data characteristics, and perturbations influence these patterns. Our findings reveal that: (1) as model scale increases, memorization expands incrementally while efficiency decreases rapidly; (2) as model scale increases, the rate of new memorization acquisition decreases while old memorization forgetting increases; (3) data characteristics (token frequency, repetition count, and uncertainty) differentially affect memorized versus non-memorized samples; and (4) prefix perturbations reduce memorization and increase generation uncertainty proportionally to perturbation strength, with low-redundancy samples showing higher vulnerability and larger models offering no additional robustness. These findings advance our understanding of memorization mechanisms, with direct implications for training optimization, privacy safeguards, and architectural improvements.
• 摘要：大型语言模型表现出了逐字记忆的非凡能力。尽管许多作品探索了影响模型记忆的因素，但动态演化记忆模式仍未得到充实。本文介绍了在前缀扰动下在不同尺度和训练步骤中对毕达斯模型家族中记忆的详细分析。使用颗粒指标，我们研究了模型架构，数据特征和扰动如何影响这些模式。我们的发现表明：（1）随着模型量表的增加，记忆会逐渐扩大，而效率迅速下降； （2）随着模型量表的增加，新的记忆采集的速度降低了，而旧的记忆忘记忘记了增加； （3）数据特征（令牌频率，重复计数和不确定性）差异地影响了记忆与非隔离样本的影响； （4）前缀扰动减少了记忆，并增加了与扰动强度成比例地增加产生不确定性的，低额外的样本显示出更高的脆弱性和更大的模型，没有额外的鲁棒性。这些发现提高了我们对记忆机制的理解，对培训优化，隐私保护和建筑改进的直接影响。

Title: Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback

• Authors: Janet Wang, Yunbei Zhang, Zhengming Ding, Jihun Hamm
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12323
• Pdf URL: https://arxiv.org/pdf/2506.12323
• Copy Paste: [[2506.12323]] Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback(https://arxiv.org/abs/2506.12323)
• Keywords: generation
• Abstract: Paucity of medical data severely limits the generalizability of diagnostic ML models, as the full spectrum of disease variability can not be represented by a small clinical dataset. To address this, diffusion models (DMs) have been considered as a promising avenue for synthetic image generation and augmentation. However, they frequently produce medically inaccurate images, deteriorating the model performance. Expert domain knowledge is critical for synthesizing images that correctly encode clinical information, especially when data is scarce and quality outweighs quantity. Existing approaches for incorporating human feedback, such as reinforcement learning (RL) and Direct Preference Optimization (DPO), rely on robust reward functions or demand labor-intensive expert evaluations. Recent progress in Multimodal Large Language Models (MLLMs) reveals their strong visual reasoning capabilities, making them adept candidates as evaluators. In this work, we propose a novel framework, coined MAGIC (Medically Accurate Generation of Images through AI-Expert Collaboration), that synthesizes clinically accurate skin disease images for data augmentation. Our method creatively translates expert-defined criteria into actionable feedback for image synthesis of DMs, significantly improving clinical accuracy while reducing the direct human workload. Experiments demonstrate that our method greatly improves the clinical quality of synthesized skin disease images, with outputs aligning with dermatologist assessments. Additionally, augmenting training data with these synthesized images improves diagnostic accuracy by +9.02% on a challenging 20-condition skin disease classification task, and by +13.89% in the few-shot setting.
• 摘要：医疗数据的匮乏严重限制了诊断ML模型的普遍性，因为疾病变异性的全范围不能由一个小的临床数据集代表。为了解决这个问题，扩散模型（DMS）被认为是合成图像产生和增强的有前途的途径。但是，它们经常产生医学上不准确的图像，从而恶化模型性能。专家领域知识对于综合正确编码临床信息的图像至关重要，尤其是当数据稀缺并且质量大于数量时。现有的合并人类反馈的方法，例如增强学习（RL）和直接偏好优化（DPO），依赖于强大的奖励功能或要求劳动密集型的专家评估。多模式大语言模型（MLLM）的最新进展揭示了它们强大的视觉推理能力，使他们成为评估者的熟练候选人。在这项工作中，我们提出了一个新颖的框架，创造的魔术（通过AI-Expert协作的医学上准确生成图像），它合成了临床准确的皮肤病图像以进行数据增强。我们的方法创造性地将专家定义的标准转化为可行的DMS综合反馈，从而显着提高了临床精度，同时减少了直接的人类工作量。实验表明，我们的方法极大地提高了合成皮肤病图像的临床质量，输出与皮肤科医生评估保持一致。此外，使用这些合成图像增强培训数据可在具有挑战性的20条条件皮肤疾病分类任务中提高诊断准确性 +9.02％，而在几次射击环境中，诊断准确性提高了诊断精度。

Title: UniDet-D: A Unified Dynamic Spectral Attention Model for Object Detection under Adverse Weathers

• Authors: Yuantao Wang, Haowei Yang, Wei Zhang, Shijian Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12324
• Pdf URL: https://arxiv.org/pdf/2506.12324
• Copy Paste: [[2506.12324]] UniDet-D: A Unified Dynamic Spectral Attention Model for Object Detection under Adverse Weathers(https://arxiv.org/abs/2506.12324)
• Keywords: restoration
• Abstract: Real-world object detection is a challenging task where the captured images/videos often suffer from complex degradations due to various adverse weather conditions such as rain, fog, snow, low-light, etc. Despite extensive prior efforts, most existing methods are designed for one specific type of adverse weather with constraints of poor generalization, under-utilization of visual features while handling various image degradations. Leveraging a theoretical analysis on how critical visual details are lost in adverse-weather images, we design UniDet-D, a unified framework that tackles the challenge of object detection under various adverse weather conditions, and achieves object detection and image restoration within a single network. Specifically, the proposed UniDet-D incorporates a dynamic spectral attention mechanism that adaptively emphasizes informative spectral components while suppressing irrelevant ones, enabling more robust and discriminative feature representation across various degradation types. Extensive experiments show that UniDet-D achieves superior detection accuracy across different types of adverse-weather degradation. Furthermore, UniDet-D demonstrates superior generalization towards unseen adverse weather conditions such as sandstorms and rain-fog mixtures, highlighting its great potential for real-world deployment.
• 摘要：现实世界中的对象检测是一项具有挑战性的任务，捕获的图像/视频通常由于各种不良天气条件（例如雨，雾，雪，雪，灯光等）而遭受复杂的降解，尽管进行了广泛的努力，但大多数现有方法设计用于一种特定类型的不利天气，限制较差，具有较差的普遍性，不足的特征，同时处理各种图像降解。我们设计了关于如何在不良天气图像中丢失关键视觉细节的理论分析，我们设计了Unidet-D，这是一个统一的框架，可以应对各种不良天气条件下对象检测的挑战，并在单个网络中实现对象检测和图像恢复。具体而言，拟议的UNIDET-D结合了动态光谱注意机制，该机制在抑制无关的频谱组件的同时适应性地强调了信息的光谱成分，从而使各种降解类型的较强和歧视性特征表示。广泛的实验表明，UNIDET-D在不同类型的不良天气降解方面达到了卓越的检测准确性。此外，UNIDET-D表现出对看不见的不利天气条件（例如沙尘暴和雨雨混合物）的较高概括，突出了其实际部署的巨大潜力。

Title: Three-dimensional Deep Shape Optimization with a Limited Dataset

• Authors: Yongmin Kwon, Namwoo Kang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12326
• Pdf URL: https://arxiv.org/pdf/2506.12326
• Copy Paste: [[2506.12326]] Three-dimensional Deep Shape Optimization with a Limited Dataset(https://arxiv.org/abs/2506.12326)
• Keywords: generative
• Abstract: Generative models have attracted considerable attention for their ability to produce novel shapes. However, their application in mechanical design remains constrained due to the limited size and variability of available datasets. This study proposes a deep learning-based optimization framework specifically tailored for shape optimization with limited datasets, leveraging positional encoding and a Lipschitz regularization term to robustly learn geometric characteristics and maintain a meaningful latent space. Through extensive experiments, the proposed approach demonstrates robustness, generalizability and effectiveness in addressing typical limitations of conventional optimization frameworks. The validity of the methodology is confirmed through multi-objective shape optimization experiments conducted on diverse three-dimensional datasets, including wheels and cars, highlighting the model's versatility in producing practical and high-quality design outcomes even under data-constrained conditions.
• 摘要：生成模型引起了他们产生新形状的能力的极大关注。但是，由于可用数据集的大小和可变性有限，它们在机械设计中的应用仍受到限制。这项研究提出了一个基于深度学习的优化框架，专门针对形状优化，使用有限的数据集，利用位置编码以及Lipschitz正则化项，以鲁棒地学习几何特征并保持有意义的潜在空间。通过广泛的实验，提出的方法证明了在解决常规优化框架的典型局限性方面的鲁棒性，可推广性和有效性。该方法的有效性是通过对包括轮子和汽车在内的不同三维数据集进行的多目标形状优化实验来确认的，即使在数据约束条件下，也强调了该模型在产生实用和高质量设计结果方面的多功能性。

Title: GroupNL: Low-Resource and Robust CNN Design over Cloud and Device

• Authors: Chuntao Ding, Jianhang Xie, Junna Zhang, Salman Raza, Shangguang Wang, Jiannong Cao
• Subjects: cs.CV, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2506.12335
• Pdf URL: https://arxiv.org/pdf/2506.12335
• Copy Paste: [[2506.12335]] GroupNL: Low-Resource and Robust CNN Design over Cloud and Device(https://arxiv.org/abs/2506.12335)
• Keywords: generation
• Abstract: It has become mainstream to deploy Convolutional Neural Network (CNN) models on ubiquitous Internet of Things (IoT) devices with the help of the cloud to provide users with a variety of high-quality services. Most existing methods have two limitations: (i) low robustness in handling corrupted image data collected by IoT devices; and (ii) high consumption of computational and transmission resources. To this end, we propose the Grouped NonLinear transformation generation method (GroupNL), which generates diversified feature maps by utilizing data-agnostic Nonlinear Transformation Functions (NLFs) to improve the robustness of the CNN model. Specifically, partial convolution filters are designated as seed filters in a convolutional layer, and a small set of feature maps, i.e., seed feature maps, are first generated based on vanilla convolution operation. Then, we split seed feature maps into several groups, each with a set of different NLFs, to generate corresponding diverse feature maps with in-place nonlinear processing. Moreover, GroupNL effectively reduces the parameter transmission between multiple nodes during model training by setting the hyperparameters of NLFs to random initialization and not updating them during model training, and reduces the computing resources by using NLFs to generate feature maps instead of most feature maps generated based on sliding windows. Experimental results on CIFAR-10, GTSRB, CIFAR-10-C, Icons50, and ImageNet-1K datasets in NVIDIA RTX GPU platforms show that the proposed GroupNL outperforms other state-of-the-art methods in model robust and training acceleration. Specifically, on the Icons-50 dataset, the accuracy of GroupNL-ResNet-18 achieves approximately 2.86% higher than the vanilla ResNet-18. GroupNL improves training speed by about 53% compared to vanilla CNN when trained on a cluster of 8 NVIDIA RTX 4090 GPUs on the ImageNet-1K dataset.
• 摘要：在云的帮助下，在无处不在的物联网（IoT）设备上部署卷积神经网络（CNN）模型已成为主流，以为用户提供各种高质量的服务。大多数现有方法都有两个局限性：（i）处理物联网设备收集的损坏的图像数据的鲁棒性低； （ii）高消耗计算和传输资源。为此，我们提出了分组的非线性转换生成方法（GroupNL），该方法通过利用数据 - 诺斯语非线性转换函数（NLF）来提高CNN模型的稳健性，从而生成多样化的特征图。具体而言，部分卷积过滤器被指定为卷积层中的种子过滤器，并且首先基于香草卷积操作生成了一小部分特征图，即种子特征图。然后，我们将种子特征图分为几组，每个组都有一组不同的NLF，以生成具有现场非线性处理的相应的不同特征图。此外，GroupNL通过将NLF的超参数设置为随机初始化而不是在模型训练期间不更新它们，从而有效地减少了模型训练过程中多个节点之间的参数传输，并通过使用NLF来生成特征映射而不是基于滑动窗口的大多数特征映射来减少计算资源。 NVIDIA RTX GPU平台中的CIFAR-10，GTSRB，CIFAR-10-C，ICONS50和ImaTEnet-1K数据集的实验结果表明，所提出的GroupNL优于模型鲁棒和训练加速度的其他最新方法。具体而言，在图标-50数据集上，GroupNL-Resnet-18的精度比Vanilla Resnet-18高约2.86％。当在ImagEnet-1K数据集上接受8个NVIDIA RTX 4090 GPU培训时，与Vanilla CNN相比，GroupNL将训练速度提高了约53％。

Title: QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm

• Authors: Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Chen Zhao, Yunji Chen
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.12355
• Pdf URL: https://arxiv.org/pdf/2506.12355
• Copy Paste: [[2506.12355]] QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm(https://arxiv.org/abs/2506.12355)
• Keywords: generation
• Abstract: The attention operator remains a critical performance bottleneck in large language models (LLMs), particularly for long-context scenarios. While FlashAttention is the most widely used and effective GPU-aware acceleration algorithm, it must require time-consuming and hardware-specific manual implementation, limiting adaptability across GPU architectures. Existing LLMs have shown a lot of promise in code generation tasks, but struggle to generate high-performance attention code. The key challenge is it cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance. To address the above challenge, we propose an LLM-friendly Thinking Language (LLM-TL) to help LLMs decouple the generation of high-level optimization logic and low-level implementation on GPU, and enhance LLMs' understanding of attention operator. Along with a 2-stage reasoning workflow, TL-Code generation and translation, the LLMs can automatically generate FlashAttention implementation on diverse GPUs, establishing a self-optimizing paradigm for generating high-performance attention operators in attention-centric algorithms. Verified on A100, RTX8000, and T4 GPUs, the performance of our methods significantly outshines that of vanilla LLMs, achieving a speed-up of up to 35.16x. Besides, our method not only surpasses human-optimized libraries (cuDNN and official library) in most scenarios but also extends support to unsupported hardware and data types, reducing development time from months to minutes compared with human experts.
• 摘要：注意操作员在大语言模型（LLM）中仍然是关键的性能瓶颈，尤其是对于长篇小说方案。虽然闪存是最广泛使用，最有效的GPU感知加速算法，但它必须需要时间耗时和特定于硬件的手动实现，从而限制了跨GPU架构的适应性。现有的LLM在代码生成任务中表现出很大的希望，但很难生成高性能的注意代码。关键的挑战是它无法理解注意操作员的复杂数据流和计算过程，并利用低级原始词来利用GPU性能。为了应对上述挑战，我们建议使用LLM友好的思维语言（LLM-TL），以帮助LLMS将高级优化逻辑和低级实施的产生产生，并增强LLMS对注意操作员的理解。除了2阶段的推理工作流程，TL代码的生成和翻译外，LLM可以自动在不同的GPU上生成闪存的实现，从而建立一个自我优化的范式，以在以注意力为中心的算法中产生高性能注意操作员。在A100，RTX8000和T4 GPU上进行了验证，我们的方法的性能大大超过了Vanilla LLM的性能，达到了高达35.16倍的速度。此外，在大多数情况下，我们的方法不仅超过了人类优化的库（Cudnn和官方图书馆），而且还将支持扩展到无支撑的硬件和数据类型，从而将开发时间从几个月到几分钟缩短到了人类专家。

Title: Scaling Probabilistic Circuits via Monarch Matrices

• Authors: Honghua Zhang, Meihua Dang, Benjie Wang, Stefano Ermon, Nanyun Peng, Guy Van den Broeck
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.12383
• Pdf URL: https://arxiv.org/pdf/2506.12383
• Copy Paste: [[2506.12383]] Scaling Probabilistic Circuits via Monarch Matrices(https://arxiv.org/abs/2506.12383)
• Keywords: generative
• Abstract: Probabilistic Circuits (PCs) are tractable representations of probability distributions allowing for exact and efficient computation of likelihoods and marginals. Recent advancements have improved the scalability of PCs either by leveraging their sparse properties or through the use of tensorized operations for better hardware utilization. However, no existing method fully exploits both aspects simultaneously. In this paper, we propose a novel sparse and structured parameterization for the sum blocks in PCs. By replacing dense matrices with sparse Monarch matrices, we significantly reduce the memory and computation costs, enabling unprecedented scaling of PCs. From a theory perspective, our construction arises naturally from circuit multiplication; from a practical perspective, compared to previous efforts on scaling up tractable probabilistic models, our approach not only achieves state-of-the-art generative modeling performance on challenging benchmarks like Text8, LM1B and ImageNet, but also demonstrates superior scaling behavior, achieving the same performance with substantially less compute as measured by the number of floating-point operations (FLOPs) during training.
• 摘要：概率电路（PC）是概率分布的可寓言表示，允许对可能性和边缘的精确计算。最近的进步通过利用其稀疏特性或使用张力操作来提高PC的可扩展性，以更好地利用硬件。但是，没有现有的方法同时完全利用这两个方面。在本文中，我们为PC中的总和块提出了一种新颖的稀疏和结构化参数化。通过用稀疏的君主矩阵代替密集的矩阵，我们可以显着降低记忆和计算成本，从而实现PC的前所未有的缩放。从理论的角度来看，我们的构建自然来自电路乘法。从实际的角度来看，与以前在扩大可进行可行的概率模型相比，我们的方法不仅在诸如Text8，LM1B和Imagenet等具有挑战性的基准上实现了最新的生成建模性能，而且还表现出了出色的缩放行为，还表现出较小的缩放性能，在浮点数（Floing Plopert）（Floing Plopert）的数量中实现相同的性能，从而实现了相同的性能（实质上）。

Title: Wireless Channel Identification via Conditional Diffusion Model

• Authors: Yuan Li, Zhong Zheng, Chang Liu, Zesong Fei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.12419
• Pdf URL: https://arxiv.org/pdf/2506.12419
• Copy Paste: [[2506.12419]] Wireless Channel Identification via Conditional Diffusion Model(https://arxiv.org/abs/2506.12419)
• Keywords: generative
• Abstract: The identification of channel scenarios in wireless systems plays a crucial role in channel modeling, radio fingerprint positioning, and transceiver design. Traditional methods to classify channel scenarios are based on typical statistical characteristics of channels, such as K-factor, path loss, delay spread, etc. However, statistic-based channel identification methods cannot accurately differentiate implicit features induced by dynamic scatterers, thus performing very poorly in identifying similar channel scenarios. In this paper, we propose a novel channel scenario identification method, formulating the identification task as a maximum a posteriori (MAP) estimation. Furthermore, the MAP estimation is reformulated by a maximum likelihood estimation (MLE), which is then approximated and solved by the conditional generative diffusion model. Specifically, we leverage a transformer network to capture hidden channel features in multiple latent noise spaces within the reverse process of the conditional generative diffusion model. These detailed features, which directly affect likelihood functions in MLE, enable highly accurate scenario identification. Experimental results show that the proposed method outperforms traditional methods, including convolutional neural networks (CNNs), back-propagation neural networks (BPNNs), and random forest-based classifiers, improving the identification accuracy by more than 10%.
• 摘要：无线系统中通道情景的识别在通道建模，无线电指纹定位和收发器设计中起着至关重要的作用。对通道情景进行分类的传统方法是基于频道的典型统计特征，例如K因子，路径丢失，延迟扩散等。但是，基于统计的基于统计的通道识别方法无法准确区分动态散射器引起的隐式特征，从而在识别相似的通道方案方面表现较差。在本文中，我们提出了一种新颖的通道情景识别方法，将标识任务作为最大a后验（MAP）估计。此外，MAP估计是由最大似然估计（MLE）重新制定的，然后通过条件生成扩散模型近似并求解。具体而言，我们利用变压器网络在条件生成扩散模型的反向过程中捕获多个潜在噪声空间中的隐藏通道特征。这些详细的特征直接影响MLE中的似然函数，可以实现高度准确的方案识别。实验结果表明，该提出的方法优于传统方法，包括卷积神经网络（CNN），后传播神经网络（BPNNS）和基于森林的随机分类器，将识别精度提高了10％以上。

Title: Fine-Grained HDR Image Quality Assessment From Noticeably Distorted to Very High Fidelity

• Authors: Mohsen Jenadeleh, Jon Sneyers, Davi Lazzarotto, Shima Mohammadi, Dominik Keller, Atanas Boev, Rakesh Rao Ramachandra Rao, António Pinheiro, Thomas Richter, Alexander Raake, Touradj Ebrahimi, João Ascenso, Dietmar Saupe
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12505
• Pdf URL: https://arxiv.org/pdf/2506.12505
• Copy Paste: [[2506.12505]] Fine-Grained HDR Image Quality Assessment From Noticeably Distorted to Very High Fidelity(https://arxiv.org/abs/2506.12505)
• Keywords: quality assessment
• Abstract: High dynamic range (HDR) and wide color gamut (WCG) technologies significantly improve color reproduction compared to standard dynamic range (SDR) and standard color gamuts, resulting in more accurate, richer, and more immersive images. However, HDR increases data demands, posing challenges for bandwidth efficiency and compression techniques. Advances in compression and display technologies require more precise image quality assessment, particularly in the high-fidelity range where perceptual differences are subtle. To address this gap, we introduce AIC-HDR2025, the first such HDR dataset, comprising 100 test images generated from five HDR sources, each compressed using four codecs at five compression levels. It covers the high-fidelity range, from visible distortions to compression levels below the visually lossless threshold. A subjective study was conducted using the JPEG AIC-3 test methodology, combining plain and boosted triplet comparisons. In total, 34,560 ratings were collected from 151 participants across four fully controlled labs. The results confirm that AIC-3 enables precise HDR quality estimation, with 95\% confidence intervals averaging a width of 0.27 at 1 JND. In addition, several recently proposed objective metrics were evaluated based on their correlation with subjective ratings. The dataset is publicly available.
• 摘要：与标准动态范围（SDR）和标准色域相比，高动态范围（HDR）和宽色域（WCG）技术可显着改善颜色繁殖，从而更准确，更丰富，更丰富，更身临其境的图像。但是，HDR增加了数据需求，对带宽效率和压缩技术提出了挑战。压缩和显示技术的进步需要更精确的图像质量评估，尤其是在知觉差异微妙的高保真范围内。为了解决此差距，我们介绍了AIC-HDR2025，这是第一个这样的HDR数据集，其中包括从五个HDR来源生成的100个测试图像，每个测试图像在五个压缩水平下使用四个编解码器压缩。它涵盖了高保真范围，从可见的畸变到低于视觉上无损阈值的压缩水平。使用JPEG AIC-3测试方法进行了主观研究，结合了平原和增强的三重态比较。总共从四个完全控制的实验室中的151名参与者收集了34,560个评分。结果证实AIC-3可以实现精确的HDR质量估计，而95 \％的置信区间平均为1 JND时宽度为0.27。此外，根据其与主观评分的相关性评估了一些最近提出的客观指标。该数据集公开可用。

Title: Retrieval Augmented Comic Image Generation

• Authors: Yunhao Shui, Xuekuan Wang, Feng Qiu, Yuqiu Huang, Jinzhu Li, Haoyu Zheng, Jinru Han, Zhuo Zeng, Pengpeng Zhang, Jiarui Han, Keqiang Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12517
• Pdf URL: https://arxiv.org/pdf/2506.12517
• Copy Paste: [[2506.12517]] Retrieval Augmented Comic Image Generation(https://arxiv.org/abs/2506.12517)
• Keywords: generation
• Abstract: We present RaCig, a novel system for generating comic-style image sequences with consistent characters and expressive gestures. RaCig addresses two key challenges: (1) maintaining character identity and costume consistency across frames, and (2) producing diverse and vivid character gestures. Our approach integrates a retrieval-based character assignment module, which aligns characters in textual prompts with reference images, and a regional character injection mechanism that embeds character features into specified image regions. Experimental results demonstrate that RaCig effectively generates engaging comic narratives with coherent characters and dynamic interactions. The source code will be publicly available to support further research in this area.
• 摘要：我们提出了Racig，这是一种新型系统，用于生成具有一致的字符和表现力的手势的漫画形象序列。 RACIG解决了两个关键挑战：（1）保持角色身份和跨框架的服装一致性，以及（2）产生多样而生动的角色手势。我们的方法集成了基于检索的字符分配模块，该模块将文本提示中的字符与参考图像保持一致，以及将字符特征嵌入特征中的区域角色注入机制。实验结果表明，RACIG有效地产生了具有连贯的特征和动态相互作用的引人入胜的漫画叙事。源代码将公开支持该领域的进一步研究。

Title: Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting

• Authors: Xingzhong Hou, Jie Wu, Boxiao Liu, Yi Zhang, Guanglu Song, Yunpeng Liu, Yu Liu, Haihang You
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12530
• Pdf URL: https://arxiv.org/pdf/2506.12530
• Copy Paste: [[2506.12530]] Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting(https://arxiv.org/abs/2506.12530)
• Keywords: generative
• Abstract: Image inpainting is the task of reconstructing missing or damaged parts of an image in a way that seamlessly blends with the surrounding content. With the advent of advanced generative models, especially diffusion models and generative adversarial networks, inpainting has achieved remarkable improvements in visual quality and coherence. However, achieving seamless continuity remains a significant challenge. In this work, we propose two novel methods to address discrepancy issues in diffusion-based inpainting models. First, we introduce a modified Variational Autoencoder that corrects color imbalances, ensuring that the final inpainted results are free of color mismatches. Second, we propose a two-step training strategy that improves the blending of generated and existing image content during the diffusion process. Through extensive experiments, we demonstrate that our methods effectively reduce discontinuity and produce high-quality inpainting results that are coherent and visually appealing.
• 摘要：图像介入是以与周围内容无缝融合的方式重建图像中缺失或损坏的部分的任务。随着高级生成模型的出现，尤其是扩散模型和生成的对抗网络，indpainting在视觉质量和连贯性方面取得了显着改善。但是，实现无缝连续性仍然是一个重大挑战。在这项工作中，我们提出了两种新的方法来解决基于扩散的镶嵌模型中的差异问题。首先，我们引入了一个修改后的变性自动编码器，该自动编码器纠正了颜色失衡，以确保最终的成分结果没有颜色不匹配。其次，我们提出了一种两步训练策略，该策略可以改善在扩散过程中生成的图像含量和现有图像内容的混合。通过广泛的实验，我们证明了我们的方法有效地减少了不连续性并产生高质量的覆盖效果，这些结果连贯且具有视觉吸引力。

Title: RAW-Explainer: Post-hoc Explanations of Graph Neural Networks on Knowledge Graphs

• Authors: Ryoji Kubo, Djellel Difallah
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.12558
• Pdf URL: https://arxiv.org/pdf/2506.12558
• Copy Paste: [[2506.12558]] RAW-Explainer: Post-hoc Explanations of Graph Neural Networks on Knowledge Graphs(https://arxiv.org/abs/2506.12558)
• Keywords: generation
• Abstract: Graph neural networks have demonstrated state-of-the-art performance on knowledge graph tasks such as link prediction. However, interpreting GNN predictions remains a challenging open problem. While many GNN explainability methods have been proposed for node or graph-level tasks, approaches for generating explanations for link predictions in heterogeneous settings are limited. In this paper, we propose RAW-Explainer, a novel framework designed to generate connected, concise, and thus interpretable subgraph explanations for link prediction. Our method leverages the heterogeneous information in knowledge graphs to identify connected subgraphs that serve as patterns of factual explanation via a random walk objective. Unlike existing methods tailored to knowledge graphs, our approach employs a neural network to parameterize the explanation generation process, which significantly speeds up the production of collective explanations. Furthermore, RAW-Explainer is designed to overcome the distribution shift issue when evaluating the quality of an explanatory subgraph which is orders of magnitude smaller than the full graph, by proposing a robust evaluator that generalizes to the subgraph distribution. Extensive quantitative results on real-world knowledge graph datasets demonstrate that our approach strikes a balance between explanation quality and computational efficiency.
• 摘要：图神经网络已证明了知识图任务（例如链接预测）的最新性能。但是，解释GNN预测仍然是一个具有挑战性的开放问题。尽管已经为节点或图形级任务提出了许多GNN解释性方法，但在异质设置中生成链接预测的解释的方法受到限制。在本文中，我们提出了原始解释器，这是一个新颖的框架，旨在生成链接预测的连接，简洁，因此可以解释的子图解释。我们的方法利用知识图中的异构信息来识别通过随机步行目标作为事实解释的连接子图。与针对知识图量身定制的现有方法不同，我们的方法采用神经网络来参数化解释生成过程，从而大大加快了集体解释的生产。此外，在评估解释性子图的质量时，原始解释剂旨在克服分布移位问题，该子图的质量比完整图小的数量级，提出了一个稳定的评估器，该评估者将其推广到子级分布。现实世界知识图数据集的广泛定量结果表明，我们的方法在解释质量和计算效率之间取得了平衡。

Title: DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty

• Authors: Mingxuan Cui, Duo Zhou, Yuxuan Han, Grani A. Hanasusanto, Qiong Wang, Huan Zhang, Zhengyuan Zhou
• Subjects: cs.LG, cs.AI, math.OC
• Abstract URL: https://arxiv.org/abs/2506.12622
• Pdf URL: https://arxiv.org/pdf/2506.12622
• Copy Paste: [[2506.12622]] DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty(https://arxiv.org/abs/2506.12622)
• Keywords: generative
• Abstract: Deep reinforcement learning (RL) has achieved significant success, yet its application in real-world scenarios is often hindered by a lack of robustness to environmental uncertainties. To solve this challenge, some robust RL algorithms have been proposed, but most are limited to tabular settings. In this work, we propose Distributionally Robust Soft Actor-Critic (DR-SAC), a novel algorithm designed to enhance the robustness of the state-of-the-art Soft Actor-Critic (SAC) algorithm. DR-SAC aims to maximize the expected value with entropy against the worst possible transition model lying in an uncertainty set. A distributionally robust version of the soft policy iteration is derived with a convergence guarantee. For settings where nominal distributions are unknown, such as offline RL, a generative modeling approach is proposed to estimate the required nominal distributions from data. Furthermore, experimental results on a range of continuous control benchmark tasks demonstrate our algorithm achieves up to $9.8$ times the average reward of the SAC baseline under common perturbations. Additionally, compared with existing robust reinforcement learning algorithms, DR-SAC significantly improves computing efficiency and applicability to large-scale problems.
• 摘要：深度强化学习（RL）取得了巨大的成功，但是在现实情况下的应用通常由于缺乏对环境不确定性的鲁棒性而阻碍。为了解决这一挑战，已经提出了一些强大的RL算法，但大多数仅限于表格设置。在这项工作中，我们提出了分布鲁棒的软critic-Critic（DR-SAC），这是一种新型算法，旨在增强最先进的软角色批评（SAC）算法的鲁棒性。 DR-SAC的目的是通过熵在不确定性集中的最坏可能的过渡模型中最大化期望值。软性策略迭代的分布强大版本是通过收敛保证得出的。对于标称分布未知的设置，例如离线RL，提出了一种生成建模方法来估计数据中所需的名义分布。此外，对一系列连续控制基准任务的实验结果表明，我们的算法达到了$ 9.8的$ 9.8 $ $倍，这是SAC基线在常见扰动下的平均奖励。此外，与现有的强大增强学习算法相比，DR-SAC显着提高了计算效率和对大规模问题的适用性。

Title: Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context

• Authors: Samarth Singhal, Sandeep Singhal
• Subjects: cs.CV, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2506.12683
• Pdf URL: https://arxiv.org/pdf/2506.12683
• Copy Paste: [[2506.12683]] Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context(https://arxiv.org/abs/2506.12683)
• Keywords: generative
• Abstract: Vision-Language Models (VLMs) have rapidly advanced alongside Large Language Models (LLMs). This study evaluates the capabilities of prominent generative VLMs, such as GPT-4.1 and Gemini 2.5 Pro, accessed via APIs, for histopathology image classification tasks, including cell typing. Using diverse datasets from public and private sources, we apply zero-shot and one-shot prompting methods to assess VLM performance, comparing them against custom-trained Convolutional Neural Networks (CNNs). Our findings demonstrate that while one-shot prompting significantly improves VLM performance over zero-shot ($p \approx 1.005 \times 10^{-5}$ based on Kappa scores), these general-purpose VLMs currently underperform supervised CNNs on most tasks. This work underscores both the promise and limitations of applying current VLMs to specialized domains like pathology via in-context learning. All code and instructions for reproducing the study can be accessed from the repository this https URL.
• 摘要：视觉语言模型（VLM）与大型语言模型（LLMS）一起迅速发展。这项研究评估了通过API访问的突出生成VLM的功能，例如GPT-4.1和Gemini 2.5 Pro，用于组织病理学图像分类任务，包括细胞分类。使用来自公共和私人资源的各种数据集，我们应用零射击和一次性提示方法来评估VLM性能，将它们与定制训练的卷积神经网络（CNN）进行比较。我们的发现表明，虽然一杆促使零射击可显着提高VLM性能（基于Kappa得分，$ P \ times 10^{ - 5} $），但这些通用的VLMS目前在大多数任务上都表现不佳。这项工作强调了将当前VLM应用于通过文化学习的专用领域（例如病理学）的希望和局限性。复制研究的所有代码和说明都可以从此HTTPS URL的存储库中访问。

Title: Generative 4D Scene Gaussian Splatting with Object View-Synthesis Priors

• Authors: Wen-Hsuan Chu, Lei Ke, Jianmeng Liu, Mingxiao Huo, Pavel Tokmakov, Katerina Fragkiadaki
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12716
• Pdf URL: https://arxiv.org/pdf/2506.12716
• Copy Paste: [[2506.12716]] Generative 4D Scene Gaussian Splatting with Object View-Synthesis Priors(https://arxiv.org/abs/2506.12716)
• Keywords: generative
• Abstract: We tackle the challenge of generating dynamic 4D scenes from monocular, multi-object videos with heavy occlusions, and introduce GenMOJO, a novel approach that integrates rendering-based deformable 3D Gaussian optimization with generative priors for view synthesis. While existing models perform well on novel view synthesis for isolated objects, they struggle to generalize to complex, cluttered scenes. To address this, GenMOJO decomposes the scene into individual objects, optimizing a differentiable set of deformable Gaussians per object. This object-wise decomposition allows leveraging object-centric diffusion models to infer unobserved regions in novel viewpoints. It performs joint Gaussian splatting to render the full scene, capturing cross-object occlusions, and enabling occlusion-aware supervision. To bridge the gap between object-centric priors and the global frame-centric coordinate system of videos, GenMOJO uses differentiable transformations that align generative and rendering constraints within a unified framework. The resulting model generates 4D object reconstructions over space and time, and produces accurate 2D and 3D point tracks from monocular input. Quantitative evaluations and perceptual human studies confirm that GenMOJO generates more realistic novel views of scenes and produces more accurate point tracks compared to existing approaches.
• 摘要：我们应对从单眼，多对象视频产生动态的4D场景的挑战，并引入Genmojo，这是一种新颖的方法，一种新的方法，将基于渲染的可变形3D高斯优化与生成的先验进行了整合，以进行综合。尽管现有模型在孤立的物体的新颖视图合成上表现良好，但它们努力将其推广到复杂的，混乱的场景。为了解决这个问题，Genmojo将场景分解为各个对象，优化了每个对象的一组可变形的高斯人。通过对象的分解，利用以对象为中心的扩散模型可以在新的观点中推断出未观察到的区域。它执行连接的高斯分裂，以渲染完整的场景，捕获横向对象的阻塞并实现闭塞性的监督。为了弥合以对象为中心的先验与视频中全球为中心的坐标系之间的差距，Genmojo使用了可区分的转换，这些转换可以使生成性和渲染约束在统一的框架内保持一致。所得模型在空间和时间上生成4D对象重建，并从单眼输入中产生准确的2D和3D点轨道。定量评估和知觉人类研究证实，Genmojo与现有方法相比产生了更现实的新颖观点，并产生更准确的点轨道。

Title: SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration

• Authors: Ye Li, Yuan Meng, Zewen Sun, Kangye Ji, Chen Tang, Jiajun Fan, Xinzhu Ma, Shutao Xia, Zhi Wang, Wenwu Zhu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12723
• Pdf URL: https://arxiv.org/pdf/2506.12723
• Copy Paste: [[2506.12723]] SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration(https://arxiv.org/abs/2506.12723)
• Keywords: generation
• Abstract: Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities. However, their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation. Existing VLA acceleration methods primarily focus on structural optimization, overlooking the fact that these models operate in sequential decision-making environments. As a result, temporal redundancy in sequential action generation and spatial redundancy in visual input remain unaddressed. To this end, we propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens. Specifically, we design an action-aware model scheduling mechanism that reduces temporal redundancy by dynamically switching between VLA model and a lightweight generator. Inspired by the human motion pattern of focusing on key decision points while relying on intuition for other actions, we categorize VLA actions into deliberative and intuitive, assigning the former to the VLA model and the latter to the lightweight generator, enabling frequency-adaptive execution through collaborative model scheduling. To address spatial redundancy, we further develop a spatio-semantic dual-aware token pruning method. Tokens are classified into spatial and semantic types and pruned based on their dual-aware importance to accelerate VLA inference. These two mechanisms work jointly to guide the VLA in focusing on critical actions and salient visual information, achieving effective acceleration while maintaining high accuracy. Experimental results demonstrate that our method achieves up to 1.5$\times$ acceleration with less than 3% drop in accuracy, outperforming existing approaches in multiple tasks.
• 摘要：视觉语言动作（VLA）模型引起了人们对其强大控制能力的越来越多的关注。但是，它们的高计算成本和低执行频率阻碍了它们对实时任务（例如机器人操纵和自动导航）的适用性。现有的VLA加速方法主要集中于结构优化，忽略了这些模型在顺序决策环境中运行的事实。结果，视觉输入的顺序动作产生和空间冗余的时间冗余尚未得到解决。为此，我们提出了SP-VLA，这是一个统一的框架，该框架通过共同调度模型和修剪令牌来加速VLA模型。具体而言，我们设计了一种动作感知模型调度机制，该机制通过在VLA模型和轻量级生成器之间动态切换来降低时间冗余。受到人类运动模式的启发，即依靠直觉采取其他动作的直觉，我们将VLA动作分类为审议和直觉，将前者分配给VLA模型，而后者将轻量级生成器分配给轻量级生成器，从而通过协作模型计划使频率适应性执行。为了解决空间冗余，我们进一步开发了一种空间语义双感知的令牌修剪方法。令牌被分类为空间和语义类型，并根据其双重意识到加速VLA推断的重要性修剪。这两种机制共同指导VLA专注于关键动作和显着的视觉信息，在保持高精度的同时达到有效的加速度。实验结果表明，我们的方法达到了高达1.5 $ \ times $加速度，准确性下降不到3％，在多个任务中的现有方法表现优于现有方法。

Title: Dynamic Modality Scheduling for Multimodal Large Models via Confidence, Uncertainty, and Semantic Consistency

• Authors: Hiroshi Tanaka, Anika Rao, Hana Satou, Michael Johnson, Sofia García
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12724
• Pdf URL: https://arxiv.org/pdf/2506.12724
• Copy Paste: [[2506.12724]] Dynamic Modality Scheduling for Multimodal Large Models via Confidence, Uncertainty, and Semantic Consistency(https://arxiv.org/abs/2506.12724)
• Keywords: generation
• Abstract: Multimodal Large Models (MLLMs) have achieved remarkable progress in vision-language understanding and generation tasks. However, existing MLLMs typically rely on static modality fusion strategies, which treat all modalities equally regardless of their instance-level reliability or semantic contribution. This often leads to suboptimal performance, especially in scenarios with noisy, missing, or misaligned modalities. In this paper, we propose Dynamic Modality Scheduling (DMS), a novel framework that adaptively adjusts the contribution of each modality at a per-sample level. DMS evaluates each modality based on three key factors: (1) \textit{confidence}, estimated from predictive entropy; (2) \textit{uncertainty}, obtained via Monte Carlo dropout; and (3) \textit{semantic consistency}, computed through inter-modal similarity. These signals are combined through a learnable or rule-based scheduler to generate soft modality weights used in downstream this http URL ensure stable training, we further introduce a \textit{Modality Weight Consistency Loss}, which regularizes the fused representation to stay close to unimodal embeddings proportionally to their assigned weights. Our method is model-agnostic and can be integrated into existing MLLMs such as BLIP-2 and LLaVA. Experimental results on VQA, image-text retrieval, and captioning tasks show that DMS significantly improves both clean and robust performance, especially under modality corruption or dropout conditions. This work provides a general and effective mechanism to enable instance-aware and robustness-enhanced multimodal modeling.
• 摘要：多模式大型模型（MLLM）在视觉理解和发电任务方面取得了显着进步。但是，现有的MLLM通常依赖于静态方式融合策略，无论其实例级别的可靠性或语义贡献如何，它们都同样对待所有方式。这通常会导致次优的性能，尤其是在嘈杂，缺失或未对准方式的情况下。在本文中，我们提出了动态方式调度（DMS），这是一个新颖的框架，可以自适应地调整每种样本级别的贡献。 DMS基于三个关键因素评估每种模式：（1）\ textit {profors}，根据预测性熵估算； （2）\ textit {不确定性}，通过蒙特卡洛辍学获得； （3）\ textit {语义一致性}，通过模式间相似性计算。这些信号是通过可学习或基于规则的调度程序组合在一起的，以生成在下游的柔软模态权重，该HTTP URL可确保稳定训练，我们进一步引入了\ textIt {模态重量一致性损失}，该{模态重量一致性损失}将融合的表示形式正规化，以保持接近单型的嵌入与分配的权重相称。我们的方法是模型不合时式，可以集成到现有的MLLM中，例如Blip-2和Llava。 VQA，图像文本检索和字幕任务的实验结果表明，DMS显着改善了清洁和健壮的性能，尤其是在模式损坏或辍学条件下。这项工作提供了一种通用有效的机制，可以使实例感知和鲁棒性增强的多模型建模。

Title: Adaptive Dropout: Unleashing Dropout across Layers for Generalizable Image Super-Resolution

• Authors: Hang Xu, Wei Yu, Jiangtong Tan, Zhen Zou, Feng Zhao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12738
• Pdf URL: https://arxiv.org/pdf/2506.12738
• Copy Paste: [[2506.12738]] Adaptive Dropout: Unleashing Dropout across Layers for Generalizable Image Super-Resolution(https://arxiv.org/abs/2506.12738)
• Keywords: restoration, super-resolution
• Abstract: Blind Super-Resolution (blind SR) aims to enhance the model's generalization ability with unknown degradation, yet it still encounters severe overfitting issues. Some previous methods inspired by dropout, which enhances generalization by regularizing features, have shown promising results in blind SR. Nevertheless, these methods focus solely on regularizing features before the final layer and overlook the need for generalization in features at intermediate layers. Without explicit regularization of features at intermediate layers, the blind SR network struggles to obtain well-generalized feature representations. However, the key challenge is that directly applying dropout to intermediate layers leads to a significant performance drop, which we attribute to the inconsistency in training-testing and across layers it introduced. Therefore, we propose Adaptive Dropout, a new regularization method for blind SR models, which mitigates the inconsistency and facilitates application across intermediate layers of networks. Specifically, for training-testing inconsistency, we re-design the form of dropout and integrate the features before and after dropout adaptively. For inconsistency in generalization requirements across different layers, we innovatively design an adaptive training strategy to strengthen feature propagation by layer-wise annealing. Experimental results show that our method outperforms all past regularization methods on both synthetic and real-world benchmark datasets, also highly effective in other image restoration tasks. Code is available at \href{this https URL}{this https URL}.
• 摘要：盲目的超分辨率（盲目SR）旨在通过未知的降解来增强模型的概括能力，但仍然遇到严重的过度拟合问题。通过辍学启发的一些以前的方法，通过正规化特征来增强概括，在盲目的SR中显示出令人鼓舞的结果。然而，这些方法仅着眼于在最后一层之前正规化特征，而忽略了中间层的特征的概括。如果没有明确的特征正规化中间层的正规化，则盲目的SR网络努力获得良好的特征表示。但是，关键的挑战是，将辍学直接应用于中间层导致了显着的性能下降，我们将其归因于训练测试的不一致以及所引入的层。因此，我们提出了适应性辍学，这是一种用于盲目SR模型的新正规化方法，可减轻不一致的情况并促进跨网络中间层的应用。具体而言，对于训练测试不一致，我们重新设计了辍学的形式，并在辍学之前和之后都会集成功能。对于跨不同层的概括要求的不一致，我们创新设计了一种自适应训练策略，以通过层次退火来增强特征传播。实验结果表明，我们的方法在合成和现实基准数据集上都优于过去的所有正规化方法，在其他图像恢复任务中也非常有效。代码可在\ href {此https url} {this https url}中获得。

Title: A large-scale, physically-based synthetic dataset for satellite pose estimation

• Authors: Szabolcs Velkei, Csaba Goldschmidt, Károly Vass
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12782
• Pdf URL: https://arxiv.org/pdf/2506.12782
• Copy Paste: [[2506.12782]] A large-scale, physically-based synthetic dataset for satellite pose estimation(https://arxiv.org/abs/2506.12782)
• Keywords: generation
• Abstract: The Deep Learning Visual Space Simulation System (DLVS3) introduces a novel synthetic dataset generator and a simulation pipeline specifically designed for training and testing satellite pose estimation solutions. This work introduces the DLVS3-HST-V1 dataset, which focuses on the Hubble Space Telescope (HST) as a complex, articulated target. The dataset is generated using advanced real-time and offline rendering technologies, integrating high-fidelity 3D models, dynamic lighting (including secondary sources like Earth reflection), and physically accurate material properties. The pipeline supports the creation of large-scale, richly annotated image sets with ground-truth 6-DoF pose and keypoint data, semantic segmentation, depth, and normal maps. This enables the training and benchmarking of deep learning-based pose estimation solutions under realistic, diverse, and challenging visual conditions. The paper details the dataset generation process, the simulation architecture, and the integration with deep learning frameworks, and positions DLVS3 as a significant step toward closing the domain gap for autonomous spacecraft operations in proximity and servicing missions.
• 摘要：深度学习的视觉空间仿真系统（DLVS3）引入了一种新型的合成数据集发电机和专门设计用于训练和测试卫星姿势估计解决方案的模拟管道。这项工作介绍了DLVS3-HST-V1数据集，该数据集的重点是哈勃太空望远镜（HST）作为一个复杂的，明确的目标。该数据集是使用高级实时和离线渲染技术生成的，集成了高保真3D模型，动态照明（包括辅助源（例如地球反射）以及物理上准确的材料属性。该管道支持具有地面6-DOF姿势和关键点数据，语义分割，深度和正常地图的大规模，注释的图像集的创建。这使得在现实，多样化和具有挑战性的视觉条件下对基于深度学习的姿势估计解决方案进行培训和基准测试。本文详细介绍了数据集生成过程，模拟体系结构以及与深度学习框架的集成，并将DLVS3定位为缩小域间隙的重要一步，以实现自主航天器的操作，以实现接近和维修任务。

Title: Leveraging MIMIC Datasets for Better Digital Health: A Review on Open Problems, Progress Highlights, and Future Promises

• Authors: Afifa Khaled, Mohammed Sabir, Rizwan Qureshi, Camillo Maria Caruso, Valerio Guarrasi, Suncheng Xiang, S Kevin Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12808
• Pdf URL: https://arxiv.org/pdf/2506.12808
• Copy Paste: [[2506.12808]] Leveraging MIMIC Datasets for Better Digital Health: A Review on Open Problems, Progress Highlights, and Future Promises(https://arxiv.org/abs/2506.12808)
• Keywords: generation
• Abstract: The Medical Information Mart for Intensive Care (MIMIC) datasets have become the Kernel of Digital Health Research by providing freely accessible, deidentified records from tens of thousands of critical care admissions, enabling a broad spectrum of applications in clinical decision support, outcome prediction, and healthcare analytics. Although numerous studies and surveys have explored the predictive power and clinical utility of MIMIC based models, critical challenges in data integration, representation, and interoperability remain underexplored. This paper presents a comprehensive survey that focuses uniquely on open problems. We identify persistent issues such as data granularity, cardinality limitations, heterogeneous coding schemes, and ethical constraints that hinder the generalizability and real-time implementation of machine learning models. We highlight key progress in dimensionality reduction, temporal modelling, causal inference, and privacy preserving analytics, while also outlining promising directions including hybrid modelling, federated learning, and standardized preprocessing pipelines. By critically examining these structural limitations and their implications, this survey offers actionable insights to guide the next generation of MIMIC powered digital health innovations.
• 摘要：医疗信息集市（MIMIC）数据集已通过提供数以万计的数以万计的重症监护入院记录，从而成为数字健康研究的内核，从而在临床决策支持，结果预测和医疗保健分析中提供了广泛的应用程序。尽管大量研究和调查探索了基于模拟的模型的预测能力和临床实用性，但数据整合，表示和互操作性的关键挑战仍未得到充实。本文介绍了一项全面的调查，该调查专注于开放问题。我们确定了持续存在的问题，例如数据粒度，基数限制，异质编码方案以及道德约束，从而阻碍了机器学习模型的普遍性和实时实现。我们重点介绍了降低维度，时间建模，因果推理和隐私保护分析方面的关键进展，同时还概述了有希望的方向，包括混合建模，联合学习和标准化的预处理管道。通过严格检查这些结构限制及其含义，该调查提供了可行的见解，以指导下一代模拟动力的数字健康创新。

Title: Flow-Based Policy for Online Reinforcement Learning

• Authors: Lei Lv, Yunfei Li, Yu Luo, Fuchun Sun, Tao Kong, Jiafeng Xu, Xiao Ma
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.12811
• Pdf URL: https://arxiv.org/pdf/2506.12811
• Copy Paste: [[2506.12811]] Flow-Based Policy for Online Reinforcement Learning(https://arxiv.org/abs/2506.12811)
• Keywords: generative
• Abstract: We present \textbf{FlowRL}, a novel framework for online reinforcement learning that integrates flow-based policy representation with Wasserstein-2-regularized optimization. We argue that in addition to training signals, enhancing the expressiveness of the policy class is crucial for the performance gains in RL. Flow-based generative models offer such potential, excelling at capturing complex, multimodal action distributions. However, their direct application in online RL is challenging due to a fundamental objective mismatch: standard flow training optimizes for static data imitation, while RL requires value-based policy optimization through a dynamic buffer, leading to difficult optimization landscapes. FlowRL first models policies via a state-dependent velocity field, generating actions through deterministic ODE integration from noise. We derive a constrained policy search objective that jointly maximizes Q through the flow policy while bounding the Wasserstein-2 distance to a behavior-optimal policy implicitly derived from the replay buffer. This formulation effectively aligns the flow optimization with the RL objective, enabling efficient and value-aware policy learning despite the complexity of the policy class. Empirical evaluations on DMControl and Humanoidbench demonstrate that FlowRL achieves competitive performance in online reinforcement learning benchmarks.
• 摘要：我们提出\ textbf {flowrl}，这是一个在线增强学习的新型框架，将基于流量的策略表示与Wasserstein-2调节优化整合在一起。我们认为，除了培训信号外，增强政策类别的表现力对于RL的性能提高至关重要。基于流量的生成模型具有这种潜力，擅长捕获复杂的多模式动作分布。但是，由于基本的目标不匹配，它们在在线RL中的直接应用是具有挑战性的：标准流训练优化了静态数据模仿，而RL则需要通过动态缓冲区进行基于价值的策略优化，从而导致难以优化的景观。 FlowRL首先通过依赖状态的速度场对策略进行建模，从而通过噪声从确定性的ODE集成来生成动作。我们得出了一个受限制的策略搜索目标，该目标通过流动策略共同最大化Q，同时将Wasserstein-2距离界定到​​隐式从重播缓冲区派生的行为最佳策略。这种表述有效地使流程优化与RL目标保持一致，尽管政策类别的复杂性，但仍可以提高和有价值的政策学习。对DMCONTROL和HUMUNOIDBENCH的经验评估表明，FlowRL在在线增强学习基准中实现了竞争性能。

Title: Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Models

• Authors: Tung Minh Luu, Younghwan Lee, Donghoon Lee, Sunho Kim, Min Jun Kim, Chang D. Yoo
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2506.12822
• Pdf URL: https://arxiv.org/pdf/2506.12822
• Copy Paste: [[2506.12822]] Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Models(https://arxiv.org/abs/2506.12822)
• Keywords: generation
• Abstract: Designing effective reward functions remains a fundamental challenge in reinforcement learning (RL), as it often requires extensive human effort and domain expertise. While RL from human feedback has been successful in aligning agents with human intent, acquiring high-quality feedback is costly and labor-intensive, limiting its scalability. Recent advancements in foundation models present a promising alternative--leveraging AI-generated feedback to reduce reliance on human supervision in reward learning. Building on this paradigm, we introduce ERL-VLM, an enhanced rating-based RL method that effectively learns reward functions from AI feedback. Unlike prior methods that rely on pairwise comparisons, ERL-VLM queries large vision-language models (VLMs) for absolute ratings of individual trajectories, enabling more expressive feedback and improved sample efficiency. Additionally, we propose key enhancements to rating-based RL, addressing instability issues caused by data imbalance and noisy labels. Through extensive experiments across both low-level and high-level control tasks, we demonstrate that ERL-VLM significantly outperforms existing VLM-based reward generation methods. Our results demonstrate the potential of AI feedback for scaling RL with minimal human intervention, paving the way for more autonomous and efficient reward learning.
• 摘要：设计有效的奖励功能仍然是强化学习（RL）的基本挑战，因为它通常需要广泛的人类努力和领域的专业知识。尽管来自人类反馈的RL已经成功地使代理人与人类意图保持一致，但获得高质量的反馈是昂贵且劳动密集型的，从而限制了其可扩展性。基础模型中的最新进展提出了一种有希望的替代方案 - 掌握AI生成的反馈，以减少对奖励学习中人类监督的依赖。在此范式的基础上，我们介绍了ERL-VLM，这是一种增强的基于评分的RL方法，可以有效地从AI反馈中学习奖励功能。与依靠成对比较的先前方法不同，ERL-VLM查询大型视觉模型（VLM），以实现单个轨迹的绝对等级，从而实现了更具表现力的反馈和提高样品效率。此外，我们提出了基于评级的RL的关键增强功能，以解决由数据不平衡和嘈杂标签引起的不稳定性问题。通过在低级和高级控制任务上进行的广泛实验，我们证明了ERL-VLM显着优于现有的基于VLM的奖励生成方法。我们的结果表明，AI反馈通过最少的人类干预来扩展RL的潜力，为更加自主和有效的奖励学习铺平了道路。

Title: Learning Unpaired Image Dehazing with Physics-based Rehazy Generation

• Authors: Haoyou Deng, Zhiqiang Li, Feng Zhang, Qingbo Lu, Zisheng Cao, Yuanjie Shao, Shuhang Gu, Changxin Gao, Nong Sang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12824
• Pdf URL: https://arxiv.org/pdf/2506.12824
• Copy Paste: [[2506.12824]] Learning Unpaired Image Dehazing with Physics-based Rehazy Generation(https://arxiv.org/abs/2506.12824)
• Keywords: generation
• Abstract: Overfitting to synthetic training pairs remains a critical challenge in image dehazing, leading to poor generalization capability to real-world scenarios. To address this issue, existing approaches utilize unpaired realistic data for training, employing CycleGAN or contrastive learning frameworks. Despite their progress, these methods often suffer from training instability, resulting in limited dehazing performance. In this paper, we propose a novel training strategy for unpaired image dehazing, termed Rehazy, to improve both dehazing performance and training stability. This strategy explores the consistency of the underlying clean images across hazy images and utilizes hazy-rehazy pairs for effective learning of real haze characteristics. To favorably construct hazy-rehazy pairs, we develop a physics-based rehazy generation pipeline, which is theoretically validated to reliably produce high-quality rehazy images. Additionally, leveraging the rehazy strategy, we introduce a dual-branch framework for dehazing network training, where a clean branch provides a basic dehazing capability in a synthetic manner, and a hazy branch enhances the generalization ability with hazy-rehazy pairs. Moreover, we design a new dehazing network within these branches to improve the efficiency, which progressively restores clean scenes from coarse to fine. Extensive experiments on four benchmarks demonstrate the superior performance of our approach, exceeding the previous state-of-the-art methods by 3.58 dB on the SOTS-Indoor dataset and by 1.85 dB on the SOTS-Outdoor dataset in PSNR. Our code will be publicly available.
• 摘要：过度适合合成训练对仍然是图像去悬式的一个关键挑战，从而导致对现实世界情景的概括能力不佳。为了解决这个问题，现有的方法利用了未配对的现实数据进行培训，采用自行车或对比学习框架。尽管取得了进展，但这些方法通常会遭受训练不稳定的困扰，从而导致表现有限。在本文中，我们提出了一种新颖的培训策略，用于改善不成对的图像除尘，以提高除狂的性能和训练稳定性。该策略探讨了跨朦胧图像的潜在干净图像的一致性，并利用朦胧的幽默对有效学习真实的雾霾特征。为了有利地构建朦胧的hehazy对，我们开发了基于物理的rehazy生成管道，从理论上讲，该管道可可靠地产生高质量的重新rehazy图像。此外，利用rehazy策略，我们引入了一个双分支框架，以进行除去的网络培训，其中一个干净的分支以合成的方式提供了基本的除吊索功能，并且一个朦胧的分支通过朦胧的hear-rehazy Pairs增强了概括能力。此外，我们在这些分支机构内设计了一个新的飞行网络，以提高效率，从而逐步恢复了从粗糙到细小的清洁场景。对四个基准测试的广泛实验证明了我们方法的出色性能，在SOTS-Indoor数据集上超过了先前的最新方法，在PSNR中的SOTS-OUTDOOR数据集上超过了1.85 dB。我们的代码将公开可用。

Title: DiffS-NOCS: 3D Point Cloud Reconstruction through Coloring Sketches to NOCS Maps Using Diffusion Models

• Authors: Di Kong, Qianhui Wan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12835
• Pdf URL: https://arxiv.org/pdf/2506.12835
• Copy Paste: [[2506.12835]] DiffS-NOCS: 3D Point Cloud Reconstruction through Coloring Sketches to NOCS Maps Using Diffusion Models(https://arxiv.org/abs/2506.12835)
• Keywords: generation
• Abstract: Reconstructing a 3D point cloud from a given conditional sketch is challenging. Existing methods often work directly in 3D space, but domain variability and difficulty in reconstructing accurate 3D structures from 2D sketches remain significant obstacles. Moreover, ideal models should also accept prompts for control, in addition with the sparse sketch, posing challenges in multi-modal fusion. We propose DiffS-NOCS (Diffusion-based Sketch-to-NOCS Map), which leverages ControlNet with a modified multi-view decoder to generate NOCS maps with embedded 3D structure and position information in 2D space from sketches. The 3D point cloud is reconstructed by combining multiple NOCS maps from different views. To enhance sketch understanding, we integrate a viewpoint encoder for extracting viewpoint features. Additionally, we design a feature-level multi-view aggregation network as the denoising module, facilitating cross-view information exchange and improving 3D consistency in NOCS map generation. Experiments on ShapeNet demonstrate that DiffS-NOCS achieves controllable and fine-grained point cloud reconstruction aligned with sketches.
• 摘要：从给定的条件草图中重建3D点云是具有挑战性的。现有方法通常直接在3D空间中起作用，但是域的可变性和难以从2D草图重建准确的3D结构仍然是重要的障碍。此外，理想的模型还应接受控制提示，除了稀疏的草图外，还应在多模式融合中提出挑战。我们提出了DIFFS-NOC（基于扩散的草图到NOCS映射），它利用使用修改的多视图解码器来利用ControlNet生成具有嵌入式3D结构的NOCS映射，并从草图中从2D空间中进行了位置信息。 3D点云是通过从不同视图的多个NOC映射组合来重建的。为了增强草图理解，我们集成了一个视点编码器，以提取观点特征。此外，我们将功能级的多视图聚合网络设计为denoising模块，促进了跨视图信息交换并提高了NOCS MAP生成中的3D一致性。对塑料的实验表明，差异-NOC达到了与草图一致的可控且细粒的点云重建。

Title: CAPO: Reinforcing Consistent Reasoning in Medical Decision-Making

• Authors: Songtao Jiang, Yuan Wang, Ruizhe Chen, Yan Zhang, Ruilin Luo, Bohan Lei, Sibo Song, Yang Feng, Jimeng Sun, Jian Wu, Zuozhu Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12849
• Pdf URL: https://arxiv.org/pdf/2506.12849
• Copy Paste: [[2506.12849]] CAPO: Reinforcing Consistent Reasoning in Medical Decision-Making(https://arxiv.org/abs/2506.12849)
• Keywords: generation
• Abstract: In medical visual question answering (Med-VQA), achieving accurate responses relies on three critical steps: precise perception of medical imaging data, logical reasoning grounded in visual input and textual questions, and coherent answer derivation from the reasoning process. Recent advances in general vision-language models (VLMs) show that large-scale reinforcement learning (RL) could significantly enhance both reasoning capabilities and overall model performance. However, their application in medical domains is hindered by two fundamental challenges: 1) misalignment between perceptual understanding and reasoning stages, and 2) inconsistency between reasoning pathways and answer generation, both compounded by the scarcity of high-quality medical datasets for effective large-scale RL. In this paper, we first introduce Med-Zero-17K, a curated dataset for pure RL-based training, encompassing over 30 medical image modalities and 24 clinical tasks. Moreover, we propose a novel large-scale RL framework for Med-VLMs, Consistency-Aware Preference Optimization (CAPO), which integrates rewards to ensure fidelity between perception and reasoning, consistency in reasoning-to-answer derivation, and rule-based accuracy for final responses. Extensive experiments on both in-domain and out-of-domain scenarios demonstrate the superiority of our method over strong VLM baselines, showcasing strong generalization capability to 3D Med-VQA benchmarks and R1-like training paradigms.
• 摘要：在医学视觉问题回答（MED-VQA）中，实现准确的答案取决于三个关键步骤：对医学成像数据的精确感知，逻辑推理基于视觉输入和文本问题，以及从推理过程中的一致答案。一般视觉模型（VLM）的最新进展表明，大规模的增强学习（RL）可以显着增强推理能力和整体模型性能。但是，两个基本挑战阻碍了它们在医疗领域中的应用：1）感知理解和推理阶段之间的错位，以及2）推理途径和答案产生之间的不一致，这两者都因缺乏有效的大型大型RL而缺乏高质量的医疗数据集。在本文中，我们首先引入了Med-Zero-17K，这是一个用于纯RL培训的策划数据集，涵盖了30多种医疗图像方式和24个临床任务。此外，我们提出了一个新型的大型RL RL框架，用于Med-vlms，一致性吸引偏好优化（CAPO），该框架集成了奖励，以确保感知和推理之间的忠诚度，推理对答案衍生的一致性以及基于规则的基于规则的准确性。对内域和室外场景的广泛实验证明了我们方法比强VLM基准的优越性，表明了对3D Med-VQA基准和R1样训练范式的强概括能力。

Title: EraserDiT: Fast Video Inpainting with Diffusion Transformer Model

• Authors: Jie Liu, Zheng Hui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.12853
• Pdf URL: https://arxiv.org/pdf/2506.12853
• Copy Paste: [[2506.12853]] EraserDiT: Fast Video Inpainting with Diffusion Transformer Model(https://arxiv.org/abs/2506.12853)
• Keywords: restoration
• Abstract: Video object removal and inpainting are critical tasks in the fields of computer vision and multimedia processing, aimed at restoring missing or corrupted regions in video sequences. Traditional methods predominantly rely on flow-based propagation and spatio-temporal Transformers, but these approaches face limitations in effectively leveraging long-term temporal features and ensuring temporal consistency in the completion results, particularly when dealing with large masks. Consequently, performance on extensive masked areas remains suboptimal. To address these challenges, this paper introduces a novel video inpainting approach leveraging the Diffusion Transformer (DiT). DiT synergistically combines the advantages of diffusion models and transformer architectures to maintain long-term temporal consistency while ensuring high-quality inpainting results. We propose a Circular Position-Shift strategy to further enhance long-term temporal consistency during the inference stage. Additionally, the proposed method automatically detects objects within videos, interactively removes specified objects, and generates corresponding prompts. In terms of processing speed, it takes only 180 seconds (testing on one NVIDIA A100 GPU) to complete a video with a resolution of $1080 \times 1920$ with 121 frames without any acceleration method. Experimental results indicate that the proposed method demonstrates superior performance in content fidelity, texture restoration, and temporal consistency. Project page: this https URL.
• 摘要：视频对象的删除和介绍是计算机视觉和多媒体处理领域的关键任务，旨在恢复视频序列中缺失或损坏的区域。传统方法主要依赖于基于流的传播和时空变压器，但是这些方法在有效利用长期时间特征并确保完成结果中的时间一致性方面面临局限性，尤其是在处理大型面具时。因此，在广泛的掩蔽区域上的性能仍然不佳。为了应对这些挑战，本文介绍了一种新型的视频介绍方法，利用扩散变压器（DIT）。 DIT协同结合了扩散模型和变压器体系结构的优势，以保持长期的时间一致性，同时确保高质量的填充结果。我们提出了一种循环迁移策略，以进一步提高推理阶段的长期时间一致性。此外，提出的方法会自动检测到视频中的对象，交互式删除指定的对象，并生成相应的提示。就处理速度而言，仅需180秒（在一个NVIDIA A100 GPU上进行测试）即可完成一个带有$ 1080 \ times 1920 $的视频，但没有121帧，没有任何加速方法。实验结果表明，所提出的方法表明内容保真度，纹理恢复和时间一致性方面表现出色。项目页面：此HTTPS URL。

Title: WildCAT3D: Appearance-Aware Multi-View Diffusion in the Wild

• Authors: Morris Alper, David Novotny, Filippos Kokkinos, Hadar Averbuch-Elor, Tom Monnier
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13030
• Pdf URL: https://arxiv.org/pdf/2506.13030
• Copy Paste: [[2506.13030]] WildCAT3D: Appearance-Aware Multi-View Diffusion in the Wild(https://arxiv.org/abs/2506.13030)
• Keywords: generation
• Abstract: Despite recent advances in sparse novel view synthesis (NVS) applied to object-centric scenes, scene-level NVS remains a challenge. A central issue is the lack of available clean multi-view training data, beyond manually curated datasets with limited diversity, camera variation, or licensing issues. On the other hand, an abundance of diverse and permissively-licensed data exists in the wild, consisting of scenes with varying appearances (illuminations, transient occlusions, etc.) from sources such as tourist photos. To this end, we present WildCAT3D, a framework for generating novel views of scenes learned from diverse 2D scene image data captured in the wild. We unlock training on these data sources by explicitly modeling global appearance conditions in images, extending the state-of-the-art multi-view diffusion paradigm to learn from scene views of varying appearances. Our trained model generalizes to new scenes at inference time, enabling the generation of multiple consistent novel views. WildCAT3D provides state-of-the-art results on single-view NVS in object- and scene-level settings, while training on strictly less data sources than prior methods. Additionally, it enables novel applications by providing global appearance control during generation.
• 摘要：尽管稀疏的新型视图合成（NVS）的最新进展应用于以对象为中心的场景，但场景级的NVS仍然是一个挑战。一个核心问题是缺乏可用的清洁多视图培训数据，除了手动策划的数据集具有有限的多样性，摄像头变化或许可问题外。另一方面，野外存在大量和允许许可的数据，其中包括来自旅游照片等来源的场景（照明，瞬态遮挡等）的场景。为此，我们介绍了Wildcat3d，这是一个框架，用于生成从野外捕获的不同2D场景图像数据中学到的新景的框架。我们通过在图像中明确建模全球外观条件，扩展了最新的多视图扩散范式来解锁这些数据源的培训，以从不同外观的场景视图中学习。我们受过训练的模型在推理时期将新场景推广到了新的场景，从而能够产生多种一致的新颖观点。 Wildcat3d在对象和场景级别设置中为单视NV提供了最先进的结果，而与先前的方法相比，严格的数据源训练更少。此外，它通过在发电期间提供全球外观控制来实现新颖的应用。

Title: A Comprehensive Survey on Continual Learning in Generative Models

• Authors: Haiyang Guo, Fanhu Zeng, Fei Zhu, Jiayi Wang, Xukai Wang, Jingang Zhou, Hongbo Zhao, Wenzhuo Liu, Shijie Ma, Xu-Yao Zhang, Cheng-Lin Liu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13045
• Pdf URL: https://arxiv.org/pdf/2506.13045
• Copy Paste: [[2506.13045]] A Comprehensive Survey on Continual Learning in Generative Models(https://arxiv.org/abs/2506.13045)
• Keywords: generative
• Abstract: The rapid advancement of generative models has enabled modern AI systems to comprehend and produce highly sophisticated content, even achieving human-level performance in specific domains. However, these models remain fundamentally constrained by catastrophic forgetting - a persistent challenge where adapting to new tasks typically leads to significant degradation in performance on previously learned tasks. To address this practical limitation, numerous approaches have been proposed to enhance the adaptability and scalability of generative models in real-world applications. In this work, we present a comprehensive survey of continual learning methods for mainstream generative models, including large language models, multimodal large language models, vision language action models, and diffusion models. Drawing inspiration from the memory mechanisms of the human brain, we systematically categorize these approaches into three paradigms: architecture-based, regularization-based, and replay-based methods, while elucidating their underlying methodologies and motivations. We further analyze continual learning setups for different generative models, including training objectives, benchmarks, and core backbones, offering deeper insights into the field. The project page of this paper is available at this https URL.
• 摘要：生成模型的快速发展使现代的AI系统能够理解和生成高度复杂的内容，甚至可以在特定领域中实现人类水平的性能。但是，这些模型从根本上仍然受到灾难性遗忘的限制 - 这是一个持续的挑战，即适应新任务通常会导致以前学到的任务的绩效显着下降。为了解决这一实际限制，已经提出了许多方法来增强现实世界应用中生成模型的适应性和可伸缩性。在这项工作中，我们对主流生成模型的持续学习方法进行了全面调查，包括大语言模型，多模式大语言模型，视觉语言动作模型和扩散模型。从人脑的记忆机制中汲取灵感，我们将这些方法系统地分为三种范式：基于建筑，基于正规化和基于重播的方法，同时阐明了它们的基本方法和动机。我们进一步分析了不同生成模型的持续学习设置，包括培训目标，基准和核心骨干，为该领域提供了更深入的见解。本文的项目页面可在此HTTPS URL上找到。

Title: Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning

• Authors: Can Polat, Hasan Kurban, Erchin Serpedin, Mustafa Kurban
• Subjects: cs.CV, cond-mat.mtrl-sci, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.13051
• Pdf URL: https://arxiv.org/pdf/2506.13051
• Copy Paste: [[2506.13051]] Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning(https://arxiv.org/abs/2506.13051)
• Keywords: generative
• Abstract: Evaluating foundation models for crystallographic reasoning requires benchmarks that isolate generalization behavior while enforcing physical constraints. This work introduces a multiscale multicrystal dataset with two physically grounded evaluation protocols to stress-test multimodal generative models. The Spatial-Exclusion benchmark withholds all supercells of a given radius from a diverse dataset, enabling controlled assessments of spatial interpolation and extrapolation. The Compositional-Exclusion benchmark omits all samples of a specific chemical composition, probing generalization across stoichiometries. Nine vision--language foundation models are prompted with crystallographic images and textual context to generate structural annotations. Responses are evaluated via (i) relative errors in lattice parameters and density, (ii) a physics-consistency index penalizing volumetric violations, and (iii) a hallucination score capturing geometric outliers and invalid space-group predictions. These benchmarks establish a reproducible, physically informed framework for assessing generalization, consistency, and reliability in large-scale multimodal models. Dataset and code are available at this https URL.
• 摘要：评估晶体学推理的基础模型需要在实施物理约束的同时隔离概括行为的基准。这项工作介绍了一个多尺度多晶数据集，其中包含两个物理扎根的评估协议，以进行压力测试多模式生成模型。空间排斥基准从不同的数据集中扣除给定半径的所有超级细胞，从而可以对空间插值和外推的受控评估。组成 - 排斥基准省略了特定化学组成的所有样品，从而探测了跨石化的概括。九个视觉 - 语言基础模型是通过晶体学图像和文本上下文引起的，以生成结构注释。通过（i）晶格参数和密度的相对误差评估响应，（ii）物理矛盾指数惩罚体积违规，以及（iii）幻觉评分捕获几何异常值和无效的空间组预测。这些基准建立了一个可再现的，有理由的框架，用于评估大规模多模型模型中的概括，一致性和可靠性。数据集和代码可在此HTTPS URL上找到。

Title: DualFast: Dual-Speedup Framework for Fast Sampling of Diffusion Models

• Authors: Hu Yu, Hao Luo, Fan Wang, Feng Zhao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13058
• Pdf URL: https://arxiv.org/pdf/2506.13058
• Copy Paste: [[2506.13058]] DualFast: Dual-Speedup Framework for Fast Sampling of Diffusion Models(https://arxiv.org/abs/2506.13058)
• Keywords: generation
• Abstract: Diffusion probabilistic models (DPMs) have achieved impressive success in visual generation. While, they suffer from slow inference speed due to iterative sampling. Employing fewer sampling steps is an intuitive solution, but this will also introduces discretization error. Existing fast samplers make inspiring efforts to reduce discretization error through the adoption of high-order solvers, potentially reaching a plateau in terms of optimization. This raises the question: can the sampling process be accelerated further? In this paper, we re-examine the nature of sampling errors, discerning that they comprise two distinct elements: the widely recognized discretization error and the less explored approximation error. Our research elucidates the dynamics between these errors and the step by implementing a dual-error disentanglement strategy. Building on these foundations, we introduce an unified and training-free acceleration framework, DualFast, designed to enhance the speed of DPM sampling by concurrently accounting for both error types, thereby minimizing the total sampling error. DualFast is seamlessly compatible with existing samplers and significantly boost their sampling quality and speed, particularly in extremely few sampling steps. We substantiate the effectiveness of our framework through comprehensive experiments, spanning both unconditional and conditional sampling domains, across both pixel-space and latent-space DPMs.
• 摘要：扩散概率模型（DPM）在视觉产生中取得了令人印象深刻的成功。而由于迭代采样，他们的推理速度缓慢。采用较少的采样步骤是一个直观的解决方案，但这也将引入离散化错误。现有的快速采样器通过采用高阶求解器来减少离散错误的努力，从而有可能在优化方面达到平稳性。这提出了一个问题：可以进一步加速抽样过程吗？在本文中，我们重新检查了抽样错误的性质，并辨别出它们构成了两个不同的元素：广泛认可的离散误差和较少探索的近似误差。我们的研究通过实施双回交解散策略来阐明这些错误与步骤之间的动态。在这些基础的基础上，我们引入了DualFast的统一且无训练的加速框架，旨在通过同时考虑这两种错误类型来提高DPM采样速度，从而最大程度地减少了总采样错误。 Dualfast与现有采样器无缝兼容，并显着提高了它们的采样质量和速度，尤其是在很少的采样步骤中。我们通过全面的实验来证实框架在像素空间和潜在空间DPM的无条件和条件采样域之间的有效性。

Title: Dynamic Graph Condensation

• Authors: Dong Chen, Shuai Zheng, Yeyu Yan, Muhao Xu, Zhenfeng Zhu, Yao Zhao, Kunlun He
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13099
• Pdf URL: https://arxiv.org/pdf/2506.13099
• Copy Paste: [[2506.13099]] Dynamic Graph Condensation(https://arxiv.org/abs/2506.13099)
• Keywords: generation
• Abstract: Recent research on deep graph learning has shifted from static to dynamic graphs, motivated by the evolving behaviors observed in complex real-world systems. However, the temporal extension in dynamic graphs poses significant data efficiency challenges, including increased data volume, high spatiotemporal redundancy, and reliance on costly dynamic graph neural networks (DGNNs). To alleviate the concerns, we pioneer the study of dynamic graph condensation (DGC), which aims to substantially reduce the scale of dynamic graphs for data-efficient DGNN training. Accordingly, we propose DyGC, a novel framework that condenses the real dynamic graph into a compact version while faithfully preserving the inherent spatiotemporal characteristics. Specifically, to endow synthetic graphs with realistic evolving structures, a novel spiking structure generation mechanism is introduced. It draws on the dynamic behavior of spiking neurons to model temporally-aware connectivity in dynamic graphs. Given the tightly coupled spatiotemporal dependencies, DyGC proposes a tailored distribution matching approach that first constructs a semantically rich state evolving field for dynamic graphs, and then performs fine-grained spatiotemporal state alignment to guide the optimization of the condensed graph. Experiments across multiple dynamic graph datasets and representative DGNN architectures demonstrate the effectiveness of DyGC. Notably, our method retains up to 96.2% DGNN performance with only 0.5% of the original graph size, and achieves up to 1846 times training speedup.
• 摘要：深图学习的最新研究已从静态图表转变为动态图，这是由在复杂的现实世界中观察到的不断发展的行为所激发的。但是，动态图中的时间扩展会带来重大的数据效率挑战，包括增加数据量，高时空冗余性以及对昂贵的动态图神经网络（DGNN）的依赖。为了减轻关注点，我们开创了动态图凝结（DGC）的研究，该研究旨在大大减少用于数据效率的DGNN训练的动态图的规模。因此，我们提出了DYGC，这是一个新颖的框架，将真实动态图凝结成紧凑的版本，同时忠实地保留了固有的时空特征。具体而言，要赋予具有逼真的结构的合成图，引入了一种新型的尖峰结构生成机制。它利用尖峰神经元的动态行为来模拟动态图中的时间感知的连接性。鉴于紧密耦合的时空依赖性，DYGC提出了一种定制的分布匹配方法，该方法首先构建了动态图的语义上富含语言状态的状态进化的场，然后执行细粒的时空状态对齐，以指导凝结图的优化。跨多个动态图数据集和代表性DGNN体系结构进行的实验证明了DYGC的有效性。值得注意的是，我们的方法仅保留96.2％的DGNN性能，仅占原始图形尺寸的0.5％，并且达到了1846次训练速度。

Title: STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulation

• Authors: Jiamin Wang, Yichen Yao, Xiang Feng, Hang Wu, Yaming Wang, Qingqiu Huang, Yuexin Ma, Xinge Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13138
• Pdf URL: https://arxiv.org/pdf/2506.13138
• Copy Paste: [[2506.13138]] STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulation(https://arxiv.org/abs/2506.13138)
• Keywords: generation, generative
• Abstract: The generation of temporally consistent, high-fidelity driving videos over extended horizons presents a fundamental challenge in autonomous driving world modeling. Existing approaches often suffer from error accumulation and feature misalignment due to inadequate decoupling of spatio-temporal dynamics and limited cross-frame feature propagation mechanisms. To address these limitations, we present STAGE (Streaming Temporal Attention Generative Engine), a novel auto-regressive framework that pioneers hierarchical feature coordination and multi-phase optimization for sustainable video synthesis. To achieve high-quality long-horizon driving video generation, we introduce Hierarchical Temporal Feature Transfer (HTFT) and a novel multi-stage training strategy. HTFT enhances temporal consistency between video frames throughout the video generation process by modeling the temporal and denoising process separately and transferring denoising features between frames. The multi-stage training strategy is to divide the training into three stages, through model decoupling and auto-regressive inference process simulation, thereby accelerating model convergence and reducing error accumulation. Experiments on the Nuscenes dataset show that STAGE has significantly surpassed existing methods in the long-horizon driving video generation task. In addition, we also explored STAGE's ability to generate unlimited-length driving videos. We generated 600 frames of high-quality driving videos on the Nuscenes dataset, which far exceeds the maximum length achievable by existing methods.
• 摘要：通过扩展视野在时间上一致，高保真驱动视频的产生给自动驾驶世界建模带来了根本挑战。现有的方法通常由于空间动力学不足和有限的跨帧特征传播机制而受到误差积累和特征错位的影响。为了解决这些局限性，我们介绍了阶段（流式暂时关注生成引擎），这是一个新型的自动回归框架，开创了层次层次特征协调和用于可持续视频综合的多相优化。为了实现高质量的长途驱动视频，我们介绍了分层时间特征转移（HTFT）和一种新颖的多阶段训练策略。 HTFT通过分别对时间和降解过程进行建模并在框架之间传递DeNo的特征，从而增强了视频生成过程中视频帧之间的时间一致性。多阶段训练策略是通过模型解耦和自动回归推理过程模拟将训练分为三个阶段，从而加速模型收敛并减少误差积累。 Nuscenes数据集上的实验表明，阶段已经显着超过了长途驾驶视频生成任务中的现有方法。此外，我们还探索了Stage生成无限长度驾驶视频的能力。我们在Nuscenes数据集上生成了600帧的高质量驾驶视频，这远远超过了现有方法可实现的最大长度。

Title: StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation

• Authors: Jiashu He, Jiayi He, Shengeng Tang, Huixia Ben, Lechao Cheng, Richang Hong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13156
• Pdf URL: https://arxiv.org/pdf/2506.13156
• Copy Paste: [[2506.13156]] StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation(https://arxiv.org/abs/2506.13156)
• Keywords: generation
• Abstract: Sign language transition generation seeks to convert discrete sign language segments into continuous sign videos by synthesizing smooth transitions. However,most existing methods merely concatenate isolated signs, resulting in poor visual coherence and semantic accuracy in the generated videos. Unlike textual languages,sign language is inherently rich in spatial-temporal cues, making it more complex to model. To address this,we propose StgcDiff, a graph-based conditional diffusion framework that generates smooth transitions between discrete signs by capturing the unique spatial-temporal dependencies of sign language. Specifically, we first train an encoder-decoder architecture to learn a structure-aware representation of spatial-temporal skeleton sequences. Next, we optimize a diffusion denoiser conditioned on the representations learned by the pre-trained encoder, which is tasked with predicting transition frames from noise. Additionally, we design the Sign-GCN module as the key component in our framework, which effectively models the spatial-temporal features. Extensive experiments conducted on the PHOENIX14T, USTC-CSL100,and USTC-SLR500 datasets demonstrate the superior performance of our method.
• 摘要：手语过渡生成试图通过综合平稳过渡来将离散的手语段转换为连续的符号视频。但是，大多数现有的方法只是连接孤立的符号，导致视觉视频中的视觉连贯性和语义精度差。与文本语言不同，手语本质上富含空间 - 时空提示，使其更加复杂。为了解决这个问题，我们提出了STGCDIFF，这是一个基于图的条件扩散框架，通过捕获手语的唯一时空依赖性，从而在离散标志之间生成平稳的过渡。具体而言，我们首先训练编码器架构来学习时空骨骼序列的结构感知表示。接下来，我们优化了一个基于预训练编码器所学的表示的扩散DeNoiser，该表示器的任务是预测噪声的过渡帧。此外，我们将SIGN-GCN模块设计为框架中的关键组件，该组件有效地建模了时空特征。在Phoenix14T，USTC-CSL100和USTC-SLR500数据集上进行的广泛实验证明了我们方法的出色性能。

Title: High-Quality Facial Albedo Generation for 3D Face Reconstruction from a Single Image using a Coarse-to-Fine Approach

• Authors: Jiashu Dai, Along Wang, Binfan Ni, Tao Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13233
• Pdf URL: https://arxiv.org/pdf/2506.13233
• Copy Paste: [[2506.13233]] High-Quality Facial Albedo Generation for 3D Face Reconstruction from a Single Image using a Coarse-to-Fine Approach(https://arxiv.org/abs/2506.13233)
• Keywords: generation
• Abstract: Facial texture generation is crucial for high-fidelity 3D face reconstruction from a single image. However, existing methods struggle to generate UV albedo maps with high-frequency details. To address this challenge, we propose a novel end-to-end coarse-to-fine approach for UV albedo map generation. Our method first utilizes a UV Albedo Parametric Model (UVAPM), driven by low-dimensional coefficients, to generate coarse albedo maps with skin tones and low-frequency texture details. To capture high-frequency details, we train a detail generator using a decoupled albedo map dataset, producing high-resolution albedo maps. Extensive experiments demonstrate that our method can generate high-fidelity textures from a single image, outperforming existing methods in terms of texture quality and realism. The code and pre-trained model are publicly available at this https URL, facilitating reproducibility and further research.
• 摘要：面部纹理产生对于来自单个图像的高保真3D面部重建至关重要。但是，现有的方法难以生成具有高频细节的紫外线反照率图。为了应对这一挑战，我们为UV反照率图的生成提出了一种新颖的端到端的粗到最佳方法。我们的方法首先利用由低维系数驱动的UV反照率参数模型（UVAPM）来生成带有肤色和低频纹理细节的粗制反照率图。为了捕获高频细节，我们使用脱钩的反照率映射数据集训练细节生成器，从而产生高分辨率的反照率图。广泛的实验表明，我们的方法可以从单个图像中生成高保真纹理，从而在纹理质量和现实主义方面优于现有方法。该代码和预培训模型在此HTTPS URL上公开可用，可促进可重复性和进一步的研究。

Title: Fair Generation without Unfair Distortions: Debiasing Text-to-Image Generation with Entanglement-Free Attention

• Authors: Jeonghoon Park, Juyoung Lee, Chaeyeon Chung, Jaeseong Lee, Jaegul Choo, Jindong Gu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13298
• Pdf URL: https://arxiv.org/pdf/2506.13298
• Copy Paste: [[2506.13298]] Fair Generation without Unfair Distortions: Debiasing Text-to-Image Generation with Entanglement-Free Attention(https://arxiv.org/abs/2506.13298)
• Keywords: generation
• Abstract: Recent advancements in diffusion-based text-to-image (T2I) models have enabled the generation of high-quality and photorealistic images from text descriptions. However, they often exhibit societal biases related to gender, race, and socioeconomic status, thereby reinforcing harmful stereotypes and shaping public perception in unintended ways. While existing bias mitigation methods demonstrate effectiveness, they often encounter attribute entanglement, where adjustments to attributes relevant to the bias (i.e., target attributes) unintentionally alter attributes unassociated with the bias (i.e., non-target attributes), causing undesirable distribution shifts. To address this challenge, we introduce Entanglement-Free Attention (EFA), a method that accurately incorporates target attributes (e.g., White, Black, Asian, and Indian) while preserving non-target attributes (e.g., background details) during bias mitigation. At inference time, EFA randomly samples a target attribute with equal probability and adjusts the cross-attention in selected layers to incorporate the sampled attribute, achieving a fair distribution of target attributes. Extensive experiments demonstrate that EFA outperforms existing methods in mitigating bias while preserving non-target attributes, thereby maintaining the output distribution and generation capability of the original model.
• 摘要：基于扩散的文本形象（T2I）模型的最新进展已使文本描述中的高质量和影像学图像产生。但是，它们经常表现出与性别，种族和社会经济地位有关的社会偏见，从而增强了有害的刻板印象，并以意想不到的方式塑造了公众的看法。尽管现有缓解方法表现出有效性，但它们经常遇到属性纠缠，其中调整与偏见相关的属性（即目标属性）的调整无意中改变了与偏见（即非目标属性）无关的属性，从而导致不可能的分布变化。为了应对这一挑战，我们引入了无纠缠的注意力（EFA），这种方法可以准确地包含目标属性（例如，白色，黑人，亚洲和印度），同时保留了偏置缓解期间的非目标属性（例如，背景细节）。在推论时，EFA随机对目标属性进行同等概率采样，并调整所选层中的交叉注意，以结合采样属性，从而达到目标属性的公平分布。广泛的实验表明，EFA在缓解偏差的同时优于现有方法，同时保留非目标属性，从而维持原始模型的输出分布和生成能力。

Title: Quantitative Comparison of Fine-Tuning Techniques for Pretrained Latent Diffusion Models in the Generation of Unseen SAR Image Concepts

• Authors: Solène Debuysère, Nicolas Trouvé, Nathan Letheule, Olivier Lévêque, Elise Colin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13307
• Pdf URL: https://arxiv.org/pdf/2506.13307
• Copy Paste: [[2506.13307]] Quantitative Comparison of Fine-Tuning Techniques for Pretrained Latent Diffusion Models in the Generation of Unseen SAR Image Concepts(https://arxiv.org/abs/2506.13307)
• Keywords: generation, generative
• Abstract: This work investigates the adaptation of large pre-trained latent diffusion models to a radically new imaging domain: Synthetic Aperture Radar (SAR). While these generative models, originally trained on natural images, demonstrate impressive capabilities in text-to-image synthesis, they are not natively adapted to represent SAR data, which involves different physics, statistical distributions, and visual characteristics. Using a sizeable SAR dataset (on the order of 100,000 to 1 million images), we address the fundamental question of fine-tuning such models for this unseen modality. We explore and compare multiple fine-tuning strategies, including full model fine-tuning and parameter-efficient approaches like Low-Rank Adaptation (LoRA), focusing separately on the UNet diffusion backbone and the text encoder components. To evaluate generative quality, we combine several metrics: statistical distance from real SAR distributions, textural similarity via GLCM descriptors, and semantic alignment assessed with a CLIP model fine-tuned on SAR data. Our results show that a hybrid tuning strategy yields the best performance: full fine-tuning of the UNet is better at capturing low-level SAR-specific patterns, while LoRA-based partial tuning of the text encoder, combined with embedding learning of the token, suffices to preserve prompt alignment. This work provides a methodical strategy for adapting foundation models to unconventional imaging modalities beyond natural image domains.
• 摘要：这项工作研究了大型预训练的潜在扩散模型对根本新成像域的适应：合成孔径雷达（SAR）。尽管这些生成模型最初是在自然图像上训练的，但在文本对图像合成中表现出了令人印象深刻的功能，但它们并未在本地适应代表SAR数据，而SAR数据涉及不同的物理学，统计分布和视觉特征。使用相当大的SAR数据集（按100,000至100万张图像的顺序），我们解决了这种看不见模式的微调模型的基本问题。我们探索和比较了多种微调策略，包括完整的模型微调和参数效率高效方法，例如低级适应（LORA），分别集中在UNET扩散式主链和文本编码器组件上。为了评估生成质量，我们结合了几个指标：与真实SAR分布的统计距离，通过GLCM描述符的纹理相似性以及使用SAR数据进行微调的剪辑模型评估的语义对齐。我们的结果表明，混合调整策略可以产生最佳性能：UNET的完整微调更好地捕获低级SAR特异性模式，而基于Lora的基于Lora的部分调整文本编码器，结合嵌入学习 token的学习，​​足以保留及时的对齐。这项工作为将基础模型调整为自然图像领域以外的非常规成像方式提供了有条理的策略。

Title: VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation

• Authors: Bo Pan, Yixiao Fu, Ke Wang, Junyu Lu, Lunke Pan, Ziyang Qian, Yuhan Chen, Guoliang Wang, Yitao Zhou, Li Zheng, Yinghao Tang, Zhen Wen, Yuchen Wu, Junhua Lu, Biao Zhu, Minfeng Zhu, Bo Zhang, Wei Chen
• Subjects: cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2506.13326
• Pdf URL: https://arxiv.org/pdf/2506.13326
• Copy Paste: [[2506.13326]] VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation(https://arxiv.org/abs/2506.13326)
• Keywords: generation
• Abstract: Data visualization generation using Large Language Models (LLMs) has shown promising results but often produces suboptimal visualizations that require human intervention for improvement. In this work, we introduce VIS-Shepherd, a specialized Multimodal Large Language Model (MLLM)-based critic to evaluate and provide feedback for LLM-generated data visualizations. At the core of our approach is a framework to construct a high-quality visualization critique dataset, where we collect human-created visualization instances, synthesize corresponding LLM-generated instances, and construct high-quality critiques. We conduct both model-based automatic evaluation and human preference studies to evaluate the effectiveness of our approach. Our experiments show that even small (7B parameters) open-source MLLM models achieve substantial performance gains by leveraging our high-quality visualization critique dataset, reaching levels comparable to much larger open-source or even proprietary models. Our work demonstrates significant potential for MLLM-based automated visualization critique and indicates promising directions for enhancing LLM-based data visualization generation. Our project page: this https URL.
• 摘要：使用大语言模型（LLM）的数据可视化生成已显示出令人鼓舞的结果，但经常会产生次优的可视化，需要人体干预才能改进。在这项工作中，我们介绍了一种专门的多模式大型语言模型（MLLM）的评论家Vis-Shepherd，以评估和提供LLM生成数据可视化的反馈。我们方法的核心是构建高质量可视化批评数据集的框架，在该数据集中我们收集了人类创建的可视化实例，合成相应的LLM生成实例并构建高质量的批评。我们进行基于模型的自动评估和人类偏好研究，以评估我们方法的有效性。我们的实验表明，即使是小（7b参数）开源MLLM模型，也可以通过利用我们的高质量可视化评论数据集来实现可观的性能增长，达到的水平与更大的开源源甚至专有模型相当。我们的工作证明了基于MLLM的自动化可视化批评的巨大潜力，并指出了增强基于LLM的数据可视化生成的有希望的方向。我们的项目页面：此HTTPS URL。

Title: LapDDPM: A Conditional Graph Diffusion Model for scRNA-seq Generation with Spectral Adversarial Perturbations

• Authors: Lorenzo Bini, Stephane Marchand-Maillet
• Subjects: cs.LG, cs.AI, q-bio.BM, q-bio.CB, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2506.13344
• Pdf URL: https://arxiv.org/pdf/2506.13344
• Copy Paste: [[2506.13344]] LapDDPM: A Conditional Graph Diffusion Model for scRNA-seq Generation with Spectral Adversarial Perturbations(https://arxiv.org/abs/2506.13344)
• Keywords: generation, generative
• Abstract: Generating high-fidelity and biologically plausible synthetic single-cell RNA sequencing (scRNA-seq) data, especially with conditional control, is challenging due to its high dimensionality, sparsity, and complex biological variations. Existing generative models often struggle to capture these unique characteristics and ensure robustness to structural noise in cellular networks. We introduce LapDDPM, a novel conditional Graph Diffusion Probabilistic Model for robust and high-fidelity scRNA-seq generation. LapDDPM uniquely integrates graph-based representations with a score-based diffusion model, enhanced by a novel spectral adversarial perturbation mechanism on graph edge weights. Our contributions are threefold: we leverage Laplacian Positional Encodings (LPEs) to enrich the latent space with crucial cellular relationship information; we develop a conditional score-based diffusion model for effective learning and generation from complex scRNA-seq distributions; and we employ a unique spectral adversarial training scheme on graph edge weights, boosting robustness against structural variations. Extensive experiments on diverse scRNA-seq datasets demonstrate LapDDPM's superior performance, achieving high fidelity and generating biologically-plausible, cell-type-specific samples. LapDDPM sets a new benchmark for conditional scRNA-seq data generation, offering a robust tool for various downstream biological applications.
• 摘要：由于其高维度，稀疏性和复杂的生物学变化，生成高保真性和生物学上合成的单细胞RNA测序（SCRNA-SEQ）数据，尤其是在条件控制的情况下，具有挑战性。现有的生成模型通常难以捕获这些独特的特征并确保细胞网络中结构噪声的鲁棒性。我们介绍了LAPDDPM，这是一种新型的条件图扩散概率模型，用于鲁棒和高保真SCRNA-SCRNA-SEQ IDEN。 LAPDDPM唯一地将基于图的表示与基于得分的扩散模型整合在一起，并通过图形边缘权重的新型光谱对抗扰动机制增强。我们的贡献是三重的：我们利用拉普拉斯的位置编码（LPES）通过重要的细胞关系信息来丰富潜在空间；我们开发了一个基于条件分数的扩散模型，以从复杂的SCRNA-SEQ分布中有效学习和生成；我们在图边缘的重量上采用了独特的光谱对抗训练方案，从而提高了针对结构变化的鲁棒性。关于不同SCRNA-SEQ数据集的广泛实验证明了LAPDDPM的出色性能，达到了高保真度并生成了生物学上的细胞类型特异性样品。 LAPDDPM为有条件的SCRNA-SEQ数据生成设定了新的基准，为各种下游生物学应用提供了可靠的工具。

Title: DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration

• Authors: Yan Chen, Hanlin Shang, Ce Liu, Yuxuan Chen, Hui Li, Weihao Yuan, Hao Zhu, Zilong Dong, Siyu Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13355
• Pdf URL: https://arxiv.org/pdf/2506.13355
• Copy Paste: [[2506.13355]] DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration(https://arxiv.org/abs/2506.13355)
• Keywords: restoration
• Abstract: Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at this https URL.
• 摘要：视频面部恢复面临着保持时间一致性的关键挑战，同时从退化的输入中恢复了细微的面部细节。本文提出了一种新颖的方法，该方法通过静态高质量的肖像预测的矢量定量变分自动编码器（VQ-VAE）通过变异的潜在空间建模介绍了视频修复框架。我们的关键创新在于将离散代码书的表示形式重新编制为dirichlet分布的连续变量，从而实现了跨帧面部特征之间的概率过渡。时空变压器架构共同模拟了框架间的依赖性并预测潜在分布，而拉普拉斯受限的重建损失与感知（LPIPS）正则化相结合，提高了像素精度和视觉质量。对盲面修复，视频介绍和面部着色任务的全面评估表明了最先进的表现。这项工作建立了一个有效的范式，用于调整强化图像先验，鉴定在高质量图像上，以恢复视频，同时解决闪烁伪影的关键挑战。源代码已开源，可在此HTTPS URL上找到。

Title: Zero-Shot Solving of Imaging Inverse Problems via Noise-Refined Likelihood Guided Diffusion Models

• Authors: Zhen Wang, Hongyi Liu, Zhihui Wei
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.13391
• Pdf URL: https://arxiv.org/pdf/2506.13391
• Copy Paste: [[2506.13391]] Zero-Shot Solving of Imaging Inverse Problems via Noise-Refined Likelihood Guided Diffusion Models(https://arxiv.org/abs/2506.13391)
• Keywords: restoration, generative
• Abstract: Diffusion models have achieved remarkable success in imaging inverse problems owing to their powerful generative capabilities. However, existing approaches typically rely on models trained for specific degradation types, limiting their generalizability to various degradation scenarios. To address this limitation, we propose a zero-shot framework capable of handling various imaging inverse problems without model retraining. We introduce a likelihood-guided noise refinement mechanism that derives a closed-form approximation of the likelihood score, simplifying score estimation and avoiding expensive gradient computations. This estimated score is subsequently utilized to refine the model-predicted noise, thereby better aligning the restoration process with the generative framework of diffusion models. In addition, we integrate the Denoising Diffusion Implicit Models (DDIM) sampling strategy to further improve inference efficiency. The proposed mechanism can be applied to both optimization-based and sampling-based schemes, providing an effective and flexible zero-shot solution for imaging inverse problems. Extensive experiments demonstrate that our method achieves superior performance across multiple inverse problems, particularly in compressive sensing, delivering high-quality reconstructions even at an extremely low sampling rate (5%).
• 摘要：由于其强大的生成能力，扩散模型在成像逆问题中取得了显着的成功。但是，现有方法通常依赖于经过特定降解类型的训练的模型，从而将其推广性限制在各种降解方案中。为了解决这一限制，我们提出了一个零拍的框架，该框架能够处理各种成像逆问题而无需模型再培训。我们介绍了一种可能引导的噪声精炼机制，该机制得出了可能分数的封闭形式近似，简化了得分估计并避免了昂贵的梯度计算。随后，该估计的分数用于完善模型预测的噪声，从而更好地将恢复过程与扩散模型的生成框架保持一致。此外，我们整合了denoising扩散隐式模型（DDIM）采样策略，以进一步提高推理效率。提出的机制可以应用于基于优化的基于优化的方案，为成像逆问题提供了有效且灵活的零击解决方案。广泛的实验表明，我们的方法在多个反问题上达到了卓越的性能，尤其是在压缩感应中，即使以极低的采样率（5％），也可以提供高质量的重建。

Title: Overcoming Occlusions in the Wild: A Multi-Task Age Head Approach to Age Estimation

• Authors: Waqar Tanveer, Laura Fernández-Robles, Eduardo Fidalgo, Víctor González-Castro, Enrique Alegre
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.13445
• Pdf URL: https://arxiv.org/pdf/2506.13445
• Copy Paste: [[2506.13445]] Overcoming Occlusions in the Wild: A Multi-Task Age Head Approach to Age Estimation(https://arxiv.org/abs/2506.13445)
• Keywords: generative
• Abstract: Facial age estimation has achieved considerable success under controlled conditions. However, in unconstrained real-world scenarios, which are often referred to as 'in the wild', age estimation remains challenging, especially when faces are partially occluded, which may obscure their visibility. To address this limitation, we propose a new approach integrating generative adversarial networks (GANs) and transformer architectures to enable robust age estimation from occluded faces. We employ an SN-Patch GAN to effectively remove occlusions, while an Attentive Residual Convolution Module (ARCM), paired with a Swin Transformer, enhances feature representation. Additionally, we introduce a Multi-Task Age Head (MTAH) that combines regression and distribution learning, further improving age estimation under occlusion. Experimental results on the FG-NET, UTKFace, and MORPH datasets demonstrate that our proposed approach surpasses existing state-of-the-art techniques for occluded facial age estimation by achieving an MAE of $3.00$, $4.54$, and $2.53$ years, respectively.
• 摘要：面部年龄估计在受控条件下取得了巨大的成功。但是，在不受约束的现实情况下，通常称为“野外”，年龄估计仍然具有挑战性，尤其是当面部被部分遮挡时，这可能会掩盖其可见性。为了解决这一限制，我们提出了一种新的方法，该方法集成了生成对抗网络（GAN）和变压器体系结构，以从遮挡的面孔中实现强大的年龄估计。我们使用SN-Patch GAN有效地消除闭塞，而细心的残留卷积模块（ARCM）与Swin Transformer配对，增强了特征表示。此外，我们引入了一个多任务年龄头（MTAH），该年龄率（MTAH）结合了回归和分布学习，进一步改善了遮挡下的年龄估计。 FG-NET，UTKFACE和MORPH数据集的实验结果表明，我们所提出的方法分别通过达到$ 3.00 $，$ 4.54 $和$ 2.53 $ $ 2.53美元的MAE，超过了现有的面部面部年龄估计的最新技术。

Title: SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer

• Authors: Zerui Gong, Zhonghua Wu, Qingyi Tao, Qinyue Li, Chen Change Loy
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.13465
• Pdf URL: https://arxiv.org/pdf/2506.13465
• Copy Paste: [[2506.13465]] SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer(https://arxiv.org/abs/2506.13465)
• Keywords: generation
• Abstract: Photorealistic style transfer (PST) enables real-world color grading by adapting reference image colors while preserving content structure. Existing methods mainly follow either approaches: generation-based methods that prioritize stylistic fidelity at the cost of content integrity and efficiency, or global color transformation methods such as LUT, which preserve structure but lack local adaptability. To bridge this gap, we propose Spatial Adaptive 4D Look-Up Table (SA-LUT), combining LUT efficiency with neural network adaptability. SA-LUT features: (1) a Style-guided 4D LUT Generator that extracts multi-scale features from the style image to predict a 4D LUT, and (2) a Context Generator using content-style cross-attention to produce a context map. This context map enables spatially-adaptive adjustments, allowing our 4D LUT to apply precise color transformations while preserving structural integrity. To establish a rigorous evaluation framework for photorealistic style transfer, we introduce PST50, the first benchmark specifically designed for PST assessment. Experiments demonstrate that SA-LUT substantially outperforms state-of-the-art methods, achieving a 66.7% reduction in LPIPS score compared to 3D LUT approaches, while maintaining real-time performance at 16 FPS for video stylization. Our code and benchmark are available at this https URL
• 摘要：光真逼真的样式转移（PST）可以通过在保留内容结构的同时调整参考图像颜色来实现现实世界的颜色分级。现有方法主要遵循哪种方法：基于生成的方法，以内容完整性和效率成本优先考虑风格忠诚度，或者是LUT等全球色彩转换方法，这些方法可以保留结构但缺乏局部适应性。为了弥合这一差距，我们提出了空间自适应4D查找表（SA-LUT），将LUT效率与神经网络的适应性相结合。 SA-LUT功能：（1）一种样式引导的4D LUT发电机，从样式图像中提取多尺度功能以预测4D LUT，以及（2）使用内容式跨注意的上下文生成器来生成上下文图。该上下文图可实现空​​间自适应的调整，使我们的4D LUT可以在保持结构完整性的同时进行精确的色彩转换。为了建立一个严格的评估框架，我们介绍了PST50，这是专门为PST评估设计的第一个基准。实验表明，与3D LUT方法相比，SA-LUT基本上优于最先进的方法，LPIPS得分降低了66.7％，同时以16 fps的速度保持实时性能以进行视频风格化。我们的代码和基准可在此HTTPS URL上找到

Title: ESRPCB: an Edge guided Super-Resolution model and Ensemble learning for tiny Printed Circuit Board Defect detection

• Authors: Xiem HoangVan, Dang Bui Dinh, Thanh Nguyen Canh, Van-Truong Nguyen
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.13476
• Pdf URL: https://arxiv.org/pdf/2506.13476
• Copy Paste: [[2506.13476]] ESRPCB: an Edge guided Super-Resolution model and Ensemble learning for tiny Printed Circuit Board Defect detection(https://arxiv.org/abs/2506.13476)
• Keywords: super-resolution
• Abstract: Printed Circuit Boards (PCBs) are critical components in modern electronics, which require stringent quality control to ensure proper functionality. However, the detection of defects in small-scale PCBs images poses significant challenges as a result of the low resolution of the captured images, leading to potential confusion between defects and noise. To overcome these challenges, this paper proposes a novel framework, named ESRPCB (edgeguided super-resolution for PCBs defect detection), which combines edgeguided super-resolution with ensemble learning to enhance PCBs defect detection. The framework leverages the edge information to guide the EDSR (Enhanced Deep Super-Resolution) model with a novel ResCat (Residual Concatenation) structure, enabling it to reconstruct high-resolution images from small PCBs inputs. By incorporating edge features, the super-resolution process preserves critical structural details, ensuring that tiny defects remain distinguishable in the enhanced image. Following this, a multi-modal defect detection model employs ensemble learning to analyze the super-resolved
• 摘要：印刷电路板（PCB）是现代电子中的关键组件，需要严格的质量控制才能确保正确的功能。但是，由于捕获的图像的低分辨率，小规模PCBS图像中缺陷的检测构成了重大挑战，从而导致缺陷和噪声之间的潜在混淆。为了克服这些挑战，本文提出了一个新的框架，名为ESRPCB（用于PCBS缺陷检测的边缘超级分辨率），该框架结合了边缘指导的超级分辨率与集合学习以增强PCBS缺陷检测。该框架利用边缘信息来指导EDSR（增强的深层超分辨率）模型，具有新型的救援（残留串联）结构，使其能够从小型PCBS输入中重建高分辨率的图像。通过合并边缘特征，超分辨率过程保留了关键的结构细节，以确保在增强图像中保持微小的缺陷仍然可以区分。之后，多模式缺陷检测模型采用集合学习来分析超级分解

Title: Deep Diffusion Models and Unsupervised Hyperspectral Unmixing for Realistic Abundance Map Synthesis

• Authors: Martina Pastorino, Michael Alibani, Nicola Acito, Gabriele Moser
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.13484
• Pdf URL: https://arxiv.org/pdf/2506.13484
• Copy Paste: [[2506.13484]] Deep Diffusion Models and Unsupervised Hyperspectral Unmixing for Realistic Abundance Map Synthesis(https://arxiv.org/abs/2506.13484)
• Keywords: generative
• Abstract: This paper presents a novel methodology for generating realistic abundance maps from hyperspectral imagery using an unsupervised, deep-learning-driven approach. Our framework integrates blind linear hyperspectral unmixing with state-of-the-art diffusion models to enhance the realism and diversity of synthetic abundance maps. First, we apply blind unmixing to extract endmembers and abundance maps directly from raw hyperspectral data. These abundance maps then serve as inputs to a diffusion model, which acts as a generative engine to synthesize highly realistic spatial distributions. Diffusion models have recently revolutionized image synthesis by offering superior performance, flexibility, and stability, making them well-suited for high-dimensional spectral data. By leveraging this combination of physically interpretable unmixing and deep generative modeling, our approach enables the simulation of hyperspectral sensor outputs under diverse imaging conditions--critical for data augmentation, algorithm benchmarking, and model evaluation in hyperspectral analysis. Notably, our method is entirely unsupervised, ensuring adaptability to different datasets without the need for labeled training data. We validate our approach using real hyperspectral imagery from the PRISMA space mission for Earth observation, demonstrating its effectiveness in producing realistic synthetic abundance maps that capture the spatial and spectral characteristics of natural scenes.
• 摘要：本文提出了一种新的方法，用于使用无监督的，深度学习驱动的方法从高光谱图像中产生逼真的丰度图。我们的框架将盲线性高光谱脉冲与最先进的扩散模型相结合，以增强合成丰度图的现实性和多样性。首先，我们将盲液构成直接从原始的高光谱数据中提取最终成员和丰度图。然后，这些丰度图是扩散模型的输入，该模型充当了合成高度逼真的空间分布的生成引擎。扩散模型最近通过提供出色的性能，柔韧性和稳定性来彻底改变图像合成，使其非常适合高维光谱数据。通过利用这种物理解释的不混合和深层生成建模的组合，我们的方法可以在多种成像条件下模拟高光谱传感器输出 - 对于数据增强，算法基准测试和高光谱分析中的模型评估。值得注意的是，我们的方法完全不受监督，可确保对不同数据集的适应性，而无需标记培训数据。我们使用Prisma空间任务中的真实高光谱图像来验证我们的方法，以进行地球观察，并证明了其在生成逼真的合成丰度图中的有效性，从而捕获了自然场景的空间和光谱特征。

Title: GeoSDF: Plane Geometry Diagram Synthesis via Signed Distance Field

• Authors: Chengrui Zhang, Maizhen Ning, Zihao Zhou, Jie Sun, Kaizhu Huang, Qiufeng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13492
• Pdf URL: https://arxiv.org/pdf/2506.13492
• Copy Paste: [[2506.13492]] GeoSDF: Plane Geometry Diagram Synthesis via Signed Distance Field(https://arxiv.org/abs/2506.13492)
• Keywords: generation
• Abstract: Plane Geometry Diagram Synthesis has been a crucial task in computer graphics, with applications ranging from educational tools to AI-driven mathematical reasoning. Traditionally, we rely on computer tools (e.g., Matplotlib and GeoGebra) to manually generate precise diagrams, but it usually requires huge, complicated calculations cost. Recently, researchers start to work on learning-based methods (e.g., Stable Diffusion and GPT4) to automatically generate diagrams, saving operational cost but usually suffering from limited realism and insufficient accuracy. In this paper, we propose a novel framework GeoSDF to automatically generate diagrams efficiently and accurately with Signed Distance Field (SDF). Specifically, we first represent geometric elements in the SDF, then construct a series of constraint functions to represent geometric relationships, next we optimize such constraint functions to get an optimized field of both elements and constraints, finally by rendering the optimized field, we can obtain the synthesized diagram. In our GeoSDF, we define a symbolic language to easily represent geometric elements and those constraints, and our synthesized geometry diagrams can be self-verified in the SDF, ensuring both mathematical accuracy and visual plausibility. In experiments, our GeoSDF synthesized both normal high-school level and IMO-level geometry diagrams. Through both qualitative and quantitative analysis, we can see that synthesized diagrams are realistic and accurate, and our synthesizing process is simple and efficient. Furthermore, we obtain a very high accuracy of solving geometry problems (over 95\% while the current SOTA accuracy is around 75%) by leveraging our self-verification property. All of these demonstrate the advantage of GeoSDF, paving the way for more sophisticated, accurate, and flexible generation of geometric diagrams for a wide array of applications.
• 摘要：平面几何图的合成一直是计算机图形学的至关重要的任务，从教育工具到AI驱动的数学推理，应用程序不等。传统上，我们依靠计算机工具（例如Matplotlib和Geogebra）手动生成精确的图表，但通常需要大量，复杂的计算成本。最近，研究人员开始研究基于学习的方法（例如稳定的扩散和GPT4），以自动生成图表，节省运营成本，但通常会遭受有限的现实主义和不足的准确性。在本文中，我们提出了一个新颖的框架GEOSDF，以自动使用签名距离字段（SDF）自动生成图表。具体而言，我们首先表示SDF中的几何元素，然后构建一系列约束函数来表示几何关系，接下来，我们优化了此类约束功能以获得元素和约束的优化字段，最后通过呈现优化的字段，我们可以获取合成图。在我们的geosdf中，我们定义了一种符号语言，可以轻松表示几何元素和这些约束，并且我们的合成几何图可以在SDF中进行自我验证，从而确保数学准确性和视觉通用性。在实验中，我们的GEOSDF合成了正常的高中水平和IMO级几何图。通过定性和定量分析，我们可以看到合成图是现实且准确的，并且我们的合成过程简单有效。此外，通过利用我们的自我验证属性，我们获得了解决几何问题（超过95 \％而当前的SOTA精度约为75％）的高精度。所有这些都证明了GEOSDF的优势，为更复杂，准确且灵活地生成几何图的几何图为广泛的应用铺平了道路。

Title: Seismic Acoustic Impedance Inversion Framework Based on Conditional Latent Generative Diffusion Model

• Authors: Jie Chen, Hongling Chen, Jinghuai Gao, Chuangji Meng, Tao Yang, XinXin Liang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13529
• Pdf URL: https://arxiv.org/pdf/2506.13529
• Copy Paste: [[2506.13529]] Seismic Acoustic Impedance Inversion Framework Based on Conditional Latent Generative Diffusion Model(https://arxiv.org/abs/2506.13529)
• Keywords: generative
• Abstract: Seismic acoustic impedance plays a crucial role in lithological identification and subsurface structure interpretation. However, due to the inherently ill-posed nature of the inversion problem, directly estimating impedance from post-stack seismic data remains highly challenging. Recently, diffusion models have shown great potential in addressing such inverse problems due to their strong prior learning and generative capabilities. Nevertheless, most existing methods operate in the pixel domain and require multiple iterations, limiting their applicability to field data. To alleviate these limitations, we propose a novel seismic acoustic impedance inversion framework based on a conditional latent generative diffusion model, where the inversion process is made in latent space. To avoid introducing additional training overhead when embedding conditional inputs, we design a lightweight wavelet-based module into the framework to project seismic data and reuse an encoder trained on impedance to embed low-frequency impedance into the latent space. Furthermore, we propose a model-driven sampling strategy during the inversion process of this framework to enhance accuracy and reduce the number of required diffusion steps. Numerical experiments on a synthetic model demonstrate that the proposed method achieves high inversion accuracy and strong generalization capability within only a few diffusion steps. Moreover, application to field data reveals enhanced geological detail and higher consistency with well-log measurements, validating the effectiveness and practicality of the proposed approach.
• 摘要：地震声阻抗在岩性鉴定和地下结构解释中起着至关重要的作用。但是，由于反转问题的本质性质固有的性质不足，直接估算堆栈后地震数据的阻抗仍然是高度挑战性的。最近，由于其强大的先前学习和生成能力，扩散模型在解决此类反问题方面表现出了巨大的潜力。但是，大多数现有方法在像素域中运行，需要多次迭代，从而将其适用性限制为现场数据。为了减轻这些局限性，我们提出了一个基于条件潜在生成扩散模型的新型地震声阻抗反转框架，其中反转过程是在潜在空间中进行的。为了避免在嵌入条件输入时引入其他训练开销，我们将基于轻量的小波的模块设计到框架中，以投影地震数据，并重用对阻抗进行训练的编码器，以将低频阻抗嵌入潜在空间。此外，我们在此框架的反转过程中提出了一种模型驱动的采样策略，以提高准确性并减少所需的扩散步骤的数量。合成模型上的数值实验表明，所提出的方法仅在几个扩散步骤内实现了高反转精度和强大的概括能力。此外，对现场数据的应用揭示了增强的地质细节和更高的一致性，并通过良好的测量结果验证了拟议方法的有效性和实用性。

Title: X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability

• Authors: Yu Yang, Alan Liang, Jianbiao Mei, Yukai Ma, Yong Liu, Gim Hee Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13558
• Pdf URL: https://arxiv.org/pdf/2506.13558
• Copy Paste: [[2506.13558]] X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability(https://arxiv.org/abs/2506.13558)
• Keywords: generation
• Abstract: Diffusion models are advancing autonomous driving by enabling realistic data synthesis, predictive end-to-end planning, and closed-loop simulation, with a primary focus on temporally consistent generation. However, the generation of large-scale 3D scenes that require spatial coherence remains underexplored. In this paper, we propose X-Scene, a novel framework for large-scale driving scene generation that achieves both geometric intricacy and appearance fidelity, while offering flexible controllability. Specifically, X-Scene supports multi-granular control, including low-level conditions such as user-provided or text-driven layout for detailed scene composition and high-level semantic guidance such as user-intent and LLM-enriched text prompts for efficient customization. To enhance geometrical and visual fidelity, we introduce a unified pipeline that sequentially generates 3D semantic occupancy and the corresponding multiview images, while ensuring alignment between modalities. Additionally, we extend the generated local region into a large-scale scene through consistency-aware scene outpainting, which extrapolates new occupancy and images conditioned on the previously generated area, enhancing spatial continuity and preserving visual coherence. The resulting scenes are lifted into high-quality 3DGS representations, supporting diverse applications such as scene exploration. Comprehensive experiments demonstrate that X-Scene significantly advances controllability and fidelity for large-scale driving scene generation, empowering data generation and simulation for autonomous driving.
• 摘要：扩散模型通过启用现实的数据综合，预测性端到端计划和闭环模拟来提高自动驾驶，并主要关注时间一致的生成。但是，需要空间相干性的大规模3D场景的产生仍未被逐渐倍增。在本文中，我们提出了X-Scene，这是一个大规模驾驶场景的新型框架，既可以实现几何复杂性和外观保真度，同时提供灵活的可控性。具体而言，X-Scene支持多粒子控制，包括用于详细场景组成的用户提供或文本驱动的布局等低级条件，以及高级语义指导，例如用户大量和LLM增强的文本提示，以有效自定义。为了增强几何和视觉保真度，我们引入了一条统一的管道，该管道顺序生成3D语义占用和相应的多视图像，同时确保模态之间的对齐。此外，我们通过一致性感知的场景将生成的局部区域扩展到大型场景，从而推断了新的占用率和在先前生成的区域进行的图像，从而增强了空间连续性并保留了视觉相干性。由此产生的场景被提升为高质量的3DGS表示，支持各种应用程序，例如场景探索。全面的实验表明，X景烯可显着提高大规模驾驶场景的可控性和忠诚度，赋予自动驾驶的数据生成和仿真。

Title: Flexible-length Text Infilling for Discrete Diffusion Models

• Authors: Andrew Zhang, Anushka Sivakumar, Chiawei Tang, Chris Thomas
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13579
• Pdf URL: https://arxiv.org/pdf/2506.13579
• Copy Paste: [[2506.13579]] Flexible-length Text Infilling for Discrete Diffusion Models(https://arxiv.org/abs/2506.13579)
• Keywords: generation
• Abstract: Discrete diffusion models are a new class of text generators that offer advantages such as bidirectional context use, parallelizable generation, and flexible prompting compared to autoregressive models. However, a critical limitation of discrete diffusion models is their inability to perform flexible-length or flexible-position text infilling without access to ground-truth positional data. We introduce \textbf{DDOT} (\textbf{D}iscrete \textbf{D}iffusion with \textbf{O}ptimal \textbf{T}ransport Position Coupling), the first discrete diffusion model to overcome this challenge. DDOT jointly denoises token values and token positions, employing a novel sample-level Optimal Transport (OT) coupling. This coupling preserves relative token ordering while dynamically adjusting the positions and length of infilled segments, a capability previously missing in text diffusion. Our method is orthogonal to existing discrete text diffusion methods and is compatible with various pretrained text denoisers. Extensive experiments on text infilling benchmarks such as One-Billion-Word and Yelp demonstrate that DDOT outperforms naive diffusion baselines. Furthermore, DDOT achieves performance on par with state-of-the-art non-autoregressive models and enables significant improvements in training efficiency and flexibility.
• 摘要：离散扩散模型是一种新的文本生成器，具有与自回归模型相比，具有双向上下文使用，可行的生成和灵活提示等优点。但是，离散扩散模型的关键局限性在于它们无法执行柔性长度或灵活位置文本填充，而无需访问地面位置数据。我们介绍\ textbf {ddot}（\ textbf {d} Iscrete \ textbf {d} iffusion fr \ textbf {o} ptimal \ textbf {t} ransport位置耦合），是克服这一挑战的第一个区分扩散模型。 DDOT采用新型样品级最佳运输（OT）耦合，共同降低令牌值和令牌位置。这种耦合可以保留相对的令牌排序，同时动态调整填充段的位置和长度，这是文本扩散中先前缺少的功能。我们的方法与现有的离散文本扩散方法正交，并且与各种预验证的文本Denoiser兼容。对文本填充基准（例如十亿字和Yelp）的广泛实验表明，DDOT的表现优于天真的扩散基线。此外，DDOT可以与最先进的非自动回归模型相同，并在训练效率和灵活性方面取得了重大提高。

Title: Omni-AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented for Efficient Long Video Understanding

• Authors: Zhucun Xue, Jiangning Zhang, Xurong Xie, Yuxuan Cai, Yong Liu, Xiangtai Li, Dacheng Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13589
• Pdf URL: https://arxiv.org/pdf/2506.13589
• Copy Paste: [[2506.13589]] Omni-AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented for Efficient Long Video Understanding(https://arxiv.org/abs/2506.13589)
• Keywords: generation
• Abstract: Multimodal Large Language Models (MLLMs) struggle with long videos due to fixed context windows and weak long-term dependency modeling. Existing Retrieval-Augmented Generation (RAG) methods for videos use static retrieval strategies, leading to inefficiencies for simple queries and information loss for complex tasks. To address this, we propose AdaVideoRAG, a novel framework that dynamically adapts retrieval granularity based on query complexity using a lightweight intent classifier. Our framework employs an Omni-Knowledge Indexing module to build hierarchical databases from text (captions, ASR, OCR), visual features, and semantic graphs, enabling optimal resource allocation across tasks. We also introduce the HiVU benchmark for comprehensive evaluation. Experiments demonstrate improved efficiency and accuracy for long-video understanding, with seamless integration into existing MLLMs. AdaVideoRAG establishes a new paradigm for adaptive retrieval in video analysis. Codes will be open-sourced at this https URL.
• 摘要：由于固定的上下文窗口和弱长期依赖建模，多模式大型语言模型（MLLM）与长视频斗争。视频的现有检索效果生成（RAG）方法使用静态检索策略，从而导致效率低下，以实现简单的查询和复杂任务的信息丢失。为了解决这个问题，我们提出了Adavideorag，这是一个新颖的框架，该框架使用轻量级意图分类器基于查询复杂性动态调整检索粒度。我们的框架采用Omni-知识索引模块来从文本（字幕，ASR，OCR），视觉特征和语义图构建层次数据库，从而在任务中启用最佳资源分配。我们还介绍了HIVU基准进行全面评估。实验表明，通过无缝整合到现有的MLLM中，可以提高效率和准确性。 Adavideorag在视频分析中建立了一种新的自适应检索范式。代码将在此HTTPS URL上开源。

Title: Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMs

• Authors: Hen Davidov, Gilad Freidkin, Shai Feldman, Yaniv Romano
• Subjects: cs.LG, stat.AP, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.13593
• Pdf URL: https://arxiv.org/pdf/2506.13593
• Copy Paste: [[2506.13593]] Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMs(https://arxiv.org/abs/2506.13593)
• Keywords: generation, generative
• Abstract: We develop a framework to quantify the time-to-unsafe-sampling - the number of large language model (LLM) generations required to trigger an unsafe (e.g., toxic) response. Estimating this quantity is challenging, since unsafe responses are exceedingly rare in well-aligned LLMs, potentially occurring only once in thousands of generations. As a result, directly estimating time-to-unsafe-sampling would require collecting training data with a prohibitively large number of generations per prompt. However, with realistic sampling budgets, we often cannot generate enough responses to observe an unsafe outcome for every prompt, leaving the time-to-unsafe-sampling unobserved in many cases, making the estimation and evaluation tasks particularly challenging. To address this, we frame this estimation problem as one of survival analysis and develop a provably calibrated lower predictive bound (LPB) on the time-to-unsafe-sampling of a given prompt, leveraging recent advances in conformal prediction. Our key innovation is designing an adaptive, per-prompt sampling strategy, formulated as a convex optimization problem. The objective function guiding this optimized sampling allocation is designed to reduce the variance of the estimators used to construct the LPB, leading to improved statistical efficiency over naive methods that use a fixed sampling budget per prompt. Experiments on both synthetic and real data support our theoretical results and demonstrate the practical utility of our method for safety risk assessment in generative AI models.
• 摘要：我们开发了一个框架来量化不安全的时间采样 - 触发不安全（例如有毒）响应所需的大语言模型（LLM）代的数量。估计该数量是具有挑战性的，因为在良好的LLM中，不安全的响应极为罕见，可能仅发生一次数千代。结果，直接估计无安全的时间采样的时间将需要每个提示的培训数据，每个提示都大量世代。但是，借助现实的抽样预算，我们通常无法产生足够的响应来观察每个提示的不安全结果，在许多情况下，在许多情况下都没有观察到的时间，这使得估计和评估任务尤其具有挑战性。为了解决这个问题，我们将此估计问题构成了生存分析之一，并在给定提示的时间上开发了可证明的较低的预测性约束（LPB），并利用了保形预测的最新进展。我们的关键创新是设计一种自适应，每次提取的采样策略，该策略被制定为凸优化问题。指导此优化抽样分配的目标函数旨在减少用于构建LPB的估计值的方差，从而提高了统计效率，而不是使用固定采样预算的幼稚方法。合成和实际数据的实验都支持我们的理论结果，并证明了我们在生成AI模型中进行安全风险评估方法的实际实用性。

Title: Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching

• Authors: Weimin Bai, Yubo Li, Wenzheng Chen, Weijian Luo, He Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13594
• Pdf URL: https://arxiv.org/pdf/2506.13594
• Copy Paste: [[2506.13594]] Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching(https://arxiv.org/abs/2506.13594)
• Keywords: generation
• Abstract: Distilling pre-trained 2D diffusion models into 3D assets has driven remarkable advances in text-to-3D synthesis. However, existing methods typically rely on Score Distillation Sampling (SDS) loss, which involves asymmetric KL divergence--a formulation that inherently favors mode-seeking behavior and limits generation diversity. In this paper, we introduce Dive3D, a novel text-to-3D generation framework that replaces KL-based objectives with Score Implicit Matching (SIM) loss, a score-based objective that effectively mitigates mode collapse. Furthermore, Dive3D integrates both diffusion distillation and reward-guided optimization under a unified divergence perspective. Such reformulation, together with SIM loss, yields significantly more diverse 3D outputs while improving text alignment, human preference, and overall visual fidelity. We validate Dive3D across various 2D-to-3D prompts and find that it consistently outperforms prior methods in qualitative assessments, including diversity, photorealism, and aesthetic appeal. We further evaluate its performance on the GPTEval3D benchmark, comparing against nine state-of-the-art baselines. Dive3D also achieves strong results on quantitative metrics, including text-asset alignment, 3D plausibility, text-geometry consistency, texture quality, and geometric detail.
• 摘要：将预训练的2D扩散模型蒸馏到3D资产中已促进了文本到3D综合方面的显着进步。但是，现有方法通常依赖于评分蒸馏采样（SDS）损失，涉及不对称的KL差异 - 一种固有地利用模式的行为并限制了产生多样性的表述。在本文中，我们介绍了Dive3D，这是一种新颖的文本到3D生成框架，用分数隐式匹配（SIM）损失取代基于KL的目标，这是一个基于分数的目标，可有效地减轻模式崩溃。此外，Dive3D在统一的分歧透视图下既整合了扩散蒸馏和奖励引导的优化。这种重新制定以及SIM丢失，在改善文本一致性，人类偏好和整体视觉保真度的同时，产生了更多的3D输出。我们验证了各种2到3D提示中的潜水3D，发现它在定性评估中始终超过先前的方法，包括多样性，光逼现实主义和审美吸引力。我们进一步评估了其在GPTEVAL3D基准测试中的性能，并与9个最先进的基准相比。 Dive3D还可以在定量指标上取得良好的结果，包括文本资产对齐，3D合理性，文本几何形式一致性，纹理质量和几何细节。

Title: Graph-Convolution-Beta-VAE for Synthetic Abdominal Aorta Aneurysm Generation

• Authors: Francesco Fabbri, Martino Andrea Scarpolini, Angelo Iollo, Francesco Viola, Francesco Tudisco
• Subjects: cs.LG, cs.AI, q-bio.TO
• Abstract URL: https://arxiv.org/abs/2506.13628
• Pdf URL: https://arxiv.org/pdf/2506.13628
• Copy Paste: [[2506.13628]] Graph-Convolution-Beta-VAE for Synthetic Abdominal Aorta Aneurysm Generation(https://arxiv.org/abs/2506.13628)
• Keywords: generation
• Abstract: Synthetic data generation plays a crucial role in medical research by mitigating privacy concerns and enabling large-scale patient data analysis. This study presents a beta-Variational Autoencoder Graph Convolutional Neural Network framework for generating synthetic Abdominal Aorta Aneurysms (AAA). Using a small real-world dataset, our approach extracts key anatomical features and captures complex statistical relationships within a compact disentangled latent space. To address data limitations, low-impact data augmentation based on Procrustes analysis was employed, preserving anatomical integrity. The generation strategies, both deterministic and stochastic, manage to enhance data diversity while ensuring realism. Compared to PCA-based approaches, our model performs more robustly on unseen data by capturing complex, nonlinear anatomical variations. This enables more comprehensive clinical and statistical analyses than the original dataset alone. The resulting synthetic AAA dataset preserves patient privacy while providing a scalable foundation for medical research, device testing, and computational modeling.
• 摘要：通过减轻隐私问题并实现大规模的患者数据分析，合成数据生成在医学研究中起着至关重要的作用。这项研究提出了一个β变量自动编码器图卷积神经网络框架，用于产生合成腹主动脉瘤（AAA）。使用一个小的现实世界数据集，我们的方法提取了关键的解剖特征，并捕获紧凑的分离潜在空间内的复杂统计关系。为了解决数据限制，采用了基于crodrustes分析的低影响数据增强，从而保留了解剖完整性。确定性和随机性的一代策略都设法在确保现实主义的同时增强了数据多样性。与基于PCA的方法相比，我们的模型通过捕获复杂的非线性解剖变异来对看不见的数据更加强大。与单独的数据集相比，这可以实现更全面的临床和统计分析。由此产生的合成AAA数据集可保留患者的隐私，同时为医学研究，设备测试和计算建模提供了可扩展的基础。

Title: FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understanding

• Authors: Chenlu Zhan, Gaoang Wang, Hongwei Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13629
• Pdf URL: https://arxiv.org/pdf/2506.13629
• Copy Paste: [[2506.13629]] FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understanding(https://arxiv.org/abs/2506.13629)
• Keywords: generation
• Abstract: Semantic querying in complex 3D scenes through free-form language presents a significant challenge. Existing 3D scene understanding methods use large-scale training data and CLIP to align text queries with 3D semantic features. However, their reliance on predefined vocabulary priors from training data hinders free-form semantic querying. Besides, recent advanced methods rely on LLMs for scene understanding but lack comprehensive 3D scene-level information and often overlook the potential inconsistencies in LLM-generated outputs. In our paper, we propose FreeQ-Graph, which enables Free-form Querying with a semantic consistent scene Graph for 3D scene understanding. The core idea is to encode free-form queries from a complete and accurate 3D scene graph without predefined vocabularies, and to align them with 3D consistent semantic labels, which accomplished through three key steps. We initiate by constructing a complete and accurate 3D scene graph that maps free-form objects and their relations through LLM and LVLM guidance, entirely free from training data or predefined priors. Most importantly, we align graph nodes with accurate semantic labels by leveraging 3D semantic aligned features from merged superpoints, enhancing 3D semantic consistency. To enable free-form semantic querying, we then design an LLM-based reasoning algorithm that combines scene-level and object-level information to intricate reasoning. We conducted extensive experiments on 3D semantic grounding, segmentation, and complex querying tasks, while also validating the accuracy of graph generation. Experiments on 6 datasets show that our model excels in both complex free-form semantic queries and intricate relational reasoning.
• 摘要：通过自由形式语言在复杂的3D场景中进行语义查询带来了重大挑战。现有的3D场景理解方法使用大规模培训数据和剪辑与3D语义特征对齐文本查询。但是，他们依赖于培训数据阻碍自由形式语义查询的预定词汇先验的依赖。此外，最近的高级方法依靠LLM来理解场景，但缺乏全面的3D场景级信息，并且经常忽略LLM生成的输出中潜在的不一致之处。在我们的论文中，我们提出了FreeQ-Graph，该图纸可以通过语义一致的场景图进行自由格式查询，以了解3D场景的理解。核心想法是从没有预定义词汇的完整，准确的3D场景图中编码自由形式的查询，并将它们与3D一致的语义标签对齐，这些语义标签通过三个关键步骤完成。我们通过构建完整而准确的3D场景图来启动，该图形图及其通过LLM和LVLM指南绘制自由形式的对象及其关系，完全没有训练数据或预定义的先验。最重要的是，我们通过利用合并超级点的3D语义对齐特征来使图形节点与准确的语义标签对齐，从而增强了3D语义一致性。为了启用自由形式的语义查询，我们然后设计了一种基于LLM的推理算法，将场景级别和对象级信息结合到复杂的推理。我们对3D语义接地，分割和复杂的查询任务进行了广泛的实验，同时还验证了图生成的准确性。 6个数据集的实验表明，我们的模型在复杂的自由形式语义查询和复杂的关系推理中都表现出色。

Title: UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions

• Authors: Zhucun Xue, Jiangning Zhang, Teng Hu, Haoyang He, Yinan Chen, Yuxuan Cai, Yabiao Wang, Chengjie Wang, Yong Liu, Xiangtai Li, Dacheng Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13691
• Pdf URL: https://arxiv.org/pdf/2506.13691
• Copy Paste: [[2506.13691]] UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions(https://arxiv.org/abs/2506.13691)
• Keywords: generation
• Abstract: The quality of the video dataset (image quality, resolution, and fine-grained caption) greatly influences the performance of the video generation model. The growing demand for video applications sets higher requirements for high-quality video generation models. For example, the generation of movie-level Ultra-High Definition (UHD) videos and the creation of 4K short video content. However, the existing public datasets cannot support related research and applications. In this paper, we first propose a high-quality open-sourced UHD-4K (22.4\% of which are 8K) text-to-video dataset named UltraVideo, which contains a wide range of topics (more than 100 kinds), and each video has 9 structured captions with one summarized caption (average of 824 words). Specifically, we carefully design a highly automated curation process with four stages to obtain the final high-quality dataset: \textit{i)} collection of diverse and high-quality video clips. \textit{ii)} statistical data filtering. \textit{iii)} model-based data purification. \textit{iv)} generation of comprehensive, structured captions. In addition, we expand Wan to UltraWan-1K/-4K, which can natively generate high-quality 1K/4K videos with more consistent text controllability, demonstrating the effectiveness of our data this http URL believe that this work can make a significant contribution to future research on UHD video generation. UltraVideo dataset and UltraWan models are available at this https URL.
• 摘要：视频数据集（图像质量，分辨率和细粒字幕）的质量极大地影响了视频生成模型的性能。对视频应用程序的需求不断增长，为高质量视频生成模型设定了更高的要求。例如，电影级超高定义（UHD）视频的生成和4K简短视频内容的创建。但是，现有的公共数据集不能支持相关的研究和应用程序。在本文中，我们首先提出了一个高质量的开源UHD-4K（22.4 \％\％，其中8K）的文本到视频数据集名为Ultravideo，其中包含广泛的主题（超过100种），并且每个视频具有9个结构性字幕，带有一个带有一个简单的字幕（平均824个字符）。具体来说，我们仔细设计了一个高度自动化的策展过程，具有四个阶段，以获取最终的高质量数据集：\ textit {i）}集合的多样和高质量的视频剪辑。 \ textit {ii）}统计数据过滤。 \ textit {iii）}基于模型的数据净化。 \ textit {iv）}生成全面的结构化字幕。此外，我们将WAN扩展到Ultrawan-1K/-4K，它可以在本质上产生具有更一致的文本可控性的高质量的1K/4K视频，这证明了我们数据的有效性这一HTTP URL相信这项工作可以为未来对UHD视频的研究做出重大贡献。在此HTTPS URL上可以使用Ultravideo数据集和Ultrawan型号。

Title: Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry

• Authors: Junyoung Seo, Jisang Han, Jaewoo Jung, Siyoon Jin, Joungbin Lee, Takuya Narihira, Kazumi Fukuda, Takashi Shibuya, Donghoon Ahn, Shoukang Hu, Seungryong Kim, Yuki Mitsufuji
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13697
• Pdf URL: https://arxiv.org/pdf/2506.13697
• Copy Paste: [[2506.13697]] Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry(https://arxiv.org/abs/2506.13697)
• Keywords: generative
• Abstract: We introduce Vid-CamEdit, a novel framework for video camera trajectory editing, enabling the re-synthesis of monocular videos along user-defined camera paths. This task is challenging due to its ill-posed nature and the limited multi-view video data for training. Traditional reconstruction methods struggle with extreme trajectory changes, and existing generative models for dynamic novel view synthesis cannot handle in-the-wild videos. Our approach consists of two steps: estimating temporally consistent geometry, and generative rendering guided by this geometry. By integrating geometric priors, the generative model focuses on synthesizing realistic details where the estimated geometry is uncertain. We eliminate the need for extensive 4D training data through a factorized fine-tuning framework that separately trains spatial and temporal components using multi-view image and video data. Our method outperforms baselines in producing plausible videos from novel camera trajectories, especially in extreme extrapolation scenarios on real-world footage.
• 摘要：我们介绍了VID-CAMEDIT，这是一种用于摄像机轨迹编辑的新型框架，使能够沿用户定义的摄像机路径重新合成单眼视频。由于其范围不足的性质和用于培训的多视频视频数据，此任务是具有挑战性的。传统的重建方法在极端的轨迹变化中挣扎，现有的动态新型视图合成模型无法处理野外视频。我们的方法由两个步骤组成：估计时间一致的几何形状，并在此几何形状引导下呈现生成呈现。通过整合几何先验，生成模型的重点是综合估计几何形状不确定的现实细节。我们通过一个分解的微调框架消除了对大量4D训练数据的需求，该框架分别使用多视图图像和视频数据分别训练空间和时间组件。我们的方法在从新颖的相机轨迹中制作出合理的视频方面优于基准，尤其是在现实世界中的极端推断场景中。

Title: VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Models

• Authors: Edward Li, Zichen Wang, Jiahe Huang, Jeong Joon Park
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13754
• Pdf URL: https://arxiv.org/pdf/2506.13754
• Copy Paste: [[2506.13754]] VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Models(https://arxiv.org/abs/2506.13754)
• Keywords: generative
• Abstract: We present a unified framework for solving partial differential equations (PDEs) using video-inpainting diffusion transformer models. Unlike existing methods that devise specialized strategies for either forward or inverse problems under full or partial observation, our approach unifies these tasks under a single, flexible generative framework. Specifically, we recast PDE-solving as a generalized inpainting problem, e.g., treating forward prediction as inferring missing spatiotemporal information of future states from initial conditions. To this end, we design a transformer-based architecture that conditions on arbitrary patterns of known data to infer missing values across time and space. Our method proposes pixel-space video diffusion models for fine-grained, high-fidelity inpainting and conditioning, while enhancing computational efficiency through hierarchical modeling. Extensive experiments show that our video inpainting-based diffusion model offers an accurate and versatile solution across a wide range of PDEs and problem setups, outperforming state-of-the-art baselines.
• 摘要：我们提出了一个统一的框架，用于使用视频扩散变压器模型来求解部分微分方程（PDE）。与现有的方法在完整或部分观察下为前进或反问题设计专业策略的方法不同，我们的方法将这些任务统一了一个灵活的生成框架。具体而言，我们将解决PDE的解决问题重新销售为普遍的授予问题，例如将前瞻性预测视为从初始条件中推断未来状态的缺失时空信息。为此，我们设计了一个基于变压器的体系结构，该体系结构对已知数据的任意模式进行条件，以推断在时间和空间之间的缺失值。我们的方法提出了像素空间视频扩散模型，用于细粒度，高保真介绍和调理，同时通过层次建模提高计算效率。广泛的实验表明，我们基于视频的基于视频的扩散模型为广泛的PDE和问题设置提供了精确且通用的解决方案，表现优于最先进的基线。

Title: AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

• Authors: Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, Jiaqi Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.13757
• Pdf URL: https://arxiv.org/pdf/2506.13757
• Copy Paste: [[2506.13757]] AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning(https://arxiv.org/abs/2506.13757)
• Keywords: generation
• Abstract: Recent advancements in Vision-Language-Action (VLA) models have shown promise for end-to-end autonomous driving by leveraging world knowledge and reasoning capabilities. However, current VLA models often struggle with physically infeasible action outputs, complex model structures, or unnecessarily long reasoning. In this paper, we propose AutoVLA, a novel VLA model that unifies reasoning and action generation within a single autoregressive generation model for end-to-end autonomous driving. AutoVLA performs semantic reasoning and trajectory planning directly from raw visual inputs and language instructions. We tokenize continuous trajectories into discrete, feasible actions, enabling direct integration into the language model. For training, we employ supervised fine-tuning to equip the model with dual thinking modes: fast thinking (trajectory-only) and slow thinking (enhanced with chain-of-thought reasoning). To further enhance planning performance and efficiency, we introduce a reinforcement fine-tuning method based on Group Relative Policy Optimization (GRPO), reducing unnecessary reasoning in straightforward scenarios. Extensive experiments across real-world and simulated datasets and benchmarks, including nuPlan, nuScenes, Waymo, and CARLA, demonstrate the competitive performance of AutoVLA in both open-loop and closed-loop settings. Qualitative results showcase the adaptive reasoning and accurate planning capabilities of AutoVLA in diverse scenarios.
• 摘要：视力语言行动（VLA）模型的最新进展已通过利用世界知识和推理能力来端对端自主驾驶的希望。但是，当前的VLA模型通常会在物理上不可行的动作输出，复杂的模型结构或不必要的长期推理中挣扎。在本文中，我们提出了Autovla，这是一种新型的VLA模型，该模型在端到端自动驾驶的单个自回旋生成模型中统一了推理和动作生成。 Autovla直接通过原始视觉输入和语言说明直接执行语义推理和轨迹计划。我们将连续的轨迹归为离散的，可行的动作，使直接集成到语言模型中。对于培训，我们采用监督的微调来为模型配备双重思考模式：快速思考（仅轨迹）和缓慢的思维（通过思想链的推理增强）。为了进一步提高计划绩效和效率，我们基于小组相对策略优化（GRPO）引入了加强微调方法，从而减少了直接场景中的不必要的推理。在现实世界和模拟数据集和基准测试中进行了广泛的实验，包括NUPLAN，NUSCENES，WAYMO和CARLA，展示了Autovla在开环和闭环设置中的竞争性能。定性结果展示了在各种情况下，Autovla的自适应推理和准确的计划功能。

Title: Discrete Diffusion in Large Language and Multimodal Models: A Survey

• Authors: Runpeng Yu, Qi Li, Xinchao Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.13759
• Pdf URL: https://arxiv.org/pdf/2506.13759
• Copy Paste: [[2506.13759]] Discrete Diffusion in Large Language and Multimodal Models: A Survey(https://arxiv.org/abs/2506.13759)
• Keywords: generation
• Abstract: In this work, we provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs). Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token, parallel decoding paradigm using full attention and a denoising-based generation strategy. This paradigm naturally enables parallel generation, fine-grained output controllability, and dynamic, response-aware perception. These capabilities are previously difficult to achieve with AR models. Recently, a growing number of industrial-scale proprietary d(M)LLMs, as well as a large number of open-source academic d(M)LLMs, have demonstrated performance comparable to their autoregressive counterparts, while achieving up to 10x acceleration in inference speed. The advancement of discrete diffusion LLMs and MLLMs has been largely driven by progress in two domains. The first is the development of autoregressive LLMs and MLLMs, which has accumulated vast amounts of data, benchmarks, and foundational infrastructure for training and inference. The second contributing domain is the evolution of the mathematical models underlying discrete diffusion. Together, these advancements have catalyzed a surge in dLLMs and dMLLMs research in early 2025. In this work, we present a comprehensive overview of the research in the dLLM and dMLLM domains. We trace the historical development of dLLMs and dMLLMs, formalize the underlying mathematical frameworks, and categorize representative models. We further analyze key techniques for training and inference, and summarize emerging applications across language, vision-language, and biological domains. We conclude by discussing future directions for research and deployment. Paper collection: this https URL
• 摘要：在这项工作中，我们提供了对离散扩散语言模型（DLLM）和离散扩散多模式模型（DMLLMS）的系统调查。与自动回归（AR）模型不同，DLLM和DMLLMS使用全面关注和基于Denoising的发电策略采用了多键，平行的解码范式。该范式自然可以实现并行生成，细粒的输出可控性和动态响应感知感知。这些功能以前很难通过AR模型实现。最近，越来越多的工业规模的专有D（M）LLM以及大量开源学术D（M）LLMS表现出与他们自动回归的同行相当的性能，而推理速度则达到了10倍加速。离散扩散LLM和MLLM的进步主要由两个领域的进步驱动。首先是开发自回旋的LLM和MLLM，它们积累了大量数据，基准和基础基础设施，用于培训和推理。第二个贡献域是离散扩散基础数学模型的演变。这些进步共同促进了2025年初的DLLM和DMLLMS研究的激增。在这项工作中，我们介绍了DLLM和DMLLM域研究的全面概述。我们追踪DLLM和DMLLM的历史发展，正式化基础数学框架，并对代表性模型进行分类。我们进一步分析了培训和推理的关键技术，并跨语言，视觉语言和生物领域总结了新兴应用。我们通过讨论未来的研究和部署方向来结束。纸收藏：此HTTPS URL

Title: Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value

• Authors: Yixian Xu, Shengjie Luo, Liwei Wang, Di He, Chang Liu
• Subjects: cs.LG, cs.AI, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.13763
• Pdf URL: https://arxiv.org/pdf/2506.13763
• Copy Paste: [[2506.13763]] Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value(https://arxiv.org/abs/2506.13763)
• Keywords: generative
• Abstract: Diffusion models have achieved remarkable success in generative modeling. Despite more stable training, the loss of diffusion models is not indicative of absolute data-fitting quality, since its optimal value is typically not zero but unknown, leading to confusion between large optimal loss and insufficient model capacity. In this work, we advocate the need to estimate the optimal loss value for diagnosing and improving diffusion models. We first derive the optimal loss in closed form under a unified formulation of diffusion models, and develop effective estimators for it, including a stochastic variant scalable to large datasets with proper control of variance and bias. With this tool, we unlock the inherent metric for diagnosing the training quality of mainstream diffusion model variants, and develop a more performant training schedule based on the optimal loss. Moreover, using models with 120M to 1.5B parameters, we find that the power law is better demonstrated after subtracting the optimal loss from the actual training loss, suggesting a more principled setting for investigating the scaling law for diffusion models.
• 摘要：扩散模型在生成建模方面取得了巨大的成功。尽管训练更稳定，但扩散模型的损失并不表示绝对数据拟合质量，因为其最佳值通常不是零，而是未知的，导致大型最佳损失和不足模型容量之间的混淆。在这项工作中，我们提倡需要估计诊断和改进扩散模型的最佳损失价值。我们首先在扩散模型的统一公式下以封闭形式以封闭形式得出最佳损失，并为其开发有效的估计器，包括可与大型数据集的随机变体可扩展，并适当控制方差和偏差。使用此工具，我们解锁了固有的指标，用于诊断主流扩散模型变体的训练质量，并根据最佳损失制定更具性能的培训时间表。此外，使用具有120m至1.5b参数的模型，我们发现在减去实际训练损失的最佳损失后，更好地证明了功率定律，这表明对扩散模型的扩展定律进行了更有原则的设置。