2025-03-05

Title: Larger or Smaller Reward Margins to Select Preferences for Alignment?

• Authors: Kexin Huang, Junkang Wu, Ziqian Chen, Xue Wang, Jinyang Gao, Bolin Ding, Jiancan Wu, Xiangnan He, Xiang Wang
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01864
• Pdf URL: https://arxiv.org/pdf/2503.01864
• Copy Paste: [[2503.01864]] Larger or Smaller Reward Margins to Select Preferences for Alignment?(https://arxiv.org/abs/2503.01864)
• Keywords: generation
• Abstract: Preference learning is critical for aligning large language models (LLMs) with human values, with the quality of preference datasets playing a crucial role in this process. While existing metrics primarily assess data quality based on either explicit or implicit reward margins, they often provide contradictory evaluations for the same data. To address this issue, we introduce the alignment potential metric, which quantifies the gap from the model's current implicit reward margin to the target explicit reward margin, thereby estimating the model's potential to align with the preference data. Empirical results demonstrate that training on data selected by this metric consistently enhances alignment performance, surpassing existing metrics across different base models and optimization objectives. Furthermore, our method extends to self-play data generation frameworks, where the metric is used to identify high-quality data within the self-generated content by LLMs. Under this data generation scenario, our method surpasses current state-of-the-art (SOTA) results across various training settings and demonstrates continuous improvements in alignment performance as dataset size and training iterations increase.
• 摘要：偏好学习对于使大语言模型（LLM）与人类价值观保持一致，而偏好数据集的质量在此过程中起着至关重要的作用。尽管现有指标主要基于明确或隐式奖励利润率评估数据质量，但它们通常会为相同数据提供矛盾的评估。为了解决这个问题，我们介绍了对齐电位指标，该指标将差距从模型当前的隐式奖励利润率到目标显式奖励余量，从而估计该模型与偏好数据保持一致的潜力。经验结果表明，该指标选择的数据培训一致地增强了对齐性能，超过了不同基本模型和优化目标的现有指标。此外，我们的方法扩展到自我播放的数据生成框架，该框架用于识别LLMS中自生成内容中的高质量数据。在此数据生成方案下，我们的方法超过了各种培训环境中当前的最新结果（SOTA）结果，并证明随着数据集大小和训练迭代的增加，对齐性能的持续改进。

Title: Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale

• Authors: Jerome Ku, Eric Nguyen, David W. Romero, Garyk Brixi, Brandon Yang, Anton Vorontsov, Ali Taghibakhshi, Amy X. Lu, Dave P. Burke, Greg Brockman, Stefano Massaroli, Christopher Ré, Patrick D. Hsu, Brian L. Hie, Stefano Ermon, Michael Poli
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01868
• Pdf URL: https://arxiv.org/pdf/2503.01868
• Copy Paste: [[2503.01868]] Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale(https://arxiv.org/abs/2503.01868)
• Keywords: generation
• Abstract: We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations. First, operators in hybrid models can be tailored to token manipulation tasks such as in-context recall, multi-token recall, and compression, with input-dependent convolutions and attention offering complementary performance. Second, co-designing convolution operators and hardware-aware algorithms enables efficiency gains in regimes where previous alternative architectures struggle to surpass Transformers. At the 40 billion parameter scale, we train end-to-end 1.2 to 2.9 times faster than optimized Transformers, and 1.1 to 1.4 times faster than previous generation hybrids. On H100 GPUs and model width 4096, individual operators in the proposed multi-hybrid StripedHyena 2 architecture achieve two-fold throughput improvement over linear attention and state-space models. Multi-hybrids excel at sequence modeling over byte-tokenized data, as demonstrated by the Evo 2 line of models. We discuss the foundations that enable these results, including architecture design, overlap-add blocked kernels for tensor cores, and dedicated all-to-all and point-to-point context parallelism strategies.
• 摘要：我们介绍了卷积多杂交结构，其设计基于两个简单的观测。首先，混合模型中的运营商可以根据象征性的操纵任务进行量身定制，例如内在召回，多言式召回和压缩，并具有输入依赖性的卷积和提供互补性能的注意力。其次，共同设计的卷积操作员和硬件感知算法可以在以前的替代体系结构难以超越变形金刚的政权中提高效率。在400亿个参数量表上，我们端到端1.2至2.9倍的速度比优化的变压器快1.2倍，比上一代混合动力车快1.1至1.4倍。在H100 GPU和模型宽度4096上，提议的多杂交条纹2架构中的单个操作员比线性注意和状态空间模型实现了两倍的吞吐量改进。如EVO 2线模型所证明的那样，多杂交在序列建模上表现出色。我们讨论了启用这些结果的基础，包括体系结构设计，重叠量阻止了张量核心的内核，以及专门的全部和所有点对点上下文的平行策略。

Title: FairGen: Controlling Sensitive Attributes for Fair Generations in Diffusion Models via Adaptive Latent Guidance

• Authors: Mintong Kang, Vinayshekhar Bannihatti Kumar, Shamik Roy, Abhishek Kumar, Sopan Khosla, Balakrishnan Murali Narayanaswamy, Rashmi Gangadharaiah
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01872
• Pdf URL: https://arxiv.org/pdf/2503.01872
• Copy Paste: [[2503.01872]] FairGen: Controlling Sensitive Attributes for Fair Generations in Diffusion Models via Adaptive Latent Guidance(https://arxiv.org/abs/2503.01872)
• Keywords: generation
• Abstract: Text-to-image diffusion models often exhibit biases toward specific demographic groups, such as generating more males than females when prompted to generate images of engineers, raising ethical concerns and limiting their adoption. In this paper, we tackle the challenge of mitigating generation bias towards any target attribute value (e.g., "male" for "gender") in diffusion models while preserving generation quality. We propose FairGen, an adaptive latent guidance mechanism which controls the generation distribution during inference. In FairGen, a latent guidance module dynamically adjusts the diffusion process to enforce specific attributes, while a memory module tracks the generation statistics and steers latent guidance to align with the targeted fair distribution of the attribute values. Further, given the limitations of existing datasets in comprehensively assessing bias in diffusion models, we introduce a holistic bias evaluation benchmark HBE, covering diverse domains and incorporating complex prompts across various applications. Extensive evaluations on HBE and Stable Bias datasets demonstrate that FairGen outperforms existing bias mitigation approaches, achieving substantial bias reduction (e.g., 68.5% gender bias reduction on Stable Diffusion 2). Ablation studies highlight FairGen's ability to flexibly and precisely control generation distribution at any user-specified granularity, ensuring adaptive and targeted bias mitigation.
• 摘要：文本到图像扩散模型通常会对特定的人群群体表现出偏见，例如，当提示产生工程师的图像，提出道德问题并限制其采用时，产生的男性比女性更多。在本文中，我们应对在扩散模型中减轻任何目标属性值（例如“男性”）的挑战，同时保留发电质量。我们提出了Fairgen，这是一种自适应潜在的指导机制，可以控制推理过程中的发电分布。在Fairgen中，潜在的指导模块动态调整了扩散过程以执行特定属性，而内存模块则跟踪生成统计信息并引导潜在的指导，以与属性值的目标公平分布保持一致。此外，鉴于现有数据集在扩散模型中全面评估偏见中的局限性，我们引入了整体偏见评估基准HBE，涵盖了各种域，并在各种应用程序中结合了复杂的提示。对HBE和稳定偏置数据集的广泛评估表明，Fairgen的表现优于现有的缓解方法，从而实现了大幅度的偏差降低（例如，稳定扩散2的68.5％性别偏差减少2）。消融研究突出了费尔根（Fairgen）在任何用户指定的粒度上灵活，精确地控制发电分布的能力，从而确保适应性和有针对性的偏差缓解。

Title: Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis

• Authors: Long Cheng, Qichen Liao, Fan Wu, Junlin Mu, Tengfei Han, Zhe Qiu, Lianqiang Li, Tianyi Liu, Fangzheng Miao, Keming Gao, Liang Wang, Zhen Zhang, Qiande Yin
• Subjects: cs.LG, cs.AI, cs.PF, math.NA
• Abstract URL: https://arxiv.org/abs/2503.01873
• Pdf URL: https://arxiv.org/pdf/2503.01873
• Copy Paste: [[2503.01873]] Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis(https://arxiv.org/abs/2503.01873)
• Keywords: generation
• Abstract: Attention calculation is extremely time-consuming for long-sequence inference tasks, such as text or image/video generation, in large models. To accelerate this process, we developed a low-precision, mathematically-equivalent algorithm called PASA, based on Flash Attention. PASA introduces two novel techniques: online pseudo-average shifting and global recovering. These techniques enable the use of half-precision computation throughout the Flash Attention process without incurring overflow instability or unacceptable numerical accuracy loss. This algorithm enhances performance on memory-restricted AI hardware architectures, such as the Ascend Neural-network Processing Unit(NPU), by reducing data movement and increasing computational FLOPs. The algorithm is validated using both designed random benchmarks and real large models. We find that the large bias and amplitude of attention input data are critical factors contributing to numerical overflow ($>65504$ for half precision) in two different categories of large models (Qwen2-7B language models and Stable-Video-Diffusion multi-modal models). Specifically, overflow arises due to the large bias in the sequence dimension and the resonance mechanism between the query and key in the head dimension of the Stable-Video-Diffusion models. The resonance mechanism is defined as phase coincidence or 180-degree phase shift between query and key matrices. It will remarkably amplify the element values of attention score matrix. This issue also applies to the Qwen models. Additionally, numerical accuracy is assessed through root mean square error (RMSE) and by comparing the final generated texts and videos to those produced using high-precision attention.
• 摘要：对于长期推断任务（例如文本或图像/视频生成），注意力计算非常耗时。为了加速这一过程，我们基于闪光的注意，开发了一种低精确的数学等效算法，称为PASA。 PASA介绍了两种新颖的技术：在线伪平均转移和全球恢复。这些技术可以在整个闪光注意过程中使用半精度计算，而不会产生溢出不稳定性或不可接受的数值精度损失。该算法通过减少数据移动和增加计算拖船来提高内存限制的AI硬件架构（例如上升神经网络处理单元（NPU））的性能。使用设计的随机基准和真实的大型模型对该算法进行验证。我们发现，注意力输入数据的巨大偏见和幅度是两个不同类别的大型模型（QWEN2-7B语言模型和稳定的Video-Diffusion Multioododal模型）的数值溢出（$> 65504 $）的关键因素。具体而言，由于序列维度的偏差以及查询和键之间的共振机理的偏差很大，因此溢出是溢出的。共振机制定义为相巧合或查询和键矩阵之间的180度相位移位。它将显着扩大注意力评分矩阵的元素值。此问题也适用于QWEN模型。此外，通过均方根误差（RMSE）以及将最终生成的文本和视频与使用高精度关注的最终生成的文本和视频进行比较来评估数值精度。

Title: Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models

• Authors: Zhanpeng He, Yifeng Cao, Matei Ciocarlie
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.01876
• Pdf URL: https://arxiv.org/pdf/2503.01876
• Copy Paste: [[2503.01876]] Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models(https://arxiv.org/abs/2503.01876)
• Keywords: generative
• Abstract: Human-in-the-loop (HitL) robot deployment has gained significant attention in both academia and industry as a semi-autonomous paradigm that enables human operators to intervene and adjust robot behaviors at deployment time, improving success rates. However, continuous human monitoring and intervention can be highly labor-intensive and impractical when deploying a large number of robots. To address this limitation, we propose a method that allows diffusion policies to actively seek human assistance only when necessary, reducing reliance on constant human oversight. To achieve this, we leverage the generative process of diffusion policies to compute an uncertainty-based metric based on which the autonomous agent can decide to request operator assistance at deployment time, without requiring any operator interaction during training. Additionally, we show that the same method can be used for efficient data collection for fine-tuning diffusion policies in order to improve their autonomous performance. Experimental results from simulated and real-world environments demonstrate that our approach enhances policy performance during deployment for a variety of scenarios.
• 摘要：作为半自治的范式，人类的机器人部署在学术界和行业中都引起了人们的重大关注，使人类操作员能够在部署时间进行干预和调整机器人行为，从而提高了成功率。但是，在部署大量机器人时，持续的人类监测和干预可能是高度劳动密集型和不切实际的。为了解决这一局限性，我们提出了一种允许扩散政策只能在必要时积极寻求人类援助的方法，从而减少对不断的人类监督的依赖。为了实现这一目标，我们利用扩散政策的生成过程来计算基于不确定性的度量，自主代理可以决定在部署时间请求操作员帮助，而无需在培训期间进行任何操作员交互。此外，我们表明，可以使用相同的方法来收集微调扩散策略，以提高其自主性能。模拟和现实世界环境的实验结果表明，我们的方法可以在各种情况下的部署过程中提高政策绩效。

Title: AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs

• Authors: Zihao Zeng, Chubo Liu, Xin He, Juan Hu, Yong Jiang, Fei Huang, Kenli Li, Wei Yang Bryan Lim
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01890
• Pdf URL: https://arxiv.org/pdf/2503.01890
• Copy Paste: [[2503.01890]] AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs(https://arxiv.org/abs/2503.01890)
• Keywords: generation
• Abstract: Transformer-based large language models (LLMs) have demonstrated exceptional capabilities in sequence modeling and text generation, with improvements scaling proportionally with model size. However, the limitations of GPU memory have restricted LLM training accessibility for many researchers. Existing heterogeneous training methods significantly expand the scale of trainable models but introduce substantial communication overheads and CPU workloads. In this work, we propose AutoHete, an automatic and efficient heterogeneous training system compatible with both single-GPU and multi-GPU environments. AutoHete dynamically adjusts activation checkpointing, parameter offloading, and optimizer offloading based on the specific hardware configuration and LLM training needs. Additionally, we design a priority-based scheduling mechanism that maximizes the overlap between operations across training iterations, enhancing throughput. Compared to state-of-the-art heterogeneous training systems, AutoHete delivers a 1.32x~1.91x throughput improvement across various model sizes and training configurations.
• 摘要：基于变压器的大型语言模型（LLMS）已经证明了序列建模和文本生成的非凡功能，并与模型大小相称地缩放。但是，GPU内存的局限性限制了许多研究人员的LLM培训可访问性。现有的异质培训方法显着扩大了可训练模型的规模，但引入了实质性的沟通开销和CPU工作量。在这项工作中，我们提出了AutoHete，这是一种自动有效的异质训练系统，均与单GPU和多GPU环境兼容。 AutoHete根据特定的硬件配置和LLM培训需求动态调整激活检查点，参数卸载和优化器卸载。此外，我们设计了一种基于优先级的调度机制，该机制可以最大程度地提高训练迭代的操作之间的重叠，从而增强吞吐量。与最先进的异质训练系统相比，AutoHete在各种模型尺寸和训练配置方面提供了1.32x〜1.91x的吞吐量改进。

Title: Recognition of Dysarthria in Amyotrophic Lateral Sclerosis patients using Hypernetworks

• Authors: Loukas Ilias, Dimitris Askounis
• Subjects: cs.LG, cs.CV, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.01892
• Pdf URL: https://arxiv.org/pdf/2503.01892
• Copy Paste: [[2503.01892]] Recognition of Dysarthria in Amyotrophic Lateral Sclerosis patients using Hypernetworks(https://arxiv.org/abs/2503.01892)
• Keywords: generative
• Abstract: Amyotrophic Lateral Sclerosis (ALS) constitutes a progressive neurodegenerative disease with varying symptoms, including decline in speech intelligibility. Existing studies, which recognize dysarthria in ALS patients by predicting the clinical standard ALSFRS-R, rely on feature extraction strategies and the design of customized convolutional neural networks followed by dense layers. However, recent studies have shown that neural networks adopting the logic of input-conditional computations enjoy a series of benefits, including faster training, better performance, and flexibility. To resolve these issues, we present the first study incorporating hypernetworks for recognizing dysarthria. Specifically, we use audio files, convert them into log-Mel spectrogram, delta, and delta-delta, and pass the resulting image through a pretrained modified AlexNet model. Finally, we use a hypernetwork, which generates weights for a target network. Experiments are conducted on a newly collected publicly available dataset, namely VOC-ALS. Results showed that the proposed approach reaches Accuracy up to 82.66% outperforming strong baselines, including multimodal fusion methods, while findings from an ablation study demonstrated the effectiveness of the introduced methodology. Overall, our approach incorporating hypernetworks obtains valuable advantages over state-of-the-art results in terms of generalization ability, parameter efficiency, and robustness.
• 摘要：肌萎缩性侧索硬化症（ALS）构成一种症状不同的进行性神经退行性疾病，包括言语清晰度的下降。现有研究通过预测临床标准ALSFRS-R来识别ALS患者的构造障碍，依赖于特征提取策略和定制的卷积神经网络的设计，然后是密集的层。但是，最近的研究表明，采用输入条件计算的逻辑的神经网络具有一系列好处，包括更快的训练，更好的性能和灵活性。为了解决这些问题，我们提出了一项结合了用于识别构音障碍的超网络的研究。具体来说，我们使用音频文件，将其转换为log-mel频谱图，delta和delta-delta，并通过预验证的修改后的Alexnet模型传递所得图像。最后，我们使用了一个超网络，该网络为目标网络生成权重。实验是在新收集的公开数据集（即VOC-als）上进行的。结果表明，所提出的方法达到了高达82.66％的精度，包括多模式融合方法，而消融研究的发现证明了引入方法的有效性。总体而言，我们包含超级核武器的方法在概括能力，参数效率和鲁棒性方面获得了与最先进的结果相比的有价值的优势。

Title: LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces

• Authors: Rashid Mushkani, Shravan Nayak, Hugo Berard, Allison Cohen, Shin Koseki, Hadrien Bertrand
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01894
• Pdf URL: https://arxiv.org/pdf/2503.01894
• Copy Paste: [[2503.01894]] LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces(https://arxiv.org/abs/2503.01894)
• Keywords: generative
• Abstract: We introduce the Local Intersectional Visual Spaces (LIVS) dataset, a benchmark for multi-criteria alignment of text-to-image (T2I) models in inclusive urban planning. Developed through a two-year participatory process with 30 community organizations, LIVS encodes diverse spatial preferences across 634 initial concepts, consolidated into six core criteria: Accessibility, Safety, Comfort, Invitingness, Inclusivity, and Diversity, through 37,710 pairwise comparisons. Using Direct Preference Optimization (DPO) to fine-tune Stable Diffusion XL, we observed a measurable increase in alignment with community preferences, though a significant proportion of neutral ratings highlights the complexity of modeling intersectional needs. Additionally, as annotation volume increases, accuracy shifts further toward the DPO-tuned model, suggesting that larger-scale preference data enhances fine-tuning effectiveness. LIVS underscores the necessity of integrating context-specific, stakeholder-driven criteria into generative modeling and provides a resource for evaluating AI alignment methodologies across diverse socio-spatial contexts.
• 摘要：我们介绍了当地的交叉视觉空间（LIVS）数据集，这是包容性城市规划中文本对图像（T2I）模型多标准对齐的基准。 LIV通过与30个社区组织的两年参与过程开发，在634个初始概念中编码了各种空间偏好，并通过37,710个成对比较巩固了六个核心标准：可访问性，安全性，安全性，舒适性，邀请，包容性和多样性。使用直接偏好优化（DPO）来微调稳定的扩散XL，我们观察到可衡量的与社区偏好的比对增加，尽管中性等级的很大一部分突出了建模交叉点需求的复杂性。此外，随着注释量的增加，准确性进一步转移到了DPO调整的模型，这表明较大规模的偏好数据提高了微调效率。 LIV强调了将特定于上下文的，利益相关者驱动的标准集成到生成建模中的必要性，并为评估各种社会空间环境中的AI一致性方法提供了资源。

Title: Continual Learning-Aided Super-Resolution Scheme for Channel Reconstruction and Generalization in OFDM Systems

• Authors: Jianqiao Chen, Nan Ma, Wenkai Liu, Xiaodong Xu, Ping Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01897
• Pdf URL: https://arxiv.org/pdf/2503.01897
• Copy Paste: [[2503.01897]] Continual Learning-Aided Super-Resolution Scheme for Channel Reconstruction and Generalization in OFDM Systems(https://arxiv.org/abs/2503.01897)
• Keywords: super-resolution, generation
• Abstract: Channel reconstruction and generalization capability are of equal importance for developing channel estimation schemes within deep learning (DL) framework. In this paper, we exploit a novel DL-based scheme for efficient OFDM channel estimation where the neural networks for channel reconstruction and generalization are respectively designed. For the former, we propose a dual-attention-aided super-resolution neural network (DA-SRNN) to map the channels at pilot positions to the whole time-frequency channels. Specifically, the channel-spatial attention mechanism is first introduced to sequentially infer attention maps along two separate dimensions corresponding to two types of underlying channel correlations, and then the lightweight SR module is developed for efficient channel reconstruction. For the latter, we introduce continual learning (CL)-aided training strategies to make the neural network adapt to different channel distributions. Specifically, the elastic weight consolidation (EWC) is introduced as the regularization term in regard to loss function of channel reconstruction, which can constrain the direction and space of updating the important weights of neural networks among different channel distributions. Meanwhile, the corresponding training process is provided in detail. By evaluating under 3rd Generation Partnership Project (3GPP) channel models, numerical results verify the superiority of the proposed channel estimation scheme with significantly improved channel reconstruction and generalization performance over counterparts.
• 摘要：对于在深度学习（DL）框架内制定信道估计方案（DL）框架中的渠道重建和概括能力都具有同等的重要性。在本文中，我们利用了一种基于DL的新方案来进行有效的OFDM通道估计，其中分别设计用于通道重建和泛化的神经网络。对于前者，我们提出了一个双重注意的超分辨率神经网络（DA-SRNN），以将试点位置的通道映射到整个时频渠道。具体而言，首先引入了通道空间注意机制，以沿两个单独的维度依次地推断出注意力图，与两种类型的基础通道相关性相对应，然后开发了轻质SR模块以进行有效的通道重建。对于后者，我们引入了持续学习（CL）辅助培训策略，以使神经网络适应不同的渠道分布。具体而言，弹性重量合并（EWC）作为通道重建的损耗函数的正则化项，这可以约束更新不同通道分布之间神经网络重要权重的方向和空间。同时，提供了相应的培训过程。通过评估第三代合作伙伴项目（3GPP）渠道模型，数值结果验证了所提出的通道估计方案的优越性，并具有明显改善的通道重建和泛化性能，而不是对应物。

Title: VAEs and GANs: Implicitly Approximating Complex Distributions with Simple Base Distributions and Deep Neural Networks -- Principles, Necessity, and Limitations

• Authors: Yuan-Hao Wei
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01898
• Pdf URL: https://arxiv.org/pdf/2503.01898
• Copy Paste: [[2503.01898]] VAEs and GANs: Implicitly Approximating Complex Distributions with Simple Base Distributions and Deep Neural Networks -- Principles, Necessity, and Limitations(https://arxiv.org/abs/2503.01898)
• Keywords: generative
• Abstract: This tutorial focuses on the fundamental architectures of Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN), disregarding their numerous variations, to highlight their core principles. Both VAE and GAN utilize simple distributions, such as Gaussians, as a basis and leverage the powerful nonlinear transformation capabilities of neural networks to approximate arbitrarily complex distributions. The theoretical basis lies in that a linear combination of multiple Gaussians can almost approximate any probability distribution, while neural networks enable further refinement through nonlinear transformations. Both methods approximate complex data distributions implicitly. This implicit approximation is crucial because directly modeling high-dimensional distributions explicitly is often intractable. However, the choice of a simple latent prior, while computationally convenient, introduces limitations. In VAEs, the fixed Gaussian prior forces the posterior distribution to align with it, potentially leading to loss of information and reduced expressiveness. This restriction affects both the interpretability of the model and the quality of generated samples.
• 摘要：该教程侧重于变异自动编码器（VAE）和生成对抗网络（GAN）的基本体系结构，无视它们的众多变体，以突出其核心原则。 VAE和GAN都利用简单的分布（例如高斯人）作为基础，并利用神经网络的强大非线性转化能力来近似任意复杂的分布。理论基础在于，多个高斯人的线性组合几乎可以近似任何概率分布，而神经网络则可以通过非线性转化进行进一步的细化。两种方法都隐含地近似复杂的数据分布。这种隐式近似是至关重要的，因为直接对高维分布进行建模通常是棘手的。但是，选择简单的潜在先验，虽然计算方便，但却引入了限制。在VAE中，固定的高斯先验迫使后验分布与之保持一致，可能导致信息丢失并降低表现力。这种限制既影响模型的可解释性，也影响了生成样品的质量。

Title: How to Steer LLM Latents for Hallucination Detection?

• Authors: Seongheon Park, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, Yixuan Li
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01917
• Pdf URL: https://arxiv.org/pdf/2503.01917
• Copy Paste: [[2503.01917]] How to Steer LLM Latents for Hallucination Detection?(https://arxiv.org/abs/2503.01917)
• Keywords: generation
• Abstract: Hallucinations in LLMs pose a significant concern to their safe deployment in real-world applications. Recent approaches have leveraged the latent space of LLMs for hallucination detection, but their embeddings, optimized for linguistic coherence rather than factual accuracy, often fail to clearly separate truthful and hallucinated content. To this end, we propose the Truthfulness Separator Vector (TSV), a lightweight and flexible steering vector that reshapes the LLM's representation space during inference to enhance the separation between truthful and hallucinated outputs, without altering model parameters. Our two-stage framework first trains TSV on a small set of labeled exemplars to form compact and well-separated clusters. It then augments the exemplar set with unlabeled LLM generations, employing an optimal transport-based algorithm for pseudo-labeling combined with a confidence-based filtering process. Extensive experiments demonstrate that TSV achieves state-of-the-art performance with minimal labeled data, exhibiting strong generalization across datasets and providing a practical solution for real-world LLM applications.
• 摘要：LLMS中的幻觉对他们在现实世界应用中的安全部署引起了重大关注。最近的方法利用LLM的潜在空间进行幻觉检测，但是它们的嵌入以语言连贯性而不是事实准确性进行了优化，通常无法清楚地分离出真实和幻觉的内容。为此，我们提出了真实性分离器向量（TSV），这是一种轻巧且灵活的转向向量，可以重塑推断期间LLM的表示空间，以增强不改变模型参数的真实和幻觉输出之间的分离。我们的两阶段框架首先在一小部分标记的示例上训练TSV，形成紧凑且分离良好的群集。然后，它使用未标记的LLM世代增强了示例集，采用了基于最佳传输算法的伪标记算法与基于置信的过滤过程相结合。广泛的实验表明，TSV通过最小的标记数据实现了最先进的性能，在数据集跨数据集表现出强烈的概括，并为现实世界中LLM应用程序提供了实用的解决方案。

Title: Adversarial Generative Flow Network for Solving Vehicle Routing Problems

• Authors: Ni Zhang, Jingfeng Yang, Zhiguang Cao, Xu Chi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01931
• Pdf URL: https://arxiv.org/pdf/2503.01931
• Copy Paste: [[2503.01931]] Adversarial Generative Flow Network for Solving Vehicle Routing Problems(https://arxiv.org/abs/2503.01931)
• Keywords: generative
• Abstract: Recent research into solving vehicle routing problems (VRPs) has gained significant traction, particularly through the application of deep (reinforcement) learning for end-to-end solution construction. However, many current construction-based neural solvers predominantly utilize Transformer architectures, which can face scalability challenges and struggle to produce diverse solutions. To address these limitations, we introduce a novel framework beyond Transformer-based approaches, i.e., Adversarial Generative Flow Networks (AGFN). This framework integrates the generative flow network (GFlowNet)-a probabilistic model inherently adept at generating diverse solutions (routes)-with a complementary model for discriminating (or evaluating) the solutions. These models are trained alternately in an adversarial manner to improve the overall solution quality, followed by a proposed hybrid decoding method to construct the solution. We apply the AGFN framework to solve the capacitated vehicle routing problem (CVRP) and travelling salesman problem (TSP), and our experimental results demonstrate that AGFN surpasses the popular construction-based neural solvers, showcasing strong generalization capabilities on synthetic and real-world benchmark instances.
• 摘要：最近针对解决车辆路线问题（VRP）的研究已获得了巨大的吸引力，尤其是通过将深入（加固）学习用于端到端溶液构建。但是，许多目前的基于施工的神经求解器主要利用变压器体系结构，这可能面临可扩展性挑战并难以生产各种解决方案。为了解决这些局限性，我们引入了除基于变压器的方法（即对抗性生成流网络（AGFN））之外的新型框架。该框架集成了生成流网络（GFLOWNET）-A概率模型固有地擅长生成各种解决方案（路由） - 与区分（或评估）解决方案的互补模型。这些模型以对抗性方式进行了交替训练，以提高整体溶液质量，然后采用拟议的混合解码方法来构建解决方案。我们应用AGFN框架来解决电容的车辆路由问题（CVRP）和旅行推销员问题（TSP），我们的实验结果表明，AGFN超过了基于建筑的流行神经求解器，展示了在合成和现实的基准标准实例上展示了强大的概括能力。

Title: Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA

• Authors: Zhusi Zhong, Yuli Wang, Lulu Bi, Zhuoqi Ma, Sun Ho Ahn, Christopher J. Mullin, Colin F. Greineder, Michael K. Atalay, Scott Collins, Grayson L. Baird, Cheng Ting Lin, Webster Stayman, Todd M. Kolb, Ihab Kamel, Harrison X. Bai, Zhicheng Jiao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02034
• Pdf URL: https://arxiv.org/pdf/2503.02034
• Copy Paste: [[2503.02034]] Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA(https://arxiv.org/abs/2503.02034)
• Keywords: generation
• Abstract: Medical imaging plays a pivotal role in modern healthcare, with computed tomography pulmonary angiography (CTPA) being a critical tool for diagnosing pulmonary embolism and other thoracic conditions. However, the complexity of interpreting CTPA scans and generating accurate radiology reports remains a significant challenge. This paper introduces Abn-BLIP (Abnormality-aligned Bootstrapping Language-Image Pretraining), an advanced diagnosis model designed to align abnormal findings to generate the accuracy and comprehensiveness of radiology reports. By leveraging learnable queries and cross-modal attention mechanisms, our model demonstrates superior performance in detecting abnormalities, reducing missed findings, and generating structured reports compared to existing methods. Our experiments show that Abn-BLIP outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance. These results highlight the potential of integrating multimodal learning strategies for improving radiology reporting. The source code is available at this https URL.
• 摘要：医学成像在现代医疗保健中起关键作用，计算机断层扫描肺血管造影（CTPA）是诊断肺栓塞和其他胸腔状况的关键工具。但是，解释CTPA扫描和产生准确的放射学报告的复杂性仍然是一个重大挑战。本文介绍了ABN-Blip（与异常一致的自举语言图像​​预处理），这是一种高级诊断模型，旨在使异常发现与放射学报告的准确性和全面性相结合。通过利用可学习的查询和跨模式注意机制，我们的模型在检测异常，减少缺失的发现以及与现有方法相比生成结构化报告时表现出了卓越的性能。我们的实验表明，ABN-Blip在准确性和临床相关性上都优于最先进的医学视觉模型和3D报告生成方法。这些结果突出了整合多模式学习策略以改善放射学报告的潜力。源代码可在此HTTPS URL上找到。

Title: Dynamic Search for Inference-Time Alignment in Diffusion Models

• Authors: Xiner Li, Masatoshi Uehara, Xingyu Su, Gabriele Scalia, Tommaso Biancalani, Aviv Regev, Sergey Levine, Shuiwang Ji
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02039
• Pdf URL: https://arxiv.org/pdf/2503.02039
• Copy Paste: [[2503.02039]] Dynamic Search for Inference-Time Alignment in Diffusion Models(https://arxiv.org/abs/2503.02039)
• Keywords: generation, generative
• Abstract: Diffusion models have shown promising generative capabilities across diverse domains, yet aligning their outputs with desired reward functions remains a challenge, particularly in cases where reward functions are non-differentiable. Some gradient-free guidance methods have been developed, but they often struggle to achieve optimal inference-time alignment. In this work, we newly frame inference-time alignment in diffusion as a search problem and propose Dynamic Search for Diffusion (DSearch), which subsamples from denoising processes and approximates intermediate node rewards. It also dynamically adjusts beam width and tree expansion to efficiently explore high-reward generations. To refine intermediate decisions, DSearch incorporates adaptive scheduling based on noise levels and a lookahead heuristic function. We validate DSearch across multiple domains, including biological sequence design, molecular optimization, and image generation, demonstrating superior reward optimization compared to existing approaches.
• 摘要：扩散模型已经显示出各种域之间有希望的生成能力，但是将其输出与所需的奖励功能保持一致仍然是一个挑战，尤其是在奖励功能不可差异的情况下。已经开发了一些无梯度的指导方法，但是它们通常很难实现最佳的推理时间对齐。在这项工作中，我们新将扩散中的推理时间对齐作为搜索问题，并提出了对扩散的动态搜索（dsearch），该搜索是从剥离过程中的样本和近似中间节点奖励的。它还动态调节梁的宽度和树木膨胀，以有效地探索高回报的世代。为了完善中间决策，DSEarch根据噪声水平和lookahead启发式功能结合了自适应调度。我们验证了跨多个领域的研究，包括生物序列设计，分子优化和图像产生，与现有方法相比表明了优越的奖励优化。

Title: Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detection

• Authors: Boyong He, Yuxiang Ji, Qianwen Ye, Zhuoyue Tan, Liaoni Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02101
• Pdf URL: https://arxiv.org/pdf/2503.02101
• Copy Paste: [[2503.02101]] Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detection(https://arxiv.org/abs/2503.02101)
• Keywords: generation
• Abstract: Domain generalization (DG) for object detection aims to enhance detectors' performance in unseen scenarios. This task remains challenging due to complex variations in real-world applications. Recently, diffusion models have demonstrated remarkable capabilities in diverse scene generation, which inspires us to explore their potential for improving DG tasks. Instead of generating images, our method extracts multi-step intermediate features during the diffusion process to obtain domain-invariant features for generalized detection. Furthermore, we propose an efficient knowledge transfer framework that enables detectors to inherit the generalization capabilities of diffusion models through feature and object-level alignment, without increasing inference time. We conduct extensive experiments on six challenging DG benchmarks. The results demonstrate that our method achieves substantial improvements of 14.0% mAP over existing DG approaches across different domains and corruption types. Notably, our method even outperforms most domain adaptation methods without accessing any target domain data. Moreover, the diffusion-guided detectors show consistent improvements of 15.9% mAP on average compared to the baseline. Our work aims to present an effective approach for domain-generalized detection and provide potential insights for robust visual recognition in real-world scenarios. The code is available at \href{this https URL}{Generalized Diffusion Detector}
• 摘要：对象检测的域概括（DG）旨在在看不见的情况下增强检测器的性能。由于现实世界应用中的复杂变化，此任务仍然具有挑战性。最近，扩散模型在不同的场景生成中表现出了非凡的功能，这激发了我们探索他们改善DG任务的潜力。我们的方法没有生成图像，而是在扩散过程中提取多步中间特征，以获取用于广义检测的域不变特征。此外，我们提出了一个有效的知识转移框架，该框架使检测器能够通过特征和对象级比对继承扩散模型的概括能力，而不会增加推理时间。我们对六个具有挑战性的DG基准进行了广泛的实验。结果表明，我们的方法可在不同领域和腐败类型的现有DG方法上实现14.0％的映射。值得注意的是，我们的方法甚至胜过大多数域适应方法，而无需访问任何目标域数据。此外，与基线相比，扩散引导的探测器平均显示出15.9％的地图的一致性提高。我们的工作旨在提出一种有效的域名检测方法，并在现实情况下为强大的视觉识别提供潜在的见解。该代码可在\ href {此https url} {概括性扩散检测器}中获得

Title: HanDrawer: Leveraging Spatial Information to Render Realistic Hands Using a Conditional Diffusion Model in Single Stage

• Authors: Qifan Fu, Xu Chen, Muhammad Asad, Shanxin Yuan, Changjae Oh, Gregory Slabaugh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02127
• Pdf URL: https://arxiv.org/pdf/2503.02127
• Copy Paste: [[2503.02127]] HanDrawer: Leveraging Spatial Information to Render Realistic Hands Using a Conditional Diffusion Model in Single Stage(https://arxiv.org/abs/2503.02127)
• Keywords: generation
• Abstract: Although diffusion methods excel in text-to-image generation, generating accurate hand gestures remains a major challenge, resulting in severe artifacts, such as incorrect number of fingers or unnatural gestures. To enable the diffusion model to learn spatial information to improve the quality of the hands generated, we propose HanDrawer, a module to condition the hand generation process. Specifically, we apply graph convolutional layers to extract the endogenous spatial structure and physical constraints implicit in MANO hand mesh vertices. We then align and fuse these spatial features with other modalities via cross-attention. The spatially fused features are used to guide a single stage diffusion model denoising process for high quality generation of the hand region. To improve the accuracy of spatial feature fusion, we propose a Position-Preserving Zero Padding (PPZP) fusion strategy, which ensures that the features extracted by HanDrawer are fused into the region of interest in the relevant layers of the diffusion model. HanDrawer learns the entire image features while paying special attention to the hand region thanks to an additional hand reconstruction loss combined with the denoising loss. To accurately train and evaluate our approach, we perform careful cleansing and relabeling of the widely used HaGRID hand gesture dataset and obtain high quality multimodal data. Quantitative and qualitative analyses demonstrate the state-of-the-art performance of our method on the HaGRID dataset through multiple evaluation metrics. Source code and our enhanced dataset will be released publicly if the paper is accepted.
• 摘要：尽管扩散方法在文本到图像的生成中表现出色，但产生准确的手势仍然是一个重大挑战，导致严重的伪像，例如手指数量不正确或不自然的手势。为了使扩散模型能够学习空间信息以提高生成的手的质量，我们提出了Handrawer，这是一个调节手工生成过程的模块。具体而言，我们应用图形卷积层来提取Mano手击网顶点中隐含的内源空间结构和物理约束。然后，我们通过交叉注意将这些空间特征与其他方式融合并融合。空间融合的特征用于指导单阶段扩散模型denoising过程，用于高质量的手部区域。为了提高空间特征融合的准确性，我们提出了一个保存位置的零填充（PPZP）融合策略，该策略可确保在扩散模型的相关层中将用档案提取的特征融合到感兴趣的区域中。掌握者了解了整个图像功能，同时要特别注意手动区域，这要归功于额外的手动重建损失与降解损失相结合。为了准确训练和评估我们的方法，我们对广泛使用的海格手势数据集进行了仔细的清洁和重新标记，并获得高质量的多模式数据。定量和定性分析证明了通过多个评估指标在海格数据集上我们方法的最新性能。源代码和我们的增强数据集将在接受论文的情况下公开发布。

Title: Aerial Infrared Health Monitoring of Solar Photovoltaic Farms at Scale

• Authors: Isaac Corley, Conor Wallace, Sourav Agrawal, Burton Putrah, Jonathan Lwowski
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02128
• Pdf URL: https://arxiv.org/pdf/2503.02128
• Copy Paste: [[2503.02128]] Aerial Infrared Health Monitoring of Solar Photovoltaic Farms at Scale(https://arxiv.org/abs/2503.02128)
• Keywords: generation
• Abstract: Solar photovoltaic (PV) farms represent a major source of global renewable energy generation, yet their true operational efficiency often remains unknown at scale. In this paper, we present a comprehensive, data-driven framework for large-scale airborne infrared inspection of North American solar installations. Leveraging high-resolution thermal imagery, we construct and curate a geographically diverse dataset encompassing thousands of PV sites, enabling machine learning-based detection and localization of defects that are not detectable in the visible spectrum. Our pipeline integrates advanced image processing, georeferencing, and airborne thermal infrared anomaly detection to provide rigorous estimates of performance losses. We highlight practical considerations in aerial data collection, annotation methodologies, and model deployment across a wide range of environmental and operational conditions. Our work delivers new insights into the reliability of large-scale solar assets and serves as a foundation for ongoing research on performance trends, predictive maintenance, and scalable analytics in the renewable energy sector.
• 摘要：太阳能光伏（PV）农场是全球可再生能源产生的主要来源，但其真正的运营效率通常在大规模上仍然未知。在本文中，我们提出了一个全面的，数据驱动的框架，用于对北美太阳能装置的大规模空降红外检查。利用高分辨率的热图像，我们构建和策划了一个包含数千个PV站点的地理上不同的数据集，从而使基于机器学习的检测和可见频谱中无法检测到的缺陷的定位。我们的管道集成了高级图像处理，地理发射和空气传播的热红外异常检测，以提供严格的性能损失估计。我们重点介绍了在广泛的环境和操作条件范围内的航空数据收集，注释方法和模型部署中的实际考虑。我们的工作为大型太阳能资产的可靠性提供了新的见解，并为可再生能源领域的绩效趋势，预测性维护和可扩展分析提供了持续研究的基础。

Title: Four Principles for Physically Interpretable World Models

• Authors: Jordan Peper, Zhenjiang Mao, Yuang Geng, Siyuan Pan, Ivan Ruchkin
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.02143
• Pdf URL: https://arxiv.org/pdf/2503.02143
• Copy Paste: [[2503.02143]] Four Principles for Physically Interpretable World Models(https://arxiv.org/abs/2503.02143)
• Keywords: generative
• Abstract: As autonomous systems are increasingly deployed in open and uncertain settings, there is a growing need for trustworthy world models that can reliably predict future high-dimensional observations. The learned latent representations in world models lack direct mapping to meaningful physical quantities and dynamics, limiting their utility and interpretability in downstream planning, control, and safety verification. In this paper, we argue for a fundamental shift from physically informed to physically interpretable world models - and crystallize four principles that leverage symbolic knowledge to achieve these ends: (1) structuring latent spaces according to the physical intent of variables, (2) learning aligned invariant and equivariant representations of the physical world, (3) adapting training to the varied granularity of supervision signals, and (4) partitioning generative outputs to support scalability and verifiability. We experimentally demonstrate the value of each principle on two benchmarks. This paper opens several intriguing research directions to achieve and capitalize on full physical interpretability in world models.
• 摘要：随着自主系统越来越多地部署在开放和不确定的环境中，人们对可信赖的世界模型的需求越来越大，可以可靠地预测未来的高维观察。世界模型中博学的潜在表示缺乏直接映射到有意义的物理数量和动力学上，从而限制了它们在下游计划，控制和安全验证中的效用和解释性。在本文中，我们主张从物理知情到物理可解释的世界模型的基本转变，并结晶了四个原则，以实现这些目的，以实现这些目的：（1）根据变量的物理意图来构造潜在空间，（2）学习使世界的不变和等效性培训，（3）培训的统一性，（3）均为（3）变化的分区，（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）变量（3）。输出以支持可伸缩性和可验证性。我们在实验上证明了每个原理在两个基准上的价值。本文打开了一些有趣的研究方向，以实现和利用世界模型中的全部物理解释性。

Title: LLM-TabFlow: Synthetic Tabular Data Generation with Inter-column Logical Relationship Preservation

• Authors: Yunbo Long, Liming Xu, Alexandra Brintrup
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02161
• Pdf URL: https://arxiv.org/pdf/2503.02161
• Copy Paste: [[2503.02161]] LLM-TabFlow: Synthetic Tabular Data Generation with Inter-column Logical Relationship Preservation(https://arxiv.org/abs/2503.02161)
• Keywords: generation, generative
• Abstract: Synthetic tabular data have widespread applications in industrial domains such as healthcare, finance, and supply chains, owing to their potential to protect privacy and mitigate data scarcity. However, generating realistic synthetic tabular data while preserving inter-column logical relationships remains a significant challenge for the existing generative models. To address these challenges, we propose LLM-TabFlow, a novel approach that leverages Large Language Model (LLM) reasoning to capture complex inter-column relationships and compress tabular data, while using Score-based Diffusion to model the distribution of the compressed data in latent space. Additionally, we introduce an evaluation framework, which is absent in literature, to fairly assess the performance of synthetic tabular data generation methods in real-world contexts. Using this framework, we conduct extensive experiments on two real-world industrial datasets, evaluating LLM-TabFlow against other five baseline methods, including SMOTE (an interpolation-based approach) and other state-of-the-art generative models. Our results show that LLM-TabFlow outperforms all baselines, fully preserving inter-column relationships while achieving the best balance between data fidelity, utility, and privacy. This study is the first to explicitly address inter-column relationship preservation in synthetic tabular data generation, offering new insights for developing more realistic and reliable tabular data generation methods.
• 摘要：合成表格数据在医疗保健，金融和供应链等工业领域中具有广泛的应用程序，因为它们可以保护隐私和减轻数据稀缺性。但是，在保留柱间逻辑关系的同时生成现实的合成表格数据对于现有生成模型来说仍然是一个重大挑战。为了应对这些挑战，我们提出了LLM-TABFLOF，这是一种利用大型语言模型（LLM）推理来捕获复杂的列之间关系和压缩表格数据的新方法，同时使用基于得分的扩散来模拟潜在空间中压缩数据的分布。此外，我们引入了一个评估框架，该框架在文献中不存在，以公平地评估现实世界中环境中综合表格数据生成方法的性能。使用此框架，我们对两个实际工业数据集进行了广泛的实验，对其他五种基线方法评估LLM-TABFLOF，包括SMOTE（一种基于插值的方法）和其他最先进的生成模型。我们的结果表明，LLM-TabFlow优于所有基线，完全保留了列之间的关系，同时在数据保真，公用事业和隐私之间取得了最佳平衡。这项研究是第一个在合成表格数据生成中明确解决柱间关系保存的方法，为开发更现实和可靠的表格数据生成方法提供了新的见解。

Title: HyperGCT: A Dynamic Hyper-GNN-Learned Geometric Constraint for 3D Registration

• Authors: Xiyu Zhang, Jiayi Ma, Jianwei Guo, Wei Hu, Zhaoshuai Qi, Fei Hui, Jiaqi Yang, Yanning Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02195
• Pdf URL: https://arxiv.org/pdf/2503.02195
• Copy Paste: [[2503.02195]] HyperGCT: A Dynamic Hyper-GNN-Learned Geometric Constraint for 3D Registration(https://arxiv.org/abs/2503.02195)
• Keywords: generation
• Abstract: Geometric constraints between feature matches are critical in 3D point cloud registration problems. Existing approaches typically model unordered matches as a consistency graph and sample consistent matches to generate hypotheses. However, explicit graph construction introduces noise, posing great challenges for handcrafted geometric constraints to render consistency among matches. To overcome this, we propose HyperGCT, a flexible dynamic Hyper-GNN-learned geometric constraint that leverages high-order consistency among 3D correspondences. To our knowledge, HyperGCT is the first method that mines robust geometric constraints from dynamic hypergraphs for 3D registration. By dynamically optimizing the hypergraph through vertex and edge feature aggregation, HyperGCT effectively captures the correlations among correspondences, leading to accurate hypothesis generation. Extensive experiments on 3DMatch, 3DLoMatch, KITTI-LC, and ETH show that HyperGCT achieves state-of-the-art performance. Furthermore, our method is robust to graph noise, demonstrating a significant advantage in terms of generalization. The code will be released.
• 摘要：特征匹配之间的几何约束在3D点云注册问题中至关重要。现有方法通常将无序的匹配模拟为一致性图和样本一致匹配，以生成假设。但是，显式的图形构造引入了噪音，对手工制作的几何约束构成了巨大的挑战，以使比赛之间保持一致性。为了克服这一点，我们提出了HyperGCT，这是一种灵活的动态超网络学习的几何约束，它利用了3D对应关系之间的高阶一致性。据我们所知，HyperGCT是第一种从动态超图中挖掘出强大的几何约束的方法。通过通过顶点和边缘特征聚集动态优化超图，HyperGCT有效地捕获了对应之间的相关性，从而导致准确的假设产生。 3DMatch，3Dlomatch，Kitti-LC和ETH的广泛实验表明，HyperGCT可以实现最先进的性能。此外，我们的方法对图形噪声具有鲁棒性，在概括方面表现出显着的优势。代码将发布。

Title: Language-Guided Visual Perception Disentanglement for Image Quality Assessment and Conditional Image Generation

• Authors: Zhichao Yang, Leida Li, Pengfei Chen, Jinjian Wu, Giuseppe Valenzise
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02206
• Pdf URL: https://arxiv.org/pdf/2503.02206
• Copy Paste: [[2503.02206]] Language-Guided Visual Perception Disentanglement for Image Quality Assessment and Conditional Image Generation(https://arxiv.org/abs/2503.02206)
• Keywords: generation, quality assessment
• Abstract: Contrastive vision-language models, such as CLIP, have demonstrated excellent zero-shot capability across semantic recognition tasks, mainly attributed to the training on a large-scale I&1T (one Image with one Text) dataset. This kind of multimodal representations often blend semantic and perceptual elements, placing a particular emphasis on semantics. However, this could be problematic for popular tasks like image quality assessment (IQA) and conditional image generation (CIG), which typically need to have fine control on perceptual and semantic features. Motivated by the above facts, this paper presents a new multimodal disentangled representation learning framework, which leverages disentangled text to guide image disentanglement. To this end, we first build an I&2T (one Image with a perceptual Text and a semantic Text) dataset, which consists of disentangled perceptual and semantic text descriptions for an image. Then, the disentangled text descriptions are utilized as supervisory signals to disentangle pure perceptual representations from CLIP's original `coarse' feature space, dubbed DeCLIP. Finally, the decoupled feature representations are used for both image quality assessment (technical quality and aesthetic quality) and conditional image generation. Extensive experiments and comparisons have demonstrated the advantages of the proposed method on the two popular tasks. The dataset, code, and model will be available.
• 摘要：诸如剪辑之类的对比视觉模型已经在语义识别任务中表现出了出色的零击功能，这主要归因于大规模I＆1T（一个带有一个文本）数据集的训练。这种多模式表示通常将语义和感知元素融合在一起，从而特别强调语义。但是，这对于流行的任务（例如图像质量评估（IQA）和有条件的图像生成（CIG））可能是有问题的，这些任务通常需要对知觉和语义特征进行良好的控制。在上述事实的激励下，本文提出了一个新的多模式删节表示框架，该框架利用文本进行了删除文本来指导图像删除。为此，我们首先构建了一个I＆2T（具有感知文本和语义文本的图像）数据集，该数据集由图像的分离感知和语义文本描述组成。然后，将删除的文本说明用作监督信号，以将纯感知表示与剪辑的原始“粗'特征空间”（称为dectip）删除。最后，将解耦的特征表示形式用于图像质量评估（技术质量和美学质量）和有条件的图像产生。广泛的实验和比较证明了该方法在这两个流行任务上的优势。数据集，代码和模型将可用。

Title: $\mathbfΦ$-GAN: Physics-Inspired GAN for Generating SAR Images Under Limited Data

• Authors: Xidan Zhang, Yihan Zhuang, Qian Guo, Haodong Yang, Xuelin Qian, Gong Cheng, Junwei Han, Zhongling Huang
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.02242
• Pdf URL: https://arxiv.org/pdf/2503.02242
• Copy Paste: [[2503.02242]] $\mathbfΦ$-GAN: Physics-Inspired GAN for Generating SAR Images Under Limited Data(https://arxiv.org/abs/2503.02242)
• Keywords: generative
• Abstract: Approaches for improving generative adversarial networks (GANs) training under a few samples have been explored for natural images. However, these methods have limited effectiveness for synthetic aperture radar (SAR) images, as they do not account for the unique electromagnetic scattering properties of SAR. To remedy this, we propose a physics-inspired regularization method dubbed $\Phi$-GAN, which incorporates the ideal point scattering center (PSC) model of SAR with two physical consistency losses. The PSC model approximates SAR targets using physical parameters, ensuring that $\Phi$-GAN generates SAR images consistent with real physical properties while preventing discriminator overfitting by focusing on PSC-based decision cues. To embed the PSC model into GANs for end-to-end training, we introduce a physics-inspired neural module capable of estimating the physical parameters of SAR targets efficiently. This module retains the interpretability of the physical model and can be trained with limited data. We propose two physical loss functions: one for the generator, guiding it to produce SAR images with physical parameters consistent with real ones, and one for the discriminator, enhancing its robustness by basing decisions on PSC attributes. We evaluate $\Phi$-GAN across several conditional GAN (cGAN) models, demonstrating state-of-the-art performance in data-scarce scenarios on three SAR image datasets.
• 摘要：已经探索了自然图像的一些样本下改善生成对抗网络（GAN）培训的方法。但是，这些方法对合成孔径雷达（SAR）图像的有效性有限，因为它们不考虑SAR的独特电磁散射特性。为了解决这个问题，我们提出了一种称为$ \ phi $ gan的物理风格的正则化方法，该方法将SAR的理想点散射中心（PSC）模型与两个物理一致性损失结合在一起。 PSC模型使用物理参数近似SAR目标，以确保$ \ phi $ -gan生成与实际物理属性一致的SAR图像，同时通过关注基于PSC的决策提示来防止歧视器过度适应。为了将PSC模型嵌入GAN中以进行端到端训练，我们引入了一个由物理启发的神经模块，能够有效地估算SAR目标的物理参数。该模块保留了物理模型的可解释性，可以使用有限的数据进行培训。我们提出了两个物理损失功能：一个用于发电机，引导其以与真实参数一致的物理参数产生SAR图像，一个用于歧视者，通过根据PSC属性来确定决策来增强其鲁棒性。我们在几种有条件的GAN（CGAN）模型中评估了$ \ phi $ gan，在三个SAR图像数据集的数据筛分方案中演示了最先进的性能。

Title: Exploring Simple Siamese Network for High-Resolution Video Quality Assessment

• Authors: Guotao Shen, Ziheng Yan, Xin Jin, Longhai Wu, Jie Chen, Ilhyun Cho, Cheul-Hee Hahm
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02330
• Pdf URL: https://arxiv.org/pdf/2503.02330
• Copy Paste: [[2503.02330]] Exploring Simple Siamese Network for High-Resolution Video Quality Assessment(https://arxiv.org/abs/2503.02330)
• Keywords: quality assessment
• Abstract: In the research of video quality assessment (VQA), two-branch network has emerged as a promising solution. It decouples VQA with separate technical and aesthetic branches to measure the perception of low-level distortions and high-level semantics respectively. However, we argue that while technical and aesthetic perspectives are complementary, the technical perspective itself should be measured in semantic-aware manner. We hypothesize that existing technical branch struggles to perceive the semantics of high-resolution videos, as it is trained on local mini-patches sampled from videos. This issue can be hidden by apparently good results on low-resolution videos, but indeed becomes critical for high-resolution VQA. This work introduces SiamVQA, a simple but effective Siamese network for highre-solution VQA. SiamVQA shares weights between technical and aesthetic branches, enhancing the semantic perception ability of technical branch to facilitate technical-quality representation learning. Furthermore, it integrates a dual cross-attention layer for fusing technical and aesthetic features. SiamVQA achieves state-of-the-art accuracy on high-resolution benchmarks, and competitive results on lower-resolution benchmarks. Codes will be available at: this https URL
• 摘要：在视频质量评估（VQA）的研究中，两个分支网络已成为有前途的解决方案。它分别用独立的技术和美学分支将VQA分别衡量，分别衡量了低水平扭曲和高级语义的感知。但是，我们认为，尽管技术和审美观点是互补的，但技术观点本身应以语义意识的方式衡量。我们假设现有的技术分支在努力感知高分辨率视频的语义，因为它经过了从视频中采样的本地迷你绘图进行培训。在低分辨率视频中，显然可以通过很好的结果来掩盖这个问题，但对于高分辨率VQA来说确实至关重要。这项工作介绍了Siamvqa，这是一个简单但有效的暹罗网络，用于高温VQA。 Siamvqa分享了技术和美学分支之间的权重，从而增强了技术分支的语义感知能力，以促进技术质量的表示学习。此外，它集成了双重跨注意层，以融合技术和美学特征。 SiamVQA在高分辨率基准测试中实现了最先进的准确性，并在低分辨率基准上实现了竞争成果。代码将可用：此HTTPS URL

Title: GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoning

• Authors: Zhun Mou, Bin Xia, Zhengchao Huang, Wenming Yang, Jiaya Jia
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02341
• Pdf URL: https://arxiv.org/pdf/2503.02341
• Copy Paste: [[2503.02341]] GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoning(https://arxiv.org/abs/2503.02341)
• Keywords: generation
• Abstract: Recent great advances in video generation models have demonstrated their potential to produce high-quality videos, bringing challenges to effective evaluation. Unlike human evaluation, existing automated evaluation metrics lack high-level semantic understanding and reasoning capabilities for video, thus making them infeasible and unexplainable. To fill this gap, we curate GRADEO-Instruct, a multi-dimensional T2V evaluation instruction tuning dataset, including 3.3k videos from over 10 existing video generation models and multi-step reasoning assessments converted by 16k human annotations. We then introduce GRADEO, one of the first specifically designed video evaluation models, which grades AI-generated videos for explainable scores and assessments through multi-step reasoning. Experiments show that our method aligns better with human evaluations than existing methods. Furthermore, our benchmarking reveals that current video generation models struggle to produce content that aligns with human reasoning and complex real-world scenarios. The models, datasets, and codes will be released soon.
• 摘要：视频生成模型的最新进展证明了它们产生高质量视频的潜力，为有效评估带来了挑战。与人类评估不同，现有的自动化评估指标缺乏视频的高级语义理解和推理能力，因此使其无法理解和无法解释。为了填补这一差距，我们策划了级别Infruct，这是一种多维T2V评估指令调整数据集，包括来自10多种现有视频生成模型的3.3k视频和通过16K人类注释转换的多步推理评估。然后，我们介绍了第一个专门设计的视频评​​估模型之一Gradeo，该模型是AI生成的视频，以通过多步推理来解释分数和评估。实验表明，我们的方法与人类评估比现有方法更好。此外，我们的基准测试表明，当前的视频生成模型难以生成与人类推理和复杂的现实世界相吻合的内容。模型，数据集和代码将很快发布。

Title: Teaching Metric Distance to Autoregressive Multimodal Foundational Models

• Authors: Jiwan Chung, Saejin Kim, Yongrae Jo, Jaewoo Park, Dongjun Min, Youngjae Yu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02379
• Pdf URL: https://arxiv.org/pdf/2503.02379
• Copy Paste: [[2503.02379]] Teaching Metric Distance to Autoregressive Multimodal Foundational Models(https://arxiv.org/abs/2503.02379)
• Keywords: generation, generative
• Abstract: As large language models expand beyond natural language to domains such as mathematics, multimodal understanding, and embodied agents, tokens increasingly reflect metric relationships rather than purely linguistic meaning. We introduce DIST2Loss, a distance-aware framework designed to train autoregressive discrete models by leveraging predefined distance relationships among output tokens. At its core, DIST2Loss transforms continuous exponential family distributions derived from inherent distance metrics into discrete, categorical optimization targets compatible with the models' architectures. This approach enables the models to learn and preserve meaningful distance relationships during token generation while maintaining compatibility with existing architectures. Empirical evaluations show consistent performance gains in diverse multimodal applications, including visual grounding, robotic manipulation, generative reward modeling, and image generation using vector-quantized features. These improvements are pronounced in cases of limited training data, highlighting DIST2Loss's effectiveness in resource-constrained settings.
• 摘要：随着大型语言模型将自然语言超越自然语言扩展到数学，多模式理解和体现的代理等领域，令牌越来越反映公制关系，而不是纯粹的语言意义。我们介绍了Dist2Loss，这是一个远距离感知的框架，旨在通过利用输出令牌之间的预定距离关系来训练自回旋离散模型。 Dist2Loss以固有距离指标得出的连续指数家庭分布将其转换为离散的，分类优化目标与模型架构兼容。这种方法使模型能够在代币生成过程中学习并保留有意义的距离关系，同时保持与现有体系结构的兼容性。经验评估表明，使用矢量定量功能，包括视觉接地，机器人操纵，生成奖励建模以及图像产生的一致性增长。在培训数据有限的情况下，这些改进是明显的，强调了Dist2Loss在资源受限设置中的有效性。

Title: Exploring Model Quantization in GenAI-based Image Inpainting and Detection of Arable Plants

• Authors: Sourav Modak, Ahmet Oğuz Saltık, Anthony Stein
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02420
• Pdf URL: https://arxiv.org/pdf/2503.02420
• Copy Paste: [[2503.02420]] Exploring Model Quantization in GenAI-based Image Inpainting and Detection of Arable Plants(https://arxiv.org/abs/2503.02420)
• Keywords: generative
• Abstract: Deep learning-based weed control systems often suffer from limited training data diversity and constrained on-board computation, impacting their real-world performance. To overcome these challenges, we propose a framework that leverages Stable Diffusion-based inpainting to augment training data progressively in 10% increments -- up to an additional 200%, thus enhancing both the volume and diversity of samples. Our approach is evaluated on two state-of-the-art object detection models, YOLO11(l) and RT-DETR(l), using the mAP50 metric to assess detection performance. We explore quantization strategies (FP16 and INT8) for both the generative inpainting and detection models to strike a balance between inference speed and accuracy. Deployment of the downstream models on the Jetson Orin Nano demonstrates the practical viability of our framework in resource-constrained environments, ultimately improving detection accuracy and computational efficiency in intelligent weed management systems.
• 摘要：基于深度学习的杂草控制系统通常会遭受有限的培训数据多样性，并限制了车载计算，从而影响了他们的现实性能。为了克服这些挑战，我们提出了一个框架，该框架利用稳定的基于扩散的镶嵌物以增加10％的增量增强培训数据 - 额外增长了200％，从而增强了样品的数量和多样性。使用MAP50公制来评估检测性能，对我们的两个最先进的对象检测模型Yolo11（L）和RT-Det（L）评估了我们的方法。我们探索了量化策略（FP16和INT8），以供生成介质和检测模型，以在推理速度和准确性之间取得平衡。 Jetson Orin Nano上下游模型的部署表明了我们在资源受限环境中框架的实际生存能力，最终提高了智能杂草管理系统中的检测准确性和计算效率。

Title: A Transformer-Based Framework for Greek Sign Language Production using Extended Skeletal Motion Representations

• Authors: Chrysa Pratikaki, Panagiotis Filntisis, Athanasios Katsamanis, Anastasios Roussos, Petros Maragos
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02421
• Pdf URL: https://arxiv.org/pdf/2503.02421
• Copy Paste: [[2503.02421]] A Transformer-Based Framework for Greek Sign Language Production using Extended Skeletal Motion Representations(https://arxiv.org/abs/2503.02421)
• Keywords: generation
• Abstract: Sign Languages are the primary form of communication for Deaf communities across the world. To break the communication barriers between the Deaf and Hard-of-Hearing and the hearing communities, it is imperative to build systems capable of translating the spoken language into sign language and vice versa. Building on insights from previous research, we propose a deep learning model for Sign Language Production (SLP), which to our knowledge is the first attempt on Greek SLP. We tackle this task by utilizing a transformer-based architecture that enables the translation from text input to human pose keypoints, and the opposite. We evaluate the effectiveness of the proposed pipeline on the Greek SL dataset Elementary23, through a series of comparative analyses and ablation studies. Our pipeline's components, which include data-driven gloss generation, training through video to text translation and a scheduling algorithm for teacher forcing - auto-regressive decoding seem to actively enhance the quality of produced SL videos.
• 摘要：符号语言是全世界聋人社区的交流的主要形式。为了打破聋人与听力障碍与听力社区之间的沟通障碍，必须建立能够将口语转化为手语的系统，反之亦然。在先前研究的见解的基础上，我们提出了一种对手语生产（SLP）的深度学习模型，据我们所知，这是希腊SLP的首次尝试。我们通过利用基于变压器的体系结构来解决这项任务，该体系结构可以从文本输入到人姿势关键点的翻译，而相反。我们通过一系列比较分析和消融研究评估了提出的管道对希腊SL数据集基本23的有效性。我们的管道的组件包括数据驱动的光泽生成，通过视频到文本翻译的培训以及用于教师强迫的日程安排算法 - 自动回归解码似乎可以积极提高生产的SL视频的质量。

Title: BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modelling

• Authors: Hao Li, Yu-Hao Huang, Chang Xu, Viktor Schlegel, Ren-He Jiang, Riza Batista-Navarro, Goran Nenadic, Jiang Bian
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.02445
• Pdf URL: https://arxiv.org/pdf/2503.02445
• Copy Paste: [[2503.02445]] BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modelling(https://arxiv.org/abs/2503.02445)
• Keywords: generation
• Abstract: Time-series Generation (TSG) is a prominent research area with broad applications in simulations, data augmentation, and counterfactual analysis. While existing methods have shown promise in unconditional single-domain TSG, real-world applications demand for cross-domain approaches capable of controlled generation tailored to domain-specific constraints and instance-level requirements. In this paper, we argue that text can provide semantic insights, domain information and instance-specific temporal patterns, to guide and improve TSG. We introduce ``Text-Controlled TSG'', a task focused on generating realistic time series by incorporating textual descriptions. To address data scarcity in this setting, we propose a novel LLM-based Multi-Agent framework that synthesizes diverse, realistic text-to-TS datasets. Furthermore, we introduce BRIDGE, a hybrid text-controlled TSG framework that integrates semantic prototypes with text description for supporting domain-level guidance. This approach achieves state-of-the-art generation fidelity on 11 of 12 datasets, and improves controllability by 12.52% on MSE and 6.34% MAE compared to no text input generation, highlighting its potential for generating tailored time-series data.
• 摘要：时间序列产生（TSG）是一个著名的研究领域，在模拟，数据增强和反事实分析中进行了广泛的应用。尽管现有方法在无条件的单域TSG中显示出了承诺，但现实世界的应用程序需求对能够为域特异性约束和实例级别的要求量身定制的跨域方法。在本文中，我们认为文本可以提供语义见解，域信息和特定于实例的时间模式，以指导和改善TSG。我们介绍了``文本控制的TSG''，这是一项旨在通过合并文本描述来生成现实时间序列的任务。为了解决这种情况下的数据稀缺性，我们提出了一个基于LLM的新型多代理框架，该框架综合了多样化的现实文本对TS数据集。此外，我们介绍了Bridge，这是一种混合文本控制的TSG框架，将语义原型与文本说明集成在一起，以支持域级别的指导。这种方法可在12个数据集中的11个数据集中实现最新的一代忠诚度，而MSE的可控性则提高了12.52％，而没有文本输入生成的MAE为6.34％，强调了其生成量身定制的时间序列数据的潜力。

Title: ERetinex: Event Camera Meets Retinex Theory for Low-Light Image Enhancement

• Authors: Xuejian Guo, Zhiqiang Tian, Yuehang Wang, Siqi Li, Yu Jiang, Shaoyi Du, Yue Gao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02484
• Pdf URL: https://arxiv.org/pdf/2503.02484
• Copy Paste: [[2503.02484]] ERetinex: Event Camera Meets Retinex Theory for Low-Light Image Enhancement(https://arxiv.org/abs/2503.02484)
• Keywords: restoration
• Abstract: Low-light image enhancement aims to restore the under-exposure image captured in dark scenarios. Under such scenarios, traditional frame-based cameras may fail to capture the structure and color information due to the exposure time limitation. Event cameras are bio-inspired vision sensors that respond to pixel-wise brightness changes asynchronously. Event cameras' high dynamic range is pivotal for visual perception in extreme low-light scenarios, surpassing traditional cameras and enabling applications in challenging dark environments. In this paper, inspired by the success of the retinex theory for traditional frame-based low-light image restoration, we introduce the first methods that combine the retinex theory with event cameras and propose a novel retinex-based low-light image restoration framework named ERetinex. Among our contributions, the first is developing a new approach that leverages the high temporal resolution data from event cameras with traditional image information to estimate scene illumination accurately. This method outperforms traditional image-only techniques, especially in low-light environments, by providing more precise lighting information. Additionally, we propose an effective fusion strategy that combines the high dynamic range data from event cameras with the color information of traditional images to enhance image quality. Through this fusion, we can generate clearer and more detail-rich images, maintaining the integrity of visual information even under extreme lighting conditions. The experimental results indicate that our proposed method outperforms state-of-the-art (SOTA) methods, achieving a gain of 1.0613 dB in PSNR while reducing FLOPS by \textbf{84.28}\%.
• 摘要：低光图像增强旨在恢复黑暗场景中捕获的暴露不足图像。在这种情况下，由于暴露时间限制，基于框架的传统相机可能无法捕获结构和颜色信息。事件摄像头是受生物启发的视觉传感器，它们对像素的亮度响应异步而响应。事件摄像机的高动态范围是极端弱光场景中视觉感知的关键，超越了传统的摄像头并在挑战性的黑暗环境中启用应用。在本文中，灵感来自于Etinex理论在传统的基于框架的低光图像修复方面的成功，我们介绍了将视网膜理论与事件摄像机相结合的第一个方法，并提出了一种新型的基于Etinex的低光图像恢复框架，名为Eretinex。在我们的贡献中，首先是开发一种新方法，该方法利用传统图像信息从事件摄像机中利用高的时间分辨率数据来准确估算场景照明。通过提供更精确的照明信息，该方法的表现优于传统的仅限图像技术，尤其是在弱光环境中。此外，我们提出了一种有效的融合策略，该策略将事件摄像机的高动态范围数据与传统图像的颜色信息结合在一起，以增强图像质量。通过这种融合，我们可以生成更清晰，更细节的图像，即使在极端的照明条件下，也可以保持视觉信息的完整性。实验结果表明，我们提出的方法优于最先进的方法（SOTA）方法，在PSNR中获得了1.0613 dB的增益，而通过\ textbf {84.28} \％降低FLOPS。

Title: Deepfake Detection via Knowledge Injection

• Authors: Tonghui Li, Yuanfang Guo, Zeming Liu, Heqi Peng, Yunhong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02503
• Pdf URL: https://arxiv.org/pdf/2503.02503
• Copy Paste: [[2503.02503]] Deepfake Detection via Knowledge Injection(https://arxiv.org/abs/2503.02503)
• Keywords: generative
• Abstract: Deepfake detection technologies become vital because current generative AI models can generate realistic deepfakes, which may be utilized in malicious purposes. Existing deepfake detection methods either rely on developing classification methods to better fit the distributions of the training data, or exploiting forgery synthesis mechanisms to learn a more comprehensive forgery distribution. Unfortunately, these methods tend to overlook the essential role of real data knowledge, which limits their generalization ability in processing the unseen real and fake data. To tackle these challenges, in this paper, we propose a simple and novel approach, named Knowledge Injection based deepfake Detection (KID), by constructing a multi-task learning based knowledge injection framework, which can be easily plugged into existing ViT-based backbone models, including foundation models. Specifically, a knowledge injection module is proposed to learn and inject necessary knowledge into the backbone model, to achieve a more accurate modeling of the distributions of real and fake data. A coarse-grained forgery localization branch is constructed to learn the forgery locations in a multi-task learning manner, to enrich the learned forgery knowledge for the knowledge injection module. Two layer-wise suppression and contrast losses are proposed to emphasize the knowledge of real data in the knowledge injection module, to further balance the portions of the real and fake knowledge. Extensive experiments have demonstrated that our KID possesses excellent compatibility with different scales of Vit-based backbone models, and achieves state-of-the-art generalization performance while enhancing the training convergence speed.
• 摘要：DeepFake检测技术变得至关重要，因为当前的生成AI模型可以产生逼真的深击，这可能用于恶意目的。现有的DeepFake检测方法要么依赖于开发分类方法来更好地拟合培训数据的分布，要么利用伪造的合成机制来学习更全面的伪造分布。不幸的是，这些方法倾向于忽略真实数据知识的重要作用，这限制了其在处理未见的真实和虚假数据中的概括能力。为了应对这些挑战，在本文中，我们提出了一种简单而新颖的方法，即通过构建基于多任务的基于多任务的知识注入框架，称为基于知识注入的深层摄影检测（KID），可以轻松地将其插入现有的基于VIT的骨干模型，包括基础模型。具体而言，提出了一个知识注入模块来学习和注入必要的知识，以实现对真实数据和伪造数据的分布进行更准确的建模。建立了一个粗粒的伪造定位分支，以多任务学习方式学习伪造地点，以丰富知识注入模块的学习伪造知识。提出了两层抑制和对比度损失，以强调知识注入模块中真实数据的知识，以进一步平衡真实和假知识的部分。广泛的实验表明，我们的孩子与不同尺度的基于VIT的骨干模型具有出色的兼容性，并在提高训练融合速度的同时，达到了最新的概括性能。

Title: Q&C: When Quantization Meets Cache in Efficient Image Generation

• Authors: Xin Ding, Xin Li, Haotong Qin, Zhibo Chen
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.02508
• Pdf URL: https://arxiv.org/pdf/2503.02508
• Copy Paste: [[2503.02508]] Q&C: When Quantization Meets Cache in Efficient Image Generation(https://arxiv.org/abs/2503.02508)
• Keywords: generation
• Abstract: Quantization and cache mechanisms are typically applied individually for efficient Diffusion Transformers (DiTs), each demonstrating notable potential for acceleration. However, the promoting effect of combining the two mechanisms on efficient generation remains under-explored. Through empirical investigation, we find that the combination of quantization and cache mechanisms for DiT is not straightforward, and two key challenges lead to severe catastrophic performance degradation: (i) the sample efficacy of calibration datasets in post-training quantization (PTQ) is significantly eliminated by cache operation; (ii) the combination of the above mechanisms introduces more severe exposure bias within sampling distribution, resulting in amplified error accumulation in the image generation process. In this work, we take advantage of these two acceleration mechanisms and propose a hybrid acceleration method by tackling the above challenges, aiming to further improve the efficiency of DiTs while maintaining excellent generation capability. Concretely, a temporal-aware parallel clustering (TAP) is designed to dynamically improve the sample selection efficacy for the calibration within PTQ for different diffusion steps. A variance compensation (VC) strategy is derived to correct the sampling distribution. It mitigates exposure bias through an adaptive correction factor generation. Extensive experiments have shown that our method has accelerated DiTs by 12.7x while preserving competitive generation capability. The code will be available at this https URL.
• 摘要：通常将量化和缓存机制单独应用于有效的扩散变压器（DIT），每种变压器都表现出显着的加速潜力。但是，将两种机制组合对有效产生的促进效应仍然不足。通过实证研究，我们发现DIT的量化和缓存机制的组合并不直接，两个关键挑战导致严重的灾难性绩效降解：（i）校准数据集在训练后量化（PTQ）中的样本疗效（PTQ）被缓存操作显着消除； （ii）上述机制的组合在抽样分布中引入了更严重的暴露偏置，从而导致图像生成过程中的放大误差积累。在这项工作中，我们利用了这两种加速机制，并通过应对上述挑战提出了混合加速方法，旨在进一步提高DIT的效率，同时保持出色的发电能力。具体而言，在不同的扩散步骤中，旨在动态地提高PTQ内校准的样品选择功效，从而动态地提高样品选择功效。得出了差异补偿（VC）策略以纠正采样分布。它通过自适应校正因子的产生来减轻暴露偏差。广泛的实验表明，我们的方法在保持竞争性生成能力的同时，加速了12.7倍。该代码将在此HTTPS URL上可用。

Title: SAGE-Amine: Generative Amine Design with Multi-Property Optimization for Efficient CO2 Capture

• Authors: Hocheol Lim, Hyein Cho, Jeonghoon Kim
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2503.02534
• Pdf URL: https://arxiv.org/pdf/2503.02534
• Copy Paste: [[2503.02534]] SAGE-Amine: Generative Amine Design with Multi-Property Optimization for Efficient CO2 Capture(https://arxiv.org/abs/2503.02534)
• Keywords: generative
• Abstract: Efficient CO2 capture is vital for mitigating climate change, with amine-based solvents being widely used due to their strong reactivity with CO2. However, optimizing key properties such as basicity, viscosity, and absorption capacity remains challenging, as traditional methods rely on labor-intensive experimentation and predefined chemical databases, limiting the exploration of novel solutions. Here, SAGE-Amine was introduced, a generative modeling approach that integrates Scoring-Assisted Generative Exploration (SAGE) with quantitative structure-property relationship models to design new amines tailored for CO2 capture. Unlike conventional virtual screening restricted to existing compounds, SAGE-Amine generates novel amines by leveraging autoregressive natural language processing models trained on amine datasets. SAGE-Amine identified known amines for CO2 capture from scratch and successfully performed single-property optimization, increasing basicity or reducing viscosity or vapor pressure. Furthermore, it facilitated multi-property optimization, simultaneously achieving high basicity with low viscosity and vapor pressure. The 10 top-ranked amines were suggested using SAGE-Amine and their thermodynamic properties were further assessed using COSMO-RS simulations, confirming their potential for CO2 capture. These results highlight the potential of generative modeling in accelerating the discovery of amine solvents and expanding the possibilities for industrial CO2 capture applications.
• 摘要：有效的二氧化碳捕获对于缓解气候变化至关重要，由于胺与二氧化碳的反应性强，基于胺的溶剂被广泛使用。但是，优化诸如碱性，粘度和吸收能力之类的关键特性仍然具有挑战性，因为传统方法依赖于劳动密集型的实验和预定义的化学数据库，从而限制了对新颖溶液的探索。在这里，引入了Sage-Amine，这是一种生成建模方法，将评分辅助的生成探索（SAGE）与定量结构 - 繁星关系模型集成在一起，以设计针对CO2捕获的新胺。与常规的虚拟筛选仅限于现有化合物不同，Sage-Amine通过利用在胺数据集中训练的自回归自然语言处理模型来生成新颖的胺。 Sage-Amine鉴定出已知的胺从头开始捕获二氧化碳，并成功执行了单特性优化，增加了碱性或降低粘度或蒸气压力。此外，它促进了多型优化，同时以低粘度和蒸气压力达到了高碱度。建议使用Sage-Amine提出10个顶级胺，并使用COSMO-RS模拟进一步评估其热力学特性，从而证实了它们的二氧化碳捕获潜力。这些结果突出了生成建模在加速发现胺溶剂并扩大工业二氧化碳捕获应用的可能性方面的潜力。

Title: RectifiedHR: Enable Efficient High-Resolution Image Generation via Energy Rectification

• Authors: Zhen Yang, Guibao Shen, Liang Hou, Mushui Liu, Luozhou Wang, Xin Tao, Pengfei Wan, Di Zhang, Ying-Cong Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02537
• Pdf URL: https://arxiv.org/pdf/2503.02537
• Copy Paste: [[2503.02537]] RectifiedHR: Enable Efficient High-Resolution Image Generation via Energy Rectification(https://arxiv.org/abs/2503.02537)
• Keywords: generation
• Abstract: Diffusion models have achieved remarkable advances in various image generation tasks. However, their performance notably declines when generating images at resolutions higher than those used during the training period. Despite the existence of numerous methods for producing high-resolution images, they either suffer from inefficiency or are hindered by complex operations. In this paper, we propose RectifiedHR, an efficient and straightforward solution for training-free high-resolution image generation. Specifically, we introduce the noise refresh strategy, which theoretically only requires a few lines of code to unlock the model's high-resolution generation ability and improve efficiency. Additionally, we first observe the phenomenon of energy decay that may cause image blurriness during the high-resolution image generation process. To address this issue, we propose an Energy Rectification strategy, where modifying the hyperparameters of the classifier-free guidance effectively improves the generation performance. Our method is entirely training-free and boasts a simple implementation logic. Through extensive comparisons with numerous baseline methods, our RectifiedHR demonstrates superior effectiveness and efficiency.
• 摘要：扩散模型在各种图像生成任务中取得了显着进步。但是，在分辨率上生成图像时，它们的性能显着下降，高于培训期间使用的图像。尽管存在许多用于产生高分辨率图像的方法，但它们要么患有效率低下，要么受到复杂操作的阻碍。在本文中，我们提出了RectifiedHR，这是一种有效且直接的解决方案，用于训练无高分辨率图像的生成。具体而言，我们介绍了噪声刷新策略，从理论上讲，该策略仅需要几行代码来解锁该模型的高分辨率生成能力并提高效率。此外，我们首先观察到能量衰减的现象，这些现象可能在高分辨率图像生成过程中导致图像模糊。为了解决这个问题，我们提出了一种能量矫正策略，其中修改无分类器指导的超参数可以有效地改善生成性能。我们的方法完全无训练，并具有简单的实现逻辑。通过与多种基线方法的广泛比较，我们的整流HHR表现出较高的有效性和效率。

Title: PVTree: Realistic and Controllable Palm Vein Generation for Recognition Tasks

• Authors: Sheng Shang, Chenglong Zhao, Ruixin Zhang, Jianlong Jin, Jingyun Zhang, Rizen Guo, Shouhong Ding, Yunsheng Wu, Yang Zhao, Wei Jia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02547
• Pdf URL: https://arxiv.org/pdf/2503.02547
• Copy Paste: [[2503.02547]] PVTree: Realistic and Controllable Palm Vein Generation for Recognition Tasks(https://arxiv.org/abs/2503.02547)
• Keywords: generation, generative
• Abstract: Palm vein recognition is an emerging biometric technology that offers enhanced security and privacy. However, acquiring sufficient palm vein data for training deep learning-based recognition models is challenging due to the high costs of data collection and privacy protection constraints. This has led to a growing interest in generating pseudo-palm vein data using generative models. Existing methods, however, often produce unrealistic palm vein patterns or struggle with controlling identity and style attributes. To address these issues, we propose a novel palm vein generation framework named PVTree. First, the palm vein identity is defined by a complex and authentic 3D palm vascular tree, created using an improved Constrained Constructive Optimization (CCO) algorithm. Second, palm vein patterns of the same identity are generated by projecting the same 3D vascular tree into 2D images from different views and converting them into realistic images using a generative model. As a result, PVTree satisfies the need for both identity consistency and intra-class diversity. Extensive experiments conducted on several publicly available datasets demonstrate that our proposed palm vein generation method surpasses existing methods and achieves a higher TAR@FAR=1e-4 under the 1:1 Open-set protocol. To the best of our knowledge, this is the first time that the performance of a recognition model trained on synthetic palm vein data exceeds that of the recognition model trained on real data, which indicates that palm vein image generation research has a promising future.
• 摘要：棕榈静脉识别是一种新兴的生物识别技术，可提供增强的安全性和隐私性。但是，由于数据收集和隐私保护限制的高成本，获取足够的棕榈静脉数据进行培训基于深度学习的识别模型，这是具有挑战性的。这引起了人们对使用生成模型生成伪palm静脉数据的兴趣。但是，现有方法通常会产生不切实际的棕榈静脉模式，或者在控制身份和样式属性方面挣扎。为了解决这些问题，我们提出了一个新颖的棕榈静脉生成框架，名为Pvtree。首先，棕榈静脉的身份是由复杂而真实的3D棕榈血管树定义的，该树使用改进的约束构造优化（CCO）算法创建。其次，通过将相同的3D血管树投射到来自不同视图的2D图像中，并使用生成模型将它们转换为逼真的图像，从而生成了相同身份的棕榈静脉模式。结果，PVTREE满足对身份一致性和类内多样性的需求。在几个公开可用数据集上进行的广泛实验表明，我们提出的棕榈静脉生成方法超过了现有方法，并在1：1开放设定协议下实现了更高的tar@far = 1e-4。据我们所知，这是第一次在合成棕榈静脉数据上训练的识别模型的性能超过了对真实数据训练的识别模型的表现，这表明棕榈静脉图像生成研究具有有希望的未来。

Title: SPG: Improving Motion Diffusion by Smooth Perturbation Guidance

• Authors: Boseong Jeon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02577
• Pdf URL: https://arxiv.org/pdf/2503.02577
• Copy Paste: [[2503.02577]] SPG: Improving Motion Diffusion by Smooth Perturbation Guidance(https://arxiv.org/abs/2503.02577)
• Keywords: generation
• Abstract: This paper presents a test-time guidance method to improve the output quality of the human motion diffusion models without requiring additional training. To have negative guidance, Smooth Perturbation Guidance (SPG) builds a weak model by temporally smoothing the motion in the denoising steps. Compared to model-agnostic methods originating from the image generation field, SPG effectively mitigates out-of-distribution issues when perturbing motion diffusion models. In SPG guidance, the nature of motion structure remains intact. This work conducts a comprehensive analysis across distinct model architectures and tasks. Despite its extremely simple implementation and no need for additional training requirements, SPG consistently enhances motion fidelity. Project page can be found at this https URL
• 摘要：本文提出了一种测试时间指导方法，以提高人类运动扩散模型的输出质量，而无需进行额外的培训。为了有负面的指导，平滑的扰动指导（SPG）通过在降级步骤中暂时平滑运动来构建弱模型。与源自图像生成场的模型 - 不足方法相比，SPG在扰动运动扩散模型时会有效地减轻分布外的问题。在SPG指导中，运动结构的性质保持完整。这项工作对不同的模型架构和任务进行了全面的分析。尽管实施极为简单，并且不需要额外的培训要求，但SPG仍始终提高运动保真度。可以在此HTTPS URL上找到项目页面

Title: TS-CGNet: Temporal-Spatial Fusion Meets Centerline-Guided Diffusion for BEV Mapping

• Authors: Xinying Hong, Siyu Li, Kang Zeng, Hao Shi, Bomin Peng, Kailun Yang, Zhiyong Li
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.02578
• Pdf URL: https://arxiv.org/pdf/2503.02578
• Copy Paste: [[2503.02578]] TS-CGNet: Temporal-Spatial Fusion Meets Centerline-Guided Diffusion for BEV Mapping(https://arxiv.org/abs/2503.02578)
• Keywords: generation
• Abstract: Bird's Eye View (BEV) perception technology is crucial for autonomous driving, as it generates top-down 2D maps for environment perception, navigation, and decision-making. Nevertheless, the majority of current BEV map generation studies focusing on visual map generation lack depth-aware reasoning capabilities. They exhibit limited efficacy in managing occlusions and handling complex environments, with a notable decline in perceptual performance under adverse weather conditions or low-light scenarios. Therefore, this paper proposes TS-CGNet, which leverages Temporal-Spatial fusion with Centerline-Guided diffusion. This visual framework, grounded in prior knowledge, is designed for integration into any existing network for building BEV maps. Specifically, this framework is decoupled into three parts: Local mapping system involves the initial generation of semantic maps using purely visual information; The Temporal-Spatial Aligner Module (TSAM) integrates historical information into mapping generation by applying transformation matrices; The Centerline-Guided Diffusion Model (CGDM) is a prediction module based on the diffusion model. CGDM incorporates centerline information through spatial-attention mechanisms to enhance semantic segmentation reconstruction. We construct BEV semantic segmentation maps by our methods on the public nuScenes and the robustness benchmarks under various corruptions. Our method improves 1.90%, 1.73%, and 2.87% for perceived ranges of 60x30m, 120x60m, and 240x60m in the task of BEV HD mapping. TS-CGNet attains an improvement of 1.92% for perceived ranges of 100x100m in the task of BEV semantic mapping. Moreover, TS-CGNet achieves an average improvement of 2.92% in detection accuracy under varying weather conditions and sensor interferences in the perception range of 240x60m. The source code will be publicly available at this https URL.
• 摘要：Bird's Eye View（BEV）感知技术对于自动驾驶至关重要，因为它为环境感知，导航和决策生成自上而下的2D地图。然而，重点关注视觉地图生成的当前BEV地图生成研究缺乏深度感知的推理能力。它们在管理阻塞和处理复杂环境方面表现出有限的功效，在不利天气条件或弱光场景下，感知性能的下降显着下降。因此，本文提出了TS-CGNET，该TS-CGNET利用中心线引导的扩散来利用时间空间融合。该视觉框架以先验知识为基础，旨在集成到任何现有的网络以构建BEV地图。具体而言，该框架分为三个部分：本地映射系统涉及使用纯粹的视觉信息的初始生成语义图；时间空间对准器模块（TSAM）通过应用转换矩阵将历史信息整合到映射生成中；中心线引导的扩散模型（CGDM）是基于扩散模型的预测模块。 CGDM通过空间注意机制结合了中心线信息，以增强语义分割重建。我们通过对公共Nuscenes的方法和各种腐败下的稳健性基准来构建BEV语义分割图。在BEV HD映射的任务中，我们的方法可提高1.90％，1.73％和2.87％的60x30m，120x60m和240x60m的范围。在BEV语义映射的任务中，TS-CGNET的感知范围为100x100m的范围为1.92％。此外，在不同的天气条件和传感器干扰范围内，TS-CGNET在240x60m的感知范围内的检测准确性平均提高了2.92％。源代码将在此HTTPS URL上公开可用。

Title: MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments

• Authors: Ege Özsoy, Chantal Pellegrini, Tobias Czempiel, Felix Tristram, Kun Yuan, David Bani-Harouni, Ulrich Eck, Benjamin Busam, Matthias Keicher, Nassir Navab
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02579
• Pdf URL: https://arxiv.org/pdf/2503.02579
• Copy Paste: [[2503.02579]] MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments(https://arxiv.org/abs/2503.02579)
• Keywords: generation
• Abstract: Operating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assistance, situational awareness, and patient safety. Current datasets fall short in scale, realism and do not capture the multimodal nature of OR scenes, limiting progress in OR modeling. To this end, we introduce MM-OR, a realistic and large-scale multimodal spatiotemporal OR dataset, and the first dataset to enable multimodal scene graph generation. MM-OR captures comprehensive OR scenes containing RGB-D data, detail views, audio, speech transcripts, robotic logs, and tracking data and is annotated with panoptic segmentations, semantic scene graphs, and downstream task labels. Further, we propose MM2SG, the first multimodal large vision-language model for scene graph generation, and through extensive experiments, demonstrate its ability to effectively leverage multimodal inputs. Together, MM-OR and MM2SG establish a new benchmark for holistic OR understanding, and open the path towards multimodal scene analysis in complex, high-stakes environments. Our code, and data is available at this https URL.
• 摘要：手术室（OR）是复杂的高风险环境，需要对医务人员，工具和设备之间的相互作用进行精确了解，以增强手术援助，情境意识和患者安全。当前的数据集的规模不足，现实主义，并且不会捕获场景的多模式性质，从而限制了或建模的进度。为此，我们介绍了MM-OR，这是一种现实且大规模的多模式时空或数据集，也是第一个启用多模式场景图生成的数据集。 MM-OR捕获包含RGB-D数据，详细视图，音频，语音记录，机器人日志和跟踪数据的全面或场景，并用全盘细分，语义场景图和下游任务标签进行注释。此外，我们提出了MM2SG，这是场景图生成的第一个多模式大型视觉模型，并通过广泛的实验证明了其有效利用多模式输入的能力。 MM-OR和MM2SG共同为整体或理解建立了新的基准，并在复杂的高风险环境中打开了多模式场景分析的道路。我们的代码和数据可在此HTTPS URL上找到。

Title: StageDesigner: Artistic Stage Generation for Scenography via Theater Scripts

• Authors: Zhaoxing Gan, Mengtian Li, Ruhua Chen, Zhongxia Ji, Sichen Guo, Huanling Hu, Guangnan Ye, Zuo Hu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02595
• Pdf URL: https://arxiv.org/pdf/2503.02595
• Copy Paste: [[2503.02595]] StageDesigner: Artistic Stage Generation for Scenography via Theater Scripts(https://arxiv.org/abs/2503.02595)
• Keywords: generation
• Abstract: In this work, we introduce StageDesigner, the first comprehensive framework for artistic stage generation using large language models combined with layout-controlled diffusion models. Given the professional requirements of stage scenography, StageDesigner simulates the workflows of seasoned artists to generate immersive 3D stage scenes. Specifically, our approach is divided into three primary modules: Script Analysis, which extracts thematic and spatial cues from input scripts; Foreground Generation, which constructs and arranges essential 3D objects; and Background Generation, which produces a harmonious background aligned with the narrative atmosphere and maintains spatial coherence by managing occlusions between foreground and background elements. Furthermore, we introduce the StagePro-V1 dataset, a dedicated dataset with 276 unique stage scenes spanning different historical styles and annotated with scripts, images, and detailed 3D layouts, specifically tailored for this task. Finally, evaluations using both standard and newly proposed metrics, along with extensive user studies, demonstrate the effectiveness of StageDesigner. Project can be found at: this https URL
• 摘要：在这项工作中，我们介绍了StaideSigner，这是使用大型语言模型与布局控制的扩散模型结合使用的大型语言模型的第一个全面框架。鉴于舞台场景的专业要求，上演者模拟了经验丰富的艺术家的工作流程，以产生沉浸式的3D舞台场景。具体而言，我们的方法分为三个主要模块：脚本分析，从输入脚本中提取主题和空间提示；前景一代，构建和安排必需的3D对象；背景生成，产生与叙事氛围保持一致的和谐背景，并通过管理前景和背景元素之间的阻塞来保持空间连贯性。此外，我们介绍了StagePro-V1数据集，这是一个专用数据集，具有276个独特的舞台场景，涵盖了不同的历史样式，并带有脚本，图像和详细的3D布局注释，专门针对此任务量身定制。最后，使用标准和新提出的指标的评估以及大量的用户研究表明了阶级签名者的有效性。可以在以下网址找到项目

Title: Quantum Geometry insights in Deep Learning

• Authors: Noémie C. Combe
• Subjects: cs.LG, cs.IT, math.DG
• Abstract URL: https://arxiv.org/abs/2503.02655
• Pdf URL: https://arxiv.org/pdf/2503.02655
• Copy Paste: [[2503.02655]] Quantum Geometry insights in Deep Learning(https://arxiv.org/abs/2503.02655)
• Keywords: generative
• Abstract: In this paper, we explore the fundamental role of the Monge-Ampère equation in deep learning, particularly in the context of Boltzmann machines and energy-based models. We first review the structure of Boltzmann learning and its relation to free energy minimization. We then establish a connection between optimal transport theory and deep learning, demonstrating how the Monge-Ampère equation governs probability transformations in generative models. Additionally, we provide insights from quantum geometry, showing that the space of covariance matrices arising in the learning process coincides with the Connes-Araki-Haagerup (CAH) cone in von Neumann algebra theory. Furthermore, we introduce an alternative approach based on renormalization group (RG) flow, which, while distinct from the optimal transport perspective, reveals another manifestation of the Monge-Ampère domain in learning dynamics. This dual perspective offers a deeper mathematical understanding of hierarchical feature learning, bridging concepts from statistical mechanics, quantum geometry, and deep learning theory.
• 摘要：在本文中，我们探讨了Monge-Ampère方程在深度学习中的基本作用，尤其是在Boltzmann机器和基于能量的模型的背景下。我们首先回顾了Boltzmann学习的结构及其与自由能量最小化的关系。然后，我们在最佳运输理论和深度学习之间建立了联系，证明了Monge-ampère方程如何控制生成模型中的概率转换。此外，我们还提供了量子几何形状的见解，表明在学习过程中产生的协方差矩阵空间与von Neumann代数理论中的Connes-Araki-Haagerup（CAH）锥相吻合。此外，我们引入了一种基于重新归一化组（RG）流的替代方法，虽然从最佳传输角度不同，但它揭示了Monge-Ampère领域在学习动力学中的另一种体现。这种双重视角为分层特征学习，统计力学，量子几何学和深度学习理论提供了更深入的数学理解。

Title: Creating Sorted Grid Layouts with Gradient-based Optimization

• Authors: Kai Uwe Barthel, Florian Tim Barthel, Peter Eisert, Nico Hezel, Konstantin Schall
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02730
• Pdf URL: https://arxiv.org/pdf/2503.02730
• Copy Paste: [[2503.02730]] Creating Sorted Grid Layouts with Gradient-based Optimization(https://arxiv.org/abs/2503.02730)
• Keywords: generation
• Abstract: Visually sorted grid layouts provide an efficient method for organizing high-dimensional vectors in two-dimensional space by aligning spatial proximity with similarity relationships. This approach facilitates the effective sorting of diverse elements ranging from data points to images, and enables the simultaneous visualization of a significant number of elements. However, sorting data on two-dimensional grids is a challenge due to its high complexity. Even for a small 8-by-8 grid with 64 elements, the number of possible arrangements exceeds $1.3 \cdot 10^{89}$ - more than the number of atoms in the universe - making brute-force solutions impractical. Although various methods have been proposed to address the challenge of determining sorted grid layouts, none have investigated the potential of gradient-based optimization. In this paper, we present a novel method for grid-based sorting that exploits gradient optimization for the first time. We introduce a novel loss function that balances two opposing goals: ensuring the generation of a "valid" permutation matrix, and optimizing the arrangement on the grid to reflect the similarity between vectors, inspired by metrics that assess the quality of sorted grids. While learning-based approaches are inherently computationally complex, our method shows promising results in generating sorted grid layouts with superior sorting quality compared to existing techniques.
• 摘要：视觉排序的网格布局提供了一种有效的方法，可以通过将空间接近度与相似性关系对齐，以在二维空间中组织高维矢量。这种方法促进了从数据点到图像的各种元素的有效分类，并可以同时可视化大量元素。但是，由于其高复杂性，对二维网格进行分类是一个挑战。即使对于具有64个元素的小8乘8个网格，可能的安排数量超过$ 1.3 \ cdot 10^{89} $ - 超过宇宙中原子的数量 - 使蛮力解决方案不切实际。尽管已经提出了各种方法来解决确定分类网格布局的挑战，但没有人研究了基于梯度的优化的潜力。在本文中，我们提出了一种用于首次利用梯度优化的基于网格的排序的新方法。我们引入了一种新颖的损失功能，可以平衡两个相反的目标：确保生成“有效”置换矩阵，并优化网格上的布置以反映向量之间的相似性，灵感来自评估排序网格质量的指标。尽管基于学习的方法本质上是计算复杂的，但与现有技术相比，我们的方法在生成具有优质排序质量的分类网格布局方面显示出令人鼓舞的结果。

Title: A Causal Framework for Aligning Image Quality Metrics and Deep Neural Network Robustness

• Authors: Nathan Drenkow, Mathias Unberath
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02797
• Pdf URL: https://arxiv.org/pdf/2503.02797
• Copy Paste: [[2503.02797]] A Causal Framework for Aligning Image Quality Metrics and Deep Neural Network Robustness(https://arxiv.org/abs/2503.02797)
• Keywords: quality assessment
• Abstract: Image quality plays an important role in the performance of deep neural networks (DNNs) and DNNs have been widely shown to exhibit sensitivity to changes in imaging conditions. Large-scale datasets often contain images under a wide range of conditions prompting a need to quantify and understand their underlying quality distribution in order to better characterize DNN performance and robustness. Aligning the sensitivities of image quality metrics and DNNs ensures that estimates of quality can act as proxies for image/dataset difficulty independent of the task models trained/evaluated on the data. Conventional image quality assessment (IQA) seeks to measure and align quality relative to human perceptual judgments, but here we seek a quality measure that is not only sensitive to imaging conditions but also well-aligned with DNN sensitivities. We first ask whether conventional IQA metrics are also informative of DNN performance. In order to answer this question, we reframe IQA from a causal perspective and examine conditions under which quality metrics are predictive of DNN performance. We show theoretically and empirically that current IQA metrics are weak predictors of DNN performance in the context of classification. We then use our causal framework to provide an alternative formulation and a new image quality metric that is more strongly correlated with DNN performance and can act as a prior on performance without training new task models. Our approach provides a means to directly estimate the quality distribution of large-scale image datasets towards characterizing the relationship between dataset composition and DNN performance.
• 摘要：图像质量在深神网络（DNN）的性能中起着重要作用，并且已广泛证明DNNs对成像条件变化具有敏感性。大规模数据集通常在各种条件下包含图像，促使需要量化和理解其潜在质量分布，以更好地表征DNN性能和鲁棒性。对齐图像质量指标和DNN的敏感性确保质量估计可以充当图像/数据集难度的代理，而与对数据训练/评估的任务模型无关。传统的图像质量评估（IQA）试图衡量相对于人类感知判断的质量和对齐质量，但是在这里，我们寻求一种质量量度，不仅对成像条件敏感，而且对DNN敏感性敏感。我们首先询问常规的IQA指标是否也可以提供DNN性能的信息。为了回答这个问题，我们从因果的角度重新构架IQA，并检查条件在哪些质量指标可以预测DNN性能。我们从理论和经验上表明当前的IQA指标是分类背景下DNN性能的弱预测指标。然后，我们使用因果框架提供替代表述和新的图像质量指标，该指标与DNN性能更加密切相关，并且可以在不训练新任务模型的情况下作为绩效的先验。我们的方法提供了一种直接估算大型图像数据集的质量分布的方法，以表征数据集组成与DNN性能之间的关系。

Title: MX-Font++: Mixture of Heterogeneous Aggregation Experts for Few-shot Font Generation

• Authors: Weihang Wang, Duolin Sun, Jielei Zhang, Longwen Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02799
• Pdf URL: https://arxiv.org/pdf/2503.02799
• Copy Paste: [[2503.02799]] MX-Font++: Mixture of Heterogeneous Aggregation Experts for Few-shot Font Generation(https://arxiv.org/abs/2503.02799)
• Keywords: generation
• Abstract: Few-shot Font Generation (FFG) aims to create new font libraries using limited reference glyphs, with crucial applications in digital accessibility and equity for low-resource languages, especially in multilingual artificial intelligence systems. Although existing methods have shown promising performance, transitioning to unseen characters in low-resource languages remains a significant challenge, especially when font glyphs vary considerably across training sets. MX-Font considers the content of a character from the perspective of a local component, employing a Mixture of Experts (MoE) approach to adaptively extract the component for better transition. However, the lack of a robust feature extractor prevents them from adequately decoupling content and style, leading to sub-optimal generation results. To alleviate these problems, we propose Heterogeneous Aggregation Experts (HAE), a powerful feature extraction expert that helps decouple content and style downstream from being able to aggregate information in channel and spatial dimensions. Additionally, we propose a novel content-style homogeneity loss to enhance the untangling. Extensive experiments on several datasets demonstrate that our MX-Font++ yields superior visual results in FFG and effectively outperforms state-of-the-art methods. Code and data are available at this https URL.
• 摘要：很少有弹药字体生成（FFG）旨在使用有限的参考字形创建新的字体库，并在数字可访问性中使用至关重要的应用程序，尤其是在多语言人工智能系统中。尽管现有的方法表现出了令人鼓舞的表现，但是在低资源语言中过渡到看不见的角色仍然是一个巨大的挑战，尤其是当字体在培训集中差异很大时。 MX-Font从本地组件的角度考虑了角色的内容，采用专家（MOE）方法的混合物（MOE）方法来适应性地提取组件以更好地过渡。但是，缺乏强大的特征提取器可阻止它们充分地脱钩的内容和样式，从而导致次优的结果。为了减轻这些问题，我们提出了异构的聚合专家（HAE），这是一位强大的功能提取专家，可帮助将内容和样式解除下游的样式，从而能够在频道和空间维度中汇总信息。此外，我们提出了一种新颖的内容式同质性损失，以增强毫无用处。在几个数据集上进行的广泛实验表明，我们的MX-FONT ++在FFG中产生了出色的视觉结果，并且有效地超过了最先进的方法。代码和数据可在此HTTPS URL上找到。

Title: RAAD-LLM: Adaptive Anomaly Detection Using LLMs and RAG Integration

• Authors: Alicia Russell-Gilbert, Sudip Mittal, Shahram Rahimi, Maria Seale, Joseph Jabour, Thomas Arnold, Joshua Church
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2503.02800
• Pdf URL: https://arxiv.org/pdf/2503.02800
• Copy Paste: [[2503.02800]] RAAD-LLM: Adaptive Anomaly Detection Using LLMs and RAG Integration(https://arxiv.org/abs/2503.02800)
• Keywords: generation
• Abstract: Anomaly detection in complex industrial environments poses unique challenges, particularly in contexts characterized by data sparsity and evolving operational conditions. Predictive maintenance (PdM) in such settings demands methodologies that are adaptive, transferable, and capable of integrating domain-specific knowledge. In this paper, we present RAAD-LLM, a novel framework for adaptive anomaly detection, leveraging large language models (LLMs) integrated with Retrieval-Augmented Generation (RAG). This approach addresses the aforementioned PdM challenges. By effectively utilizing domain-specific knowledge, RAAD-LLM enhances the detection of anomalies in time series data without requiring fine-tuning on specific datasets. The framework's adaptability mechanism enables it to adjust its understanding of normal operating conditions dynamically, thus increasing detection accuracy. We validate this methodology through a real-world application for a plastics manufacturing plant and the Skoltech Anomaly Benchmark (SKAB). Results show significant improvements over our previous model with an accuracy increase from 70.7 to 89.1 on the real-world dataset. By allowing for the enriching of input series data with semantics, RAAD-LLM incorporates multimodal capabilities that facilitate more collaborative decision-making between the model and plant operators. Overall, our findings support RAAD-LLM's ability to revolutionize anomaly detection methodologies in PdM, potentially leading to a paradigm shift in how anomaly detection is implemented across various industries.
• 摘要：复杂的工业环境中的异常检测提出了独特的挑战，尤其是在数据稀疏和不断发展的操作条件的环境中。在这种情况下，预测维护（PDM）需要适应性，可转移且能够整合特定领域知识的方法。在本文中，我们提出了RAAD-LLM，这是一种用于自适应异常检测的新型框架，利用与检索功能增强生成（RAG）集成的大型语言模型（LLM）。这种方法解决了上述PDM挑战。通过有效利用特定领域的知识，RAAD-LLM可以增强时间序列数据中异常的检测，而无需在特定数据集上进行微调。该框架的适应性机制使其能够动态调整对正常工作条件的理解，从而提高检测准确性。我们通过用于塑料制造工厂和Skoltech异常基准（SKAB）的现实世界应用来验证这种方法。结果表明，与我们以前的模型相比，现实世界数据集的准确性从70.7增加到89.1。通过允许使用语义的输入系列数据丰富，RAAD-LLM结合了多模式功能，可促进模型和工厂运营商之间更加协作的决策。总体而言，我们的发现支持RAAD-LLM在PDM中彻底改变异常检测方法的能力，这有可能导致各个行业实施异常检测的范式转移。

Title: Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts

• Authors: Marta Skreta, Tara Akhound-Sadegh, Viktor Ohanesian, Roberto Bondesan, Alán Aspuru-Guzik, Arnaud Doucet, Rob Brekelmans, Alexander Tong, Kirill Neklyudov
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02819
• Pdf URL: https://arxiv.org/pdf/2503.02819
• Copy Paste: [[2503.02819]] Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts(https://arxiv.org/abs/2503.02819)
• Keywords: generation, generative
• Abstract: While score-based generative models are the model of choice across diverse domains, there are limited tools available for controlling inference-time behavior in a principled manner, e.g. for composing multiple pretrained models. Existing classifier-free guidance methods use a simple heuristic to mix conditional and unconditional scores to approximately sample from conditional distributions. However, such methods do not approximate the intermediate distributions, necessitating additional 'corrector' steps. In this work, we provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models. We derive a weighted simulation scheme which we call Feynman-Kac Correctors (FKCs) based on the celebrated Feynman-Kac formula by carefully accounting for terms in the appropriate partial differential equations (PDEs). To simulate these PDEs, we propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality. We empirically demonstrate the utility of our methods by proposing amortized sampling via inference-time temperature annealing, improving multi-objective molecule generation using pretrained models, and improving classifier-free guidance for text-to-image generation. Our code is available at this https URL.
• 摘要：虽然基于得分的生成模型是跨不同领域的首选模型，但有限的工具可用于以原则性的方式控制推理时间行为，例如用于组成多个审慎的模型。现有的无分类器指导方法使用简单的启发式方法将条件和无条件得分混合在一起，以大约从条件分布中进行样本。但是，此类方法不近似中间分布，需要采取其他“校正”步骤。在这项工作中，我们提供了一种有效的原则方法，用于从一系列从基于得分的模型中得出的退火，几何平均或产品分布来取样。我们得出了一个加权模拟方案，该方案通过仔细考虑适当的部分微分方程（PDES）中的条款，根据著名的Feynman-Kac公式称为Feynman-Kac校正器（FKC）。为了模拟这些PDE，我们提出了连续的蒙特卡洛（SMC）重新采样算法，以利用推理时间缩放来提高采样质量。我们通过通过推理时间温度退火提出摊销采样，通过预审前的模型提出摊销采样来证明我们的方法的实用性，并使用预审前的模型来改善多目标分子的产生，并改善对文本对图像生成的无分类器指导。我们的代码可在此HTTPS URL上找到。

Title: Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024

• Authors: Nuria Alina Chandra, Ryan Murtfeldt, Lin Qiu, Arnab Karmakar, Hannah Lee, Emmanuel Tanumihardja, Kevin Farhat, Ben Caffee, Sejin Paik, Changyeon Lee, Jongwook Choi, Aerin Kim, Oren Etzioni
• Subjects: cs.CV, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.02857
• Pdf URL: https://arxiv.org/pdf/2503.02857
• Copy Paste: [[2503.02857]] Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024(https://arxiv.org/abs/2503.02857)
• Keywords: generative
• Abstract: In the age of increasingly realistic generative AI, robust deepfake detection is essential for mitigating fraud and disinformation. While many deepfake detectors report high accuracy on academic datasets, we show that these academic benchmarks are out of date and not representative of recent deepfakes. We introduce Deepfake-Eval-2024, a new deepfake detection benchmark consisting of in-the-wild deepfakes collected from social media and deepfake detection platform users in 2024. Deepfake-Eval-2024 consists of 44 hours of videos, 56.5 hours of audio, and 1,975 images, encompassing the latest manipulation technologies. The benchmark contains diverse media content from 88 different websites in 52 different languages. We find that the performance of open-source state-of-the-art deepfake detection models drops precipitously when evaluated on Deepfake-Eval-2024, with AUC decreasing by 50% for video, 48% for audio, and 45% for image models compared to previous benchmarks. We also evaluate commercial deepfake detection models and models finetuned on Deepfake-Eval-2024, and find that they have superior performance to off-the-shelf open-source models, but they do not yet reach the accuracy of human deepfake forensic analysts. The dataset is available at this https URL.
• 摘要：在越来越现实的生成AI时代，鲁棒的深泡检测对于缓解欺诈和虚假信息至关重要。尽管许多DeepFake探测器都报告了学术数据集的高精度，但我们表明这些学术基准已过时，并且不代表最近的DeepFakes。我们介绍了DeepFake-eval-2024，这是一种新的DeepFake检测基准，该基准由2024年的社交媒体和Deepfake检测平台用户收集的野外深色组成。2024年的DeepFake-Eval-2024由44小时的视频组成，56.5小时的音频和1,975张图像，以及1,975张图像，最新的Manipaumperss技术。该基准包含来自52种不同语言的88个不同网站的各种媒体内容。我们发现，当在DeepFake-Eval-2024上评估开源最新的DeepFake检测模型的性能会急剧下降，视频的AUC降低了50％，音频48％，与以前的基准分析相比，图像模型的表现为45％。我们还评估了在DeepFake-eval-2024上进行的商业DeepFake检测模型和模型，并发现它们具有优于现成的开源模型的性能，但它们尚未达到人类深层法医分析师的准确性。该数据集可在此HTTPS URL上找到。

Title: ARINAR: Bi-Level Autoregressive Feature-by-Feature Generative Models

• Authors: Qinyu Zhao, Stephen Gould, Liang Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02883
• Pdf URL: https://arxiv.org/pdf/2503.02883
• Copy Paste: [[2503.02883]] ARINAR: Bi-Level Autoregressive Feature-by-Feature Generative Models(https://arxiv.org/abs/2503.02883)
• Keywords: generation, generative
• Abstract: Existing autoregressive (AR) image generative models use a token-by-token generation schema. That is, they predict a per-token probability distribution and sample the next token from that distribution. The main challenge is how to model the complex distribution of high-dimensional tokens. Previous methods either are too simplistic to fit the distribution or result in slow generation speed. Instead of fitting the distribution of the whole tokens, we explore using a AR model to generate each token in a feature-by-feature way, i.e., taking the generated features as input and generating the next feature. Based on that, we propose ARINAR (AR-in-AR), a bi-level AR model. The outer AR layer take previous tokens as input, predicts a condition vector z for the next token. The inner layer, conditional on z, generates features of the next token autoregressively. In this way, the inner layer only needs to model the distribution of a single feature, for example, using a simple Gaussian Mixture Model. On the ImageNet 256x256 image generation task, ARINAR-B with 213M parameters achieves an FID of 2.75, which is comparable to the state-of-the-art MAR-B model (FID=2.31), while five times faster than the latter.
• 摘要：现有的自动回归（AR）图像生成模型使用令牌逐个代码的模式。也就是说，他们预测了均概率分布，并从该分布中进行了接下来的令牌。主要的挑战是如何对高维令牌的复杂分布进行建模。先前的方法太简单了，无法适应分布或导致缓慢的生成速度。我们没有拟合整个代币的分布，而是使用AR模型探索以逐项功能的方式生成每个令牌，即以生成的功能为输入并生成下一个功能。基于此，我们提出了双层AR模型Arinar（AR-In-AR）。外AR层以前代币为输入，可以预测下一个令牌的条件向量z。内层（条件在Z上）会产生接下来的代币自动加压的特征。这样，内层只需要模拟单个特征的分布，例如使用简单的高斯混合模型。在Imagenet 256x256图像生成任务上，具有213m参数的Arinar-B可实现2.75的FID，与最新的MAR-B模型相当（FID = 2.31），而五倍的FID为2.75。