2025-02-25

Title: Maturity Framework for Enhancing Machine Learning Quality

• Authors: Angelantonio Castelli, Georgios Christos Chouliaras, Dmitri Goldenberg
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.15758
• Pdf URL: https://arxiv.org/pdf/2502.15758
• Copy Paste: [[2502.15758]] Maturity Framework for Enhancing Machine Learning Quality(https://arxiv.org/abs/2502.15758)
• Keywords: quality assessment
• Abstract: With the rapid integration of Machine Learning (ML) in business applications and processes, it is crucial to ensure the quality, reliability and reproducibility of such systems. We suggest a methodical approach towards ML system quality assessment and introduce a structured Maturity framework for governance of ML. We emphasize the importance of quality in ML and the need for rigorous assessment, driven by issues in ML governance and gaps in existing frameworks. Our primary contribution is a comprehensive open-sourced quality assessment method, validated with empirical evidence, accompanied by a systematic maturity framework tailored to ML systems. Drawing from applied experience at this http URL, we discuss challenges and lessons learned during large-scale adoption within organizations. The study presents empirical findings, highlighting quality improvement trends and showcasing business outcomes. The maturity framework for ML systems, aims to become a valuable resource to reshape industry standards and enable a structural approach to improve ML maturity in any organization.
• 摘要：随着机器学习（ML）在业务应用程序和流程中的快速整合，至关重要的是确保此类系统的质量，可靠性和可重复性。我们建议一种有条不紊的方法来用于ML系统质量评估，并引入了ML治理的结构化成熟框架。我们强调了质量在ML中的重要性以及对ML治理和现有框架中差距的问题驱动的严格评估的需求。我们的主要贡献是一种全面的开源质量评估方法，并用经验证据验证，并伴随着针对ML系统量身定制的系统成熟框架。从该HTTP URL的应用经验中得出，我们讨论了组织内部大规模采用期间所学到的挑战和经验教训。该研究提出了经验发现，突出了质量改进趋势并展示了业务成果。 ML系统的成熟框架旨在成为重塑行业标准的宝贵资源，并实现一种结构性方法来改善任何组织的ML成熟度。

Title: FragFM: Efficient Fragment-Based Molecular Generation via Discrete Flow Matching

• Authors: Joongwon Lee, Seonghwan Kim, Wou Youn Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15805
• Pdf URL: https://arxiv.org/pdf/2502.15805
• Copy Paste: [[2502.15805]] FragFM: Efficient Fragment-Based Molecular Generation via Discrete Flow Matching(https://arxiv.org/abs/2502.15805)
• Keywords: generation, generative
• Abstract: We introduce FragFM, a novel fragment-based discrete flow matching framework for molecular graph this http URL generates molecules at the fragment level, leveraging a coarse-to-fine autoencoding mechanism to reconstruct atom-level details. This approach reduces computational complexity while maintaining high chemical validity, enabling more efficient and scalable molecular generation. We benchmark FragFM against state-of-the-art diffusion- and flow-based models on standard molecular generation benchmarks and natural product datasets, demonstrating superior performance in validity, property control, and sampling efficiency. Notably, FragFM achieves over 99\% validity with significantly fewer sampling steps, improving scalability while preserving molecular diversity. These results highlight the potential of fragment-based generative modeling for large-scale, property-aware molecular design, paving the way for more efficient exploration of chemical space.
• 摘要：我们介绍了FragFM，这是一种基于片段的新型离散流匹配框架，用于分子图，此HTTP URL在片段级别生成分子，利用了粗到5的自动编码机制来重建原子水平的细节。这种方法可降低计算复杂性，同时保持高化学有效性，从而更有效，可扩展的分子产生。我们基于标准分子生成基准和天然产物数据集的最新扩散模型基准基准FragFM，证明了有效性，财产控制和采样效率的卓越性能。值得注意的是，FragFM的有效性超过99 \％，采样步骤明显较少，提高了可伸缩性，同时保留了分子多样性。这些结果突出了基于碎片的生成建模的潜力，用于大规模，物业感知的分子设计，为更有效探索化学空间铺平了道路。

Title: DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model

• Authors: Yi Liu, Changran Xu, Yunhao Zhou, Zeju Li, Qiang Xu
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15832
• Pdf URL: https://arxiv.org/pdf/2502.15832
• Copy Paste: [[2502.15832]] DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model(https://arxiv.org/abs/2502.15832)
• Keywords: generation
• Abstract: Recent advancements in large language models (LLMs) have shown significant potential for automating hardware description language (HDL) code generation from high-level natural language instructions. While fine-tuning has improved LLMs' performance in hardware design tasks, prior efforts have largely focused on Verilog generation, overlooking the equally critical task of Verilog understanding. Furthermore, existing models suffer from weak alignment between natural language descriptions and Verilog code, hindering the generation of high-quality, synthesizable designs. To address these issues, we present DeepRTL, a unified representation model that excels in both Verilog understanding and generation. Based on CodeT5+, DeepRTL is fine-tuned on a comprehensive dataset that aligns Verilog code with rich, multi-level natural language descriptions. We also introduce the first benchmark for Verilog understanding and take the initiative to apply embedding similarity and GPT Score to evaluate the models' understanding capabilities. These metrics capture semantic similarity more accurately than traditional methods like BLEU and ROUGE, which are limited to surface-level n-gram overlaps. By adapting curriculum learning to train DeepRTL, we enable it to significantly outperform GPT-4 in Verilog understanding tasks, while achieving performance on par with OpenAI's o1-preview model in Verilog generation tasks.
• 摘要：大型语言模型（LLMS）的最新进展显示出了从高级自然语言说明中生成的硬件说明（HDL）代码的显着潜力。虽然微调在硬件设计任务中提高了LLMS的性能，但先前的努力主要集中在Verilog生成上，忽略了Verilog理解的同样至关重要的任务。此外，现有模型在自然语言描述和Verilog代码之间的一致性较弱，从而阻碍了高质量，可综合的设计的产生。为了解决这些问题，我们提出了DeepRTL，这是一种统一的表示模型，在Verilog的理解和产生中都表现出色。基于CODET5+，DEEPRTL在一个综合数据集中进行了微调，该数据集将Verilog代码与丰富的多层次自然语言描述保持一致。我们还介绍了第一个用于Verilog理解的基准，并采用主动性应用嵌入的相似性和GPT分数来评估模型的理解能力。这些指标比Bleu和Rouge等传统方法更准确地捕获语义相似性，这些方法仅限于表面级的N-gram重叠。通过调整课程学习培训DEEPRTL，我们使其能够在Verilog理解任务中显着超过GPT-4，同时在Verilog生成任务中与OpenAI的O1-Preview模型达到绩效。

Title: Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models

• Authors: Aryan Jadon, Avinash Patil, Shashank Kumar
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15854
• Pdf URL: https://arxiv.org/pdf/2502.15854
• Copy Paste: [[2502.15854]] Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models(https://arxiv.org/abs/2502.15854)
• Keywords: generation
• Abstract: Retrieval-Augmented Generation (RAG) systems face significant performance gaps when applied to technical domains requiring precise information extraction from complex documents. Current evaluation methodologies relying on document-level metrics inadequately capture token-resolution retrieval accuracy that is critical for domain-related documents. We propose a framework combining granular evaluation metrics with synthetic data generation to optimize domain-specific RAG performance. First, we introduce token-aware metrics Precision $\Omega$ and Intersection-over-Union (IoU) that quantify context preservation versus information density trade-offs inherent in technical texts. Second, we develop a reasoning model-driven pipeline using instruction-tuned LLMs (DeepSeek-R1, DeepSeek-R1 distilled variants, and Phi-4) to generate context-anchored QA pairs with discontinuous reference spans across three specialized corpora: SEC 10-K filings (finance), biomedical abstracts (PubMed), and APT threat reports (cybersecurity). Our empirical analysis reveals critical insights: smaller chunks (less than 10 tokens) improve precision by 31-42% (IoU = 0.071 vs. baseline 0.053) at recall costs (-18%), while domain-specific embedding strategies yield 22% variance in optimal chunk sizing (5-20 tokens). The DeepSeek-R1-Distill-Qwen-32B model demonstrates superior concept alignment (+14% mean IoU over alternatives), though no configuration universally dominates. Financial texts favor larger chunks for risk factor coverage (Recall = 0.81 at size = 20), whereas cybersecurity content benefits from atomic segmentation, Precision $\Omega = 0.28$ at size = 5. Our code is available on this https URL
• 摘要：当应用于需要从复杂文档中精确的信息提取的技术领域时，检索增强的生成（RAG）系统将面临明显的性能差距。当前依赖文档级指标的当前评估方法不足地捕获了对于域相关文档至关重要的令牌分辨率检索准确性。我们提出了一个将粒状评估指标与合成数据生成的框架，以优化域特异性的破布性能。首先，我们介绍了令牌的指标精度$ \ omega $和跨工会（IOU），以量化上下文保存与信息密度密度折衷的技术文本中固有的权衡。其次，我们使用指令调整的LLM（DeepSeek-R1，DeepSeek-R1，DeepSeek-R1蒸馏变体和PHI-4）开发了推理模型驱动的管道，以生成与三个专业语料库中不连续参考跨度的上下文锚定QA对：SEC 10- K文件（财务），生物医学摘要（PubMed）和APT威胁报告（网络安全）。我们的经验分析揭示了关键见解：较小的块（少于10个令牌）以召回成本（-18％）提高了31-42％（IOU = 0.071 vs. 0.071 vs.基线0.053），而域特异性嵌入策略会产生22％的差异在最佳块尺寸（5-20​​令牌）中。 DeepSeek-R1-Distill-Qwen-32b模型表明了优越的概念对准（+14％的含义超过替代方案），尽管没有配置普遍主导。财务文本有利于较大的大块对风险因素覆盖率（召回= 0.81时= 20），而网络安全内容受原子细分受益，精度$ \ omega = 0.28 $ at Size = 5。我们的代码可在此https url上获得

Title: A Critical Assessment of Modern Generative Models' Ability to Replicate Artistic Styles

• Authors: Andrea Asperti, Franky George, Tiberio Marras, Razvan Ciprian Stricescu, Fabio Zanotti
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15856
• Pdf URL: https://arxiv.org/pdf/2502.15856
• Copy Paste: [[2502.15856]] A Critical Assessment of Modern Generative Models' Ability to Replicate Artistic Styles(https://arxiv.org/abs/2502.15856)
• Keywords: generative
• Abstract: In recent years, advancements in generative artificial intelligence have led to the development of sophisticated tools capable of mimicking diverse artistic styles, opening new possibilities for digital creativity and artistic expression. This paper presents a critical assessment of the style replication capabilities of contemporary generative models, evaluating their strengths and limitations across multiple dimensions. We examine how effectively these models reproduce traditional artistic styles while maintaining structural integrity and compositional balance in the generated images. The analysis is based on a new large dataset of AI-generated works imitating artistic styles of the past, holding potential for a wide range of applications: the "AI-pastiche" dataset. The study is supported by extensive user surveys, collecting diverse opinions on the dataset and investigation both technical and aesthetic challenges, including the ability to generate outputs that are realistic and visually convincing, the versatility of models in handling a wide range of artistic styles, and the extent to which they adhere to the content and stylistic specifications outlined in prompts. This paper aims to provide a comprehensive overview of the current state of generative tools in style replication, offering insights into their technical and artistic limitations, potential advancements in model design and training methodologies, and emerging opportunities for enhancing digital artistry, human-AI collaboration, and the broader creative landscape.
• 摘要：近年来，生成人工智能的进步导致了能够模仿多种艺术风格的复杂工具的开发，为数字创造力和艺术表达打开了新的可能性。本文对当代生成模型的样式复制能力进行了批判性评估，评估了它们在多个维度上的优势和局限性。我们研究了这些模型如何在生成图像中保持结构完整性和组成平衡的同时如何重现传统的艺术风格。该分析基于AI生成的作品的新大型数据集模仿过去的艺术风格，具有广泛应用程序的潜力：“ AI-Pastiche”数据集。这项研究得到了广泛的用户调查的支持，在数据集上收集了不同的意见，并调查了技术和美学挑战，包括产生现实且在视觉上令人信服的产量的能力，模型在处理广泛的艺术风格以及多种多样的能力，以及他们遵守提示中概述的内容和风格规格的程度。本文旨在全面概述样式复制中当前的生成工具状态，从而洞悉其技术和艺术局限性，模型设计和培训方法的潜在进步以及增强数字艺术，人类协作，人类协作，人类合作，，，以及更广泛的创意景观。

Title: RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers

• Authors: Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, Jun Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15894
• Pdf URL: https://arxiv.org/pdf/2502.15894
• Copy Paste: [[2502.15894]] RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers(https://arxiv.org/abs/2502.15894)
• Keywords: generation
• Abstract: Recent advancements in video generation have enabled models to synthesize high-quality, minute-long videos. However, generating even longer videos with temporal coherence remains a major challenge, and existing length extrapolation methods lead to temporal repetition or motion deceleration. In this work, we systematically analyze the role of frequency components in positional embeddings and identify an intrinsic frequency that primarily governs extrapolation behavior. Based on this insight, we propose RIFLEx, a minimal yet effective approach that reduces the intrinsic frequency to suppress repetition while preserving motion consistency, without requiring any additional modifications. RIFLEx offers a true free lunch--achieving high-quality $2\times$ extrapolation on state-of-the-art video diffusion transformers in a completely training-free manner. Moreover, it enhances quality and enables $3\times$ extrapolation by minimal fine-tuning without long videos. Project page and codes: \href{this https URL}{this https URL.}
• 摘要：视频生成的最新进展使模型能够合成高质量的长时间视频。但是，以时间连贯性生成更长的视频仍然是一个重大挑战，现有的外推方法导致时间重复或运动减速。在这项工作中，我们系统地分析了频率成分在位置嵌入中的作用，并确定主要控制外推行为的固有频率。基于这种见解，我们提出了Riflex，这是一种最小而有效的方法，可降低固有频率以抑制重复的同时保持运动一致性，而无需进行任何其他修改。 Riflex提供了真正的免费午餐 - 以完全无训练的方式在最先进的视频扩散变压器上获得高质量的$ 2 \ times $外推。此外，它可以提高质量，并通过最小的微调启用$ 3 \ times $ $，而无需长时间的视频。项目页面和代码：\ href {此https url} {此https url。}

Title: Directional Gradient Projection for Robust Fine-Tuning of Foundation Models

• Authors: Chengyue Huang, Junjiao Tian, Brisa Maneechotesuwan, Shivang Chopra, Zsolt Kira
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15895
• Pdf URL: https://arxiv.org/pdf/2502.15895
• Copy Paste: [[2502.15895]] Directional Gradient Projection for Robust Fine-Tuning of Foundation Models(https://arxiv.org/abs/2502.15895)
• Keywords: generative
• Abstract: Robust fine-tuning aims to adapt large foundation models to downstream tasks while preserving their robustness to distribution shifts. Existing methods primarily focus on constraining and projecting current model towards the pre-trained initialization based on the magnitudes between fine-tuned and pre-trained weights, which often require extensive hyper-parameter tuning and can sometimes result in underfitting. In this work, we propose Directional Gradient Projection (DiGraP), a novel layer-wise trainable method that incorporates directional information from gradients to bridge regularization and multi-objective optimization. Besides demonstrating our method on image classification, as another contribution we generalize this area to the multi-modal evaluation settings for robust fine-tuning. Specifically, we first bridge the uni-modal and multi-modal gap by performing analysis on Image Classification reformulated Visual Question Answering (VQA) benchmarks and further categorize ten out-of-distribution (OOD) VQA datasets by distribution shift types and degree (i.e. near versus far OOD). Experimental results show that DiGraP consistently outperforms existing baselines across Image Classfication and VQA tasks with discriminative and generative backbones, improving both in-distribution (ID) generalization and OOD robustness.
• 摘要：强大的微调旨在使大型基础模型适应下游任务，同时保留其对分配变化的稳健性。现有方法主要集中在基于微调和预训练的权重之间的幅度上，将当前模型限制为预训练的初始化，这通常需要广泛的超参数调整，有时可能导致适合。在这项工作中，我们提出了方向性梯度投影（DIGRAP），这是一种新颖的可训练方法，它结合了从梯度到桥梁正则化和多目标优化的定向信息。除了展示我们的图像分类方法之外，作为另一个贡献，我们将该领域推广到多模式评估设置以进行健壮的微调。具体而言，我们首先通过对图像分类重新重新的视觉问题答案（VQA）基准进行分析，首先桥接单模式和多模式差距，并通过分布移位类型和程度（即接近与远的OOD）。实验结果表明，DIGRAP始终在图像分类和VQA任务上均超过现有的基线，并具有歧视性和生成性骨架，从而改善了分布（ID）概括和OOD稳健性。

Title: IPAD: Inverse Prompt for AI Detection -- A Robust and Explainable LLM-Generated Text Detector

• Authors: Zheng Chen, Yushi Feng, Changyang He, Yue Deng, Hongxi Pu, Bo Li
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15902
• Pdf URL: https://arxiv.org/pdf/2502.15902
• Copy Paste: [[2502.15902]] IPAD: Inverse Prompt for AI Detection -- A Robust and Explainable LLM-Generated Text Detector(https://arxiv.org/abs/2502.15902)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinguishing between human-written and LLM-generated texts. This increases the risk of misuse and highlights the need for reliable detectors. Yet, existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios. Also, they struggle to provide explainable evidence to support their decisions, thus undermining the reliability. In light of these challenges, we propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and a Distinguisher that examines how well the input texts align with the predicted prompts. We develop and examine two versions of Distinguishers. Empirical evaluations demonstrate that both Distinguishers perform significantly better than the baseline methods, with version2 outperforming baselines by 9.73% on in-distribution data (F1-score) and 12.65% on OOD data (AUROC). Furthermore, a user study is conducted to illustrate that IPAD enhances the AI detection trustworthiness by allowing users to directly examine the decision-making evidence, which provides interpretable support for its state-of-the-art detection results.
• 摘要：大型语言模型（LLM）在文本生成中已经达到了人类水平的流利度，这使人写作和LLM生成的文本的区别变得复杂。这增加了滥用的风险，并强调了对可靠探测器的需求。然而，现有检测器对分布（OOD）数据和攻击数据表现出较差的鲁棒性，这对于实际情况至关重要。此外，他们努力提供可解释的证据来支持他们的决策，从而破坏了可靠性。鉴于这些挑战，我们提出了iPad（AI检测的逆提示），一个新颖的框架由提示逆变器组成，该逆变器识别可能生成输入文本的预测提示，以及一个差异范围，一个差异范围，一个差异范围，它可以研究输入文本与该输入文本的良好状态。预测提示。我们开发并检查了两个版本的区别者。经验评估表明，这两个差异范围的性能都明显优于基线方法，而在分布数据（F1得分）上，版本2的表现优于9.73％，而OOD数据（AUROC）的基准比基线方法要优于9.73％。此外，进行了一项用户研究，以说明iPad通过允许用户直接检查决策证据来增强AI检测可信度，该证据为其最新检测结果提供了可解释的支持。

Title: Human Motion Prediction, Reconstruction, and Generation

• Authors: Canxuan Gang, Yiran Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15956
• Pdf URL: https://arxiv.org/pdf/2502.15956
• Copy Paste: [[2502.15956]] Human Motion Prediction, Reconstruction, and Generation(https://arxiv.org/abs/2502.15956)
• Keywords: generation
• Abstract: This report reviews recent advancements in human motion prediction, reconstruction, and generation. Human motion prediction focuses on forecasting future poses and movements from historical data, addressing challenges like nonlinear dynamics, occlusions, and motion style variations. Reconstruction aims to recover accurate 3D human body movements from visual inputs, often leveraging transformer-based architectures, diffusion models, and physical consistency losses to handle noise and complex poses. Motion generation synthesizes realistic and diverse motions from action labels, textual descriptions, or environmental constraints, with applications in robotics, gaming, and virtual avatars. Additionally, text-to-motion generation and human-object interaction modeling have gained attention, enabling fine-grained and context-aware motion synthesis for augmented reality and robotics. This review highlights key methodologies, datasets, challenges, and future research directions driving progress in these fields.
• 摘要：本报告回顾了人类运动预测，重建和发电方面的最新进展。人类运动预测的重点是从历史数据中预测未来的姿势和运动，解决了非线性动力学，闭塞和运动风格变化等挑战。重建旨在从视觉输入中恢复准确的3D人体运动，通常利用基于变压器的结构，扩散模型和物理一致性损失来处理噪声和复杂的姿势。运动产生从动作标签，文本描述或环境约束中综合了现实和多样的动作，并在机器人技术，游戏和虚拟化身中的应用。此外，文本到动作的产生和人类对象的相互作用建模已引起人们的注意，从而实现了增强现实和机器人技术的细粒度和上下文感知的运动综合。这篇评论重点介绍了关键方法，数据集，挑战和未来的研究方向推动了这些领域的进步。

Title: Multi-Agent Multimodal Models for Multicultural Text to Image Generation

• Authors: Parth Bhalerao, Mounika Yalamarty, Brian Trinh, Oana Ignat
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15972
• Pdf URL: https://arxiv.org/pdf/2502.15972
• Copy Paste: [[2502.15972]] Multi-Agent Multimodal Models for Multicultural Text to Image Generation(https://arxiv.org/abs/2502.15972)
• Keywords: generation
• Abstract: Large Language Models (LLMs) demonstrate impressive performance across various multimodal tasks. However, their effectiveness in cross-cultural contexts remains limited due to the predominantly Western-centric nature of existing data and models. Meanwhile, multi-agent models have shown strong capabilities in solving complex tasks. In this paper, we evaluate the performance of LLMs in a multi-agent interaction setting for the novel task of multicultural image generation. Our key contributions are: (1) We introduce MosAIG, a Multi-Agent framework that enhances multicultural Image Generation by leveraging LLMs with distinct cultural personas; (2) We provide a dataset of 9,000 multicultural images spanning five countries, three age groups, two genders, 25 historical landmarks, and five languages; and (3) We demonstrate that multi-agent interactions outperform simple, no-agent models across multiple evaluation metrics, offering valuable insights for future research. Our dataset and models are available at this https URL.
• 摘要：大型语言模型（LLMS）在各种多模式任务中表现出令人印象深刻的性能。但是，由于现有数据和模型的主要以西方为中心的性质，它们在跨文化环境中的有效性仍然有限。同时，多代理模型在解决复杂的任务方面表现出很强的功能。在本文中，我们评估了LLM在多元文化图像生成的新任务中在多代理相互作用环境中的性能。我们的主要贡献是：（1）我们介绍了Mosaig，这是一个多代理框架，通过利用具有不同文化角色的LLM来增强多元文化形象的产生； （2）我们提供了一个跨越五个国家，三个年龄段，两个性别，25个历史地标和五种语言的9,000个多元文化图像的数据集； （3）我们证明，多代理相互作用的表现优于多个评估指标的简单，无代理的模型，为未来的研究提供了宝贵的见解。我们的数据集和模型可在此HTTPS URL上找到。

Title: CoRe: Coherency Regularization for Hierarchical Time Series

• Authors: Rares Cristian, Pavithra Harhsa, Georgia Perakis, Brian Quanz
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.15983
• Pdf URL: https://arxiv.org/pdf/2502.15983
• Copy Paste: [[2502.15983]] CoRe: Coherency Regularization for Hierarchical Time Series(https://arxiv.org/abs/2502.15983)
• Keywords: generative
• Abstract: Hierarchical time series forecasting presents unique challenges, particularly when dealing with noisy data that may not perfectly adhere to aggregation constraints. This paper introduces a novel approach to soft coherency in hierarchical time series forecasting using neural networks. We present a network coherency regularization method, which we denote as CoRe (Coherency Regularization), a technique that trains neural networks to produce forecasts that are inherently coherent across hierarchies, without strictly enforcing aggregation constraints. Our method offers several key advantages. (1) It provides theoretical guarantees on the coherency of forecasts, even for out-of-sample data. (2) It is adaptable to scenarios where data may contain errors or missing values, making it more robust than strict coherency methods. (3) It can be easily integrated into existing neural network architectures for time series forecasting. We demonstrate the effectiveness of our approach on multiple benchmark datasets, comparing it against state-of-the-art methods in both coherent and noisy data scenarios. Additionally, our method can be used within existing generative probabilistic forecasting frameworks to generate coherent probabilistic forecasts. Our results show improved generalization and forecast accuracy, particularly in the presence of data inconsistencies. On a variety of datasets, including both strictly hierarchically coherent and noisy data, our training method has either equal or better accuracy at all levels of the hierarchy while being strictly more coherent out-of-sample than existing soft-coherency methods.
• 摘要：分层时间序列预测提出了独特的挑战，尤其是在处理可能无法完全遵守聚合约束的嘈杂数据时。本文介绍了一种使用神经网络预测的层次时间序列中软性相干性的新方法。我们提出了一种网络相干正则化方法，我们将其表示为核心（相干正规化），该技术训练神经网络以产生跨层次结构本质上相干的预测，而无需严格强制执行聚合约束。我们的方法提供了几个关键优势。 （1）它为预测的相干性提供了理论上的保证，即使对于样本外数据也是如此。 （2）它适用于数据可能包含错误或缺失值的方案，这使其比严格的相干方法更强大。 （3）可以轻松地集成到时间序列预测的现有神经网络体系结构中。我们证明了我们的方法在多个基准数据集上的有效性，并将其与连贯和嘈杂的数据方案中的最新方法进行了比较。此外，我们的方法可用于现有的生成概率预测框架中，以生成相干概率预测。我们的结果表明，概括和预测准确性的提高，尤其是在存在数据不一致的情况下。在各种数据集上，包括严格的层次连贯性和嘈杂的数据，我们的培训方法在层次结构的所有级别上具有相等或更好的精度，而严格的样本比现有的软质量方法更加连贯。

Title: Good Representation, Better Explanation: Role of Convolutional Neural Networks in Transformer-Based Remote Sensing Image Captioning

• Authors: Swadhin Das, Saarthak Gupta, and Kamal Kumar, Raksha Sharma
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16095
• Pdf URL: https://arxiv.org/pdf/2502.16095
• Copy Paste: [[2502.16095]] Good Representation, Better Explanation: Role of Convolutional Neural Networks in Transformer-Based Remote Sensing Image Captioning(https://arxiv.org/abs/2502.16095)
• Keywords: generation
• Abstract: Remote Sensing Image Captioning (RSIC) is the process of generating meaningful descriptions from remote sensing images. Recently, it has gained significant attention, with encoder-decoder models serving as the backbone for generating meaningful captions. The encoder extracts essential visual features from the input image, transforming them into a compact representation, while the decoder utilizes this representation to generate coherent textual descriptions. Recently, transformer-based models have gained significant popularity due to their ability to capture long-range dependencies and contextual information. The decoder has been well explored for text generation, whereas the encoder remains relatively unexplored. However, optimizing the encoder is crucial as it directly influences the richness of extracted features, which in turn affects the quality of generated captions. To address this gap, we systematically evaluate twelve different convolutional neural network (CNN) architectures within a transformer-based encoder framework to assess their effectiveness in RSIC. The evaluation consists of two stages: first, a numerical analysis categorizes CNNs into different clusters, based on their performance. The best performing CNNs are then subjected to human evaluation from a human-centric perspective by a human annotator. Additionally, we analyze the impact of different search strategies, namely greedy search and beam search, to ensure the best caption. The results highlight the critical role of encoder selection in improving captioning performance, demonstrating that specific CNN architectures significantly enhance the quality of generated descriptions for remote sensing images. By providing a detailed comparison of multiple encoders, this study offers valuable insights to guide advances in transformer-based image captioning models.
• 摘要：遥感图像字幕（RSIC）是从遥感图像中生成有意义的描述的过程。最近，它引起了极大的关注，编码器模型是生成有意义的字幕的骨干。编码器从输入图像中提取基本视觉特征，将它们转换为紧凑的表示形式，而解码器则利用此表示形式生成相干的文本描述。最近，由于捕获长期依赖性和上下文信息的能力，基于变压器的模型已获得了很大的知名度。解码器已经进行了很好的探索文本生成，而编码器仍然相对尚未探索。但是，优化编码器至关重要，因为它直接影响提取特征的丰富性，进而影响了生成的字幕质量。为了解决这一差距，我们系统地评估了基于变压器的编码器框架中的十二个不同的卷积神经网络（CNN）架构，以评估其在RSIC中的有效性。评估包括两个阶段：首先，数值分析根据其性能将CNN分为不同的群集。然后，人类注释者从以人为中心的角度将表现最佳的CNN从以人为中心的角度进行人类评估。此外，我们分析了不同搜索策略的影响，即贪婪的搜索和光束搜索，以确保最佳标题。结果突出了编码器选择在改善字幕性能中的关键作用，这表明特定的CNN体​​系结构显着提高了遥感图像的生成描述的质量。通过提供多个编码器的详细比较，本研究提供了有价值的见解，以指导基于变压器的图像字幕模型的进步。

Title: PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models

• Authors: Xinwei Liu, Xiaojun Jia, Yuan Xun, Hua Zhang, Xiaochun Cao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16167
• Pdf URL: https://arxiv.org/pdf/2502.16167
• Copy Paste: [[2502.16167]] PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.16167)
• Keywords: generation, generative
• Abstract: Diffusion models (DMs) have revolutionized data generation, particularly in text-to-image (T2I) synthesis. However, the widespread use of personalized generative models raises significant concerns regarding privacy violations and copyright infringement. To address these issues, researchers have proposed adversarial perturbation-based protection techniques. However, these methods have notable limitations, including insufficient robustness against data transformations and the inability to fully eliminate identifiable features of protected objects in the generated output. In this paper, we introduce PersGuard, a novel backdoor-based approach that prevents malicious personalization of specific images. Unlike traditional adversarial perturbation methods, PersGuard implant backdoor triggers into pre-trained T2I models, preventing the generation of customized outputs for designated protected images while allowing normal personalization for unprotected ones. Unfortunately, existing backdoor methods for T2I diffusion models fail to be applied to personalization scenarios due to the different backdoor objectives and the potential backdoor elimination during downstream fine-tuning processes. To address these, we propose three novel backdoor objectives specifically designed for personalization scenarios, coupled with backdoor retention loss engineered to resist downstream fine-tuning. These components are integrated into a unified optimization framework. Extensive experimental evaluations demonstrate PersGuard's effectiveness in preserving data privacy, even under challenging conditions including gray-box settings, multi-object protection, and facial identity scenarios. Our method significantly outperforms existing techniques, offering a more robust solution for privacy and copyright protection.
• 摘要：扩散模型（DMS）已彻底改变了数据生成，尤其是在文本对图像（T2I）合成中。但是，对个性化生成模型的广泛使用引起了人们对侵犯隐私和侵犯版权的重大关注。为了解决这些问题，研究人员提出了基于对抗性扰动的保护技术。但是，这些方法具有明显的局限性，包括对数据转换的鲁棒性不足以及无法完全消除生成的输出中受保护对象的可识别特征。在本文中，我们介绍了Persguard，这是一种基于新型后门的方法，可防止特定图像的恶意个性化。与传统的对抗扰动方法不同，Persguard植入物后门触发器触发了预训练的T2I模型，从而防止了指定受保护图像的自定义输出的生成，同时允许对未受保护的图像进行正常的个性化化。不幸的是，由于后门目标的不同，在下游微调过程中，现有的T2I扩散模型的后门方法无法应用于个性化方案。为了解决这些问题，我们提出了三个专门设计用于个性化场景的新型后门目标，再加上设计可抵抗下游微调的后门保留损失。这些组件被整合到统一的优化框架中。广泛的实验评估表明，即使在具有挑战性的条件下，Persguard在保护数据隐私方面的有效性，包括灰色盒子设置，多对象保护和面部身份方案。我们的方法极大地胜过现有技术，为隐私和版权保护提供了更强大的解决方案。

Title: DiffFake: Exposing Deepfakes using Differential Anomaly Detection

• Authors: Sotirios Stamnas, Victor Sanchez
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16247
• Pdf URL: https://arxiv.org/pdf/2502.16247
• Copy Paste: [[2502.16247]] DiffFake: Exposing Deepfakes using Differential Anomaly Detection(https://arxiv.org/abs/2502.16247)
• Keywords: generation
• Abstract: Traditional deepfake detectors have dealt with the detection problem as a binary classification task. This approach can achieve satisfactory results in cases where samples of a given deepfake generation technique have been seen during training, but can easily fail with deepfakes generated by other techniques. In this paper, we propose DiffFake, a novel deepfake detector that approaches the detection problem as an anomaly detection task. Specifically, DiffFake learns natural changes that occur between two facial images of the same person by leveraging a differential anomaly detection framework. This is done by combining pairs of deep face embeddings and using them to train an anomaly detection model. We further propose to train a feature extractor on pseudo-deepfakes with global and local artifacts, to extract meaningful and generalizable features that can then be used to train the anomaly detection model. We perform extensive experiments on five different deepfake datasets and show that our method can match and sometimes even exceed the performance of state-of-the-art competitors.
• 摘要：传统的DeepFake探测器已将检测问题作为二进制分类任务。在训练期间已经看到给定深层产生技术的样品的情况下，这种方法可以取得令人满意的结果，但是通过其他技术产生的深泡沫很容易失败。在本文中，我们提出了Difffake，这是一种新型的深泡检测器，将检测问题作为一种异常检测任务。具体而言，Diffffake通过利用差异异常检测框架来学习同一人的两个面部图像之间发生的自然变化。这是通过结合深面嵌入并使用它们来训练异常检测模型来完成的。我们进一步建议用全球和本地人工制品训练特征提取器，以提取有意义且可推广的特征，然后将其用于训练异常检测模型。我们在五个不同的DeepFake数据集上进行了广泛的实验，并表明我们的方法可以匹配，甚至超过最先进的竞争对手的表现。

Title: Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction

• Authors: Sarah Ball, Simeon Allmendinger, Frauke Kreuter, Niklas Kühl
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16280
• Pdf URL: https://arxiv.org/pdf/2502.16280
• Copy Paste: [[2502.16280]] Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction(https://arxiv.org/abs/2502.16280)
• Keywords: generative
• Abstract: Generative AI (GenAI) is increasingly used in survey contexts to simulate human preferences. While many research endeavors evaluate the quality of synthetic GenAI data by comparing model-generated responses to gold-standard survey results, fundamental questions about the validity and reliability of using LLMs as substitutes for human respondents remain. Our study provides a technical analysis of how demographic attributes and prompt variations influence latent opinion mappings in large language models (LLMs) and evaluates their suitability for survey-based predictions. Using 14 different models, we find that LLM-generated data fails to replicate the variance observed in real-world human responses, particularly across demographic subgroups. In the political space, persona-to-party mappings exhibit limited differentiation, resulting in synthetic data that lacks the nuanced distribution of opinions found in survey data. Moreover, we show that prompt sensitivity can significantly alter outputs for some models, further undermining the stability and predictiveness of LLM-based simulations. As a key contribution, we adapt a probe-based methodology that reveals how LLMs encode political affiliations in their latent space, exposing the systematic distortions introduced by these models. Our findings highlight critical limitations in AI-generated survey data, urging caution in its use for public opinion research, social science experimentation, and computational behavioral modeling.
• 摘要：生成的AI（Genai）越来越多地用于调查环境中，以模拟人类的偏好。尽管许多研究努力通过比较模型生成的响应对金标准的调查结果来评估合成Genai数据的质量，但有关使用LLMs作为人类受访者替代品的有效性和可靠性的基本问题仍然存在。我们的研究提供了有关人口属性和及时变化如何影响大语言模型（LLMS）的潜在意见映射的技术分析，并评估了其对基于调查的预测的适用性。使用14个不同的模型，我们发现LLM生成的数据无法复制在现实世界中的人类反应中观察到的差异，尤其是在人口统计亚组之间。在政治领域，角色到党派映射的分化有限，导致合成数据缺乏调查数据中发现的观点的细微分布。此外，我们表明迅速灵敏度可以显着改变某些模型的输出，从而进一步破坏了基于LLM的模拟的稳定性和预测性。作为关键贡献，我们适应了一种基于探针的方法，该方法揭示了LLM在其潜在空间中如何编码政治隶属关系，从而揭示了这些模型引起的系统扭曲。我们的发现突出了AI生成的调查数据中的临界局限性，敦促其用于公众舆论研究，社会科学实验和计算行为建模。

Title: FHGE: A Fast Heterogeneous Graph Embedding with Ad-hoc Meta-paths

• Authors: Xuqi Mao, Zhenying He, X. Sean Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16281
• Pdf URL: https://arxiv.org/pdf/2502.16281
• Copy Paste: [[2502.16281]] FHGE: A Fast Heterogeneous Graph Embedding with Ad-hoc Meta-paths(https://arxiv.org/abs/2502.16281)
• Keywords: generation
• Abstract: Graph neural networks (GNNs) have emerged as the state of the art for a variety of graph-related tasks and have been widely used in Heterogeneous Graphs (HetGs), where meta-paths help encode specific semantics between various node types. Despite the revolutionary representation capabilities of existing heterogeneous GNNs (HGNNs) due to their focus on improving the effectiveness of heterogeneity capturing, the huge training costs hinder their practical deployment in real-world scenarios that frequently require handling ad-hoc queries with user-defined meta-paths. To address this, we propose FHGE, a Fast Heterogeneous Graph Embedding designed for efficient, retraining-free generation of meta-path-guided graph embeddings. The key design of the proposed framework is two-fold: segmentation and reconstruction modules. It employs Meta-Path Units (MPUs) to segment the graph into local and global components, enabling swift integration of node embeddings from relevant MPUs during reconstruction and allowing quick adaptation to specific meta-paths. In addition, a dual attention mechanism is applied to enhance semantics capturing. Extensive experiments across diverse datasets demonstrate the effectiveness and efficiency of FHGE in generating meta-path-guided graph embeddings and downstream tasks, such as link prediction and node classification, highlighting its significant advantages for real-time graph analysis in ad-hoc queries.
• 摘要：图形神经网络（GNN）已成为各种与图形相关的任务的艺术状态，并已广泛用于异质图（HETGS），其中元路径有助于在各种节点类型之间编码特定的语义。尽管现有异质GNN（HGNN）具有革命性的代表能力，因为它们专注于提高异质性捕获的有效性，但巨大的培训成本阻碍了他们在现实世界中的实践部署，这些场景经常需要处理Ad-Hoc Queries，并使用用户确定的元数据进行处理。 - 路径。为了解决这个问题，我们提出了FHGE，这是一种旨在有效的，无需重新训练的元数据指导的图形嵌入的快速异质图嵌入。所提出的框架的关键设计是两个方面：分割和重建模块。它采用元路径单元（MPU）将图形分割为局部和全局组件，从而在重建过程中可以快速整合来自相关MPU的节点嵌入，并允许快速适应特定的元数据。此外，还采用双重注意机制来增强语义捕获。跨不同数据集的广泛实验证明了FHGE在生成元数据指导的图形嵌入和下游任务方面的有效性和效率，例如链接预测和节点分类，突出了其在Ad-Hoc查询中实时图表分析的显着优势。

Title: DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation

• Authors: Yuxuan Xiong, Yue Shi, Yishun Dou, Bingbing Ni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16302
• Pdf URL: https://arxiv.org/pdf/2502.16302
• Copy Paste: [[2502.16302]] DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation(https://arxiv.org/abs/2502.16302)
• Keywords: generation
• Abstract: Recently, denoising diffusion models have achieved promising results in 2D image generation and editing. Instruct-NeRF2NeRF (IN2N) introduces the success of diffusion into 3D scene editing through an "Iterative dataset update" (IDU) strategy. Though achieving fascinating results, IN2N suffers from problems of blurry backgrounds and trapping in local optima. The first problem is caused by IN2N's lack of efficient guidance for background maintenance, while the second stems from the interaction between image editing and NeRF training during IDU. In this work, we introduce DualNeRF to deal with these problems. We propose a dual-field representation to preserve features of the original scene and utilize them as additional guidance to the model for background maintenance during IDU. Moreover, a simulated annealing strategy is embedded into IDU to endow our model with the power of addressing local optima issues. A CLIP-based consistency indicator is used to further improve the editing quality by filtering out low-quality edits. Extensive experiments demonstrate that our method outperforms previous methods both qualitatively and quantitatively.
• 摘要：最近，在2D图像生成和编辑中，DeNoisis扩散模型取得了有希望的结果。 Construment-nerf2nerf（IN2N）通过“迭代数据集更新”（IDU）策略引入了扩散成功到3D场景编辑中的成功。尽管取得了令人着迷的结果，但IN2N遭受了模糊背景的问题和捕获本地Optima的问题。第一个问题是由IN2N缺乏有效的背景维护指导引起的，而第二个问题是由于图像编辑和IDU期间的NERF培训之间的相互作用造成的。在这项工作中，我们介绍Dualnerf来解决这些问题。我们提出了一个双场表示，以保留原始场景的特征，并将其用作IDU期间背景维护模型的附加指导。此外，将模拟退火策略嵌入IDU中，以赋予我们的模型解决本地Optima问题的能力。基于夹的一致性指标用于通过过滤低质量的编辑来进一步提高编辑质量。广泛的实验表明，我们的方法在定性和定量上都优于先前的方法。

Title: Concept Corrector: Erase concepts on the fly for text-to-image diffusion models

• Authors: Zheling Meng, Bo Peng, Xiaochuan Jin, Yueming Lyu, Wei Wang, Jing Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16368
• Pdf URL: https://arxiv.org/pdf/2502.16368
• Copy Paste: [[2502.16368]] Concept Corrector: Erase concepts on the fly for text-to-image diffusion models(https://arxiv.org/abs/2502.16368)
• Keywords: generation
• Abstract: Text-to-image diffusion models have demonstrated the underlying risk of generating various unwanted content, such as sexual elements. To address this issue, the task of concept erasure has been introduced, aiming to erase any undesired concepts that the models can generate. Previous methods, whether training-based or training-free, have primarily focused on the input side, i.e. texts. However, they often suffer from incomplete erasure due to limitations in the generalization from limited prompts to diverse image content. In this paper, motivated by the notion that concept erasure on the output side, i.e. generated images, may be more direct and effective, we propose to check concepts based on intermediate-generated images and correct them in the remainder of the generation process. Two key challenges are identified, i.e. determining the presence of target concepts during generation and replacing them on the fly. Leveraging the generation mechanism of diffusion models, we present the Concept Corrector, which incorporates the Generation Check Mechanism and the Concept Removal Attention. This method can identify the generated features associated with target concepts and replace them using pre-defined negative prompts, thereby achieving concept erasure. It requires no changes to model parameters and only relies on a given concept name and its replacement content. To the best of our knowledge, this is the first erasure method based on intermediate-generated images. The experiments on various concepts demonstrate its impressive erasure performance. Code: this https URL.
• 摘要：文本到图像扩散模型表明，产生各种不必要的内容（例如性元素）的潜在风险。为了解决这个问题，已经引入了概念擦除的任务，旨在消除模型可以生成的任何不希望的概念。以前的方法，无论是基于培训还是无培训，都主要集中在输入方面，即文本。但是，由于从有限的提示到多样化的图像内容的限制，它们通常会遭受不完全擦除的困扰。在本文中，以这样的概念在输出侧擦除的概念（即生成的图像）​​可能更直接和有效，我们建议在其余的生成过程中检查基于中间生成的图像并纠正它们的概念。确定了两个关键的挑战，即确定生成过程中目标概念的存在并即时更换。利用扩散模型的发电机理，我们提出了概念校正器，该概念校正了，该概念结合了生成检查机制和概念去除注意力。该方法可以识别与目标概念相关的生成的功能，并使用预定义的负面提示替换它们，从而实现概念擦除。它不需要更改模型参数，仅依赖于给定的概念名称及其替换内容。据我们所知，这是基于中间生成图像的第一种擦除方法。关于各种概念的实验表明了其令人印象深刻的擦除性能。代码：此HTTPS URL。

Title: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science

• Authors: Qiuhai Zeng, Claire Jin, Xinyue Wang, Yuhan Zheng, Qunhua Li
• Subjects: cs.LG, cs.AI, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.16395
• Pdf URL: https://arxiv.org/pdf/2502.16395
• Copy Paste: [[2502.16395]] An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science(https://arxiv.org/abs/2502.16395)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have demonstrated potential for data science tasks via code generation. However, the exploratory nature of data science, alongside the stochastic and opaque outputs of LLMs, raise concerns about their reliability. While prior work focuses on benchmarking LLM accuracy, reproducibility remains underexplored, despite being critical to establishing trust in LLM-driven analysis. We propose a novel analyst-inspector framework to automatically evaluate and enforce the reproducibility of LLM-generated data science workflows - the first rigorous approach to the best of our knowledge. Defining reproducibility as the sufficiency and completeness of workflows for reproducing functionally equivalent code, this framework enforces computational reproducibility principles, ensuring transparent, well-documented LLM workflows while minimizing reliance on implicit model assumptions. Using this framework, we systematically evaluate five state-of-the-art LLMs on 1,032 data analysis tasks across three diverse benchmark datasets. We also introduce two novel reproducibility-enhancing prompting strategies. Our results show that higher reproducibility strongly correlates with improved accuracy and reproducibility-enhancing prompts are effective, demonstrating structured prompting's potential to enhance automated data science workflows and enable transparent, robust AI-driven analysis. Our code is publicly available.
• 摘要：大型语言模型（LLMS）通过代码生成证明了数据科学任务的潜力。但是，数据科学的探索性质以及LLM的随机和不透明的产出引起了人们对其可靠性的关注。尽管先前的工作着重于LLM准确性，但尽管对于建立对LLM驱动的分析的信任至关重要，但可重复性仍然没有被忽视。我们提出了一个新颖的分析师检查框架，以自动评估和执行LLM生成的数据科学工作流的可重复性 - 据我们所知，这是第一种严格的方法。该框架将重复性定义为重现功能等效代码的工作流的充分性和完整性，可以实现计算重现性原理，确保透明，文献良好的LLM工作流程，同时最大程度地减少对隐式模型假设的依赖。使用此框架，我们在三个不同基准数据集的1,032个数据分析任务上系统地评估了五个最先进的LLMS。我们还介绍了两个新颖的可重复性提高促进策略。我们的结果表明，更高的可重复性与提高的准确性和增强性提示密切相关，这表明结构化提示的潜力增强了自动化数据科学工作流，并实现了透明，稳健的AI-drion驱动分析。我们的代码公开可用。

Title: A Survey on Industrial Anomalies Synthesis

• Authors: Xichen Xu, Yanshu Wang, Yawen Huang, Jiaqi Liu, Xiaoning Lei, Guoyang Xie, Guannan Jiang, Zhichao Lu
• Subjects: cs.CV, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.16412
• Pdf URL: https://arxiv.org/pdf/2502.16412
• Copy Paste: [[2502.16412]] A Survey on Industrial Anomalies Synthesis(https://arxiv.org/abs/2502.16412)
• Keywords: generative
• Abstract: This paper comprehensively reviews anomaly synthesis methodologies. Existing surveys focus on limited techniques, missing an overall field view and understanding method interconnections. In contrast, our study offers a unified review, covering about 40 representative methods across Hand-crafted, Distribution-hypothesis-based, Generative models (GM)-based, and Vision-language models (VLM)-based synthesis. We introduce the first industrial anomaly synthesis (IAS) taxonomy. Prior works lack formal classification or use simplistic taxonomies, hampering structured comparisons and trend identification. Our taxonomy provides a fine-grained framework reflecting methodological progress and practical implications, grounding future research. Furthermore, we explore cross-modality synthesis and large-scale VLM. Previous surveys overlooked multimodal data and VLM in anomaly synthesis, limiting insights into their advantages. Our survey analyzes their integration, benefits, challenges, and prospects, offering a roadmap to boost IAS with multimodal learning. More resources are available at this https URL.
• 摘要：本文全面回顾了异常合成方法。现有的调查专注于有限的技术，缺少整体现场视图和理解方法互连。相比之下，我们的研究提供了统一的综述，涵盖了基于手工制作的，基于分布的生成模型（GM）的基于手工制作的，基于视觉模型和基于视觉模型（VLM）基于基于的综合的代表性方法。我们介绍了第一个工业异常合成（IAS）分类法。先前的工作缺乏正式分类或使用简单的分类法，妨碍结构化的比较和趋势识别。我们的分类法提供了一个精细的框架，反映了方法论进步和实际含义，从而扎根未来的研究。此外，我们探讨了跨模式合成和大规模VLM。先前的调查忽略了异常合成中的多模式数据和VLM，将见解限制在其优势中。我们的调查分析了它们的整合，收益，挑战和前景，提供了通过多模式学习来提高IAS的路线图。此HTTPS URL提供了更多资源。

Title: TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation

• Authors: Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16414
• Pdf URL: https://arxiv.org/pdf/2502.16414
• Copy Paste: [[2502.16414]] TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation(https://arxiv.org/abs/2502.16414)
• Keywords: generation
• Abstract: Large Language models (LLMs) have achieved encouraging results in tabular data generation. However, existing approaches require fine-tuning, which is computationally expensive. This paper explores an alternative: prompting a fixed LLM with in-context examples. We observe that using randomly selected in-context examples hampers the LLM's performance, resulting in sub-optimal generation quality. To address this, we propose a novel in-context learning framework: TabGen-ICL, to enhance the in-context learning ability of LLMs for tabular data generation. TabGen-ICL operates iteratively, retrieving a subset of real samples that represent the residual between currently generated samples and true data distributions. This approach serves two purposes: locally, it provides more effective in-context learning examples for the LLM in each iteration; globally, it progressively narrows the gap between generated and real data. Extensive experiments on five real-world tabular datasets demonstrate that TabGen-ICL significantly outperforms the random selection strategy. Specifically, it reduces the error rate by a margin of $3.5\%-42.2\%$ on fidelity metrics. We demonstrate for the first time that prompting a fixed LLM can yield high-quality synthetic tabular data. The code is provided in the \href{this https URL}{link}.
• 摘要：大型语言模型（LLM）在表格数据生成中取得了令人鼓舞的结果。但是，现有的方法需要微调，这在计算上很昂贵。本文探讨了一种替代方案：提示具有固定的LLM，其中包含在上下文中的示例。我们观察到，使用随机选择的中文示例阻碍了LLM的性能，从而产生了次优的生成质量。为了解决这个问题，我们提出了一个新颖的文本学习框架：TABGEN-ICL，以增强LLMS对表格数据生成的文化学习能力。 Tabgen-ICL迭代运行，检索代表当前生成的样本和真实数据分布之间残留的真实样品的子集。这种方法有两个目的：在本地，它为每次迭代中的LLM提供了更有效的内在学习示例；在全球范围内，它逐渐缩​​小了生成的数据和真实数据之间的差距。在五个现实世界表格数据集上进行的广泛实验表明，Tabgen-ICL显着优于随机选择策略。具体而言，Fidelity指标的错误率将错误率降低了$ 3.5 \％-42.2 \％$。我们首次证明提示固定的LLM可以产生高质量的合成表格数据。该代码在\ href {此https url} {link}中提供。

Title: High-resolution Rainy Image Synthesis: Learning from Rendering

• Authors: Kaibin Zhou, Shengjie Zhao, Hao Deng, Lin Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16421
• Pdf URL: https://arxiv.org/pdf/2502.16421
• Copy Paste: [[2502.16421]] High-resolution Rainy Image Synthesis: Learning from Rendering(https://arxiv.org/abs/2502.16421)
• Keywords: generation
• Abstract: Currently, there are few effective methods for synthesizing a mass of high-resolution rainy images in complex illumination conditions. However, these methods are essential for synthesizing large-scale high-quality paired rainy-clean image datasets, which can train deep learning-based single image rain removal models capable of generalizing to various illumination conditions. Therefore, we propose a practical two-stage learning-from-rendering pipeline for high-resolution rainy image synthesis. The pipeline combines the benefits of the realism of rendering-based methods and the high-efficiency of learning-based methods, providing the possibility of creating large-scale high-quality paired rainy-clean image datasets. In the rendering stage, we use a rendering-based method to create a High-resolution Rainy Image (HRI) dataset, which contains realistic high-resolution paired rainy-clean images of multiple scenes and various illumination conditions. In the learning stage, to learn illumination information from background images for high-resolution rainy image generation, we propose a High-resolution Rainy Image Generation Network (HRIGNet). HRIGNet is designed to introduce a guiding diffusion model in the Latent Diffusion Model, which provides additional guidance information for high-resolution image synthesis. In our experiments, HRIGNet is able to synthesize high-resolution rainy images up to 2048x1024 resolution. Rain removal experiments on real dataset validate that our method can help improve the robustness of deep derainers to real rainy images. To make our work reproducible, source codes and the dataset have been released at this https URL.
• 摘要：当前，在复杂的照明条件下，很少有有效的方法可以合成大量高分辨率下雨图像。但是，这些方法对于综合大规模高质量的多雨 - 清洁图像数据集至关重要，该数据集可以训练能够概括为各种照明条件的深度学习基于学习的单图像去除模型。因此，我们提出了一条实用的两阶段学习，从渲染管道上进行了高分辨率雨水图像合成。该管道结合了基于渲染的方法的现实主义的好处和基于学习的方法的高效率，从而提供了创建大型高质量配对的多雨 - 清洁图像数据集的可能性。在渲染阶段，我们使用一种基于渲染的方法来创建高分辨率下雨图像（HRI）数据集，该数据集包含了多个场景和各种照明条件的现实高分辨率配对的多雨雨清洁图像。在学习阶段，为了从高分辨率下雨图像生成的背景图像中学习照明信息，我们提出了一个高分辨率下雨的图像生成网络（Hrignet）。 Hrignet旨在在潜在扩散模型中引入指导扩散模型，该模型为高分辨率图像合成提供了其他指导信息。在我们的实验中，Hrignet能够合成高分辨率下雨的图像，直至2048x1024分辨率。在实际数据集上进行的去除降雨实验验证了我们的方法可以帮助改善深度降雨图像的鲁棒性。为了使我们的工作可重复，源代码和数据集已在此HTTPS URL上发布。

Title: Unified Prompt Attack Against Text-to-Image Generation Models

• Authors: Duo Peng, Qiuhong Ke, Mark He Huang, Ping Hu, Jun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16423
• Pdf URL: https://arxiv.org/pdf/2502.16423
• Copy Paste: [[2502.16423]] Unified Prompt Attack Against Text-to-Image Generation Models(https://arxiv.org/abs/2502.16423)
• Keywords: generation
• Abstract: Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries--common in prior methods and easily detectable by API defenders--by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM's superiority in effectiveness, efficiency, naturalness, and low query detection rates.
• 摘要：文本对图像（T2I）模型已大大提高，但是由于它们产生有害图像的潜力，它们日益普及引起了安全的关注。为了解决这些问题，我们提出了UPAM，这是一个新颖的框架，旨在从攻击的角度评估T2I模型的鲁棒性。与仅关注文本防御的先前方法不同，UPAM统一了对文本和视觉防御的攻击。此外，它可以基于梯度的优化，克服对枚举的依赖，以提高效率和有效性。为了处理T2I模型由于防御措施而导致的图像输出的情况，我们引入了球体进行学习（SPL），即使没有图像结果，也可以启用优化。在SPL之后，我们的模型绕过防御，引起了有害内容的产生。为了确保与攻击者意图的语义对齐，我们建议使用语义增强学习（SEL）进行精确的语义控制。 UPAM还优先考虑使用自然性增强（INE）的对抗提示的自然性，这使得它们难以检测到人类检查员。此外，我们通过引入可转移攻击学习（TAL）来解决迭代查询的问题 - 在先前的方法中概述，并易于被API辩护人检测到，从而可以使用最少的查询进行有效的攻击。广泛的实验验证了UPAM在有效性，效率，自然性和低查询检测率方面的优势。

Title: Fine-Grained Video Captioning through Scene Graph Consolidation

• Authors: Sanghyeok Chu, Seonguk Seo, Bohyung Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16427
• Pdf URL: https://arxiv.org/pdf/2502.16427
• Copy Paste: [[2502.16427]] Fine-Grained Video Captioning through Scene Graph Consolidation(https://arxiv.org/abs/2502.16427)
• Keywords: generation
• Abstract: Recent advances in visual language models (VLMs) have significantly improved image captioning, but extending these gains to video understanding remains challenging due to the scarcity of fine-grained video captioning datasets. To bridge this gap, we propose a novel zero-shot video captioning approach that combines frame-level scene graphs from a video to obtain intermediate representations for caption generation. Our method first generates frame-level captions using an image VLM, converts them into scene graphs, and consolidates these graphs to produce comprehensive video-level descriptions. To achieve this, we leverage a lightweight graph-to-text model trained solely on text corpora, eliminating the need for video captioning annotations. Experiments on the MSR-VTT and ActivityNet Captions datasets show that our approach outperforms zero-shot video captioning baselines, demonstrating that aggregating frame-level scene graphs yields rich video understanding without requiring large-scale paired data or high inference cost.
• 摘要：视觉语言模型（VLM）的最新进展已大大改善了图像字幕，但是由于缺乏细粒度的视频字幕数据集，将这些收益扩展到视频理解仍然具有挑战性。为了弥合这一差距，我们提出了一种新颖的零拍视频字幕方法，该方法结合了视频中的框架级场景图，以获取字幕生成的中间表示。我们的方法首先使用图像VLM生成帧级字幕，将它们转换为场景图，并合并这些图以生成全面的视频级描述。为了实现这一目标，我们利用仅在文本语料库中训练的轻量级图表模型，消除了对视频字幕注释的需求。 MSR-VTT和ActivityNet字幕数据集上的实验表明，我们的方法的表现优于零拍的视频字幕字幕基线，表明汇总帧级场景图可以产生丰富的视频理解，而无需大规模的配对数据或高推理成本。

Title: Visual Reasoning Evaluation of Grok, Deepseek Janus, Gemini, Qwen, Mistral, and ChatGPT

• Authors: Nidhal Jegham, Marwan Abdelatti, Abdeltawab Hendawi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16428
• Pdf URL: https://arxiv.org/pdf/2502.16428
• Copy Paste: [[2502.16428]] Visual Reasoning Evaluation of Grok, Deepseek Janus, Gemini, Qwen, Mistral, and ChatGPT(https://arxiv.org/abs/2502.16428)
• Keywords: generation
• Abstract: Traditional evaluations of multimodal large language models (LLMs) have been limited by their focus on single-image reasoning, failing to assess crucial aspects like contextual understanding, reasoning stability, and uncertainty calibration. This study addresses these limitations by introducing a novel benchmark that integrates multi-image reasoning tasks with rejection-based evaluation and positional bias detection. To evaluate these dimensions, we further introduce entropy as a novel metric for quantifying reasoning consistency across reordered answer variants. We applied this benchmark to assess Grok 3, ChatGPT-4o, ChatGPT-o1, Gemini 2.0 Flash Experimental, DeepSeek Janus models, Qwen2.5-VL-72B-Instruct, QVQ-72B-Preview, and Pixtral 12B across eight visual reasoning tasks, including difference spotting and diagram interpretation. Our findings reveal ChatGPT-o1 leading in overall accuracy (82.5\%) and rejection accuracy (70.0\%), closely followed by Gemini 2.0 Flash Experimental (70.8\%). QVQ-72B-Preview demonstrated superior rejection accuracy (85.5\%). Notably, Pixtral 12B (51.7\%) showed promise in specific domains, while Janus models exhibited challenges in bias and uncertainty calibration, reflected in low rejection accuracies and high entropy scores. High entropy scores in Janus models (Janus 7B: 0.8392, Janus 1B: 0.787) underscore their susceptibility to positional bias and unstable reasoning, contrasting with the low entropy and robust reasoning of ChatGPT models. The study further demonstrates that model size is not the sole determinant of performance, as evidenced by Grok 3 underperformance despite its substantial parameter count. By employing multi-image contexts, rejection mechanisms, and entropy-based consistency metrics, this benchmark sets a new standard for evaluating multimodal LLMs, enabling a more robust and reliable assessment of next-generation AI systems.
• 摘要：多模式大语言模型（LLM）的传统评估受到关注单像推理的限制，无法评估关键方面，例如上下文理解，推理稳定性和不确定性校准。这项研究通过引入一个新的基准测试来解决这些局限性，该基准将多图像推理任务与基于拒绝的评估和位置偏见检测相结合。为了评估这些维度，我们进一步引入熵作为一种新的度量标准，用于量化重新排序的答案变体的推理一致性。我们将此基准应用用于评估Grok 3，Chatgpt-4O，Chatgpt-O1，Gemini 2.0 Flash实验，DeepSeek Janus模型，QWEN2.5-VL-72B-INSTRUCT ，包括差异发现和图表解释。我们的发现揭示了CHATGPT-O1的总体准确性（82.5 \％）和排斥准确性（70.0 \％），紧随其后的是Gemini 2.0 Flash实验性（70.8 \％）。 QVQ-72B-preiview表现出较高的排斥准确性（85.5 \％）。值得注意的是，PixTral 12b（51.7 \％）在特定领域显示出希望，而Janus模型在偏置和不确定性校准方面表现出挑战，反映在低排斥精度和高熵分数中。 Janus模型（Janus 7b：0.8392，Janus 1b：0.787）的高熵得分强调了它们对位置偏见和不稳定推理的敏感性，与低熵和Chatgpt模型的稳健推理形成鲜明对比。该研究进一步表明，模型大小不是性能的唯一决定因素，尽管Grok 3的表现不佳，尽管其大量参数计数。通过采用多图像上下文，拒绝机制和基于熵的一致性指标，该基准为评估多模式LLM的新标准设定了新标准，从而实现了对下一代AI系统的更强大和可靠的评估。

Title: UniDyG: A Unified and Effective Representation Learning Approach for Large Dynamic Graphs

• Authors: Yuanyuan Xu, Wenjie Zhang, Xuemin Lin, Ying Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16431
• Pdf URL: https://arxiv.org/pdf/2502.16431
• Copy Paste: [[2502.16431]] UniDyG: A Unified and Effective Representation Learning Approach for Large Dynamic Graphs(https://arxiv.org/abs/2502.16431)
• Keywords: generation
• Abstract: Dynamic graphs are formulated in continuous-time or discrete-time dynamic graphs. They differ in temporal granularity: Continuous-Time Dynamic Graphs (CTDGs) exhibit rapid, localized changes, while Discrete-Time Dynamic Graphs (DTDGs) show gradual, global updates. This difference leads to isolated developments in representation learning for each type. To advance representation learning, recent research attempts to design a unified model capable of handling both CTDGs and DTDGs. However, it typically focuses on local dynamic propagation for temporal structure learning in the time domain, failing to accurately capture the structural evolution associated with each temporal granularity. In addition, existing works-whether specific or unified-often overlook the issue of temporal noise, compromising the model robustness and effectiveness. To better model both types of dynamic graphs, we propose UniDyG, a unified and effective representation learning approach, which scales to large dynamic graphs. We first propose a novel Fourier Graph Attention (FGAT) mechanism that can model local and global structural correlations based on recent neighbors and complex-number selective aggregation, while theoretically ensuring consistent representations of dynamic graphs over time. Based on approximation theory, we demonstrate that FGAT is well-suited to capture the underlying structures in CTDGs and DTDGs. We further enhance FGAT to resist temporal noise by designing an energy-gated unit, which adaptively filters out high-frequency noise according to the energy. Last, we leverage our FGAT mechanisms for temporal structure learning and employ the frequency-enhanced linear function for node-level dynamic updates, facilitating the generation of high-quality temporal embeddings. Extensive experiments show that our UniDyG achieves an average improvement of 14.4% over sixteen baselines across nine dynamic graphs.
• 摘要：动态图在连续时间或离散时间动态图中配制。它们在时间粒度上有所不同：连续时间动态图（CTDG）表现出快速的局部变化，而离散时间动态图（DTDGS）显示出逐渐的全局更新。这种差异导致每种类型的表示形式学习中孤立的发展。为了推动表示形式学习，最近的研究试图设计一个能够处理CTDG和DTDG的统一模型。但是，它通常着重于时间域中的时间结构学习的局部动态传播，无法准确捕获与每个时间粒度相关的结构演化。此外，现有的工作 - 特定的或统一的，通常会忽略时间噪声的问题，从而损害了模型的稳健性和有效性。为了更好地模拟两种类型的动态图，我们建议Unidyg，unidyg是一种统一有效的表示学习方法，该方法将其扩展到大型动态图。我们首先提出了一种新型的傅立叶图（FGAT）机制，该机制可以基于最近的邻居和复杂的数字选择性聚合来对局部和全局结构相关进行建模，而理论上可以确保随着时间的推移的动态图的一致表示。基于近似理论，我们证明了FGAT非常适合捕获CTDG和DTDG中的基础结构。我们通过设计能量门控单元来进一步增强FGAT以抵抗时间噪声，该单元根据能量自适应地滤除高频噪声。最后，我们利用FGAT机制进行时间结构学习，并采用频率增强的线性函数来用于节点级动态更新，从而促进了高质量的时间嵌入的生成。广泛的实验表明，在九个动态图中，我们的Unidyg在16个基准中的平均提高14.4％。

Title: Iterative Flow Matching -- Path Correction and Gradual Refinement for Enhanced Generative Modeling

• Authors: Eldad Haber, Shadab Ahamed, Md. Shahriar Rahim Siddiqui, Niloufar Zakariaei, Moshe Eliasof
• Subjects: cs.LG, cs.AI, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.16445
• Pdf URL: https://arxiv.org/pdf/2502.16445
• Copy Paste: [[2502.16445]] Iterative Flow Matching -- Path Correction and Gradual Refinement for Enhanced Generative Modeling(https://arxiv.org/abs/2502.16445)
• Keywords: generation, generative
• Abstract: Generative models for image generation are now commonly used for a wide variety of applications, ranging from guided image generation for entertainment to solving inverse problems. Nonetheless, training a generator is a non-trivial feat that requires fine-tuning and can lead to so-called hallucinations, that is, the generation of images that are unrealistic. In this work, we explore image generation using flow matching. We explain and demonstrate why flow matching can generate hallucinations, and propose an iterative process to improve the generation process. Our iterative process can be integrated into virtually $\textit{any}$ generative modeling technique, thereby enhancing the performance and robustness of image synthesis systems.
• 摘要：现在，用于图像生成的生成模型现在通常用于多种应用，从娱乐的指导图像产生到解决反问题。尽管如此，训练生成器是一种非平凡的壮举，需要进行微调，并可能导致所谓的幻觉，也就是说，即不现实的图像的产生。在这项工作中，我们使用流匹配探索图像生成。我们解释并证明了为什么流匹配可以产生幻觉，并提出了改善生成过程的迭代过程。我们的迭代过程几乎可以集成到$ \ textit {any} $生成建模技术，从而增强图像合成系统的性能和鲁棒性。

Title: Auxiliary Discrminator Sequence Generative Adversarial Networks (ADSeqGAN) for Few Sample Molecule Generation

• Authors: Haocheng Tang, Jing Long, Junmei Wang
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.16446
• Pdf URL: https://arxiv.org/pdf/2502.16446
• Copy Paste: [[2502.16446]] Auxiliary Discrminator Sequence Generative Adversarial Networks (ADSeqGAN) for Few Sample Molecule Generation(https://arxiv.org/abs/2502.16446)
• Keywords: generation, generative
• Abstract: In this work, we introduce Auxiliary Discriminator Sequence Generative Adversarial Networks (ADSeqGAN), a novel approach for molecular generation in small-sample datasets. Traditional generative models often struggle with limited training data, particularly in drug discovery, where molecular datasets for specific therapeutic targets, such as nucleic acids binders and central nervous system (CNS) drugs, are scarce. ADSeqGAN addresses this challenge by integrating an auxiliary random forest classifier as an additional discriminator into the GAN framework, significantly improves molecular generation quality and class specificity. Our method incorporates pretrained generator and Wasserstein distance to enhance training stability and diversity. We evaluate ADSeqGAN on a dataset comprising nucleic acid-targeting and protein-targeting small molecules, demonstrating its superior ability to generate nucleic acid binders compared to baseline models such as SeqGAN, ORGAN, and MolGPT. Through an oversampling strategy, ADSeqGAN also significantly improves CNS drug generation, achieving a higher yield than traditional de novo models. Critical assessments, including docking simulations and molecular property analysis, confirm that ADSeqGAN-generated molecules exhibit strong binding affinities, enhanced chemical diversity, and improved synthetic feasibility. Overall, ADSeqGAN presents a novel framework for generative molecular design in data-scarce scenarios, offering potential applications in computational drug discovery. We have demonstrated the successful applications of ADSeqGAN in generating synthetic nucleic acid-targeting and CNS drugs in this work.
• 摘要：在这项工作中，我们引入了辅助歧视序列生成对抗网络（ADSEQGAN），这是一种在小样本数据集中分子产生的新方法。传统的生成模型通常在有限的训练数据中（尤其是在药物发现中）困难，在药物发现中，针对特定治疗靶标的分子数据集（例如核酸粘合剂和中枢神经系统（CNS）药物）很少。 Adseqgan通过将辅助随机森林分类器作为额外的歧视器整合到GAN框架中来应对这一挑战，从而显着提高了分子的产生质量和类别的特异性。我们的方法结合了预据的发电机和Wasserstein距离，以增强训练稳定性和多样性。我们在包含核酸靶向和蛋白质靶向的小分子的数据集上评估了Adseqgan，与基线模型（如Seqgan，Organ和Molgpt）相比，它证明了其产生核酸粘合剂的优势能力。通过过度采样策略，Adseqgan还显着改善了CNS药物的产生，比传统的从头模型的产量更高。关键评估，包括对接模拟和分子特性分析，证实Adseqgan生成的分子表现出强大的结合亲和力，增强的化学多样性和提高的合成可行性。总体而言，Adseqgan为数据筛选场景中的生成分子设计提供了新的框架，为计算药物发现提供了潜在的应用。我们已经证明了Adseqgan在这项工作中生成合成核酸靶向和CNS药物的成功应用。

Title: MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation

• Authors: Xuqi Mao, Zhenying He, X. Sean Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16454
• Pdf URL: https://arxiv.org/pdf/2502.16454
• Copy Paste: [[2502.16454]] MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation(https://arxiv.org/abs/2502.16454)
• Keywords: generation
• Abstract: Graph neural networks (GNNs) have become the state of the art for various graph-related tasks and are particularly prominent in heterogeneous graphs (HetGs). However, several issues plague this paradigm: first, the difficulty in fully utilizing long-range information, known as over-squashing; second, the tendency for excessive message-passing layers to produce indistinguishable representations, referred to as over-smoothing; and finally, the inadequacy of conventional MPNNs to train effectively on large sparse graphs. To address these challenges in deep neural networks for large-scale heterogeneous graphs, this paper introduces the Mamba-based Asynchronous Propagation Network (MAPN), which enhances the representation of heterogeneous sparse graphs. MAPN consists of two primary components: node sequence generation and semantic information aggregation. Node sequences are initially generated based on meta-paths through random walks, which serve as the foundation for a spatial state model that extracts essential information from nodes at various distances. It then asynchronously aggregates semantic information across multiple hops and layers, effectively preserving unique node characteristics and mitigating issues related to deep network degradation. Extensive experiments across diverse datasets demonstrate the effectiveness of MAPN in graph embeddings for various downstream tasks underscoring its substantial benefits for graph representation in large sparse heterogeneous graphs.
• 摘要：图形神经网络（GNN）已成为各种与图形相关任务的技术状态，在异质图（HETG）中尤为突出。但是，有几个问题困扰着这个范式：首先，完全利用长期信息（称为过度方面）的困难；其次，过多的消息层产生无法区分的表示的趋势，称为过度光滑；最后，传统的MPNN在大型稀疏图上有效训练的不足。为了应对大型异质图的深神经网络中的这些挑战，本文介绍了基于Mamba的异步传播网络（MAPN），从而增强了异质稀疏图的表示。 MAPN由两个主要组成部分组成：节点序列生成和语义信息聚集。节点序列最初是基于元路径通过随机步行而生成的，该步行是空间态模型的基础，该模型从各个距离的节点中提取基本信息。然后，它异步地汇总了多个啤酒花和层之间的语义信息，从而有效地保留了独特的节点特性并减轻与深网降解有关的问题。跨不同数据集的广泛实验证明了MAPN在图形嵌入中的有效性，用于各种下游任务，强调了其在大型稀疏异质图中图形表示的实质优势。

Title: Cross-domain Few-shot Object Detection with Multi-modal Textual Enrichment

• Authors: Zeyu Shangguan, Daniel Seita, Mohammad Rostami
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16469
• Pdf URL: https://arxiv.org/pdf/2502.16469
• Copy Paste: [[2502.16469]] Cross-domain Few-shot Object Detection with Multi-modal Textual Enrichment(https://arxiv.org/abs/2502.16469)
• Keywords: generation
• Abstract: Advancements in cross-modal feature extraction and integration have significantly enhanced performance in few-shot learning tasks. However, current multi-modal object detection (MM-OD) methods often experience notable performance degradation when encountering substantial domain shifts. We propose that incorporating rich textual information can enable the model to establish a more robust knowledge relationship between visual instances and their corresponding language descriptions, thereby mitigating the challenges of domain shift. Specifically, we focus on the problem of Cross-Domain Multi-Modal Few-Shot Object Detection (CDMM-FSOD) and introduce a meta-learning-based framework designed to leverage rich textual semantics as an auxiliary modality to achieve effective domain adaptation. Our new architecture incorporates two key components: (i) A multi-modal feature aggregation module, which aligns visual and linguistic feature embeddings to ensure cohesive integration across modalities. (ii) A rich text semantic rectification module, which employs bidirectional text feature generation to refine multi-modal feature alignment, thereby enhancing understanding of language and its application in object detection. We evaluate the proposed method on common cross-domain object detection benchmarks and demonstrate that it significantly surpasses existing few-shot object detection approaches.
• 摘要：在几次学习任务中，跨模式特征提取和集成的进步可以显着提高性能。但是，当遇到实质域移动时，当前的多模式对象检测（MM-OD）方法通常会出现显着的性能降解。我们建议，合并丰富的文本信息可以使模型能够在视觉实例及其相应的语言描述之间建立更强大的知识关系，从而减轻域转移的挑战。具体而言，我们专注于跨域多模式的几射击对象检测（CDMM-FSOD）的问题，并引入了一个基于元学习的框架，旨在利用丰富的文本语义作为辅助模式，以实现有效的域适应性。我们的新体系结构包含了两个关键组成部分：（i）多模式特征聚合模块，该模块将视觉和语言特征嵌入对齐，以确保跨模态的凝聚力整合。 （ii）一种丰富的文本语义整流模块，该模块采用双向文本特征生成来完善多模式特征对齐，从而增强对语言及其在对象检测中的应用的理解。我们在常见的跨域对象检测基准上评估了提出的方法，并证明它显着超过了现有的少数射击对象检测方法。

Title: Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control

• Authors: Jinbo Yan, Alan Zhao, Yixin Hu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16475
• Pdf URL: https://arxiv.org/pdf/2502.16475
• Copy Paste: [[2502.16475]] Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control(https://arxiv.org/abs/2502.16475)
• Keywords: generation
• Abstract: Single-image 3D generation has emerged as a prominent research topic, playing a vital role in virtual reality, 3D modeling, and digital content creation. However, existing methods face challenges such as a lack of multi-view geometric consistency and limited controllability during the generation process, which significantly restrict their usability. % To tackle these challenges, we introduce Dragen3D, a novel approach that achieves geometrically consistent and controllable 3D generation leveraging 3D Gaussian Splatting (3DGS). We introduce the Anchor-Gaussian Variational Autoencoder (Anchor-GS VAE), which encodes a point cloud and a single image into anchor latents and decode these latents into 3DGS, enabling efficient latent-space generation. To enable multi-view geometry consistent and controllable generation, we propose a Seed-Point-Driven strategy: first generate sparse seed points as a coarse geometry representation, then map them to anchor latents via the Seed-Anchor Mapping Module. Geometric consistency is ensured by the easily learned sparse seed points, and users can intuitively drag the seed points to deform the final 3DGS geometry, with changes propagated through the anchor latents. To the best of our knowledge, we are the first to achieve geometrically controllable 3D Gaussian generation and editing without relying on 2D diffusion priors, delivering comparable 3D generation quality to state-of-the-art methods.
• 摘要：单图3D代表已成为一个著名的研究主题，在虚拟现实，3D建模和数字内容创建中起着至关重要的作用。但是，现有方法面临挑战，例如缺乏多视图几何一致性和在生成过程中的可控性有限，这极大地限制了它们的可用性。 ％为了应对这些挑战，我们引入了Dragen3D，这是一种新颖的方法，可以实现几何一致且可控制的3D代利用3D高斯分裂（3DGS）。我们介绍了锚定 - 高斯变量自动编码器（Anchor-GS VAE），该变量编码器将点云和单个图像编码为锚固潜在，并将这些潜伏期解码为3DGS，从而实现了有效的潜在空间生成。为了启用多视图几何形状一致且可控制的生成，我们提出了一个种子点驱动的策略：首先生成稀疏的种子点作为粗几何表示，然后将它们映射到通过种子锚定映射模块固定潜在的。易于学到的稀疏种子点可以确保几何一致性，并且用户可以直观地拖动种子点以变形最终的3DGS几何形状，并通过锚固潜伏期传播变化。据我们所知，我们是第一个实现几何可控的3D高斯生成和编辑的人，而无需依赖2D扩散先验，从而提供了可比的3D生成质量与最先进的方法。

Title: On Computational Limits of FlowAR Models: Expressivity and Efficiency

• Authors: Chengyue Gong, Yekun Ke, Xiaoyu Li, Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song
• Subjects: cs.LG, cs.AI, cs.CC, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16490
• Pdf URL: https://arxiv.org/pdf/2502.16490
• Copy Paste: [[2502.16490]] On Computational Limits of FlowAR Models: Expressivity and Efficiency(https://arxiv.org/abs/2502.16490)
• Keywords: generative
• Abstract: The expressive power and computational complexity of deep visual generative models, such as flow-based and autoregressive (AR) models, have gained considerable interest for their wide-ranging applications in generative tasks. However, the theoretical characterization of their expressiveness through the lens of circuit complexity remains underexplored, particularly for the state-of-the-art architecture like FlowAR proposed by [Ren et al., 2024], which integrates flow-based and autoregressive mechanisms. This gap limits our understanding of their inherent computational limits and practical efficiency. In this study, we address this gap by analyzing the circuit complexity of the FlowAR architecture. We demonstrate that when the largest feature map produced by the FlowAR model has dimensions $n \times n \times c$, the FlowAR model is simulable by a family of threshold circuits $\mathsf{TC}^0$, which have constant depth $O(1)$ and polynomial width $\mathrm{poly}(n)$. This is the first study to rigorously highlight the limitations in the expressive power of FlowAR models. Furthermore, we identify the conditions under which the FlowAR model computations can achieve almost quadratic time. To validate our theoretical findings, we present efficient model variant constructions based on low-rank approximations that align with the derived criteria. Our work provides a foundation for future comparisons with other generative paradigms and guides the development of more efficient and expressive implementations.
• 摘要：深层视觉生成模型（例如基于流量和自回归（AR）模型）的表达能力和计算复杂性已对其在生成任务中的广泛应用引起了极大的兴趣。然而，通过电路复杂性镜头的表达性的理论表征仍然没有被忽视，特别是对于[Ren等，2024]提出的最先进的架构，该结构集成了基于流动和自动性机制。这一差距限制了我们对它们固有的计算限制和实际效率的理解。在这项研究中，我们通过分析流动体系结构的电路复杂性来解决这一差距。我们证明，当流量模型产生的最大特征图具有尺寸$ n \ times n \ times c $时，流量模型将通过一个阈值电路$ \ mathsf {tc}^0 $模拟，它们具有恒定的深度$ O（1）$和多项式宽度$ \ MATHRM {poly}（n）$。这是第一个严格强调流动模型表达能力的局限性的研究。此外，我们确定流动模型计算几乎达到二次时间的条件。为了验证我们的理论发现，我们基于与派生标准一致的低级别近似值提出有效的模型变体构造。我们的工作为将来与其他生成范式进行了比较奠定了基础，并指导开发更有效和表现力的实施。

Title: PMAT: Optimizing Action Generation Order in Multi-Agent Reinforcement Learning

• Authors: Kun Hu, Muning Wen, Xihuai Wang, Shao Zhang, Yiwei Shi, Minne Li, Minglong Li, Ying Wen
• Subjects: cs.LG, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2502.16496
• Pdf URL: https://arxiv.org/pdf/2502.16496
• Copy Paste: [[2502.16496]] PMAT: Optimizing Action Generation Order in Multi-Agent Reinforcement Learning(https://arxiv.org/abs/2502.16496)
• Keywords: generation
• Abstract: Multi-agent reinforcement learning (MARL) faces challenges in coordinating agents due to complex interdependencies within multi-agent systems. Most MARL algorithms use the simultaneous decision-making paradigm but ignore the action-level dependencies among agents, which reduces coordination efficiency. In contrast, the sequential decision-making paradigm provides finer-grained supervision for agent decision order, presenting the potential for handling dependencies via better decision order management. However, determining the optimal decision order remains a challenge. In this paper, we introduce Action Generation with Plackett-Luce Sampling (AGPS), a novel mechanism for agent decision order optimization. We model the order determination task as a Plackett-Luce sampling process to address issues such as ranking instability and vanishing gradient during the network training process. AGPS realizes credit-based decision order determination by establishing a bridge between the significance of agents' local observations and their decision credits, thus facilitating order optimization and dependency management. Integrating AGPS with the Multi-Agent Transformer, we propose the Prioritized Multi-Agent Transformer (PMAT), a sequential decision-making MARL algorithm with decision order optimization. Experiments on benchmarks including StarCraft II Multi-Agent Challenge, Google Research Football, and Multi-Agent MuJoCo show that PMAT outperforms state-of-the-art algorithms, greatly enhancing coordination efficiency.
• 摘要：由于多代理系统中复杂的相互依存关系，多代理增强学习（MARL）在协调代理方面面临挑战。大多数MALL算法都使用同时决策范式，但忽略了代理之间的动作水平依赖性，从而降低了协调效率。相比之下，顺序的决策范式为代理决策令提供了更细粒度的监督，从而通过更好的决策订单管理呈现了处理依赖性的潜力。但是，确定最佳决策顺序仍然是一个挑战。在本文中，我们用Plackett-luce采样（AGP）引入了动作生成，这是代理决策顺序优化的新型机制。我们将订单确定任务建模为一个私有的抽样过程，以解决网络培训过程中排名不稳定性和消失梯度等问题。 AGP通过在代理人本地观察值及其决策信用额度之间建立桥梁来实现基于信用的决策命令确定，从而促进订单优化和依赖管理。将AGP与多代理变压器整合在一起，我们提出了优先的多代理变压器（PMAT），这是一种具有决策顺序优化的顺序决策MARL算法。在包括Starcraft II多代理挑战，Google Research Football和多代理Mujoco在内的基准测试的实验表明，PMAT的表现优于最先进的算法，极大地提高了协调效率。

Title: Color Information-Based Automated Mask Generation for Detecting Underwater Atypical Glare Areas

• Authors: Mingyu Jeon, Yeonji Paeng, Sejin Lee
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.16538
• Pdf URL: https://arxiv.org/pdf/2502.16538
• Copy Paste: [[2502.16538]] Color Information-Based Automated Mask Generation for Detecting Underwater Atypical Glare Areas(https://arxiv.org/abs/2502.16538)
• Keywords: generation
• Abstract: Underwater diving assistance and safety support robots acquire real-time diver information through onboard underwater cameras. This study introduces a breath bubble detection algorithm that utilizes unsupervised K-means clustering, thereby addressing the high accuracy demands of deep learning models as well as the challenges associated with constructing supervised datasets. The proposed method fuses color data and relative spatial coordinates from underwater images, employs CLAHE to mitigate noise, and subsequently performs pixel clustering to isolate reflective regions. Experimental results demonstrate that the algorithm can effectively detect regions corresponding to breath bubbles in underwater images, and that the combined use of RGB, LAB, and HSV color spaces significantly enhances detection accuracy. Overall, this research establishes a foundation for monitoring diver conditions and identifying potential equipment malfunctions in underwater environments.
• 摘要：水下潜水援助和安全支持机器人通过船上的水下摄像头获取实时潜水员信息。这项研究介绍了一种利用无监督的K-均值聚类的气泡检测算法，从而解决了深度学习模型的高精度需求以及与构建监督数据集相关的挑战。提出的方法将颜色数据和相对空间坐标从水下图像融合，采用Clahe来减轻噪声，然后执行像素聚类以隔离反射区域。实验结果表明，该算法可以有效地检测与水下图像中呼吸气泡相对应的区域，并且RGB，LAB和HSV颜色空间的合并使用可显着提高检测准确性。总体而言，这项研究为监测潜水员条件并确定水下环境中潜在的设备故障奠定了基础。

Title: AdverX-Ray: Ensuring X-Ray Integrity Through Frequency-Sensitive Adversarial VAEs

• Authors: Francisco Caetano, Christiaan Viviers, Lena Filatova, Peter H. N. de With, Fons van der Sommen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16610
• Pdf URL: https://arxiv.org/pdf/2502.16610
• Copy Paste: [[2502.16610]] AdverX-Ray: Ensuring X-Ray Integrity Through Frequency-Sensitive Adversarial VAEs(https://arxiv.org/abs/2502.16610)
• Keywords: quality assessment
• Abstract: Ensuring the quality and integrity of medical images is crucial for maintaining diagnostic accuracy in deep learning-based Computer-Aided Diagnosis and Computer-Aided Detection (CAD) systems. Covariate shifts are subtle variations in the data distribution caused by different imaging devices or settings and can severely degrade model performance, similar to the effects of adversarial attacks. Therefore, it is vital to have a lightweight and fast method to assess the quality of these images prior to using CAD models. AdverX-Ray addresses this need by serving as an image-quality assessment layer, designed to detect covariate shifts effectively. This Adversarial Variational Autoencoder prioritizes the discriminator's role, using the suboptimal outputs of the generator as negative samples to fine-tune the discriminator's ability to identify high-frequency artifacts. Images generated by adversarial networks often exhibit severe high-frequency artifacts, guiding the discriminator to focus excessively on these components. This makes the discriminator ideal for this approach. Trained on patches from X-ray images of specific machine models, AdverX-Ray can evaluate whether a scan matches the training distribution, or if a scan from the same machine is captured under different settings. Extensive comparisons with various OOD detection methods show that AdverX-Ray significantly outperforms existing techniques, achieving a 96.2% average AUROC using only 64 random patches from an X-ray. Its lightweight and fast architecture makes it suitable for real-time applications, enhancing the reliability of medical imaging systems. The code and pretrained models are publicly available.
• 摘要：确保医学图像的质量和完整性对​​于维持基于深度学习的计算机辅助诊断和计算机辅助检测（CAD）系统的诊断准确性至关重要。协变量转移是由不同的成像设备或设置引起的数据分布的微妙变化，并且可能会严重降低模型性能，类似于对抗性攻击的影响。因此，在使用CAD模型之前，具有轻巧和快速的方法来评估这些图像的质量至关重要。 Adverx-Ray通过用作图像质量评估层来满足这一需求，该层旨在有效地检测协变量转移。这种对抗性变异自动编码器优先考虑歧视者的角色，使用发电机的次优输出作为负样本，以微调鉴别鉴定高频伪像的能力。对抗网络产生的图像通常表现出严重的高频伪像，从而指导歧视者过度专注于这些组件。这使得歧视者是这种方法的理想选择。 Adverx射线对特定机器型号的X射线图像进行了培训，可以评估扫描是否与训练分布相匹配，或者是否在不同的设置下捕获了同一机器的扫描。与各种OOD检测方法的广泛比较表明，Adverx射线显着胜过现有技术，仅使用X射线的64个随机斑块实现了96.2％的平均AUROC。它的轻巧且快速的体系结构使其适用于实时应用程序，从而增强了医学成像系统的可靠性。代码和验证的模型公开可用。

Title: Can Large Vision-Language Models Detect Images Copyright Infringement from GenAI?

• Authors: Qipan Xu, Zhenting Wang, Xiaoxiao He, Ligong Han, Ruixiang Tang
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16618
• Pdf URL: https://arxiv.org/pdf/2502.16618
• Copy Paste: [[2502.16618]] Can Large Vision-Language Models Detect Images Copyright Infringement from GenAI?(https://arxiv.org/abs/2502.16618)
• Keywords: generation, generative
• Abstract: Generative AI models, renowned for their ability to synthesize high-quality content, have sparked growing concerns over the improper generation of copyright-protected material. While recent studies have proposed various approaches to address copyright issues, the capability of large vision-language models (LVLMs) to detect copyright infringements remains largely unexplored. In this work, we focus on evaluating the copyright detection abilities of state-of-the-art LVLMs using a various set of image samples. Recognizing the absence of a comprehensive dataset that includes both IP-infringement samples and ambiguous non-infringement negative samples, we construct a benchmark dataset comprising positive samples that violate the copyright protection of well-known IP figures, as well as negative samples that resemble these figures but do not raise copyright concerns. This dataset is created using advanced prompt engineering techniques. We then evaluate leading LVLMs using our benchmark dataset. Our experimental results reveal that LVLMs are prone to overfitting, leading to the misclassification of some negative samples as IP-infringement cases. In the final section, we analyze these failure cases and propose potential solutions to mitigate the overfitting problem.
• 摘要：生成的AI模型以合成高质量内容的能力而闻名，引发了人们对受版权保护的材料不当产生的越来越关注。尽管最近的研究提出了解决版权问题的各种方法，但大型视觉模型（LVLMS）检测版权侵权的能力仍然很大程度上尚未得到探索。在这项工作中，我们专注于使用各种图像样本来评估最先进的LVLM的版权检测能力。认识到缺乏包括IP插入样本和模棱两可的非侵入负面样本的全面数据集，我们构建了一个基准数据集，其中包含违反众所周知IP的版权保护的正面​​样本以及与这些相似的负面样本的版权保护的基准数据集数字，但不会引起版权问题。该数据集是使用高级提示工程技术创建的。然后，我们使用基准数据集评估领先的LVLM。我们的实验结果表明，LVLM易于过度拟合，从而导致某些负样本作为IP插入案例的错误分类。在最后一部分中，我们分析了这些故障案例，并提出了潜在的解决方案以减轻过度拟合问题。

Title: Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines

• Authors: Xinwei Long, Zhiyuan Ma, Ermo Hua, Kaiyan Zhang, Biqing Qi, Bowen Zhou
• Subjects: cs.CV, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.16641
• Pdf URL: https://arxiv.org/pdf/2502.16641
• Copy Paste: [[2502.16641]] Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines(https://arxiv.org/abs/2502.16641)
• Keywords: generation, generative
• Abstract: Retrieval-augmented generation (RAG) has emerged to address the knowledge-intensive visual question answering (VQA) task. Current methods mainly employ separate retrieval and generation modules to acquire external knowledge and generate answers, respectively. We propose ReAuSE, an alternative to the previous RAG model for the knowledge-based VQA task, which seamlessly integrates knowledge retriever into the generative multi-modal large language model, serving as a built-in search engine. Specifically, our model functions both as a generative retriever and an accurate answer generator. It not only helps retrieve documents from the knowledge base by producing identifiers for each document, but it also answers visual questions based on the retrieved documents. Furthermore, we propose a reinforced retrieval calibration module from relevance feedback to improve retrieval performance and align with the preferences for accurate answer generation. Extensive experiments on two representative OKVQA and A-OKVQA datasets demonstrate significant improvements ranging from 2.9\% to 9.6\% across all evaluation metrics when compared to strong baselines.
• 摘要：检索增强的一代（RAG）已出现，以解决知识密集的视觉问题回答（VQA）任务。当前方法主要采用单独的检索和发电模块来获取外部知识并产生答案。我们提出了Reause，这是基于知识的VQA任务的先前抹布模型的替代方法，该模型将知识回收者无缝地集成到生成的多模式大语言模型中，并用作内置的搜索引擎。具体而言，我们的模型既可以作为生成试器和准确的答案生成器发挥作用。它不仅通过为每个文档生成标识符来帮助从知识库中检索文档，而且还根据检索到的文档回答视觉问题。此外，我们提出了一个从相关反馈的加强检索校准模块，以提高检索性能并与偏好保持准确的答案生成。与强质基线相比，对两个代表性OKVQA和A-OKVQA数据集进行了广泛的实验表明，所有评估指标的显着改善范围从2.9 \％到9.6 \％。

Title: Hierarchical Semantic Compression for Consistent Image Semantic Restoration

• Authors: Shengxi Li, Zifu Zhang, Mai Xu, Lai Jiang, Yufan Liu, Ce Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16799
• Pdf URL: https://arxiv.org/pdf/2502.16799
• Copy Paste: [[2502.16799]] Hierarchical Semantic Compression for Consistent Image Semantic Restoration(https://arxiv.org/abs/2502.16799)
• Keywords: restoration, generative
• Abstract: The emerging semantic compression has been receiving increasing research efforts most recently, capable of achieving high fidelity restoration during compression, even at extremely low bitrates. However, existing semantic compression methods typically combine standard pipelines with either pre-defined or high-dimensional semantics, thus suffering from deficiency in compression. To address this issue, we propose a novel hierarchical semantic compression (HSC) framework that purely operates within intrinsic semantic spaces from generative models, which is able to achieve efficient compression for consistent semantic restoration. More specifically, we first analyse the entropy models for the semantic compression, which motivates us to employ a hierarchical architecture based on a newly developed general inversion encoder. Then, we propose the feature compression network (FCN) and semantic compression network (SCN), such that the middle-level semantic feature and core semantics are hierarchically compressed to restore both accuracy and consistency of image semantics, via an entropy model progressively shared by channel-wise context. Experimental results demonstrate that the proposed HSC framework achieves the state-of-the-art performance on subjective quality and consistency for human vision, together with superior performances on machine vision tasks given compressed bitstreams. This essentially coincides with human visual system in understanding images, thus providing a new framework for future image/video compression paradigms. Our code shall be released upon acceptance.
• 摘要：新兴的语义压缩最近正在接受越来越多的研究工作，即使在极低的比特率下，也能够在压缩过程中实现高保真恢复。但是，现有的语义压缩方法通常将标准管道与预定义或高维语义相结合，因此遭受压缩的缺乏。为了解决这个问题，我们提出了一种新型的分层语义压缩（HSC）框架，该框架纯粹是在生成模型的内在语义空间内运行的，该模型能够实现有效的压缩以实现一致的语义恢复。更具体地说，我们首先分析语义压缩的熵模型，这促使我们采用基于新开发的一般反转编码器的层次结构。然后，我们提出特征压缩网络（FCN）和语义压缩网络（SCN），使得中级语义特征和核心语义在层次上被压缩，以恢复图像语义的准确性和一致性，这是通过熵模型逐渐共享的渠道的上下文。实验结果表明，所提出的HSC框架在人类视觉的主观质量和一致性上实现了最新的性能，以及在给定压缩bitstreams的机器视觉任务上的出色表现。这本质上与人类的视觉系统一致理解图像，从而为未来的图像/视频压缩范式提供了一个新的框架。我们的守则应在接受后发布。

Title: CLIP-SENet: CLIP-based Semantic Enhancement Network for Vehicle Re-identification

• Authors: Liping Lu, Zihao Fu, Duanfeng Chu, Wei Wang, Bingrong Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16815
• Pdf URL: https://arxiv.org/pdf/2502.16815
• Copy Paste: [[2502.16815]] CLIP-SENet: CLIP-based Semantic Enhancement Network for Vehicle Re-identification(https://arxiv.org/abs/2502.16815)
• Keywords: generation
• Abstract: Vehicle re-identification (Re-ID) is a crucial task in intelligent transportation systems (ITS), aimed at retrieving and matching the same vehicle across different surveillance cameras. Numerous studies have explored methods to enhance vehicle Re-ID by focusing on semantic enhancement. However, these methods often rely on additional annotated information to enable models to extract effective semantic features, which brings many limitations. In this work, we propose a CLIP-based Semantic Enhancement Network (CLIP-SENet), an end-to-end framework designed to autonomously extract and refine vehicle semantic attributes, facilitating the generation of more robust semantic feature representations. Inspired by zero-shot solutions for downstream tasks presented by large-scale vision-language models, we leverage the powerful cross-modal descriptive capabilities of the CLIP image encoder to initially extract general semantic information. Instead of using a text encoder for semantic alignment, we design an adaptive fine-grained enhancement module (AFEM) to adaptively enhance this general semantic information at a fine-grained level to obtain robust semantic feature representations. These features are then fused with common Re-ID appearance features to further refine the distinctions between vehicles. Our comprehensive evaluation on three benchmark datasets demonstrates the effectiveness of CLIP-SENet. Our approach achieves new state-of-the-art performance, with 92.9% mAP and 98.7% Rank-1 on VeRi-776 dataset, 90.4% Rank-1 and 98.7% Rank-5 on VehicleID dataset, and 89.1% mAP and 97.9% Rank-1 on the more challenging VeRi-Wild dataset.
• 摘要：车辆重新识别（RE-ID）是智能运输系统（ITS）的至关重要任务，旨在通过不同的监视摄像机检索和匹配同一车辆。许多研究通过着眼于语义增强来探索增强车辆重新ID的方法。但是，这些方法通常依靠其他注释的信息来使模型提取有效的语义特征，从而带来许多局限性。在这项工作中，我们提出了一个基于夹子的语义增强网络（剪辑 - 磁带），该网络是一个旨在自主提取和完善车辆语义属性的端到端框架，从而促进了更强大的语义特征表示。受到大规模视觉语言模型提出的下游任务的零击解决方案的启发，我们利用了剪辑图像编码器的功能强大的跨模式描述功能来最初提取一般的语义信息。我们设计一个自适应的细粒增强模块（AFEM），而不是使用文本编码器进行语义对齐，以在细粒度级别自适应增强此一般语义信息，以获得强大的语义特征表示。然后将这些功能与常见的重新外观特征融合在一起，以进一步完善车辆之间的区别。我们对三个基准数据集的全面评估证明了剪贴仪的有效性。我们的方法实现了新的最先进性能，在Veri-776数据集中拥有92.9％的地图和98.7％的排名，90.4％的排名1和98.7％的ParthID数据集排名-5，地图89.1％和97.9 ％排名在更具挑战性的Veri-Wild数据集中。

Title: Fast, Accurate Manifold Denoising by Tunneling Riemannian Optimization

• Authors: Shiyu Wang, Mariam Avagyan, Yihan Shen, Arnaud Lamy, Tingran Wang, Szabolcs Márka, Zsuzsa Márka, John Wright
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2502.16819
• Pdf URL: https://arxiv.org/pdf/2502.16819
• Copy Paste: [[2502.16819]] Fast, Accurate Manifold Denoising by Tunneling Riemannian Optimization(https://arxiv.org/abs/2502.16819)
• Keywords: generation
• Abstract: Learned denoisers play a fundamental role in various signal generation (e.g., diffusion models) and reconstruction (e.g., compressed sensing) architectures, whose success derives from their ability to leverage low-dimensional structure in data. Existing denoising methods, however, either rely on local approximations that require a linear scan of the entire dataset or treat denoising as generic function approximation problems, often sacrificing efficiency and interpretability. We consider the problem of efficiently denoising a new noisy data point sampled from an unknown $d$-dimensional manifold $M \in \mathbb{R}^D$, using only noisy samples. This work proposes a framework for test-time efficient manifold denoising, by framing the concept of "learning-to-denoise" as "learning-to-optimize". We have two technical innovations: (i) online learning methods which learn to optimize over the manifold of clean signals using only noisy data, effectively "growing" an optimizer one sample at a time. (ii) mixed-order methods which guarantee that the learned optimizers achieve global optimality, ensuring both efficiency and near-optimal denoising performance. We corroborate these claims with theoretical analyses of both the complexity and denoising performance of mixed-order traversal. Our experiments on scientific manifolds demonstrate significantly improved complexity-performance tradeoffs compared to nearest neighbor search, which underpins existing provable denoising approaches based on exhaustive search.
• 摘要：学识渊博的DeNoiser在各种信号产生（例如扩散模型）和重建（例如压缩传感）架构中起着基本作用，它们的成功源于其利用数据中低维结构的能力。但是，现有的denoising方法要么依赖于需要对整个数据集进行线性扫描的局部近似值，要么依赖于将denoising视为通用函数近似问题，通常会牺牲效率和解释性。我们认为有效地从未知的$ d $ d $ dimensional歧管$ m \ in \ mathbb {r}^d $中采样了一个新的嘈杂数据点的问题，仅使用嘈杂的样本。这项工作提出了一个通过将“学习到denoise”的概念构建为“学习至上”的概念，提出了一个用于测试时间有效的歧管。我们有两项技术创新：（i）在线学习方法，仅使用嘈杂的数据学会优化对清洁信号的多种多样，从而有效地“增长”优化器一个样本。 （ii）确保学习优化者实现全球最优性，确保效率和近乎最佳的降解性能的混合阶方法。我们通过理论分析对这些主张的复杂性和混合遍历性能的质疑来证实这些主张。与最近的邻居搜索相比，我们对科学歧管的实验表明，与最近的邻居搜索相比，复杂性 - 表现的权衡得到了显着改善，这基础是基于详尽搜索的现有可证明的Denoising方法。

Title: Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization

• Authors: Taeyoung Yun, Kiyoung Om, Jaewoo Lee, Sujin Yun, Jinkyoo Park
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.16824
• Pdf URL: https://arxiv.org/pdf/2502.16824
• Copy Paste: [[2502.16824]] Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization(https://arxiv.org/abs/2502.16824)
• Keywords: generative
• Abstract: Optimizing high-dimensional and complex black-box functions is crucial in numerous scientific applications. While Bayesian optimization (BO) is a powerful method for sample-efficient optimization, it struggles with the curse of dimensionality and scaling to thousands of evaluations. Recently, leveraging generative models to solve black-box optimization problems has emerged as a promising framework. However, those methods often underperform compared to BO methods due to limited expressivity and difficulty of uncertainty estimation in high-dimensional spaces. To overcome these issues, we introduce \textbf{DiBO}, a novel framework for solving high-dimensional black-box optimization problems. Our method iterates two stages. First, we train a diffusion model to capture the data distribution and an ensemble of proxies to predict function values with uncertainty quantification. Second, we cast the candidate selection as a posterior inference problem to balance exploration and exploitation in high-dimensional spaces. Concretely, we fine-tune diffusion models to amortize posterior inference. Extensive experiments demonstrate that our method outperforms state-of-the-art baselines across various synthetic and real-world black-box optimization tasks. Our code is publicly available \href{this https URL}{here}
• 摘要：在众多科学应用中，优化高维和复杂的黑盒功能至关重要。虽然贝叶斯优化（BO）是样本有效优化的有力方法，但它与维度的诅咒和缩放为数千次评估斗争。最近，利用生成模型解决黑盒优化问题已成为一个有希望的框架。但是，由于表现力有限和高维空间中不确定性估计的难度，这些方法通常与BO方法相比通常不足。为了克服这些问题，我们介绍了\ textbf {dibo}，这是一个解决高维黑盒优化问题的新颖框架。我们的方法迭代两个阶段。首先，我们训练一个扩散模型以捕获数据分布和代理集合，以预测不确定性定量的功能值。其次，我们将候选选择作为后推理问题，以平衡高维空间中的探索和剥削。具体而言，我们微调扩散模型以摊销后推理。广泛的实验表明，我们的方法在各种合成和现实的黑盒优化任务上优于最先进的基线。我们的代码是公开可用的\ href {this https url} {tere}

Title: Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models

• Authors: Yaqi Sun, Kyohei Atarashi, Koh Takeuchi, Hisashi Kashima
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16842
• Pdf URL: https://arxiv.org/pdf/2502.16842
• Copy Paste: [[2502.16842]] Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models(https://arxiv.org/abs/2502.16842)
• Keywords: generation
• Abstract: Large Vision-Language Models (LVLMs) integrate image encoders with Large Language Models (LLMs) to process multi-modal inputs and perform complex visual tasks. However, they often generate hallucinations by describing non-existent objects or attributes, compromising their reliability. This study analyzes hallucination patterns in image captioning, showing that not all tokens in the generation process are influenced by image input and that image dependency can serve as a useful signal for hallucination detection. To address this, we develop an automated pipeline to identify hallucinated objects and train a token-level classifier using hidden representations from parallel inference passes-with and without image input. Leveraging this classifier, we introduce a decoding strategy that effectively controls hallucination rates in image captioning at inference time.
• 摘要：大型视觉模型（LVLM）将图像编码器与大语言模型（LLMS）集成在一起，以处理多模式输入并执行复杂的视觉任务。但是，它们通常通过描述不存在的对象或属性来产生幻觉，从而损害其可靠性。这项研究分析了图像字幕中的幻觉模式，表明并非生成过程中的所有令牌都受图像输入的影响，并且图像依赖性可以作为幻觉检测的有用信号。为了解决这个问题，我们开发了一条自动管道来识别幻觉对象，并使用并行推理中的隐藏表示形式训练令牌级别的分类器，而没有图像输入。利用此分类器，我们引入了一种解码策略，该策略可有效控制推理时图像字幕中的幻觉率。

Title: Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning

• Authors: Jiaheng Li, Donghe Li, Ye Yang, Huan Xi, Yu Xiao, Li Sun, Dou An, Qingyu Yang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16896
• Pdf URL: https://arxiv.org/pdf/2502.16896
• Copy Paste: [[2502.16896]] Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning(https://arxiv.org/abs/2502.16896)
• Keywords: generation
• Abstract: The growing penetration of renewable energy sources in power systems has increased the complexity and uncertainty of load forecasting, especially for integrated energy systems with multiple energy carriers. Traditional forecasting methods heavily rely on historical data and exhibit limited transferability across different scenarios, posing significant challenges for emerging applications in smart grids and energy internet. This paper proposes the TSLLM-Load Forecasting Mechanism, a novel zero-shot load forecasting framework based on large language models (LLMs) to address these challenges. The framework consists of three key components: a data preprocessing module that handles multi-source energy load data, a time series prompt generation module that bridges the semantic gap between energy data and LLMs through multi-task learning and similarity alignment, and a prediction module that leverages pre-trained LLMs for accurate forecasting. The framework's effectiveness was validated on a real-world dataset comprising load profiles from 20 Australian solar-powered households, demonstrating superior performance in both conventional and zero-shot scenarios. In conventional testing, our method achieved a Mean Squared Error (MSE) of 0.4163 and a Mean Absolute Error (MAE) of 0.3760, outperforming existing approaches by at least 8\%. In zero-shot prediction experiments across 19 households, the framework maintained consistent accuracy with a total MSE of 11.2712 and MAE of 7.6709, showing at least 12\% improvement over current methods. The results validate the framework's potential for accurate and transferable load forecasting in integrated energy systems, particularly beneficial for renewable energy integration and smart grid applications.
• 摘要：可再生能源在电力系统中的渗透不断增长增加了负载预测的复杂性和不确定性，尤其是对于具有多个能量载体的集成能量系统。传统的预测方法在很大程度上依赖于历史数据，并且在不同情况下的可转移性有限，这对智能电网和能源互联网中新兴应用构成了重大挑战。本文提出了TSLLM载荷预测机制，这是一种基于大语言模型（LLMS）的新型零弹负载预测框架，以应对这些挑战。该框架由三个关键组成部分组成：一个处理多源能量负载数据的数据预处理模块，这是一个时间序列提示生成模块，该模块通过多任务数据和LLM之间的语义差距通过多任务学习和相似性统一，以及一个预测模块这利用预先训练的LLM进行准确的预测。该框架的有效性已在一个现实世界中的数据集上进行了验证，该数据集包括来自20个澳大利亚太阳能家庭的负载概况，在传统和零摄影方案中都表现出了卓越的性能。在常规测试中，我们的方法达到了平均平方误差（MSE）为0.4163，平均绝对误差（MAE）为0.3760，表现的表现至少超过8％。在19个家庭的零拍预测实验中，该框架保持一致的准确性，总MSE为11.2712，MAE为7.6709，显示出比当前方法至少提高12 \％。结果证明了该框架在集成能源系统中准确且可转移的负载预测的潜力，特别有益于可再生能源集成和智能电网应用。

Title: Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinment

• Authors: Suchae Jeong, Inseong Choi, Youngsik Yun, Jihie Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16902
• Pdf URL: https://arxiv.org/pdf/2502.16902
• Copy Paste: [[2502.16902]] Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinment(https://arxiv.org/abs/2502.16902)
• Keywords: generation
• Abstract: Text-to-Image models, including Stable Diffusion, have significantly improved in generating images that are highly semantically aligned with the given prompts. However, existing models may fail to produce appropriate images for the cultural concepts or objects that are not well known or underrepresented in western cultures, such as `hangari' (Korean utensil). In this paper, we propose a novel approach, Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (Culture-TRIP), which refines the prompt in order to improve the alignment of the image with such culture nouns in text-to-image models. Our approach (1) retrieves cultural contexts and visual details related to the culture nouns in the prompt and (2) iteratively refines and evaluates the prompt based on a set of cultural criteria and large language models. The refinement process utilizes the information retrieved from Wikipedia and the Web. Our user survey, conducted with 66 participants from eight different countries demonstrates that our proposed approach enhances the alignment between the images and the prompts. In particular, C-TRIP demonstrates improved alignment between the generated images and underrepresented culture nouns. Resource can be found at this https URL.
• 摘要：文本到图像模型，包括稳定的扩散，在生成与给定提示相符的高度对齐的图像方面已显着改善。但是，现有模型可能无法为在西方文化中（例如“ hangari''（韩国餐具））（韩国餐具）等文化概念或物体（不明显）的文化概念或物体产生适当的图像。在本文中，我们提出了一种新颖的方法，具有迭代及时改进（文化 - 旅行）的文化意识，文本到图像的生成，该方法是完善了提示，以改善图像的对齐方式 - 图像模型。我们的方法（1）在提示中检索了与文化名词相关的文化背景和视觉细节，（2）迭代完善并根据一组文化标准和大语言模型来评估提示。改进过程利用了从Wikipedia和Web检索的信息。我们的用户调查与来自八个不同国家 /地区的66名参与者进行的用户调查表明，我们提出的方法可以增强图像和提示之间的一致性。尤其是，C贸易证明了生成的图像和代表性不足的培养名词之间的比对得到了改善。资源可以在此HTTPS URL上找到。

Title: HVIS: A Human-like Vision and Inference System for Human Motion Prediction

• Authors: Kedi Lyu, Haipeng Chen, Zhenguang Liu, Yifang Yin, Yukang Lin, Yingying Jiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16913
• Pdf URL: https://arxiv.org/pdf/2502.16913
• Copy Paste: [[2502.16913]] HVIS: A Human-like Vision and Inference System for Human Motion Prediction(https://arxiv.org/abs/2502.16913)
• Keywords: generation
• Abstract: Grasping the intricacies of human motion, which involve perceiving spatio-temporal dependence and multi-scale effects, is essential for predicting human motion. While humans inherently possess the requisite skills to navigate this issue, it proves to be markedly more challenging for machines to emulate. To bridge the gap, we propose the Human-like Vision and Inference System (HVIS) for human motion prediction, which is designed to emulate human observation and forecast future movements. HVIS comprises two components: the human-like vision encode (HVE) module and the human-like motion inference (HMI) module. The HVE module mimics and refines the human visual process, incorporating a retina-analog component that captures spatiotemporal information separately to avoid unnecessary crosstalk. Additionally, a visual cortex-analogy component is designed to hierarchically extract and treat complex motion features, focusing on both global and local features of human poses. The HMI is employed to simulate the multi-stage learning model of the human brain. The spontaneous learning network simulates the neuronal fracture generation process for the adversarial generation of future motions. Subsequently, the deliberate learning network is optimized for hard-to-train joints to prevent misleading learning. Experimental results demonstrate that our method achieves new state-of-the-art performance, significantly outperforming existing methods by 19.8% on Human3.6M, 15.7% on CMU Mocap, and 11.1% on G3D.
• 摘要：掌握人类运动的复杂性，涉及人们对时空的依赖性和多尺度效应，对于预测人类运动至关重要。尽管人类天生就具备解决这个问题的必要技能，但事实证明，对于机器而言，它显着挑战。为了弥合差距，我们提出了人类运动预测的人类视力和推理系统（HVI），该预测旨在模仿人类的观察和预测未来的运动。 HVIS包括两个组成部分：类似人类的视觉编码（HVE）模块和类人类运动推断（HMI）模块。 HVE模块模拟并完善了人类的视觉过程，并结合了视网膜 - 分析组件，该组件分别捕获时空信息，以避免不必要的串扰。此外，视觉皮层 - 分析成分旨在层次提取和处理复杂的运动特征，重点是人类姿势的全球和局部特征。 HMI用于模拟人脑的多阶段学习模型。自发学习网络模拟了对抗性未来动作的神经元断裂生成过程。随后，故意学习网络将针对难以训练的关节进行优化，以防止误导性学习。实验结果表明，我们的方法实现了新的最先进性能，对人类36m，CMU MOCAP的15.7％的表现显着优于19.8％，而G3D的方法则优于15.7％，而G3D的方法则优于11.1％。

Title: Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model

• Authors: Kang Fu, Huiyu Duan, Zicheng Zhang, Xiaohong Liu, Xiongkuo Min, Jia Wang, Guangtao Zhai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16915
• Pdf URL: https://arxiv.org/pdf/2502.16915
• Copy Paste: [[2502.16915]] Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model(https://arxiv.org/abs/2502.16915)
• Keywords: generation, quality assessment
• Abstract: Recent advancements in text-to-image (T2I) generation have spurred the development of text-to-3D asset (T23DA) generation, leveraging pretrained 2D text-to-image diffusion models for text-to-3D asset synthesis. Despite the growing popularity of text-to-3D asset generation, its evaluation has not been well considered and studied. However, given the significant quality discrepancies among various text-to-3D assets, there is a pressing need for quality assessment models aligned with human subjective judgments. To tackle this challenge, we conduct a comprehensive study to explore the T23DA quality assessment (T23DAQA) problem in this work from both subjective and objective perspectives. Given the absence of corresponding databases, we first establish the largest text-to-3D asset quality assessment database to date, termed the AIGC-T23DAQA database. This database encompasses 969 validated 3D assets generated from 170 prompts via 6 popular text-to-3D asset generation models, and corresponding subjective quality ratings for these assets from the perspectives of quality, authenticity, and text-asset correspondence, respectively. Subsequently, we establish a comprehensive benchmark based on the AIGC-T23DAQA database, and devise an effective T23DAQA model to evaluate the generated 3D assets from the aforementioned three perspectives, respectively.
• 摘要：文本到图像（T2I）一代的最新进展促使文本到3D资产（T23DA）生成的发展，利用了预估计的2D文本对图像扩散模型来进行文本到3D资产合成。尽管文本到3D资产的产生越来越受欢迎，但其评估尚未得到很好的考虑和研究。但是，鉴于各种文本到3D资产之间的质量差异很大，因此迫切需要质量评估模型与人类主观判断保持一致。为了应对这一挑战，我们从主观和客观的角度探讨了这项工作中T23DA质量评估（T23DAQA）问题的全面研究。鉴于没有相应的数据库，我们首先建立了迄今为止称为AIGC-T23DAQA数据库的最大文本至3D资产质量评估数据库。该数据库包括969个通过6个流行的文本到3D资产生成模型从170个提示生成的3D资产，并分别从质量，真实性和文本资产信函的角度分别对这些资产的相应主观质量评级。随后，我们基于AIGC-T23DAQA数据库建立了一个全面的基准，并设计了有效的T23DAQA模型，分别从上述三个角度评估了生成的3D资产。

Title: MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection

• Authors: Farzad Beizaee, Gregory Lodygensky, Christian Desrosiers, Jose Dolz
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.16943
• Pdf URL: https://arxiv.org/pdf/2502.16943
• Copy Paste: [[2502.16943]] MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection(https://arxiv.org/abs/2502.16943)
• Keywords: generative
• Abstract: Unsupervised anomaly detection in brain images is crucial for identifying injuries and pathologies without access to labels. However, the accurate localization of anomalies in medical images remains challenging due to the inherent complexity and variability of brain structures and the scarcity of annotated abnormal data. To address this challenge, we propose a novel approach that incorporates masking within diffusion models, leveraging their generative capabilities to learn robust representations of normal brain anatomy. During training, our model processes only normal brain MRI scans and performs a forward diffusion process in the latent space that adds noise to the features of randomly-selected patches. Following a dual objective, the model learns to identify which patches are noisy and recover their original features. This strategy ensures that the model captures intricate patterns of normal brain structures while isolating potential anomalies as noise in the latent space. At inference, the model identifies noisy patches corresponding to anomalies and generates a normal counterpart for these patches by applying a reverse diffusion process. Our method surpasses existing unsupervised anomaly detection techniques, demonstrating superior performance in generating accurate normal counterparts and localizing anomalies. The code is available at hhttps://github.com/farzad-bz/MAD-AD.
• 摘要：大脑图像中无监督的异常检测对于鉴定损伤和病理而无需获得标签至关重要。但是，由于大脑结构的固有复杂性和可变性以及带注释的异常数据的稀缺性，因此在医学图像中的准确定位仍然具有挑战性。为了应对这一挑战，我们提出了一种新颖的方法，该方法将遮罩纳入扩散模型中，利用它们的生成能力来学习正常脑解剖结构的稳健表示。在训练过程中，我们的模型过程仅进行正常的大脑MRI扫描，并在潜在空间中执行正向扩散过程，从而为随机选择的斑块的特征增加噪声。按照双重目标，该模型学会了确定哪些补丁是嘈杂的，并恢复了其原始功能。该策略确保模型捕获了普通大脑结构的复杂模式，同时将潜在异常隔离为潜在空间中的噪声。在推断时，该模型识别与异常相对应的嘈杂贴片，并通过应用反向扩散过程为这些贴片生成正常对应物。我们的方法超过了现有的无监督异常检测技术，表明在产生准确的正常对应物和定位异常方面表现出色。该代码可在hhttps：//github.com/farzad-bz/mad-ad上找到。

Title: Autoregressive Image Generation Guided by Chains of Thought

• Authors: Miaomiao Cai, Guanjie Wang, Wei Li, Zhijun Tu, Hanting Chen, Shaohui Lin, Jie Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16965
• Pdf URL: https://arxiv.org/pdf/2502.16965
• Copy Paste: [[2502.16965]] Autoregressive Image Generation Guided by Chains of Thought(https://arxiv.org/abs/2502.16965)
• Keywords: generation
• Abstract: In the field of autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases. However, directly applying LLMs to complex image generation can struggle with reconstructing the structure and details of the image, impacting the accuracy and stability of generation. Additionally, the 'next-token prediction' paradigm in the AR model does not align with the contextual scanning and logical reasoning processes involved in human visual perception, limiting effective image generation. Chain-of-Thought (CoT), as a key reasoning capability of LLMs, utilizes reasoning prompts to guide the model, improving reasoning performance on complex natural language process (NLP) tasks, enhancing accuracy and stability of generation, and helping the model maintain contextual coherence and logical consistency, similar to human reasoning. Inspired by CoT from the field of NLP, we propose autoregressive Image Generation with Thoughtful Reasoning (IGTR) to enhance autoregressive image generation. IGTR adds reasoning prompts without modifying the model structure or raster generation order. Specifically, we design specialized image-related reasoning prompts for AR image generation to simulate the human reasoning process, which enhances contextual reasoning by allowing the model to first perceive overall distribution information before generating the image, and improve generation stability by increasing the inference steps. Compared to the AR method without prompts, our method shows outstanding performance and achieves an approximate improvement of 20%.
• 摘要：在自回旋（AR）图像生成领域中，基于LLM的“下一步预测”范式的模型通过减少电感偏差显示出与扩散模型相当的性能。但是，将LLM直接应用于复杂的图像生成可能会在重建图像的结构和细节上挣扎，从而影响发电的准确性和稳定性。此外，AR模型中的“下一预测”范式与人类视觉感知所涉及的上下文扫描和逻辑推理过程不符，从而限制了有效的图像产生。作为LLM的关键推理能力，经过思考链（COT）利用推理提示指导模型，改善了复杂自然语言过程（NLP）任务的推理性能，提高了生成的准确性和稳定性，并帮助模型维护上下文连贯性和逻辑一致性，类似于人类推理。受NLP领域的COT的启发，我们提出了具有周到推理（IGTR）的自回归图像生成，以增强自回归图像的产生。 IGTR添加了推理提示，而无需修改模型结构或栅格生成顺序。具体而言，我们为AR图像生成设计了与图像相关的专业推理提示，以模拟人类的推理过程，从而通过允许模型在生成图像之前先感知总体分布信息来增强上下文推理，并通过增加推理步骤来提高生成稳定性。与没有提示的AR方法相比，我们的方法显示出出色的性能，并实现了20％的近似改善。

Title: TraFlow: Trajectory Distillation on Pre-Trained Rectified Flow

• Authors: Zhangkai Wu, Xuhui Fan, Hongyu Wu, Longbing Cao
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16972
• Pdf URL: https://arxiv.org/pdf/2502.16972
• Copy Paste: [[2502.16972]] TraFlow: Trajectory Distillation on Pre-Trained Rectified Flow(https://arxiv.org/abs/2502.16972)
• Keywords: generation
• Abstract: Majorities of distillation methods on pre-trained diffusion models or on pre-trained rectified flow, focus on either the distillation outputs or the trajectories between random noises and clean images to speed up sample generations from pre-trained models. In those trajectory-based distillation methods, consistency distillation requires the self-consistent trajectory projection to regulate the trajectory, which might avoid the common ODE approximation error {while still be concerning about sampling efficiencies}. At the same time, rectified flow distillations enforce straight trajectory for fast sampling, although an ODE solver is still required. In this work, we propose a trajectory distillation method, \modelname, that enjoys the benefits of both and enables few-step generations. TraFlow adopts the settings of consistency trajectory models, and further enforces the properties of self-consistency and straightness throughout the entire trajectory. These two properties are pursued by reaching a balance with following three targets: (1) reconstruct the output from pre-trained models; (2) learn the amount of changes by pre-trained models; (3) satisfy the self-consistency over its trajectory. Extensive experimental results have shown the effectiveness of our proposed method.
• 摘要：大多数蒸馏方法在预训练的扩散模型或预训练的整流流上，专注于蒸馏输出或随机噪声和清洁图像之间的轨迹，以加快从预训练的模型中加快样品世代相传的速度。在那些基于轨迹的蒸馏方法中，一致性蒸馏需要自洽的轨迹投影以调节轨迹，这可能避免了常见的ODE近似误差{虽然仍然与采样效率有关}。同时，尽管仍然需要ode求解器，但纠正的流动蒸馏强迫直轨迹进行快速采样。在这项工作中，我们提出了一种轨迹蒸馏方法\ modelname，它享有两者的好处，并可以使几代人产生。 Traflow采用了一致性轨迹模型的设置，并在整个轨迹中进一步实施了自符势和直率的属性。通过与以下三个目标达到平衡来追求这两个属性：（1）重建预训练模型的输出； （2）通过预先训练的模型了解更改的数量； （3）满足其轨迹上的自洽性。广泛的实验结果表明了我们提出的方法的有效性。

Title: PQDAST: Depth-Aware Arbitrary Style Transfer for Games via Perceptual Quality-Guided Distillation

• Authors: Eleftherios Ioannou, Steve Maddock
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.16996
• Pdf URL: https://arxiv.org/pdf/2502.16996
• Copy Paste: [[2502.16996]] PQDAST: Depth-Aware Arbitrary Style Transfer for Games via Perceptual Quality-Guided Distillation(https://arxiv.org/abs/2502.16996)
• Keywords: generation
• Abstract: Artistic style transfer is concerned with the generation of imagery that combines the content of an image with the style of an artwork. In the realm of computer games, most work has focused on post-processing video frames. Some recent work has integrated style transfer into the game pipeline, but it is limited to single styles. Integrating an arbitrary style transfer method into the game pipeline is challenging due to the memory and speed requirements of games. We present PQDAST, the first solution to address this. We use a perceptual quality-guided knowledge distillation framework and train a compressed model using the FLIP evaluator, which substantially reduces both memory usage and processing time with limited impact on stylisation quality. For better preservation of depth and fine details, we utilise a synthetic dataset with depth and temporal considerations during training. The developed model is injected into the rendering pipeline to further enforce temporal stability and avoid diminishing post-process effects. Quantitative and qualitative experiments demonstrate that our approach achieves superior performance in temporal consistency, with comparable style transfer quality, to state-of-the-art image, video and in-game methods.
• 摘要：艺术风格的转移与将图像内容与艺术品风格结合在一起的图像的产生。在计算机游戏领域，大多数工作都集中在后处理视频帧上。最近的一些工作将样式转移整合到了游戏管道中，但仅限于单一样式。由于游戏的内存和速度要求，将任意样式转移方法集成到游戏管道中是具有挑战性的。我们提出PQDAST，这是第一个解决此问题的解决方案。我们使用感知质量引导的知识蒸馏框架，并使用Flip评估器训练压缩模型，该框架大大减少了内存使用和处理时间，对样式质量的影响有限。为了更好地保存深度和细节，我们在培训过程中使用具有深度和时间考虑的合成数据集。开发的模型被注入渲染管道中，以进一步执行时间稳定性并避免减少后处理效果。定量和定性实验表明，我们的方法在时间一致性方面取得了卓越的性能，具有可比的样式转移质量，与最先进的图像，视频和游戏中的方法相当。

Title: Improving the Transferability of Adversarial Examples by Inverse Knowledge Distillation

• Authors: Wenyuan Wu, Zheng Liu, Yong Chen, Chao Su, Dezhong Peng, Xu Wang
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17003
• Pdf URL: https://arxiv.org/pdf/2502.17003
• Copy Paste: [[2502.17003]] Improving the Transferability of Adversarial Examples by Inverse Knowledge Distillation(https://arxiv.org/abs/2502.17003)
• Keywords: generation
• Abstract: In recent years, the rapid development of deep neural networks has brought increased attention to the security and robustness of these models. While existing adversarial attack algorithms have demonstrated success in improving adversarial transferability, their performance remains suboptimal due to a lack of consideration for the discrepancies between target and source models. To address this limitation, we propose a novel method, Inverse Knowledge Distillation (IKD), designed to enhance adversarial transferability effectively. IKD introduces a distillation-inspired loss function that seamlessly integrates with gradient-based attack methods, promoting diversity in attack gradients and mitigating overfitting to specific model architectures. By diversifying gradients, IKD enables the generation of adversarial samples with superior generalization capabilities across different models, significantly enhancing their effectiveness in black-box attack scenarios. Extensive experiments on the ImageNet dataset validate the effectiveness of our approach, demonstrating substantial improvements in the transferability and attack success rates of adversarial samples across a wide range of models.
• 摘要：近年来，深层神经网络的快速发展引起了人们对这些模型的安全性和鲁棒性的关注。尽管现有的对抗攻击算法在改善对抗性转移性方面取得了成功，但由于缺乏对目标和源模型之间的差异的考虑，它们的性能仍然不错。为了解决这一局限性，我们提出了一种新颖的方法，即倒数知识蒸馏（IKD），旨在有效增强对抗性转移性。 IKD引入了蒸馏启发的损失函数，该功能与基于梯度的攻击方法无缝集成，促进攻击梯度的多样性并减轻对特定模型体系结构的过度拟合。通过使IKD多样化，IKD可以在不同模型之间产生具有出色概括能力的对抗样本，从而显着提高了它们在黑盒攻击方案中的有效性。在Imagenet数据集上进行的广泛实验验证了我们方法的有效性，证明了在广泛模型范围内的对抗样本的转移性和攻击成功率的显着提高。

Title: Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence

• Authors: Wenzhe Yin, Zehao Xiao, Pan Zhou, Shujian Yu, Jiayi Shen, Jan-Jakob Sonke, Efstratios Gavves
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17028
• Pdf URL: https://arxiv.org/pdf/2502.17028
• Copy Paste: [[2502.17028]] Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence(https://arxiv.org/abs/2502.17028)
• Keywords: generation
• Abstract: Multimodal alignment is crucial for various downstream tasks such as cross-modal generation and retrieval. Previous multimodal approaches like CLIP maximize the mutual information mainly by aligning pairwise samples across modalities while overlooking the distributional differences, leading to suboptimal alignment with modality gaps. In this paper, to overcome the limitation, we propose CS-Aligner, a novel and straightforward framework that performs distributional vision-language alignment by integrating Cauchy-Schwarz (CS) divergence with mutual information. In the proposed framework, we find that the CS divergence and mutual information serve complementary roles in multimodal alignment, capturing both the global distribution information of each modality and the pairwise semantic relationships, yielding tighter and more precise alignment. Moreover, CS-Aligher enables incorporating additional information from unpaired data and token-level representations, enhancing flexible and fine-grained alignment in practice. Experiments on text-to-image generation and cross-modality retrieval tasks demonstrate the effectiveness of our method on vision-language alignment.
• 摘要：多模式比对对于诸如跨模式生成和检索等各种下游任务至关重要。先前的多模式方法（例如剪辑）诸如剪辑之类的方法主要通过跨模态对齐样品，同时忽略分布差异，从而最大程度地提高了相互信息。在本文中，为了克服限制，我们提出了CS-Aligner，这是一个新颖而直接的框架，通过将Cauchy-Schwarz（CS）差异与相互信息整合在一起，通过将Cauchy-Schwarz（CS）差异整合来执行分布视觉的路线。在拟议的框架中，我们发现CS差异和相互信息在多模式对齐中起互补的作用，从而捕获了每种模态的全局分布信息和成对的语义关系，从而产生了更严格，更精确的对准。此外，CS-Eligher启用了从未配对的数据和令牌级表示中的其他信息，从而在实践中增强了灵活和细粒度的对齐方式。关于文本形象生成和跨模式检索任务的实验证明了我们方法对视觉对齐的有效性。

Title: SpecDM: Hyperspectral Dataset Synthesis with Pixel-level Semantic Annotations

• Authors: Wendi Liu, Pei Yang, Wenhui Hong, Xiaoguang Mei, Jiayi Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17056
• Pdf URL: https://arxiv.org/pdf/2502.17056
• Copy Paste: [[2502.17056]] SpecDM: Hyperspectral Dataset Synthesis with Pixel-level Semantic Annotations(https://arxiv.org/abs/2502.17056)
• Keywords: generation, generative
• Abstract: In hyperspectral remote sensing field, some downstream dense prediction tasks, such as semantic segmentation (SS) and change detection (CD), rely on supervised learning to improve model performance and require a large amount of manually annotated data for training. However, due to the needs of specific equipment and special application scenarios, the acquisition and annotation of hyperspectral images (HSIs) are often costly and time-consuming. To this end, our work explores the potential of generative diffusion model in synthesizing HSIs with pixel-level annotations. The main idea is to utilize a two-stream VAE to learn the latent representations of images and corresponding masks respectively, learn their joint distribution during the diffusion model training, and finally obtain the image and mask through their respective decoders. To the best of our knowledge, it is the first work to generate high-dimensional HSIs with annotations. Our proposed approach can be applied in various kinds of dataset generation. We select two of the most widely used dense prediction tasks: semantic segmentation and change detection, and generate datasets suitable for these tasks. Experiments demonstrate that our synthetic datasets have a positive impact on the improvement of these downstream tasks.
• 摘要：在高光谱遥感领域中，一些下游密集的预测任务，例如语义细分（SS）和变更检测（CD），依靠监督学习来提高模型性能，并需要大量手动注释数据进行培训。但是，由于特定设备和特殊应用方案的需求，高光谱图像（HSIS）的获取和注释通常是昂贵且耗时的。为此，我们的工作探讨了用像素级注释合成HSI的生成扩散模型的潜力。主要思想是利用两流VAE分别学习图像的潜在表示和相应的掩码，在扩散模型训练期间学习其联合分布，并最终通过其各自的解码器获得图像和掩盖。据我们所知，这是第一项用注释产生高维HSI的工作。我们提出的方法可以应用于各种数据集生成。我们选择了两个最广泛的密集预测任务：语义分割和更改检测，并生成适合这些任务的数据集。实验表明，我们的合成数据集对这些下游任务的改进有积极影响。

Title: Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence

• Authors: Bolin Chen, Hanwei Zhu, Shanzhi Yin, Lingyu Zhu, Jie Chen, Ru-Ling Liao, Shiqi Wang, Yan Ye
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.17085
• Pdf URL: https://arxiv.org/pdf/2502.17085
• Copy Paste: [[2502.17085]] Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence(https://arxiv.org/abs/2502.17085)
• Keywords: generation, generative
• Abstract: Generative model based compact video compression is typically operated within a relative narrow range of bitrates, and often with an emphasis on ultra-low rate applications. There has been an increasing consensus in the video communication industry that full bitrate coverage should be enabled by generative coding. However, this is an extremely difficult task, largely because generation and compression, although related, have distinct goals and trade-offs. The proposed Pleno-Generation (PGen) framework distinguishes itself through its exceptional capabilities in ensuring the robustness of video coding by utilizing a wider range of bandwidth for generation via bandwidth intelligence. In particular, we initiate our research of PGen with face video coding, and PGen offers a paradigm shift that prioritizes high-fidelity reconstruction over pursuing compact bitstream. The novel PGen framework leverages scalable representation and layered reconstruction for Generative Face Video Compression (GFVC), in an attempt to imbue the bitstream with intelligence in different granularity. Experimental results illustrate that the proposed PGen framework can facilitate existing GFVC algorithms to better deliver high-fidelity and faithful face videos. In addition, the proposed framework can allow a greater space of flexibility for coding applications and show superior RD performance with a much wider bitrate range in terms of various quality evaluations. Moreover, in comparison with the latest Versatile Video Coding (VVC) codec, the proposed scheme achieves competitive Bjøntegaard-delta-rate savings for perceptual-level evaluations.
• 摘要：基于生成模型的紧凑视频压缩通常在相对狭窄的比特率范围内进行操作，并且通常强调超低速率应用。在视频通信行业中，人们达成了越来越多的共识，应通过生成编码来实现全比特率覆盖率。但是，这是一项极其艰巨的任务，主要是因为产生和压缩虽然相关，但具有不同的目标和权衡。拟议的PLENO生成（PGEN）框架通过其出色的功能来区分自己，从而通过带宽智能利用更广泛的带宽来确保视频编码的鲁棒性。特别是，我们通过面部视频编码启动对PGEN的研究，PGEN提供了范式转变，优先考虑高保真重建而不是追求紧凑的Bitstream。新颖的PGEN框架利用可扩展的表示和分层的重建进行生成的面部视频压缩（GFVC），以试图将比特斯流与不同的智能相同。实验结果表明，提出的PGEN框架可以促进现有的GFVC算法，以更好地提供高保真和忠实的面部视频。此外，所提出的框架可以为编码应用提供更大的灵活性，并在各种质量评估方面显示出优异的RD性能，比特率范围更大。此外，与最新的多功能视频编码（VVC）编解码器相比，所提出的方案可实现竞争性的Bjøntegaard-Delta-Rate-Rate节省，以进行感知级别的评估。

Title: Improved Diffusion-based Generative Model with Better Adversarial Robustness

• Authors: Zekun Wang, Mingyang Yi, Shuchen Xue, Zhenguo Li, Ming Liu, Bing Qin, Zhi-Ming Ma
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17099
• Pdf URL: https://arxiv.org/pdf/2502.17099
• Copy Paste: [[2502.17099]] Improved Diffusion-based Generative Model with Better Adversarial Robustness(https://arxiv.org/abs/2502.17099)
• Keywords: generation, generative
• Abstract: Diffusion Probabilistic Models (DPMs) have achieved significant success in generative tasks. However, their training and sampling processes suffer from the issue of distribution mismatch. During the denoising process, the input data distributions differ between the training and inference stages, potentially leading to inaccurate data generation. To obviate this, we analyze the training objective of DPMs and theoretically demonstrate that this mismatch can be alleviated through Distributionally Robust Optimization (DRO), which is equivalent to performing robustness-driven Adversarial Training (AT) on DPMs. Furthermore, for the recently proposed Consistency Model (CM), which distills the inference process of the DPM, we prove that its training objective also encounters the mismatch issue. Fortunately, this issue can be mitigated by AT as well. Based on these insights, we propose to conduct efficient AT on both DPM and CM. Finally, extensive empirical studies validate the effectiveness of AT in diffusion-based models. The code is available at this https URL.
• 摘要：扩散概率模型（DPM）在生成任务上取得了重大成功。但是，他们的培训和抽样过程遇到了分配不匹配的问题。在降级过程中，培训和推理阶段之间的输入数据分布有所不同，可能导致数据生成不准确。为了消除这一点，我们分析了DPM的训练目标，并理论上证明可以通过分布强大的优化（DRO）来缓解这种不匹配，这相当于在DPMS上进行鲁棒性驱动的对抗性训练（AT）。此外，对于最近提出的一致性模型（CM），它提炼了DPM的推理过程，我们证明其训练目标还遇到了不匹配问题。幸运的是，此问题也可以通过AT减轻。基于这些见解，我们建议在DPM和CM上进行有效的效率。最后，广泛的经验研究验证了基于扩散模型的AT有效性。该代码可在此HTTPS URL上找到。

Title: Generative Models in Decision Making: A Survey

• Authors: Yinchuan Li, Xinyu Shao, Jianping Zhang, Haozhi Wang, Leo Maxime Brunswic, Kaiwen Zhou, Jiqian Dong, Kaiyang Guo, Xiu Li, Zhitang Chen, Jun Wang, Jianye Hao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17100
• Pdf URL: https://arxiv.org/pdf/2502.17100
• Copy Paste: [[2502.17100]] Generative Models in Decision Making: A Survey(https://arxiv.org/abs/2502.17100)
• Keywords: generation, generative
• Abstract: In recent years, the exceptional performance of generative models in generative tasks has sparked significant interest in their integration into decision-making processes. Due to their ability to handle complex data distributions and their strong model capacity, generative models can be effectively incorporated into decision-making systems by generating trajectories that guide agents toward high-reward state-action regions or intermediate sub-goals. This paper presents a comprehensive review of the application of generative models in decision-making tasks. We classify seven fundamental types of generative models: energy-based models, generative adversarial networks, variational autoencoders, normalizing flows, diffusion models, generative flow networks, and autoregressive models. Regarding their applications, we categorize their functions into three main roles: controllers, modelers and optimizers, and discuss how each role contributes to decision-making. Furthermore, we examine the deployment of these models across five critical real-world decision-making scenarios. Finally, we summarize the strengths and limitations of current approaches and propose three key directions for advancing next-generation generative directive models: high-performance algorithms, large-scale generalized decision-making models, and self-evolving and adaptive models.
• 摘要：近年来，生成模型在生成任务中的出色表现引发了对它们整合到决策过程中的浓厚兴趣。由于它们可以处理复杂的数据分布及其强大的模型能力，因此可以通过生成指导代理到高奖励状态行动区域或中间子目标来有效地纳入决策系统中的生成模型。本文对生成模型在决策任务中的应用进行了全面综述。我们对生成模型的七种基本类型进行了分类：基于能量的模型，生成的对抗网络，变化自动编码器，标准化流量，扩散模型，生成流动网络和自回旋模型。关于他们的应用，我们将其功能分为三个主要角色：控制器，建模者和优化者，并讨论每个角色如何对决策做出贡献。此外，我们研究了这些模型在五个关键现实世界决策方案中的部署。最后，我们总结了当前方法的优势和局限性，并提出了前进下一代生成指令模型的三个关键方向：高性能算法，大规模的广义决策模型以及自我发展和自适应模型。

Title: SFLD: Reducing the content bias for AI-generated Image Detection

• Authors: Seoyeon Gye, Junwon Ko, Hyounguk Shon, Minchan Kwon, Junmo Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17105
• Pdf URL: https://arxiv.org/pdf/2502.17105
• Copy Paste: [[2502.17105]] SFLD: Reducing the content bias for AI-generated Image Detection(https://arxiv.org/abs/2502.17105)
• Keywords: generation, generative
• Abstract: Identifying AI-generated content is critical for the safe and ethical use of generative AI. Recent research has focused on developing detectors that generalize to unknown generators, with popular methods relying either on high-level features or low-level fingerprints. However, these methods have clear limitations: biased towards unseen content, or vulnerable to common image degradations, such as JPEG compression. To address these issues, we propose a novel approach, SFLD, which incorporates PatchShuffle to integrate high-level semantic and low-level textural information. SFLD applies PatchShuffle at multiple levels, improving robustness and generalization across various generative models. Additionally, current benchmarks face challenges such as low image quality, insufficient content preservation, and limited class diversity. In response, we introduce TwinSynths, a new benchmark generation methodology that constructs visually near-identical pairs of real and synthetic images to ensure high quality and content preservation. Our extensive experiments and analysis show that SFLD outperforms existing methods on detecting a wide variety of fake images sourced from GANs, diffusion models, and TwinSynths, demonstrating the state-of-the-art performance and generalization capabilities to novel generative models.
• 摘要：识别AI生成的内容对于生成AI的安全和道德使用至关重要。最近的研究集中在开发概括为未知发电机的检测器，其流行方法依赖于高级功能或低级指纹。但是，这些方法具有明显的局限性：对看不见的内容有偏见，或容易受到常见图像降解的影响，例如JPEG压缩。为了解决这些问题，我们提出了一种新颖的方法SFLD，该方法结合了补丁程序，以整合高级语义和低级纹理信息。 SFLD在多个层面上应用斑块扣，从而改善了各种生成模型的鲁棒性和概括。此外，当前的基准测试面临诸如低图像质量，内容保存不足和班级多样性等挑战。作为回应，我们介绍了Twinsynths，这是一种新的基准生成方法，该方法在视觉上构建了几乎相同的真实和合成图像，以确保高质量和内容保存。我们的广泛实验和分析表明，SFLD优于现有方法，该方法在检测来自gan，扩散模型和Twinsynts的各种假图像方面，证明了最先进的性能和概括能力以及新颖的生成模型。

Title: Diffusion Models for Tabular Data: Challenges, Current Progress, and Future Directions

• Authors: Zhong Li, Qi Huang, Lincen Yang, Jiayang Shi, Zhao Yang, Niki van Stein, Thomas Bäck, Matthijs van Leeuwen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17119
• Pdf URL: https://arxiv.org/pdf/2502.17119
• Copy Paste: [[2502.17119]] Diffusion Models for Tabular Data: Challenges, Current Progress, and Future Directions(https://arxiv.org/abs/2502.17119)
• Keywords: generation, generative
• Abstract: In recent years, generative models have achieved remarkable performance across diverse applications, including image generation, text synthesis, audio creation, video generation, and data augmentation. Diffusion models have emerged as superior alternatives to Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) by addressing their limitations, such as training instability, mode collapse, and poor representation of multimodal distributions. This success has spurred widespread research interest. In the domain of tabular data, diffusion models have begun to showcase similar advantages over GANs and VAEs, achieving significant performance breakthroughs and demonstrating their potential for addressing unique challenges in tabular data modeling. However, while domains like images and time series have numerous surveys summarizing advancements in diffusion models, there remains a notable gap in the literature for tabular data. Despite the increasing interest in diffusion models for tabular data, there has been little effort to systematically review and summarize these developments. This lack of a dedicated survey limits a clear understanding of the challenges, progress, and future directions in this critical area. This survey addresses this gap by providing a comprehensive review of diffusion models for tabular data. Covering works from June 2015, when diffusion models emerged, to December 2024, we analyze nearly all relevant studies, with updates maintained in a \href{this https URL}{GitHub repository}. Assuming readers possess foundational knowledge of statistics and diffusion models, we employ mathematical formulations to deliver a rigorous and detailed review, aiming to promote developments in this emerging and exciting area.
• 摘要：近年来，生成模型在各种应用程序中取得了出色的性能，包括图像生成，文本综合，音频创建，视频生成和数据增强。扩散模型已成为生成对抗网络（GAN）和变化自动编码器（VAE）的优越替代品，例如解决它们的局限性，例如训练不稳定性，模式崩溃和多模式分布的表示不良。这一成功激发了广泛的研究兴趣。在表格数据的域中，扩散模型已经开始展示与gan和vaes相似的优势，实现了显着的性能突破，并展示了它们在表达表格数据建模中的独特挑战的潜力。但是，尽管图像和时间序列之类的域有许多调查总结了扩散模型中的进步，但文献中对于表格数据仍然存在显着差距。尽管对表格数据的扩散模型的兴趣越来越大，但系统地审查和总结了这些发展的努力很少。缺乏专门的调查限制了对这个关键领域中的挑战，进步和未来方向的清晰了解。该调查通过对表格数据的扩散模型进行全面审查来解决这一差距。涵盖2015年6月扩散模型的工作，到2024年12月，我们分析了几乎所有相关的研究，并在\ href {this HTTPS url} {github reposority}中维护了更新。假设读者具有统计和扩散模型的基本知识，我们采用数学公式来提供严格而详细的审查，旨在促进这一新兴和令人兴奋的领域的发展。

Title: DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks

• Authors: Canyu Zhao, Mingyu Liu, Huanyi Zheng, Muzhi Zhu, Zhiyue Zhao, Hao Chen, Tong He, Chunhua Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17157
• Pdf URL: https://arxiv.org/pdf/2502.17157
• Copy Paste: [[2502.17157]] DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks(https://arxiv.org/abs/2502.17157)
• Keywords: generation
• Abstract: Our primary goal here is to create a good, generalist perception model that can tackle multiple tasks, within limits on computational resources and training data. To achieve this, we resort to text-to-image diffusion models pre-trained on billions of images. Our exhaustive evaluation metrics demonstrate that DICEPTION effectively tackles multiple perception tasks, achieving performance on par with state-of-the-art models. We achieve results on par with SAM-vit-h using only 0.06% of their data (e.g., 600K vs. 1B pixel-level annotated images). Inspired by Wang et al., DICEPTION formulates the outputs of various perception tasks using color encoding; and we show that the strategy of assigning random colors to different instances is highly effective in both entity segmentation and semantic segmentation. Unifying various perception tasks as conditional image generation enables us to fully leverage pre-trained text-to-image models. Thus, DICEPTION can be efficiently trained at a cost of orders of magnitude lower, compared to conventional models that were trained from scratch. When adapting our model to other tasks, it only requires fine-tuning on as few as 50 images and 1% of its parameters. DICEPTION provides valuable insights and a more promising solution for visual generalist models.
• 摘要：我们的主要目标是创建一个良好的通才感知模型，该模型可以在计算资源和培训数据的限制内处理多个任务。为了实现这一目标，我们求助于在数十亿张图像上预先训练的文本对图像扩散模型。我们详尽的评估指标表明，掷骰子有效地解决了多个感知任务，从而在最新模型上达到了绩效。我们仅使用其数据的0.06％的数据（例如600K与1B像素级注释的图像）以SAM-VIT-H的标准率达到结果。受Wang等人的启发，使用颜色编码来制定各种感知任务的输出。我们表明，将随机颜色分配给不同实例的策略在实体细分和语义分割中都非常有效。将各种感知任务统一为有条件的图像生成，使我们能够完全利用预先训练的文本对图像模型。因此，与从头开始训练的传统模型相比，可以以低数量级的成本进行固定训练。将我们的模型调整到其他任务时，仅需要对50张图像和1％的参数进行微调。骰子吸收为视觉通用模型提供了宝贵的见解和更有希望的解决方案。

Title: A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis

• Authors: Yuli Wu, Fucheng Liu, Rüveyda Yilmaz, Henning Konermann, Peter Walter, Johannes Stegmaier
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17160
• Pdf URL: https://arxiv.org/pdf/2502.17160
• Copy Paste: [[2502.17160]] A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis(https://arxiv.org/abs/2502.17160)
• Keywords: generative
• Abstract: Fréchet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic data match real data in many image synthesis tasks, the primary goal in biomedical generative models is often to enrich training datasets ideally with corresponding annotations. For this purpose, the gold standard for evaluating generative models is to incorporate synthetic data into downstream task training, such as classification and segmentation, to pragmatically assess its performance. In this paper, we examine cases from retinal imaging modalities, including color fundus photography and optical coherence tomography, where FID and its related metrics misalign with task-specific evaluation goals in classification and segmentation. We highlight the limitations of using various metrics, represented by FID and its variants, as evaluation criteria for these applications and address their potential caveats in broader biomedical imaging modalities and downstream tasks.
• 摘要：FréchetInception距离（FID），使用鉴定的Inception-V3网络计算的Fréchet距离（FID）被广泛用作生成模型的最新评估度量。它假设Inception-V3的特征向量遵循多元高斯分布，并根据其平均值和协方差计算2-Wasserstein距离。尽管FID有效地衡量了综合数据在许多图像合成任务中的实际数据匹配程度，但生物医学生成模型的主要目标通常是通过相应的注释理想地富集训练数据集。为此，评估生成模型的黄金标准是将综合数据纳入下游任务训练，例如分类和分割，以务实评估其性能。在本文中，我们研究了视网膜成像方式中的案例，包括颜色底面摄影和光学相干断层扫描，其中FID及其相关的指标与分类和分割的特定于任务评估目标失调。我们强调了使用由FID及其变体代表的各种指标作为这些应用的评估标准的局限性，并解决了其在更广泛的生物医学成像方式和下游任务中的潜在警告。

Title: Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation

• Authors: Baptiste Chopin, Tashvik Dhamija, Pranav Balaji, Yaohui Wang, Antitza Dantcheva
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17198
• Pdf URL: https://arxiv.org/pdf/2502.17198
• Copy Paste: [[2502.17198]] Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation(https://arxiv.org/abs/2502.17198)
• Keywords: generation
• Abstract: We propose Dimitra, a novel framework for audio-driven talking head generation, streamlined to learn lip motion, facial expression, as well as head pose motion. Specifically, we train a conditional Motion Diffusion Transformer (cMDT) by modeling facial motion sequences with 3D representation. We condition the cMDT with only two input signals, an audio-sequence, as well as a reference facial image. By extracting additional features directly from audio, Dimitra is able to increase quality and realism of generated videos. In particular, phoneme sequences contribute to the realism of lip motion, whereas text transcript to facial expression and head pose realism. Quantitative and qualitative experiments on two widely employed datasets, VoxCeleb2 and HDTF, showcase that Dimitra is able to outperform existing approaches for generating realistic talking heads imparting lip motion, facial expression, and head pose.
• 摘要：我们提出了Dimitra，这是一个用于音频驱动的会说话的新颖框架，简化以学习唇部运动，面部表情以及头部姿势运动。具体而言，我们通过用3D表示的面部运动序列对条件运动扩散变压器（CMDT）训练条件运动扩散变压器（CMDT）。我们只有两个输入信号，一个音频序列以及一个参考面部图像来调节CMDT。通过直接从音频中提取其他功能，Dimitra能够提高生成视频的质量和现实主义。特别是，音素序列有助于唇部运动的现实主义，而文本转录本为面部表达和头部姿势现实主义。在两个广泛使用的数据集（Voxceleb2和HDTF）上进行的定量和定性实验展示了Dimitra能够超过现有的方法，以产生逼真的说话头，从而赋予唇部运动，面部表情和头部姿势。

Title: VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing

• Authors: Xiangpeng Yang, Linchao Zhu, Hehe Fan, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17258
• Pdf URL: https://arxiv.org/pdf/2502.17258
• Copy Paste: [[2502.17258]] VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing(https://arxiv.org/abs/2502.17258)
• Keywords: generation
• Abstract: Recent advancements in diffusion models have significantly improved video generation and editing capabilities. However, multi-grained video editing, which encompasses class-level, instance-level, and part-level modifications, remains a formidable challenge. The major difficulties in multi-grained editing include semantic misalignment of text-to-region control and feature coupling within the diffusion model. To address these difficulties, we present VideoGrain, a zero-shot approach that modulates space-time (cross- and self-) attention mechanisms to achieve fine-grained control over video content. We enhance text-to-region control by amplifying each local prompt's attention to its corresponding spatial-disentangled region while minimizing interactions with irrelevant areas in cross-attention. Additionally, we improve feature separation by increasing intra-region awareness and reducing inter-region interference in self-attention. Extensive experiments demonstrate our method achieves state-of-the-art performance in real-world scenarios. Our code, data, and demos are available at this https URL
• 摘要：扩散模型的最新进展已大大提高了视频生成和编辑功能。但是，多元熟悉的视频编辑包括班级级别，实例级别和部分级别的修改，仍然是一个巨大的挑战。多元透彻编辑的主要困难包括在扩散模型中的文本对区域控制和特征耦合的语义错位。为了解决这些困难，我们提出了视频：一种零拍的方法，可调节时空（交叉和自我）注意机制，以实现对视频内容的细粒度控制。我们通过将每个当地提示的注意力放大其相应的空间 - 触发区域，同时最大程度地减少与跨注意区域无关区域的相互作用，从而增强文本对区域的控制。此外，我们通过提高区域内意识并减少自我注意力区域间干扰来改善特征分离。广泛的实验证明了我们的方法在现实情况下实现了最先进的表现。我们的代码，数据和演示可在此HTTPS URL上找到

Title: KV-Edit: Training-Free Image Editing for Precise Background Preservation

• Authors: Tianrui Zhu, Shiyi Zhang, Jiawei Shao, Yansong Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17363
• Pdf URL: https://arxiv.org/pdf/2502.17363
• Copy Paste: [[2502.17363]] KV-Edit: Training-Free Image Editing for Precise Background Preservation(https://arxiv.org/abs/2502.17363)
• Keywords: generative
• Abstract: Background consistency remains a significant challenge in image editing tasks. Despite extensive developments, existing works still face a trade-off between maintaining similarity to the original image and generating content that aligns with the target. Here, we propose KV-Edit, a training-free approach that uses KV cache in DiTs to maintain background consistency, where background tokens are preserved rather than regenerated, eliminating the need for complex mechanisms or expensive training, ultimately generating new content that seamlessly integrates with the background within user-provided regions. We further explore the memory consumption of the KV cache during editing and optimize the space complexity to $O(1)$ using an inversion-free method. Our approach is compatible with any DiT-based generative model without additional training. Experiments demonstrate that KV-Edit significantly outperforms existing approaches in terms of both background and image quality, even surpassing training-based methods. Project webpage is available at this https URL
• 摘要：背景一致性仍然是图像编辑任务的重大挑战。尽管有广泛的发展，但现有作品仍然在保持与原始图像相似的相似性和生成与目标保持一致的内容之间面临权衡。在这里，我们提出了KV-EDIT，这是一种无培训的方法，它使用DIT中的KV缓存来维持背景一致性，在这里保留了背景令牌而不是再生，消除了对复杂机制或昂贵培训的需求，最终生成了无缝集成的新内容具有用户提供的区域内的背景。我们进一步探讨了编辑过程中KV缓存的内存消耗，并使用无反转方法优化了$ O（1）$的空间复杂性。我们的方法与任何基于DIT的生成模型都兼容，没有其他培训。实验表明，KV-EDIT在背景和图像质量方面显着优于现有方法，甚至超过了基于培训的方法。 Project网页可在此HTTPS URL上找到

Title: Sustainable Greenhouse Management: A Comparative Analysis of Recurrent and Graph Neural Networks

• Authors: Emiliano Seri, Marcello Petitta, Cristina Cornaro
• Subjects: cs.LG, stat.AP
• Abstract URL: https://arxiv.org/abs/2502.17371
• Pdf URL: https://arxiv.org/pdf/2502.17371
• Copy Paste: [[2502.17371]] Sustainable Greenhouse Management: A Comparative Analysis of Recurrent and Graph Neural Networks(https://arxiv.org/abs/2502.17371)
• Keywords: generation
• Abstract: The integration of photovoltaic (PV) systems into greenhouses not only optimizes land use but also enhances sustainable agricultural practices by enabling dual benefits of food production and renewable energy generation. However, accurate prediction of internal environmental conditions is crucial to ensure optimal crop growth while maximizing energy production. This study introduces a novel application of Spatio-Temporal Graph Neural Networks (STGNNs) to greenhouse microclimate modeling, comparing their performance with traditional Recurrent Neural Networks (RNNs). While RNNs excel at temporal pattern recognition, they cannot explicitly model the directional relationships between environmental variables. Our STGNN approach addresses this limitation by representing these relationships as directed graphs, enabling the model to capture both spatial dependencies and their directionality. Using high-frequency data collected at 15-minute intervals from a greenhouse in Volos, Greece, we demonstrate that RNNs achieve exceptional accuracy in winter conditions (R^2 = 0.985) but show limitations during summer cooling system operation. Though STGNNs currently show lower performance (winter R^2 = 0.947), their architecture offers greater potential for integrating additional variables such as PV generation and crop growth indicators.
• 摘要：将光伏（PV）系统整合到温室中，不仅可以优化土地使用，而且通过实现粮食生产和可再生能源产生的双重益处来增强可持续的农业实践。但是，对内部环境条件的准确预测对于确保最佳农作物生长至关重要，同时最大化能源生产至关重要。这项研究介绍了时空图神经网络（STGNN）在温室微气候建模中的新应用，并将其与传统的复发神经网络（RNN）进行了比较。尽管RNN在时间模式识别方面表现出色，但它们无法明确对环境变量之间的方向关系进行建模。我们的STGNN方法通过将这些关系按照指示图表示这些关系来解决此限制，从而使模型能够同时捕获空间依赖性及其方向性。使用来自希腊沃罗斯温室的15分钟间隔收集的高频数据，我们证明了RNN在冬季条件下达到了非凡的精度（R^2 = 0.985），但在夏季冷却系统操作中显示出限制。尽管STGNN当前显示出较低的性能（冬季R^2 = 0.947），但它们的架构为整合其他变量（例如PV生成和作物增长指标）提供了更大的潜力。

Title: X-Dancer: Expressive Music to Human Dance Video Generation

• Authors: Zeyuan Chen, Hongyi Xu, Guoxian Song, You Xie, Chenxu Zhang, Xin Chen, Chao Wang, Di Chang, Linjie Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17414
• Pdf URL: https://arxiv.org/pdf/2502.17414
• Copy Paste: [[2502.17414]] X-Dancer: Expressive Music to Human Dance Video Generation(https://arxiv.org/abs/2502.17414)
• Keywords: generation
• Abstract: We present X-Dancer, a novel zero-shot music-driven image animation pipeline that creates diverse and long-range lifelike human dance videos from a single static image. As its core, we introduce a unified transformer-diffusion framework, featuring an autoregressive transformer model that synthesize extended and music-synchronized token sequences for 2D body, head and hands poses, which then guide a diffusion model to produce coherent and realistic dance video frames. Unlike traditional methods that primarily generate human motion in 3D, X-Dancer addresses data limitations and enhances scalability by modeling a wide spectrum of 2D dance motions, capturing their nuanced alignment with musical beats through readily available monocular videos. To achieve this, we first build a spatially compositional token representation from 2D human pose labels associated with keypoint confidences, encoding both large articulated body movements (e.g., upper and lower body) and fine-grained motions (e.g., head and hands). We then design a music-to-motion transformer model that autoregressively generates music-aligned dance pose token sequences, incorporating global attention to both musical style and prior motion context. Finally we leverage a diffusion backbone to animate the reference image with these synthesized pose tokens through AdaIN, forming a fully differentiable end-to-end framework. Experimental results demonstrate that X-Dancer is able to produce both diverse and characterized dance videos, substantially outperforming state-of-the-art methods in term of diversity, expressiveness and realism. Code and model will be available for research purposes.
• 摘要：我们介绍了X-Dancer，这是一种小说的零拍音乐驱动的图像动画管道，它从单个静态图像中创建了多样化且远距离的人类舞蹈视频。作为其核心，我们引入了一个统一的变压器扩散框架，具有自回归的变压器模型，该模型合成了2D身体，头部和手姿势的扩展和音乐同步令牌序列。与主要在3D中产生人类运动的传统方法不同，X舞者通过建模2D舞蹈动作的广泛范围来解决数据限制并增强可扩展性，从而通过随时可用的单眼视频来捕获其与音乐节拍的细微对齐。为了实现这一目标，我们首先从与关键点知识相关的2D人姿势标签中构建了一个空间组成令牌表示，编码大型铰接的身体运动（例如上半身和下半身）和细粒度的运动（例如，头部和手）。然后，我们设计了一种音乐到动作变压器模型，可自动重18S生成音乐一致的舞蹈姿势令牌序列，从而纳入了全球对音乐风格和先前运动环境的关注。最后，我们利用扩散的主链通过这些合成的姿势令牌通过ADAIN来对参考图像进行动画化，从而形成一个完全可区分的端到端框架。实验结果表明，X型舞者能够制作出多样化和特征的舞蹈视频，在多样性，表现力和现实主义方面大大优于最先进的方法。代码和模型将用于研究目的。

Title: S4S: Solving for a Diffusion Model Solver

• Authors: Eric Frankel, Sitan Chen, Jerry Li, Pang Wei Koh, Lillian J. Ratliff, Sewoong Oh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17423
• Pdf URL: https://arxiv.org/pdf/2502.17423
• Copy Paste: [[2502.17423]] S4S: Solving for a Diffusion Model Solver(https://arxiv.org/abs/2502.17423)
• Keywords: generation
• Abstract: Diffusion models (DMs) create samples from a data distribution by starting from random noise and iteratively solving a reverse-time ordinary differential equation (ODE). Because each step in the iterative solution requires an expensive neural function evaluation (NFE), there has been significant interest in approximately solving these diffusion ODEs with only a few NFEs without modifying the underlying model. However, in the few NFE regime, we observe that tracking the true ODE evolution is fundamentally impossible using traditional ODE solvers. In this work, we propose a new method that learns a good solver for the DM, which we call Solving for the Solver (S4S). S4S directly optimizes a solver to obtain good generation quality by learning to match the output of a strong teacher solver. We evaluate S4S on six different pre-trained DMs, including pixel-space and latent-space DMs for both conditional and unconditional sampling. In all settings, S4S uniformly improves the sample quality relative to traditional ODE solvers. Moreover, our method is lightweight, data-free, and can be plugged in black-box on top of any discretization schedule or architecture to improve performance. Building on top of this, we also propose S4S-Alt, which optimizes both the solver and the discretization schedule. By exploiting the full design space of DM solvers, with 5 NFEs, we achieve an FID of 3.73 on CIFAR10 and 13.26 on MS-COCO, representing a $1.5\times$ improvement over previous training-free ODE methods.
• 摘要：扩散模型（DMS）通过从随机噪声开始并迭代求解反时普通微分方程（ODE）来从数据分布中创建样本。由于迭代解决方案中的每个步骤都需要昂贵的神经功能评估（NFE），因此对仅使用少数NFE求解这些扩散ODE的近似值而没有修改基础模型引起了重大兴趣。但是，在少数NFE制度中，我们观察到使用传统的ODE求解器在根本上是不可能的。在这项工作中，我们提出了一种新方法，该方法可以学习DM的良好求解器，我们称其为求解器（S4S）。 S4S直接优化求解器，通过学习与强大的教师求解器的产出相匹配，以获得良好的发电质量。我们在六个不同的预训练的DM上评估了S4，包括用于条件和无条件采样的像素空间和潜在空间DM。在所有设置中，S4S均匀地提高了相对于传统ode求解器的样品质量。此外，我们的方法是轻巧的，无数据的，并且可以在任何离散的时间表或体系结构以提高性能的基础上插入黑框。在此基础上，我们还提出了S4S-Alt，该ALT优化了求解器和离散时间表。通过利用DM求解器的完整设计空间，具有5个NFE，我们在CIFAR10上的FID为3.73，MS-Coco上的FID为13.26，代表$ 1.5 \ times $的改进，比以前的无培训ODE方法提高了。

Title: GCC: Generative Color Constancy via Diffusing a Color Checker

• Authors: Chen-Wei Chang, Cheng-De Fan, Chia-Che Chang, Yi-Chen Lo, Yu-Chee Tseng, Jiun-Long Huang, Yu-Lun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17435
• Pdf URL: https://arxiv.org/pdf/2502.17435
• Copy Paste: [[2502.17435]] GCC: Generative Color Constancy via Diffusing a Color Checker(https://arxiv.org/abs/2502.17435)
• Keywords: generative
• Abstract: Color constancy methods often struggle to generalize across different camera sensors due to varying spectral sensitivities. We present GCC, which leverages diffusion models to inpaint color checkers into images for illumination estimation. Our key innovations include (1) a single-step deterministic inference approach that inpaints color checkers reflecting scene illumination, (2) a Laplacian decomposition technique that preserves checker structure while allowing illumination-dependent color adaptation, and (3) a mask-based data augmentation strategy for handling imprecise color checker annotations. GCC demonstrates superior robustness in cross-camera scenarios, achieving state-of-the-art worst-25% error rates of 5.15° and 4.32° in bi-directional evaluations. These results highlight our method's stability and generalization capability across different camera characteristics without requiring sensor-specific training, making it a versatile solution for real-world applications.
• 摘要：由于光谱敏感性的不同，颜色恒定方法通常很难跨越不同的相机传感器。我们提出了GCC，该海湾合作委员会利用扩散模型将彩色检查器注入图像以进行照明估计。我们的关键创新包括（1）一种单步确定性推理方法，该方法对反映场景照明的配色检查器进行了涂抹，（2）一种laplacian分解技术，可保留检查器结构，同时允许依赖照明的颜色适应，以及（3）基于掩模的数据处理不精确的彩色检查器注释的增强策略。 GCC在跨摄像机场景中表现出了出色的鲁棒性，在双向评估中达到了最新的最差25％错误率为5.15°和4.32°。这些结果突出了我们方法在不同摄像头特征上的稳定性和概括能力，而无需特定于传感器的训练，这使其成为现实世界应用的多功能解决方案。

Title: Towards Hierarchical Rectified Flow

• Authors: Yichi Zhang, Yici Yan, Alex Schwing, Zhizhen Zhao
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17436
• Pdf URL: https://arxiv.org/pdf/2502.17436
• Copy Paste: [[2502.17436]] Towards Hierarchical Rectified Flow(https://arxiv.org/abs/2502.17436)
• Keywords: generation
• Abstract: We formulate a hierarchical rectified flow to model data distributions. It hierarchically couples multiple ordinary differential equations (ODEs) and defines a time-differentiable stochastic process that generates a data distribution from a known source distribution. Each ODE resembles the ODE that is solved in a classic rectified flow, but differs in its domain, i.e., location, velocity, acceleration, etc. Unlike the classic rectified flow formulation, which formulates a single ODE in the location domain and only captures the expected velocity field (sufficient to capture a multi-modal data distribution), the hierarchical rectified flow formulation models the multi-modal random velocity field, acceleration field, etc., in their entirety. This more faithful modeling of the random velocity field enables integration paths to intersect when the underlying ODE is solved during data generation. Intersecting paths in turn lead to integration trajectories that are more straight than those obtained in the classic rectified flow formulation, where integration paths cannot intersect. This leads to modeling of data distributions with fewer neural function evaluations. We empirically verify this on synthetic 1D and 2D data as well as MNIST, CIFAR-10, and ImageNet-32 data. Code is available at: this https URL.
• 摘要：我们制定了分层整流的流程，以建模数据分布。它从层次上耦合了多个普通微分方程（ODE），并定义了一个时间差异的随机过程，该过程从已知的源分布中生成数据分布。每种颂歌都类似于在经典的整流流中求解的ODE，但在其域中有所不同，即位置，速度，加速度等。与经典的整流流式配方不同，它在位置域中制定了单个ODE，仅捕获单个ODE预期速度字段（足以捕获多模式数据分布），分层整流的流程公式模型多模式随机速度字段，加速场等。随机速度场的这种更忠实的建模可以使整合路径在数据生成过程中求解底层ode时相交。相交的路径反过来导致集成轨迹比在经典整流的流动公式中获得的轨迹更直，在整合路径无法相交的情况下。这导致具有更少神经功能评估的数据分布建模。我们在合成的1D和2D数据以及MNIST，CIFAR-10和Imagenet-32数据上进行经验验证这一点。代码可用：此HTTPS URL。

Title: Fractal Generative Models

• Authors: Tianhong Li, Qinyi Sun, Lijie Fan, Kaiming He
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17437
• Pdf URL: https://arxiv.org/pdf/2502.17437
• Copy Paste: [[2502.17437]] Fractal Generative Models(https://arxiv.org/abs/2502.17437)
• Keywords: generation, generative
• Abstract: Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at this https URL.
• 摘要：模块化是计算机科学的基石，将复杂的功能抽象成原子构建块。在本文中，我们通过将生成模型抽象成原子生成模块来引入新的模块化水平。类似于数学中的分形，我们的方法通过递归调用原子生成模块构建了一种新型的生成模型，从而产生了自相似的分形体系结构，我们称为分形生成模型。作为一个运行的示例，我们使用自回归模型作为原子生成模块实例化了分形框架，并将其审查逐个像素图像生成的挑战性任务，在可能性估计和发电质量中都表现出强烈的性能。我们希望这项工作可以为生成建模开辟新的范式，并为将来的研究提供肥沃的基础。代码可在此HTTPS URL上找到。