2025-04-25

Title: Safety Pretraining: Toward the Next Generation of Safe AI

• Authors: Pratyush Maini, Sachin Goyal, Dylan Sam, Alex Robey, Yash Savani, Yiding Jiang, Andy Zou, Zacharcy C. Lipton, J. Zico Kolter
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.16980
• Pdf URL: https://arxiv.org/pdf/2504.16980
• Copy Paste: [[2504.16980]] Safety Pretraining: Toward the Next Generation of Safe AI(https://arxiv.org/abs/2504.16980)
• Keywords: generation
• Abstract: As large language models (LLMs) are increasingly deployed in high-stakes settings, the risk of generating harmful or toxic content remains a central challenge. Post-hoc alignment methods are brittle: once unsafe patterns are learned during pretraining, they are hard to remove. We present a data-centric pretraining framework that builds safety into the model from the start. Our contributions include: (i) a safety classifier trained on 10,000 GPT-4 labeled examples, used to filter 600B tokens; (ii) the largest synthetic safety dataset to date (100B tokens) generated via recontextualization of harmful web data; (iii) RefuseWeb and Moral Education datasets that convert harmful prompts into refusal dialogues and web-style educational material; (iv) Harmfulness-Tag annotations injected during pretraining to flag unsafe content and steer away inference from harmful generations; and (v) safety evaluations measuring base model behavior before instruction tuning. Our safety-pretrained models reduce attack success rates from 38.8% to 8.4% with no performance degradation on standard LLM safety benchmarks.
• 摘要：随着大型语言模型（LLM）越来越多地部署在高风险环境中，产生有害或有毒内容的风险仍然是一个核心挑战。事后比对方法是脆弱的：一旦在预处理过程中学习了不安全的模式，就很难去除它们。我们提出了一个以数据为中心的预处理框架，该框架从一开始就将安全性建立到模型中。我们的贡献包括：（i）在10,000 GPT-4标记的示例上培训的安全分类器，用于过滤600B令牌； （ii）通过重新定义有害网络数据生成的迄今为止最大的合成安全数据集（100B令牌）； （iii）拒绝将有害提示转换为拒绝对话和网络风格的教育材料的拒绝和道德教育数据集； （iv）在预处理期间注入的有害标签注释不安全的含量，并从有害世代开始推断； （v）指导调整之前测量基本模型行为的安全评估。我们的安全预测模型将攻击成功率从38.8％降低至8.4％，而在标准LLM安全基准上没有绩效降解。

Title: (Im)possibility of Automated Hallucination Detection in Large Language Models

• Authors: Amin Karbasi, Omar Montasser, John Sous, Grigoris Velegkas
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.17004
• Pdf URL: https://arxiv.org/pdf/2504.17004
• Copy Paste: [[2504.17004]] (Im)possibility of Automated Hallucination Detection in Large Language Models(https://arxiv.org/abs/2504.17004)
• Keywords: generation
• Abstract: Is automated hallucination detection possible? In this work, we introduce a theoretical framework to analyze the feasibility of automatically detecting hallucinations produced by large language models (LLMs). Inspired by the classical Gold-Angluin framework for language identification and its recent adaptation to language generation by Kleinberg and Mullainathan, we investigate whether an algorithm, trained on examples drawn from an unknown target language $K$ (selected from a countable collection) and given access to an LLM, can reliably determine whether the LLM's outputs are correct or constitute hallucinations. First, we establish an equivalence between hallucination detection and the classical task of language identification. We prove that any hallucination detection method can be converted into a language identification method, and conversely, algorithms solving language identification can be adapted for hallucination detection. Given the inherent difficulty of language identification, this implies that hallucination detection is fundamentally impossible for most language collections if the detector is trained using only correct examples from the target language. Second, we show that the use of expert-labeled feedback, i.e., training the detector with both positive examples (correct statements) and negative examples (explicitly labeled incorrect statements), dramatically changes this conclusion. Under this enriched training regime, automated hallucination detection becomes possible for all countable language collections. These results highlight the essential role of expert-labeled examples in training hallucination detectors and provide theoretical support for feedback-based methods, such as reinforcement learning with human feedback (RLHF), which have proven critical for reliable LLM deployment.
• 摘要：自动幻觉检测是否可能？在这项工作中，我们引入了一个理论框架，以分析自动检测大语模型（LLMS）产生的幻觉的可行性。 Inspired by the classical Gold-Angluin framework for language identification and its recent adaptation to language generation by Kleinberg and Mullainathan, we investigate whether an algorithm, trained on examples drawn from an unknown target language $K$ (selected from a countable collection) and given access to an LLM, can reliably determine whether the LLM's outputs are correct or constitute hallucinations.首先，我们在幻觉检测与语言识别的经典任务之间建立了等效性。我们证明，任何幻觉检测方法都可以转换为语言识别方法，相反，可以将算法求解语言识别进行调整以进行幻觉检测。鉴于语言识别的固有难度，这意味着，如果仅使用目标语言中的正确示例对检测器进行培训，那么大多数语言收集的幻觉检测是不可能的。其次，我们表明，使用专家标记的反馈，即用积极的示例（正确的陈述）和负面示例（明确标记为错误的陈述）训练检测器，从而极大地改变了这一结论。在这个丰富的培训制度下，所有可数语言收集都有可能自动化的幻觉检测。这些结果突出了专家标记的示例在训练幻觉探测器中的重要作用，并为基于反馈的方法提供了理论支持，例如使用人类反馈（RLHF）的增强学习，这对于可靠的LLM部署至关重要。

Title: Democracy of AI Numerical Weather Models: An Example of Global Forecasting with FourCastNetv2 Made by a University Research Lab Using GPU

• Authors: Iman Khadir, Shane Stevenson, Henry Li, Kyle Krick, Abram Burrows, David Hall, Stan Posey, Samuel S.P. Shen
• Subjects: cs.LG, cs.AI, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2504.17028
• Pdf URL: https://arxiv.org/pdf/2504.17028
• Copy Paste: [[2504.17028]] Democracy of AI Numerical Weather Models: An Example of Global Forecasting with FourCastNetv2 Made by a University Research Lab Using GPU(https://arxiv.org/abs/2504.17028)
• Keywords: generation
• Abstract: This paper demonstrates the feasibility of democratizing AI-driven global weather forecasting models among university research groups by leveraging Graphics Processing Units (GPUs) and freely available AI models, such as NVIDIA's FourCastNetv2. FourCastNetv2 is an NVIDIA's advanced neural network for weather prediction and is trained on a 73-channel subset of the European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5) dataset at single levels and different pressure levels. Although the training specifications for FourCastNetv2 are not released to the public, the training documentation of the model's first generation, FourCastNet, is available to all users. The training had 64 A100 GPUs and took 16 hours to complete. Although NVIDIA's models offer significant reductions in both time and cost compared to traditional Numerical Weather Prediction (NWP), reproducing published forecasting results presents ongoing challenges for resource-constrained university research groups with limited GPU availability. We demonstrate both (i) leveraging FourCastNetv2 to create predictions through the designated application programming interface (API) and (ii) utilizing NVIDIA hardware to train the original FourCastNet model. Further, this paper demonstrates the capabilities and limitations of NVIDIA A100's for resource-limited research groups in universities. We also explore data management, training efficiency, and model validation, highlighting the advantages and challenges of using limited high-performance computing resources. Consequently, this paper and its corresponding GitHub materials may serve as an initial guide for other university research groups and courses related to machine learning, climate science, and data science to develop research and education programs on AI weather forecasting, and hence help democratize the AI NWP in the digital economy.
• 摘要：本文证明了通过利用图形处理单元（GPU）和自由使用的AI模型（例如NVIDIA的FourcastNetv2）来使AI驱动的AI驱动全球天气预测模型民主化的可行性。 FourcastNETV2是NVIDIA的天气预测的高级神经网络，并接受了欧洲中范围内天气预测中心（ECMWF）重新分析V5（ERE5）数据集的73渠道子集的培训。尽管未向公众发布了FourcastNETV2的培训规格，但该模型的第一代训练文档四castnet可供所有用户使用。该培训有64个A100 GPU，花了16个小时才能完成。尽管NVIDIA的模型与传统的数值天气预测（NWP）相比，时间和成本都大大减少，但复制已发布的预测结果对GPU可用性有限的资源受限大学研究小组提出了持续的挑战。我们演示了（i）利用FourcastNETV2通过指定的应用程序编程接口（API）和（ii）利用NVIDIA硬件来训练原始的四castnet模型来创建预测。此外，本文展示了NVIDIA A100对大学资源有限的研究小组的功能和局限性。我们还探索数据管理，培训效率和模型验证，突出了使用有限的高性能计算资源的优点和挑战。因此，本文及其相应的GITHUB材料可以作为其他大学研究小组和与机器学习，气候科学和数据科学有关的课程的初步指南，以制定有关AI天气预测的研究和教育计划，因此有助于使数字经济中的AI NWP民主化。

Title: Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation

• Authors: Rahul Vishwakarma
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17058
• Pdf URL: https://arxiv.org/pdf/2504.17058
• Copy Paste: [[2504.17058]] Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation(https://arxiv.org/abs/2504.17058)
• Keywords: generation, generative
• Abstract: The generation of high-quality synthetic data presents significant challenges in machine learning research, particularly regarding statistical fidelity and uncertainty quantification. Existing generative models produce compelling synthetic samples but lack rigorous statistical guarantees about their relation to the underlying data distribution, limiting their applicability in critical domains requiring robust error bounds. We address this fundamental limitation by presenting a novel framework that incorporates conformal prediction methodologies into Generative Adversarial Networks (GANs). By integrating multiple conformal prediction paradigms including Inductive Conformal Prediction (ICP), Mondrian Conformal Prediction, Cross-Conformal Prediction, and Venn-Abers Predictors, we establish distribution-free uncertainty quantification in generated samples. This approach, termed Conformalized GAN (cGAN), demonstrates enhanced calibration properties while maintaining the generative power of traditional GANs, producing synthetic data with provable statistical guarantees. We provide rigorous mathematical proofs establishing finite-sample validity guarantees and asymptotic efficiency properties, enabling the reliable application of synthetic data in high-stakes domains including healthcare, finance, and autonomous systems.
• 摘要：高质量合成数据的产生在机器学习研究中提出了重大挑战，尤其是在统计保真度和不确定性量化方面。现有的生成模型会产生引人注目的合成样本，但缺乏对它们与基础数据分布的关系的严格统计保证，从而限制了其在需要强大误差范围的关键域中的适用性。我们通过提出一个新颖的框架来解决这种基本限制，该框架将共形预测方法纳入生成对抗网络（GAN）。通过整合包括电感综合预测（ICP），蒙德里亚综合预测，跨符号预测和Venn-abers预测因子在内的多个共形预测范例，我们在生成的样品中建立了无分布不确定性量化。这种方法称为保融gAn（CGAN），表现出增强的校准属性，同时保持传统gan的生成能力，从而产生具有可证明的统计保证的合成数据。我们提供严格的数学证明，以确立有限样本的有效性保证和渐近效率属性，从而使合成数据可靠地应用在高风险领域，包括医疗保健，金融和自主系统。

Title: Distilling semantically aware orders for autoregressive image generation

• Authors: Rishav Pramanik, Antoine Poupon, Juan A. Rodriguez, Masih Aminbeidokhti, David Vazquez, Christopher Pal, Zhaozheng Yin, Marco Pedersoli
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17069
• Pdf URL: https://arxiv.org/pdf/2504.17069
• Copy Paste: [[2504.17069]] Distilling semantically aware orders for autoregressive image generation(https://arxiv.org/abs/2504.17069)
• Keywords: generation
• Abstract: Autoregressive patch-based image generation has recently shown competitive results in terms of image quality and scalability. It can also be easily integrated and scaled within Vision-Language models. Nevertheless, autoregressive models require a defined order for patch generation. While a natural order based on the dictation of the words makes sense for text generation, there is no inherent generation order that exists for image generation. Traditionally, a raster-scan order (from top-left to bottom-right) guides autoregressive image generation models. In this paper, we argue that this order is suboptimal, as it fails to respect the causality of the image content: for instance, when conditioned on a visual description of a sunset, an autoregressive model may generate clouds before the sun, even though the color of clouds should depend on the color of the sun and not the inverse. In this work, we show that first by training a model to generate patches in any-given-order, we can infer both the content and the location (order) of each patch during generation. Secondly, we use these extracted orders to finetune the any-given-order model to produce better-quality images. Through our experiments, we show on two datasets that this new generation method produces better images than the traditional raster-scan approach, with similar training costs and no extra annotations.
• 摘要：基于回归补丁的图像生成最近在图像质量和可扩展性方面显示了竞争性结果。它也可以轻松地集成和缩放在视觉语言模型中。然而，自回归模型需要定义的补丁订单。尽管基于单词的概念的自然顺序对于文本生成有意义，但图像生成没有固有的生成顺序。传统上，栅格扫描订单（从左上到右下角）指导自回归图像生成模型。在本文中，我们认为该顺序是次优的，因为它未能尊重图像内容的因果关系：例如，当以日落的视觉描述为条件时，自动回归模型也可能在太阳之前产生云，即使云的颜色应取决于太阳的颜色，而不是逆向。在这项工作中，我们首先通过训练模型以生成任何传递订单的贴片来证明，我们可以在生成过程中推断每个补丁的内容和位置（顺序）。其次，我们使用这些提取的订单来验证任何赋予的阶模型以产生更高质量的图像。通过我们的实验，我们在两个数据集上表明，这种新一代方法比传统的栅格扫描方法产生更好的图像，具有相似的培训成本，没有额外的注释。

Title: Sparse Phased Array Optimization Using Deep Learning

• Authors: David Lu, Lior Maman, Jackson Earls, Amir Boag, Pierre Baldi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17073
• Pdf URL: https://arxiv.org/pdf/2504.17073
• Copy Paste: [[2504.17073]] Sparse Phased Array Optimization Using Deep Learning(https://arxiv.org/abs/2504.17073)
• Keywords: generation
• Abstract: Antenna arrays are widely used in wireless communication, radar systems, radio astronomy, and military defense to enhance signal strength, directivity, and interference suppression. We introduce a deep learning-based optimization approach that enhances the design of sparse phased arrays by reducing grating lobes. This approach begins by generating sparse array configurations to address the non-convex challenges and extensive degrees of freedom inherent in array design. We use neural networks to approximate the non-convex cost function that estimates the energy ratio between the main and side lobes. This differentiable approximation facilitates cost function minimization through gradient descent, optimizing the antenna elements' coordinates and leading to an improved layout. Additionally, we incorporate a tailored penalty mechanism that includes various physical and design constraints into the optimization process, enhancing its robustness and practical applicability. We demonstrate the effectiveness of our method by applying it to the ten array configurations with the lowest initial costs, achieving further cost reductions ranging from 411% to 643%, with an impressive average improvement of 552%. By significantly reducing side lobe levels in antenna arrays, this breakthrough paves the way for ultra-precise beamforming, enhanced interference mitigation, and next-generation wireless and radar systems with unprecedented efficiency and clarity.
• 摘要：天线阵列广泛用于无线通信，雷达系统，射电天文学和军事防御，以增强信号强度，方向性和干扰抑制。我们介绍了一种基于深度学习的优化方法，该方法通过减少光栅裂片来增强稀疏分阶段的设计。这种方法首先生成稀疏的数组配置，以解决数组设计中固有的非凸挑战和广泛的自由度。我们使用神经网络来近似非凸成本函数，该函数估计主叶和侧叶之间的能量比。这种可区分的近似通过梯度下降促进成本函数最小化，从而优化了天线元件的坐标并导致布局改善。此外，我们将量身定制的惩罚机制纳入了优化过程中，该机制包括各种物理和设计约束，从而增强了其鲁棒性和实际适用性。我们通过将其应用于初始成本最低的十个阵列配置，从而证明了我们方法的有效性，从而实现了从411％到643％的进一步降低，平均平均提高了552％。通过显着降低天线阵列中的侧叶水平，这一突破为超正面的外观形成，增强的干扰缓解和下一代无线和雷达系统铺平了道路，具有前所未有的效率和清晰度。

Title: Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection

• Authors: Jens Petersen, Davide Abati, Amirhossein Habibian, Auke Wiggers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17076
• Pdf URL: https://arxiv.org/pdf/2504.17076
• Copy Paste: [[2504.17076]] Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection(https://arxiv.org/abs/2504.17076)
• Keywords: generative
• Abstract: Generative image models are increasingly being used for training data augmentation in vision tasks. In the context of automotive object detection, methods usually focus on producing augmented frames that look as realistic as possible, for example by replacing real objects with generated ones. Others try to maximize the diversity of augmented frames, for example by pasting lots of generated objects onto existing backgrounds. Both perspectives pay little attention to the locations of objects in the scene. Frame layouts are either reused with little or no modification, or they are random and disregard realism entirely. In this work, we argue that optimal data augmentation should also include realistic augmentation of layouts. We introduce a scene-aware probabilistic location model that predicts where new objects can realistically be placed in an existing scene. By then inpainting objects in these locations with a generative model, we obtain much stronger augmentation performance than existing approaches. We set a new state of the art for generative data augmentation on two automotive object detection tasks, achieving up to $2.8\times$ higher gains than the best competing approach ($+1.4$ vs. $+0.5$ mAP boost). We also demonstrate significant improvements for instance segmentation.
• 摘要：生成图像模型越来越多地用于训练视觉任务中的数据增强。在汽车对象检测的背景下，方法通常专注于产生尽可能逼真的增强帧，例如，通过用生成的对象替换真实对象。其他人则尝试最大程度地提高增强框架的多样性，例如，将许多生成的对象粘贴到现有背景上。这两种观点都很少关注场景中对象的位置。框架布局要么很少或没有修改，要么是随机的，或者完全无视现实主义。在这项工作中，我们认为最佳数据增强还应包括对布局的现实增强。我们介绍了一个场景感知的概率位置模型，该模型可预测现有的新对象可以将新对象置于现有场景中。那时，通过生成模型在这些位置进行覆盖对象，我们获得的增强性能比现有方法要强得多。我们为两项汽车对象检测任务的生成数据增强设置了新的最新技术状态，比最佳竞争方法（$+1.4 $ vs. $+0.5 $ MAP BOOST）获得了高达$ 2.8 \ tims $ $的收益。我们还显示出明显的改进，例如分割。

Title: We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedback

• Authors: Minkyu Choi, S P Sharan, Harsh Goel, Sahil Shah, Sandeep Chinchali
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17180
• Pdf URL: https://arxiv.org/pdf/2504.17180
• Copy Paste: [[2504.17180]] We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedback(https://arxiv.org/abs/2504.17180)
• Keywords: generation
• Abstract: Current text-to-video (T2V) generation models are increasingly popular due to their ability to produce coherent videos from textual prompts. However, these models often struggle to generate semantically and temporally consistent videos when dealing with longer, more complex prompts involving multiple objects or sequential events. Additionally, the high computational costs associated with training or fine-tuning make direct improvements impractical. To overcome these limitations, we introduce \(\projectname\), a novel zero-training video refinement pipeline that leverages neuro-symbolic feedback to automatically enhance video generation, achieving superior alignment with the prompts. Our approach first derives the neuro-symbolic feedback by analyzing a formal video representation and pinpoints semantically inconsistent events, objects, and their corresponding frames. This feedback then guides targeted edits to the original video. Extensive empirical evaluations on both open-source and proprietary T2V models demonstrate that \(\projectname\) significantly enhances temporal and logical alignment across diverse prompts by almost $40\%$.
• 摘要：当前的文本对视频（T2V）生成模型由于能够从文本提示中制作连贯的视频而越来越受欢迎。但是，在处理涉及多个对象或顺序事件的更长，更复杂的提示时，这些模型通常很难在语义和时间上生成一致的视频。此外，与培训或微调相关的高计算成本使直接改进不切实际。为了克服这些局限性，我们介绍了\（\ projectName \），这是一种新型的零训练视频改进管道，利用神经符号的反馈自动增强视频生成，从而与提示达到了较高的一致性。我们的方法首先通过分析形式的视频表示形式来得出神经符号反馈，并确定语义上不一致的事件，对象及其相应帧。然后，此反馈将针对原始视频的针对编辑。对开源和专有T2V模型的广泛经验评估表明，\（\ projectName \）显着提高了不同提示的时间和逻辑对齐，近40美元\％$ $。

Title: Synthetic Power Flow Data Generation Using Physics-Informed Denoising Diffusion Probabilistic Models

• Authors: Junfei Wang, Darshana Upadhyay, Marzia Zaman, Pirathayini Srikantha
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17210
• Pdf URL: https://arxiv.org/pdf/2504.17210
• Copy Paste: [[2504.17210]] Synthetic Power Flow Data Generation Using Physics-Informed Denoising Diffusion Probabilistic Models(https://arxiv.org/abs/2504.17210)
• Keywords: generation, generative
• Abstract: Many data-driven modules in smart grid rely on access to high-quality power flow data; however, real-world data are often limited due to privacy and operational constraints. This paper presents a physics-informed generative framework based on Denoising Diffusion Probabilistic Models (DDPMs) for synthesizing feasible power flow data. By incorporating auxiliary training and physics-informed loss functions, the proposed method ensures that the generated data exhibit both statistical fidelity and adherence to power system feasibility. We evaluate the approach on the IEEE 14-bus and 30-bus benchmark systems, demonstrating its ability to capture key distributional properties and generalize to out-of-distribution scenarios. Comparative results show that the proposed model outperforms three baseline models in terms of feasibility, diversity, and accuracy of statistical features. This work highlights the potential of integrating generative modelling into data-driven power system applications.
• 摘要：智能电网中许多数据驱动的模块都取决于访问高质量的功率流数据。但是，由于隐私和操作约束，实际数据通常受到限制。本文介绍了一个基于脱氧扩散概率模型（DDPM）的物理信息生成框架，用于合成可行的功率流数据。通过纳入辅助培训和物理知识的损失功能，提出的方法可确保生成的数据表现出统计保真度和对电力系统可行性的依从性。我们评估了IEEE 14公共汽车和30个公共汽车基准系统的方法，证明了其捕获关键分布属性并推广到分布外情景的能力。比较结果表明，在可行性，多样性和统计特征的准确性方面，所提出的模型优于三个基线模型。这项工作突出了将生成建模集成到数据驱动的电源系统应用程序中的潜力。

Title: Enhancing Variational Autoencoders with Smooth Robust Latent Encoding

• Authors: Hyomin Lee, Minseon Kim, Sangwon Jang, Jongheon Jeong, Sung Ju Hwang
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.17219
• Pdf URL: https://arxiv.org/pdf/2504.17219
• Copy Paste: [[2504.17219]] Enhancing Variational Autoencoders with Smooth Robust Latent Encoding(https://arxiv.org/abs/2504.17219)
• Keywords: generation, generative
• Abstract: Variational Autoencoders (VAEs) have played a key role in scaling up diffusion-based generative models, as in Stable Diffusion, yet questions regarding their robustness remain largely underexplored. Although adversarial training has been an established technique for enhancing robustness in predictive models, it has been overlooked for generative models due to concerns about potential fidelity degradation by the nature of trade-offs between performance and robustness. In this work, we challenge this presumption, introducing Smooth Robust Latent VAE (SRL-VAE), a novel adversarial training framework that boosts both generation quality and robustness. In contrast to conventional adversarial training, which focuses on robustness only, our approach smooths the latent space via adversarial perturbations, promoting more generalizable representations while regularizing with originality representation to sustain original fidelity. Applied as a post-training step on pre-trained VAEs, SRL-VAE improves image robustness and fidelity with minimal computational overhead. Experiments show that SRL-VAE improves both generation quality, in image reconstruction and text-guided image editing, and robustness, against Nightshade attacks and image editing attacks. These results establish a new paradigm, showing that adversarial training, once thought to be detrimental to generative models, can instead enhance both fidelity and robustness.
• 摘要：如稳定的扩散，有关扩散的生成模型在扩展基于扩散的生成模型方面发挥了关键作用，但有关其鲁棒性的问题仍然很大程度上尚未得到充实。尽管对抗性训练一直是一种增强预测模型中鲁棒性的既定技术，但由于担心绩效和稳健性之间权衡的性质，因此对生成模型的生成模型被忽略了。在这项工作中，我们挑战了这一推定，引入了平稳的稳健潜在VAE（SRL-VAE），这是一个新型的对抗训练框架，可提高发电质量和稳健性。与仅关注鲁棒性的常规对抗训练相反，我们的方法通过对抗性扰动使潜在空间平滑，从而促进了更具概括的表示，同时与原创性代表进行正规化以维持原始的忠诚度。 SRL-VAE是在训练前的VAE上应用后的训练步骤，可通过最小的计算开销来提高图像的鲁棒性和忠诚度。实验表明，SRL-VAE在图像重建和文本引导的图像编辑和鲁棒性方面都可以提高发电质量，以防止夜代攻击和图像编辑攻击。这些结果建立了一个新的范式，表明对抗性训练曾经被认为对生成模型有害，可以提高忠诚度和鲁棒性。

Title: Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusion

• Authors: Mengyu Qiao, Runze Tian, Yang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17223
• Pdf URL: https://arxiv.org/pdf/2504.17223
• Copy Paste: [[2504.17223]] Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusion(https://arxiv.org/abs/2504.17223)
• Keywords: generative
• Abstract: The rapid evolution of deep generative models poses a critical challenge to deepfake detection, as detectors trained on forgery-specific artifacts often suffer significant performance degradation when encountering unseen forgeries. While existing methods predominantly rely on spatial domain analysis, frequency domain operations are primarily limited to feature-level augmentation, leaving frequency-native artifacts and spatial-frequency interactions insufficiently exploited. To address this limitation, we propose a novel detection framework that integrates multi-scale spatial-frequency analysis for universal deepfake detection. Our framework comprises three key components: (1) a local spectral feature extraction pipeline that combines block-wise discrete cosine transform with cascaded multi-scale convolutions to capture subtle spectral artifacts; (2) a global spectral feature extraction pipeline utilizing scale-invariant differential accumulation to identify holistic forgery distribution patterns; and (3) a multi-stage cross-modal fusion mechanism that incorporates shallow-layer attention enhancement and deep-layer dynamic modulation to model spatial-frequency interactions. Extensive evaluations on widely adopted benchmarks demonstrate that our method outperforms state-of-the-art deepfake detection methods in both accuracy and generalizability.
• 摘要：深层生成模型的快速发展对深泡检测构成了关键的挑战，因为在伪造特定的伪影训练时，在遇到看不见的伪造时通常会遭受明显的性能降解。尽管现有方法主要依赖于空间域分析，但频域操作主要仅限于特征级增强，而频率本质的人工制品和空间频率相互作用不足。为了解决这一限制，我们提出了一个新的检测框架，该框架整合了多尺度的空间频率分析以进行通用深层检测。我们的框架包括三个关键组成部分：（1）局部光谱特征提取管道，将块的离散余弦变换与级联的多尺度卷积结合在一起，以捕获微妙的光谱伪像； （2）利用规模不变差分积累来识别整体伪造分布模式的全球光谱提取管道； （3）一种多阶段的跨模式融合机制，该机制结合了浅层注意力的增强和深层动态调制，以模拟空间频率相互作用。对广泛采用的基准测试的广泛评估表明，我们的方法在准确性和概括性方面都优于最先进的深层检测方法。

Title: Scene Perceived Image Perceptual Score (SPIPS): combining global and local perception for image quality assessment

• Authors: Zhiqiang Lao, Heather Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17234
• Pdf URL: https://arxiv.org/pdf/2504.17234
• Copy Paste: [[2504.17234]] Scene Perceived Image Perceptual Score (SPIPS): combining global and local perception for image quality assessment(https://arxiv.org/abs/2504.17234)
• Keywords: restoration, generation, quality assessment
• Abstract: The rapid advancement of artificial intelligence and widespread use of smartphones have resulted in an exponential growth of image data, both real (camera-captured) and virtual (AI-generated). This surge underscores the critical need for robust image quality assessment (IQA) methods that accurately reflect human visual perception. Traditional IQA techniques primarily rely on spatial features - such as signal-to-noise ratio, local structural distortions, and texture inconsistencies - to identify artifacts. While effective for unprocessed or conventionally altered images, these methods fall short in the context of modern image post-processing powered by deep neural networks (DNNs). The rise of DNN-based models for image generation, enhancement, and restoration has significantly improved visual quality, yet made accurate assessment increasingly complex. To address this, we propose a novel IQA approach that bridges the gap between deep learning methods and human perception. Our model disentangles deep features into high-level semantic information and low-level perceptual details, treating each stream separately. These features are then combined with conventional IQA metrics to provide a more comprehensive evaluation framework. This hybrid design enables the model to assess both global context and intricate image details, better reflecting the human visual process, which first interprets overall structure before attending to fine-grained elements. The final stage employs a multilayer perceptron (MLP) to map the integrated features into a concise quality score. Experimental results demonstrate that our method achieves improved consistency with human perceptual judgments compared to existing IQA models.
• 摘要：人工智能和智能手机广泛使用的快速发展导致图像数据的指数增长，包括真实（摄像头捕获）和虚拟（AI生成）。这种激增强调了对可靠的图像质量评估（IQA）方法的关键需求，该方法准确地反映了人类的视觉感知。传统的IQA技术主要依赖于空间特征，例如信噪比，局部结构扭曲和纹理矛盾，以识别伪影。尽管对未经加工或常规更改的图像有效，但这些方法在由深神经网络（DNNS）提供支持的现代图像后处理的背景下不足。基于DNN的模型的图像产生，增强和恢复的兴起已显着提高了视觉质量，但使准确的评估越来越复杂。为了解决这个问题，我们提出了一种新型的IQA方法，该方法弥合了深度学习方法与人类感知之间的差距。我们的模型将深度特征分解为高级语义信息和低级感知细节，分别处理每个流。然后将这些功能与常规的IQA指标相结合，以提供更全面的评估框架。这种混合设计使该模型能够评估全球环境和复杂的图像细节，更好地反映人类的视觉过程，该过程首先解释了整体结构，然后再参与细粒元素。最后阶段采用多层感知器（MLP）将集成功能映射到简洁的质量分数中。实验结果表明，与现有的IQA模型相比，我们的方法与人类感知判断的一致性提高了。

Title: Targeted AMP generation through controlled diffusion with efficient embeddings

• Authors: Diogo Soares, Leon Hetzel, Paulina Szymczak, Fabian Theis, Stephan Günnemann, Ewa Szczurek
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2504.17247
• Pdf URL: https://arxiv.org/pdf/2504.17247
• Copy Paste: [[2504.17247]] Targeted AMP generation through controlled diffusion with efficient embeddings(https://arxiv.org/abs/2504.17247)
• Keywords: generation, generative
• Abstract: Deep learning-based antimicrobial peptide (AMP) discovery faces critical challenges such as low experimental hit rates as well as the need for nuanced controllability and efficient modeling of peptide properties. To address these challenges, we introduce OmegAMP, a framework that leverages a diffusion-based generative model with efficient low-dimensional embeddings, precise controllability mechanisms, and novel classifiers with drastically reduced false positive rates for candidate filtering. OmegAMP enables the targeted generation of AMPs with specific physicochemical properties, activity profiles, and species-specific effectiveness. Moreover, it maximizes sample diversity while ensuring faithfulness to the underlying data distribution during generation. We demonstrate that OmegAMP achieves state-of-the-art performance across all stages of the AMP discovery pipeline, significantly advancing the potential of computational frameworks in combating antimicrobial resistance.
• 摘要：基于深度学习的抗微生物肽（AMP）发现面临着关键的挑战，例如低实验性命中率，以及对肽性能细微的可控性和有效建模的需求。为了应对这些挑战，我们引入了Omegamp，该框架利用具有有效的低维嵌入，精确的可控性机制和新型分类器的基于扩散的生成模型，并大大降低了候选滤波的假阳性率。 Omegamp可以实现具有特定物理化学特性，活性特征和物种特异性的有针对性的AMP。此外，它可以最大化样本多样性，同时确保在发电期间对基础数据分布的忠诚。我们证明，Omegamp在AMP发现管道的所有阶段都达到了最先进的性能，从而显着提高了计算框架在打击抗菌抗性方面的潜力。

Title: DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks

• Authors: Yinqi Li, Hong Chang, Ruibing Hou, Shiguang Shan, Xilin Chen
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.17253
• Pdf URL: https://arxiv.org/pdf/2504.17253
• Copy Paste: [[2504.17253]] DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks(https://arxiv.org/abs/2504.17253)
• Keywords: generation, generative
• Abstract: Diffusion models have shown remarkable progress in various generative tasks such as image and video generation. This paper studies the problem of leveraging pretrained diffusion models for performing discriminative tasks. Specifically, we extend the discriminative capability of pretrained frozen generative diffusion models from the classification task to the more complex object detection task, by "inverting" a pretrained layout-to-image diffusion model. To this end, a gradient-based discrete optimization approach for replacing the heavy prediction enumeration process, and a prior distribution model for making more accurate use of the Bayes' rule, are proposed respectively. Empirical results show that this method is on par with basic discriminative object detection baselines on COCO dataset. In addition, our method can greatly speed up the previous diffusion-based method for classification without sacrificing accuracy. Code and models are available at this https URL .
• 摘要：扩散模型在各种生成任务（例如图像和视频生成）中显示出了显着的进展。本文研究了利用预处理扩散模型来执行歧视任务的问题。具体而言，我们通过“反相”预验证的布局到图像扩散模型，将预验证的冷冻生成扩散模型从分类任务扩展到更复杂的对象检测任务的歧视能力。为此，分别提出了一种基于梯度的离散优化方法，用于替换大量预测过程，并分别提出了更准确地使用贝叶斯规则的先前分配模型。经验结果表明，此方法与可可数据集上的基本判别对象检测基线相当。此外，我们的方法可以大大加快基于基于扩散的分类方法，而无需牺牲准确性。代码和型号可在此HTTPS URL上找到。

Title: Symbolic Representation for Any-to-Any Generative Tasks

• Authors: Jiaqi Chen, Xiaoye Zhu, Yue Wang, Tianyang Liu, Xinhui Chen, Ying Chen, Chak Tou Leong, Yifei Ke, Joseph Liu, Yiwen Yuan, Julian McAuley, Li-jia Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17261
• Pdf URL: https://arxiv.org/pdf/2504.17261
• Copy Paste: [[2504.17261]] Symbolic Representation for Any-to-Any Generative Tasks(https://arxiv.org/abs/2504.17261)
• Keywords: generative
• Abstract: We propose a symbolic generative task description language and a corresponding inference engine capable of representing arbitrary multimodal tasks as structured symbolic flows. Unlike conventional generative models that rely on large-scale training and implicit neural representations to learn cross-modal mappings, often at high computational cost and with limited flexibility, our framework introduces an explicit symbolic representation comprising three core primitives: functions, parameters, and topological logic. Leveraging a pre-trained language model, our inference engine maps natural language instructions directly to symbolic workflows in a training-free manner. Our framework successfully performs over 12 diverse multimodal generative tasks, demonstrating strong performance and flexibility without the need for task-specific tuning. Experiments show that our method not only matches or outperforms existing state-of-the-art unified models in content quality, but also offers greater efficiency, editability, and interruptibility. We believe that symbolic task representations provide a cost-effective and extensible foundation for advancing the capabilities of generative AI.
• 摘要：我们提出了一个符号生成任务描述语言和相应的推理引擎，能够将任意多模式任务表示为结构化符号流。与传统的生成模型不同，这些模型依靠大规模训练和隐式神经表示来学习跨模式映射，通常以高计算成本和有限的灵活性来学习，我们的框架引入了一个明确的符号表示，其中包括三个核心基础：功能，参数和拓扑逻辑。利用预先训练的语言模型，我们的推理引擎以无训练的方式直接将自然语言指令映射到符号工作流程。我们的框架成功执行了12种多种模式生成任务，表明了强大的性能和灵活性，而无需特定于任务的调整。实验表明，我们的方法不仅在内容质量上匹配现有的最新统一模型，而且还提供了更高的效率，编辑性和中断性。我们认为，符号任务表示为推进生成AI的能力提供了一个具有成本效益且可扩展的基础。

Title: Towards Generalized and Training-Free Text-Guided Semantic Manipulation

• Authors: Yu Hong, Xiao Cai, Pengpeng Zeng, Shuai Zhang, Jingkuan Song, Lianli Gao, Heng Tao Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17269
• Pdf URL: https://arxiv.org/pdf/2504.17269
• Copy Paste: [[2504.17269]] Towards Generalized and Training-Free Text-Guided Semantic Manipulation(https://arxiv.org/abs/2504.17269)
• Keywords: generative
• Abstract: Text-guided semantic manipulation refers to semantically editing an image generated from a source prompt to match a target prompt, enabling the desired semantic changes (e.g., addition, removal, and style transfer) while preserving irrelevant contents. With the powerful generative capabilities of the diffusion model, the task has shown the potential to generate high-fidelity visual content. Nevertheless, existing methods either typically require time-consuming fine-tuning (inefficient), fail to accomplish multiple semantic manipulations (poorly extensible), and/or lack support for different modality tasks (limited generalizability). Upon further investigation, we find that the geometric properties of noises in the diffusion model are strongly correlated with the semantic changes. Motivated by this, we propose a novel $\textit{GTF}$ for text-guided semantic manipulation, which has the following attractive capabilities: 1) $\textbf{Generalized}$: our $\textit{GTF}$ supports multiple semantic manipulations (e.g., addition, removal, and style transfer) and can be seamlessly integrated into all diffusion-based methods (i.e., Plug-and-play) across different modalities (i.e., modality-agnostic); and 2) $\textbf{Training-free}$: $\textit{GTF}$ produces high-fidelity results via simply controlling the geometric relationship between noises without tuning or optimization. Our extensive experiments demonstrate the efficacy of our approach, highlighting its potential to advance the state-of-the-art in semantics manipulation.
• 摘要：文本指导的语义操作是指从源提示符中生成的图像以匹配目标提示，从而实现所需的语义更改（例如，加法，删除和样式传输），同时保留无关的内容，从而符合目标提示。借助扩散模型的强大生成能力，该任务表明了产生高保真视觉内容的潜力。然而，现有方法通常需要耗时的微调（效率低下），无法完成多种语义操作（差可扩展），也无法完成对不同模态任务（有限的可推广性）的支持。经过进一步研究，我们发现扩散模型中噪声的几何特性与语义变化密切相关。 Motivated by this, we propose a novel $\textit{GTF}$ for text-guided semantic manipulation, which has the following attractive capabilities: 1) $\textbf{Generalized}$: our $\textit{GTF}$ supports multiple semantic manipulations (e.g., addition, removal, and style transfer) and can be seamlessly integrated into all diffusion-based methods (i.e.,跨不同模态的插件）（即模态 - 不合时宜）; 2）$ \ textbf {triagh-free} $：$ \ textit {gtf} $通过简单地控制噪声之间的几何关系而不调整或优化，就会产生高效率结果。我们的广泛实验证明了我们方法的功效，强调了它在语义操纵中推进最新技术的潜力。

Title: DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition

• Authors: Yiyan Xu, Wuqiang Zheng, Wenjie Wang, Fengbin Zhu, Xinting Hu, Yang Zhang, Fuli Feng, Tat-Seng Chua
• Subjects: cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.17349
• Pdf URL: https://arxiv.org/pdf/2504.17349
• Copy Paste: [[2504.17349]] DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition(https://arxiv.org/abs/2504.17349)
• Keywords: generation
• Abstract: Personalized image generation has emerged as a promising direction in multimodal content creation. It aims to synthesize images tailored to individual style preferences (e.g., color schemes, character appearances, layout) and semantic intentions (e.g., emotion, action, scene contexts) by leveraging user-interacted history images and multimodal instructions. Despite notable progress, existing methods -- whether based on diffusion models, large language models, or Large Multimodal Models (LMMs) -- struggle to accurately capture and fuse user style preferences and semantic intentions. In particular, the state-of-the-art LMM-based method suffers from the entanglement of visual features, leading to Guidance Collapse, where the generated images fail to preserve user-preferred styles or reflect the specified semantics. To address these limitations, we introduce DRC, a novel personalized image generation framework that enhances LMMs through Disentangled Representation Composition. DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively, to form user-specific latent instructions that guide image generation within LMMs. Specifically, it involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation. Extensive experiments on two benchmarks demonstrate that DRC shows competitive performance while effectively mitigating the guidance collapse issue, underscoring the importance of disentangled representation learning for controllable and effective personalized image generation.
• 摘要：个性化的图像生成已成为多模式内容创建的有希望的方向。它旨在通过利用用户相互作用的历史图像和多模式指令来综合针对各个样式偏好（例如配色方案，角色外观，布局）和语义意图（例如，情感，动作，场景上下文）的图像。尽管取得了显着的进展，但现有方法（无论是基于扩散模型，大语言模型还是大型多模型模型（LMM））而难以准确捕获和融合用户样式的偏好和语义意图。特别是，基于LMM的最先进的方法遭受了视觉特征的纠缠，导致引导崩溃，其中生成的图像无法保留用户偏爱的样式或反映指定的语义。为了解决这些局限性，我们介绍了DRC，这是一个新型的个性化图像生成框架，可通过分离的表示形式组成来增强LMM。 DRC分别从历史图像和参考图像中明确提取用户样式的偏好和语义意图，以形成特定于用户的潜在指令，以指导LMMS中的图像生成。具体而言，它涉及两个关键的学习阶段：1）分解学习学习，该学习采用双重驱动器脱节器来明确分开样式和语义特征，并通过重建驱动的范式优化，并具有难以感知的重要性采样； 2）个性化建模，该建模应用于语义保存增强，以有效地适应了稳固的个性化发电的分离表示形式。在两个基准上进行的广泛实验表明，刚果民主共和国在有效地减轻指导崩溃问题的同时表现出竞争性能，强调了分解的表示学习的重要性，以实现可控和有效的个性化图像生成。

Title: I-INR: Iterative Implicit Neural Representations

• Authors: Ali Haider, Muhammad Salman Ali, Maryam Qamar, Tahir Khalil, Soo Ye Kim, Jihyong Oh, Enzo Tartaglione, Sung-Ho Bae
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17364
• Pdf URL: https://arxiv.org/pdf/2504.17364
• Copy Paste: [[2504.17364]] I-INR: Iterative Implicit Neural Representations(https://arxiv.org/abs/2504.17364)
• Keywords: restoration
• Abstract: Implicit Neural Representations (INRs) have revolutionized signal processing and computer vision by modeling signals as continuous, differentiable functions parameterized by neural networks. However, their inherent formulation as a regression problem makes them prone to regression to the mean, limiting their ability to capture fine details, retain high-frequency information, and handle noise effectively. To address these challenges, we propose Iterative Implicit Neural Representations (I-INRs) a novel plug-and-play framework that enhances signal reconstruction through an iterative refinement process. I-INRs effectively recover high-frequency details, improve robustness to noise, and achieve superior reconstruction quality. Our framework seamlessly integrates with existing INR architectures, delivering substantial performance gains across various tasks. Extensive experiments show that I-INRs outperform baseline methods, including WIRE, SIREN, and Gauss, in diverse computer vision applications such as image restoration, image denoising, and object occupancy prediction.
• 摘要：隐式神经表示（INRS）通过将信号建模为连续的，可区分的功能，从而彻底改变了信号处理和计算机视觉。但是，它们作为回归问题的固有表述使它们容易回归到平均值，限制了捕获细节，保留高频信息并有效地处理噪声的能力。为了应对这些挑战，我们提出了迭代隐式神经表示（i-Inrs）一个新颖的插件框架，该框架通过迭代完善过程增强了信号重建。 I-Inrs有效地恢复了高频细节，提高噪声的鲁棒性并实现了出色的重建质量。我们的框架与现有的INR体系结构无缝集成，从而在各种任务中带来了可观的性能增长。广泛的实验表明，在不同的计算机视觉应用中，I-Inrs优于基线方法（包括电线，警笛和高斯），例如图像恢复，图像降解和对象占用预测。

Title: TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation

• Authors: Ling You, Wenxuan Huang, Xinni Xie, Xiangyi Wei, Bangyan Li, Shaohui Lin, Yang Li, Changbo Wang
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.17365
• Pdf URL: https://arxiv.org/pdf/2504.17365
• Copy Paste: [[2504.17365]] TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation(https://arxiv.org/abs/2504.17365)
• Keywords: generation
• Abstract: Soccer is a globally popular sporting event, typically characterized by long matches and distinctive highlight moments. Recent advances in Multimodal Large Language Models (MLLMs) offer promising capabilities in temporal grounding and video understanding, soccer commentary generation often requires precise temporal localization and semantically rich descriptions over long-form video. However, existing soccer MLLMs often rely on the temporal a priori for caption generation, so they cannot process the soccer video end-to-end. While some traditional approaches follow a two-step paradigm that is complex and fails to capture the global context to achieve suboptimal performance. To solve the above issues, we present TimeSoccer, the first end-to-end soccer MLLM for Single-anchor Dense Video Captioning (SDVC) in full-match soccer videos. TimeSoccer jointly predicts timestamps and generates captions in a single pass, enabling global context modeling across 45-minute matches. To support long video understanding of soccer matches, we introduce MoFA-Select, a training-free, motion-aware frame compression module that adaptively selects representative frames via a coarse-to-fine strategy, and incorporates complementary training paradigms to strengthen the model's ability to handle long temporal sequences. Extensive experiments demonstrate that our TimeSoccer achieves State-of-The-Art (SoTA) performance on the SDVC task in an end-to-end form, generating high-quality commentary with accurate temporal alignment and strong semantic relevance.
• 摘要：足球是一项全球受欢迎的体育赛事，通常以长比赛和独特的精彩片刻为特征。多模式大语模型（MLLM）的最新进展在时间基础和视频理解方面提供了有希望的功能，足球评论的生成通常需要精确的时间定位，而语义上的语义丰富的描述则是长期视频。但是，现有的足球MLLM通常依赖于先验的时间来产生字幕，因此他们无法端对端处理足球视频。尽管某些传统方法遵循复杂的两步范式，并且未能捕获全球上下文以实现次优性能。为了解决上述问题，我们介绍了TimeOccer，这是第一个在全匹配足球视频中单台密集视频字幕（SDVC）的端到端足球MLLM。 TimeOccer共同预测时间戳并在单个通行证中生成字幕，从而使45分钟匹配的全局上下文建模。为了支持对足球比赛的长期视频理解，我们介绍了MOFA-Select，这是一种无训练，运动吸引的框架压缩模块，通过粗到5的策略自适应地选择了代表性的框架，并结合了互补的训练范式，以增强模型的能力处理长时间序列。广泛的实验表明，我们的时代occer在端到端形式的SDVC任务上实现了最先进的表现（SOTA），从而产生了具有准确的时间对齐和强烈的语义相关性的高质量评论。

Title: Doubly Adaptive Social Learning

• Authors: Marco Carpentiero, Virginia Bordignon, Vincenzo Matta, Ali H. Sayed
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17370
• Pdf URL: https://arxiv.org/pdf/2504.17370
• Copy Paste: [[2504.17370]] Doubly Adaptive Social Learning(https://arxiv.org/abs/2504.17370)
• Keywords: generation
• Abstract: In social learning, a network of agents assigns probability scores (beliefs) to some hypotheses of interest, which rule the generation of local streaming data observed by each agent. Belief formation takes place by means of an iterative two-step procedure where: i) the agents update locally their beliefs by using some likelihood model; and ii) the updated beliefs are combined with the beliefs of the neighboring agents, using a pooling rule. This procedure can fail to perform well in the presence of dynamic drifts, leading the agents to incorrect decision making. Here, we focus on the fully online setting where both the true hypothesis and the likelihood models can change over time. We propose the doubly adaptive social learning ($\text{A}^2\text{SL}$) strategy, which infuses social learning with the necessary adaptation capabilities. This goal is achieved by exploiting two adaptation stages: i) a stochastic gradient descent update to learn and track the drifts in the decision model; ii) and an adaptive belief update to track the true hypothesis changing over time. These stages are controlled by two adaptation parameters that govern the evolution of the error probability for each agent. We show that all agents learn consistently for sufficiently small adaptation parameters, in the sense that they ultimately place all their belief mass on the true hypothesis. In particular, the probability of choosing the wrong hypothesis converges to values on the order of the adaptation parameters. The theoretical analysis is illustrated both on synthetic data and by applying the $\text{A}^2\text{SL}$ strategy to a social learning problem in the online setting using real data.
• 摘要：在社会学习中，一个代理网络将概率分数（信念）分配给一些感兴趣的假设，这些假设统治着每个代理观察到的本地流数据。信仰形成是通过迭代两步程序进行的，其中：i）代理人通过使用某些可能性模型在本地更新他们的信念； ii）更新的信念与相邻代理的信念相结合，使用合并规则。在有动态漂移的情况下，此过程无法表现良好，导致代理做出错误的决策。在这里，我们专注于充分的在线设置，其中真正的假设和可能性模型都可以随着时间而变化。我们提出了双重自适应的社交学习（$ \ text {a}^2 \ text {sl} $）策略，该策略以必要的适应能力为社会学习注入。通过利用两个适应阶段来实现此目标：i）随机梯度下降更新，以学习和跟踪决策模型中的漂移； ii）和自适应信念更新，以跟踪随着时间的推移的真实假设的变化。这些阶段由两个调整参数控制，这些适应参数控制着每个代理的误差概率的演变。我们表明，所有代理人都一贯学习，以实现足够小的适应参数，因为他们最终将所有信仰质量都放在了真正的假设上。特别是，选择错误的假设的概率在适应参数的顺序上收敛到值。在综合数据和应用$ \ text {a}^2 \ text {sl} $策略中使用$ \ text {a}^2 \ tragent在在线设置中使用真实数据来说明理论分析。

Title: Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset

• Authors: Oussema Dhaouadi, Johannes Meier, Luca Wahl, Jacques Kaiser, Luca Scalerandi, Nick Wandelburg, Zhuolun Zhou, Nijanthan Berinpanathan, Holger Banzhaf, Daniel Cremers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17371
• Pdf URL: https://arxiv.org/pdf/2504.17371
• Copy Paste: [[2504.17371]] Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset(https://arxiv.org/abs/2504.17371)
• Keywords: generative
• Abstract: Accurate 3D trajectory data is crucial for advancing autonomous driving. Yet, traditional datasets are usually captured by fixed sensors mounted on a car and are susceptible to occlusion. Additionally, such an approach can precisely reconstruct the dynamic environment in the close vicinity of the measurement vehicle only, while neglecting objects that are further away. In this paper, we introduce the DeepScenario Open 3D Dataset (DSC3D), a high-quality, occlusion-free dataset of 6 degrees of freedom bounding box trajectories acquired through a novel monocular camera drone tracking pipeline. Our dataset includes more than 175,000 trajectories of 14 types of traffic participants and significantly exceeds existing datasets in terms of diversity and scale, containing many unprecedented scenarios such as complex vehicle-pedestrian interaction on highly populated urban streets and comprehensive parking maneuvers from entry to exit. DSC3D dataset was captured in five various locations in Europe and the United States and include: a parking lot, a crowded inner-city, a steep urban intersection, a federal highway, and a suburban intersection. Our 3D trajectory dataset aims to enhance autonomous driving systems by providing detailed environmental 3D representations, which could lead to improved obstacle interactions and safety. We demonstrate its utility across multiple applications including motion prediction, motion planning, scenario mining, and generative reactive traffic agents. Our interactive online visualization platform and the complete dataset are publicly available at this http URL, facilitating research in motion prediction, behavior modeling, and safety validation.
• 摘要：准确的3D轨迹数据对于推进自动驾驶至关重要。然而，传统数据集通常是由安装在汽车上的固定传感器捕获的，并且容易受到阻塞。此外，这种方法只能精确地重建测量工具附近的动态环境，同时忽略较远的物体。在本文中，我们介绍了DeepScenario Open 3D数据集（DSC3D），这是一种通过新颖的单眼相机无人机跟踪管道获取的高质量，无咬合的数据集，该数据集是6度自由度界限框轨迹。我们的数据集包括超过175,000种交通参与者的175,000多个轨迹，并在多样性和规模上大大超过了现有的数据集，其中包含许多前所未有的场景，例如人口稠密的城市街道上的复杂车辆互动互动以及从进入进入的全面停车位上的全面停车位。 DSC3D数据集被捕获在欧洲和美国的五个不同地点，其中包括：一个停车场，拥挤的市中心，陡峭的城市十字路口，联邦高速公路和郊区交叉路口。我们的3D轨迹数据集旨在通过提供详细的环境3D表示来增强自动驾驶系统，从而可以改善障碍互动和安全性。我们在多个应用程序中演示了它的实用性，包括运动预测，运动计划，场景挖掘和生成反应性交通代理。我们的交互式在线可视化平台和完整的数据集可在此HTTP URL上公开获得，从而促进了运动预测，行为建模和安全验证的研究。

Title: FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding

• Authors: De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, Jan Kautz
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17447
• Pdf URL: https://arxiv.org/pdf/2504.17447
• Copy Paste: [[2504.17447]] FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding(https://arxiv.org/abs/2504.17447)
• Keywords: generation
• Abstract: There has been impressive progress in Large Multimodal Models (LMMs). Recent works extend these models to long inputs, including multi-page documents and long videos. However, the model size and performance of these long context models are still limited due to the computational cost in both training and inference. In this work, we explore an orthogonal direction and process long inputs without long context LMMs. We propose Frame Selection Augmented Generation (FRAG), where the model first selects relevant frames within the input, and then only generates the final outputs based on the selected frames. The core of the selection process is done by scoring each frame independently, which does not require long context processing. The frames with the highest scores are then selected by a simple Top-K selection. We show that this frustratingly simple framework is applicable to both long videos and multi-page documents using existing LMMs without any fine-tuning. We consider two models, LLaVA-OneVision and InternVL2, in our experiments and show that FRAG consistently improves the performance and achieves state-of-the-art performances for both long video and long document understanding. For videos, FRAG substantially improves InternVL2-76B by 5.8% on MLVU and 3.7% on Video-MME. For documents, FRAG achieves over 20% improvements on MP-DocVQA compared with recent LMMs specialized in long document understanding. Code is available at: this https URL
• 摘要：大型多模型（LMM）取得了令人印象深刻的进步。最近的作品将这些模型扩展到了长期输入，包括多页文档和长视频。但是，由于培训和推理的计算成本，这些长上下文模型的模型大小和性能仍然受到限制。在这项工作中，我们探讨了没有长上下文LMM的正交方向和过程长输入。我们提出了框架选择增强生成（FRAG），其中模型首先选择输入中的相关帧，然后仅根据所选框架生成最终输出。选择过程的核心是通过独立评分每个帧来完成的，这不需要长上下文处理。然后通过简单的TOP-K选择选择具有最高分数的帧。我们表明，这种令人沮丧的简单框架适用于使用现有LMM的长视频和多页文档，而无需进行任何微调。我们在实验中考虑了两个模型，即Llava-onevision和internvl2，并表明Fraf始终如一地提高了性能，并实现了长期视频和长期文档理解的最新性能。对于视频，Frag在MLVU上实际上将Intervl2-76B提高了5.8％，Video-MME上的3.7％提高了3.7％。对于文档，与最新的LMMS相比，Frag在MP-DOCVQA方面取得了20％以上的改善。代码可用：此HTTPS URL

Title: Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks

• Authors: Zhiying Li, Yeying Jin, Fan Shen, Zhi Liu, Weibin Chen, Pengju Zhang, Xiaomei Zhang, Boyu Chen, Michael Shen, Kejian Wu, Zhaoxin Fan, Jin Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17457
• Pdf URL: https://arxiv.org/pdf/2504.17457
• Copy Paste: [[2504.17457]] Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks(https://arxiv.org/abs/2504.17457)
• Keywords: generation
• Abstract: Expressive human pose and shape estimation (EHPS) is crucial for digital human generation, especially in applications like live streaming. While existing research primarily focuses on reducing estimation errors, it largely neglects robustness and security aspects, leaving these systems vulnerable to adversarial attacks. To address this significant challenge, we propose the \textbf{Tangible Attack (TBA)}, a novel framework designed to generate adversarial examples capable of effectively compromising any digital human generation model. Our approach introduces a \textbf{Dual Heterogeneous Noise Generator (DHNG)}, which leverages Variational Autoencoders (VAE) and ControlNet to produce diverse, targeted noise tailored to the original image features. Additionally, we design a custom \textbf{adversarial loss function} to optimize the noise, ensuring both high controllability and potent disruption. By iteratively refining the adversarial sample through multi-gradient signals from both the noise and the state-of-the-art EHPS model, TBA substantially improves the effectiveness of adversarial attacks. Extensive experiments demonstrate TBA's superiority, achieving a remarkable 41.0\% increase in estimation error, with an average improvement of approximately 17.0\%. These findings expose significant security vulnerabilities in current EHPS models and highlight the need for stronger defenses in digital human generation systems.
• 摘要：富有表现力的人姿势和形状估计（EHP）对于数字人类一代至关重要，尤其是在现场流中的应用中。尽管现有的研究主要侧重于减少估计错误，但它在很大程度上忽略了稳健性和安全性方面，而这些系统使这些系统容易受到对抗性攻击的影响。为了应对这一重大挑战，我们提出了\ textbf {有形攻击（TBA）}，这是一个新颖的框架，旨在生成能够有效损害任何数字人类生成模型的对抗性示例。我们的方法引入了\ textbf {双重异质噪声发生器（DHNG）}，该噪声发生器（DHNG）}利用变异自动编码器（VAE）和ControlNet产生了针对原始图像特征量身定制的多样的，有针对性的噪声。此外，我们设计了一个自定义\ textbf {对抗损耗函数}，以优化噪声，以确保高可控性和有效的破坏。通过迭代地通过来自噪声和最新EHPS模型的多速率信号来精炼对抗性样本，TBA实质上提高了对抗性攻击的有效性。广泛的实验表明了TBA的优势，估计误差的41.0 \％增加，平均提高约为17.0 \％。这些发现在当前的EHPS模型中暴露了重大的安全漏洞，并强调了在数字人类发电系统中对更强大防御的需求。

Title: RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation

• Authors: Aviv Slobodkin, Hagai Taitelbaum, Yonatan Bitton, Brian Gordon, Michal Sokolik, Nitzan Bitton Guetta, Almog Gueta, Royi Rassin, Itay Laish, Dani Lischinski, Idan Szpektor
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17502
• Pdf URL: https://arxiv.org/pdf/2504.17502
• Copy Paste: [[2504.17502]] RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation(https://arxiv.org/abs/2504.17502)
• Keywords: generation
• Abstract: Subject-driven text-to-image (T2I) generation aims to produce images that align with a given textual description, while preserving the visual identity from a referenced subject image. Despite its broad downstream applicability -- ranging from enhanced personalization in image generation to consistent character representation in video rendering -- progress in this field is limited by the lack of reliable automatic evaluation. Existing methods either assess only one aspect of the task (i.e., textual alignment or subject preservation), misalign with human judgments, or rely on costly API-based evaluation. To address this, we introduce RefVNLI, a cost-effective metric that evaluates both textual alignment and subject preservation in a single prediction. Trained on a large-scale dataset derived from video-reasoning benchmarks and image perturbations, RefVNLI outperforms or matches existing baselines across multiple benchmarks and subject categories (e.g., \emph{Animal}, \emph{Object}), achieving up to 6.4-point gains in textual alignment and 8.5-point gains in subject consistency. It also excels with lesser-known concepts, aligning with human preferences at over 87\% accuracy.
• 摘要：主题驱动的文本对图像（T2I）生成旨在产生与给定文本描述一致的图像，同时从引用的主题图像中保留视觉标识。尽管它的下游适用性广泛 - 从图像生成中的个性化增强到视频渲染中的一致性角色表示 - 该领域的进展受到缺乏可靠的自动评估的限制。现有方法仅评估任务的一个方面（即文本一致性或主题保存），与人类判断不一致，或依靠基于API的昂贵评估。为了解决这个问题，我们介绍了Refvnli，这是一种具有成本效益的度量标准，可以评估单个预测中的文本一致性和主题保存。在一个大规模的数据集上训练，该数据集源自视频策划的基准和图像扰动，Refvnli的表现或匹配多个基准和主题类别的现有基准（例如\ \ emph {andial}，\ emph}，\ emph {object}），可在6.4点（6.4点）中添加到6.4点的文本升级和8.5-point in 8.5-point in 8.5-point in 8.5-point。它还具有鲜为人知的概念，与超过87％精度的人类偏好保持一致。

Title: Text-to-Image Alignment in Denoising-Based Models through Step Selection

• Authors: Paul Grimal, Hervé Le Borgne, Olivier Ferret
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17525
• Pdf URL: https://arxiv.org/pdf/2504.17525
• Copy Paste: [[2504.17525]] Text-to-Image Alignment in Denoising-Based Models through Step Selection(https://arxiv.org/abs/2504.17525)
• Keywords: generation, generative
• Abstract: Visual generative AI models often encounter challenges related to text-image alignment and reasoning limitations. This paper presents a novel method for selectively enhancing the signal at critical denoising steps, optimizing image generation based on input semantics. Our approach addresses the shortcomings of early-stage signal modifications, demonstrating that adjustments made at later stages yield superior results. We conduct extensive experiments to validate the effectiveness of our method in producing semantically aligned images on Diffusion and Flow Matching model, achieving state-of-the-art performance. Our results highlight the importance of a judicious choice of sampling stage to improve performance and overall image alignment.
• 摘要：视觉生成的AI模型通常会遇到与文本图像对齐和推理限制有关的挑战。本文提出了一种新颖的方法，可在关键的denoising步骤中有选择地增强信号，从而根据输入语义优化图像生成。我们的方法解决了早期信号修改的缺点，表明在较晚阶段进行的调整会产生较高的结果。我们进行了广泛的实验，以验证方法在扩散和流匹配模型上产生语义对齐图像的有效性，从而实现最新性能。我们的结果强调了采样阶段明智选择以提高性能和整体图像对齐的重要性。

Title: TarDiff: Target-Oriented Diffusion Guidance for Synthetic Electronic Health Record Time Series Generation

• Authors: Bowen Deng, Chang Xu, Hao Li, Yuhao Huang, Min Hou, Jiang Bian
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17613
• Pdf URL: https://arxiv.org/pdf/2504.17613
• Copy Paste: [[2504.17613]] TarDiff: Target-Oriented Diffusion Guidance for Synthetic Electronic Health Record Time Series Generation(https://arxiv.org/abs/2504.17613)
• Keywords: generation
• Abstract: Synthetic Electronic Health Record (EHR) time-series generation is crucial for advancing clinical machine learning models, as it helps address data scarcity by providing more training data. However, most existing approaches focus primarily on replicating statistical distributions and temporal dependencies of real-world data. We argue that fidelity to observed data alone does not guarantee better model performance, as common patterns may dominate, limiting the representation of rare but important conditions. This highlights the need for generate synthetic samples to improve performance of specific clinical models to fulfill their target outcomes. To address this, we propose TarDiff, a novel target-oriented diffusion framework that integrates task-specific influence guidance into the synthetic data generation process. Unlike conventional approaches that mimic training data distributions, TarDiff optimizes synthetic samples by quantifying their expected contribution to improving downstream model performance through influence functions. Specifically, we measure the reduction in task-specific loss induced by synthetic samples and embed this influence gradient into the reverse diffusion process, thereby steering the generation towards utility-optimized data. Evaluated on six publicly available EHR datasets, TarDiff achieves state-of-the-art performance, outperforming existing methods by up to 20.4% in AUPRC and 18.4% in AUROC. Our results demonstrate that TarDiff not only preserves temporal fidelity but also enhances downstream model performance, offering a robust solution to data scarcity and class imbalance in healthcare analytics.
• 摘要：合成电子健康记录（EHR）时间序列的产生对于推进临床机器学习模型至关重要，因为它通过提供更多的培训数据来帮助解决数据稀缺。但是，大多数现有方法主要侧重于复制现实世界数据的统计分布和时间依赖性。我们认为，仅观察到的数据的忠诚并不能保证更好的模型性能，因为共同模式可能会占主导地位，从而限制了稀有但重要条件的表示。这凸显了生成合成样本以提高特定临床模型的性能以实现其目标结果的需求。为了解决这个问题，我们提出了Tardiff，这是一个新型面向目标的扩散框架，将特定于任务的影响指导集成到合成数据生成过程中。与模仿培训数据分布的常规方法不同，Tardiff通过量化其通过影响功能来改善下游模型性能的预期贡献来优化合成样本。具体而言，我们测量了合成样本引起的特定任务损失的减少，并将这种影响梯度嵌入反向扩散过程中，从而将生成转向了公用事业优化的数据。 Tardiff在六个公开可用的EHR数据集上进行了评估，在AUPRC中，最大的方法优于现有方法高达20.4％，而AUROC的表现高达20.4％。我们的结果表明，TARDIFF不仅保留了时间忠诚，而且还提高了下游模型性能，为数据稀缺性和医疗保健分析中的阶级失衡提供了强有力的解决方案。

Title: DiMeR: Disentangled Mesh Reconstruction Model

• Authors: Lutao Jiang, Jiantao Lin, Kanghao Chen, Wenhang Ge, Xin Yang, Yifan Jiang, Yuanhuiyi Lyu, Xu Zheng, Yingcong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17670
• Pdf URL: https://arxiv.org/pdf/2504.17670
• Copy Paste: [[2504.17670]] DiMeR: Disentangled Mesh Reconstruction Model(https://arxiv.org/abs/2504.17670)
• Keywords: generative
• Abstract: With the advent of large-scale 3D datasets, feed-forward 3D generative models, such as the Large Reconstruction Model (LRM), have gained significant attention and achieved remarkable success. However, we observe that RGB images often lead to conflicting training objectives and lack the necessary clarity for geometry reconstruction. In this paper, we revisit the inductive biases associated with mesh reconstruction and introduce DiMeR, a novel disentangled dual-stream feed-forward model for sparse-view mesh reconstruction. The key idea is to disentangle both the input and framework into geometry and texture parts, thereby reducing the training difficulty for each part according to the Principle of Occam's Razor. Given that normal maps are strictly consistent with geometry and accurately capture surface variations, we utilize normal maps as exclusive input for the geometry branch to reduce the complexity between the network's input and output. Moreover, we improve the mesh extraction algorithm to introduce 3D ground truth supervision. As for texture branch, we use RGB images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust capabilities across various tasks, including sparse-view reconstruction, single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR significantly outperforms previous methods, achieving over 30% improvement in Chamfer Distance on the GSO and OmniObject3D dataset.
• 摘要：随着大规模3D数据集的出现，饲料前3D生成模型（例如大型重建模型（LRM））已获得了引起的重大关注并取得了巨大的成功。但是，我们观察到RGB图像通常会导致训练目标相互矛盾，并且缺乏几何重建的必要清晰度。在本文中，我们重新审视了与网格重建相关的电感偏差，并引入了二聚体，这是一种新型的散布双流馈送模型，用于稀疏视图网格重建。关键的想法是将输入和框架分解为几何和纹理部分，从而根据Occam剃须刀的原理减少每个部分的训练难度。鉴于正常地图严格与几何形状一致并准确捕获表面变化，因此我们利用正常地图作为几何分支的独家输入来降低网络输入和输出之间的复杂性。此外，我们改进了网格提取算法，以引入3D地面真相监督。至于纹理分支，我们使用RGB图像作为输入来获得纹理网格。总体而言，二聚体展示了各种任务的强大功能，包括稀疏视图重建，单图像到3D和文本到3D。许多实验表明，二聚体的表现明显优于先前的方法，在GSO和OmniObject3D数据集上的倒角距离提高了30％以上。

Title: Fault Diagnosis in New Wind Turbines using Knowledge from Existing Turbines by Generative Domain Adaptation

• Authors: Stefan Jonas, Angela Meyer
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2504.17709
• Pdf URL: https://arxiv.org/pdf/2504.17709
• Copy Paste: [[2504.17709]] Fault Diagnosis in New Wind Turbines using Knowledge from Existing Turbines by Generative Domain Adaptation(https://arxiv.org/abs/2504.17709)
• Keywords: generative
• Abstract: Intelligent condition monitoring of wind turbines is essential for reducing downtimes. Machine learning models trained on wind turbine operation data are commonly used to detect anomalies and, eventually, operation faults. However, data-driven normal behavior models (NBMs) require a substantial amount of training data, as NBMs trained with scarce data may result in unreliable fault diagnosis. To overcome this limitation, we present a novel generative deep learning approach to make SCADA samples from one wind turbine lacking training data resemble SCADA data from wind turbines with representative training data. Through CycleGAN-based domain mapping, our method enables the application of an NBM trained on an existing wind turbine to one with severely limited data. We demonstrate our approach on field data mapping SCADA samples across 7 substantially different WTs. Our findings show significantly improved fault diagnosis in wind turbines with scarce data. Our method achieves the most similar anomaly scores to an NBM trained with abundant data, outperforming NBMs trained on scarce training data with improvements of +10.3% in F1-score when 1 month of training data is available and +16.8% when 2 weeks are available. The domain mapping approach outperforms conventional fine-tuning at all considered degrees of data scarcity, ranging from 1 to 8 weeks of training data. The proposed technique enables earlier and more reliable fault diagnosis in newly installed wind farms, demonstrating a novel and promising research direction to improve anomaly detection when faced with training data scarcity.
• 摘要：风力涡轮机的智能状况监测对于减少降落至关重要。经过风力涡轮机操作数据训练的机器学习模型通常用于检测异常情况，最终用于操作故障。但是，数据驱动的正常行为模型（NBMS）需要大量的培训数据，因为接受稀缺数据训练的NBM可能会导致故障诊断不可靠。为了克服这一局限性，我们提出了一种新颖的生成深度学习方法，以从一个风力涡轮机中制作出SCADA样品，缺少训练数据，类似于来自风力涡轮机的SCADA数据，并具有代表性的培训数据。通过基于自行车的域映射，我们的方法可以将在现有风力涡轮机上训练的NBM应用于具有严重数据的培训。我们证明了我们在7个基本不同的WT中的现场数据映射SCADA样品的方法。我们的发现显示，由于数据稀缺，风力涡轮机的故障诊断显着改善。我们的方法达到了与接受丰富数据培训的NBM的最相似异常得分，在可用的1个月培训数据可用时，在F1评分中的稀缺培训数据训练的NBM均超过了稀缺培训数据，而在2周可用时 +16.8％。在所有考虑的数据稀缺程度上，域映射方法的表现优于常规微调，范围从1到8周的培训数据。该提出的技术可以在新安装的风电场中较早，更可靠的故障诊断，这表明面对培训数据稀缺时，可以提高新颖而有希望的研究方向，以改善异常检测。

Title: Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields

• Authors: Zhuo He, Paul Henderson, Nicolas Pugeault
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17712
• Pdf URL: https://arxiv.org/pdf/2504.17712
• Copy Paste: [[2504.17712]] Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields(https://arxiv.org/abs/2504.17712)
• Keywords: generation, generative
• Abstract: StyleGAN has demonstrated the ability of GANs to synthesize highly-realistic faces of imaginary people from random noise. One limitation of GAN-based image generation is the difficulty of controlling the features of the generated image, due to the strong entanglement of the low-dimensional latent space. Previous work that aimed to control StyleGAN with image or text prompts modulated sampling in W latent space, which is more expressive than Z latent space. However, W space still has restricted expressivity since it does not control the feature synthesis directly; also the feature embedding in W space requires a pre-training process to reconstruct the style signal, limiting its application. This paper introduces the concept of "generative fields" to explain the hierarchical feature synthesis in StyleGAN, inspired by the receptive fields of convolution neural networks (CNNs). Additionally, we propose a new image editing pipeline for StyleGAN using generative field theory and the channel-wise style latent space S, utilizing the intrinsic structural feature of CNNs to achieve disentangled control of feature synthesis at synthesis time.
• 摘要：Stylegan证明了甘斯从随机噪声中综合想象中的人的高度现实面孔的能力。基于GAN的图像生成的一个局限性是控制生成图像的特征，这是由于低维潜在空间的牢固纠缠所致。旨在用图像或文本提示在W潜在空间中调制采样的先前工作，该采样比Z潜在空间更具表现力。但是，W空间仍然具有限制的表达性，因为它不能直接控制特征合成。此外，在W空间中嵌入的功能还需要一个预训练的过程来重建样式信号，从而限制其应用程序。本文介绍了“生成领域”的概念，以解释stylegan中的层次特征综合，灵感来自卷积神经网络（CNN）的接受领域。此外，我们利用CNN的固有结构特征来实现合成时特征综合的固有结构特征，为Persinative Field Theory和渠道风格的潜在空间S提出了一个新的图像编辑管道，使用生成场理论和频道风格的潜在空间s。

Title: DPMambaIR:All-in-One Image Restoration via Degradation-Aware Prompt State Space Model

• Authors: Zhanwen Liu, Sai Zhou, Yuchao Dai, Yang Wang, Yisheng An, Xiangmo Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17732
• Pdf URL: https://arxiv.org/pdf/2504.17732
• Copy Paste: [[2504.17732]] DPMambaIR:All-in-One Image Restoration via Degradation-Aware Prompt State Space Model(https://arxiv.org/abs/2504.17732)
• Keywords: restoration
• Abstract: All-in-One image restoration aims to address multiple image degradation problems using a single model, significantly reducing training costs and deployment complexity compared to traditional methods that design dedicated models for each degradation type. Existing approaches typically rely on Degradation-specific models or coarse-grained degradation prompts to guide image restoration. However, they lack fine-grained modeling of degradation information and face limitations in balancing multi-task conflicts. To overcome these limitations, we propose DPMambaIR, a novel All-in-One image restoration framework. By integrating a Degradation-Aware Prompt State Space Model (DP-SSM) and a High-Frequency Enhancement Block (HEB), DPMambaIR enables fine-grained modeling of complex degradation information and efficient global integration, while mitigating the loss of high-frequency details caused by task competition. Specifically, the DP-SSM utilizes a pre-trained degradation extractor to capture fine-grained degradation features and dynamically incorporates them into the state space modeling process, enhancing the model's adaptability to diverse degradation types. Concurrently, the HEB supplements high-frequency information, effectively addressing the loss of critical details, such as edges and textures, in multi-task image restoration scenarios. Extensive experiments on a mixed dataset containing seven degradation types show that DPMambaIR achieves the best performance, with 27.69dB and 0.893 in PSNR and SSIM, respectively. These results highlight the potential and superiority of DPMambaIR as a unified solution for All-in-One image restoration.
• 摘要：与为每种降解类型设计专门模型的传统方法相比，多合一图像恢复旨在使用单个模型解决多个图像降解问题，从而大大降低了培训成本和部署复杂性。现有方法通常依赖于特定于降解的模型或粗粒降解提示来指导图像恢复。但是，他们缺乏平衡多任务冲突的降解信息和面部限制的细粒度建模。为了克服这些局限性，我们提出了DPMambair，这是一个新型的多合一图像恢复框架。通过整合降解感知的及时状态空间模型（DP-SSM）和高频增强块（HEB），DPMambair可以对复杂的降级信息和有效的全球整合进行细粒度建模，同时减轻由任务竞争引起的高频详细信息的损失。具体而言，DP-SSM利用预训练的降解提取器来捕获细粒的降解功能，并将其动态地纳入状态空间建模过程中，从而增强了模型对各种降解类型的适应性。同时，HEB补充了高频信息，可有效解决关键细节的丢失，例如边缘和纹理，在多任务图像恢复方案中。在包含七种降解类型的混合数据集上进行的广泛实验表明，DPMambair的性能分别为27.69db和0.893，在PSNR和SSIM中分别为0.893。这些结果突出了DPMambair作为多合一图像恢复的统一解决方案的潜力和优势。

Title: Interpretable Early Detection of Parkinson's Disease through Speech Analysis

• Authors: Lorenzo Simone, Mauro Giuseppe Camporeale, Vito Marco Rubino, Vincenzo Gervasi, Giovanni Dimauro
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17739
• Pdf URL: https://arxiv.org/pdf/2504.17739
• Copy Paste: [[2504.17739]] Interpretable Early Detection of Parkinson's Disease through Speech Analysis(https://arxiv.org/abs/2504.17739)
• Keywords: generative
• Abstract: Parkinson's disease is a progressive neurodegenerative disorder affecting motor and non-motor functions, with speech impairments among its earliest symptoms. Speech impairments offer a valuable diagnostic opportunity, with machine learning advances providing promising tools for timely detection. In this research, we propose a deep learning approach for early Parkinson's disease detection from speech recordings, which also highlights the vocal segments driving predictions to enhance interpretability. This approach seeks to associate predictive speech patterns with articulatory features, providing a basis for interpreting underlying neuromuscular impairments. We evaluated our approach using the Italian Parkinson's Voice and Speech Database, containing 831 audio recordings from 65 participants, including both healthy individuals and patients. Our approach showed competitive classification performance compared to state-of-the-art methods, while providing enhanced interpretability by identifying key speech features influencing predictions.
• 摘要：帕金森氏病是影响运动和非运动功能的一种进行性神经退行性疾病，其最早的症状是言语障碍。语音障碍提供了宝贵的诊断机会，机器学习的进步为及时检测提供了有希望的工具。在这项研究中，我们为早期帕金森氏病从语音录音中检测出一种深度学习方法，这也凸显了人声段推动预测以增强可解释性。这种方法试图将预测性语音模式与关节特征联系起来，从而为解释潜在的神经肌肉损伤提供了基础。我们使用意大利帕金森的语音和语音数据库评估了我们的方法，其中包含来自65名参与者的831次录音，包括健康的个体和患者。与最新方法相比，我们的方法显示出竞争性的分类性能，同时通过识别影响预测的关键语音特征来提供增强的解释性。

Title: Step1X-Edit: A Practical Framework for General Image Editing

• Authors: Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, Guopeng Li, Yuang Peng, Quan Sun, Jingwei Wu, Yan Cai, Zheng Ge, Ranchen Ming, Lei Xia, Xianfang Zeng, Yibo Zhu, Binxing Jiao, Xiangyu Zhang, Gang Yu, Daxin Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17761
• Pdf URL: https://arxiv.org/pdf/2504.17761
• Copy Paste: [[2504.17761]] Step1X-Edit: A Practical Framework for General Image Editing(https://arxiv.org/abs/2504.17761)
• Keywords: generation
• Abstract: In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.
• 摘要：近年来，图像编辑模型见证了巨大而快速的发展。 GPT-4O和Gemini2 Flash等尖端多模型的最新揭幕引入了非常有前途的图像编辑功能。这些模型表明了满足绝大多数用户驱动的编辑要求的令人印象深刻的才能，这标志着图像操纵领域的显着进步。但是，这些封闭源型号之间的开源算法之间仍然存在很大的差距。因此，在本文中，我们旨在发布一个名为STEP1X-EDIT的最先进的图像编辑模型，该模型可以与GPT-4O和Gemini2 Flash（Gemini2 Flash）相比提供可比的性能。更具体地说，我们采用多模式LLM来处理参考图像和用户的编辑指令。已提取潜在嵌入并与扩散图像解码器集成以获得目标图像。为了训练模型，我们构建了数据生成管道以生成高质量的数据集。为了进行评估，我们开发了Gedit Bench，这是一种植根于现实世界用户说明的新颖基准。 GEDIT板台上的实验结果表明，STEP1X-EDIT的表现优于现有的开源基线，并接近领先的专有模型的性能，从而为图像编辑领域做出了重大贡献。

Title: Dynamic Camera Poses and Where to Find Them

• Authors: Chris Rockwell, Joseph Tung, Tsung-Yi Lin, Ming-Yu Liu, David F. Fouhey, Chen-Hsuan Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17788
• Pdf URL: https://arxiv.org/pdf/2504.17788
• Copy Paste: [[2504.17788]] Dynamic Camera Poses and Where to Find Them(https://arxiv.org/abs/2504.17788)
• Keywords: generation
• Abstract: Annotating camera poses on dynamic Internet videos at scale is critical for advancing fields like realistic video generation and simulation. However, collecting such a dataset is difficult, as most Internet videos are unsuitable for pose estimation. Furthermore, annotating dynamic Internet videos present significant challenges even for state-of-theart methods. In this paper, we introduce DynPose-100K, a large-scale dataset of dynamic Internet videos annotated with camera poses. Our collection pipeline addresses filtering using a carefully combined set of task-specific and generalist models. For pose estimation, we combine the latest techniques of point tracking, dynamic masking, and structure-from-motion to achieve improvements over the state-of-the-art approaches. Our analysis and experiments demonstrate that DynPose-100K is both large-scale and diverse across several key attributes, opening up avenues for advancements in various downstream applications.
• 摘要：对动态互联网视频的注释摄像头构成对前进的视频生成和仿真等领域至关重要。但是，收集此类数据集很困难，因为大多数互联网视频不适合姿势估算。此外，注释动态的互联网视频即使对于最先进的方法也带来了重大挑战。在本文中，我们介绍了Dynpose-100K，这是一个大规模的数据集，其中包括带有相机姿势的动态互联网视频。我们的收集管道使用一组精心组合的任务特定和通才模型进行过滤。为了进行姿势估计，我们结合了点跟踪，动态掩蔽和结构效率的最新技术，以实现对最新方法的改进。我们的分析和实验表明，Dynpose-100K在几个关键属性中既大规模又多样，这为各种下游应用程序的进步开辟了途径。

Title: Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models

• Authors: Xu Ma, Peize Sun, Haoyu Ma, Hao Tang, Chih-Yao Ma, Jialiang Wang, Kunpeng Li, Xiaoliang Dai, Yujun Shi, Xuan Ju, Yushi Hu, Artsiom Sanakoyeu, Felix Juefei-Xu, Ji Hou, Junjiao Tian, Tao Xu, Tingbo Hou, Yen-Cheng Liu, Zecheng He, Zijian He, Matt Feiszli, Peizhao Zhang, Peter Vajda, Sam Tsai, Yun Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17789
• Pdf URL: https://arxiv.org/pdf/2504.17789
• Copy Paste: [[2504.17789]] Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models(https://arxiv.org/abs/2504.17789)
• Keywords: generation
• Abstract: Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models. A primary limitation is the substantial number of image tokens required for AR models, which constrains both training and inference efficiency, as well as image resolution. To address this, we present Token-Shuffle, a novel yet simple method that reduces the number of image tokens in Transformer. Our key insight is the dimensional redundancy of visual vocabularies in Multimodal Large Language Models (MLLMs), where low-dimensional visual codes from visual encoder are directly mapped to high-dimensional language vocabularies. Leveraging this, we consider two key operations: token-shuffle, which merges spatially local tokens along channel dimension to decrease the input token number, and token-unshuffle, which untangles the inferred tokens after Transformer blocks to restore the spatial arrangement for output. Jointly training with textual prompts, our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis in a unified next-token prediction way while maintaining efficient training and inference. For the first time, we push the boundary of AR text-to-image generation to a resolution of 2048x2048 with gratifying generation performance. In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15. Exhaustive large-scale human evaluations also demonstrate our prominent image generation ability in terms of text-alignment, visual flaw, and visual appearance. We hope that Token-Shuffle can serve as a foundational design for efficient high-resolution image generation within MLLMs.
• 摘要：在语言产生中长期占主导地位的自回旋（AR）模型越来越多地应用于图像合成，但通常被认为比基于扩散的模型不那么竞争。主要限制是AR模型所需的大量图像令牌，这既约束训练和推理效率，又要限制图像分辨率。为了解决这个问题，我们提出了令牌Shuffle，这是一种新颖而简单的方法，可减少变压器中图像令牌的数量。我们的主要见解是多模式大型语言模型（MLLMS）中视觉词汇的尺寸冗余，其中视觉编码器的低维视觉代码直接映射到高维语言词汇表中。利用这一点，我们考虑了两个关键的操作：令牌shuffle，它沿通道维度合并了空间局部令牌，以减少输入令牌数字和令牌 - 毫不掩饰，而令牌则可以在变压器块后解开推断的令牌，以恢复输出空间排列。通过文本提示共同培训，我们的策略不需要额外的预预读文本编码器，并且使MLLM能够以统一的下一步预测方式支持极高的高分辨率图像合成，同时保持有效的培训和推理。我们第一次将AR文本对象生成的边界推向了2048x2048的分辨率，具有令人满意的生成性能。在Genai基准测试中，我们的2.7b模型在硬提示上达到0.77总分，表现优于AR模型的Lamagen 0.18，而扩散模型LDM则达到0.15。详尽的大规模人体评估还证明了我们在文本平衡，视觉缺陷和视觉外观方面的突出形象产生能力。我们希望令牌转移可以作为MLLM中有效高分辨率图像生成的基础设计。