2025-04-09

Title: Level Generation with Constrained Expressive Range

• Authors: Mahsa Bazzaz, Seth Cooper
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05334
• Pdf URL: https://arxiv.org/pdf/2504.05334
• Copy Paste: [[2504.05334]] Level Generation with Constrained Expressive Range(https://arxiv.org/abs/2504.05334)
• Keywords: generation, generative
• Abstract: Expressive range analysis is a visualization-based technique used to evaluate the performance of generative models, particularly in game level generation. It typically employs two quantifiable metrics to position generated artifacts on a 2D plot, offering insight into how content is distributed within a defined metric space. In this work, we use the expressive range of a generator as the conceptual space of possible creations. Inspired by the quality diversity paradigm, we explore this space to generate levels. To do so, we use a constraint-based generator that systematically traverses and generates levels in this space. To train the constraint-based generator we use different tile patterns to learn from the initial example levels. We analyze how different patterns influence the exploration of the expressive range. Specifically, we compare the exploration process based on time, the number of successful and failed sample generations, and the overall interestingness of the generated levels. Unlike typical quality diversity approaches that rely on random generation and hope to get good coverage of the expressive range, this approach systematically traverses the grid ensuring more coverage. This helps create unique and interesting game levels while also improving our understanding of the generator's strengths and limitations.
• 摘要：表达范围分析是一种基于可视化的技术，用于评估生成模型的性能，尤其是在游戏水平生成中。它通常使用两个可量化的指标来定位在2D图上生成的工件，从而深入了解内容如何在定义的度量空间内分布。在这项工作中，我们将发电机的表达范围用作可能创造的概念空间。受质量多样性范式的启发，我们探索了这个空间以产生水平。为此，我们使用基于约束的生成器，该发电机系统地遍历和生成该空间中的水平。为了训练基于约束的发电机，我们使用不同的图块模式从初始示例级别中学习。我们分析了不同的模式如何影响表达范围的探索。具体而言，我们根据时间，成功和失败的样本世代的数量以及生成水平的整体兴趣比较探索过程。与依靠随机发电并希望获得良好覆盖表达范围的典型质量多样性方法不同，这种方法可以系统地穿越网格，从而确保更多的覆盖范围。这有助于创建独特而有趣的游戏水平，同时也提高了我们对发电机优势和局限性的理解。

Title: Time-adaptive Video Frame Interpolation based on Residual Diffusion

• Authors: Victor Fonte Chavez, Claudia Esteves, Jean-Bernard Hayet
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05402
• Pdf URL: https://arxiv.org/pdf/2504.05402
• Copy Paste: [[2504.05402]] Time-adaptive Video Frame Interpolation based on Residual Diffusion(https://arxiv.org/abs/2504.05402)
• Keywords: super-resolution
• Abstract: In this work, we propose a new diffusion-based method for video frame interpolation (VFI), in the context of traditional hand-made animation. We introduce three main contributions: The first is that we explicitly handle the interpolation time in our model, which we also re-estimate during the training process, to cope with the particularly large variations observed in the animation domain, compared to natural videos; The second is that we adapt and generalize a diffusion scheme called ResShift recently proposed in the super-resolution community to VFI, which allows us to perform a very low number of diffusion steps (in the order of 10) to produce our estimates; The third is that we leverage the stochastic nature of the diffusion process to provide a pixel-wise estimate of the uncertainty on the interpolated frame, which could be useful to anticipate where the model may be wrong. We provide extensive comparisons with respect to state-of-the-art models and show that our model outperforms these models on animation videos.
• 摘要：在这项工作中，我们在传统的手工动画的背景下，提出了一种基于视频框架插值（VFI）的新的基于扩散的方法。我们介绍了三个主要贡献：第一个是我们在模型中明确处理插值时间，在训练过程中我们也重新估算了与自然视频相比，在动画域中观察到的特别差异；第二是，我们适应并概括了最近在超分辨率社区中提出的称为Resshift的扩散方案，该方案使我们能够执行非常少数的扩散步骤（以10顺序）来产生我们的估计值；第三个是我们利用扩散过程的随机性质来提供插值框架上不确定性的像素估计，这对于预测模型可能是错误的位置可能很有用。我们提供了与最先进模型的广泛比较，并表明我们的模型在动画视频上的表现优于这些模型。

Title: EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representations

• Authors: Yue Yao, Mohamed-Khalil Bouzidi, Daniel Goehring, Joerg Reichardt
• Subjects: cs.CV, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.05422
• Pdf URL: https://arxiv.org/pdf/2504.05422
• Copy Paste: [[2504.05422]] EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representations(https://arxiv.org/abs/2504.05422)
• Keywords: generation, generative
• Abstract: As the prediction horizon increases, predicting the future evolution of traffic scenes becomes increasingly difficult due to the multi-modal nature of agent motion. Most state-of-the-art (SotA) prediction models primarily focus on forecasting the most likely future. However, for the safe operation of autonomous vehicles, it is equally important to cover the distribution for plausible motion alternatives. To address this, we introduce EP-Diffuser, a novel parameter-efficient diffusion-based generative model designed to capture the distribution of possible traffic scene evolutions. Conditioned on road layout and agent history, our model acts as a predictor and generates diverse, plausible scene continuations. We benchmark EP-Diffuser against two SotA models in terms of accuracy and plausibility of predictions on the Argoverse 2 dataset. Despite its significantly smaller model size, our approach achieves both highly accurate and plausible traffic scene predictions. We further evaluate model generalization ability in an out-of-distribution (OoD) test setting using Waymo Open dataset and show superior robustness of our approach. The code and model checkpoints can be found here: this https URL.
• 摘要：随着预测范围的增加，由于代理运动的多模式性质，预测交通场景的未来演变变得越来越困难。大多数最先进的（SOTA）预测模型主要集中于预测最可能的未来。但是，对于自动驾驶汽车的安全运行，涵盖合理运动替代方案的分布同样重要。为了解决这个问题，我们介绍了EP-Diffuser，这是一种新型的基于参数效率扩散的生成模型，旨在捕获可能的流量场景演变的分布。以道路布局和代理历史记录为条件，我们的模型充当预测指标，并产生了多样的，合理的场景连续性。我们根据Argoverse 2数据集对预测的准确性和合理性，对两个SOTA模型进行基准测试。尽管模型大小明显较小，但我们的方法既可以实现高度准确又合理的交通现场预测。我们进一步评估了使用Waymo打开数据集的分布（OOD）测试设置中的模型泛化能力，并显示出我们方法的较高鲁棒性。代码和模型检查点可以在此处找到：此HTTPS URL。

Title: Generative Adversarial Networks with Limited Data: A Survey and Benchmarking

• Authors: Omar De Mitri, Ruyu Wang, Marco F. Huber
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05456
• Pdf URL: https://arxiv.org/pdf/2504.05456
• Copy Paste: [[2504.05456]] Generative Adversarial Networks with Limited Data: A Survey and Benchmarking(https://arxiv.org/abs/2504.05456)
• Keywords: generative
• Abstract: Generative Adversarial Networks (GANs) have shown impressive results in various image synthesis tasks. Vast studies have demonstrated that GANs are more powerful in feature and expression learning compared to other generative models and their latent space encodes rich semantic information. However, the tremendous performance of GANs heavily relies on the access to large-scale training data and deteriorates rapidly when the amount of data is limited. This paper aims to provide an overview of GANs, its variants and applications in various vision tasks, focusing on addressing the limited data issue. We analyze state-of-the-art GANs in limited data regime with designed experiments, along with presenting various methods attempt to tackle this problem from different perspectives. Finally, we further elaborate on remaining challenges and trends for future research.
• 摘要：生成对抗网络（GAN）在各种图像综合任务中显示出令人印象深刻的结果。大量的研究表明，与其他生成模型及其潜在空间相比，GAN在功能和表达学习方面更强大，可以编码丰富的语义信息。但是，甘恩斯的巨大性能在很大程度上依赖于获得大规模训练数据的访问，并且在数据量受到限制时会迅速恶化。本文旨在概述gan，其在各种视觉任务中的变体和应用程序，重点是解决有限的数据问题。我们通过设计的实验在有限的数据制度中分析了最先进的gan，并提出各种方法试图从不同的角度解决此问题。最后，我们进一步阐述了未来研究的剩余挑战和趋势。

Title: GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases

• Authors: Alfred Clemedtson, Borun Shi
• Subjects: cs.LG, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.05478
• Pdf URL: https://arxiv.org/pdf/2504.05478
• Copy Paste: [[2504.05478]] GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases(https://arxiv.org/abs/2504.05478)
• Keywords: generation
• Abstract: Large language models have shown remarkable language processing and reasoning ability but are prone to hallucinate when asked about private data. Retrieval-augmented generation (RAG) retrieves relevant data that fit into an LLM's context window and prompts the LLM for an answer. GraphRAG extends this approach to structured Knowledge Graphs (KGs) and questions regarding entities multiple hops away. The majority of recent GraphRAG methods either overlook the retrieval step or have ad hoc retrieval processes that are abstract or inefficient. This prevents them from being adopted when the KGs are stored in graph databases supporting graph query languages. In this work, we present GraphRAFT, a retrieve-and-reason framework that finetunes LLMs to generate provably correct Cypher queries to retrieve high-quality subgraph contexts and produce accurate answers. Our method is the first such solution that can be taken off-the-shelf and used on KGs stored in native graph DBs. Benchmarks suggest that our method is sample-efficient and scales with the availability of training data. Our method achieves significantly better results than all state-of-the-art models across all four standard metrics on two challenging Q\&As on large text-attributed KGs.
• 摘要：大型语言模型已显示出非凡的语言处理和推理能力，但是当被问及私人数据时，很容易幻觉。检索增强的生成（RAG）检索适合LLM上下文窗口的相关数据，并提示LLM以获取答案。 GraphRag将这种方法扩展到结构化知识图（kgs）和有关实体多个啤酒花的问题。最近的大多数GraphRAG方法要么忽略检索步骤，要么具有抽象或效率低下的临时检索过程。当将kg存储在支持图查询语言的图形数据库中时，这可以防止它们被采用。在这项工作中，我们提出了Graphraft，这是一种检索和季节的框架，该框架对LLM进行了验证，以生成可证明的正确的Cypher查询以检索高质量的子图背景并产生准确的答案。我们的方法是第一个可以在本机图DBS中存储的kgs上使用的解决方案。基准表明我们的方法是样本效率，并且具有训练数据的可用性。与在大型文本征收的KG上一样，我们的方法比所有四个标准指标的所有最先进的模型都取得了明显的更好的结果。

Title: SelfMAD: Enhancing Generalization and Robustness in Morphing Attack Detection via Self-Supervised Learning

• Authors: Marija Ivanovska, Leon Todorov, Naser Damer, Deepak Kumar Jain, Peter Peer, Vitomir Štruc
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05504
• Pdf URL: https://arxiv.org/pdf/2504.05504
• Copy Paste: [[2504.05504]] SelfMAD: Enhancing Generalization and Robustness in Morphing Attack Detection via Self-Supervised Learning(https://arxiv.org/abs/2504.05504)
• Keywords: generative
• Abstract: With the continuous advancement of generative models, face morphing attacks have become a significant challenge for existing face verification systems due to their potential use in identity fraud and other malicious activities. Contemporary Morphing Attack Detection (MAD) approaches frequently rely on supervised, discriminative models trained on examples of bona fide and morphed images. These models typically perform well with morphs generated with techniques seen during training, but often lead to sub-optimal performance when subjected to novel unseen morphing techniques. While unsupervised models have been shown to perform better in terms of generalizability, they typically result in higher error rates, as they struggle to effectively capture features of subtle artifacts. To address these shortcomings, we present SelfMAD, a novel self-supervised approach that simulates general morphing attack artifacts, allowing classifiers to learn generic and robust decision boundaries without overfitting to the specific artifacts induced by particular face morphing methods. Through extensive experiments on widely used datasets, we demonstrate that SelfMAD significantly outperforms current state-of-the-art MADs, reducing the detection error by more than 64% in terms of EER when compared to the strongest unsupervised competitor, and by more than 66%, when compared to the best performing discriminative MAD model, tested in cross-morph settings. The source code for SelfMAD is available at this https URL.
• 摘要：随着生成模型的不断发展，由于其潜在使用身份欺诈和其他恶意活动，面部变形攻击已成为现有面部验证系统的重大挑战。当代的变形攻击检测（MAD）方法经常取决于受到监督的，歧视性的模型，该模型接受了真正的图像和变形图像的例子。这些模型通常会通过在训练过程中看到的技术产生的形态表现良好，但经过新颖的不见变形技术，通常会导致次优性能。尽管已证明无监督的模型在概括性方面的性能更好，但它们通常会导致较高的错误率，因为它们难以有效捕获微妙的人工制品的特征。为了解决这些缺点，我们提出了一种新颖的自我监督方法，可以模拟一般的变形攻击伪像，使分类器能够学习通用和健壮的决策界限，而不会过分适合特定面部变形方法引起的特定伪像。通过对广泛使用的数据集进行的广泛实验，我们证明，与最强大的无人看待的竞争对手相比，SelfMAD的表现显着胜过当前的最新疯狂，而与最强大的无人监督的竞争对手相比，EER的检测错误将超过64％，并且与最佳性能歧视性疯狂模型相比，在交叉界面上进行了鉴别。 SelfMad的源代码可在此HTTPS URL上获得。

Title: PartStickers: Generating Parts of Objects for Rapid Prototyping

• Authors: Mo Zhou, Josh Myers-Dean, Danna Gurari
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05508
• Pdf URL: https://arxiv.org/pdf/2504.05508
• Copy Paste: [[2504.05508]] PartStickers: Generating Parts of Objects for Rapid Prototyping(https://arxiv.org/abs/2504.05508)
• Keywords: generation
• Abstract: Design prototyping involves creating mockups of products or concepts to gather feedback and iterate on ideas. While prototyping often requires specific parts of objects, such as when constructing a novel creature for a video game, existing text-to-image methods tend to only generate entire objects. To address this, we propose a novel task and method of ``part sticker generation", which entails generating an isolated part of an object on a neutral background. Experiments demonstrate our method outperforms state-of-the-art baselines with respect to realism and text alignment, while preserving object-level generation capabilities. We publicly share our code and models to encourage community-wide progress on this new task: this https URL.
• 摘要：设计原型制作涉及创建产品或概念的模型，以收集反馈并迭代思想。虽然原型制作通常需要对象的特定部分，例如为视频游戏构造新颖的生物时，现有的文本对图像方法往往只生成整个对象。为了解决这个问题，我们提出了一种新颖的任务和方法``部分贴纸生成''，它需要在中性背景下生成一个孤立的对象的一部分。实验证明了我们的方法优于现实主义和文本一致性的最新基准，同时为对象级别的生成能力提供了共享的代码和模型。

Title: Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling

• Authors: Tasmiah Haque, Md. Asif Bin Syed, Byungheon Jeong, Xue Bai, Sumit Mohan, Somdyuti Paul, Imtiaz Ahmed, Srinjoy Das
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05537
• Pdf URL: https://arxiv.org/pdf/2504.05537
• Copy Paste: [[2504.05537]] Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling(https://arxiv.org/abs/2504.05537)
• Keywords: generative
• Abstract: We propose a deep learning framework designed to significantly optimize bandwidth for motion-transfer-enabled video applications, including video conferencing, virtual reality interactions, health monitoring systems, and vision-based real-time anomaly detection. To capture complex motion effectively, we utilize the First Order Motion Model (FOMM), which encodes dynamic objects by detecting keypoints and their associated local affine transformations. These keypoints are identified using a self-supervised keypoint detector and arranged into a time series corresponding to the successive frames. Forecasting is performed on these keypoints by integrating two advanced generative time series models into the motion transfer pipeline, namely the Variational Recurrent Neural Network (VRNN) and the Gated Recurrent Unit with Normalizing Flow (GRU-NF). The predicted keypoints are subsequently synthesized into realistic video frames using an optical flow estimator paired with a generator network, thereby facilitating accurate video forecasting and enabling efficient, low-frame-rate video transmission. We validate our results across three datasets for video animation and reconstruction using the following metrics: Mean Absolute Error, Joint Embedding Predictive Architecture Embedding Distance, Structural Similarity Index, and Average Pair-wise Displacement. Our results confirm that by utilizing the superior reconstruction property of the Variational Autoencoder, the VRNN integrated FOMM excels in applications involving multi-step ahead forecasts such as video conferencing. On the other hand, by leveraging the Normalizing Flow architecture for exact likelihood estimation, and enabling efficient latent space sampling, the GRU-NF based FOMM exhibits superior capabilities for producing diverse future samples while maintaining high visual quality for tasks like real-time video-based anomaly detection.
• 摘要：我们提出了一个深度学习框架，旨在显着优化支持运动传输的视频应用程序的带宽，包括视频会议，虚拟现实互动，健康监测系统和基于视觉的实时异常检测。为了有效地捕获复杂的运动，我们利用了一阶运动模型（FOMM），该模型（FOMM）通过检测关键点及其相关的局部仿射变换来编码动态对象。使用自我监督的关键点检测器识别这些关键点，并安排为与连续帧相对应的时间序列。通过将两个先进的生成时间序列模型集成到运动转移管道中，即变异复发神经网络（VRNN）和带有归一化流量（GRU-NF）的门控复发单元（GRU-NF），在这些关键点上进行了预测。随后使用与发电机网络配对的光流估计器将预测的关键点合成为逼真的视频帧，从而促进了准确的视频预测并启用有效的，低框架速率的视频传输。我们使用以下指标验证了三个数据集中的结果，以进行视频动画和重建：平均绝对误差，关节嵌入预测性架构嵌入距离，结构相似性指数和平均成对位移。我们的结果证实，通过利用变异自动编码器的卓越重建属性，VRNN集成的Fomm在涉及多步骤的预测（例如视频会议）的应用中出色。另一方面，通过利用标准化流程结构进行精确的可能性估计，并实现有效的潜在空间采样，基于GRU-NF的FOMM具有较高的功能，可以产生不同的未来样品，同时维持高视觉质量，例如实时视频异常检测。

Title: Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

• Authors: Yi Peng, Chris, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu, Yahui Zhou
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05599
• Pdf URL: https://arxiv.org/pdf/2504.05599
• Copy Paste: [[2504.05599]] Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought(https://arxiv.org/abs/2504.05599)
• Keywords: generation
• Abstract: We introduce Skywork R1V, a multimodal reasoning model extending the an R1-series Large language models (LLM) to visual modalities via an efficient multimodal transfer method. Leveraging a lightweight visual projector, Skywork R1V facilitates seamless multimodal adaptation without necessitating retraining of either the foundational language model or the vision encoder. To strengthen visual-text alignment, we propose a hybrid optimization strategy that combines Iterative Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO), significantly enhancing cross-modal integration efficiency. Additionally, we introduce an adaptive-length Chain-of-Thought distillation approach for reasoning data generation. This approach dynamically optimizes reasoning chain lengths, thereby enhancing inference efficiency and preventing excessive reasoning overthinking. Empirical evaluations demonstrate that Skywork R1V, with only 38B parameters, delivers competitive performance, achieving a score of 69.0 on the MMMU benchmark and 67.5 on MathVista. Meanwhile, it maintains robust textual reasoning performance, evidenced by impressive scores of 72.0 on AIME and 94.0 on MATH500. The Skywork R1V model weights have been publicly released to promote openness and reproducibility.
• 摘要：我们介绍了Skywork R1V，这是一种多模式推理模型，通过有效的多模式传输方法将R1系列大型语言模型（LLM）扩展到视觉方式。 Skywork R1V利用轻巧的视觉投影仪，促进了无缝的多模式适应，而无需重新训练基础语言模型或视觉编码器。为了加强视觉文本对齐，我们提出了一种混合优化策略，将迭代监督的微调（SFT）与小组相对策略优化（GRPO）相结合，从而显着提高了交叉模式的整合效率。此外，我们引入了一种自适应长度链条蒸馏方法，用于推理数据的生成。这种方法动态优化了推理链长度，从而提高了推理效率并防止过多的推理过度思考。经验评估表明，Skywork R1V只有38B参数，提供了竞争性能，在MMMU基准测试中获得了69.0的得分，而在Mathvista上的得分为67.5。同时，它保持了强大的文本推理性能，这在AIME上的72.0分数和Math500上的94.0得分证明了。 Skywork R1V模型权重已公开发布，以促进开放性和可重复性。

Title: Falcon: Fractional Alternating Cut with Overcoming Minima in Unsupervised Segmentation

• Authors: Xiao Zhang, Xiangyu Han, Xiwen Lai, Yao Sun, Pei Zhang, Konrad Kording
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05613
• Pdf URL: https://arxiv.org/pdf/2504.05613
• Copy Paste: [[2504.05613]] Falcon: Fractional Alternating Cut with Overcoming Minima in Unsupervised Segmentation(https://arxiv.org/abs/2504.05613)
• Keywords: generation
• Abstract: Today's unsupervised image segmentation algorithms often segment suboptimally. Modern graph-cut based approaches rely on high-dimensional attention maps from Transformer-based foundation models, typically employing a relaxed Normalized Cut solved recursively via the Fiedler vector (the eigenvector of the second smallest eigenvalue). Consequently, they still lag behind supervised methods in both mask generation speed and segmentation accuracy. We present a regularized fractional alternating cut (Falcon), an optimization-based K-way Normalized Cut without relying on recursive eigenvector computations, achieving substantially improved speed and accuracy. Falcon operates in two stages: (1) a fast K-way Normalized Cut solved by extending into a fractional quadratic transformation, with an alternating iterative procedure and regularization to avoid local minima; and (2) refinement of the resulting masks using complementary low-level information, producing high-quality pixel-level segmentations. Experiments show that Falcon not only surpasses existing state-of-the-art methods by an average of 2.5% across six widely recognized benchmarks (reaching up to 4.3\% improvement on Cityscapes), but also reduces runtime by around 30% compared to prior graph-based approaches. These findings demonstrate that the semantic information within foundation-model attention can be effectively harnessed by a highly parallelizable graph cut framework. Consequently, Falcon can narrow the gap between unsupervised and supervised segmentation, enhancing scalability in real-world applications and paving the way for dense prediction-based vision pre-training in various downstream tasks. The code is released in this https URL.
• 摘要：当今的无监督图像分割算法通常次优。现代基于图形的方法依赖于基于变压器的基础模型的高维注意图，通常采用轻松的归一化切割方法通过Fiedler Vector（第二小最小值的特征向量）递归地解决。因此，它们仍然落后于蒙版生成速度和分割精度的监督方法。我们提出了一个正则分数交替切割（FALCON），这是一种基于优化的K道归一化切割，而无需依赖递归特征向量计算，从而实现了实质上提高的速度和准确性。 Falcon分为两个阶段运行：（1）通过延伸到分数二次变换来解决的快速K道归一化切割，并采用交替的迭代程序和正则化，以避免局部最小值； （2）使用互补的低级信息对所得面具进行改进，从而产生高质量的像素级分段。实验表明，猎鹰不仅超过现有的最新方法，在六个众所周知的基准中平均超过2.5％（在城市景观方面提高了4.3％\％），而且与先前的基于图的方法相比，运行时也将运行时降低了约30％。这些发现表明，可以通过高度可行的图形切割框架有效地利用基础模型注意力中的语义信息。因此，猎鹰可以缩小无监督和监督分割之间的差距，增强现实世界应用中的可扩展性，并为各种下游任务中的基于密集的预测视力预训练铺平道路。该代码在此HTTPS URL中发布。

Title: Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation

• Authors: Tianshui Chen, Jianman Lin, Zhijing Yang, Chumei Qing, Yukai Shi, Liang Lin
• Subjects: cs.CV, cs.SD
• Abstract URL: https://arxiv.org/abs/2504.05672
• Pdf URL: https://arxiv.org/pdf/2504.05672
• Copy Paste: [[2504.05672]] Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation(https://arxiv.org/abs/2504.05672)
• Keywords: generation
• Abstract: Speech-preserving facial expression manipulation (SPFEM) aims to modify a talking head to display a specific reference emotion while preserving the mouth animation of source spoken contents. Thus, emotion and content information existing in reference and source inputs can provide direct and accurate supervision signals for SPFEM models. However, the intrinsic intertwining of these elements during the talking process poses challenges to their effectiveness as supervisory signals. In this work, we propose to learn content and emotion priors as guidance augmented with contrastive learning to learn decoupled content and emotion representation via an innovative Contrastive Decoupled Representation Learning (CDRL) algorithm. Specifically, a Contrastive Content Representation Learning (CCRL) module is designed to learn audio feature, which primarily contains content information, as content priors to guide learning content representation from the source input. Meanwhile, a Contrastive Emotion Representation Learning (CERL) module is proposed to make use of a pre-trained visual-language model to learn emotion prior, which is then used to guide learning emotion representation from the reference input. We further introduce emotion-aware and emotion-augmented contrastive learning to train CCRL and CERL modules, respectively, ensuring learning emotion-independent content representation and content-independent emotion representation. During SPFEM model training, the decoupled content and emotion representations are used to supervise the generation process, ensuring more accurate emotion manipulation together with audio-lip synchronization. Extensive experiments and evaluations on various benchmarks show the effectiveness of the proposed algorithm.
• 摘要：言语保护的面部表情操纵（SPFEM）旨在修改会说话的头，以显示特定的参考情绪，同时保留源说话内容的口腔动画。因此，参考和源输入中存在的情绪和内容信息可以为SPFEM模型提供直接，准确的监督信号。但是，在谈话过程中，这些要素的内在交织构成了其作为监督信号的有效性的挑战。在这项工作中，我们建议学习内容和情感先验，随着对比度学习的增强指导，通过创新的对比度解耦表示学习（CDRL）算法学习解耦内容和情感表示。具体来说，对比内容表示学习（CCRL）模块旨在学习音频功能，该模块主要包含内容信息，作为内容先验，可从源输入指导学习内容表示。同时，提出了一种对比情绪表示学习（CERL）模块，以利用预先训练的视觉语言模型来学习情感，然后将其用于指导参考输入中的学习情感表征。我们进一步介绍了情感感知和情感上的对比学习，分别训练CCRL和CERL模块，以确保学习与情感无关的内容表示和与内容无关的情感表示。在SPFEM模型培训期间，使用脱钩的内容和情感表征用于监督生成过程，从而确保更准确的情绪操纵以及音频唇同步。对各种基准测试的广泛实验和评估表明了所提出的算法的有效性。

Title: Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment

• Authors: Gen Li, Li Chen, Cheng Tang, Valdemar Švábenský, Daisuke Deguchi, Takayoshi Yamashita, Atsushi Shimada
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2504.05716
• Pdf URL: https://arxiv.org/pdf/2504.05716
• Copy Paste: [[2504.05716]] Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment(https://arxiv.org/abs/2504.05716)
• Keywords: generative
• Abstract: We explore the use of Large Language Models (LLMs) for automated assessment of open-text student reflections and prediction of academic performance. Traditional methods for evaluating reflections are time-consuming and may not scale effectively in educational settings. In this work, we employ LLMs to transform student reflections into quantitative scores using two assessment strategies (single-agent and multi-agent) and two prompting techniques (zero-shot and few-shot). Our experiments, conducted on a dataset of 5,278 reflections from 377 students over three academic terms, demonstrate that the single-agent with few-shot strategy achieves the highest match rate with human evaluations. Furthermore, models utilizing LLM-assessed reflection scores outperform baselines in both at-risk student identification and grade prediction tasks. These findings suggest that LLMs can effectively automate reflection assessment, reduce educators' workload, and enable timely support for students who may need additional assistance. Our work emphasizes the potential of integrating advanced generative AI technologies into educational practices to enhance student engagement and academic success.
• 摘要：我们探讨了大型语言模型（LLM）的使用，以自动评估开放文本的学生思考和对学业表现的预测。评估反思的传统方法是耗时的，在教育环境中可能无法有效地扩展。在这项工作中，我们采用LLM使用两种评估策略（单一代理和多代理）和两种提示技术（零射击和少数）来将学生的思考转化为定量得分。我们的实验在三个学术术语中的377名学生的5,278个反射的数据集上进行，这表明，具有很少射击策略的单位代理可以通过人类评估达到最高的匹配率。此外，利用LLM评估的反射分数在高危学生识别和等级预测任务中的模型都超过了基线。这些发现表明，LLM可以有效地自动化反思评估，减少教育者的工作量，并为可能需要额外帮助的学生提供及时的支持。我们的工作强调了将先进的生成AI技术纳入教育实践的潜力，以增强学生的参与和学业上的成功。

Title: QEMesh: Employing A Quadric Error Metrics-Based Representation for Mesh Generation

• Authors: Jiaqi Li, Ruowei Wang, Yu Liu, Qijun Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05720
• Pdf URL: https://arxiv.org/pdf/2504.05720
• Copy Paste: [[2504.05720]] QEMesh: Employing A Quadric Error Metrics-Based Representation for Mesh Generation(https://arxiv.org/abs/2504.05720)
• Keywords: generation, generative
• Abstract: Mesh generation plays a crucial role in 3D content creation, as mesh is widely used in various industrial applications. Recent works have achieved impressive results but still face several issues, such as unrealistic patterns or pits on surfaces, thin parts missing, and incomplete structures. Most of these problems stem from the choice of shape representation or the capabilities of the generative network. To alleviate these, we extend PoNQ, a Quadric Error Metrics (QEM)-based representation, and propose a novel model, QEMesh, for high-quality mesh generation. PoNQ divides the shape surface into tiny patches, each represented by a point with its normal and QEM matrix, which preserves fine local geometry information. In our QEMesh, we regard these elements as generable parameters and design a unique latent diffusion model containing a novel multi-decoder VAE for PoNQ parameters generation. Given the latent code generated by the diffusion model, three parameter decoders produce several PoNQ parameters within each voxel cell, and an occupancy decoder predicts which voxel cells containing parameters to form the final shape. Extensive evaluations demonstrate that our method generates results with watertight surfaces and is comparable to state-of-the-art methods in several main metrics.
• 摘要：网格生成在3D含量创建中起着至关重要的作用，因为网格广泛用于各种工业应用中。最近的作品取得了令人印象深刻的结果，但仍然面临着几个问题，例如表面上的不切实际模式或凹坑，缺少薄的零件和不完整的结构。这些问题中的大多数源于形状表示的选择或生成网络的功能。为了减轻这些方法，我们扩展了PONQ，即基于二次误差指标（QEM）的表示，并提出了一种新型模型Qemesh，以生成高质量的网格。 PONQ将形状表面划分为微小的斑块，每个斑块都由其正常和QEM矩阵的点表示，该点保留了良好的局部几何信息。在我们的Qemesh中，我们将这些元素视为可生成参数，并设计了一个独特的潜扩散模型，该模型包含用于生成PONQ参数的新型多码数vae。给定扩散模型生成的潜在代码，三个参数解码器在每个体素电池内产生多个PONQ参数，并且占用解码器预测哪些含有参数的素单元形成最终形状。广泛的评估表明，我们的方法在水密表面产生结果，并且与几种主要指标的最新方法相媲美。

Title: DDT: Decoupled Diffusion Transformer

• Authors: Shuai Wang, Zhi Tian, Weilin Huang, Limin Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05741
• Pdf URL: https://arxiv.org/pdf/2504.05741
• Copy Paste: [[2504.05741]] DDT: Decoupled Diffusion Transformer(https://arxiv.org/abs/2504.05741)
• Keywords: generation
• Abstract: Diffusion transformers have demonstrated remarkable generation quality, albeit requiring longer training iterations and numerous inference steps. In each denoising step, diffusion transformers encode the noisy inputs to extract the lower-frequency semantic component and then decode the higher frequency with identical modules. This scheme creates an inherent optimization dilemma: encoding low-frequency semantics necessitates reducing high-frequency components, creating tension between semantic encoding and high-frequency decoding. To resolve this challenge, we propose a new \textbf{\color{ddt}D}ecoupled \textbf{\color{ddt}D}iffusion \textbf{\color{ddt}T}ransformer~(\textbf{\color{ddt}DDT}), with a decoupled design of a dedicated condition encoder for semantic extraction alongside a specialized velocity decoder. Our experiments reveal that a more substantial encoder yields performance improvements as model size increases. For ImageNet $256\times256$, Our DDT-XL/2 achieves a new state-of-the-art performance of {1.31 FID}~(nearly $4\times$ faster training convergence compared to previous diffusion transformers). For ImageNet $512\times512$, Our DDT-XL/2 achieves a new state-of-the-art FID of 1.28. Additionally, as a beneficial by-product, our decoupled architecture enhances inference speed by enabling the sharing self-condition between adjacent denoising steps. To minimize performance degradation, we propose a novel statistical dynamic programming approach to identify optimal sharing strategies.
• 摘要：扩散变压器表现出了出色的产生质量，尽管需要更长的训练迭代和许多推理步骤。在每个降解步骤中，扩散变压器编码嘈杂的输入以提取低频语义分量，然后用相同的模块解码较高的频率。该方案产生了固有的优化难题：编码低频语义需要减少高频组件，从而在语义编码和高频解码之间产生张力。要解决这个挑战，我们提出了一个新的\ textbf {\ color {ddt} d} ecoupled \ textbf {\ color {ddt} d} iffusion \ textbf {\ textbf {\ color {ddt} t} t} t} t} t} ransformer〜（专用条件编码器，用于语义提取，并与专门的速度解码器一起进行。我们的实验表明，随着模型大小的增加，更实质性的编码器会产生性能的改善。对于ImagEnet $ 256 \ times256 $，我们的DDT-XL/2实现了{1.31 FID}〜的新最先进的性能（与以前的扩散变形金刚相比，近4美元\ $ 4 \ times $更快的训练收敛）。对于Imagenet $ 512 \ times512 $，我们的DDT-XL/2实现了1.28的新最新FID。此外，作为一种有益的副产品，我们的解耦结构通过实现相邻的DeNoising步骤之间的共享自我条件来增强推理速度。为了最大程度地减少性能降解，我们提出了一种新型的统计动态编程方法，以识别最佳共享策略。

Title: Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation

• Authors: Zhihua Xu, Tianshui Chen, Zhijing Yang, Siyuan Peng, Keze Wang, Liang Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05746
• Pdf URL: https://arxiv.org/pdf/2504.05746
• Copy Paste: [[2504.05746]] Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation(https://arxiv.org/abs/2504.05746)
• Keywords: generation
• Abstract: The paramount challenge in audio-driven One-shot Talking Head Animation (ADOS-THA) lies in capturing subtle imperceptible changes between adjacent video frames. Inherently, the temporal relationship of adjacent audio clips is highly correlated with that of the corresponding adjacent video frames, offering supplementary information that can be pivotal for guiding and supervising talking head animations. In this work, we propose to learn audio-visual correlations and integrate the correlations to help enhance feature representation and regularize final generation by a novel Temporal Audio-Visual Correlation Embedding (TAVCE) framework. Specifically, it first learns an audio-visual temporal correlation metric, ensuring the temporal audio relationships of adjacent clips are aligned with the temporal visual relationships of corresponding adjacent video frames. Since the temporal audio relationship contains aligned information about the visual frame, we first integrate it to guide learning more representative features via a simple yet effective channel attention mechanism. During training, we also use the alignment correlations as an additional objective to supervise generating visual frames. We conduct extensive experiments on several publicly available benchmarks (i.e., HDTF, LRW, VoxCeleb1, and VoxCeleb2) to demonstrate its superiority over existing leading algorithms.
• 摘要：音频驱动的一声谈话头动画（ADOS-THA）中的最重要挑战在于捕捉相邻视频帧之间微妙的不可察觉的变化。固有地，相邻音频剪辑的时间关系与相应的相邻视频帧高度相关，提供了补充信息，这对于指导和监督说话的头部动画可能是关键的。在这项工作中，我们建议学习视听相关性并整合相关性，以帮助增强特征表示并通过一种新型的时间视听相关性嵌入（TAVCE）框架正规化最终代理。具体而言，它首先学习了视听时间相关度量，以确保相邻剪辑的时间音频关系与相应相邻视频帧的时间视觉关系对齐。由于时间音频关系包含有关视觉框架的一致信息，因此我们首先将其集成以指导通过简单而有效的频道注意机制学习更多的代表性特征。在培训期间，我们还使用对齐相关性作为监督产生视觉帧的附加目标。我们对几个公开可用的基准（即HDTF，LRW，Voxceleb1和Voxceleb2）进行了广泛的实验，以证明其优于现有领先算法的优势。

Title: MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

• Authors: Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05782
• Pdf URL: https://arxiv.org/pdf/2504.05782
• Copy Paste: [[2504.05782]] MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models(https://arxiv.org/abs/2504.05782)
• Keywords: generation
• Abstract: Multimodal reasoning, which integrates language and visual cues into problem solving and decision making, is a fundamental aspect of human intelligence and a crucial step toward artificial general intelligence. However, the evaluation of multimodal reasoning capabilities in Multimodal Large Language Models (MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained by limited data size, narrow domain coverage, and unstructured knowledge distribution. To close these gaps, we introduce MDK12-Bench, a multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via real-world K-12 examinations. Spanning six disciplines (math, physics, chemistry, biology, geography, and information science), our benchmark comprises 140K reasoning instances across diverse difficulty levels from primary school to 12th grade. It features 6,827 instance-level knowledge point annotations based on a well-organized knowledge structure, detailed answer explanations, difficulty labels and cross-year partitions, providing a robust platform for comprehensive evaluation. Additionally, we present a novel dynamic evaluation framework to mitigate data contamination issues by bootstrapping question forms, question types, and image styles during evaluation. Extensive experiment on MDK12-Bench reveals the significant limitation of current MLLMs in multimodal reasoning. The findings on our benchmark provide insights into the development of the next-generation models. Our data and codes are available at this https URL.
• 摘要：多模式推理将语言和视觉线索整合到解决问题和决策中，是人类智力的基本方面，也是迈向人工通用智能的关键步骤。但是，多模式大语言模型（MLLM）中多模式推理能力的评估仍然不足。大多数现有的推理基准受到有限的数据大小，狭窄的域覆盖范围和非结构化知识分布的限制。为了缩小这些差距，我们介绍了MDK12基础，这是一个多学科基准测试，可通过现实世界中的K-12检查评估MLLM的推理能力。我们的基准涵盖了六个学科（数学，物理，化学，生物学，地理学和信息科学），包括从小学到12年级的各种难度水平的140K推理实例。它具有基于组织良好的知识结构，详细的答案说明，难度标签和跨年分区的6,827个实例级知识点注释，为全面评估提供了强大的平台。此外，我们提出了一个新颖的动态评估框架，以通过自举问题，问题类型和图像样式来减轻数据污染问题。对MDK12基板的广泛实验揭示了当前MLLM在多模式推理中的显着局限性。我们的基准测试结果提供了有关下一代模型发展的见解。我们的数据和代码可在此HTTPS URL上找到。

Title: Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling

• Authors: Jaskirat Singh, Junshen Kevin Chen, Jonas Kohler, Michael Cohen
• Subjects: cs.CV, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.05800
• Pdf URL: https://arxiv.org/pdf/2504.05800
• Copy Paste: [[2504.05800]] Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling(https://arxiv.org/abs/2504.05800)
• Keywords: generation
• Abstract: Training-free consistent text-to-image generation depicting the same subjects across different images is a topic of widespread recent interest. Existing works in this direction predominantly rely on cross-frame self-attention; which improves subject-consistency by allowing tokens in each frame to pay attention to tokens in other frames during self-attention computation. While useful for single subjects, we find that it struggles when scaling to multiple characters. In this work, we first analyze the reason for these limitations. Our exploration reveals that the primary-issue stems from self-attention-leakage, which is exacerbated when trying to ensure consistency across multiple-characters. This happens when tokens from one subject pay attention to other characters, causing them to appear like each other (e.g., a dog appearing like a duck). Motivated by these findings, we propose StoryBooth: a training-free approach for improving multi-character consistency. In particular, we first leverage multi-modal chain-of-thought reasoning and region-based generation to apriori localize the different subjects across the desired story outputs. The final outputs are then generated using a modified diffusion model which consists of two novel layers: 1) a bounded cross-frame self-attention layer for reducing inter-character attention leakage, and 2) token-merging layer for improving consistency of fine-grain subject details. Through both qualitative and quantitative results we find that the proposed approach surpasses prior state-of-the-art, exhibiting improved consistency across both multiple-characters and fine-grain subject details.
• 摘要：无培训一致的文本对图像生成描绘了不同图像的相同主题是广泛关注的话题。朝这个方向的现有作品主要依赖跨框架的自我注意力；通过允许在自我注意计算过程中其他框架中的令牌上的令牌来提高主题一致性。虽然对单一主题有用，但我们发现它在缩放到多个字符时会挣扎。在这项工作中，我们首先分析了这些限制的原因。我们的探索表明，主要问题源于自我发挥链，这在试图确保多个字符的一致性时会加剧。当一个主题的令牌注意其他角色时，这会发生这种情况，导致它们彼此像彼此一样（例如，一只狗看起来像鸭子）。在这些发现的激励下，我们提出了StoryBooth：一种无训练的方法，用于提高多字符的一致性。特别是，我们首先利用多模式链的推理和基于区域的生成来将其定位在所需的故事输出中。然后，使用修改的扩散模型生成最终输出，该模型由两个新层组成：1）一个有界的跨框架自我发言层，用于减少字符间的注意力泄漏，以及2）令牌层，以提高细粒主体细节的一致性。通过定性和定量结果，我们发现所提出的方法超过了先前的最新方法，在多个字符和细谷物主题细节上都表现出提高的一致性。

Title: Parasite: A Steganography-based Backdoor Attack Framework for Diffusion Models

• Authors: Jiahao Chen, Yu Pan, Yi Du, Chunkai Wu, Lin Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05815
• Pdf URL: https://arxiv.org/pdf/2504.05815
• Copy Paste: [[2504.05815]] Parasite: A Steganography-based Backdoor Attack Framework for Diffusion Models(https://arxiv.org/abs/2504.05815)
• Keywords: generation
• Abstract: Recently, the diffusion model has gained significant attention as one of the most successful image generation models, which can generate high-quality images by iteratively sampling noise. However, recent studies have shown that diffusion models are vulnerable to backdoor attacks, allowing attackers to enter input data containing triggers to activate the backdoor and generate their desired output. Existing backdoor attack methods primarily focused on target noise-to-image and text-to-image tasks, with limited work on backdoor attacks in image-to-image tasks. Furthermore, traditional backdoor attacks often rely on a single, conspicuous trigger to generate a fixed target image, lacking concealability and flexibility. To address these limitations, we propose a novel backdoor attack method called "Parasite" for image-to-image tasks in diffusion models, which not only is the first to leverage steganography for triggers hiding, but also allows attackers to embed the target content as a backdoor trigger to achieve a more flexible attack. "Parasite" as a novel attack method effectively bypasses existing detection frameworks to execute backdoor attacks. In our experiments, "Parasite" achieved a 0 percent backdoor detection rate against the mainstream defense frameworks. In addition, in the ablation study, we discuss the influence of different hiding coefficients on the attack results. You can find our code at this https URL.
• 摘要：最近，扩散模型已成为最成功的图像生成模型之一，可以通过迭代采样噪声来产生高质量的图像。但是，最近的研究表明，扩散模型容易受到后门攻击的影响，使攻击者可以输入包含触发器的输入数据，以激活后门并生成所需的输出。现有的后门攻击方法主要集中在目标噪声到图像和文本对象任务上，对图像到图像任务中的后门攻击工作有限。此外，传统的后门攻击通常依靠一个明显的触发器来产生固定的目标图像，缺乏隐藏性和灵活性。为了解决这些限制，我们提出了一种新型的后门攻击方法，称为“寄生虫”，用于扩散模型中的图像到图像任务，这不仅是第一个利用触发器隐藏的隐肌造影的人，而且还允许攻击者将目标内容嵌入后门触发器以实现更灵活的攻击。 “寄生虫”作为一种新型攻击方法有效地绕过了现有的检测框架以执行后门攻击。在我们的实验中，“寄生虫”在主流防御框架中达到了0％的后门检测率。此外，在消融研究中，我们讨论了不同隐藏系数对攻击结果的影响。您可以在此HTTPS URL上找到我们的代码。

Title: Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking

• Authors: Junxi Chen, Junhao Dong, Xiaohua Xie
• Subjects: cs.CV, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.05838
• Pdf URL: https://arxiv.org/pdf/2504.05838
• Copy Paste: [[2504.05838]] Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking(https://arxiv.org/abs/2504.05838)
• Keywords: generation
• Abstract: Recently, the Image Prompt Adapter (IP-Adapter) has been increasingly integrated into text-to-image diffusion models (T2I-DMs) to improve controllability. However, in this paper, we reveal that T2I-DMs equipped with the IP-Adapter (T2I-IP-DMs) enable a new jailbreak attack named the hijacking attack. We demonstrate that, by uploading imperceptible image-space adversarial examples (AEs), the adversary can hijack massive benign users to jailbreak an Image Generation Service (IGS) driven by T2I-IP-DMs and mislead the public to discredit the service provider. Worse still, the IP-Adapter's dependency on open-source image encoders reduces the knowledge required to craft AEs. Extensive experiments verify the technical feasibility of the hijacking attack. In light of the revealed threat, we investigate several existing defenses and explore combining the IP-Adapter with adversarially trained models to overcome existing defenses' limitations. Our code is available at this https URL.
• 摘要：最近，图像提示适配器（IP-ADAPTER）已越来越多地集成到文本对图像扩散模型（T2I-DMS）中，以提高可控性。但是，在本文中，我们揭示了配备IP-ADAPTER（T2I-IP-DMS）的T2I-DMS使新的越狱攻击称为“劫持攻击”。我们证明，通过上传不察觉的图像空间对抗示例（AES），对手可以劫持大量的良性用户来越狱由T2I-IP-DMS驱动的图像生成服务（IGS），并误导公众以抹黑服务提供商。更糟糕的是，IP-Adapter对开源图像编码的依赖性减少了制作AES所需的知识。广泛的实验验证了劫持攻击的技术可行性。鉴于揭示的威胁，我们研究了几种现有的防御措施，并探索将IP-ADAPTER与对抗训练的模型相结合，以克服现有的防御限制。我们的代码可在此HTTPS URL上找到。

Title: SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation

• Authors: Hao Du, Bo Wu, Yan Lu, Zhendong Mao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05925
• Pdf URL: https://arxiv.org/pdf/2504.05925
• Copy Paste: [[2504.05925]] SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation(https://arxiv.org/abs/2504.05925)
• Keywords: generation
• Abstract: Vision-language temporal alignment is a crucial capability for human dynamic recognition and cognition in real-world scenarios. While existing research focuses on capturing vision-language relevance, it faces limitations due to biased temporal distributions, imprecise annotations, and insufficient compositionally. To achieve fair evaluation and comprehensive exploration, our objective is to investigate and evaluate the ability of models to achieve alignment from a temporal perspective, specifically focusing on their capacity to synchronize visual scenarios with linguistic context in a temporally coherent manner. As a preliminary step, we present the statistical analysis of existing benchmarks and reveal the existing challenges from a decomposed perspective. To this end, we introduce SVLTA, the Synthetic Vision-Language Temporal Alignment derived via a well-designed and feasible control generation method within a simulation environment. The approach considers commonsense knowledge, manipulable action, and constrained filtering, which generates reasonable, diverse, and balanced data distributions for diagnostic evaluations. Our experiments reveal diagnostic insights through the evaluations in temporal question answering, distributional shift sensitiveness, and temporal alignment adaptation.
• 摘要：视觉语言的时间对齐是现实情况下人类动态识别和认知的关键能力。尽管现有的研究重点是捕获视觉语言相关性，但由于时间分布，不精确的注释和构图不足，它面临局限性。为了实现公平的评估和全面的探索，我们的目标是调查和评估模型从时间角度实现一致性的能力，特别关注他们以时间连贯的方式将视觉场景与语言环境同步的能力。作为初步步骤，我们介绍了现有基准的统计分析，并从分解的角度揭示了现有的挑战。为此，我们介绍了SVLTA，这是通过模拟环境中设计良好且可行的控制生成方法得出的合成视觉时间对齐。该方法考虑了常识性知识，可操作的动作和受约束的过滤，从而为诊断评估产生合理，多样化和平衡的数据分布。我们的实验通过时间问题回答，分布转移敏感性和时间对齐适应性的评估来揭示诊断见解。

Title: Evaluation of the impact of expert knowledge: How decision support scores impact the effectiveness of automatic knowledge-driven feature engineering (aKDFE)

• Authors: Olof Björneld, Tora Hammar, Daniel Nilsson, Alisa Lincke, Welf Löwe
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05928
• Pdf URL: https://arxiv.org/pdf/2504.05928
• Copy Paste: [[2504.05928]] Evaluation of the impact of expert knowledge: How decision support scores impact the effectiveness of automatic knowledge-driven feature engineering (aKDFE)(https://arxiv.org/abs/2504.05928)
• Keywords: generation
• Abstract: Adverse Drug Events (ADEs), harmful medication effects, pose significant healthcare challenges, impacting patient safety and costs. This study evaluates automatic Knowledge-Driven Feature Engineering (aKDFE) for improved ADE prediction from Electronic Health Record (EHR) data, comparing it with automated event-based Knowledge Discovery in Databases (KDD). We investigated how incorporating domain-specific ADE risk scores for prolonged heart QT interval, extracted from the Janusmed Riskprofile (Janusmed) Clinical Decision Support System (CDSS), affects prediction performance using EHR data and medication handling events. Results indicate that, while aKDFE step 1 (event-based feature generation) alone did not significantly improve ADE prediction performance, aKDFE step 2 (patient-centric transformation) enhances the prediction performance. High Area Under the Receiver Operating Characteristic curve (AUROC) values suggest strong feature correlations to the outcome, aligning with the predictive power of patients' prior healthcare history for ADEs. Statistical analysis did not confirm that incorporating the Janusmed information (i) risk scores and (ii) medication route of administration into the model's feature set enhanced predictive performance. However, the patient-centric transformation applied by aKDFE proved to be a highly effective feature engineering approach. Limitations include a single-project focus, potential bias from machine learning pipeline methods, and reliance on AUROC. In conclusion, aKDFE, particularly with patient-centric transformation, improves ADE prediction from EHR data. Future work will explore attention-based models, event feature sequences, and automatic methods for incorporating domain knowledge into the aKDFE framework.
• 摘要：不良药物事件（ADE），有害药物效果，构成了重大的医疗挑战，影响了患者的安全性和成本。这项研究评估了自动知识驱动的功能工程（AKDFE），以改善电子健康记录（EHR）数据的ADE预测，并将其与数据库中基于自动事件的知识发现（KDD）进行了比较。我们调查了从延长的心脏QT间隔中纳入域特异性ADE风险评分，该间隔是从Janusmed Imparfile（JANUSMED）临床决策支持系统（CDSS）中提取的，使用EHR数据和药物处理事件影响预测性能。结果表明，尽管AKDFE步骤1（基于事件的功能生成）并不能显着提高ADE预测性能，但AKDFE步骤2（以患者为中心的转换）可以增强预测性能。接收器操作特征曲线（AUROC）值的高面积表明与结果的特征相关性很强，与患者先前的ADE医疗保健历史的预测能力保持一致。统计分析没有确认将概括的信息（i）风险分数和（ii）用药途径纳入模型的功能集增强的预测性能。但是，AKDFE应用的以患者为中心的转变被证明是一种非常有效的功能工程方法。局限性包括单个项目的焦点，机器学习管道方法的潜在偏见以及对AUROC的依赖。总之，AKDFE，尤其是以患者为中心的转化，可以改善EHR数据的ADE预测。未来的工作将探索基于注意力的模型，事件特征序列以及将域知识纳入AKDFE框架的自动方法。

Title: CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics

• Authors: Kuntian Zhang, Simin Yu, Yaoshu Wang, Makoto Onizuka, Chuan Xiao
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05945
• Pdf URL: https://arxiv.org/pdf/2504.05945
• Copy Paste: [[2504.05945]] CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics(https://arxiv.org/abs/2504.05945)
• Keywords: generative
• Abstract: In this paper, we propose CKGAN, a novel generative adversarial network (GAN) variant based on an integral probability metrics framework with characteristic kernel (CKIPM). CKIPM, as a distance between two probability distributions, is designed to optimize the lowerbound of the maximum mean discrepancy (MMD) in a reproducing kernel Hilbert space, and thus can be used to train GANs. CKGAN mitigates the notorious problem of mode collapse by mapping the generated images back to random noise. To save the effort of selecting the kernel function manually, we propose a soft selection method to automatically learn a characteristic kernel function. The experimental evaluation conducted on a set of synthetic and real image benchmarks (MNIST, CelebA, etc.) demonstrates that CKGAN generally outperforms other MMD-based GANs. The results also show that at the cost of moderately more training time, the automatically selected kernel function delivers very close performance to the best of manually fine-tuned one on real image benchmarks and is able to improve the performances of other MMD-based GANs.
• 摘要：在本文中，我们提出了基于具有特征内核（CKIPM）的积分概率指标框架的新型生成对抗网络（GAN）变体Ckgan。 CKIPM作为两个概率分布之间的距离，旨在优化再现核Hilbert空间中最大平均差异（MMD）的下部，因此可以用于训练gans。 Ckgan通过将生成的图像映射回随机噪声来减轻臭名昭著的模式崩溃问题。为了节省手动选择内核函数的努力，我们提出了一种自动学习特征内核函数的软选择方法。在一组合成和真实图像基准（MNIST，CELEBA等）上进行的实验评估表明，Ckgan通常比其他基于MMD的gans的表现胜过。结果还表明，以中等程度的训练时间为代价，自动选择的内核功能可在真实的图像基准上提供非常接近的性能，并在真实的图像基准上进行了精心调整的功能，并能够改善其他基于MMD的GAN的性能。

Title: Diffusion Based Ambiguous Image Segmentation

• Authors: Jakob Lønborg Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl, Vedrana Andersen Dahl
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05977
• Pdf URL: https://arxiv.org/pdf/2504.05977
• Copy Paste: [[2504.05977]] Diffusion Based Ambiguous Image Segmentation(https://arxiv.org/abs/2504.05977)
• Keywords: generative
• Abstract: Medical image segmentation often involves inherent uncertainty due to variations in expert annotations. Capturing this uncertainty is an important goal and previous works have used various generative image models for the purpose of representing the full distribution of plausible expert ground truths. In this work, we explore the design space of diffusion models for generative segmentation, investigating the impact of noise schedules, prediction types, and loss weightings. Notably, we find that making the noise schedule harder with input scaling significantly improves performance. We conclude that x- and v-prediction outperform epsilon-prediction, likely because the diffusion process is in the discrete segmentation domain. Many loss weightings achieve similar performance as long as they give enough weight to the end of the diffusion process. We base our experiments on the LIDC-IDRI lung lesion dataset and obtain state-of-the-art (SOTA) performance. Additionally, we introduce a randomly cropped variant of the LIDC-IDRI dataset that is better suited for uncertainty in image segmentation. Our model also achieves SOTA in this harder setting.
• 摘要：医疗图像细分通常涉及专家注释的变化而固有的不确定性。捕获这种不确定性是一个重要的目标，以前的作品使用了各种生成图像模型，目的是代表合理的专家基础真理的完整分布。在这项工作中，我们探讨了生成分割的扩散模型的设计空间，研究噪声表，预测类型和减肥权重的影响。值得注意的是，我们发现通过输入缩放更难使噪声时间表变得更加困难，从而显着提高了性能。我们得出的结论是，X-和V预测的表现优于Epsilon的预测，这可能是因为扩散过程在离散的分割域中。只要许多减肥的重量能够达到相似的性能，只要它们为扩散过程的结束提供足够的权重即可。我们将实验基于LIDC-IDRI肺病变数据集并获得最新性能（SOTA）性能。此外，我们引入了LIDC-IDRI数据集的随机裁剪变体，该变体更适合图像分割中的不确定性。我们的模型在这种更困难的环境中也可以实现SOTA。

Title: An Empirical Study of GPT-4o Image Generation Capabilities

• Authors: Sixiang Chen, Jinbin Bai, Zhuoran Zhao, Tian Ye, Qingyu Shi, Donghao Zhou, Wenhao Chai, Xin Lin, Jianzong Wu, Chao Tang, Shilin Xu, Tao Zhang, Haobo Yuan, Yikang Zhou, Wei Chow, Linfeng Li, Xiangtai Li, Lei Zhu, Lu Qi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05979
• Pdf URL: https://arxiv.org/pdf/2504.05979
• Copy Paste: [[2504.05979]] An Empirical Study of GPT-4o Image Generation Capabilities(https://arxiv.org/abs/2504.05979)
• Keywords: generation, generative
• Abstract: The landscape of image generation has rapidly evolved, from early GAN-based approaches to diffusion models and, most recently, to unified generative architectures that seek to bridge understanding and generation tasks. Recent advances, especially the GPT-4o, have demonstrated the feasibility of high-fidelity multimodal generation, their architectural design remains mysterious and unpublished. This prompts the question of whether image and text generation have already been successfully integrated into a unified framework for those methods. In this work, we conduct an empirical study of GPT-4o's image generation capabilities, benchmarking it against leading open-source and commercial models. Our evaluation covers four main categories, including text-to-image, image-to-image, image-to-3D, and image-to-X generation, with more than 20 tasks. Our analysis highlights the strengths and limitations of GPT-4o under various settings, and situates it within the broader evolution of generative modeling. Through this investigation, we identify promising directions for future unified generative models, emphasizing the role of architectural design and data scaling.
• 摘要：从基于GAN的早期方法到扩散模型，再到最近试图弥合理解和生成任务的统一生成架构，形象生成的景观已经迅速发展。最近的进步，尤其是GPT-4O，证明了高保真多模式的可行性，其建筑设计仍然神秘且未出版。这提示了图像和文本生成是否已经成功集成到这些方法的统一框架中的问题。在这项工作中，我们对GPT-4O的图像产生功能进行了实证研究，从而对其进行了基准测试，以防止领先的开源和商业模型。我们的评估涵盖了四个主要类别，包括文本对图像，图像到图像，图像到3D和图像到X的生成，具有20多个任务。我们的分析强调了在各种环境下GPT-4O的优势和局限性，并将其置于生成建模的更广泛演变之内。通过这项调查，我们确定了未来统一生成模型的有希望的方向，强调了建筑设计和数据扩展的作用。

Title: CamContextI2V: Context-aware Controllable Video Generation

• Authors: Luis Denninger, Sina Mokhtarzadeh Azar, Juergen Gall
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06022
• Pdf URL: https://arxiv.org/pdf/2504.06022
• Copy Paste: [[2504.06022]] CamContextI2V: Context-aware Controllable Video Generation(https://arxiv.org/abs/2504.06022)
• Keywords: generation, generative
• Abstract: Recently, image-to-video (I2V) diffusion models have demonstrated impressive scene understanding and generative quality, incorporating image conditions to guide generation. However, these models primarily animate static images without extending beyond their provided context. Introducing additional constraints, such as camera trajectories, can enhance diversity but often degrades visual quality, limiting their applicability for tasks requiring faithful scene representation. We propose CamContextI2V, an I2V model that integrates multiple image conditions with 3D constraints alongside camera control to enrich both global semantics and fine-grained visual details. This enables more coherent and context-aware video generation. Moreover, we motivate the necessity of temporal awareness for an effective context representation. Our comprehensive study on the RealEstate10K dataset demonstrates improvements in visual quality and camera controllability. We make our code and models publicly available at: this https URL.
• 摘要：最近，图像到视频（I2V）扩散模型表现出令人印象深刻的场景理解和生成质量，并结合了图像条件以引导生成。但是，这些模型主要是对静态图像进行动画动画，而无需扩展其提供的上下文。引入其他约束（例如摄像机轨迹）可以增强多样性，但经常会降低视觉质量，从而限制了其对需要忠实场景表示的任务的适用性。我们提出了CamContexti2V，这是一种I2V模型，将多个图像条件与3D约束与摄像机控制一起集成在一起，以丰富全局语义和细粒的视觉细节。这可以使更多连贯和上下文感知的视频生成。此外，我们激发了对有效上下文表示的时间意识的必要性。我们对RealEstate10K数据集的全面研究表明，视觉质量和相机可控性的改善。我们将代码和模型公开可用：此HTTPS URL。

Title: Trust-Region Twisted Policy Improvement

• Authors: Joery A. de Vries, Jinke He, Yaniv Oren, Matthijs T.J. Spaan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06048
• Pdf URL: https://arxiv.org/pdf/2504.06048
• Copy Paste: [[2504.06048]] Trust-Region Twisted Policy Improvement(https://arxiv.org/abs/2504.06048)
• Keywords: generation
• Abstract: Monte-Carlo tree search (MCTS) has driven many recent breakthroughs in deep reinforcement learning (RL). However, scaling MCTS to parallel compute has proven challenging in practice which has motivated alternative planners like sequential Monte-Carlo (SMC). Many of these SMC methods adopt particle filters for smoothing through a reformulation of RL as a policy inference problem. Yet, persisting design choices of these particle filters often conflict with the aim of online planning in RL, which is to obtain a policy improvement at the start of planning. Drawing inspiration from MCTS, we tailor SMC planners specifically for RL by improving data generation within the planner through constrained action sampling and explicit terminal state handling, as well as improving policy and value target estimation. This leads to our Trust-Region Twisted SMC (TRT-SMC), which shows improved runtime and sample-efficiency over baseline MCTS and SMC methods in both discrete and continuous domains.
• 摘要：蒙特 - 卡洛树搜索（MCT）驱动了许多最近的深入增强学习（RL）的突破。但是，将MCT缩放到并行计算中已被证明在实践中具有挑战性，这促使替代计划者（如Sequential Monte-Carlo（SMC））。这些SMC方法中的许多方法采用粒子过滤器，通过对RL的重新制定为政策推断问题进行平滑。但是，这些粒子过滤器的持续设计选择通常会与RL在线规划的目的冲突，这是在计划开始时获得政策改进。我们从MCT中汲取灵感，我们通过限制的动作采样和显式终端处理以及改善政策和价值目标估计来改善计划者中的数据生成，从而为RL量身定制。这导致了我们的信任区域扭曲的SMC（TRT-SMC），它显示了离散和连续域中基线MCT和SMC方法的运行时和样本效率的提高。

Title: Explainable AI for building energy retrofitting under data scarcity

• Authors: Panagiota Rempi, Sotiris Pelekis, Alexandros Menelaos Tzortzis, Evangelos Karakolis, Christos Ntanos, Dimitris Askounis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06055
• Pdf URL: https://arxiv.org/pdf/2504.06055
• Copy Paste: [[2504.06055]] Explainable AI for building energy retrofitting under data scarcity(https://arxiv.org/abs/2504.06055)
• Keywords: generative
• Abstract: Enhancing energy efficiency in residential buildings is a crucial step toward mitigating climate change and reducing greenhouse gas emissions. Retrofitting existing buildings, which account for a significant portion of energy consumption, is critical particularly in regions with outdated and inefficient building stocks. This study presents an Artificial Intelligence (AI) and Machine Learning (ML)-based framework to recommend energy efficiency measures for residential buildings, leveraging accessible building characteristics to achieve energy class targets. Using Latvia as a case study, the methodology addresses challenges associated with limited datasets, class imbalance and data scarcity. The proposed approach integrates Conditional Tabular Generative Adversarial Networks (CTGAN) to generate synthetic data, enriching and balancing the dataset. A Multi-Layer Perceptron (MLP) model serves as the predictive model performing multi-label classification to predict appropriate retrofit strategies. Explainable Artificial Intelligence (XAI), specifically SHapley Additive exPlanations (SHAP), ensures transparency and trust by identifying key features that influence recommendations and guiding feature engineering choices for improved reliability and performance. The evaluation of the approach shows that it notably overcomes data limitations, achieving improvements up to 54% in precision, recall and F1 score. Although this study focuses on Latvia, the methodology is adaptable to other regions, underscoring the potential of AI in reducing the complexity and cost of building energy retrofitting overcoming data limitations. By facilitating decision-making processes and promoting stakeholders engagement, this work supports the global transition toward sustainable energy use in the residential building sector.
• 摘要：提高住宅建筑物的能源效率是缓解气候变化并减少温室气体排放的关键一步。改造现有建筑物的大部分能源消耗，尤其是在建筑物过时且效率低下的地区至关重要。这项研究提出了一个基于人工智能（AI）和机器学习（ML）的框架，以推荐住宅建筑的能源效率措施，利用可访问的建筑特征来实现能源类别的目标。该方法将拉脱维亚作为案例研究，解决了与有限的数据集，类失衡和数据稀缺性相关的挑战。提出的方法集成了条件表格生成的对抗网络（CTGAN），以生成合成数据，丰富和平衡数据集。多层感知器（MLP）模型是执行多标签分类的预测模型，以预测适当的改造策略。可解释的人工智能（XAI），特别是Shapley添加说明（SHAP），通过识别影响建议和指导功能工程选择的关键功能来确保透明度和信任，以提高可靠性和性能。该方法的评估表明，它显着克服了数据限制，在精确度，召回和F1分数方面取得了高达54％的改进。尽管这项研究的重点是拉脱维亚，但该方法可适应其他地区，强调了AI在降低建筑能源改造克服数据限制的复杂性和成本方面的潜力。通过促进决策流程并促进利益相关者的参与，这项工作支持了全球向住宅建筑部门可持续能源使用的过渡。

Title: FaceCloak: Learning to Protect Face Templates

• Authors: Sudipta Banerjee, Anubhav Jain, Chinmay Hegde, Nasir Memon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06131
• Pdf URL: https://arxiv.org/pdf/2504.06131
• Copy Paste: [[2504.06131]] FaceCloak: Learning to Protect Face Templates(https://arxiv.org/abs/2504.06131)
• Keywords: generative
• Abstract: Generative models can reconstruct face images from encoded representations (templates) bearing remarkable likeness to the original face raising security and privacy concerns. We present FaceCloak, a neural network framework that protects face templates by generating smart, renewable binary cloaks. Our method proactively thwarts inversion attacks by cloaking face templates with unique disruptors synthesized from a single face template on the fly while provably retaining biometric utility and unlinkability. Our cloaked templates can suppress sensitive attributes while generalizing to novel feature extraction schemes and outperforms leading baselines in terms of biometric matching and resiliency to reconstruction attacks. FaceCloak-based matching is extremely fast (inference time cost=0.28ms) and light-weight (0.57MB).
• 摘要：生成模型可以从编码表示（模板）中重建面部图像，与原始面部提高安全性和隐私问题相似。我们提出Facecloak，这是一种神经网络框架，可通过产生智能，可再生的二进制斗篷来保护面部模板。我们的方法通过将面部模板与独特的干扰器掩盖，从单个面部模板中合成，同时可以保留生物特征识别实用程序和无链接，从而主动阻碍了反转攻击。我们的隐藏模板可以抑制敏感属性，同时将新型特征提取方案推广，并且在生物识别匹配和对重建攻击的弹性方面胜过领先的基线。基于Facecloak的匹配非常快（推理时间成本= 0.28ms）和轻量重量（0.57MB）。

Title: A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model

• Authors: Jihun Park, Jongmin Gim, Kyoungmin Lee, Minseok Oh, Minwoo Choi, Jaeyeul Kim, Woo Chool Park, Sunghoon Im
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06144
• Pdf URL: https://arxiv.org/pdf/2504.06144
• Copy Paste: [[2504.06144]] A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model(https://arxiv.org/abs/2504.06144)
• Keywords: generation
• Abstract: We present a training-free style-aligned image generation method that leverages a scale-wise autoregressive model. While large-scale text-to-image (T2I) models, particularly diffusion-based methods, have demonstrated impressive generation quality, they often suffer from style misalignment across generated image sets and slow inference speeds, limiting their practical usability. To address these issues, we propose three key components: initial feature replacement to ensure consistent background appearance, pivotal feature interpolation to align object placement, and dynamic style injection, which reinforces style consistency using a schedule function. Unlike previous methods requiring fine-tuning or additional training, our approach maintains fast inference while preserving individual content details. Extensive experiments show that our method achieves generation quality comparable to competing approaches, significantly improves style alignment, and delivers inference speeds over six times faster than the fastest model.
• 摘要：我们提出了一种由无训练样式的图像生成方法，该方法利用了尺度自回归模型的模型。尽管大规模的文本形象图（T2I）模型，尤其是基于扩散的方法，表现出令人印象深刻的生成质量，但它们通常会遇到跨产生的图像集和缓慢推理速度的风格错位，从而限制了它们的实际可用性。为了解决这些问题，我们提出了三个关键组成部分：初始功能更换，以确保背景外观，关键特征插值以对准对象放置以及动态样式注入，从而使用计划功能来增强样式一致性。与以前需要微调或额外培训的方法不同，我们的方法在保留个人内容细节的同时保持快速推理。广泛的实验表明，我们的方法可以达到与竞争方法相当的发电质量，可显着提高样式的一致性，并提供的推理速度比最快的模型快六倍。

Title: A Self-Supervised Framework for Space Object Behaviour Characterisation

• Authors: Ian Groves, Andrew Campbell, James Fernandes, Diego Rodriguez, Paul Murray, Massimiliano Vasile, Victoria Nockles
• Subjects: cs.LG, cs.AI, physics.space-ph
• Abstract URL: https://arxiv.org/abs/2504.06176
• Pdf URL: https://arxiv.org/pdf/2504.06176
• Copy Paste: [[2504.06176]] A Self-Supervised Framework for Space Object Behaviour Characterisation(https://arxiv.org/abs/2504.06176)
• Keywords: generation
• Abstract: Foundation Models, pre-trained on large unlabelled datasets before task-specific fine-tuning, are increasingly being applied to specialised domains. Recent examples include ClimaX for climate and Clay for satellite Earth observation, but a Foundation Model for Space Object Behavioural Analysis has not yet been developed. As orbital populations grow, automated methods for characterising space object behaviour are crucial for space safety. We present a Space Safety and Sustainability Foundation Model focusing on space object behavioural analysis using light curves (LCs). We implemented a Perceiver-Variational Autoencoder (VAE) architecture, pre-trained with self-supervised reconstruction and masked reconstruction on 227,000 LCs from the MMT-9 observatory. The VAE enables anomaly detection, motion prediction, and LC generation. We fine-tuned the model for anomaly detection & motion prediction using two independent LC simulators (CASSANDRA and GRIAL respectively), using CAD models of boxwing, Sentinel-3, SMOS, and Starlink platforms. Our pre-trained model achieved a reconstruction error of 0.01%, identifying potentially anomalous light curves through reconstruction difficulty. After fine-tuning, the model scored 88% and 82% accuracy, with 0.90 and 0.95 ROC AUC scores respectively in both anomaly detection and motion mode prediction (sun-pointing, spin, etc.). Analysis of high-confidence anomaly predictions on real data revealed distinct patterns including characteristic object profiles and satellite glinting. Here, we demonstrate how self-supervised learning can simultaneously enable anomaly detection, motion prediction, and synthetic data generation from rich representations learned in pre-training. Our work therefore supports space safety and sustainability through automated monitoring and simulation capabilities.
• 摘要：在特定于任务的微调之前，在大型未标记数据集中进行了预先培训的基础模型越来越多地应用于专用域。最近的例子包括气候高潮和粘土的卫星观测，但是尚未开发出空间对象行为分析的基础模型。随着轨道种群的增长，表征空间对象行为的自动化方法对于空间安全至关重要。我们提出了一个专注于使用光曲线（LCS）的空间对象行为分析的空间安全和可持续性基础模型。我们实施了一个感知者变量自动编码器（VAE）体系结构，并在MMT-9天文台对227,000 LC进行了自我监管的重建和掩盖重建。 VAE可实现异常检测，运动预测和LC产生。我们使用两个独立的LC模拟器（分别使用Cassandra和Grial）微调了用于异常检测和运动预测的模型，该模型使用Boxwing，Sentinel-3，SMOS和Starlink平台的CAD模型。我们的预训练模型的重建误差为0.01％，通过重建难度确定了潜在的异常光曲线。微调后，该模型在异常检测和运动模式预测（日落，旋转等）中分别为88％和82％的精度得分，ROC AUC分别为0.90和0.95。对实际数据的高信心异常预测的分析揭示了不同的模式，包括特征对象概况和卫星闪烁。在这里，我们证明了自我监督的学习如何同时从培训中学到的丰富表示形式同时实现异常检测，运动预测和合成数据生成。因此，我们的工作通过自动监控和模拟功能来支持空间安全和可持续性。

Title: NNN: Next-Generation Neural Networks for Marketing Mix Modeling

• Authors: Thomas Mulc, Mike Anderson, Paul Cubre, Huikun Zhang, Ivy Liu, Saket Kumar
• Subjects: cs.LG, stat.AP
• Abstract URL: https://arxiv.org/abs/2504.06212
• Pdf URL: https://arxiv.org/pdf/2504.06212
• Copy Paste: [[2504.06212]] NNN: Next-Generation Neural Networks for Marketing Mix Modeling(https://arxiv.org/abs/2504.06212)
• Keywords: generation
• Abstract: We present NNN, a Transformer-based neural network approach to Marketing Mix Modeling (MMM) designed to address key limitations of traditional methods. Unlike conventional MMMs which rely on scalar inputs and parametric decay functions, NNN uses rich embeddings to capture both quantitative and qualitative aspects of marketing and organic channels (e.g., search queries, ad creatives). This, combined with its attention mechanism, enables NNN to model complex interactions, capture long-term effects, and potentially improve sales attribution accuracy. We show that L1 regularization permits the use of such expressive models in typical data-constrained settings. Evaluating NNN on simulated and real-world data demonstrates its efficacy, particularly through considerable improvement in predictive power. Beyond attribution, NNN provides valuable, complementary insights through model probing, such as evaluating keyword or creative effectiveness, enhancing model interpretability.
• 摘要：我们提出了NNN，这是一种基于变压器的神经网络方法来营销组合建模（MMM），旨在解决传统方法的关键局限性。与依赖标量输入和参数衰减功能的常规MMM不同，NNN使用丰富的嵌入方式来捕获营销和有机渠道的定量和定性方面（例如，搜索查询，广告创建剂）。这与其注意力机制相结合，使NNN能够对复杂的相互作用进行建模，捕获长期效果并有可能提高销售归因准确性。我们表明，L1正则化允许在典型的数据约束设置中使用此类表达模型。在模拟和现实世界中评估NNN证明了其功效，尤其是通过大量提高预测能力。除归因之外，NNN通过模型探测提供了有价值的互补见解，例如评估关键字或创造性效果，增强模型的解释性。

Title: HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance

• Authors: Jiazi Bu, Pengyang Ling, Yujie Zhou, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Dahua Lin, Jiaqi Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06232
• Pdf URL: https://arxiv.org/pdf/2504.06232
• Copy Paste: [[2504.06232]] HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance(https://arxiv.org/abs/2504.06232)
• Keywords: generation
• Abstract: Text-to-image (T2I) diffusion/flow models have drawn considerable attention recently due to their remarkable ability to deliver flexible visual creations. Still, high-resolution image synthesis presents formidable challenges due to the scarcity and complexity of high-resolution content. To this end, we present HiFlow, a training-free and model-agnostic framework to unlock the resolution potential of pre-trained flow models. Specifically, HiFlow establishes a virtual reference flow within the high-resolution space that effectively captures the characteristics of low-resolution flow information, offering guidance for high-resolution generation through three key aspects: initialization alignment for low-frequency consistency, direction alignment for structure preservation, and acceleration alignment for detail fidelity. By leveraging this flow-aligned guidance, HiFlow substantially elevates the quality of high-resolution image synthesis of T2I models and demonstrates versatility across their personalized variants. Extensive experiments validate HiFlow's superiority in achieving superior high-resolution image quality over current state-of-the-art methods.
• 摘要：文本对图像（T2I）扩散/流模型最近引起了广泛的关注，因为它们具有出色的传递柔性视觉创作的能力。尽管如此，由于高分辨率含量的稀缺性和复杂性，高分辨率图像综合构成了巨大的挑战。为此，我们提出了Hiflow，这是一种无训练和模型不合时宜的框架，可释放预训练流动模型的分辨率潜力。具体而言，Hiflow在高分辨率空间内建立了一个虚拟参考流，该流量有效地捕获了低分辨率流量信息的特征，通过三个关键方面为高分辨率生成提供了指导：低频一致性的初始化一致性，低频一致性，结构保存方向对准方向一致性和加速对齐的详细信息。通过利用这一与流动对准的指导，Hiflow显着提高了T2I模型的高分辨率图像合成的质量，并在其个性化变体中证明了多功能性。广泛的实验验证了Hiflow在实现优越的高分辨率图像质量上优于当前最新方法的优势。

Title: Transfer between Modalities with MetaQueries

• Authors: Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, Ji Hou, Saining Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06256
• Pdf URL: https://arxiv.org/pdf/2504.06256
• Copy Paste: [[2504.06256]] Transfer between Modalities with MetaQueries(https://arxiv.org/abs/2504.06256)
• Keywords: generation, generative
• Abstract: Unified multimodal models aim to integrate understanding (text output) and generation (pixel output), but aligning these different modalities within a single architecture often demands complex training recipes and careful data balancing. We introduce MetaQueries, a set of learnable queries that act as an efficient interface between autoregressive multimodal LLMs (MLLMs) and diffusion models. MetaQueries connects the MLLM's latents to the diffusion decoder, enabling knowledge-augmented image generation by leveraging the MLLM's deep understanding and reasoning capabilities. Our method simplifies training, requiring only paired image-caption data and standard diffusion objectives. Notably, this transfer is effective even when the MLLM backbone remains frozen, thereby preserving its state-of-the-art multimodal understanding capabilities while achieving strong generative performance. Additionally, our method is flexible and can be easily instruction-tuned for advanced applications such as image editing and subject-driven generation.
• 摘要：统一的多模式模型旨在整合理解（文本输出）和发电（像素输出），但是单个体系结构中的这些不同模式通常需要复杂的培训配方和仔细的数据平衡。我们介绍了Metaqueries，这是一组可学习的查询，它们是自回归多模式LLM（MLLM）和扩散模型之间有效界面的。 Metaqueries通过利用MLLM的深刻理解和推理能力，将MLLM的潜在的潜在解码器连接到扩散解码器。我们的方法简化了训练，仅需要配对的图像捕获数据和标准扩散目标。值得注意的是，即使MLLM骨架保持冷冻，这种转移也是有效的，从而保留其最先进的多模式理解能力，同时实现强大的生成性能。此外，我们的方法是灵活的，可以轻松地针对高级应用程序进行指导，例如图像编辑和主题驱动的生成。

Title: Hogwild! Inference: Parallel LLM Generation via Concurrent Attention

• Authors: Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, Dan Alistarh
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06261
• Pdf URL: https://arxiv.org/pdf/2504.06261
• Copy Paste: [[2504.06261]] Hogwild! Inference: Parallel LLM Generation via Concurrent Attention(https://arxiv.org/abs/2504.06261)
• Keywords: generation
• Abstract: Large Language Models (LLMs) have demonstrated the ability to tackle increasingly complex tasks through advanced reasoning, long-form content generation, and tool use. Solving these tasks often involves long inference-time computations. In human problem solving, a common strategy to expedite work is collaboration: by dividing the problem into sub-tasks, exploring different strategies concurrently, etc. Recent research has shown that LLMs can also operate in parallel by implementing explicit cooperation frameworks, such as voting mechanisms or the explicit creation of independent sub-tasks that can be executed in parallel. However, each of these frameworks may not be suitable for all types of tasks, which can hinder their applicability. In this work, we propose a different design approach: we run LLM "workers" in parallel , allowing them to synchronize via a concurrently-updated attention cache and prompt these workers to decide how best to collaborate. Our approach allows the instances to come up with their own collaboration strategy for the problem at hand, all the while "seeing" each other's partial progress in the concurrent cache. We implement this approach via Hogwild! Inference: a parallel LLM inference engine where multiple instances of the same LLM run in parallel with the same attention cache, with "instant" access to each other's generated tokens. Hogwild! inference takes advantage of Rotary Position Embeddings (RoPE) to avoid recomputation while improving parallel hardware utilization. We find that modern reasoning-capable LLMs can perform inference with shared Key-Value cache out of the box, without additional fine-tuning.
• 摘要：大型语言模型（LLMS）已经证明了通过高级推理，长形成内容生成和工具使用来解决日益复杂的任务的能力。解决这些任务通常涉及长时间的推理时间计算。在人类问题解决中，加快工作的共同策略是协作：通过将问题分为子任务，同时探索不同的策略等。最近的研究表明，LLM还可以通过实施明确的合作框架，例如投票机制或明确创建可以并行执行的独立子任务来平行运行。但是，这些框架中的每一个都不适合所有类型的任务，这可能会阻碍其适用性。在这项工作中，我们提出了一种不同的设计方法：我们并行运行LLM“工人”，使他们能够通过同时升级的注意缓存同步，并促使这些工人决定如何最好地协作。我们的方法使实例可以提出他们自己的问题的协作策略，同时“看到”彼此在并发缓存中的部分进步。我们通过Hogwild实施这种方法！推理：平行的LLM推理引擎，其中相同LLM的多个实例与同一注意缓存并行运行，并且“即时”访问彼此的生成令牌。霍格维尔！推理利用旋转位置嵌入（绳索），以避免重新计算，同时改善并行硬件利用率。我们发现，具有现代推理能力的LLM可以通过开箱即用的共享键值缓存来执行推理，而无需进行其他微调。

Title: OmniSVG: A Unified Scalable Vector Graphics Generation Model

• Authors: Yiying Yang, Wei Cheng, Sijin Chen, Xianfang Zeng, Jiaxu Zhang, Liao Wang, Gang Yu, Xingjun Ma, Yu-Gang Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06263
• Pdf URL: https://arxiv.org/pdf/2504.06263
• Copy Paste: [[2504.06263]] OmniSVG: A Unified Scalable Vector Graphics Generation Model(https://arxiv.org/abs/2504.06263)
• Keywords: generation
• Abstract: Scalable Vector Graphics (SVG) is an important image format widely adopted in graphic design because of their resolution independence and editability. The study of generating high-quality SVG has continuously drawn attention from both designers and researchers in the AIGC community. However, existing methods either produces unstructured outputs with huge computational cost or is limited to generating monochrome icons of over-simplified structures. To produce high-quality and complex SVG, we propose OmniSVG, a unified framework that leverages pre-trained Vision-Language Models (VLMs) for end-to-end multimodal SVG generation. By parameterizing SVG commands and coordinates into discrete tokens, OmniSVG decouples structural logic from low-level geometry for efficient training while maintaining the expressiveness of complex SVG structure. To further advance the development of SVG synthesis, we introduce MMSVG-2M, a multimodal dataset with two million richly annotated SVG assets, along with a standardized evaluation protocol for conditional SVG generation tasks. Extensive experiments show that OmniSVG outperforms existing methods and demonstrates its potential for integration into professional SVG design workflows.
• 摘要：可扩展的向量图形（SVG）是图形设计中广泛采用的重要图像格式，因为它们具有分辨率的独立性和编辑性。生成高质量SVG的研究不断引起AIGC社区设计师和研究人员的关注。但是，现有方法要么产生具有巨大计算成本的非结构化输出，要么仅限于生成过度简化结构的单色图标。为了产生高质量且复杂的SVG，我们提出了OmniSVG，这是一个统一的框架，该框架利用预先训练的视觉模型（VLM）来端到端多模式SVG生成。通过将SVG命令参数化并坐标为离散令牌，Omnisvg将结构逻辑从低级几何形状中脱离，以有效地训练，同时保持复杂SVG结构的表现力。为了进一步推进SVG合成的开发，我们引入了MMSVG-2M，这是一个具有200万个注释SVG资产的多模式数据集，以及针对有条件的SVG生成任务的标准化评估协议。广泛的实验表明，OmniSVG优于现有方法，并证明了其将其集成到专业SVG设计工作流程中的潜力。