2025-02-21

Title: Smaller But Better: Unifying Layout Generation with Smaller Large Language Models

• Authors: Peirong Zhang, Jiaxin Zhang, Jiahuan Cao, Hongliang Li, Lianwen Jin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.14005
• Pdf URL: https://arxiv.org/pdf/2502.14005
• Copy Paste: [[2502.14005]] Smaller But Better: Unifying Layout Generation with Smaller Large Language Models(https://arxiv.org/abs/2502.14005)
• Keywords: generation
• Abstract: We propose LGGPT, an LLM-based model tailored for unified layout generation. First, we propose Arbitrary Layout Instruction (ALI) and Universal Layout Response (ULR) as the uniform I/O template. ALI accommodates arbitrary layout generation task inputs across multiple layout domains, enabling LGGPT to unify both task-generic and domain-generic layout generation hitherto unexplored. Collectively, ALI and ULR boast a succinct structure that forgoes superfluous tokens typically found in existing HTML-based formats, facilitating efficient instruction tuning and boosting unified generation performance. In addition, we propose an Interval Quantization Encoding (IQE) strategy that compresses ALI into a more condensed structure. IQE precisely preserves valid layout clues while eliminating the less informative placeholders, facilitating LGGPT to capture complex and variable layout generation conditions during the unified training process. Experimental results demonstrate that LGGPT achieves superior or on par performance compared to existing methods. Notably, LGGPT strikes a prominent balance between proficiency and efficiency with a compact 1.5B parameter LLM, which beats prior 7B or 175B models even in the most extensive and challenging unified scenario. Furthermore, we underscore the necessity of employing LLMs for unified layout generation and suggest that 1.5B could be an optimal parameter size by comparing LLMs of varying scales. Code is available at this https URL.
• 摘要：我们提出了LGGPT，这是一种基于LLM的模型，该模型量身定制为统一布局生成。首先，我们将任意布局指令（ALI）和通用布局响应（ULR）作为统一I/O模板。 ALI可以在多个布局域上容纳任意布局生成任务输入，从而使LGGPT能够统一任务生成和域的布局生成迄今未探索的生成。总体而言，Ali和Ulr拥有一种简洁的结构，它放弃了通常以HTML基于HTML的格式发现的多余令牌，从而促进了有效的教学调整并提高统一的生成性能。此外，我们提出了一个间隔量化编码（IQE）策略，该策略将ALI压缩为更凝结的结构。 IQE准确地保留了有效的布局线索，同时消除了较少信息的占位符，从而促进LGGPT在统一培训过程中捕获复杂而可变的布局生​​成条件。实验结果表明，与现有方法相比，LGGPT取得了优越或par性能。值得注意的是，LGGPT与紧凑的1.5B参数LLM之间的熟练程度和效率之间达到了显着的平衡，即使在最广泛，最具挑战性的统一场景中，它也比以前的7B或175B模型击败了先前的7B或175B模型。此外，我们强调了使用LLM进行统一布局生成的必要性，并建议1.5B可以通过比较不同尺度的LLM来成为最佳参数大小。代码可在此HTTPS URL上找到。

Title: A Racing Dataset and Baseline Model for Track Detection in Autonomous Racing

• Authors: Shreya Ghosh, Yi-Huan Chen, Ching-Hsiang Huang, Abu Shafin Mohammad Mahdee Jameel, Chien Chou Ho, Aly El Gamal, Samuel Labi
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.14068
• Pdf URL: https://arxiv.org/pdf/2502.14068
• Copy Paste: [[2502.14068]] A Racing Dataset and Baseline Model for Track Detection in Autonomous Racing(https://arxiv.org/abs/2502.14068)
• Keywords: generative
• Abstract: A significant challenge in racing-related research is the lack of publicly available datasets containing raw images with corresponding annotations for the downstream task. In this paper, we introduce RoRaTrack, a novel dataset that contains annotated multi-camera image data from racing scenarios for track detection. The data is collected on a Dallara AV-21 at a racing circuit in Indiana, in collaboration with the Indy Autonomous Challenge (IAC). RoRaTrack addresses common problems such as blurriness due to high speed, color inversion from the camera, and absence of lane markings on the track. Consequently, we propose RaceGAN, a baseline model based on a Generative Adversarial Network (GAN) that effectively addresses these challenges. The proposed model demonstrates superior performance compared to current state-of-the-art machine learning models in track detection. The dataset and code for this work are available at this http URL.
• 摘要：与赛车相关的研究中的一个重大挑战是缺乏包含针对下游任务的相应注释的原始图像的公开数据集。在本文中，我们介绍了Roratrack，Roratrack是一个新颖的数据集，其中包含来自赛车场景的带注释的多相机图像数据，以进行轨道检测。该数据是在印第安纳州的赛车巡回赛上与Indy Autonomous Challenge（IAC）合作收集的。 Roratrack解决了常见问题，例如由于高速，摄像机的颜色反转以及轨道上没有车道标记而导致的颜色。因此，我们建议Racegan，这是一种基于生成对抗网络（GAN）的基线模型，可有效解决这些挑战。所提出的模型与轨道检测中的当前最新机器学习模型相比表明了卓越的性能。此工作的数据集和代码可在此HTTP URL上找到。

Title: DiffExp: Efficient Exploration in Reward Fine-tuning for Text-to-Image Diffusion Models

• Authors: Daewon Chae, June Suk Choi, Jinkyu Kim, Kimin Lee
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14070
• Pdf URL: https://arxiv.org/pdf/2502.14070
• Copy Paste: [[2502.14070]] DiffExp: Efficient Exploration in Reward Fine-tuning for Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.14070)
• Keywords: generation
• Abstract: Fine-tuning text-to-image diffusion models to maximize rewards has proven effective for enhancing model performance. However, reward fine-tuning methods often suffer from slow convergence due to online sample generation. Therefore, obtaining diverse samples with strong reward signals is crucial for improving sample efficiency and overall performance. In this work, we introduce DiffExp, a simple yet effective exploration strategy for reward fine-tuning of text-to-image models. Our approach employs two key strategies: (a) dynamically adjusting the scale of classifier-free guidance to enhance sample diversity, and (b) randomly weighting phrases of the text prompt to exploit high-quality reward signals. We demonstrate that these strategies significantly enhance exploration during online sample generation, improving the sample efficiency of recent reward fine-tuning methods, such as DDPO and AlignProp.
• 摘要：证明可以最大程度地提高奖励的文本对图像扩散模型已被证明有效地增强了模型性能。但是，奖励微调方法通常由于在线样本的生成而经常遭受缓慢的收敛性。因此，获得具有强烈奖励信号的不同样品对于提高样本效率和整体性能至关重要。在这项工作中，我们介绍了DIFFEXP，这是一种简单而有效的探索策略，用于奖励文本对图像模型的微调。我们的方法采用了两种关键策略：（a）动态调整无分类器指导的规模以增强样本多样性，以及（b）文本提示的随机加权短语，以利用高质量的奖励信号。我们证明，这些策略可以显着增强在线样本生成期间的探索，从而提高了最近的奖励微调方法的样本效率，例如DDPO和AlignProp。

Title: Understanding SGD with Exponential Moving Average: A Case Study in Linear Regression

• Authors: Xuheng Li, Quanquan Gu
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.14123
• Pdf URL: https://arxiv.org/pdf/2502.14123
• Copy Paste: [[2502.14123]] Understanding SGD with Exponential Moving Average: A Case Study in Linear Regression(https://arxiv.org/abs/2502.14123)
• Keywords: generative
• Abstract: Exponential moving average (EMA) has recently gained significant popularity in training modern deep learning models, especially diffusion-based generative models. However, there have been few theoretical results explaining the effectiveness of EMA. In this paper, to better understand EMA, we establish the risk bound of online SGD with EMA for high-dimensional linear regression, one of the simplest overparameterized learning tasks that shares similarities with neural networks. Our results indicate that (i) the variance error of SGD with EMA is always smaller than that of SGD without averaging, and (ii) unlike SGD with iterate averaging from the beginning, the bias error of SGD with EMA decays exponentially in every eigen-subspace of the data covariance matrix. Additionally, we develop proof techniques applicable to the analysis of a broad class of averaging schemes.
• 摘要：指数移动平均线（EMA）最近在培训现代深度学习模型，尤其是基于扩散的生成模型方面获得了巨大的知名度。但是，几乎没有理论上的结果来解释EMA的有效性。在本文中，为了更好地理解EMA，我们建立了与EMA的在线SGD的风险限制，用于高维线性回归，这是最简单的过度参数学习任务之一，与神经网络具有相似之处。我们的结果表明，（i）EMA的SGD的差异误差总是比SGD的差异差异，而没有平均值，并且（ii）与SGD不同，从一开始就迭代的SGD，SGD的偏差误差与EMA衰减的偏差误差在每个EIGEN-中呈指数级别。数据协方差矩阵的子空间。此外，我们开发了适用于分析一类平均方案的证明技术。

Title: ModSkill: Physical Character Skill Modularization

• Authors: Yiming Huang, Zhiyang Dou, Lingjie Liu
• Subjects: cs.CV, cs.GR, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.14140
• Pdf URL: https://arxiv.org/pdf/2502.14140
• Copy Paste: [[2502.14140]] ModSkill: Physical Character Skill Modularization(https://arxiv.org/abs/2502.14140)
• Keywords: generation, generative
• Abstract: Human motion is highly diverse and dynamic, posing challenges for imitation learning algorithms that aim to generalize motor skills for controlling simulated characters. Previous methods typically rely on a universal full-body controller for tracking reference motion (tracking-based model) or a unified full-body skill embedding space (skill embedding). However, these approaches often struggle to generalize and scale to larger motion datasets. In this work, we introduce a novel skill learning framework, ModSkill, that decouples complex full-body skills into compositional, modular skills for independent body parts. Our framework features a skill modularization attention layer that processes policy observations into modular skill embeddings that guide low-level controllers for each body part. We also propose an Active Skill Learning approach with Generative Adaptive Sampling, using large motion generation models to adaptively enhance policy learning in challenging tracking scenarios. Our results show that this modularized skill learning framework, enhanced by generative sampling, outperforms existing methods in precise full-body motion tracking and enables reusable skill embeddings for diverse goal-driven tasks.
• 摘要：人类的运动是高度多样和动态的，对模仿学习算法的挑战构成挑战，旨在概括控制模拟角色的运动技能。以前的方法通常依靠通用全身控制器来跟踪参考运动（基于跟踪的模型）或统一的全身技能嵌入空间（技能嵌入）。但是，这些方法通常很难概括和扩展到更大的运动数据集。在这项工作中，我们介绍了一个新颖的技能学习框架Modskill，该框架将复杂的全身技能纳入了独立身体部位的组成，模块化技能。我们的框架具有一个技能模块化注意力层，该层将策略观察过程处理为模块化技能嵌入，可指导每个身体部位的低级控制器。我们还提出了一种具有生成自适应抽样的积极的技能学习方法，使用大型运动生成模型在挑战性跟踪方案中适应性地增强政策学习。我们的结果表明，通过生成抽样增强了这个模块化的技能学习框架，在精确的全身运动跟踪中优于现有方法，并启用可重复使用的技能嵌入，以实现各种目标驱动的任务。

Title: Accurate Forgetting for Heterogeneous Federated Continual Learning

• Authors: Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, Masashi Sugiyama
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14205
• Pdf URL: https://arxiv.org/pdf/2502.14205
• Copy Paste: [[2502.14205]] Accurate Forgetting for Heterogeneous Federated Continual Learning(https://arxiv.org/abs/2502.14205)
• Keywords: generative
• Abstract: Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging FL and continual learning (CL) gives rise to a challenging practical problem: federated continual learning (FCL). Existing research in FCL primarily focuses on mitigating the catastrophic forgetting issue of continual learning while collaborating with other clients. We argue that the forgetting phenomena are not invariably detrimental. In this paper, we consider a more practical and challenging FCL setting characterized by potentially unrelated or even antagonistic data/tasks across different clients. In the FL scenario, statistical heterogeneity and data noise among clients may exhibit spurious correlations which result in biased feature learning. While existing CL strategies focus on a complete utilization of previous knowledge, we found that forgetting biased information is beneficial in our study. Therefore, we propose a new concept accurate forgetting (AF) and develop a novel generative-replay method~\method~which selectively utilizes previous knowledge in federated networks. We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge. Comprehensive experiments affirm the superiority of our method over baselines.
• 摘要：近年来，人们对联邦学习（FL）产生了兴趣。但是，客户参与顺序学习的上下文仍然不足。桥接FL和持续学习（CL）引起了一个具有挑战性的实际问题：联合持续学习（FCL）。 FCL的现有研究主要集中于减轻与其他客户合作时持续学习的灾难性遗忘问题。我们认为，遗忘现象并非总是有害。在本文中，我们考虑了一个更实用，更具挑战性的FCL设置，其特征是不同客户的潜在无关甚至对抗数据/任务。在FL情况下，客户之间的统计异质性和数据噪声可能会显示出虚假的相关性，从而导致特征学习。尽管现有的策略专注于以前知识的完全利用，但我们发现忘记偏见的信息对我们的研究是有益的。因此，我们提出了一个新的概念准确遗忘（AF），并开发了一种新颖的生成复制方法〜\方法〜，该方法有选择地利用联合网络中的先前知识。我们采用基于归一化流量模型的概率框架来量化先前知识的信誉。全面的实验肯定了我们方法比基线的优越性。

Title: Spatial and Frequency Domain Adaptive Fusion Network for Image Deblurring

• Authors: Hu Gao, Depeng Dang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.14209
• Pdf URL: https://arxiv.org/pdf/2502.14209
• Copy Paste: [[2502.14209]] Spatial and Frequency Domain Adaptive Fusion Network for Image Deblurring(https://arxiv.org/abs/2502.14209)
• Keywords: generation
• Abstract: Image deblurring aims to reconstruct a latent sharp image from its corresponding blurred one. Although existing methods have achieved good performance, most of them operate exclusively in either the spatial domain or the frequency domain, rarely exploring solutions that fuse both domains. In this paper, we propose a spatial-frequency domain adaptive fusion network (SFAFNet) to address this limitation. Specifically, we design a gated spatial-frequency domain feature fusion block (GSFFBlock), which consists of three key components: a spatial domain information module, a frequency domain information dynamic generation module (FDGM), and a gated fusion module (GFM). The spatial domain information module employs the NAFBlock to integrate local information. Meanwhile, in the FDGM, we design a learnable low-pass filter that dynamically decomposes features into separate frequency subbands, capturing the image-wide receptive field and enabling the adaptive exploration of global contextual information. Additionally, to facilitate information flow and the learning of complementary representations. In the GFM, we present a gating mechanism (GATE) to re-weight spatial and frequency domain features, which are then fused through the cross-attention mechanism (CAM). Experimental results demonstrate that our SFAFNet performs favorably compared to state-of-the-art approaches on commonly used benchmarks.
• 摘要：图像DeBlurring旨在重建其相应模糊的潜在尖锐图像。尽管现有方法已经达到了良好的性能，但其中大多数仅在空间域或频域中仅运行，很少探索融合两个域的解决方案。在本文中，我们提出了一个空间频率域自适应融合网络（SFAFNET）来解决此限制。具体而言，我们设计了一个封闭式的空间频率域特征融合块（GSFFBLOCK），该特征融合块由三个关键组件组成：空间域信息模块，频域信息信息动态生成模块（FDGM）和一个门控融合模块（GFM）。空间域信息模块采用Nafblock来集成本地信息。同时，在FDGM中，我们设计了一个可学习的低通滤波器，将功能动态分解为单独的频率子带，捕获整个图像的接收场并启用全局上下文信息的自适应探索。此外，以促进信息流和互补表示的学习。在GFM中，我们提出了一个登陆机制（门），以重新体重空间和频域特征，然后通过交叉注意机制（CAM）融合。实验结果表明，与常用基准的最新方法相比，我们的SFAFNET表现出色。

Title: Designing Parameter and Compute Efficient Diffusion Transformers using Distillation

• Authors: Vignesh Sundaresha
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.14226
• Pdf URL: https://arxiv.org/pdf/2502.14226
• Copy Paste: [[2502.14226]] Designing Parameter and Compute Efficient Diffusion Transformers using Distillation(https://arxiv.org/abs/2502.14226)
• Keywords: generation
• Abstract: Diffusion Transformers (DiTs) with billions of model parameters form the backbone of popular image and video generation models like DALL.E, Stable-Diffusion and SORA. Though these models are necessary in many low-latency applications like Augmented/Virtual Reality, they cannot be deployed on resource-constrained Edge devices (like Apple Vision Pro or Meta Ray-Ban glasses) due to their huge computational complexity. To overcome this, we turn to knowledge distillation and perform a thorough design-space exploration to achieve the best DiT for a given parameter size. In particular, we provide principles for how to choose design knobs such as depth, width, attention heads and distillation setup for a DiT. During the process, a three-way trade-off emerges between model performance, size and speed that is crucial for Edge implementation of diffusion. We also propose two distillation approaches - Teaching Assistant (TA) method and Multi-In-One (MI1) method - to perform feature distillation in the DiT context. Unlike existing solutions, we demonstrate and benchmark the efficacy of our approaches on practical Edge devices such as NVIDIA Jetson Orin Nano.
• 摘要：具有数十亿个模型参数的扩散变压器（DIT）形成了流行图像和视频生成模型（例如dall.e，稳定扩散和Sora）的骨干。尽管这些模型在许多低延迟的应用中是必需的，例如增强/虚拟现实，但由于其巨大的计算复杂性，它们无法在资源受限的边缘设备（例如Apple Vision Pro或Meta Ray-Ban眼镜）上部署。为了克服这一点，我们转向知识蒸馏并进行彻底的设计空间探索，以达到给定参数大小的最佳DIT。特别是，我们提供了如何选择设计旋钮的原理，例如深度，宽度，注意力头和DIT的蒸馏设置。在此过程中，模型性能，尺寸和速度之间出现了三向权衡，这对于扩散的边缘实施至关重要。我们还提出了两种蒸馏方法 - 助教方法（TA）方法和多合一方法（MI1）方法 - 在DIT上下文中执行特征蒸馏。与现有的解决方案不同，我们证明并基准了我们在Nvidia Jetson Orin Nano等实用边缘设备上的方法的功效。

Title: Correcting Noisy Multilabel Predictions: Modeling Label Noise through Latent Space Shifts

• Authors: Weipeng Huang, Qin Li, Yang Xiao, Cheng Qiao, Tie Cai, Junwei Liao, Neil J. Hurley, Guangyuan Piao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14281
• Pdf URL: https://arxiv.org/pdf/2502.14281
• Copy Paste: [[2502.14281]] Correcting Noisy Multilabel Predictions: Modeling Label Noise through Latent Space Shifts(https://arxiv.org/abs/2502.14281)
• Keywords: generative
• Abstract: Noise in data appears to be inevitable in most real-world machine learning applications and would cause severe overfitting problems. Not only can data features contain noise, but labels are also prone to be noisy due to human input. In this paper, rather than noisy label learning in multiclass classifications, we instead focus on the less explored area of noisy label learning for multilabel classifications. Specifically, we investigate the post-correction of predictions generated from classifiers learned with noisy labels. The reasons are two-fold. Firstly, this approach can directly work with the trained models to save computational resources. Secondly, it could be applied on top of other noisy label correction techniques to achieve further improvements. To handle this problem, we appeal to deep generative approaches that are possible for uncertainty estimation. Our model posits that label noise arises from a stochastic shift in the latent variable, providing a more robust and beneficial means for noisy learning. We develop both unsupervised and semi-supervised learning methods for our model. The extensive empirical study presents solid evidence to that our approach is able to consistently improve the independent models and performs better than a number of existing methods across various noisy label settings. Moreover, a comprehensive empirical analysis of the proposed method is carried out to validate its robustness, including sensitivity analysis and an ablation study, among other elements.
• 摘要：在大多数真实世界的机器学习应用中，数据中的噪声似乎是不可避免的，并且会导致严重的过度拟合问题。数据功能不仅可以包含噪声，而且由于人类输入，标签也容易出现嘈杂。在本文中，我们不是在多类分类中进行嘈杂的标签学习，而是专注于用于多标签分类的嘈杂标签学习的较少探索领域。具体而言，我们研究了用嘈杂标签学到的分类器产生的预测后的校正。原因是两个方面。首先，这种方法可以直接与受过训练的模型合作以节省计算资源。其次，可以将其应用于其他嘈杂的标签校正技术，以实现进一步的改进。为了解决这个问题，我们呼吁进行不确定性估计的深层生成方法。我们的模型认为，标签噪声源于潜在变量的随机变化，为嘈杂的学习提供了更强大和有益的手段。我们为模型开发了无监督和半监督的学习方法。广泛的实证研究为我们的方法提供了可靠的证据，表明我们的方法能够始终如一地改善独立模型，并且比各种嘈杂标签设置的许多现有方法更好。此外，对所提出的方法进行了全面的经验分析，以验证其鲁棒性，包括灵敏度分析和消融研究以及其他元素。

Title: Textured 3D Regenerative Morphing with 3D Diffusion Prior

• Authors: Songlin Yang, Yushi Lan, Honghua Chen, Xingang Pan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14316
• Pdf URL: https://arxiv.org/pdf/2502.14316
• Copy Paste: [[2502.14316]] Textured 3D Regenerative Morphing with 3D Diffusion Prior(https://arxiv.org/abs/2502.14316)
• Keywords: generative
• Abstract: Textured 3D morphing creates smooth and plausible interpolation sequences between two 3D objects, focusing on transitions in both shape and texture. This is important for creative applications like visual effects in filmmaking. Previous methods rely on establishing point-to-point correspondences and determining smooth deformation trajectories, which inherently restrict them to shape-only morphing on untextured, topologically aligned datasets. This restriction leads to labor-intensive preprocessing and poor generalization. To overcome these challenges, we propose a method for 3D regenerative morphing using a 3D diffusion prior. Unlike previous methods that depend on explicit correspondences and deformations, our method eliminates the additional need for obtaining correspondence and uses the 3D diffusion prior to generate morphing. Specifically, we introduce a 3D diffusion model and interpolate the source and target information at three levels: initial noise, model parameters, and condition features. We then explore an Attention Fusion strategy to generate more smooth morphing sequences. To further improve the plausibility of semantic interpolation and the generated 3D surfaces, we propose two strategies: (a) Token Reordering, where we match approximate tokens based on semantic analysis to guide implicit correspondences in the denoising process of the diffusion model, and (b) Low-Frequency Enhancement, where we enhance low-frequency signals in the tokens to improve the quality of generated surfaces. Experimental results show that our method achieves superior smoothness and plausibility in 3D morphing across diverse cross-category object pairs, offering a novel regenerative method for 3D morphing with textured representations.
• 摘要：纹理的3D变形在两个3D对象之间形成平滑而合理的插值序列，重点是形状和纹理的过渡。这对于诸如电影制作中的视觉效果之类的创意应用很重要。以前的方法依赖于建立点对点对应关系并确定平滑的变形轨迹，这些轨迹本质地将它们限制为在不介入的，拓扑上对齐的数据集上形状的变形。这种限制会导致劳动密集型的预处理和不良的概括。为了克服这些挑战，我们提出了一种使用3D扩散先验的3D再生变形的方法。与以前取决于显式对应关系和变形的方法不同，我们的方法消除了获得对应关系的额外需求，并在生成变形之前使用3D扩散。具体而言，我们引入了一个3D扩散模型，并在三个级别插入源和目标信息：初始噪声，模型参数和条件特征。然后，我们探索一种注意力融合策略，以产生更平滑的变形序列。为了进一步提高语义插值和生成的3D表面的合理性，我们提出了两种策略：（a）代币重新排序，我们基于语义分析匹配近似令牌，以指导扩散模型的降解过程中的隐式对应关系，以及（b ）低频增强，在其中增强令牌中的低频信号以提高产生的表面质量。实验结果表明，我们的方法在跨不同跨类别对象对的3D变形方面实现了卓越的平滑度和合理性，提供了一种新型的再生方法，用于3D变形，并具有纹理表示。

Title: PPO-MI: Efficient Black-Box Model Inversion via Proximal Policy Optimization

• Authors: Xinpeng Shou
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.14370
• Pdf URL: https://arxiv.org/pdf/2502.14370
• Copy Paste: [[2502.14370]] PPO-MI: Efficient Black-Box Model Inversion via Proximal Policy Optimization(https://arxiv.org/abs/2502.14370)
• Keywords: generative
• Abstract: Model inversion attacks pose a significant privacy risk by attempting to reconstruct private training data from trained models. Most of the existing methods either depend on gradient estimation or require white-box access to model parameters, which limits their applicability in practical scenarios. In this paper, we propose PPO-MI, a novel reinforcement learning-based framework for black-box model inversion attacks. Our approach formulates the inversion task as a Markov Decision Process, where an agent navigates the latent space of a generative model to reconstruct private training samples using only model predictions. By employing Proximal Policy Optimization (PPO) with a momentum-based state transition mechanism, along with a reward function balancing prediction accuracy and exploration, PPO-MI ensures efficient latent space exploration and high query efficiency. We conduct extensive experiments illustrates that PPO-MI outperforms the existing methods while require less attack knowledge, and it is robust across various model architectures and datasets. These results underline its effectiveness and generalizability in practical black-box scenarios, raising important considerations for the privacy vulnerabilities of deployed machine learning models.
• 摘要：模型反演攻击通过尝试从训练有素的模型中重建私人培训数据，从而构成了重大的隐私风险。大多数现有方法要么取决于梯度估计，要么需要对模型参数的访问，这在实际情况下限制了其适用性。在本文中，我们提出了PPO-MI，这是一种基于黑框模型反转攻击的新型增强学习框架。我们的方法将反演任务制定为马尔可夫决策过程，在该过程中，代理将生成模型的潜在空间导航，以仅使用模型预测来重建私人训练样本。通过采用基于动量的状态过渡机制的近端政策优化（PPO），以及奖励功能平衡预测准确性和探索，PPO-MI确保有效的潜在空间探索和高查询效率。我们进行广泛的实验表明，PPO-MI在需要更少的攻击知识的情况下优于现有方法，并且在各种模型体系结构和数据集中都有坚固耐用。这些结果强调了其在实用的黑盒情景中的有效性和可推广性，从而提高了针对已部署机器学习模型的隐私脆弱性的重要考虑因素。

Title: RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers

• Authors: Ke Cao, Jing Wang, Ao Ma, Jiasong Feng, Zhanjie Zhang, Xuanhua He, Shanyuan Liu, Bo Cheng, Dawei Leng, Yuhui Yin, Jie Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.14377
• Pdf URL: https://arxiv.org/pdf/2502.14377
• Copy Paste: [[2502.14377]] RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers(https://arxiv.org/abs/2502.14377)
• Keywords: generation
• Abstract: The Diffusion Transformer plays a pivotal role in advancing text-to-image and text-to-video generation, owing primarily to its inherent scalability. However, existing controlled diffusion transformer methods incur significant parameter and computational overheads and suffer from inefficient resource allocation due to their failure to account for the varying relevance of control information across different transformer layers. To address this, we propose the Relevance-Guided Efficient Controllable Generation framework, RelaCtrl, enabling efficient and resource-optimized integration of control signals into the Diffusion Transformer. First, we evaluate the relevance of each layer in the Diffusion Transformer to the control information by assessing the "ControlNet Relevance Score"-i.e., the impact of skipping each control layer on both the quality of generation and the control effectiveness during inference. Based on the strength of the relevance, we then tailor the positioning, parameter scale, and modeling capacity of the control layers to reduce unnecessary parameters and redundant computations. Additionally, to further improve efficiency, we replace the self-attention and FFN in the commonly used copy block with the carefully designed Two-Dimensional Shuffle Mixer (TDSM), enabling efficient implementation of both the token mixer and channel mixer. Both qualitative and quantitative experimental results demonstrate that our approach achieves superior performance with only 15% of the parameters and computational complexity compared to PixArt-delta. More examples are available at this https URL.
• 摘要：扩散变压器主要归因于其固有的可扩展性，在推进文本形象和文本对视频生成方面起着关键作用。但是，现有的受控扩散变压器方法产生了重要的参数和计算开销，并且由于其未能说明不同变压器层的控制信息的不同相关性而遭受效率低下的资源分配。为了解决这个问题，我们提出了相关性的有效可控生成框架，Relactrl，使控制信号的有效和资源优化的集成能够将控制信号集成到扩散变压器中。首先，我们通过评估“ ControlNet相关性得分” -I.E。评估扩散变压器中每一层与控制信息的相关性，跳过每个控制层对推理过程中发电质量和控制效果的影响。然后，我们根据相关性的强度，然后根据控制层的定位，参数量表和建模能力来减少不必要的参数和冗余计算。此外，为了进一步提高效率，我们用经过精心设计的二维洗牌搅拌机（TDSM）代替了常用的复制块中的自我注意事项和FFN，从而有效实现了令牌搅拌机和通道混音器。定性和定量实验结果都表明，与Pixart-Delta相比，我们的方法仅具有15％的参数和计算复杂性。此HTTPS URL提供了更多示例。

Title: S*: Test Time Scaling for Code Generation

• Authors: Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E. Gonzalez, Ion Stoica
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14382
• Pdf URL: https://arxiv.org/pdf/2502.14382
• Copy Paste: [[2502.14382]] S*: Test Time Scaling for Code Generation(https://arxiv.org/abs/2502.14382)
• Keywords: generation
• Abstract: Increasing test-time compute for LLMs shows promise across domains but remains underexplored in code generation, despite extensive study in math. In this paper, we propose S*, the first hybrid test-time scaling framework that substantially improves the coverage and selection accuracy of generated code. S* extends the existing parallel scaling paradigm with sequential scaling to push performance boundaries. It further leverages a novel selection mechanism that adaptively generates distinguishing inputs for pairwise comparison, combined with execution-grounded information to robustly identify correct solutions. We evaluate across 12 Large Language Models and Large Reasoning Model and show: (1) S* consistently improves performance across model families and sizes, enabling a 3B model to outperform GPT-4o-mini; (2) S* enables non-reasoning models to surpass reasoning models - GPT-4o-mini with S* outperforms o1-preview by 3.7% on LiveCodeBench; (3) S* further boosts state-of-the-art reasoning models - DeepSeek-R1-Distill-Qwen-32B with S* achieves 85.7% on LiveCodeBench, approaching o1 (high) at 88.5%. Code will be available under this https URL.
• 摘要：LLMS的测试时间计算的增加显示了跨域的希望，但尽管广泛研究了数学研究，但在代码生成中仍未充满信心。在本文中，我们提出了S*，这是第一个混合测试时间缩放框架，可大大提高生成代码的覆盖范围和选择精度。 S*通过顺序缩放扩展了现有的并行缩放范式以推动性能边界。它进一步利用了一种新型的选择机制，该机制可自适应地生成分别的成对比较的输入，并结合执行的信息以鲁棒性识别正确的解决方案。我们评估了12个大型语言模型和大型推理模型，并显示：（1）S*始终提高模型家族和大小的性能，从而使3B模型胜过GPT-4O-Mini； （2）S*使非争议模型能够超越推理模型-LiveCodeBench上的gpt-4o-mini均优于O1-preview的GPT-4O-Mini胜过3.7％； （3）S*进一步提高了最新推理模型-S* deepSeek-r1-distill-qwen-32b（s*）在livecodebench上实现了85.7％，以88.5％的速度接近O1（高）。代码将在此HTTPS URL下可用。

Title: Cardiac Evidence Backtracking for Eating Behavior Monitoring using Collocative Electrocardiogram Imagining

• Authors: Xu-Lu Zhang, Zhen-Qun Yang, Dong-Mei Jiang, Ga Liao, Qing Li, Ramesh Jain, Xiao-Yong Wei
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.14430
• Pdf URL: https://arxiv.org/pdf/2502.14430
• Copy Paste: [[2502.14430]] Cardiac Evidence Backtracking for Eating Behavior Monitoring using Collocative Electrocardiogram Imagining(https://arxiv.org/abs/2502.14430)
• Keywords: generation
• Abstract: Eating monitoring has remained an open challenge in medical research for years due to the lack of non-invasive sensors for continuous monitoring and the reliable methods for automatic behavior detection. In this paper, we present a pilot study using the wearable 24-hour ECG for sensing and tailoring the sophisticated deep learning for ad-hoc and interpretable detection. This is accomplished using a collocative learning framework in which 1) we construct collocative tensors as pseudo-images from 1D ECG signals to improve the feasibility of 2D image-based deep models; 2) we formulate the cardiac logic of analyzing the ECG data in a comparative way as periodic attention regulators so as to guide the deep inference to collect evidence in a human comprehensible manner; and 3) we improve the interpretability of the framework by enabling the backtracking of evidence with a set of methods designed for Class Activation Mapping (CAM) decoding and decision tree/forest generation. The effectiveness of the proposed framework has been validated on the largest ECG dataset of eating behavior with superior performance over conventional models, and its capacity of cardiac evidence mining has also been verified through the consistency of the evidence it backtracked and that of the previous medical studies.
• 摘要：由于缺乏连续监测的非侵入性传感器以及可靠的自动行为检测方法，饮食监测多年来一直是医学研究的挑战。在本文中，我们提出了一项使用可穿戴的24小时心电图的试点研究，以感测和调整精致的深度学习，以进行临时和可解释的检测。这是使用完善的学习框架来完成的，其中1）我们从1D ECG信号中构建了串联张量作为伪图像，以提高基于2D图像的深层模型的可行性； 2）我们以比较的方式为周期性注意调节剂来制定心脏逻辑，以指导以人类可理解的方式收集证据的深度推断； 3）我们通过通过一组专为类激活映射（CAM）解码和决策树/森林生成的方法进行证据的回溯来提高框架的解释性。拟议框架的有效性已在最大的ECG饮食行为数据集中得到验证，其性能优于常规模型，并且通过证据的一致性以及以前的医学研究的证据的一致性，也通过其心脏证据挖掘的能力进行了验证。 。

Title: Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generation

• Authors: Austin A. Barr, Robert Rozman, Eddie Guo
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.14523
• Pdf URL: https://arxiv.org/pdf/2502.14523
• Copy Paste: [[2502.14523]] Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generation(https://arxiv.org/abs/2502.14523)
• Keywords: generation, generative
• Abstract: We propose a new framework for zero-shot generation of synthetic tabular data. Using the large language model (LLM) GPT-4o and plain-language prompting, we demonstrate the ability to generate high-fidelity tabular data without task-specific fine-tuning or access to real-world data (RWD) for pre-training. To benchmark GPT-4o, we compared the fidelity and privacy of LLM-generated synthetic data against data generated with the conditional tabular generative adversarial network (CTGAN), across three open-access datasets: Iris, Fish Measurements, and Real Estate Valuation. Despite the zero-shot approach, GPT-4o outperformed CTGAN in preserving means, 95% confidence intervals, bivariate correlations, and data privacy of RWD, even at amplified sample sizes. Notably, correlations between parameters were consistently preserved with appropriate direction and strength. However, refinement is necessary to better retain distributional characteristics. These findings highlight the potential of LLMs in tabular data synthesis, offering an accessible alternative to generative adversarial networks and variational autoencoders.
• 摘要：我们提出了一个新的框架，用于零发出的合成表格数据。使用大型语言模型（LLM）GPT-4O和普通语言提示，我们证明了在没有特定任务的微调或访问现实世界数据（RWD）的情况下生成高保真表格数据的能力。为了基准GPT-4O，我们将LLM生成的合成数据的忠诚度和隐私与在三个开放式数据集中：IRIS，FISH测量和房地产估值相比，LLM生成的合成数据与条件表格生成对抗网络（CTGAN）产生的数据。尽管使用了零拍的方法，但GPT-4O的表现优于保存平均值，95％的置信区间，双变量相关性和RWD的数据隐私，即使在放大样本量也是如此。值得注意的是，参数之间的相关性始终以适当的方向和强度保存。但是，精炼对于更好地保留分布特征是必要的。这些发现突出了LLM在表格数据合成中的潜力，为生成对抗网络和变异自动编码器提供了可访问的替代方案。

Title: ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification

• Authors: Hyunseok Lee, Seunghyuk Oh, Jaehyung Kim, Jinwoo Shin, Jihoon Tack
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.14565
• Pdf URL: https://arxiv.org/pdf/2502.14565
• Copy Paste: [[2502.14565]] ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification(https://arxiv.org/abs/2502.14565)
• Keywords: generation
• Abstract: Self-awareness, i.e., the ability to assess and correct one's own generation, is a fundamental aspect of human intelligence, making its replication in large language models (LLMs) an important yet challenging task. Previous works tackle this by employing extensive reinforcement learning or rather relying on large external verifiers. In this work, we propose Refine via Intrinsic Self-Verification (ReVISE), an efficient and effective framework that enables LLMs to self-correct their outputs through self-verification. The core idea of ReVISE is to enable LLMs to verify their reasoning processes and continually rethink reasoning trajectories based on its verification. We introduce a structured curriculum based upon online preference learning to implement this efficiently. Specifically, as ReVISE involves two challenging tasks (i.e., self-verification and reasoning correction), we tackle each task sequentially using curriculum learning, collecting both failed and successful reasoning paths to construct preference pairs for efficient training. During inference, our approach enjoys natural test-time scaling by integrating self-verification and correction capabilities, further enhanced by our proposed confidence-aware decoding mechanism. Our experiments on various reasoning tasks demonstrate that ReVISE achieves efficient self-correction and significantly improves reasoning performance.
• 摘要：自我意识，即评估和纠正自己的一代的能力，是人类智力的基本方面，使其在大语言模型（LLMS）中复制成为重要但充满挑战的任务。以前的作品通过采用广泛的强化学习或依靠大型外部验证符来解决这一问题。在这项工作中，我们通过内在的自我验证（修订）提出完善，这是一个有效有效的框架，使LLMS能够通过自我验证自我纠正其输出。修订的核心思想是使LLMS能够根据其验证验证其推理过程，并不断重新考虑推理轨迹。我们基于在线偏好学习来有效地实施这项结构化课程。具体而言，由于修订涉及两个具有挑战性的任务（即自我验证和推理校正），因此我们使用课程学习顺序处理每个任务，收集失败和成功的推理路径，以构建优先培训，以进行有效的培训。在推断期间，我们的方法通过整合自我验证和校正能力来享受自然的测试时间缩放，并通过我们提出的意识到的解码机制进一步增强。我们对各种推理任务的实验表明，修改可以实现有效的自我纠正，并显着提高了推理绩效。

Title: A Theory for Conditional Generative Modeling on Multiple Data Sources

• Authors: Rongzhen Wang, Yan Zhang, Chenyu Zheng, Chongxuan Li, Guoqiang Wu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14583
• Pdf URL: https://arxiv.org/pdf/2502.14583
• Copy Paste: [[2502.14583]] A Theory for Conditional Generative Modeling on Multiple Data Sources(https://arxiv.org/abs/2502.14583)
• Keywords: generative
• Abstract: The success of large generative models has driven a paradigm shift, leveraging massive multi-source data to enhance model capabilities. However, the interaction among these sources remains theoretically underexplored. This paper takes the first step toward a rigorous analysis of multi-source training in conditional generative modeling, where each condition represents a distinct data source. Specifically, we establish a general distribution estimation error bound in average total variation distance for conditional maximum likelihood estimation based on the bracketing number. Our result shows that when source distributions share certain similarities and the model is expressive enough, multi-source training guarantees a sharper bound than single-source training. We further instantiate the general theory on conditional Gaussian estimation and deep generative models including autoregressive and flexible energy-based models, by characterizing their bracketing numbers. The results highlight that the number of sources and similarity among source distributions improve the advantage of multi-source training. Simulations and real-world experiments validate our theory. Code is available at: \url{this https URL}.
• 摘要：大型生成模型的成功驱动了范式转移，利用大量的多源数据来增强模型功能。但是，这些来源之间的相互作用在理论上仍然没有被忽略。本文迈出了对条件生成建模中多源训练的严格分析的第一步，其中每个条件代表不同的数据源。具体而言，我们建立一个基于括号数的有条件最大似然估计的一般分布估计误差的平均总变化距离。我们的结果表明，当源分布共享某些相似之处并且该模型表现力足够时，多源培训可以保证比单源培训更清晰。我们通过表征其括号数字来进一步实例化有关条件高斯估计和深层生成模型，包括自回归和灵活的基于能量的模型的一般理论。结果表明，来源分布之间的来源数量和相似性提高了多源培训的优势。模拟和现实实验验证了我们的理论。代码可在：\ url {this HTTPS url}中获得。

Title: CER: Confidence Enhanced Reasoning in LLMs

• Authors: Ali Razghandi, Seyed Mohammad Hadi Hosseini, Mahdieh Soleymani Baghshah
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.14634
• Pdf URL: https://arxiv.org/pdf/2502.14634
• Copy Paste: [[2502.14634]] CER: Confidence Enhanced Reasoning in LLMs(https://arxiv.org/abs/2502.14634)
• Keywords: generation
• Abstract: Ensuring the reliability of Large Language Models (LLMs) in complex reasoning tasks remains a formidable challenge, particularly in scenarios that demand precise mathematical calculations and knowledge-intensive open-domain generation. In this work, we introduce an uncertainty-aware framework designed to enhance the accuracy of LLM responses by systematically incorporating model confidence at critical decision points. We propose an approach that encourages multi-step reasoning in LLMs and quantify the confidence of intermediate answers such as numerical results in mathematical reasoning and proper nouns in open-domain generation. Then, the overall confidence of each reasoning chain is evaluated based on confidence of these critical intermediate steps. Finally, we aggregate the answer of generated response paths in a way that reflects the reliability of each generated content (as opposed to self-consistency in which each generated chain contributes equally to majority voting). We conducted extensive experiments in five datasets, three mathematical datasets and two open-domain datasets, using four LLMs. The results consistently validate the effectiveness of our novel confidence aggregation method, leading to an accuracy improvement of up to 7.4% and 5.8% over baseline approaches in math and open-domain generation tasks, respectively. Code is publicly available at this https URL Aquasar11/CER.
• 摘要：在复杂的推理任务中确保大语言模型（LLM）的可靠性仍然是一个巨大的挑战，尤其是在需要精确的数学计算和知识密集的开放域产生的情况下。在这项工作中，我们引入了一个不确定性感知的框架，旨在通过系统地将模型置信度在关键决策点上加以提高LLM响应的准确性。我们提出了一种鼓励LLM中多步推理的方法，并量化了中间答案的信心，例如数学推理中的数值结果和开放域中的适当名词。然后，根据这些关键中间步骤的置信度评估了每个推理链的总体置信度。最后，我们以反映每个生成内容的可靠性的方式汇总了生成的响应路径的答案（与每个生成链对多数投票的自洽相反）。我们使用四个LLMS在五个数据集，三个数学数据集和两个开放域数据集中进行了广泛的实验。结果始终验证了我们新型置信度聚集方法的有效性，从而使数学和开放域生成任务的准确性分别比基线方法的准确性提高了7.4％和5.8％。代码可在此HTTPS URL Aquasar11/CER上公开获得。

Title: ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation

• Authors: Angxiao Yue, Zichong Wang, Hongteng Xu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14637
• Pdf URL: https://arxiv.org/pdf/2502.14637
• Copy Paste: [[2502.14637]] ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation(https://arxiv.org/abs/2502.14637)
• Keywords: generation, generative
• Abstract: Protein backbone generation plays a central role in de novo protein design and is significant for many biological and medical applications. Although diffusion and flow-based generative models provide potential solutions to this challenging task, they often generate proteins with undesired designability and suffer computational inefficiency. In this study, we propose a novel rectified quaternion flow (ReQFlow) matching method for fast and high-quality protein backbone generation. In particular, our method generates a local translation and a 3D rotation from random noise for each residue in a protein chain, which represents each 3D rotation as a unit quaternion and constructs its flow by spherical linear interpolation (SLERP) in an exponential format. We train the model by quaternion flow (QFlow) matching with guaranteed numerical stability and rectify the QFlow model to accelerate its inference and improve the designability of generated protein backbones, leading to the proposed ReQFlow model. Experiments show that ReQFlow achieves state-of-the-art performance in protein backbone generation while requiring much fewer sampling steps and significantly less inference time (e.g., being 37x faster than RFDiffusion and 62x faster than Genie2 when generating a backbone of length 300), demonstrating its effectiveness and efficiency. The code is available at this https URL.
• 摘要：蛋白质主链产生在从头蛋白质设计中起着核心作用，对于许多生物学和医学应用而言是重要的。尽管扩散和基于流的生成模型为这项具有挑战性的任务提供了潜在的解决方案，但它们通常会产生具有不希望的可设计性并遭受计算效率低下的蛋白质。在这项研究中，我们提出了一种新型的校正四元流（REQFLOW）匹配方法，用于快速和高质量的蛋白质主链产生。特别是，我们的方法生成了局部翻译和蛋白质链中每个残基的随机噪声的3D旋转，该蛋白链中的每个残基代表每个3D旋转作为单位季节，并通过球形线性插值（SLERP）以指数形式构造其流动。我们通过与保证的数值稳定性匹配的四元流（QFlow）匹配训练模型，并纠正QFlow模型以加速其推断并提高生成的蛋白质骨架的可设计性，从而导致提出的Reqflow模型。实验表明，Reqflow在蛋白质主链产生中实现最新性能，同时需要更少的采样步骤和明显少得多的推理时间（例如，比RfDiffusion的37倍速度比RfDiffusion快37倍，并且比Genie2快62x时产生长度为300的骨干时），证明其有效性和效率。该代码可在此HTTPS URL上找到。

Title: DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models

• Authors: Hongji Yang, Wencheng Han, Yucheng Zhou, Jianbing Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.14779
• Pdf URL: https://arxiv.org/pdf/2502.14779
• Copy Paste: [[2502.14779]] DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models(https://arxiv.org/abs/2502.14779)
• Keywords: generation, generative
• Abstract: In this paper, we introduce DC (Decouple)-ControlNet, a highly flexible and precisely controllable framework for multi-condition image generation. The core idea behind DC-ControlNet is to decouple control conditions, transforming global control into a hierarchical system that integrates distinct elements, contents, and layouts. This enables users to mix these individual conditions with greater flexibility, leading to more efficient and accurate image generation control. Previous ControlNet-based models rely solely on global conditions, which affect the entire image and lack the ability of element- or region-specific control. This limitation reduces flexibility and can cause condition misunderstandings in multi-conditional image generation. To address these challenges, we propose both intra-element and Inter-element Controllers in DC-ControlNet. The Intra-Element Controller handles different types of control signals within individual elements, accurately describing the content and layout characteristics of the object. For interactions between elements, we introduce the Inter-Element Controller, which accurately handles multi-element interactions and occlusion based on user-defined relationships. Extensive evaluations show that DC-ControlNet significantly outperforms existing ControlNet models and Layout-to-Image generative models in terms of control flexibility and precision in multi-condition control.
• 摘要：在本文中，我们介绍了DC（Decouple）-Controlnet，这是一个高度灵活且精确控制的多条件图像生成框架。 DC-Controlnet背后的核心思想是将控制条件解散，将全局控制转换为一个集成了不同元素，内容和布局的分层系统。这使用户可以将这些个人条件与更大的灵活性相结合，从而导致更有效，准确的图像生成控制。以前的基于控制网络的模型仅依赖于全局条件，这些条件会影响整个图像，并且缺乏特定于元素或区域特异性控制的能力。这种限制可降低灵活性，并可能导致多条件图像生成中的误解。为了应对这些挑战，我们提出了DC-Controlnet中的元素内和元素间控制器。元素内控制器在单个元素中处理不同类型的控制信号，准确地描述了对象的内容和布局特征。对于元素之间的交互，我们介绍了元素间控制器，该控制器可以准确处理基于用户定义的关系的多元素交互和遮挡。广泛的评估表明，DC-Controlnet在多条件控制中的控制灵活性和精度方面显着优于现有控制网络模型和布局到图像生成模型。

Title: A Survey on Text-Driven 360-Degree Panorama Generation

• Authors: Hai Wang, Xiaoyu Xiang, Weihao Xia, Jing-Hao Xue
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14799
• Pdf URL: https://arxiv.org/pdf/2502.14799
• Copy Paste: [[2502.14799]] A Survey on Text-Driven 360-Degree Panorama Generation(https://arxiv.org/abs/2502.14799)
• Keywords: generation
• Abstract: The advent of text-driven 360-degree panorama generation, enabling the synthesis of 360-degree panoramic images directly from textual descriptions, marks a transformative advancement in immersive visual content creation. This innovation significantly simplifies the traditionally complex process of producing such content. Recent progress in text-to-image diffusion models has accelerated the rapid development in this emerging field. This survey presents a comprehensive review of text-driven 360-degree panorama generation, offering an in-depth analysis of state-of-the-art algorithms and their expanding applications in 360-degree 3D scene generation. Furthermore, we critically examine current limitations and propose promising directions for future research. A curated project page with relevant resources and research papers is available at this https URL.
• 摘要：文本驱动的360度全景生成的出现，直接从文本描述中直接从360度全景图中综合，标志着沉浸式视觉内容创建的变革性进步。这项创新大大简化了传统上生产此类内容的复杂过程。文本到图像扩散模型的最新进展加速了这一新兴领域的快速发展。这项调查对文本驱动的360度全景一代进行了全面评论，对最先进的算法进行了深入的分析及其在360度3D场景生成中的扩展应用程序。此外，我们批判性地检查了当前的局限性，并为未来的研究提出了有希望的方向。此HTTPS URL提供了带有相关资源和研究论文的策划项目页面。

Title: Improving the Diffusability of Autoencoders

• Authors: Ivan Skorokhodov, Sharath Girish, Benran Hu, Willi Menapace, Yanyu Li, Rameen Abdal, Sergey Tulyakov, Aliaksandr Siarohin
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.14831
• Pdf URL: https://arxiv.org/pdf/2502.14831
• Copy Paste: [[2502.14831]] Improving the Diffusability of Autoencoders(https://arxiv.org/abs/2502.14831)
• Keywords: generation
• Abstract: Latent diffusion models have emerged as the leading approach for generating high-quality images and videos, utilizing compressed latent representations to reduce the computational burden of the diffusion process. While recent advancements have primarily focused on scaling diffusion backbones and improving autoencoder reconstruction quality, the interaction between these components has received comparatively less attention. In this work, we perform a spectral analysis of modern autoencoders and identify inordinate high-frequency components in their latent spaces, which are especially pronounced in the autoencoders with a large bottleneck channel size. We hypothesize that this high-frequency component interferes with the coarse-to-fine nature of the diffusion synthesis process and hinders the generation quality. To mitigate the issue, we propose scale equivariance: a simple regularization strategy that aligns latent and RGB spaces across frequencies by enforcing scale equivariance in the decoder. It requires minimal code changes and only up to 20K autoencoder fine-tuning steps, yet significantly improves generation quality, reducing FID by 19% for image generation on ImageNet-1K 256x256 and FVD by at least 44% for video generation on Kinetics-700 17x256x256.
• 摘要：潜在扩散模型已成为生成高质量图像和视频的领先方法，利用压缩潜在表示减轻了扩散过程的计算负担。尽管最近的进步主要集中在扩散骨架和提高自动编码器重建质量上，但这些组件之间的相互作用受到相对较少的关注。在这项工作中，我们对现代自动编码器进行频谱分析，并在其潜在空间中识别出非频率的组件，这些高频组件在具有较大瓶颈通道大小的自动编码器中尤为明显。我们假设这种高频成分会干扰扩散合成过程的粗到细性，并阻碍了生成质量。为了减轻问题，我们提出了量表等效性：一种简单的正则化策略，通过在解码器中执行量表等值范围来使频率的潜在和RGB空间保持一致。它需要最小的代码更改，最多只需20K自动编码器微调步骤，但显着提高了发电质量，在Imagenet-1k 256x256上生成图像生成的FID降低了19％，而FVD的图像生成至少为44％，以便在Kinetics-700 17X256X2256上为视频生成。 。

Title: Probabilistic Robustness in Deep Learning: A Concise yet Comprehensive Guide

• Authors: Xingyu Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.14833
• Pdf URL: https://arxiv.org/pdf/2502.14833
• Copy Paste: [[2502.14833]] Probabilistic Robustness in Deep Learning: A Concise yet Comprehensive Guide(https://arxiv.org/abs/2502.14833)
• Keywords: generative
• Abstract: Deep learning (DL) has demonstrated significant potential across various safety-critical applications, yet ensuring its robustness remains a key challenge. While adversarial robustness has been extensively studied in worst-case scenarios, probabilistic robustness (PR) offers a more practical perspective by quantifying the likelihood of failures under stochastic perturbations. This paper provides a concise yet comprehensive overview of PR, covering its formal definitions, evaluation and enhancement methods. We introduce a reformulated ''min-max'' optimisation framework for adversarial training specifically designed to improve PR. Furthermore, we explore the integration of PR verification evidence into system-level safety assurance, addressing challenges in translating DL model-level robustness to system-level claims. Finally, we highlight open research questions, including benchmarking PR evaluation methods, extending PR to generative AI tasks, and developing rigorous methodologies and case studies for system-level integration.
• 摘要：深度学习（DL）在各种安全至关重要的应用中都表现出了巨大的潜力，但确保其鲁棒性仍然是一个关键的挑战。尽管对抗性的鲁棒性在最坏情况的情况下进行了广泛的研究，但概率鲁棒性（PR）通过量化随机扰动下的失败可能性来提供更实际的视角。本文提供了PR的简洁而全面的概述，涵盖了其正式定义，评估和增强方法。我们为针对改善PR的对抗性训练提供了一个重新制定的“ Min-Max”优化框架。此外，我们探讨了将PR验证证据集成到系统级安全保证中的集成，从而解决了将DL模型级鲁棒性转化为系统级别主张的挑战。最后，我们重点介绍了开放研究问题，包括基准测试PR评估方法，将PR扩展到生成的AI任务，并开发严格的方法论和案例研究以进行系统级集成。

Title: LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models

• Authors: Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, Juanzi Li
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.14834
• Pdf URL: https://arxiv.org/pdf/2502.14834
• Copy Paste: [[2502.14834]] LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models(https://arxiv.org/abs/2502.14834)
• Keywords: generation
• Abstract: Existing Large Vision-Language Models (LVLMs) can process inputs with context lengths up to 128k visual and text tokens, yet they struggle to generate coherent outputs beyond 1,000 words. We find that the primary limitation is the absence of long output examples during supervised fine-tuning (SFT). To tackle this issue, we introduce LongWriter-V-22k, a SFT dataset comprising 22,158 examples, each with multiple input images, an instruction, and corresponding outputs ranging from 0 to 10,000 words. Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model. Given the high cost of collecting human feedback for lengthy outputs (e.g., 3,000 words), we propose IterDPO, which breaks long outputs into segments and uses iterative corrections to form preference pairs with the original outputs. Additionally, we develop MMLongBench-Write, a benchmark featuring six tasks to evaluate the long-generation capabilities of VLMs. Our 7B parameter model, trained with LongWriter-V-22k and IterDPO, achieves impressive performance on this benchmark, outperforming larger proprietary models like GPT-4o. Code and data: this https URL
• 摘要：现有的大型视觉模型（LVLM）可以处理上下文长度最高128K的视觉和文本令牌的输入，但它们很难生成超过1,000个单词的相干输出。我们发现，主要限制是监督微调（SFT）期间没有长输出示例。为了解决此问题，我们介绍了Longwriter-V-22K，这是一个包含22,158个示例的SFT数据集，每个示例都有多个输入图像，指令和相应的输出，范围为0到10,000个单词。此外，为了实现对输入图像的高保真性的长输出，我们将直接的偏好优化（DPO）用于SFT模型。鉴于收集人类反馈的冗长输出（例如3,000个单词）的高成本，我们提出了ITERDPO，该输出将很长的输出分为段，并使用迭代校正来形成与原始输出的优先级对。此外，我们开发了Mmlongbench-Write，这是一个基准，该基准具有六个任务，以评估VLM的长期能力。我们的7B参数模型，经过长作者-V-22K和ITERDPO训练，在此基准上取得了令人印象深刻的性能，表现优于GPT-4O（例如GPT-4O）。代码和数据：此HTTPS URL

Title: Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation

• Authors: Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.14846
• Pdf URL: https://arxiv.org/pdf/2502.14846
• Copy Paste: [[2502.14846]] Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation(https://arxiv.org/abs/2502.14846)
• Keywords: generation
• Abstract: Reasoning about images with rich text, such as charts and documents, is a critical application of vision-language models (VLMs). However, VLMs often struggle in these domains due to the scarcity of diverse text-rich vision-language data. To address this challenge, we present CoSyn, a framework that leverages the coding capabilities of text-only large language models (LLMs) to automatically create synthetic text-rich multimodal data. Given input text describing a target domain (e.g., "nutrition fact labels"), CoSyn prompts an LLM to generate code (Python, HTML, LaTeX, etc.) for rendering synthetic images. With the underlying code as textual representations of the synthetic images, CoSyn can generate high-quality instruction-tuning data, again relying on a text-only LLM. Using CoSyn, we constructed a dataset comprising 400K images and 2.7M rows of vision-language instruction-tuning data. Comprehensive experiments on seven benchmarks demonstrate that models trained on our synthetic data achieve state-of-the-art performance among competitive open-source models, including Llama 3.2, and surpass proprietary models such as GPT-4V and Gemini 1.5 Flash. Furthermore, CoSyn can produce synthetic pointing data, enabling VLMs to ground information within input images, showcasing its potential for developing multimodal agents capable of acting in real-world environments.
• 摘要：关于图像具有丰富文本的推理，例如图表和文档，是视觉模型（VLM）的关键应用。但是，由于多种文本丰富的视觉语言数据的稀缺性，VLM经常在这些领域中挣扎。为了应对这一挑战，我们提出了Cosyn，该框架利用了仅文本大型语言模型（LLMS）的编码功能来自动创建合成文本丰富的多模式数据。给定的输入文本描述了目标域（例如“营养事实标签”），COSYN提示LLM生成代码（Python，HTML，乳胶等），以渲染合成图像。以基础代码为合成图像的文本表示形式，Cosyn可以生成高质量的指令数据，再次依靠仅使用文本LLM。使用COSYN，我们构建了一个包含400K图像和270万视觉语言指令数据的数据集。对七个基准测试的综合实验表明，经过合成数据训练的模型在竞争性开源模型中实现了最先进的性能，包括Llama 3.2，以及超越专有模型，例如GPT-4V和Gemini 1.5 1.5 Flash。此外，Cosyn可以产生合成指向数据，从而使VLMS能够在输入图像中接地信息，从而展示了其开发能够在现实世界环境中起作用的多模式剂的潜力。