2025-02-07

Title: Teaching Language Models to Critique via Reinforcement Learning

• Authors: Zhihui Xie, Jie chen, Liyu Chen, Weichao Mao, Jingjing Xu, Lingpeng Kong
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.03492
• Pdf URL: https://arxiv.org/pdf/2502.03492
• Copy Paste: [[2502.03492]] Teaching Language Models to Critique via Reinforcement Learning(https://arxiv.org/abs/2502.03492)
• Keywords: generation, generative
• Abstract: Teaching large language models (LLMs) to critique and refine their outputs is crucial for building systems that can iteratively improve, yet it is fundamentally limited by the ability to provide accurate judgments and actionable suggestions. In this work, we study LLM critics for code generation and propose $\texttt{CTRL}$, a framework for $\texttt{C}$ritic $\texttt{T}$raining via $\texttt{R}$einforcement $\texttt{L}$earning, which trains a critic model to generate feedback that maximizes correction performance for a fixed generator model without human supervision. Our results demonstrate that critics trained with $\texttt{CTRL}$ significantly enhance pass rates and mitigate compounding errors across both base and stronger generator models. Furthermore, we show that these critic models act as accurate generative reward models and enable test-time scaling through iterative critique-revision, achieving up to 106.1% relative improvements across challenging code generation benchmarks.
• 摘要：

Title: Path Planning for Masked Diffusion Model Sampling

• Authors: Fred Zhangzhi Peng, Zachary Bezemek, Sawan Patel, Sherwood Yao, Jarrid Rector-Brooks, Alexander Tong, Pranam Chatterjee
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.03540
• Pdf URL: https://arxiv.org/pdf/2502.03540
• Copy Paste: [[2502.03540]] Path Planning for Masked Diffusion Model Sampling(https://arxiv.org/abs/2502.03540)
• Keywords: generation, generative
• Abstract: In this paper, we investigate how the order in which tokens are unmasked during masked diffusion models (MDMs) inference affects generative quality. We derive an expanded evidence lower bound (ELBO) that introduces a planner, responsible for selecting which tokens to unmask at each step. Our analysis suggests that alternative unmasking strategies can improve generative performance. Based on these insights, we propose Path Planning (P2), a sampling framework that leverages pre-trained BERT or the denoiser itself to guide unmasking decisions. P2 generalizes all known MDM sampling strategies and enables significant improvements across diverse domains including language generation (in-context learning, code generation, story infilling, mathematical reasoning, reverse curse correction) and biological sequence generation (protein and RNA sequences).
• 摘要：

Title: TD-M(PC)$^2$: Improving Temporal Difference MPC Through Policy Constraint

• Authors: Haotian Lin, Pengcheng Wang, Jeff Schneider, Guanya Shi
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.03550
• Pdf URL: https://arxiv.org/pdf/2502.03550
• Copy Paste: [[2502.03550]] TD-M(PC)$^2$: Improving Temporal Difference MPC Through Policy Constraint(https://arxiv.org/abs/2502.03550)
• Keywords: generation
• Abstract: Model-based reinforcement learning algorithms that combine model-based planning and learned value/policy prior have gained significant recognition for their high data efficiency and superior performance in continuous control. However, we discover that existing methods that rely on standard SAC-style policy iteration for value learning, directly using data generated by the planner, often result in \emph{persistent value overestimation}. Through theoretical analysis and experiments, we argue that this issue is deeply rooted in the structural policy mismatch between the data generation policy that is always bootstrapped by the planner and the learned policy prior. To mitigate such a mismatch in a minimalist way, we propose a policy regularization term reducing out-of-distribution (OOD) queries, thereby improving value learning. Our method involves minimum changes on top of existing frameworks and requires no additional computation. Extensive experiments demonstrate that the proposed approach improves performance over baselines such as TD-MPC2 by large margins, particularly in 61-DoF humanoid tasks. View qualitative results at this https URL.
• 摘要：

Title: Controllable Sequence Editing for Counterfactual Generation

• Authors: Michelle M. Li, Kevin Li, Yasha Ektefaie, Shvat Messica, Marinka Zitnik
• Subjects: cs.LG, q-bio.GN, q-bio.PE
• Abstract URL: https://arxiv.org/abs/2502.03569
• Pdf URL: https://arxiv.org/pdf/2502.03569
• Copy Paste: [[2502.03569]] Controllable Sequence Editing for Counterfactual Generation(https://arxiv.org/abs/2502.03569)
• Keywords: generation
• Abstract: Sequence models generate counterfactuals by modifying parts of a sequence based on a given condition, enabling reasoning about "what if" scenarios. While these models excel at conditional generation, they lack fine-grained control over when and where edits occur. Existing approaches either focus on univariate sequences or assume that interventions affect the entire sequence globally. However, many applications require precise, localized modifications, where interventions take effect only after a specified time and impact only a subset of co-occurring variables. We introduce CLEF, a controllable sequence editing model for counterfactual reasoning about both immediate and delayed effects. CLEF learns temporal concepts that encode how and when interventions should influence a sequence. With these concepts, CLEF selectively edits relevant time steps while preserving unaffected portions of the sequence. We evaluate CLEF on cellular and patient trajectory datasets, where gene regulation affects only certain genes at specific time steps, or medical interventions alter only a subset of lab measurements. CLEF improves immediate sequence editing by up to 36.01% in MAE compared to baselines. Unlike prior methods, CLEF enables one-step generation of counterfactual sequences at any future time step, outperforming baselines by up to 65.71% in MAE. A case study on patients with type 1 diabetes mellitus shows that CLEF identifies clinical interventions that shift patient trajectories toward healthier outcomes.
• 摘要：

Title: The Logical Implication Steering Method for Conditional Interventions on Transformer Generation

• Authors: Damjan Kalajdzievski
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03618
• Pdf URL: https://arxiv.org/pdf/2502.03618
• Copy Paste: [[2502.03618]] The Logical Implication Steering Method for Conditional Interventions on Transformer Generation(https://arxiv.org/abs/2502.03618)
• Keywords: generation
• Abstract: The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the idea that high level concepts are encoded as vectors in the space of activations of a model. Studies also show that model generation behavior can be steered toward a given concept by adding the concept's vector to the corresponding activations. We show how to leverage these properties to build a form of logical implication into models, enabling transparent and interpretable adjustments that induce a chosen generation behavior in response to the presence of any given concept. Our method, Logical Implication Model Steering (LIMS), unlocks new hand engineered reasoning capabilities by integrating neuro-symbolic logic into pre-trained transformer models.
• 摘要：

Title: The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering

• Authors: Zhuowei Li, Haizhou Shi, Yunhe Gao, Di Liu, Zhenting Wang, Yuxiao Chen, Ting Liu, Long Zhao, Hao Wang, Dimitris N. Metaxas
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03628
• Pdf URL: https://arxiv.org/pdf/2502.03628
• Copy Paste: [[2502.03628]] The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering(https://arxiv.org/abs/2502.03628)
• Keywords: generation
• Abstract: Large Vision-Language Models (LVLMs) can reason effectively over both textual and visual inputs, but they tend to hallucinate syntactically coherent yet visually ungrounded contents. In this paper, we investigate the internal dynamics of hallucination by examining the tokens logits rankings throughout the generation process, revealing three key patterns in how LVLMs process information: (1) gradual visual information loss -- visually grounded tokens gradually become less favored throughout generation, and (2) early excitation -- semantically meaningful tokens achieve peak activation in the layers earlier than the final layer. (3) hidden genuine information -- visually grounded tokens though not being eventually decided still retain relatively high rankings at inference. Based on these insights, we propose VISTA (Visual Information Steering with Token-logit Augmentation), a training-free inference-time intervention framework that reduces hallucination while promoting genuine information. VISTA works by combining two complementary approaches: reinforcing visual information in activation space and leveraging early layer activations to promote semantically meaningful decoding. Compared to existing methods, VISTA requires no external supervision and is applicable to various decoding strategies. Extensive experiments show that VISTA on average reduces hallucination by abount 40% on evaluated open-ended generation task, and it consistently outperforms existing methods on four benchmarks across four architectures under three decoding strategies.
• 摘要：

Title: Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach

• Authors: Yunuo Chen, Junli Cao, Anil Kag, Vidit Goel, Sergei Korolev, Chenfanfu Jiang, Sergey Tulyakov, Jian Ren
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03639
• Pdf URL: https://arxiv.org/pdf/2502.03639
• Copy Paste: [[2502.03639]] Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach(https://arxiv.org/abs/2502.03639)
• Keywords: generation
• Abstract: We present a novel video generation framework that integrates 3-dimensional geometry and dynamic awareness. To achieve this, we augment 2D videos with 3D point trajectories and align them in pixel space. The resulting 3D-aware video dataset, PointVid, is then used to fine-tune a latent diffusion model, enabling it to track 2D objects with 3D Cartesian coordinates. Building on this, we regularize the shape and motion of objects in the video to eliminate undesired artifacts, \eg, nonphysical deformation. Consequently, we enhance the quality of generated RGB videos and alleviate common issues like object morphing, which are prevalent in current video models due to a lack of shape awareness. With our 3D augmentation and regularization, our model is capable of handling contact-rich scenarios such as task-oriented videos. These videos involve complex interactions of solids, where 3D information is essential for perceiving deformation and contact. Furthermore, our model improves the overall quality of video generation by promoting the 3D consistency of moving objects and reducing abrupt changes in shape and motion.
• 摘要：

Title: All-in-One Image Compression and Restoration

• Authors: Huimin Zeng, Jiacheng Li, Ziqiang Zheng, Zhiwei Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03649
• Pdf URL: https://arxiv.org/pdf/2502.03649
• Copy Paste: [[2502.03649]] All-in-One Image Compression and Restoration(https://arxiv.org/abs/2502.03649)
• Keywords: restoration
• Abstract: Visual images corrupted by various types and levels of degradations are commonly encountered in practical image compression. However, most existing image compression methods are tailored for clean images, therefore struggling to achieve satisfying results on these images. Joint compression and restoration methods typically focus on a single type of degradation and fail to address a variety of degradations in practice. To this end, we propose a unified framework for all-in-one image compression and restoration, which incorporates the image restoration capability against various degradations into the process of image compression. The key challenges involve distinguishing authentic image content from degradations, and flexibly eliminating various degradations without prior knowledge. Specifically, the proposed framework approaches these challenges from two perspectives: i.e., content information aggregation, and degradation representation aggregation. Extensive experiments demonstrate the following merits of our model: 1) superior rate-distortion (RD) performance on various degraded inputs while preserving the performance on clean data; 2) strong generalization ability to real-world and unseen scenarios; 3) higher computing efficiency over compared methods. Our code is available at this https URL.
• 摘要：

Title: A Study in Dataset Distillation for Image Super-Resolution

• Authors: Tobias Dietz, Brian B. Moser, Tobias Nauen, Federico Raue, Stanislav Frolov, Andreas Dengel
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03656
• Pdf URL: https://arxiv.org/pdf/2502.03656
• Copy Paste: [[2502.03656]] A Study in Dataset Distillation for Image Super-Resolution(https://arxiv.org/abs/2502.03656)
• Keywords: super-resolution
• Abstract: Dataset distillation is the concept of condensing large datasets into smaller but highly representative synthetic samples. While previous research has primarily focused on image classification, its application to image Super-Resolution (SR) remains underexplored. This exploratory work studies multiple dataset distillation techniques applied to SR, including pixel- and latent-space approaches under different aspects. Our experiments demonstrate that a 91.12% dataset size reduction can be achieved while maintaining comparable SR performance to the full dataset. We further analyze initialization strategies and distillation methods to optimize memory efficiency and computational costs. Our findings provide new insights into dataset distillation for SR and set the stage for future advancements.
• 摘要：

Title: Privacy-Preserving Generative Models: A Comprehensive Survey

• Authors: Debalina Padariya, Isabel Wagner, Aboozar Taherkhani, Eerke Boiten
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.03668
• Pdf URL: https://arxiv.org/pdf/2502.03668
• Copy Paste: [[2502.03668]] Privacy-Preserving Generative Models: A Comprehensive Survey(https://arxiv.org/abs/2502.03668)
• Keywords: generative
• Abstract: Despite the generative model's groundbreaking success, the need to study its implications for privacy and utility becomes more urgent. Although many studies have demonstrated the privacy threats brought by GANs, no existing survey has systematically categorized the privacy and utility perspectives of GANs and VAEs. In this article, we comprehensively study privacy-preserving generative models, articulating the novel taxonomies for both privacy and utility metrics by analyzing 100 research publications. Finally, we discuss the current challenges and future research directions that help new researchers gain insight into the underlying concepts.
• 摘要：

Title: Unrealized Expectations: Comparing AI Methods vs Classical Algorithms for Maximum Independent Set

• Authors: Yikai Wu, Haoyu Zhao, Sanjeev Arora
• Subjects: cs.LG, cs.AI, cs.DM, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.03669
• Pdf URL: https://arxiv.org/pdf/2502.03669
• Copy Paste: [[2502.03669]] Unrealized Expectations: Comparing AI Methods vs Classical Algorithms for Maximum Independent Set(https://arxiv.org/abs/2502.03669)
• Keywords: generative
• Abstract: AI methods, such as generative models and reinforcement learning, have recently been applied to combinatorial optimization (CO) problems, especially NP-hard ones. This paper compares such GPU-based methods with classical CPU-based methods on Maximum Independent Set (MIS). Experiments on standard graph families show that AI-based algorithms fail to outperform and, in many cases, to match the solution quality of the state-of-art classical solver KaMIS running on a single CPU. Some GPU-based methods even perform similarly to the simplest heuristic, degree-based greedy. Even with post-processing techniques like local search, AI-based methods still perform worse than CPU-based solvers. We develop a new mode of analysis to reveal that non-backtracking AI methods, e.g. LTFT (which is based on GFlowNets), end up reasoning similarly to the simplest degree-based greedy approach, and thus worse than KaMIS. We also find that CPU-based algorithms, notably KaMIS, have strong performance on sparse random graphs, which appears to refute a well-known conjectured upper bound for efficient algorithms from Coja-Oghlan & Efthymiou (2015).
• 摘要：

Title: Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free

• Authors: Gian Mario Favero, Parham Saremi, Emily Kaczmarek, Brennan Nichyporuk, Tal Arbel
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03687
• Pdf URL: https://arxiv.org/pdf/2502.03687
• Copy Paste: [[2502.03687]] Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free(https://arxiv.org/abs/2502.03687)
• Keywords: generative
• Abstract: Discriminative classifiers have become a foundational tool in deep learning for medical imaging, excelling at learning separable features of complex data distributions. However, these models often need careful design, augmentation, and training techniques to ensure safe and reliable deployment. Recently, diffusion models have become synonymous with generative modeling in 2D. These models showcase robustness across a range of tasks including natural image classification, where classification is performed by comparing reconstruction errors across images generated for each possible conditioning input. This work presents the first exploration of the potential of class conditional diffusion models for 2D medical image classification. First, we develop a novel majority voting scheme shown to improve the performance of medical diffusion classifiers. Next, extensive experiments on the CheXpert and ISIC Melanoma skin cancer datasets demonstrate that foundation and trained-from-scratch diffusion models achieve competitive performance against SOTA discriminative classifiers without the need for explicit supervision. In addition, we show that diffusion classifiers are intrinsically explainable, and can be used to quantify the uncertainty of their predictions, increasing their trustworthiness and reliability in safety-critical, clinical contexts. Further information is available on our project page: this https URL
• 摘要：

Title: DICE: Distilling Classifier-Free Guidance into Text Embeddings

• Authors: Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, Siwei Lyu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03726
• Pdf URL: https://arxiv.org/pdf/2502.03726
• Copy Paste: [[2502.03726]] DICE: Distilling Classifier-Free Guidance into Text Embeddings(https://arxiv.org/abs/2502.03726)
• Keywords: generation, generative
• Abstract: Text-to-image diffusion models are capable of generating high-quality images, but these images often fail to align closely with the given text prompts. Classifier-free guidance (CFG) is a popular and effective technique for improving text-image alignment in the generative process. However, using CFG introduces significant computational overhead and deviates from the established theoretical foundations of diffusion models. In this paper, we present DIstilling CFG by enhancing text Embeddings (DICE), a novel approach that removes the reliance on CFG in the generative process while maintaining the benefits it provides. DICE distills a CFG-based text-to-image diffusion model into a CFG-free version by refining text embeddings to replicate CFG-based directions. In this way, we avoid the computational and theoretical drawbacks of CFG, enabling high-quality, well-aligned image generation at a fast sampling speed. Extensive experiments on multiple Stable Diffusion v1.5 variants, SDXL and PixArt-$\alpha$ demonstrate the effectiveness of our method. Furthermore, DICE supports negative prompts for image editing to improve image quality further. Code will be available soon.
• 摘要：

Title: Distribution learning via neural differential equations: minimal energy regularization and approximation theory

• Authors: Youssef Marzouk, Zhi Ren, Jakob Zech
• Subjects: cs.LG, math.CA, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.03795
• Pdf URL: https://arxiv.org/pdf/2502.03795
• Copy Paste: [[2502.03795]] Distribution learning via neural differential equations: minimal energy regularization and approximation theory(https://arxiv.org/abs/2502.03795)
• Keywords: generative
• Abstract: Neural ordinary differential equations (ODEs) provide expressive representations of invertible transport maps that can be used to approximate complex probability distributions, e.g., for generative modeling, density estimation, and Bayesian inference. We show that for a large class of transport maps $T$, there exists a time-dependent ODE velocity field realizing a straight-line interpolation $(1-t)x + tT(x)$, $t \in [0,1]$, of the displacement induced by the map. Moreover, we show that such velocity fields are minimizers of a training objective containing a specific minimum-energy regularization. We then derive explicit upper bounds for the $C^k$ norm of the velocity field that are polynomial in the $C^k$ norm of the corresponding transport map $T$; in the case of triangular (Knothe--Rosenblatt) maps, we also show that these bounds are polynomial in the $C^k$ norms of the associated source and target densities. Combining these results with stability arguments for distribution approximation via ODEs, we show that Wasserstein or Kullback--Leibler approximation of the target distribution to any desired accuracy $\epsilon > 0$ can be achieved by a deep neural network representation of the velocity field whose size is bounded explicitly in terms of $\epsilon$, the dimension, and the smoothness of the source and target densities. The same neural network ansatz yields guarantees on the value of the regularized training objective.
• 摘要：

Title: DeblurDiff: Real-World Image Deblurring with Generative Diffusion Models

• Authors: Lingshun Kong, Jiawei Zhang, Dongqing Zou, Jimmy Ren, Xiaohe Wu, Jiangxin Dong, Jinshan Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03810
• Pdf URL: https://arxiv.org/pdf/2502.03810
• Copy Paste: [[2502.03810]] DeblurDiff: Real-World Image Deblurring with Generative Diffusion Models(https://arxiv.org/abs/2502.03810)
• Keywords: restoration, generation, generative
• Abstract: Diffusion models have achieved significant progress in image generation. The pre-trained Stable Diffusion (SD) models are helpful for image deblurring by providing clear image priors. However, directly using a blurry image or pre-deblurred one as a conditional control for SD will either hinder accurate structure extraction or make the results overly dependent on the deblurring network. In this work, we propose a Latent Kernel Prediction Network (LKPN) to achieve robust real-world image deblurring. Specifically, we co-train the LKPN in latent space with conditional diffusion. The LKPN learns a spatially variant kernel to guide the restoration of sharp images in the latent space. By applying element-wise adaptive convolution (EAC), the learned kernel is utilized to adaptively process the input feature, effectively preserving the structural information of the input. This process thereby more effectively guides the generative process of Stable Diffusion (SD), enhancing both the deblurring efficacy and the quality of detail reconstruction. Moreover, the results at each diffusion step are utilized to iteratively estimate the kernels in LKPN to better restore the sharp latent by EAC. This iterative refinement enhances the accuracy and robustness of the deblurring process. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art image deblurring methods on both benchmark and real-world images.
• 摘要：

Title: FairT2I: Mitigating Social Bias in Text-to-Image Generation via Large Language Model-Assisted Detection and Attribute Rebalancing

• Authors: Jinya Sakurai, Issei Sato
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03826
• Pdf URL: https://arxiv.org/pdf/2502.03826
• Copy Paste: [[2502.03826]] FairT2I: Mitigating Social Bias in Text-to-Image Generation via Large Language Model-Assisted Detection and Attribute Rebalancing(https://arxiv.org/abs/2502.03826)
• Keywords: generation
• Abstract: The proliferation of Text-to-Image (T2I) models has revolutionized content creation, providing powerful tools for diverse applications ranging from artistic expression to educational material development and marketing. Despite these technological advancements, significant ethical concerns arise from these models' reliance on large-scale datasets that often contain inherent societal biases. These biases are further amplified when AI-generated content is included in training data, potentially reinforcing and perpetuating stereotypes in the generated outputs. In this paper, we introduce FairT2I, a novel framework that harnesses large language models to detect and mitigate social biases in T2I generation. Our framework comprises two key components: (1) an LLM-based bias detection module that identifies potential social biases in generated images based on text prompts, and (2) an attribute rebalancing module that fine-tunes sensitive attributes within the T2I model to mitigate identified biases. Our extensive experiments across various T2I models and datasets show that FairT2I can significantly reduce bias while maintaining high-quality image generation. We conducted both qualitative user studies and quantitative non-parametric analyses in the generated image feature space, building upon the occupational dataset introduced in the Stable Bias study. Our results show that FairT2I successfully mitigates social biases and enhances the diversity of sensitive attributes in generated images. We further demonstrate, using the P2 dataset, that our framework can detect subtle biases that are challenging for human observers to perceive, extending beyond occupation-related prompts. On the basis of these findings, we introduce a new benchmark dataset for evaluating bias in T2I models.
• 摘要：

Title: Taking A Closer Look at Interacting Objects: Interaction-Aware Open Vocabulary Scene Graph Generation

• Authors: Lin Li, Chuhan Zhang, Dong Zhang, Chong Sun, Chen Li, Long Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03856
• Pdf URL: https://arxiv.org/pdf/2502.03856
• Copy Paste: [[2502.03856]] Taking A Closer Look at Interacting Objects: Interaction-Aware Open Vocabulary Scene Graph Generation(https://arxiv.org/abs/2502.03856)
• Keywords: generation
• Abstract: Today's open vocabulary scene graph generation (OVSGG) extends traditional SGG by recognizing novel objects and relationships beyond predefined categories, leveraging the knowledge from pre-trained large-scale models. Most existing methods adopt a two-stage pipeline: weakly supervised pre-training with image captions and supervised fine-tuning (SFT) on fully annotated scene graphs. Nonetheless, they omit explicit modeling of interacting objects and treat all objects equally, resulting in mismatched relation pairs. To this end, we propose an interaction-aware OVSGG framework INOVA. During pre-training, INOVA employs an interaction-aware target generation strategy to distinguish interacting objects from non-interacting ones. In SFT, INOVA devises an interaction-guided query selection tactic to prioritize interacting objects during bipartite graph matching. Besides, INOVA is equipped with an interaction-consistent knowledge distillation to enhance the robustness by pushing interacting object pairs away from the background. Extensive experiments on two benchmarks (VG and GQA) show that INOVA achieves state-of-the-art performance, demonstrating the potential of interaction-aware mechanisms for real-world applications.
• 摘要：

Title: Bridging the inference gap in Mutimodal Variational Autoencoders

• Authors: Agathe Senellart, Stéphanie Allassonnière
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.03952
• Pdf URL: https://arxiv.org/pdf/2502.03952
• Copy Paste: [[2502.03952]] Bridging the inference gap in Mutimodal Variational Autoencoders(https://arxiv.org/abs/2502.03952)
• Keywords: generation
• Abstract: From medical diagnosis to autonomous vehicles, critical applications rely on the integration of multiple heterogeneous data modalities. Multimodal Variational Autoencoders offer versatile and scalable methods for generating unobserved modalities from observed ones. Recent models using mixturesof-experts aggregation suffer from theoretically grounded limitations that restrict their generation quality on complex datasets. In this article, we propose a novel interpretable model able to learn both joint and conditional distributions without introducing mixture aggregation. Our model follows a multistage training process: first modeling the joint distribution with variational inference and then modeling the conditional distributions with Normalizing Flows to better approximate true posteriors. Importantly, we also propose to extract and leverage the information shared between modalities to improve the conditional coherence of generated samples. Our method achieves state-of-the-art results on several benchmark datasets.
• 摘要：

Title: MultiFloodSynth: Multi-Annotated Flood Synthetic Dataset Generation

• Authors: YoonJe Kang, Yonghoon Jung, Wonseop Shin, Bumsoo Kim, Sanghyun Seo
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03966
• Pdf URL: https://arxiv.org/pdf/2502.03966
• Copy Paste: [[2502.03966]] MultiFloodSynth: Multi-Annotated Flood Synthetic Dataset Generation(https://arxiv.org/abs/2502.03966)
• Keywords: generation, generative
• Abstract: In this paper, we present synthetic data generation framework for flood hazard detection system. For high fidelity and quality, we characterize several real-world properties into virtual world and simulate the flood situation by controlling them. For the sake of efficiency, recent generative models in image-to-3D and urban city synthesis are leveraged to easily composite flood environments so that we avoid data bias due to the hand-crafted manner. Based on our framework, we build the flood synthetic dataset with 5 levels, dubbed MultiFloodSynth which contains rich annotation types like normal map, segmentation, 3D bounding box for a variety of downstream task. In experiments, our dataset demonstrate the enhanced performance of flood hazard detection with on-par realism compared with real dataset.
• 摘要：

Title: Tight Bounds on Jensen's Gap: Novel Approach with Applications in Generative Modeling

• Authors: Marcin Mazur, Piotr Kościelniak, Łukasz Struski
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.03988
• Pdf URL: https://arxiv.org/pdf/2502.03988
• Copy Paste: [[2502.03988]] Tight Bounds on Jensen's Gap: Novel Approach with Applications in Generative Modeling(https://arxiv.org/abs/2502.03988)
• Keywords: generative
• Abstract: Among various mathematical tools of particular interest are those that provide a common basis for researchers in different scientific fields. One of them is Jensen's inequality, which states that the expectation of a convex function is greater than or equal to the function evaluated at the expectation. The resulting difference, known as Jensen's gap, became the subject of investigation by both the statistical and machine learning communities. Among many related topics, finding lower and upper bounds on Jensen's gap (under different assumptions on the underlying function and distribution) has recently become a problem of particular interest. In our paper, we take another step in this direction by providing a novel general and mathematically rigorous technique, motivated by the recent results of Struski et al. (2023). In addition, by studying in detail the case of the logarithmic function and the log-normal distribution, we explore a method for tightly estimating the log-likelihood of generative models trained on real-world datasets. Furthermore, we present both analytical and experimental arguments in support of the superiority of our approach in comparison to existing state-of-the-art solutions, contingent upon fulfillment of the criteria set forth by theoretical studies and corresponding experiments on synthetic data.
• 摘要：

Title: CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing

• Authors: Yu Yuan, Shizhao Sun, Qi Liu, Jiang Bian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.03997
• Pdf URL: https://arxiv.org/pdf/2502.03997
• Copy Paste: [[2502.03997]] CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing(https://arxiv.org/abs/2502.03997)
• Keywords: generation
• Abstract: Computer Aided Design (CAD) is indispensable across various industries. \emph{Text-based CAD editing}, which automates the modification of CAD models based on textual instructions, holds great potential but remains underexplored. Existing methods primarily focus on design variation generation or text-based CAD generation, either lacking support for text-based control or neglecting existing CAD models as constraints. We introduce \emph{CAD-Editor}, the first framework for text-based CAD editing. To address the challenge of demanding triplet data with accurate correspondence for training, we propose an automated data synthesis pipeline. This pipeline utilizes design variation models to generate pairs of original and edited CAD models and employs Large Vision-Language Models (LVLMs) to summarize their differences into editing instructions. To tackle the composite nature of text-based CAD editing, we propose a locate-then-infill framework that decomposes the task into two focused sub-tasks: locating regions requiring modification and infilling these regions with appropriate edits. Large Language Models (LLMs) serve as the backbone for both sub-tasks, leveraging their capabilities in natural language understanding and CAD knowledge. Experiments show that CAD-Editor achieves superior performance both quantitatively and qualitatively.
• 摘要：

Title: Probe-Free Low-Rank Activation Intervention

• Authors: Chonghe Jiang, Bao Nguyen, Anthony Man-Cho So, Viet Anh Nguyen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.04043
• Pdf URL: https://arxiv.org/pdf/2502.04043
• Copy Paste: [[2502.04043]] Probe-Free Low-Rank Activation Intervention(https://arxiv.org/abs/2502.04043)
• Keywords: generation
• Abstract: Language models (LMs) can produce texts that appear accurate and coherent but contain untruthful or toxic content. Inference-time interventions that edit the hidden activations have shown promising results in steering the LMs towards desirable generations. Existing activation intervention methods often comprise an activation probe to detect undesirable generation, triggering the activation modification to steer subsequent generation. This paper proposes a probe-free intervention method FLORAIN for all attention heads in a specific activation layer. It eliminates the need to train classifiers for probing purposes. The intervention function is parametrized by a sample-wise nonlinear low-rank mapping, which is trained by minimizing the distance between the modified activations and their projection onto the manifold of desirable content. Under specific constructions of the manifold and projection distance, we show that the intervention strategy can be computed efficiently by solving a smooth optimization problem. The empirical results, benchmarked on multiple base models, demonstrate that FLORAIN consistently outperforms several baseline methods in enhancing model truthfulness and quality across generation and multiple-choice tasks.
• 摘要：

Title: Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory

• Authors: Sascha Marton, Moritz Schneider
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.04052
• Pdf URL: https://arxiv.org/pdf/2502.04052
• Copy Paste: [[2502.04052]] Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory(https://arxiv.org/abs/2502.04052)
• Keywords: generation
• Abstract: Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.
• 摘要：

Title: Evaluating Inter-Column Logical Relationships in Synthetic Tabular Data Generation

• Authors: Yunbo Long, Liming Xu, Alexandra Brintrup
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.04055
• Pdf URL: https://arxiv.org/pdf/2502.04055
• Copy Paste: [[2502.04055]] Evaluating Inter-Column Logical Relationships in Synthetic Tabular Data Generation(https://arxiv.org/abs/2502.04055)
• Keywords: generation
• Abstract: Current evaluations of synthetic tabular data mainly focus on how well joint distributions are modeled, often overlooking the assessment of their effectiveness in preserving realistic event sequences and coherent entity relationships across this http URL paper proposes three evaluation metrics designed to assess the preservation of logical relationships among columns in synthetic tabular data. We validate these metrics by assessing the performance of both classical and state-of-the-art generation methods on a real-world industrial this http URL results reveal that existing methods often fail to rigorously maintain logical consistency (e.g., hierarchical relationships in geography or organization) and dependencies (e.g., temporal sequences or mathematical relationships), which are crucial for preserving the fine-grained realism of real-world tabular data. Building on these insights, this study also discusses possible pathways to better capture logical relationships while modeling the distribution of synthetic tabular data.
• 摘要：

Title: TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers

• Authors: Younghye Hwang, Hyojin Lee, Joonhyuk Kang
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2502.04056
• Pdf URL: https://arxiv.org/pdf/2502.04056
• Copy Paste: [[2502.04056]] TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers(https://arxiv.org/abs/2502.04056)
• Keywords: generative
• Abstract: Diffusion transformers (DiTs) combine transformer architectures with diffusion models. However, their computational complexity imposes significant limitations on real-time applications and sustainability of AI systems. In this study, we aim to enhance the computational efficiency through model quantization, which represents the weights and activation values with lower precision. Multi-region quantization (MRQ) is introduced to address the asymmetric distribution of network values in DiT blocks by allocating two scaling parameters to sub-regions. Additionally, time-grouping quantization (TGQ) is proposed to reduce quantization error caused by temporal variation in activations. The experimental results show that the proposed algorithm achieves performance comparable to the original full-precision model with only a 0.29 increase in FID at W8A8. Furthermore, it outperforms other baselines at W6A6, thereby confirming its suitability for low-bit quantization. These results highlight the potential of our method to enable efficient real-time generative models.
• 摘要：

Title: Content-Rich AIGC Video Quality Assessment via Intricate Text Alignment and Motion-Aware Consistency

• Authors: Shangkun Sun, Xiaoyu Liang, Bowen Qu, Wei Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.04076
• Pdf URL: https://arxiv.org/pdf/2502.04076
• Copy Paste: [[2502.04076]] Content-Rich AIGC Video Quality Assessment via Intricate Text Alignment and Motion-Aware Consistency(https://arxiv.org/abs/2502.04076)
• Keywords: generation, quality assessment
• Abstract: The advent of next-generation video generation models like \textit{Sora} poses challenges for AI-generated content (AIGC) video quality assessment (VQA). These models substantially mitigate flickering artifacts prevalent in prior models, enable longer and complex text prompts and generate longer videos with intricate, diverse motion patterns. Conventional VQA methods designed for simple text and basic motion patterns struggle to evaluate these content-rich videos. To this end, we propose \textbf{CRAVE} (\underline{C}ontent-\underline{R}ich \underline{A}IGC \underline{V}ideo \underline{E}valuator), specifically for the evaluation of Sora-era AIGC videos. CRAVE proposes the multi-granularity text-temporal fusion that aligns long-form complex textual semantics with video dynamics. Additionally, CRAVE leverages the hybrid motion-fidelity modeling to assess temporal artifacts. Furthermore, given the straightforward prompts and content in current AIGC VQA datasets, we introduce \textbf{CRAVE-DB}, a benchmark featuring content-rich videos from next-generation models paired with elaborate prompts. Extensive experiments have shown that the proposed CRAVE achieves excellent results on multiple AIGC VQA benchmarks, demonstrating a high degree of alignment with human perception. All data and code will be publicly available at this https URL.
• 摘要：

Title: Generative Adversarial Networks Bridging Art and Machine Intelligence

• Authors: Junhao Song, Yichao Zhang, Ziqian Bi, Tianyang Wang, Keyu Chen, Ming Li, Qian Niu, Junyu Liu, Benji Peng, Sen Zhang, Ming Liu, Jiawei Xu, Xuanhe Pan, Jinlang Wang, Pohsun Feng, Yizhu Wen, Lawrence K.Q. Yan, Hong-Ming Tseng, Xinyuan Song, Jintao Ren, Silin Chen, Yunze Wang, Weiche Hsieh, Bowen Jing, Junjie Yang, Jun Zhou, Zheyu Yao, Chia Xin Liang
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.04116
• Pdf URL: https://arxiv.org/pdf/2502.04116
• Copy Paste: [[2502.04116]] Generative Adversarial Networks Bridging Art and Machine Intelligence(https://arxiv.org/abs/2502.04116)
• Keywords: generation, generative
• Abstract: This book begins with a detailed introduction to the fundamental principles and historical development of GANs, contrasting them with traditional generative models and elucidating the core adversarial mechanisms through illustrative Python examples. The text systematically addresses the mathematical and theoretical underpinnings including probability theory, statistics, and game theory providing a solid framework for understanding the objectives, loss functions, and optimisation challenges inherent to GAN training. Subsequent chapters review classic variants such as Conditional GANs, DCGANs, InfoGAN, and LAPGAN before progressing to advanced training methodologies like Wasserstein GANs, GANs with gradient penalty, least squares GANs, and spectral normalisation techniques. The book further examines architectural enhancements and task-specific adaptations in generators and discriminators, showcasing practical implementations in high resolution image generation, artistic style transfer, video synthesis, text to image generation and other multimedia applications. The concluding sections offer insights into emerging research trends, including self-attention mechanisms, transformer-based generative models, and a comparative analysis with diffusion models, thus charting promising directions for future developments in both academic and applied settings.
• 摘要：

Title: Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning

• Authors: Wesley A. Suttle, Aamodh Suresh, Carlos Nieto-Granda
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.04141
• Pdf URL: https://arxiv.org/pdf/2502.04141
• Copy Paste: [[2502.04141]] Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning(https://arxiv.org/abs/2502.04141)
• Keywords: generation
• Abstract: Entropy-based objectives are widely used to perform state space exploration in reinforcement learning (RL) and dataset generation for offline RL. Behavioral entropy (BE), a rigorous generalization of classical entropies that incorporates cognitive and perceptual biases of agents, was recently proposed for discrete settings and shown to be a promising metric for robotic exploration problems. In this work, we propose using BE as a principled exploration objective for systematically generating datasets that provide diverse state space coverage in complex, continuous, potentially high-dimensional domains. To achieve this, we extend the notion of BE to continuous settings, derive tractable $k$-nearest neighbor estimators, provide theoretical guarantees for these estimators, and develop practical reward functions that can be used with standard RL methods to learn BE-maximizing policies. Using standard MuJoCo environments, we experimentally compare the performance of offline RL algorithms for a variety of downstream tasks on datasets generated using BE, Rényi, and Shannon entropy-maximizing policies, as well as the SMM and RND algorithms. We find that offline RL algorithms trained on datasets collected using BE outperform those trained on datasets collected using Shannon entropy, SMM, and RND on all tasks considered, and on 80% of the tasks compared to datasets collected using Rényi entropy.
• 摘要：

Title: MRAMG-Bench: A BeyondText Benchmark for Multimodal Retrieval-Augmented Multimodal Generation

• Authors: Qinhan Yu, Zhiyou Xiao, Binghui Li, Zhengren Wang, Chong Chen, Wentao Zhang
• Subjects: cs.LG, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.04176
• Pdf URL: https://arxiv.org/pdf/2502.04176
• Copy Paste: [[2502.04176]] MRAMG-Bench: A BeyondText Benchmark for Multimodal Retrieval-Augmented Multimodal Generation(https://arxiv.org/abs/2502.04176)
• Keywords: generation, generative
• Abstract: Recent advancements in Retrieval-Augmented Generation (RAG) have shown remarkable performance in enhancing response accuracy and relevance by integrating external knowledge into generative models. However, existing RAG methods primarily focus on providing text-only answers, even in multimodal retrieval-augmented generation scenarios. In this work, we introduce the Multimodal Retrieval-Augmented Multimodal Generation (MRAMG) task, which aims to generate answers that combine both text and images, fully leveraging the multimodal data within a corpus. Despite the importance of this task, there is a notable absence of a comprehensive benchmark to effectively evaluate MRAMG performance. To bridge this gap, we introduce the MRAMG-Bench, a carefully curated, human-annotated dataset comprising 4,346 documents, 14,190 images, and 4,800 QA pairs, sourced from three categories: Web Data, Academic Papers, and Lifestyle. The dataset incorporates diverse difficulty levels and complex multi-image scenarios, providing a robust foundation for evaluating multimodal generation tasks. To facilitate rigorous evaluation, our MRAMG-Bench incorporates a comprehensive suite of both statistical and LLM-based metrics, enabling a thorough analysis of the performance of popular generative models in the MRAMG task. Besides, we propose an efficient multimodal answer generation framework that leverages both LLMs and MLLMs to generate multimodal responses. Our datasets are available at: this https URL.
• 摘要：

Title: PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models?

• Authors: Mennatullah Siam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.04192
• Pdf URL: https://arxiv.org/pdf/2502.04192
• Copy Paste: [[2502.04192]] PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models?(https://arxiv.org/abs/2502.04192)
• Keywords: generation
• Abstract: Multiple works have emerged to push the boundaries on multi-modal large language models (MLLMs) towards pixel-level understanding. Such approaches have shown strong performance on benchmarks for referring expression segmentation and grounded conversation generation. The current trend in pixel-level MLLMs is to train with pixel-level grounding supervision on large-scale labelled data. However, we show that such MLLMs when evaluated on recent challenging vision centric benchmarks, exhibit a weak ability in visual question answering. Surprisingly, some of these methods even downgrade the grounding ability of MLLMs that were never trained with such supervision. In this work, we propose two novel challenging benchmarks and show that MLLMs without pixel-level grounding supervision can outperform the state of the art in such tasks when evaluating both the pixel-level grounding and visual question answering. We propose simple baselines to extract the grounding information that can be plugged into any MLLM, which we call as PixFoundation. More importantly, we study the research question of ``When does grounding emerge in MLLMs that are not trained with pixel-level grounding supervision?'' We show that grounding can coincide with object parts or location/appearance information. Code repository is at this https URL.
• 摘要：

Title: Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retention

• Authors: Ayush K. Varshney, Vicenç Torra
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.04260
• Pdf URL: https://arxiv.org/pdf/2502.04260
• Copy Paste: [[2502.04260]] Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retention(https://arxiv.org/abs/2502.04260)
• Keywords: generative
• Abstract: Machine Unlearning allows participants to remove their data from a trained machine learning model in order to preserve their privacy, and security. However, the machine unlearning literature for generative models is rather limited. The literature for image-to-image generative model (I2I model) considers minimizing the distance between Gaussian noise and the output of I2I model for forget samples as machine unlearning. However, we argue that the machine learning model performs fairly well on unseen data i.e., a retrained model will be able to catch generic patterns in the data and hence will not generate an output which is equivalent to Gaussian noise. In this paper, we consider that the model after unlearning should treat forget samples as out-of-distribution (OOD) data, i.e., the unlearned model should no longer recognize or encode the specific patterns found in the forget samples. To achieve this, we propose a framework which decouples the model parameters with gradient ascent, ensuring that forget samples are OOD for unlearned model with theoretical guarantee. We also provide $(\epsilon, \delta)$-unlearning guarantee for model updates with gradient ascent. The unlearned model is further fine-tuned on the remaining samples to maintain its performance. We also propose an attack model to ensure that the unlearned model has effectively removed the influence of forget samples. Extensive empirical evaluation on two large-scale datasets, ImageNet-1K and Places365 highlights the superiority of our approach. To show comparable performance with retrained model, we also show the comparison of a simple AutoEncoder on various baselines on CIFAR-10 dataset.
• 摘要：

Title: MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation

• Authors: Jinbo Xing, Long Mai, Cusuh Ham, Jiahui Huang, Aniruddha Mahapatra, Chi-Wing Fu, Tien-Tsin Wong, Feng Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.04299
• Pdf URL: https://arxiv.org/pdf/2502.04299
• Copy Paste: [[2502.04299]] MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation(https://arxiv.org/abs/2502.04299)
• Keywords: generation
• Abstract: This paper presents a method that allows users to design cinematic video shots in the context of image-to-video generation. Shot design, a critical aspect of filmmaking, involves meticulously planning both camera movements and object motions in a scene. However, enabling intuitive shot design in modern image-to-video generation systems presents two main challenges: first, effectively capturing user intentions on the motion design, where both camera movements and scene-space object motions must be specified jointly; and second, representing motion information that can be effectively utilized by a video diffusion model to synthesize the image animations. To address these challenges, we introduce MotionCanvas, a method that integrates user-driven controls into image-to-video (I2V) generation models, allowing users to control both object and camera motions in a scene-aware manner. By connecting insights from classical computer graphics and contemporary video generation techniques, we demonstrate the ability to achieve 3D-aware motion control in I2V synthesis without requiring costly 3D-related training data. MotionCanvas enables users to intuitively depict scene-space motion intentions, and translates them into spatiotemporal motion-conditioning signals for video diffusion models. We demonstrate the effectiveness of our method on a wide range of real-world image content and shot-design scenarios, highlighting its potential to enhance the creative workflows in digital content creation and adapt to various image and video editing applications.
• 摘要：

Title: HOG-Diff: Higher-Order Guided Diffusion for Graph Generation

• Authors: Yiming Huang, Tolga Birdal
• Subjects: cs.LG, cs.AI, cs.SI, physics.soc-ph
• Abstract URL: https://arxiv.org/abs/2502.04308
• Pdf URL: https://arxiv.org/pdf/2502.04308
• Copy Paste: [[2502.04308]] HOG-Diff: Higher-Order Guided Diffusion for Graph Generation(https://arxiv.org/abs/2502.04308)
• Keywords: generation
• Abstract: Graph generation is a critical yet challenging task as empirical analyses require a deep understanding of complex, non-Euclidean structures. Although diffusion models have recently made significant achievements in graph generation, these models typically adapt from the frameworks designed for image generation, making them ill-suited for capturing the topological properties of graphs. In this work, we propose a novel Higher-order Guided Diffusion (HOG-Diff) model that follows a coarse-to-fine generation curriculum and is guided by higher-order information, enabling the progressive generation of plausible graphs with inherent topological structures. We further prove that our model exhibits a stronger theoretical guarantee than classical diffusion frameworks. Extensive experiments on both molecular and generic graph generation tasks demonstrate that our method consistently outperforms or remains competitive with state-of-the-art baselines. Our code is available at this https URL.
• 摘要：

Title: Consistency of augmentation graph and network approximability in contrastive learning

• Authors: Chenghui Li, A. Martina Neuman
• Subjects: cs.LG, math.AP, math.SP
• Abstract URL: https://arxiv.org/abs/2502.04312
• Pdf URL: https://arxiv.org/pdf/2502.04312
• Copy Paste: [[2502.04312]] Consistency of augmentation graph and network approximability in contrastive learning(https://arxiv.org/abs/2502.04312)
• Keywords: generation
• Abstract: Contrastive learning leverages data augmentation to develop feature representation without relying on large labeled datasets. However, despite its empirical success, the theoretical foundations of contrastive learning remain incomplete, with many essential guarantees left unaddressed, particularly the realizability assumption concerning neural approximability of an optimal spectral contrastive loss solution. In this work, we overcome these limitations by analyzing the pointwise and spectral consistency of the augmentation graph Laplacian. We establish that, under specific conditions for data generation and graph connectivity, as the augmented dataset size increases, the augmentation graph Laplacian converges to a weighted Laplace-Beltrami operator on the natural data manifold. These consistency results ensure that the graph Laplacian spectrum effectively captures the manifold geometry. Consequently, they give way to a robust framework for establishing neural approximability, directly resolving the realizability assumption in a current paradigm.
• 摘要：

Title: sshELF: Single-Shot Hierarchical Extrapolation of Latent Features for 3D Reconstruction from Sparse-Views

• Authors: Eyvaz Najafli, Marius Kästingschäfer, Sebastian Bernhard, Thomas Brox, Andreas Geiger
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.04318
• Pdf URL: https://arxiv.org/pdf/2502.04318
• Copy Paste: [[2502.04318]] sshELF: Single-Shot Hierarchical Extrapolation of Latent Features for 3D Reconstruction from Sparse-Views(https://arxiv.org/abs/2502.04318)
• Keywords: generation
• Abstract: Reconstructing unbounded outdoor scenes from sparse outward-facing views poses significant challenges due to minimal view overlap. Previous methods often lack cross-scene understanding and their primitive-centric formulations overload local features to compensate for missing global context, resulting in blurriness in unseen parts of the scene. We propose sshELF, a fast, single-shot pipeline for sparse-view 3D scene reconstruction via hierarchal extrapolation of latent features. Our key insights is that disentangling information extrapolation from primitive decoding allows efficient transfer of structural patterns across training scenes. Our method: (1) learns cross-scene priors to generate intermediate virtual views to extrapolate to unobserved regions, (2) offers a two-stage network design separating virtual view generation from 3D primitive decoding for efficient training and modular model design, and (3) integrates a pre-trained foundation model for joint inference of latent features and texture, improving scene understanding and generalization. sshELF can reconstruct 360 degree scenes from six sparse input views and achieves competitive results on synthetic and real-world datasets. We find that sshELF faithfully reconstructs occluded regions, supports real-time rendering, and provides rich latent features for downstream applications. The code will be released.
• 摘要：

Title: Ola: Pushing the Frontiers of Omni-Modal Language Model with Progressive Modality Alignment

• Authors: Zuyan Liu, Yuhao Dong, Jiahui Wang, Ziwei Liu, Winston Hu, Jiwen Lu, Yongming Rao
• Subjects: cs.CV, cs.CL, cs.MM, cs.SD, eess.AS, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.04328
• Pdf URL: https://arxiv.org/pdf/2502.04328
• Copy Paste: [[2502.04328]] Ola: Pushing the Frontiers of Omni-Modal Language Model with Progressive Modality Alignment(https://arxiv.org/abs/2502.04328)
• Keywords: generation
• Abstract: Recent advances in large language models, particularly following GPT-4o, have sparked increasing interest in developing omni-modal models capable of understanding more modalities. While some open-source alternatives have emerged, there is still a notable lag behind specialized single-modality models in performance. In this paper, we present Ola, an Omni-modal language model that achieves competitive performance across image, video, and audio understanding compared to specialized counterparts. The core design of Ola lies in its progressive modality alignment strategy that extends the supporting modality of the language model progressively. Our training pipeline begins with the most distinct modalities: image and text, then gradually expands the skill sets of the model using speech data that connects language and audio knowledge, and video data that connects all modalities. The progressive learning pipeline also enables us to maintain a relatively small size of the cross-modal alignment data, making developing omni-modal from existing vision-language models easy and less costly. Moreover, to unlock an advanced interactive experience like GPT-4o, we further design a sentence-wise decoding solution for streaming speech generation. Extensive experiments demonstrate that Ola surpasses existing open omni-modal LLMs across all modalities while achieving highly competitive performance compared to state-of-the-art specialized models of similar sizes. We aim to make Ola a fully open omni-modal understanding solution to advance future research in this emerging field. Model weights, code, and data are open-sourced at this https URL.
• 摘要：