2025-02-05

Title: Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding

• Authors: Jingming Xia, Guanqun Cao, Guang Ma, Yiben Luo, Qinzhao Li, John Oyekan
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01666
• Pdf URL: https://arxiv.org/pdf/2502.01666
• Copy Paste: [[2502.01666]] Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding(https://arxiv.org/abs/2502.01666)
• Keywords: diffusion, generative
• Abstract: Monocular depth estimation involves predicting depth from a single RGB image and plays a crucial role in applications such as autonomous driving, robotic navigation, 3D reconstruction, etc. Recent advancements in learning-based methods have significantly improved depth estimation performance. Generative models, particularly Stable Diffusion, have shown remarkable potential in recovering fine details and reconstructing missing regions through large-scale training on diverse datasets. However, models like CLIP, which rely on textual embeddings, face limitations in complex outdoor environments where rich context information is needed. These limitations reduce their effectiveness in such challenging scenarios. Here, we propose a novel image-based semantic embedding that extracts contextual information directly from visual features, significantly improving depth prediction in complex environments. Evaluated on the KITTI and Waymo datasets, our method achieves performance comparable to state-of-the-art models while addressing the shortcomings of CLIP embeddings in handling outdoor scenes. By leveraging visual semantics directly, our method demonstrates enhanced robustness and adaptability in depth estimation tasks, showcasing its potential for application to other visual perception tasks.

Title: Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking

• Authors: Jie Ren, Yuhang Zhang, Dongrui Liu, Xiaopeng Zhang, Qi Tian
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01667
• Pdf URL: https://arxiv.org/pdf/2502.01667
• Copy Paste: [[2502.01667]] Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking(https://arxiv.org/abs/2502.01667)
• Keywords: diffusion
• Abstract: Direct preference optimization (DPO) has shown success in aligning diffusion models with human preference. Previous approaches typically assume a consistent preference label between final generations and noisy samples at intermediate steps, and directly apply DPO to these noisy samples for fine-tuning. However, we theoretically identify inherent issues in this assumption and its impacts on the effectiveness of preference alignment. We first demonstrate the inherent issues from two perspectives: gradient direction and preference order, and then propose a Tailored Preference Optimization (TailorPO) framework for aligning diffusion models with human preference, underpinned by some theoretical insights. Our approach directly ranks intermediate noisy samples based on their step-wise reward, and effectively resolves the gradient direction issues through a simple yet efficient design. Additionally, we incorporate the gradient guidance of diffusion models into preference alignment to further enhance the optimization effectiveness. Experimental results demonstrate that our method significantly improves the model's ability to generate aesthetically pleasing and human-preferred images.

Title: Semantic Communication based on Generative AI: A New Approach to Image Compression and Edge Optimization

• Authors: Francesco Pezone
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01675
• Pdf URL: https://arxiv.org/pdf/2502.01675
• Copy Paste: [[2502.01675]] Semantic Communication based on Generative AI: A New Approach to Image Compression and Edge Optimization(https://arxiv.org/abs/2502.01675)
• Keywords: diffusion, generative
• Abstract: As digital technologies advance, communication networks face challenges in handling the vast data generated by intelligent devices. Autonomous vehicles, smart sensors, and IoT systems necessitate new paradigms. This thesis addresses these challenges by integrating semantic communication and generative models for optimized image compression and edge network resource allocation. Unlike bit-centric systems, semantic communication prioritizes transmitting meaningful data specifically selected to convey the meaning rather than obtain a faithful representation of the original data. The communication infrastructure can benefit to significant improvements in bandwidth efficiency and latency reduction. Central to this work is the design of semantic-preserving image compression using Generative Adversarial Networks and Denoising Diffusion Probabilistic Models. These models compress images by encoding only semantically relevant features, allowing for high-quality reconstruction with minimal transmission. Additionally, a Goal-Oriented edge network optimization framework is introduced, leveraging the Information Bottleneck principle and stochastic optimization to dynamically allocate resources and enhance efficiency. By integrating semantic communication into edge networks, this approach balances computational efficiency and communication effectiveness, making it suitable for real-time applications. The thesis compares semantic-aware models with conventional image compression techniques using classical and semantic evaluation metrics. Results demonstrate the potential of combining generative AI and semantic communication to create more efficient semantic-goal-oriented communication networks that meet the demands of modern data-driven applications.

Title: LEAD: Large Foundation Model for EEG-Based Alzheimer's Disease Detection

• Authors: Yihe Wang, Nan Huang, Nadia Mammone, Marco Cecchi, Xiang Zhang
• Subjects: cs.LG, cs.AI, cs.CE, eess.SP
• Abstract URL: https://arxiv.org/abs/2502.01678
• Pdf URL: https://arxiv.org/pdf/2502.01678
• Copy Paste: [[2502.01678]] LEAD: Large Foundation Model for EEG-Based Alzheimer's Disease Detection(https://arxiv.org/abs/2502.01678)
• Keywords: self-supervised, foundation model
• Abstract: Electroencephalogram (EEG) provides a non-invasive, highly accessible, and cost-effective solution for Alzheimer's Disease (AD) detection. However, existing methods, whether based on manual feature extraction or deep learning, face two major challenges: the lack of large-scale datasets for robust feature learning and evaluation, and poor detection performance due to inter-subject variations. To address these challenges, we curate an EEG-AD corpus containing 813 subjects, which forms the world's largest EEG-AD dataset to the best of our knowledge. Using this unique dataset, we propose LEAD, the first large foundation model for EEG-based AD detection. Our method encompasses an entire pipeline, from data selection and preprocessing to self-supervised contrastive pretraining, fine-tuning, and key setups such as subject-independent evaluation and majority voting for subject-level detection. We pre-train the model on 11 EEG datasets and unified fine-tune it on 5 AD datasets. Our self-supervised pre-training design includes sample-level and subject-level contrasting to extract useful general EEG features. Fine-tuning is performed on 5 channel-aligned datasets together. The backbone encoder incorporates temporal and channel embeddings to capture features across both temporal and spatial dimensions. Our method demonstrates outstanding AD detection performance, achieving up to a 9.86% increase in F1 score at the sample-level and up to a 9.31% at the subject-level compared to state-of-the-art methods. The results of our model strongly confirm the effectiveness of contrastive pre-training and channel-aligned unified fine-tuning for addressing inter-subject variation. The source code is at this https URL.

Title: Leveraging Joint Predictive Embedding and Bayesian Inference in Graph Self Supervised Learning

• Authors: Srinitish Srinivasan, Omkumar CU
• Subjects: cs.LG, cs.AI, cs.SI
• Abstract URL: https://arxiv.org/abs/2502.01684
• Pdf URL: https://arxiv.org/pdf/2502.01684
• Copy Paste: [[2502.01684]] Leveraging Joint Predictive Embedding and Bayesian Inference in Graph Self Supervised Learning(https://arxiv.org/abs/2502.01684)
• Keywords: self-supervised
• Abstract: Graph representation learning has emerged as a cornerstone for tasks like node classification and link prediction, yet prevailing self-supervised learning (SSL) methods face challenges such as computational inefficiency, reliance on contrastive objectives, and representation collapse. Existing approaches often depend on feature reconstruction, negative sampling, or complex decoders, which introduce training overhead and hinder generalization. Further, current techniques which address such limitations fail to account for the contribution of node embeddings to a certain prediction in the absence of labeled nodes. To address these limitations, we propose a novel joint embedding predictive framework for graph SSL that eliminates contrastive objectives and negative sampling while preserving semantic and structural information. Additionally, we introduce a semantic-aware objective term that incorporates pseudo-labels derived from Gaussian Mixture Models (GMMs), enhancing node discriminability by evaluating latent feature contributions. Extensive experiments demonstrate that our framework outperforms state-of-the-art graph SSL methods across benchmarks, achieving superior performance without contrastive loss or complex decoders. Key innovations include (1) a non-contrastive, view-invariant joint embedding predictive architecture, (2) Leveraging single context and multiple targets relationship between subgraphs, and (3) GMM-based pseudo-label scoring to capture semantic contributions. This work advances graph SSL by offering a computationally efficient, collapse-resistant paradigm that bridges spatial and semantic graph features for downstream tasks. The code for our paper can be found at this https URL

Title: HuViDPO:Enhancing Video Generation through Direct Preference Optimization for Human-Centric Alignment

• Authors: Lifan Jiang, Boxi Wu, Jiahui Zhang, Xiaotong Guan, Shuang Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01690
• Pdf URL: https://arxiv.org/pdf/2502.01690
• Copy Paste: [[2502.01690]] HuViDPO:Enhancing Video Generation through Direct Preference Optimization for Human-Centric Alignment(https://arxiv.org/abs/2502.01690)
• Keywords: diffusion
• Abstract: With the rapid development of AIGC technology, significant progress has been made in diffusion model-based technologies for text-to-image (T2I) and text-to-video (T2V). In recent years, a few studies have introduced the strategy of Direct Preference Optimization (DPO) into T2I tasks, significantly enhancing human preferences in generated images. However, existing T2V generation methods lack a well-formed pipeline with exact loss function to guide the alignment of generated videos with human preferences using DPO strategies. Additionally, challenges such as the scarcity of paired video preference data hinder effective model training. At the same time, the lack of training datasets poses a risk of insufficient flexibility and poor video generation quality in the generated videos. Based on those problems, our work proposes three targeted solutions in sequence. 1) Our work is the first to introduce the DPO strategy into the T2V tasks. By deriving a carefully structured loss function, we utilize human feedback to align video generation with human preferences. We refer to this new method as HuViDPO. 2) Our work constructs small-scale human preference datasets for each action category and fine-tune this model, improving the aesthetic quality of the generated videos while reducing training costs. 3) We adopt a First-Frame-Conditioned strategy, leveraging the rich in formation from the first frame to guide the generation of subsequent frames, enhancing flexibility in video generation. At the same time, we employ a SparseCausal Attention mechanism to enhance the quality of the generated this http URL details and examples can be accessed on our website: this https URL. this http URL.

Title: Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation

• Authors: Kim Yong Tan, Yueming Lyu, Ivor Tsang, Yew-Soon Ong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01692
• Pdf URL: https://arxiv.org/pdf/2502.01692
• Copy Paste: [[2502.01692]] Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation(https://arxiv.org/abs/2502.01692)
• Keywords: diffusion
• Abstract: Guided diffusion-model generation is a promising direction for customizing the generation process of a pre-trained diffusion-model to address the specific downstream tasks. Existing guided diffusion models either rely on training of the guidance model with pre-collected datasets or require the objective functions to be differentiable. However, for most real-world tasks, the offline datasets are often unavailable, and their objective functions are often not differentiable, such as image generation with human preferences, molecular generation for drug discovery, and material design. Thus, we need an \textbf{online} algorithm capable of collecting data during runtime and supporting a \textbf{black-box} objective function. Moreover, the \textbf{query efficiency} of the algorithm is also critical because the objective evaluation of the query is often expensive in the real-world scenarios. In this work, we propose a novel and simple algorithm, \textbf{Fast Direct}, for query-efficient online black-box target generation. Our Fast Direct builds a pseudo-target on the data manifold to update the noise sequence of the diffusion model with a universal direction, which is promising to perform query-efficient guided generation. Extensive experiments on twelve high-resolution ($\small {1024 \times 1024}$) image target generation tasks and six 3D-molecule target generation tasks show $\textbf{6}\times$ up to $\textbf{10}\times$ query efficiency improvement and $\textbf{11}\times$ up to $\textbf{44}\times$ query efficiency improvement, respectively. Our implementation is publicly available at: this https URL

Title: Al-Khwarizmi: Discovering Physical Laws with Foundation Models

• Authors: Christopher E. Mower, Haitham Bou-Ammar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01702
• Pdf URL: https://arxiv.org/pdf/2502.01702
• Copy Paste: [[2502.01702]] Al-Khwarizmi: Discovering Physical Laws with Foundation Models(https://arxiv.org/abs/2502.01702)
• Keywords: foundation model
• Abstract: Inferring physical laws from data is a central challenge in science and engineering, including but not limited to healthcare, physical sciences, biosciences, social sciences, sustainability, climate, and robotics. Deep networks offer high-accuracy results but lack interpretability, prompting interest in models built from simple components. The Sparse Identification of Nonlinear Dynamics (SINDy) method has become the go-to approach for building such modular and interpretable models. SINDy leverages sparse regression with L1 regularization to identify key terms from a library of candidate functions. However, SINDy's choice of candidate library and optimization method requires significant technical expertise, limiting its widespread applicability. This work introduces Al-Khwarizmi, a novel agentic framework for physical law discovery from data, which integrates foundational models with SINDy. Leveraging LLMs, VLMs, and Retrieval-Augmented Generation (RAG), our approach automates physical law discovery, incorporating prior knowledge and iteratively refining candidate solutions via reflection. Al-Khwarizmi operates in two steps: it summarizes system observations-comprising textual descriptions, raw data, and plots-followed by a secondary step that generates candidate feature libraries and optimizer configurations to identify hidden physics laws correctly. Evaluating our algorithm on over 198 models, we demonstrate state-of-the-art performance compared to alternatives, reaching a 20 percent increase against the best-performing alternative.

Title: Choose Your Model Size: Any Compression by a Single Gradient Descent

• Authors: Martin Genzel, Patrick Putzky, Pengfei Zhao, Sebastian Schulze, Mattes Mollenhauer, Robert Seidel, Stefan Dietzel, Thomas Wollmann
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01717
• Pdf URL: https://arxiv.org/pdf/2502.01717
• Copy Paste: [[2502.01717]] Choose Your Model Size: Any Compression by a Single Gradient Descent(https://arxiv.org/abs/2502.01717)
• Keywords: foundation model
• Abstract: The adoption of Foundation Models in resource-constrained environments remains challenging due to their large size and inference costs. A promising way to overcome these limitations is post-training compression, which aims to balance reduced model size against performance degradation. This work presents Any Compression via Iterative Pruning (ACIP), a novel algorithmic approach to determine a compression-performance trade-off from a single stochastic gradient descent run. To ensure parameter efficiency, we use an SVD-reparametrization of linear layers and iteratively prune their singular values with a sparsity-inducing penalty. The resulting pruning order gives rise to a global parameter ranking that allows us to materialize models of any target size. Importantly, the compressed models exhibit strong predictive downstream performance without the need for costly fine-tuning. We evaluate ACIP on a large selection of open-weight LLMs and tasks, and demonstrate state-of-the-art results compared to existing factorisation-based compression methods. We also show that ACIP seamlessly complements common quantization-based compression techniques.

Title: Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity

• Authors: Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, Jianfei Chen, Ion Stoica, Kurt Keutzer, Song Han
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01776
• Pdf URL: https://arxiv.org/pdf/2502.01776
• Copy Paste: [[2502.01776]] Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity(https://arxiv.org/abs/2502.01776)
• Keywords: diffusion
• Abstract: Diffusion Transformers (DiTs) dominate video generation but their high computational cost severely limits real-world applicability, usually requiring tens of minutes to generate a few seconds of video even on high-performance GPUs. This inefficiency primarily arises from the quadratic computational complexity of 3D Full Attention with respect to the context length. In this paper, we propose a training-free framework termed Sparse VideoGen (SVG) that leverages the inherent sparsity in 3D Full Attention to boost inference efficiency. We reveal that the attention heads can be dynamically classified into two groups depending on distinct sparse patterns: (1) Spatial Head, where only spatially-related tokens within each frame dominate the attention output, and (2) Temporal Head, where only temporally-related tokens across different frames dominate. Based on this insight, SVG proposes an online profiling strategy to capture the dynamic sparse patterns and predicts the type of attention head. Combined with a novel hardware-efficient tensor layout transformation and customized kernel implementations, SVG achieves up to 2.28x and 2.33x end-to-end speedup on CogVideoX-v1.5 and HunyuanVideo, respectively, while preserving generation quality.

Title: AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis

• Authors: Basit Alawode, Iyyakutti Iyappan Ganapathi, Sajid Javed, Naoufel Werghi, Mohammed Bennamoun, Arif Mahmood
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01785
• Pdf URL: https://arxiv.org/pdf/2502.01785
• Copy Paste: [[2502.01785]] AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis(https://arxiv.org/abs/2502.01785)
• Keywords: foundation model
• Abstract: The preservation of aquatic biodiversity is critical in mitigating the effects of climate change. Aquatic scene understanding plays a pivotal role in aiding marine scientists in their decision-making processes. In this paper, we introduce AquaticCLIP, a novel contrastive language-image pre-training model tailored for aquatic scene understanding. AquaticCLIP presents a new unsupervised learning framework that aligns images and texts in aquatic environments, enabling tasks such as segmentation, classification, detection, and object counting. By leveraging our large-scale underwater image-text paired dataset without the need for ground-truth annotations, our model enriches existing vision-language models in the aquatic domain. For this purpose, we construct a 2 million underwater image-text paired dataset using heterogeneous resources, including YouTube, Netflix, NatGeo, etc. To fine-tune AquaticCLIP, we propose a prompt-guided vision encoder that progressively aggregates patch features via learnable prompts, while a vision-guided mechanism enhances the language encoder by incorporating visual context. The model is optimized through a contrastive pretraining loss to align visual and textual modalities. AquaticCLIP achieves notable performance improvements in zero-shot settings across multiple underwater computer vision tasks, outperforming existing methods in both robustness and interpretability. Our model sets a new benchmark for vision-language applications in underwater environments. The code and dataset for AquaticCLIP are publicly available on GitHub at xxx.

Title: Self-supervised Subgraph Neural Network With Deep Reinforcement Walk Exploration

• Authors: Jianming Huang, Hiroyuki Kasai
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01809
• Pdf URL: https://arxiv.org/pdf/2502.01809
• Copy Paste: [[2502.01809]] Self-supervised Subgraph Neural Network With Deep Reinforcement Walk Exploration(https://arxiv.org/abs/2502.01809)
• Keywords: self-supervised
• Abstract: Graph data, with its structurally variable nature, represents complex real-world phenomena like chemical compounds, protein structures, and social networks. Traditional Graph Neural Networks (GNNs) primarily utilize the message-passing mechanism, but their expressive power is limited and their prediction lacks explainability. To address these limitations, researchers have focused on graph substructures. Subgraph neural networks (SGNNs) and GNN explainers have emerged as potential solutions, but each has its limitations. SGNNs computes graph representations based on the bags of subgraphs to enhance the expressive power. However, they often rely on predefined algorithm-based sampling strategies, which is inefficient. GNN explainers adopt data-driven approaches to generate important subgraphs to provide explanation. Nevertheless, their explanation is difficult to be translated into practical improvements on GNNs. To overcome these issues, we propose a novel self-supervised framework that integrates SGNNs with the generation approach of GNN explainers, named the Reinforcement Walk Exploration SGNN (RWE-SGNN). Our approach features a sampling model trained in an explainer fashion, optimizing subgraphs to enhance model performance. To achieve a data-driven sampling approach, unlike traditional subgraph generation approaches, we propose a novel walk exploration process, which efficiently extracts important substructures, simplifying the embedding process and avoiding isomorphism problems. Moreover, we prove that our proposed walk exploration process has equivalent generation capability to the traditional subgraph generation process. Experimental results on various graph datasets validate the effectiveness of our proposed method, demonstrating significant improvements in performance and precision.

Title: SelfCheckAgent: Zero-Resource Hallucination Detection in Generative Large Language Models

• Authors: Diyana Muhammed, Gollam Rabby, Sören Auer
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01812
• Pdf URL: https://arxiv.org/pdf/2502.01812
• Copy Paste: [[2502.01812]] SelfCheckAgent: Zero-Resource Hallucination Detection in Generative Large Language Models(https://arxiv.org/abs/2502.01812)
• Keywords: generative
• Abstract: Detecting hallucinations in Large Language Models (LLMs) remains a critical challenge for their reliable deployment in real-world applications. To address this, we introduce SelfCheckAgent, a novel framework integrating three different agents: the Symbolic Agent, the Specialized Detection Agent, and the Contextual Consistency Agent. These agents provide a robust multi-dimensional approach to hallucination detection. Notable results include the Contextual Consistency Agent leveraging Llama 3.1 with Chain-of-Thought (CoT) to achieve outstanding performance on the WikiBio dataset, with NonFactual hallucination detection scoring 93.64%, Factual 70.26%, and Ranking 78.48% respectively. On the AIME dataset, GPT-4o with CoT excels in NonFactual detection with 94.89% but reveals trade-offs in Factual with 30.58% and Ranking with 30.68%, underscoring the complexity of hallucination detection in the complex mathematical domains. The framework also incorporates a triangulation strategy, which increases the strengths of the SelfCheckAgent, yielding significant improvements in real-world hallucination identification. The comparative analysis demonstrates SelfCheckAgent's applicability across diverse domains, positioning it as a crucial advancement for trustworthy LLMs. These findings highlight the potentiality of consistency-driven methodologies in detecting hallucinations in LLMs.

Title: Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning

• Authors: Hanyang Zhao, Haoxian Chen, Ji Zhang, David D. Yao, Wenpin Tang
• Subjects: cs.LG, cs.AI, math.OC
• Abstract URL: https://arxiv.org/abs/2502.01819
• Pdf URL: https://arxiv.org/pdf/2502.01819
• Copy Paste: [[2502.01819]] Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning(https://arxiv.org/abs/2502.01819)
• Keywords: diffusion, generative
• Abstract: Reinforcement learning from human feedback (RLHF), which aligns a diffusion model with input prompt, has become a crucial step in building reliable generative AI models. Most works in this area use a discrete-time formulation, which is prone to induced errors, and often not applicable to models with higher-order/black-box solvers. The objective of this study is to develop a disciplined approach to fine-tune diffusion models using continuous-time RL, formulated as a stochastic control problem with a reward function that aligns the end result (terminal state) with input prompt. The key idea is to treat score matching as controls or actions, and thereby making connections to policy optimization and regularization in continuous-time RL. To carry out this idea, we lay out a new policy optimization framework for continuous-time RL, and illustrate its potential in enhancing the value networks design space via leveraging the structural property of diffusion models. We validate the advantages of our method by experiments in downstream tasks of fine-tuning large-scale Text2Image models of Stable Diffusion v1.5.

Title: Texture Image Synthesis Using Spatial GAN Based on Vision Transformers

• Authors: Elahe Salari, Zohreh Azimifar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01842
• Pdf URL: https://arxiv.org/pdf/2502.01842
• Copy Paste: [[2502.01842]] Texture Image Synthesis Using Spatial GAN Based on Vision Transformers(https://arxiv.org/abs/2502.01842)
• Keywords: generative
• Abstract: Texture synthesis is a fundamental task in computer vision, whose goal is to generate visually realistic and structurally coherent textures for a wide range of applications, from graphics to scientific simulations. While traditional methods like tiling and patch-based techniques often struggle with complex textures, recent advancements in deep learning have transformed this field. In this paper, we propose ViT-SGAN, a new hybrid model that fuses Vision Transformers (ViTs) with a Spatial Generative Adversarial Network (SGAN) to address the limitations of previous methods. By incorporating specialized texture descriptors such as mean-variance (mu, sigma) and textons into the self-attention mechanism of ViTs, our model achieves superior texture synthesis. This approach enhances the model's capacity to capture complex spatial dependencies, leading to improved texture quality that is superior to state-of-the-art models, especially for regular and irregular textures. Comparison experiments with metrics such as FID, IS, SSIM, and LPIPS demonstrate the substantial improvement of ViT-SGAN, which underlines its efficiency in generating diverse realistic textures.

Title: UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping

• Authors: Aashish Rai, Dilin Wang, Mihir Jain, Nikolaos Sarafianos, Arthur Chen, Srinath Sridhar, Aayush Prakash
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01846
• Pdf URL: https://arxiv.org/pdf/2502.01846
• Copy Paste: [[2502.01846]] UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping(https://arxiv.org/abs/2502.01846)
• Keywords: diffusion
• Abstract: 3D Gaussian Splatting (3DGS) has demonstrated superior quality in modeling 3D objects and scenes. However, generating 3DGS remains challenging due to their discrete, unstructured, and permutation-invariant nature. In this work, we present a simple yet effective method to overcome these challenges. We utilize spherical mapping to transform 3DGS into a structured 2D representation, termed UVGS. UVGS can be viewed as multi-channel images, with feature dimensions as a concatenation of Gaussian attributes such as position, scale, color, opacity, and rotation. We further find that these heterogeneous features can be compressed into a lower-dimensional (e.g., 3-channel) shared feature space using a carefully designed multi-branch network. The compressed UVGS can be treated as typical RGB images. Remarkably, we discover that typical VAEs trained with latent diffusion models can directly generalize to this new representation without additional training. Our novel representation makes it effortless to leverage foundational 2D models, such as diffusion models, to directly model 3DGS. Additionally, one can simply increase the 2D UV resolution to accommodate more Gaussians, making UVGS a scalable solution compared to typical 3D backbones. This approach immediately unlocks various novel generation applications of 3DGS by inherently utilizing the already developed superior 2D generation capabilities. In our experiments, we demonstrate various unconditional, conditional generation, and inpainting applications of 3DGS based on diffusion models, which were previously non-trivial.

Title: Foundation Model-Based Apple Ripeness and Size Estimation for Selective Harvesting

• Authors: Keyi Zhu, Jiajia Li, Kaixiang Zhang, Chaaran Arunachalam, Siddhartha Bhattacharya, Renfu Lu, Zhaojian Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01850
• Pdf URL: https://arxiv.org/pdf/2502.01850
• Copy Paste: [[2502.01850]] Foundation Model-Based Apple Ripeness and Size Estimation for Selective Harvesting(https://arxiv.org/abs/2502.01850)
• Keywords: foundation model
• Abstract: Harvesting is a critical task in the tree fruit industry, demanding extensive manual labor and substantial costs, and exposing workers to potential hazards. Recent advances in automated harvesting offer a promising solution by enabling efficient, cost-effective, and ergonomic fruit picking within tight harvesting windows. However, existing harvesting technologies often indiscriminately harvest all visible and accessible fruits, including those that are unripe or undersized. This study introduces a novel foundation model-based framework for efficient apple ripeness and size estimation. Specifically, we curated two public RGBD-based Fuji apple image datasets, integrating expanded annotations for ripeness ("Ripe" vs. "Unripe") based on fruit color and image capture dates. The resulting comprehensive dataset, Fuji-Ripeness-Size Dataset, includes 4,027 images and 16,257 annotated apples with ripeness and size labels. Using Grounding-DINO, a language-model-based object detector, we achieved robust apple detection and ripeness classification, outperforming other state-of-the-art models. Additionally, we developed and evaluated six size estimation algorithms, selecting the one with the lowest error and variation for optimal performance. The Fuji-Ripeness-Size Dataset and the apple detection and size estimation algorithms are made publicly available, which provides valuable benchmarks for future studies in automated and selective harvesting.

Title: Anomaly Detection via Autoencoder Composite Features and NCE

• Authors: Yalin Liao, Austin J. Brockmeier
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01920
• Pdf URL: https://arxiv.org/pdf/2502.01920
• Copy Paste: [[2502.01920]] Anomaly Detection via Autoencoder Composite Features and NCE(https://arxiv.org/abs/2502.01920)
• Keywords: generative, anomaly
• Abstract: Unsupervised anomaly detection is a challenging task. Autoencoders (AEs) or generative models are often employed to model the data distribution of normal inputs and subsequently identify anomalous, out-of-distribution inputs by high reconstruction error or low likelihood, respectively. However, AEs may generalize and achieve small reconstruction errors on abnormal inputs. We propose a decoupled training approach for anomaly detection that both an AE and a likelihood model trained with noise contrastive estimation (NCE). After training the AE, NCE estimates a probability density function, to serve as the anomaly score, on the joint space of the AE's latent representation combined with features of the reconstruction quality. To further reduce the false negative rate in NCE we systematically varying the reconstruction features to augment the training and optimize the contrastive Gaussian noise distribution. Experimental assessments on multiple benchmark datasets demonstrate that the proposed approach matches the performance of prevalent state-of-the-art anomaly detection algorithms.

Title: LAST SToP For Modeling Asynchronous Time Series

• Authors: Shubham Gupta, Thibaut Durand, Graham Taylor, Lilian W. Białokozowicz
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01922
• Pdf URL: https://arxiv.org/pdf/2502.01922
• Copy Paste: [[2502.01922]] LAST SToP For Modeling Asynchronous Time Series(https://arxiv.org/abs/2502.01922)
• Keywords: anomaly
• Abstract: We present a novel prompt design for Large Language Models (LLMs) tailored to Asynchronous Time Series. Unlike regular time series, which assume values at evenly spaced time points, asynchronous time series consist of timestamped events occurring at irregular intervals, each described in natural language. Our approach effectively utilizes the rich natural language of event descriptions, allowing LLMs to benefit from their broad world knowledge for reasoning across different domains and tasks. This allows us to extend the scope of asynchronous time series analysis beyond forecasting to include tasks like anomaly detection and data imputation. We further introduce Stochastic Soft Prompting, a novel prompt-tuning mechanism that significantly improves model performance, outperforming existing fine-tuning methods such as QLoRA. Through extensive experiments on real world datasets, we demonstrate that our approach achieves state-of-the-art performance across different tasks and datasets.

Title: Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approach

• Authors: Mohammed Alsakabi, Aidan Erickson, John M. Dolan, Ozan K. Tonguz
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.01940
• Pdf URL: https://arxiv.org/pdf/2502.01940
• Copy Paste: [[2502.01940]] Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approach(https://arxiv.org/abs/2502.01940)
• Keywords: generative
• Abstract: We present a cost-effective new approach for generating denser depth maps for Autonomous Driving (AD) and Autonomous Vehicles (AVs) by integrating the images obtained from deep neural network (DNN) 4D radar detectors with conventional camera RGB images. Our approach introduces a novel pixel positional encoding algorithm inspired by Bartlett's spatial spectrum estimation technique. This algorithm transforms both radar depth maps and RGB images into a unified pixel image subspace called the Spatial Spectrum, facilitating effective learning based on their similarities and differences. Our method effectively leverages high-resolution camera images to train radar depth map generative models, addressing the limitations of conventional radar detectors in complex vehicular environments, thus sharpening the radar output. We develop spectrum estimation algorithms tailored for radar depth maps and RGB images, a comprehensive training framework for data-driven generative models, and a camera-radar deployment scheme for AV operation. Our results demonstrate that our approach also outperforms the state-of-the-art (SOTA) by 27.95% in terms of Unidirectional Chamfer Distance (UCD).

Title: Gradient-Regularized Latent Space Modulation in Large Language Models for Structured Contextual Synthesis

• Authors: Derek Yotheringhay, Beatrix Nightingale, Maximilian Featherstone, Edmund Worthington, Hugo Ashdown
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.01979
• Pdf URL: https://arxiv.org/pdf/2502.01979
• Copy Paste: [[2502.01979]] Gradient-Regularized Latent Space Modulation in Large Language Models for Structured Contextual Synthesis(https://arxiv.org/abs/2502.01979)
• Keywords: generative
• Abstract: Generating structured textual content requires mechanisms that enforce coherence, stability, and adherence to predefined constraints while maintaining semantic fidelity. Conventional approaches often rely on rule-based heuristics or fine-tuning strategies that lack flexibility and generalizability across diverse tasks. The incorporation of Gradient-Regularized Latent Space Modulation (GRLSM) introduces a novel paradigm for guiding text generation through the application of structured constraints within the latent space. The integration of gradient-based regularization mitigates abrupt variations in latent representations, ensuring a smoother encoding process that enhances structural consistency and logical progression within generated sequences. Comparative evaluations demonstrate that latent space modulation leads to a reduction in perplexity, increased coherence scores, and improved structural alignment across multiple domains. Stability assessments further indicate that the imposition of spectral norm constraints facilitates more controlled variations in generated text, preserving semantic consistency under input perturbations. Empirical results confirm that structured latent space constraints not only refine the organization of generated outputs but also enhance interpretability through more predictable and reliable synthesis patterns. Performance metrics illustrate that the GRLSM framework substantially reduces structural inconsistencies while preserving the generative flexibility inherent in neural models.

Title: Generative Data Mining with Longtail-Guided Diffusion

• Authors: David S. Hayden, Mao Ye, Timur Garipov, Gregory P. Meyer, Carl Vondrick, Zhao Chen, Yuning Chai, Eric Wolff, Siddhartha S. Srinivasa
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01980
• Pdf URL: https://arxiv.org/pdf/2502.01980
• Copy Paste: [[2502.01980]] Generative Data Mining with Longtail-Guided Diffusion(https://arxiv.org/abs/2502.01980)
• Keywords: diffusion, generative
• Abstract: It is difficult to anticipate the myriad challenges that a predictive model will encounter once deployed. Common practice entails a reactive, cyclical approach: model deployment, data mining, and retraining. We instead develop a proactive longtail discovery process by imagining additional data during training. In particular, we develop general model-based longtail signals, including a differentiable, single forward pass formulation of epistemic uncertainty that does not impact model parameters or predictive performance but can flag rare or hard inputs. We leverage these signals as guidance to generate additional training data from a latent diffusion model in a process we call Longtail Guidance (LTG). Crucially, we can perform LTG without retraining the diffusion model or the predictive model, and we do not need to expose the predictive model to intermediate diffusion states. Data generated by LTG exhibit semantically meaningful variation, yield significant generalization improvements on image classification benchmarks, and can be analyzed to proactively discover, explain, and address conceptual gaps in a predictive model.

Title: T-SCEND: Test-time Scalable MCTS-enhanced Diffusion Model

• Authors: Tao Zhang, Jia-Shu Pan, Ruiqi Feng, Tailin Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01989
• Pdf URL: https://arxiv.org/pdf/2502.01989
• Copy Paste: [[2502.01989]] T-SCEND: Test-time Scalable MCTS-enhanced Diffusion Model(https://arxiv.org/abs/2502.01989)
• Keywords: diffusion
• Abstract: We introduce Test-time Scalable MCTS-enhanced Diffusion Model (T-SCEND), a novel framework that significantly improves diffusion model's reasoning capabilities with better energy-based training and scaling up test-time computation. We first show that naïvely scaling up inference budget for diffusion models yields marginal gain. To address this, the training of T-SCEND consists of a novel linear-regression negative contrastive learning objective to improve the performance-energy consistency of the energy landscape, and a KL regularization to reduce adversarial sampling. During inference, T-SCEND integrates the denoising process with a novel hybrid Monte Carlo Tree Search (hMCTS), which sequentially performs best-of-N random search and MCTS as denoising proceeds. On challenging reasoning tasks of Maze and Sudoku, we demonstrate the effectiveness of T-SCEND's training objective and scalable inference method. In particular, trained with Maze sizes of up to $6\times6$, our T-SCEND solves $88\%$ of Maze problems with much larger sizes of $15\times15$, while standard diffusion completely this http URL to reproduce the experiments can be found at this https URL.

Title: Rethinking Timesteps Samplers and Prediction Types

• Authors: Bin Xie, Gady Agam
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01990
• Pdf URL: https://arxiv.org/pdf/2502.01990
• Copy Paste: [[2502.01990]] Rethinking Timesteps Samplers and Prediction Types(https://arxiv.org/abs/2502.01990)
• Keywords: diffusion, generative
• Abstract: Diffusion models suffer from the huge consumption of time and resources to train. For example, diffusion models need hundreds of GPUs to train for several weeks for a high-resolution generative task to meet the requirements of an extremely large number of iterations and a large batch size. Training diffusion models become a millionaire's game. With limited resources that only fit a small batch size, training a diffusion model always fails. In this paper, we investigate the key reasons behind the difficulties of training diffusion models with limited resources. Through numerous experiments and demonstrations, we identified a major factor: the significant variation in the training losses across different timesteps, which can easily disrupt the progress made in previous iterations. Moreover, different prediction types of $x_0$ exhibit varying effectiveness depending on the task and timestep. We hypothesize that using a mixed-prediction approach to identify the most accurate $x_0$ prediction type could potentially serve as a breakthrough in addressing this issue. In this paper, we outline several challenges and insights, with the hope of inspiring further research aimed at tackling the limitations of training diffusion models with constrained resources, particularly for high-resolution tasks.

Title: Can LLMs Assist Annotators in Identifying Morality Frames? -- Case Study on Vaccination Debate on Social Media

• Authors: Tunazzina Islam, Dan Goldwasser
• Subjects: cs.CL, cs.AI, cs.CY, cs.HC, cs.SI
• Abstract URL: https://arxiv.org/abs/2502.01991
• Pdf URL: https://arxiv.org/pdf/2502.01991
• Copy Paste: [[2502.01991]] Can LLMs Assist Annotators in Identifying Morality Frames? -- Case Study on Vaccination Debate on Social Media(https://arxiv.org/abs/2502.01991)
• Keywords: in-context
• Abstract: Nowadays, social media is pivotal in shaping public discourse, especially on polarizing issues like vaccination, where diverse moral perspectives influence individual opinions. In NLP, data scarcity and complexity of psycholinguistic tasks such as identifying morality frames makes relying solely on human annotators costly, time-consuming, and prone to inconsistency due to cognitive load. To address these issues, we leverage large language models (LLMs), which are adept at adapting new tasks through few-shot learning, utilizing a handful of in-context examples coupled with explanations that connect examples to task principles. Our research explores LLMs' potential to assist human annotators in identifying morality frames within vaccination debates on social media. We employ a two-step process: generating concepts and explanations with LLMs, followed by human evaluation using a "think-aloud" tool. Our study shows that integrating LLMs into the annotation process enhances accuracy, reduces task difficulty, lowers cognitive load, suggesting a promising avenue for human-AI collaboration in complex psycholinguistic tasks.

Title: One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation

• Authors: Jianze Li, Jiezhang Cao, Yong Guo, Wenbo Li, Yulun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01993
• Pdf URL: https://arxiv.org/pdf/2502.01993
• Copy Paste: [[2502.01993]] One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation(https://arxiv.org/abs/2502.01993)
• Keywords: diffusion
• Abstract: Diffusion models (DMs) have significantly advanced the development of real-world image super-resolution (Real-ISR), but the computational cost of multi-step diffusion models limits their application. One-step diffusion models generate high-quality images in a one sampling step, greatly reducing computational overhead and inference latency. However, most existing one-step diffusion methods are constrained by the performance of the teacher model, where poor teacher performance results in image artifacts. To address this limitation, we propose FluxSR, a novel one-step diffusion Real-ISR technique based on flow matching models. We use the state-of-the-art diffusion model FLUX.1-dev as both the teacher model and the base model. First, we introduce Flow Trajectory Distillation (FTD) to distill a multi-step flow matching model into a one-step Real-ISR. Second, to improve image realism and address high-frequency artifact issues in generated images, we propose TV-LPIPS as a perceptual loss and introduce Attention Diversification Loss (ADL) as a regularization term to reduce token similarity in transformer, thereby eliminating high-frequency artifacts. Comprehensive experiments demonstrate that our method outperforms existing one-step diffusion-based Real-ISR methods. The code and model will be released at this https URL.

Title: Analytical Lyapunov Function Discovery: An RL-based Generative Approach

• Authors: Haohan Zou, Jie Feng, Hao Zhao, Yuanyuan Shi
• Subjects: cs.LG, cs.AI, cs.SC, eess.SY
• Abstract URL: https://arxiv.org/abs/2502.02014
• Pdf URL: https://arxiv.org/pdf/2502.02014
• Copy Paste: [[2502.02014]] Analytical Lyapunov Function Discovery: An RL-based Generative Approach(https://arxiv.org/abs/2502.02014)
• Keywords: generative
• Abstract: Despite advances in learning-based methods, finding valid Lyapunov functions for nonlinear dynamical systems remains challenging. Current neural network approaches face two main issues: challenges in scalable verification and limited interpretability. To address these, we propose an end-to-end framework using transformers to construct analytical Lyapunov functions (local), which simplifies formal verification, enhances interpretability, and provides valuable insights for control engineers. Our framework consists of a transformer-based trainer that generates candidate Lyapunov functions and a falsifier that verifies candidate expressions and refines the model via risk-seeking policy gradient. Unlike Alfarano et al. (2024), which utilizes pre-training and seeks global Lyapunov functions for low-dimensional systems, our model is trained from scratch via reinforcement learning (RL) and succeeds in finding local Lyapunov functions for high-dimensional and non-polynomial systems. Given the analytical nature of the candidates, we employ efficient optimization methods for falsification during training and formal verification tools for the final verification. We demonstrate the efficiency of our approach on a range of nonlinear dynamical systems with up to ten dimensions and show that it can discover Lyapunov functions not previously identified in the control literature.

Title: A Periodic Bayesian Flow for Material Generation

• Authors: Hanlin Wu, Yuxuan Song, Jingjing Gong, Ziyao Cao, Yawen Ouyang, Jianbing Zhang, Hao Zhou, Wei-Ying Ma, Jingjing Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.02016
• Pdf URL: https://arxiv.org/pdf/2502.02016
• Copy Paste: [[2502.02016]] A Periodic Bayesian Flow for Material Generation(https://arxiv.org/abs/2502.02016)
• Keywords: diffusion, generative
• Abstract: Generative modeling of crystal data distribution is an important yet challenging task due to the unique periodic physical symmetry of crystals. Diffusion-based methods have shown early promise in modeling crystal distribution. More recently, Bayesian Flow Networks were introduced to aggregate noisy latent variables, resulting in a variance-reduced parameter space that has been shown to be advantageous for modeling Euclidean data distributions with structural constraints (Song et al., 2023). Inspired by this, we seek to unlock its potential for modeling variables located in non-Euclidean manifolds e.g. those within crystal structures, by overcoming challenging theoretical issues. We introduce CrysBFN, a novel crystal generation method by proposing a periodic Bayesian flow, which essentially differs from the original Gaussian-based BFN by exhibiting non-monotonic entropy dynamics. To successfully realize the concept of periodic Bayesian flow, CrysBFN integrates a new entropy conditioning mechanism and empirically demonstrates its significance compared to time-conditioning. Extensive experiments over both crystal ab initio generation and crystal structure prediction tasks demonstrate the superiority of CrysBFN, which consistently achieves new state-of-the-art on all benchmarks. Surprisingly, we found that CrysBFN enjoys a significant improvement in sampling efficiency, e.g., ~100x speedup 10 v.s. 2000 steps network forwards) compared with previous diffusion-based methods on MP-20 dataset. Code is available at this https URL.

Title: ContinuouSP: Generative Model for Crystal Structure Prediction with Invariance and Continuity

• Authors: Yuji Tone, Masatoshi Hanai, Mitsuaki Kawamura, Kenjiro Taura, Toyotaro Suzumura
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2502.02026
• Pdf URL: https://arxiv.org/pdf/2502.02026
• Copy Paste: [[2502.02026]] ContinuouSP: Generative Model for Crystal Structure Prediction with Invariance and Continuity(https://arxiv.org/abs/2502.02026)
• Keywords: generative
• Abstract: The discovery of new materials using crystal structure prediction (CSP) based on generative machine learning models has become a significant research topic in recent years. In this paper, we study invariance and continuity in the generative machine learning for CSP. We propose a new model, called ContinuouSP, which effectively handles symmetry and periodicity in crystals. We clearly formulate the invariance and the continuity, and construct a model based on the energy-based model. Our preliminary evaluation demonstrates the effectiveness of this model with the CSP task.

Title: CASIM: Composite Aware Semantic Injection for Text to Motion Generation

• Authors: Che-Jui Chang, Qingze Tony Liu, Honglu Zhou, Vladimir Pavlovic, Mubbasir Kapadia
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.02063
• Pdf URL: https://arxiv.org/pdf/2502.02063
• Copy Paste: [[2502.02063]] CASIM: Composite Aware Semantic Injection for Text to Motion Generation(https://arxiv.org/abs/2502.02063)
• Keywords: diffusion, generative
• Abstract: Recent advances in generative modeling and tokenization have driven significant progress in text-to-motion generation, leading to enhanced quality and realism in generated motions. However, effectively leveraging textual information for conditional motion generation remains an open challenge. We observe that current approaches, primarily relying on fixed-length text embeddings (e.g., CLIP) for global semantic injection, struggle to capture the composite nature of human motion, resulting in suboptimal motion quality and controllability. To address this limitation, we propose the Composite Aware Semantic Injection Mechanism (CASIM), comprising a composite-aware semantic encoder and a text-motion aligner that learns the dynamic correspondence between text and motion tokens. Notably, CASIM is model and representation-agnostic, readily integrating with both autoregressive and diffusion-based methods. Experiments on HumanML3D and KIT benchmarks demonstrate that CASIM consistently improves motion quality, text-motion alignment, and retrieval scores across state-of-the-art methods. Qualitative analyses further highlight the superiority of our composite-aware approach over fixed-length semantic injection, enabling precise motion control from text prompts and stronger generalization to unseen text inputs.

Title: Position Paper: Building Trust in Synthetic Data for Clinical AI

• Authors: Krishan Agyakari Raja Babu, Supriti Mulay, Om Prabhu, Mohanasankar Sivaprakasam
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02076
• Pdf URL: https://arxiv.org/pdf/2502.02076
• Copy Paste: [[2502.02076]] Position Paper: Building Trust in Synthetic Data for Clinical AI(https://arxiv.org/abs/2502.02076)
• Keywords: generative
• Abstract: Deep generative models and synthetic medical data have shown significant promise in addressing key challenges in healthcare, such as privacy concerns, data bias, and the scarcity of realistic datasets. While research in this area has grown rapidly and demonstrated substantial theoretical potential, its practical adoption in clinical settings remains limited. Despite the benefits synthetic data offers, questions surrounding its reliability and credibility persist, leading to a lack of trust among clinicians. This position paper argues that fostering trust in synthetic medical data is crucial for its clinical adoption. It aims to spark a discussion on the viability of synthetic medical data in clinical practice, particularly in the context of current advancements in AI. We present empirical evidence from brain tumor segmentation to demonstrate that the quality, diversity, and proportion of synthetic data directly impact trust in clinical AI models. Our findings provide insights to improve the deployment and acceptance of synthetic data-driven AI systems in real-world clinical workflows.

Title: IPO: Iterative Preference Optimization for Text-to-Video Generation

• Authors: Xiaomeng Yang, Zhiyu Tan, Xuecheng Nie, Hao Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.02088
• Pdf URL: https://arxiv.org/pdf/2502.02088
• Copy Paste: [[2502.02088]] IPO: Iterative Preference Optimization for Text-to-Video Generation(https://arxiv.org/abs/2502.02088)
• Keywords: foundation model
• Abstract: Video foundation models have achieved significant advancement with the help of network upgrade as well as model scale-up. However, they are still hard to meet requirements of applications due to unsatisfied generation quality. To solve this problem, we propose to align video foundation models with human preferences from the perspective of post-training in this paper. Consequently, we introduce an Iterative Preference Optimization strategy to enhance generated video quality by incorporating human feedback. Specifically, IPO exploits a critic model to justify video generations for pairwise ranking as in Direct Preference Optimization or point-wise scoring as in Kahneman-Tversky Optimization. Given this, IPO optimizes video foundation models with guidance of signals from preference feedback, which helps improve generated video quality in subject consistency, motion smoothness and aesthetic quality, etc. In addition, IPO incorporates the critic model with the multi-modality large language model, which enables it to automatically assign preference labels without need of retraining or relabeling. In this way, IPO can efficiently perform multi-round preference optimization in an iterative manner, without the need of tediously manual labeling. Comprehensive experiments demonstrate that the proposed IPO can effectively improve the video generation quality of a pretrained model and help a model with only 2B parameters surpass the one with 5B parameters. Besides, IPO achieves new state-of-the-art performance on VBench benchmark. We will release our source codes, models as well as dataset to advance future research and applications.

Title: Dual-Flow: Transferable Multi-Target, Instance-Agnostic Attacks via In-the-wild Cascading Flow Optimization

• Authors: Yixiao Chen, Shikun Sun, Jianshu Li, Ruoyu Li, Zhe Li, Junliang Xing
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02096
• Pdf URL: https://arxiv.org/pdf/2502.02096
• Copy Paste: [[2502.02096]] Dual-Flow: Transferable Multi-Target, Instance-Agnostic Attacks via In-the-wild Cascading Flow Optimization(https://arxiv.org/abs/2502.02096)
• Keywords: generative
• Abstract: Adversarial attacks are widely used to evaluate model robustness, and in black-box scenarios, the transferability of these attacks becomes crucial. Existing generator-based attacks have excellent generalization and transferability due to their instance-agnostic nature. However, when training generators for multi-target tasks, the success rate of transfer attacks is relatively low due to the limitations of the model's capacity. To address these challenges, we propose a novel Dual-Flow framework for multi-target instance-agnostic adversarial attacks, utilizing Cascading Distribution Shift Training to develop an adversarial velocity function. Extensive experiments demonstrate that Dual-Flow significantly improves transferability over previous multi-target generative attacks. For example, it increases the success rate from Inception-v3 to ResNet-152 by 34.58%. Furthermore, our attack method, such as adversarially trained models, shows substantially stronger robustness against defense mechanisms.

Title: BRIDLE: Generalized Self-supervised Learning with Quantization

• Authors: Hoang M. Nguyen, Satya N. Shukla, Qiang Zhang, Hanchao Yu, Sreya D. Roy, Taipeng Tian, Lingjiong Zhu, Yuchen Liu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02118
• Pdf URL: https://arxiv.org/pdf/2502.02118
• Copy Paste: [[2502.02118]] BRIDLE: Generalized Self-supervised Learning with Quantization(https://arxiv.org/abs/2502.02118)
• Keywords: self-supervised
• Abstract: Self-supervised learning has been a powerful approach for learning meaningful representations from unlabeled data across various domains, reducing the reliance on large labeled datasets. Inspired by BERT's success in capturing deep bidirectional contexts in natural language processing, similar frameworks have been adapted to other modalities such as audio, with models like BEATs extending the bidirectional training paradigm to audio signals using vector quantization (VQ). However, these frameworks face challenges, notably their dependence on a single codebook for quantization, which may not capture the complex, multifaceted nature of signals. In addition, inefficiencies in codebook utilization lead to underutilized code vectors. To address these limitations, we introduce BRIDLE (Bidirectional Residual Quantization Interleaved Discrete Learning Encoder), a self-supervised encoder pretraining framework that incorporates residual quantization (RQ) into the bidirectional training process, and is generalized for pretraining with audio, image, and video. Using multiple hierarchical codebooks, RQ enables fine-grained discretization in the latent space, enhancing representation quality. BRIDLE involves an interleaved training procedure between the encoder and tokenizer. We evaluate BRIDLE on audio understanding tasks using classification benchmarks, achieving state-of-the-art results, and demonstrate competitive performance on image classification and video classification tasks, showing consistent improvements over traditional VQ methods in downstream performance.

Title: On the Guidance of Flow Matching

• Authors: Ruiqi Feng, Tailin Wu, Chenglei Yu, Wenhao Deng, Peiyan Hu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02150
• Pdf URL: https://arxiv.org/pdf/2502.02150
• Copy Paste: [[2502.02150]] On the Guidance of Flow Matching(https://arxiv.org/abs/2502.02150)
• Keywords: diffusion, generative
• Abstract: Flow matching has shown state-of-the-art performance in various generative tasks, ranging from image generation to decision-making, where guided generation is pivotal. However, the guidance of flow matching is more general than and thus substantially different from that of its predecessor, diffusion models. Therefore, the challenge in guidance for general flow matching remains largely underexplored. In this paper, we propose the first framework of general guidance for flow matching. From this framework, we derive a family of guidance techniques that can be applied to general flow matching. These include a new training-free asymptotically exact guidance, novel training losses for training-based guidance, and two classes of approximate guidance that cover classical gradient guidance methods as special cases. We theoretically investigate these different methods to give a practical guideline for choosing suitable methods in different scenarios. Experiments on synthetic datasets, image inverse problems, and offline reinforcement learning demonstrate the effectiveness of our proposed guidance methods and verify the correctness of our flow matching guidance framework. Code to reproduce the experiments can be found at this https URL.

Title: Generative Kernel Spectral Clustering

• Authors: David Winant, Sonny Achten, Johan A. K. Suykens
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02185
• Pdf URL: https://arxiv.org/pdf/2502.02185
• Copy Paste: [[2502.02185]] Generative Kernel Spectral Clustering(https://arxiv.org/abs/2502.02185)
• Keywords: generative
• Abstract: Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Title: ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusion

• Authors: Nissim Maruani, Wang Yifan, Matthew Fisher, Pierre Alliez, Mathieu Desbrun
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.02187
• Pdf URL: https://arxiv.org/pdf/2502.02187
• Copy Paste: [[2502.02187]] ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusion(https://arxiv.org/abs/2502.02187)
• Keywords: diffusion, generative
• Abstract: This paper proposes ShapeShifter, a new 3D generative model that learns to synthesize shape variations based on a single reference model. While generative methods for 3D objects have recently attracted much attention, current techniques often lack geometric details and/or require long training times and large resources. Our approach remedies these issues by combining sparse voxel grids and point, normal, and color sampling within a multiscale neural architecture that can be trained efficiently and in parallel. We show that our resulting variations better capture the fine details of their original input and can handle more general types of surfaces than previous SDF-based methods. Moreover, we offer interactive generation of 3D shape variants, allowing more human control in the design loop if needed.

Title: From Uncertain to Safe: Conformal Fine-Tuning of Diffusion Models for Safe PDE Control

• Authors: Peiyan Hu, Xiaowei Qian, Wenhao Deng, Rui Wang, Haodong Feng, Ruiqi Feng, Tao Zhang, Long Wei, Yue Wang, Zhi-Ming Ma, Tailin Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02205
• Pdf URL: https://arxiv.org/pdf/2502.02205
• Copy Paste: [[2502.02205]] From Uncertain to Safe: Conformal Fine-Tuning of Diffusion Models for Safe PDE Control(https://arxiv.org/abs/2502.02205)
• Keywords: diffusion
• Abstract: The application of deep learning for partial differential equation (PDE)-constrained control is gaining increasing attention. However, existing methods rarely consider safety requirements crucial in real-world applications. To address this limitation, we propose Safe Diffusion Models for PDE Control (SafeDiffCon), which introduce the uncertainty quantile as model uncertainty quantification to achieve optimal control under safety constraints through both post-training and inference phases. Firstly, our approach post-trains a pre-trained diffusion model to generate control sequences that better satisfy safety constraints while achieving improved control objectives via a reweighted diffusion loss, which incorporates the uncertainty quantile estimated using conformal prediction. Secondly, during inference, the diffusion model dynamically adjusts both its generation process and parameters through iterative guidance and fine-tuning, conditioned on control targets while simultaneously integrating the estimated uncertainty quantile. We evaluate SafeDiffCon on three control tasks: 1D Burgers' equation, 2D incompressible fluid, and controlled nuclear fusion problem. Results demonstrate that SafeDiffCon is the only method that satisfies all safety constraints, whereas other classical and deep learning baselines fail. Furthermore, while adhering to safety constraints, SafeDiffCon achieves the best control performance.

Title: InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration

• Authors: Senmao Li, Kai Wang, Joost van de Weijer, Fahad Shahbaz Khan, Chun-Le Guo, Shiqi Yang, Yaxing Wang, Jian Yang, Ming-Ming Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02215
• Pdf URL: https://arxiv.org/pdf/2502.02215
• Copy Paste: [[2502.02215]] InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration(https://arxiv.org/abs/2502.02215)
• Keywords: diffusion
• Abstract: Diffusion priors have been used for blind face restoration (BFR) by fine-tuning diffusion models (DMs) on restoration datasets to recover low-quality images. However, the naive application of DMs presents several key limitations. (i) The diffusion prior has inferior semantic consistency (e.g., ID, structure and color.), increasing the difficulty of optimizing the BFR model; (ii) reliance on hundreds of denoising iterations, preventing the effective cooperation with perceptual losses, which is crucial for faithful restoration. Observing that the latent consistency model (LCM) learns consistency noise-to-data mappings on the ODE-trajectory and therefore shows more semantic consistency in the subject identity, structural information and color preservation, we propose InterLCM to leverage the LCM for its superior semantic consistency and efficiency to counter the above issues. Treating low-quality images as the intermediate state of LCM, InterLCM achieves a balance between fidelity and quality by starting from earlier LCM steps. LCM also allows the integration of perceptual loss during training, leading to improved restoration quality, particularly in real-world scenarios. To mitigate structural and semantic uncertainties, InterLCM incorporates a Visual Module to extract visual features and a Spatial Encoder to capture spatial details, enhancing the fidelity of restored images. Extensive experiments demonstrate that InterLCM outperforms existing approaches in both synthetic and real-world datasets while also achieving faster inference speed.

Title: Flatten Graphs as Sequences: Transformers are Scalable Graph Generators

• Authors: Dexiong Chen, Markus Krimmel, Karsten Borgwardt
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.02216
• Pdf URL: https://arxiv.org/pdf/2502.02216
• Copy Paste: [[2502.02216]] Flatten Graphs as Sequences: Transformers are Scalable Graph Generators(https://arxiv.org/abs/2502.02216)
• Keywords: diffusion, foundation model
• Abstract: We introduce AutoGraph, a novel autoregressive framework for generating large attributed graphs using decoder-only transformers. At the core of our approach is a reversible "flattening" process that transforms graphs into random sequences. By sampling and learning from these sequences, AutoGraph enables transformers to model and generate complex graph structures in a manner akin to natural language. In contrast to diffusion models that rely on computationally intensive node features, our approach operates exclusively on these sequences. The sampling complexity and sequence length scale linearly with the number of edges, making AutoGraph highly scalable for generating large sparse graphs. Empirically, AutoGraph achieves state-of-the-art performance across diverse synthetic and molecular graph generation benchmarks, while delivering a 100-fold generation and a 3-fold training speedup compared to leading diffusion models. Additionally, it demonstrates promising transfer capabilities and supports substructure-conditioned generation without additional fine-tuning. By extending language modeling techniques to graph generation, this work paves the way for developing graph foundation models.

Title: Exploring the latent space of diffusion models directly through singular value decomposition

• Authors: Li Wang, Boyan Gao, Yanran Li, Zhao Wang, Xiaosong Yang, David A. Clifton, Jun Xiao
• Subjects: cs.CV, cs.AI, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.02225
• Pdf URL: https://arxiv.org/pdf/2502.02225
• Copy Paste: [[2502.02225]] Exploring the latent space of diffusion models directly through singular value decomposition(https://arxiv.org/abs/2502.02225)
• Keywords: diffusion
• Abstract: Despite the groundbreaking success of diffusion models in generating high-fidelity images, their latent space remains relatively under-explored, even though it holds significant promise for enabling versatile and interpretable image editing capabilities. The complicated denoising trajectory and high dimensionality of the latent space make it extremely challenging to interpret. Existing methods mainly explore the feature space of U-Net in Diffusion Models (DMs) instead of the latent space itself. In contrast, we directly investigate the latent space via Singular Value Decomposition (SVD) and discover three useful properties that can be used to control generation results without the requirements of data collection and maintain identity fidelity generated images. Based on these properties, we propose a novel image editing framework that is capable of learning arbitrary attributes from one pair of latent codes destined by text prompts in Stable Diffusion Models. To validate our approach, extensive experiments are conducted to demonstrate its effectiveness and flexibility in image editing. We will release our codes soon to foster further research and applications in this area.

Title: Evalita-LLM: Benchmarking Large Language Models on Italian

• Authors: Bernardo Magnini, Roberto Zanoli, Michele Resta, Martin Cimmino, Paolo Albano, Marco Madeddu, Viviana Patti
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.02289
• Pdf URL: https://arxiv.org/pdf/2502.02289
• Copy Paste: [[2502.02289]] Evalita-LLM: Benchmarking Large Language Models on Italian(https://arxiv.org/abs/2502.02289)
• Keywords: generative
• Abstract: We describe Evalita-LLM, a new benchmark designed to evaluate Large Language Models (LLMs) on Italian tasks. The distinguishing and innovative features of Evalita-LLM are the following: (i) all tasks are native Italian, avoiding issues of translating from Italian and potential cultural biases; (ii) in addition to well established multiple-choice tasks, the benchmark includes generative tasks, enabling more natural interaction with LLMs; (iii) all tasks are evaluated against multiple prompts, this way mitigating the model sensitivity to specific prompts and allowing a fairer and objective evaluation. We propose an iterative methodology, where candidate tasks and candidate prompts are validated against a set of LLMs used for development. We report experimental results from the benchmark's development phase, and provide performance statistics for several state-of-the-art LLMs.

Title: Density Ratio Estimation with Conditional Probability Paths

• Authors: Hanlin Yu, Arto Klami, Aapo Hyvärinen, Anna Korba, Omar Chehab
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02300
• Pdf URL: https://arxiv.org/pdf/2502.02300
• Copy Paste: [[2502.02300]] Density Ratio Estimation with Conditional Probability Paths(https://arxiv.org/abs/2502.02300)
• Keywords: generative
• Abstract: Density ratio estimation in high dimensions can be reframed as integrating a certain quantity, the time score, over probability paths which interpolate between the two densities. In practice, the time score has to be estimated based on samples from the two densities. However, existing methods for this problem remain computationally expensive and can yield inaccurate estimates. Inspired by recent advances in generative modeling, we introduce a novel framework for time score estimation, based on a conditioning variable. Choosing the conditioning variable judiciously enables a closed-form objective function. We demonstrate that, compared to previous approaches, our approach results in faster learning of the time score and competitive or better estimation accuracies of the density ratio on challenging tasks. Furthermore, we establish theoretical guarantees on the error of the estimated density ratio.

Title: UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training

• Authors: Jiawei Qin, Xucong Zhang, Yusuke Sugano
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02307
• Pdf URL: https://arxiv.org/pdf/2502.02307
• Copy Paste: [[2502.02307]] UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training(https://arxiv.org/abs/2502.02307)
• Keywords: self-supervised
• Abstract: Despite decades of research on data collection and model architectures, current gaze estimation models face significant challenges in generalizing across diverse data domains. While recent advances in self-supervised pre-training have shown remarkable potential for improving model generalization in various vision tasks, their effectiveness in gaze estimation remains unexplored due to the geometric nature of the gaze regression task. We propose UniGaze, which leverages large-scale, in-the-wild facial datasets through self-supervised pre-training for gaze estimation. We carefully curate multiple facial datasets that capture diverse variations in identity, lighting, background, and head poses. By directly applying Masked Autoencoder (MAE) pre-training on normalized face images with a Vision Transformer (ViT) backbone, our UniGaze learns appropriate feature representations within the specific input space required by downstream gaze estimation models. Through comprehensive experiments using challenging cross-dataset evaluation and novel protocols, including leave-one-dataset-out and joint-dataset settings, we demonstrate that UniGaze significantly improves generalization across multiple data domains while minimizing reliance on costly labeled data. The source code and pre-trained models will be released upon acceptance.

Title: DIME:Diffusion-Based Maximum Entropy Reinforcement Learning

• Authors: Onur Celik, Zechu Li, Denis Blessing, Ge Li, Daniel Palanicek, Jan Peters, Georgia Chalvatzaki, Gerhard Neumann
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02316
• Pdf URL: https://arxiv.org/pdf/2502.02316
• Copy Paste: [[2502.02316]] DIME:Diffusion-Based Maximum Entropy Reinforcement Learning(https://arxiv.org/abs/2502.02316)
• Keywords: diffusion
• Abstract: Maximum entropy reinforcement learning (MaxEnt-RL) has become the standard approach to RL due to its beneficial exploration properties. Traditionally, policies are parameterized using Gaussian distributions, which significantly limits their representational capacity. Diffusion-based policies offer a more expressive alternative, yet integrating them into MaxEnt-RL poses challenges--primarily due to the intractability of computing their marginal entropy. To overcome this, we propose Diffusion-Based Maximum Entropy RL (DIME). DIME leverages recent advances in approximate inference with diffusion models to derive a lower bound on the maximum entropy objective. Additionally, we propose a policy iteration scheme that provably converges to the optimal diffusion policy. Our method enables the use of expressive diffusion-based policies while retaining the principled exploration benefits of MaxEnt-RL, significantly outperforming other diffusion-based methods on challenging high-dimensional control benchmarks. It is also competitive with state-of-the-art non-diffusion based RL methods while requiring fewer algorithmic design choices and smaller update-to-data ratios, reducing computational complexity.

Title: SHIELD: APT Detection and Intelligent Explanation Using LLM

• Authors: Parth Atulbhai Gandhi, Prasanna N. Wudali, Yonatan Amaru, Yuval Elovici, Asaf Shabtai
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.02342
• Pdf URL: https://arxiv.org/pdf/2502.02342
• Copy Paste: [[2502.02342]] SHIELD: APT Detection and Intelligent Explanation Using LLM(https://arxiv.org/abs/2502.02342)
• Keywords: anomaly
• Abstract: Advanced persistent threats (APTs) are sophisticated cyber attacks that can remain undetected for extended periods, making their mitigation particularly challenging. Given their persistence, significant effort is required to detect them and respond effectively. Existing provenance-based attack detection methods often lack interpretability and suffer from high false positive rates, while investigation approaches are either supervised or limited to known attacks. To address these challenges, we introduce SHIELD, a novel approach that combines statistical anomaly detection and graph-based analysis with the contextual analysis capabilities of large language models (LLMs). SHIELD leverages the implicit knowledge of LLMs to uncover hidden attack patterns in provenance data, while reducing false positives and providing clear, interpretable attack descriptions. This reduces analysts' alert fatigue and makes it easier for them to understand the threat landscape. Our extensive evaluation demonstrates SHIELD's effectiveness and computational efficiency in real-world scenarios. SHIELD was shown to outperform state-of-the-art methods, achieving higher precision and recall. SHIELD's integration of anomaly detection, LLM-driven contextual analysis, and advanced graph-based correlation establishes a new benchmark for APT detection.

Title: Field Matching: an Electrostatic Paradigm to Generate and Transfer Data

• Authors: Alexander Kolesov, Manukhov Stepan, Vladimir V. Palyulin, Alexander Korotin
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02367
• Pdf URL: https://arxiv.org/pdf/2502.02367
• Copy Paste: [[2502.02367]] Field Matching: an Electrostatic Paradigm to Generate and Transfer Data(https://arxiv.org/abs/2502.02367)
• Keywords: generative
• Abstract: We propose Electrostatic Field Matching (EFM), a novel method that is suitable for both generative modeling and distribution transfer tasks. Our approach is inspired by the physics of an electrical capacitor. We place source and target distributions on the capacitor plates and assign them positive and negative charges, respectively. We then learn the electrostatic field of the capacitor using a neural network approximator. To map the distributions to each other, we start at one plate of the capacitor and move the samples along the learned electrostatic field lines until they reach the other plate. We theoretically justify that this approach provably yields the distribution transfer. In practice, we demonstrate the performance of our EFM in toy and image data experiments.

Title: CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning

• Authors: Jianfeng Pan, Senyou Deng, Shaomang Huang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.02390
• Pdf URL: https://arxiv.org/pdf/2502.02390
• Copy Paste: [[2502.02390]] CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning(https://arxiv.org/abs/2502.02390)
• Keywords: generative
• Abstract: Research on LLM technologies is rapidly emerging, with most of them employing a 'fast thinking' approach to inference. Most LLMs generate the final result based solely on a single query and LLM's reasoning capabilities. However, with the advent of OpenAI-o1, 'slow thinking' techniques have garnered increasing attention because its process is closer to the human thought process. Inspired by the human ability to constantly associate and replenish knowledge during thinking, we developed the novel Chain-of-Associated-Thoughts (CoAT) framework, which introduces an innovative synergy between the Monte Carlo Tree Search (MCTS) algorithm and a dynamic mechanism for integrating new key information, termed 'associative memory'. By combining the structured exploration capabilities of MCTS with the adaptive learning capacity of associative memory, CoAT significantly expands the LLM search space, enabling our framework to explore diverse reasoning pathways and dynamically update its knowledge base in real-time. This allows the framework to not only revisit and refine earlier inferences but also adaptively incorporate evolving information, ensuring that the final output is both accurate and comprehensive. To validate the effectiveness of our framework, we conducted extensive experiments across a range of generative and reasoning tasks. These experiments demonstrated that our framework outperforms conventional inference processes on accuracy, coherence, and diversity. The framework's ability to iteratively expand its search space while retaining contextually relevant information results.

Title: Privacy Amplification by Structured Subsampling for Deep Differentially Private Time Series Forecasting

• Authors: Jan Schuchardt, Mina Dalirrooyfard, Jed Guzelkabaagac, Anderson Schneider, Yuriy Nevmyvaka, Stephan Günnemann
• Subjects: cs.LG, cs.CR, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.02410
• Pdf URL: https://arxiv.org/pdf/2502.02410
• Copy Paste: [[2502.02410]] Privacy Amplification by Structured Subsampling for Deep Differentially Private Time Series Forecasting(https://arxiv.org/abs/2502.02410)
• Keywords: self-supervised
• Abstract: Many forms of sensitive data, such as web traffic, mobility data, or hospital occupancy, are inherently sequential. The standard method for training machine learning models while ensuring privacy for units of sensitive information, such as individual hospital visits, is differentially private stochastic gradient descent (DP-SGD). However, we observe in this work that the formal guarantees of DP-SGD are incompatible with timeseries-specific tasks like forecasting, since they rely on the privacy amplification attained by training on small, unstructured batches sampled from an unstructured dataset. In contrast, batches for forecasting are generated by (1) sampling sequentially structured time series from a dataset, (2) sampling contiguous subsequences from these series, and (3) partitioning them into context and ground-truth forecast windows. We theoretically analyze the privacy amplification attained by this structured subsampling to enable the training of forecasting models with sound and tight event- and user-level privacy guarantees. Towards more private models, we additionally prove how data augmentation amplifies privacy in self-supervised training of sequence models. Our empirical evaluation demonstrates that amplification by structured subsampling enables the training of forecasting models with strong formal privacy guarantees.

Title: Towards Fast Graph Generation via Autoregressive Noisy Filtration Modeling

• Authors: Markus Krimmel, Jenna Wiens, Karsten Borgwardt, Dexiong Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02415
• Pdf URL: https://arxiv.org/pdf/2502.02415
• Copy Paste: [[2502.02415]] Towards Fast Graph Generation via Autoregressive Noisy Filtration Modeling(https://arxiv.org/abs/2502.02415)
• Keywords: diffusion, generative
• Abstract: Graph generative models often face a critical trade-off between learning complex distributions and achieving fast generation speed. We introduce Autoregressive Noisy Filtration Modeling (ANFM), a novel approach that addresses both challenges. ANFM leverages filtration, a concept from topological data analysis, to transform graphs into short sequences of monotonically increasing subgraphs. This formulation extends the sequence families used in previous autoregressive models. To learn from these sequences, we propose a novel autoregressive graph mixer model. Our experiments suggest that exposure bias might represent a substantial hurdle in autoregressive graph generation and we introduce two mitigation strategies to address it: noise augmentation and a reinforcement learning approach. Incorporating these techniques leads to substantial performance gains, making ANFM competitive with state-of-the-art diffusion models across diverse synthetic and real-world datasets. Notably, ANFM produces remarkably short sequences, achieving a 100-fold speedup in generation time compared to diffusion models. This work marks a significant step toward high-throughput graph generation.

Title: TransformDAS: Mapping {\Phi}-OTDR Signals to Riemannian Manifold for Robust Classification

• Authors: Jiaju Kang, Puyu Han, Yang Chun, Xu Wang, Luqi Gong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02428
• Pdf URL: https://arxiv.org/pdf/2502.02428
• Copy Paste: [[2502.02428]] TransformDAS: Mapping {\Phi}-OTDR Signals to Riemannian Manifold for Robust Classification(https://arxiv.org/abs/2502.02428)
• Keywords: generative
• Abstract: Phase-sensitive optical time-domain reflectometry ({\Phi}-OTDR) is a widely used distributed fiber optic sensing system in engineering. Machine learning algorithms for {\Phi}-OTDR event classification require high volumes and quality of datasets; however, high-quality datasets are currently extremely scarce in the field, leading to a lack of robustness in models, which is manifested by higher false alarm rates in real-world scenarios. One promising approach to address this issue is to augment existing data using generative models combined with a small amount of real-world data. We explored mapping both {\Phi}-OTDR features in a GAN-based generative pipeline and signal features in a Transformer classifier to hyperbolic space to seek more effective model generalization. The results indicate that state-of-the-art models exhibit stronger generalization performance and lower false alarm rates in real-world scenarios when trained on augmented datasets. TransformDAS, in particular, demonstrates the best classification performance, highlighting the benefits of Riemannian manifold mapping in {\Phi}-OTDR data generation and model classification.

Title: Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models

• Authors: Haoran Ye, Tianze Zhang, Yuhang Xie, Liyuan Zhang, Yuanyi Ren, Xin Zhang, Guojie Song
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.02444
• Pdf URL: https://arxiv.org/pdf/2502.02444
• Copy Paste: [[2502.02444]] Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models(https://arxiv.org/abs/2502.02444)
• Keywords: generative
• Abstract: Values are core drivers of individual and collective perception, cognition, and behavior. Value systems, such as Schwartz's Theory of Basic Human Values, delineate the hierarchy and interplay among these values, enabling cross-disciplinary investigations into decision-making and societal dynamics. Recently, the rise of Large Language Models (LLMs) has raised concerns regarding their elusive intrinsic values. Despite growing efforts in evaluating, understanding, and aligning LLM values, a psychologically grounded LLM value system remains underexplored. This study addresses the gap by introducing the Generative Psycho-Lexical Approach (GPLA), a scalable, adaptable, and theoretically informed method for constructing value systems. Leveraging GPLA, we propose a psychologically grounded five-factor value system tailored for LLMs. For systematic validation, we present three benchmarking tasks that integrate psychological principles with cutting-edge AI priorities. Our results reveal that the proposed value system meets standard psychological criteria, better captures LLM values, improves LLM safety prediction, and enhances LLM alignment, when compared to the canonical Schwartz's values.

Title: Sparse Data Generation Using Diffusion Models

• Authors: Phil Ostheimer, Mayank Nagda, Marius Kloft, Sophie Fellenz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02448
• Pdf URL: https://arxiv.org/pdf/2502.02448
• Copy Paste: [[2502.02448]] Sparse Data Generation Using Diffusion Models(https://arxiv.org/abs/2502.02448)
• Keywords: diffusion
• Abstract: Sparse data is ubiquitous, appearing in numerous domains, from economics and recommender systems to astronomy and biomedical sciences. However, efficiently and realistically generating sparse data remains a significant challenge. We introduce Sparse Data Diffusion (SDD), a novel method for generating sparse data. SDD extends continuous state-space diffusion models by explicitly modeling sparsity through the introduction of Sparsity Bits. Empirical validation on image data from various domains-including two scientific applications, physics and biology-demonstrates that SDD achieves high fidelity in representing data sparsity while preserving the quality of the generated data.

Title: Personalization Toolkit: Training Free Personalization of Large Vision Language Models

• Authors: Soroush Seifi, Vaggelis Dorovatas, Daniel Olmeda Reino, Rahaf Aljundi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02452
• Pdf URL: https://arxiv.org/pdf/2502.02452
• Copy Paste: [[2502.02452]] Personalization Toolkit: Training Free Personalization of Large Vision Language Models(https://arxiv.org/abs/2502.02452)
• Keywords: foundation model
• Abstract: Large Vision Language Models (LVLMs) have significant potential to deliver personalized assistance by adapting to individual users' unique needs and preferences. Personalization of LVLMs is an emerging area that involves customizing models to recognize specific object instances and provide tailored responses. However, existing approaches rely on time-consuming test-time training for each user and object, rendering them impractical. This paper proposes a novel, training-free approach to LVLM personalization by leveraging pre-trained vision foundation models to extract distinct features, retrieval-augmented generation (RAG) techniques to recognize instances in the visual input, and visual prompting methods. Our model-agnostic vision toolkit enables flexible and efficient personalization without extensive retraining. We demonstrate state-of-the-art results, outperforming conventional training-based approaches and establish a new standard for LVLM personalization.

Title: Towards Consistent and Controllable Image Synthesis for Face Editing

• Authors: Mengting Wei, Tuomas Varanka, Yante Li, Xingxun Jiang, Huai-Qian Khor, Guoying Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02465
• Pdf URL: https://arxiv.org/pdf/2502.02465
• Copy Paste: [[2502.02465]] Towards Consistent and Controllable Image Synthesis for Face Editing(https://arxiv.org/abs/2502.02465)
• Keywords: diffusion
• Abstract: Current face editing methods mainly rely on GAN-based techniques, but recent focus has shifted to diffusion-based models due to their success in image reconstruction. However, diffusion models still face challenges in manipulating fine-grained attributes and preserving consistency of attributes that should remain unchanged. To address these issues and facilitate more convenient editing of face images, we propose a novel approach that leverages the power of Stable-Diffusion models and crude 3D face models to control the lighting, facial expression and head pose of a portrait photo. We observe that this task essentially involve combinations of target background, identity and different face attributes. We aim to sufficiently disentangle the control of these factors to enable high-quality of face editing. Specifically, our method, coined as RigFace, contains: 1) A Spatial Arrtibute Encoder that provides presise and decoupled conditions of background, pose, expression and lighting; 2) An Identity Encoder that transfers identity features to the denoising UNet of a pre-trained Stable-Diffusion model; 3) An Attribute Rigger that injects those conditions into the denoising UNet. Our model achieves comparable or even superior performance in both identity preservation and photorealism compared to existing face editing models.

Title: Mind the Gap: Evaluating Patch Embeddings from General-Purpose and Histopathology Foundation Models for Cell Segmentation and Classification

• Authors: Valentina Vadori, Antonella Peruffo, Jean-Marie Graïc, Livio Finos, Enrico Grisan
• Subjects: cs.CV, cs.AI, cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.02471
• Pdf URL: https://arxiv.org/pdf/2502.02471
• Copy Paste: [[2502.02471]] Mind the Gap: Evaluating Patch Embeddings from General-Purpose and Histopathology Foundation Models for Cell Segmentation and Classification(https://arxiv.org/abs/2502.02471)
• Keywords: foundation model
• Abstract: Recent advancements in foundation models have transformed computer vision, driving significant performance improvements across diverse domains, including digital histopathology. However, the advantages of domain-specific histopathology foundation models over general-purpose models for specialized tasks such as cell analysis remain underexplored. This study investigates the representation learning gap between these two categories by analyzing multi-level patch embeddings applied to cell instance segmentation and classification. We implement an encoder-decoder architecture with a consistent decoder and various encoders. These include convolutional, vision transformer (ViT), and hybrid encoders pre-trained on ImageNet-22K or LVD-142M, representing general-purpose foundation models. These are compared against ViT encoders from the recently released UNI, Virchow2, and Prov-GigaPath foundation models, trained on patches extracted from hundreds of thousands of histopathology whole-slide images. The decoder integrates patch embeddings from different encoder depths via skip connections to generate semantic and distance maps. These maps are then post-processed to create instance segmentation masks where each label corresponds to an individual cell and to perform cell-type classification. All encoders remain frozen during training to assess their pre-trained feature extraction capabilities. Using the PanNuke and CoNIC histopathology datasets, and the newly introduced Nissl-stained CytoDArk0 dataset for brain cytoarchitecture studies, we evaluate instance-level detection, segmentation accuracy, and cell-type classification. This study provides insights into the comparative strengths and limitations of general-purpose vs. histopathology foundation models, offering guidance for model selection in cell-focused histopathology and brain cytoarchitecture analysis workflows.

Title: Distributional Diffusion Models with Scoring Rules

• Authors: Valentin De Bortoli, Alexandre Galashov, J. Swaroop Guntupalli, Guangyao Zhou, Kevin Murphy, Arthur Gretton, Arnaud Doucet
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.02483
• Pdf URL: https://arxiv.org/pdf/2502.02483
• Copy Paste: [[2502.02483]] Distributional Diffusion Models with Scoring Rules(https://arxiv.org/abs/2502.02483)
• Keywords: diffusion
• Abstract: Diffusion models generate high-quality synthetic data. They operate by defining a continuous-time forward process which gradually adds Gaussian noise to data until fully corrupted. The corresponding reverse process progressively "denoises" a Gaussian sample into a sample from the data distribution. However, generating high-quality outputs requires many discretization steps to obtain a faithful approximation of the reverse process. This is expensive and has motivated the development of many acceleration methods. We propose to accomplish sample generation by learning the posterior {\em distribution} of clean data samples given their noisy versions, instead of only the mean of this distribution. This allows us to sample from the probability transitions of the reverse process on a coarse time scale, significantly accelerating inference with minimal degradation of the quality of the output. This is accomplished by replacing the standard regression loss used to estimate conditional means with a scoring rule. We validate our method on image and robot trajectory generation, where we consistently outperform standard diffusion models at few discretization steps.

Title: Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?

• Authors: Xiyuan Wang, Yewei Liu, Lexi Pang, Siwei Chen, Muhan Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02488
• Pdf URL: https://arxiv.org/pdf/2502.02488
• Copy Paste: [[2502.02488]] Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?(https://arxiv.org/abs/2502.02488)
• Keywords: diffusion
• Abstract: Diffusion models have gained popularity in graph generation tasks; however, the extent of their expressivity concerning the graph distributions they can learn is not fully understood. Unlike models in other domains, popular backbones for graph diffusion models, such as Graph Transformers, do not possess universal expressivity to accurately model the distribution scores of complex graph data. Our work addresses this limitation by focusing on the frequency of specific substructures as a key characteristic of target graph distributions. When evaluating existing models using this metric, we find that they fail to maintain the distribution of substructure counts observed in the training set when generating new graphs. To address this issue, we establish a theoretical connection between the expressivity of Graph Neural Networks (GNNs) and the overall performance of graph diffusion models, demonstrating that more expressive GNN backbones can better capture complex distribution patterns. By integrating advanced GNNs into the backbone architecture, we achieve significant improvements in substructure generation.

Title: A Self-Supervised Framework for Improved Generalisability in Ultrasound B-mode Image Segmentation

• Authors: Edward Ellis, Andrew Bulpitt, Nasim Parsa, Michael F Byrne, Sharib Ali
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02489
• Pdf URL: https://arxiv.org/pdf/2502.02489
• Copy Paste: [[2502.02489]] A Self-Supervised Framework for Improved Generalisability in Ultrasound B-mode Image Segmentation(https://arxiv.org/abs/2502.02489)
• Keywords: self-supervised
• Abstract: Ultrasound (US) imaging is clinically invaluable due to its noninvasive and safe nature. However, interpreting US images is challenging, requires significant expertise, and time, and is often prone to errors. Deep learning offers assistive solutions such as segmentation. Supervised methods rely on large, high-quality, and consistently labeled datasets, which are challenging to curate. Moreover, these methods tend to underperform on out-of-distribution data, limiting their clinical utility. Self-supervised learning (SSL) has emerged as a promising alternative, leveraging unlabeled data to enhance model performance and generalisability. We introduce a contrastive SSL approach tailored for B-mode US images, incorporating a novel Relation Contrastive Loss (RCL). RCL encourages learning of distinct features by differentiating positive and negative sample pairs through a learnable metric. Additionally, we propose spatial and frequency-based augmentation strategies for the representation learning on US images. Our approach significantly outperforms traditional supervised segmentation methods across three public breast US datasets, particularly in data-limited scenarios. Notable improvements on the Dice similarity metric include a 4% increase on 20% and 50% of the BUSI dataset, nearly 6% and 9% improvements on 20% and 50% of the BrEaST dataset, and 6.4% and 3.7% improvements on 20% and 50% of the UDIAT dataset, respectively. Furthermore, we demonstrate superior generalisability on the out-of-distribution UDIAT dataset with performance boosts of 20.6% and 13.6% compared to the supervised baseline using 20% and 50% of the BUSI and BrEaST training data, respectively. Our research highlights that domain-inspired SSL can improve US segmentation, especially under data-limited conditions.

Title: VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models

• Authors: Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, Shelly Sheynin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02492
• Pdf URL: https://arxiv.org/pdf/2502.02492
• Copy Paste: [[2502.02492]] VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models(https://arxiv.org/abs/2502.02492)
• Keywords: generative
• Abstract: Despite tremendous recent progress, generative video models still struggle to capture real-world motion, dynamics, and physics. We show that this limitation arises from the conventional pixel reconstruction objective, which biases models toward appearance fidelity at the expense of motion coherence. To address this, we introduce VideoJAM, a novel framework that instills an effective motion prior to video generators, by encouraging the model to learn a joint appearance-motion representation. VideoJAM is composed of two complementary units. During training, we extend the objective to predict both the generated pixels and their corresponding motion from a single learned representation. During inference, we introduce Inner-Guidance, a mechanism that steers the generation toward coherent motion by leveraging the model's own evolving motion prediction as a dynamic guidance signal. Notably, our framework can be applied to any video model with minimal adaptations, requiring no modifications to the training data or scaling of the model. VideoJAM achieves state-of-the-art performance in motion coherence, surpassing highly competitive proprietary models while also enhancing the perceived visual quality of the generations. These findings emphasize that appearance and motion can be complementary and, when effectively integrated, enhance both the visual quality and the coherence of video generation. Project website: this https URL

Title: Learning to generate physical ocean states: Towards hybrid climate modeling

• Authors: Etienne Meunier, David Kamm, Guillaume Gachon, Redouane Lguensat, Julie Deshayes
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02499
• Pdf URL: https://arxiv.org/pdf/2502.02499
• Copy Paste: [[2502.02499]] Learning to generate physical ocean states: Towards hybrid climate modeling(https://arxiv.org/abs/2502.02499)
• Keywords: generative
• Abstract: Ocean General Circulation Models require extensive computational resources to reach equilibrium states, while deep learning emulators, despite offering fast predictions, lack the physical interpretability and long-term stability necessary for climate scientists to understand climate sensitivity (to greenhouse gas emissions) and mechanisms of abrupt % variability such as tipping points. We propose to take the best from both worlds by leveraging deep generative models to produce physically consistent oceanic states that can serve as initial conditions for climate projections. We assess the viability of this hybrid approach through both physical metrics and numerical experiments, and highlight the benefits of enforcing physical constraints during generation. Although we train here on ocean variables from idealized numerical simulations, we claim that this hybrid approach, combining the computational efficiency of deep learning with the physical accuracy of numerical models, can effectively reduce the computational burden of running climate models to equilibrium, and reduce uncertainties in climate projections by minimizing drifts in baseline simulations.

Title: Generative Modeling on Lie Groups via Euclidean Generalized Score Matching

• Authors: Marco Bertolini, Tuan Le, Djork-Arné Clevert
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02513
• Pdf URL: https://arxiv.org/pdf/2502.02513
• Copy Paste: [[2502.02513]] Generative Modeling on Lie Groups via Euclidean Generalized Score Matching(https://arxiv.org/abs/2502.02513)
• Keywords: diffusion, generative
• Abstract: We extend Euclidean score-based diffusion processes to generative modeling on Lie groups. Through the formalism of Generalized Score Matching, our approach yields a Langevin dynamics which decomposes as a direct sum of Lie algebra representations, enabling generative processes on Lie groups while operating in Euclidean space. Unlike equivariant models, which restrict the space of learnable functions by quotienting out group orbits, our method can model any target distribution on any (non-Abelian) Lie group. Standard score matching emerges as a special case of our framework when the Lie group is the translation group. We prove that our generalized generative processes arise as solutions to a new class of paired stochastic differential equations (SDEs), introduced here for the first time. We validate our approach through experiments on diverse data types, demonstrating its effectiveness in real-world applications such as SO(3)-guided molecular conformer generation and modeling ligand-specific global SE(3) transformations for molecular docking, showing improvement in comparison to Riemannian diffusion on the group itself. We show that an appropriate choice of Lie group enhances learning efficiency by reducing the effective dimensionality of the trajectory space and enables the modeling of transitions between complex data distributions. Additionally, we demonstrate the universality of our approach by deriving how it extends to flow matching.

Title: Privacy Attacks on Image AutoRegressive Models

• Authors: Antoni Kowalczuk, Jan Dubiński, Franziska Boenisch, Adam Dziedzic
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02514
• Pdf URL: https://arxiv.org/pdf/2502.02514
• Copy Paste: [[2502.02514]] Privacy Attacks on Image AutoRegressive Models(https://arxiv.org/abs/2502.02514)
• Keywords: diffusion
• Abstract: Image autoregressive (IAR) models have surpassed diffusion models (DMs) in both image quality (FID: 1.48 vs. 1.58) and generation speed. However, their privacy risks remain largely unexplored. To address this, we conduct a comprehensive privacy analysis comparing IARs to DMs. We develop a novel membership inference attack (MIA) that achieves a significantly higher success rate in detecting training images (TPR@FPR=1%: 86.38% for IARs vs. 4.91% for DMs). Using this MIA, we perform dataset inference (DI) and find that IARs require as few as six samples to detect dataset membership, compared to 200 for DMs, indicating higher information leakage. Additionally, we extract hundreds of training images from an IAR (e.g., 698 from VAR-d30). Our findings highlight a fundamental privacy-utility trade-off: while IARs excel in generation quality and speed, they are significantly more vulnerable to privacy attacks. This suggests that incorporating techniques from DMs, such as per-token probability modeling using diffusion, could help mitigate IARs' privacy risks. Our code is available at this https URL.

Title: Brief analysis of DeepSeek R1 and it's implications for Generative AI

• Authors: Sarah Mercer, Samuel Spillard, Daniel P. Martin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02523
• Pdf URL: https://arxiv.org/pdf/2502.02523
• Copy Paste: [[2502.02523]] Brief analysis of DeepSeek R1 and it's implications for Generative AI(https://arxiv.org/abs/2502.02523)
• Keywords: generative
• Abstract: In late January 2025, DeepSeek released their new reasoning model (DeepSeek R1); which was developed at a fraction of the cost yet remains competitive with OpenAI's models, despite the US's GPU export ban. This report discusses the model, and what its release means for the field of Generative AI more widely. We briefly discuss other models released from China in recent weeks, their similarities; innovative use of Mixture of Experts (MoE), Reinforcement Learning (RL) and clever engineering appear to be key factors in the capabilities of these models. This think piece has been written to a tight time-scale, providing broad coverage of the topic, and serves as introductory material for those looking to understand the model's technical advancements, as well as it's place in the ecosystem. Several further areas of research are identified.

Title: Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation

• Authors: Jian Liu, Wei Sun, Hui Yang, Pengchao Deng, Chongpei Liu, Nicu Sebe, Hossein Rahmani, Ajmal Mian
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.02525
• Pdf URL: https://arxiv.org/pdf/2502.02525
• Copy Paste: [[2502.02525]] Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation(https://arxiv.org/abs/2502.02525)
• Keywords: diffusion, generative
• Abstract: Nine-degrees-of-freedom (9-DoF) object pose and size estimation is crucial for enabling augmented reality and robotic manipulation. Category-level methods have received extensive research attention due to their potential for generalization to intra-class unknown objects. However, these methods require manual collection and labeling of large-scale real-world training data. To address this problem, we introduce a diffusion-based paradigm for domain-generalized category-level 9-DoF object pose estimation. Our motivation is to leverage the latent generalization ability of the diffusion model to address the domain generalization challenge in object pose estimation. This entails training the model exclusively on rendered synthetic data to achieve generalization to real-world scenes. We propose an effective diffusion model to redefine 9-DoF object pose estimation from a generative perspective. Our model does not require any 3D shape priors during training or inference. By employing the Denoising Diffusion Implicit Model, we demonstrate that the reverse diffusion process can be executed in as few as 3 steps, achieving near real-time performance. Finally, we design a robotic grasping system comprising both hardware and software components. Through comprehensive experiments on two benchmark datasets and the real-world robotic system, we show that our method achieves state-of-the-art domain generalization performance. Our code will be made public at this https URL.

Title: TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems

• Authors: Si-Yang Liu, Han-Jia Ye
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.02527
• Pdf URL: https://arxiv.org/pdf/2502.02527
• Copy Paste: [[2502.02527]] TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems(https://arxiv.org/abs/2502.02527)
• Keywords: in-context
• Abstract: TabPFN has emerged as a promising in-context learning model for tabular data, capable of directly predicting the labels of test samples given labeled training examples. It has demonstrated competitive performance, particularly on small-scale classification tasks. However, despite its effectiveness, TabPFN still requires further refinement in several areas, including handling high-dimensional features, aligning with downstream datasets, and scaling to larger datasets. In this paper, we revisit existing variants of TabPFN and observe that most approaches focus either on reducing bias or variance, often neglecting the need to address the other side, while also increasing inference overhead. To fill this gap, we propose Beta (Bagging and Encoder-based Fine-tuning for TabPFN Adaptation), a novel and effective method designed to minimize both bias and variance. To reduce bias, we introduce a lightweight encoder to better align downstream tasks with the pre-trained TabPFN. By increasing the number of encoders in a lightweight manner, Beta mitigate variance, thereby further improving the model's performance. Additionally, bootstrapped sampling is employed to further reduce the impact of data perturbations on the model, all while maintaining computational efficiency during inference. Our approach enhances TabPFN's ability to handle high-dimensional data and scale to larger datasets. Experimental results on over 200 benchmark classification datasets demonstrate that Beta either outperforms or matches state-of-the-art methods.

Title: Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation

• Authors: Junha Lee, Chunghyun Park, Jaesung Choe, Yu-Chiang Frank Wang, Jan Kautz, Minsu Cho, Chris Choy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02548
• Pdf URL: https://arxiv.org/pdf/2502.02548
• Copy Paste: [[2502.02548]] Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation(https://arxiv.org/abs/2502.02548)
• Keywords: foundation model
• Abstract: We tackle open-vocabulary 3D scene understanding by introducing a novel data generation pipeline and training framework. Our method addresses three critical requirements for effective training: precise 3D region segmentation, comprehensive textual descriptions, and sufficient dataset scale. By leveraging state-of-the-art open-vocabulary image segmentation models and region-aware Vision-Language Models, we develop an automatic pipeline that generates high-quality 3D mask-text pairs. Applying this pipeline to multiple 3D scene datasets, we create Mosaic3D-5.6M, a dataset of over 30K annotated scenes with 5.6M mask-text pairs, significantly larger than existing datasets. Building upon this data, we propose Mosaic3D, a foundation model combining a 3D encoder trained with contrastive learning and a lightweight mask decoder for open-vocabulary 3D semantic and instance segmentation. Our approach achieves state-of-the-art results on open-vocabulary 3D semantic and instance segmentation tasks including ScanNet200, Matterport3D, and ScanNet++, with ablation studies validating the effectiveness of our large-scale training data.

Title: Open Materials Generation with Stochastic Interpolants

• Authors: Philipp Hoellmer, Thomas Egg, Maya M. Martirossyan, Eric Fuemmeler, Amit Gupta, Zeren Shui, Pawan Prakash, Adrian Roitberg, Mingjie Liu, George Karypis, Mark Transtrum, Richard G. Hennig, Ellad B. Tadmor, Stefano Martiniani
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2502.02582
• Pdf URL: https://arxiv.org/pdf/2502.02582
• Copy Paste: [[2502.02582]] Open Materials Generation with Stochastic Interpolants(https://arxiv.org/abs/2502.02582)
• Keywords: diffusion, generative
• Abstract: The discovery of new materials is essential for enabling technological advancements. Computational approaches for predicting novel materials must effectively learn the manifold of stable crystal structures within an infinite design space. We introduce Open Materials Generation (OMG), a unifying framework for the generative design and discovery of inorganic crystalline materials. OMG employs stochastic interpolants (SI) to bridge an arbitrary base distribution to the target distribution of inorganic crystals via a broad class of tunable stochastic processes, encompassing both diffusion models and flow matching as special cases. In this work, we adapt the SI framework by integrating an equivariant graph representation of crystal structures and extending it to account for periodic boundary conditions in unit cell representations. Additionally, we couple the SI flow over spatial coordinates and lattice vectors with discrete flow matching for atomic species. We benchmark OMG's performance on two tasks: Crystal Structure Prediction (CSP) for specified compositions, and 'de novo' generation (DNG) aimed at discovering stable, novel, and unique structures. In our ground-up implementation of OMG, we refine and extend both CSP and DNG metrics compared to previous works. OMG establishes a new state-of-the-art in generative modeling for materials discovery, outperforming purely flow-based and diffusion-based implementations. These results underscore the importance of designing flexible deep learning frameworks to accelerate progress in materials science.

Title: Calibrated Multi-Preference Optimization for Aligning Diffusion Models

• Authors: Kyungmin Lee, Xiaohang Li, Qifei Wang, Junfeng He, Junjie Ke, Ming-Hsuan Yang, Irfan Essa, Jinwoo Shin, Feng Yang, Yinxiao Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02588
• Pdf URL: https://arxiv.org/pdf/2502.02588
• Copy Paste: [[2502.02588]] Calibrated Multi-Preference Optimization for Aligning Diffusion Models(https://arxiv.org/abs/2502.02588)
• Keywords: diffusion
• Abstract: Aligning text-to-image (T2I) diffusion models with preference optimization is valuable for human-annotated datasets, but the heavy cost of manual data collection limits scalability. Using reward models offers an alternative, however, current preference optimization methods fall short in exploiting the rich information, as they only consider pairwise preference distribution. Furthermore, they lack generalization to multi-preference scenarios and struggle to handle inconsistencies between rewards. To address this, we present Calibrated Preference Optimization (CaPO), a novel method to align T2I diffusion models by incorporating the general preference from multiple reward models without human annotated data. The core of our approach involves a reward calibration method to approximate the general preference by computing the expected win-rate against the samples generated by the pretrained models. Additionally, we propose a frontier-based pair selection method that effectively manages the multi-preference distribution by selecting pairs from Pareto frontiers. Finally, we use regression loss to fine-tune diffusion models to match the difference between calibrated rewards of a selected pair. Experimental results show that CaPO consistently outperforms prior methods, such as Direct Preference Optimization (DPO), in both single and multi-reward settings validated by evaluation on T2I benchmarks, including GenEval and T2I-Compbench.

Title: COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generation

• Authors: Xueqing Deng, Qihang Yu, Ali Athar, Chenglin Yang, Linjie Yang, Xiaojie Jin, Xiaohui Shen, Liang-Chieh Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.02589
• Pdf URL: https://arxiv.org/pdf/2502.02589
• Copy Paste: [[2502.02589]] COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generation(https://arxiv.org/abs/2502.02589)
• Keywords: generative
• Abstract: This paper introduces the COCONut-PanCap dataset, created to enhance panoptic segmentation and grounded image captioning. Building upon the COCO dataset with advanced COCONut panoptic masks, this dataset aims to overcome limitations in existing image-text datasets that often lack detailed, scene-comprehensive descriptions. The COCONut-PanCap dataset incorporates fine-grained, region-level captions grounded in panoptic segmentation masks, ensuring consistency and improving the detail of generated captions. Through human-edited, densely annotated descriptions, COCONut-PanCap supports improved training of vision-language models (VLMs) for image understanding and generative models for text-to-image tasks. Experimental results demonstrate that COCONut-PanCap significantly boosts performance across understanding and generation tasks, offering complementary benefits to large-scale datasets. This dataset sets a new benchmark for evaluating models on joint panoptic segmentation and grounded captioning tasks, addressing the need for high-quality, detailed image-text annotations in multi-modal learning.