2025-02-25

Title: Digi-Q: Learning Q-Value Functions for Training Device-Control Agents

• Authors: Hao Bai, Yifei Zhou, Li Erran Li, Sergey Levine, Aviral Kumar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.15760
• Pdf URL: https://arxiv.org/pdf/2502.15760
• Copy Paste: [[2502.15760]] Digi-Q: Learning Q-Value Functions for Training Device-Control Agents(https://arxiv.org/abs/2502.15760)
• Keywords: foundation model
• Abstract: While a number of existing approaches for building foundation model agents rely on prompting or fine-tuning with human demonstrations, it is not sufficient in dynamic environments (e.g., mobile device control). On-policy reinforcement learning (RL) should address these limitations, but collecting actual rollouts in an environment is often undesirable in truly open-ended agentic problems such as mobile device control or interacting with humans, where each unit of interaction is associated with a cost. In such scenarios, a method for policy learning that can utilize off-policy experience by learning a trained action-value function is much more effective. In this paper, we develop an approach, called Digi-Q, to train VLM-based action-value Q-functions which are then used to extract the agent policy. We study our approach in the mobile device control setting. Digi-Q trains the Q-function using offline temporal-difference (TD) learning, on top of frozen, intermediate-layer features of a VLM. Compared to fine-tuning the whole VLM, this approach saves us compute and enhances scalability. To make the VLM features amenable for representing the Q-function, we need to employ an initial phase of fine-tuning to amplify coverage over actionable information needed for value function. Once trained, we use this Q-function via a Best-of-N policy extraction operator that imitates the best action out of multiple candidate actions from the current policy as ranked by the value function, enabling policy improvement without environment interaction. Digi-Q outperforms several prior methods on user-scale device control tasks in Android-in-the-Wild, attaining 21.2% improvement over prior best-performing method. In some cases, our Digi-Q approach already matches state-of-the-art RL methods that require interaction. The project is open-sourced at this https URL

Title: Anomaly Detection in Smart Power Grids with Graph-Regularized MS-SVDD: a Multimodal Subspace Learning Approach

• Authors: Thomas Debelle, Fahad Sohrab, Pekka Abrahamsson, Moncef Gabbouj
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2502.15793
• Pdf URL: https://arxiv.org/pdf/2502.15793
• Copy Paste: [[2502.15793]] Anomaly Detection in Smart Power Grids with Graph-Regularized MS-SVDD: a Multimodal Subspace Learning Approach(https://arxiv.org/abs/2502.15793)
• Keywords: anomaly
• Abstract: In this paper, we address an anomaly detection problem in smart power grids using Multimodal Subspace Support Vector Data Description (MS-SVDD). This approach aims to leverage better feature relations by considering the data as coming from different modalities. These data are projected into a shared lower-dimensionality subspace which aims to preserve their inner characteristics. To supplement the previous work on this subject, we introduce novel multimodal graph-embedded regularizers that leverage graph information for every modality to enhance the training process, and we consider an improved training equation that allows us to maximize or minimize each modality according to the specified criteria. We apply this regularized graph-embedded model on a 3-modalities dataset after having generalized MS-SVDD algorithms to any number of modalities. To set up our application, we propose a whole preprocessing procedure to extract One-Class Classification training instances from time-bounded event time series that are used to evaluate both the reliability and earliness of our model for Event Detection.

Title: Self-Supervised Transformers as Iterative Solution Improvers for Constraint Satisfaction

• Authors: Yudong W. Xu, Wenhao Li, Scott Sanner, Elias B. Khalil
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15794
• Pdf URL: https://arxiv.org/pdf/2502.15794
• Copy Paste: [[2502.15794]] Self-Supervised Transformers as Iterative Solution Improvers for Constraint Satisfaction(https://arxiv.org/abs/2502.15794)
• Keywords: self-supervised
• Abstract: We present a Transformer-based framework for Constraint Satisfaction Problems (CSPs). CSPs find use in many applications and thus accelerating their solution with machine learning is of wide interest. Most existing approaches rely on supervised learning from feasible solutions or reinforcement learning, paradigms that require either feasible solutions to these NP-Complete CSPs or large training budgets and a complex expert-designed reward signal. To address these challenges, we propose ConsFormer, a self-supervised framework that leverages a Transformer as a solution refiner. ConsFormer constructs a solution to a CSP iteratively in a process that mimics local search. Instead of using feasible solutions as labeled data, we devise differentiable approximations to the discrete constraints of a CSP to guide model training. Our model is trained to improve random assignments for a single step but is deployed iteratively at test time, circumventing the bottlenecks of supervised and reinforcement learning. Our method can tackle out-of-distribution CSPs simply through additional iterations.

Title: FragFM: Efficient Fragment-Based Molecular Generation via Discrete Flow Matching

• Authors: Joongwon Lee, Seonghwan Kim, Wou Youn Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15805
• Pdf URL: https://arxiv.org/pdf/2502.15805
• Copy Paste: [[2502.15805]] FragFM: Efficient Fragment-Based Molecular Generation via Discrete Flow Matching(https://arxiv.org/abs/2502.15805)
• Keywords: diffusion, generative
• Abstract: We introduce FragFM, a novel fragment-based discrete flow matching framework for molecular graph this http URL generates molecules at the fragment level, leveraging a coarse-to-fine autoencoding mechanism to reconstruct atom-level details. This approach reduces computational complexity while maintaining high chemical validity, enabling more efficient and scalable molecular generation. We benchmark FragFM against state-of-the-art diffusion- and flow-based models on standard molecular generation benchmarks and natural product datasets, demonstrating superior performance in validity, property control, and sampling efficiency. Notably, FragFM achieves over 99\% validity with significantly fewer sampling steps, improving scalability while preserving molecular diversity. These results highlight the potential of fragment-based generative modeling for large-scale, property-aware molecular design, paving the way for more efficient exploration of chemical space.

Title: CoME: An Unlearning-based Approach to Conflict-free Model Editing

• Authors: Dahyun Jung, Jaehyung Seo, Jaewook Lee, Chanjun Park, Heuiseok Lim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15826
• Pdf URL: https://arxiv.org/pdf/2502.15826
• Copy Paste: [[2502.15826]] CoME: An Unlearning-based Approach to Conflict-free Model Editing(https://arxiv.org/abs/2502.15826)
• Keywords: generative
• Abstract: Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability. While model editing methods have been developed to address such errors without full re-training, they frequently suffer from knowledge conflicts, where outdated information interferes with new knowledge. In this work, we propose Conflict-free Model Editing (CoME), a novel framework that enhances the accuracy of knowledge updates in LLMs by selectively removing outdated knowledge. CoME leverages unlearning to mitigate knowledge interference, allowing new information to be integrated without compromising relevant linguistic features. Through experiments on GPT-J and LLaMA-3 using Counterfact and ZsRE datasets, we demonstrate that CoME improves both editing accuracy and model reliability when applied to existing editing methods. Our results highlight that the targeted removal of outdated knowledge is crucial for enhancing model editing effectiveness and maintaining the model's generative performance.

Title: A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models

• Authors: Mengyang Sun, Yihao Wang, Tao Feng, Dan Zhang, Yifan Zhu, Jie Tang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15828
• Pdf URL: https://arxiv.org/pdf/2502.15828
• Copy Paste: [[2502.15828]] A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models(https://arxiv.org/abs/2502.15828)
• Keywords: foundation model
• Abstract: In order to streamline the fine-tuning of foundation models, Low-Rank Adapters (LoRAs) have been substantially adopted across various fields, including instruction tuning and domain adaptation. The underlying concept of LoRA involves decomposing a full-rank matrix into the product of two lower-rank matrices, which reduces storage consumption and accelerates the training process. Furthermore, to address the limited expressive capacity of LoRA, the Mixture-of-Expert (MoE) has been introduced for incorporating multiple LoRA adapters. The integration of LoRA experts leads to a visible improvement across several downstream scenes. However, the mixture of LoRAs (MoE-LoRA) still exhibits its low robustness during tuning and inferring. Inspired by the Riemannian Preconditioners which train LoRA as a sub-space projector, we propose a new training strategy for MoE-LoRA, to stabilize and boost its feature learning procedure by multi-space projections. Examinations on SGD and AdamW optimizers demonstrate the effectiveness of our methodology. Source code is available at this https URL.

Title: A Critical Assessment of Modern Generative Models' Ability to Replicate Artistic Styles

• Authors: Andrea Asperti, Franky George, Tiberio Marras, Razvan Ciprian Stricescu, Fabio Zanotti
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15856
• Pdf URL: https://arxiv.org/pdf/2502.15856
• Copy Paste: [[2502.15856]] A Critical Assessment of Modern Generative Models' Ability to Replicate Artistic Styles(https://arxiv.org/abs/2502.15856)
• Keywords: generative
• Abstract: In recent years, advancements in generative artificial intelligence have led to the development of sophisticated tools capable of mimicking diverse artistic styles, opening new possibilities for digital creativity and artistic expression. This paper presents a critical assessment of the style replication capabilities of contemporary generative models, evaluating their strengths and limitations across multiple dimensions. We examine how effectively these models reproduce traditional artistic styles while maintaining structural integrity and compositional balance in the generated images. The analysis is based on a new large dataset of AI-generated works imitating artistic styles of the past, holding potential for a wide range of applications: the "AI-pastiche" dataset. The study is supported by extensive user surveys, collecting diverse opinions on the dataset and investigation both technical and aesthetic challenges, including the ability to generate outputs that are realistic and visually convincing, the versatility of models in handling a wide range of artistic styles, and the extent to which they adhere to the content and stylistic specifications outlined in prompts. This paper aims to provide a comprehensive overview of the current state of generative tools in style replication, offering insights into their technical and artistic limitations, potential advancements in model design and training methodologies, and emerging opportunities for enhancing digital artistry, human-AI collaboration, and the broader creative landscape.

Title: RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers

• Authors: Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, Jun Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15894
• Pdf URL: https://arxiv.org/pdf/2502.15894
• Copy Paste: [[2502.15894]] RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers(https://arxiv.org/abs/2502.15894)
• Keywords: diffusion
• Abstract: Recent advancements in video generation have enabled models to synthesize high-quality, minute-long videos. However, generating even longer videos with temporal coherence remains a major challenge, and existing length extrapolation methods lead to temporal repetition or motion deceleration. In this work, we systematically analyze the role of frequency components in positional embeddings and identify an intrinsic frequency that primarily governs extrapolation behavior. Based on this insight, we propose RIFLEx, a minimal yet effective approach that reduces the intrinsic frequency to suppress repetition while preserving motion consistency, without requiring any additional modifications. RIFLEx offers a true free lunch--achieving high-quality $2\times$ extrapolation on state-of-the-art video diffusion transformers in a completely training-free manner. Moreover, it enhances quality and enables $3\times$ extrapolation by minimal fine-tuning without long videos. Project page and codes: \href{this https URL}{this https URL.}

Title: Directional Gradient Projection for Robust Fine-Tuning of Foundation Models

• Authors: Chengyue Huang, Junjiao Tian, Brisa Maneechotesuwan, Shivang Chopra, Zsolt Kira
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15895
• Pdf URL: https://arxiv.org/pdf/2502.15895
• Copy Paste: [[2502.15895]] Directional Gradient Projection for Robust Fine-Tuning of Foundation Models(https://arxiv.org/abs/2502.15895)
• Keywords: foundation model, generative
• Abstract: Robust fine-tuning aims to adapt large foundation models to downstream tasks while preserving their robustness to distribution shifts. Existing methods primarily focus on constraining and projecting current model towards the pre-trained initialization based on the magnitudes between fine-tuned and pre-trained weights, which often require extensive hyper-parameter tuning and can sometimes result in underfitting. In this work, we propose Directional Gradient Projection (DiGraP), a novel layer-wise trainable method that incorporates directional information from gradients to bridge regularization and multi-objective optimization. Besides demonstrating our method on image classification, as another contribution we generalize this area to the multi-modal evaluation settings for robust fine-tuning. Specifically, we first bridge the uni-modal and multi-modal gap by performing analysis on Image Classification reformulated Visual Question Answering (VQA) benchmarks and further categorize ten out-of-distribution (OOD) VQA datasets by distribution shift types and degree (i.e. near versus far OOD). Experimental results show that DiGraP consistently outperforms existing baselines across Image Classfication and VQA tasks with discriminative and generative backbones, improving both in-distribution (ID) generalization and OOD robustness.

Title: Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models

• Authors: Zheyuan Liu, Guangyao Dou, Xiangchi Yuan, Chunhui Zhang, Zhaoxuan Tan, Meng Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15910
• Pdf URL: https://arxiv.org/pdf/2502.15910
• Copy Paste: [[2502.15910]] Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models(https://arxiv.org/abs/2502.15910)
• Keywords: generative
• Abstract: Generative models such as Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) trained on massive datasets can lead them to memorize and inadvertently reveal sensitive information, raising ethical and privacy concerns. While some prior works have explored this issue in the context of LLMs, it presents a unique challenge for MLLMs due to the entangled nature of knowledge across modalities, making comprehensive unlearning more difficult. To address this challenge, we propose Modality Aware Neuron Unlearning (MANU), a novel unlearning framework for MLLMs designed to selectively clip neurons based on their relative importance to the targeted forget data, curated for different modalities. Specifically, MANU consists of two stages: important neuron selection and selective pruning. The first stage identifies and collects the most influential neurons across modalities relative to the targeted forget knowledge, while the second stage is dedicated to pruning those selected neurons. MANU effectively isolates and removes the neurons that contribute most to the forget data within each modality, while preserving the integrity of retained knowledge. Our experiments conducted across various MLLM architectures illustrate that MANU can achieve a more balanced and comprehensive unlearning in each modality without largely affecting the overall model utility.

Title: AutoMedPrompt: A New Framework for Optimizing LLM Medical Prompts Using Textual Gradients

• Authors: Sean Wu, Michael Koo, Fabien Scalzo, Ira Kurtz
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15944
• Pdf URL: https://arxiv.org/pdf/2502.15944
• Copy Paste: [[2502.15944]] AutoMedPrompt: A New Framework for Optimizing LLM Medical Prompts Using Textual Gradients(https://arxiv.org/abs/2502.15944)
• Keywords: foundation model
• Abstract: Large language models (LLMs) have demonstrated increasingly sophisticated performance in medical and other fields of knowledge. Traditional methods of creating specialist LLMs require extensive fine-tuning and training of models on large datasets. Recently, prompt engineering, instead of fine-tuning, has shown potential to boost the performance of general foundation models. However, prompting methods such as chain-of-thought (CoT) may not be suitable for all subspecialty, and k-shot approaches may introduce irrelevant tokens into the context space. We present AutoMedPrompt, which explores the use of textual gradients to elicit medically relevant reasoning through system prompt optimization. AutoMedPrompt leverages TextGrad's automatic differentiation via text to improve the ability of general foundation LLMs. We evaluated AutoMedPrompt on Llama 3, an open-source LLM, using several QA benchmarks, including MedQA, PubMedQA, and the nephrology subspecialty-specific NephSAP. Our results show that prompting with textual gradients outperforms previous methods on open-source LLMs and surpasses proprietary models such as GPT-4, Claude 3 Opus, and Med-PaLM 2. AutoMedPrompt sets a new state-of-the-art (SOTA) performance on PubMedQA with an accuracy of 82.6$\%$, while also outperforming previous prompting strategies on open-sourced models for MedQA (77.7$\%$) and NephSAP (63.8$\%$).

Title: MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning

• Authors: Zaifu Zhan, Jun Wang, Shuang Zhou, Jiawen Deng, Rui Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15954
• Pdf URL: https://arxiv.org/pdf/2502.15954
• Copy Paste: [[2502.15954]] MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning(https://arxiv.org/abs/2502.15954)
• Keywords: in-context
• Abstract: Objective: To optimize in-context learning in biomedical natural language processing by improving example selection. Methods: We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates four retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on three core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with two large language models (Llama2-7B, Llama3-8B) and three retrievers (Contriever, MedCPT, BGE-Large) to assess performance across different retrieval strategies. Results: The results from the Random mode indicate that providing more examples in the prompt improves the model's generation performance. Meanwhile, Top mode and Diversity mode significantly outperform Random mode on the RE (DDI) task, achieving an F1 score of 0.9669, a 26.4% improvement. Among the three retrievers tested, Contriever outperformed the other two in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks. Conclusion: MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.

Title: Human Motion Prediction, Reconstruction, and Generation

• Authors: Canxuan Gang, Yiran Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15956
• Pdf URL: https://arxiv.org/pdf/2502.15956
• Copy Paste: [[2502.15956]] Human Motion Prediction, Reconstruction, and Generation(https://arxiv.org/abs/2502.15956)
• Keywords: diffusion
• Abstract: This report reviews recent advancements in human motion prediction, reconstruction, and generation. Human motion prediction focuses on forecasting future poses and movements from historical data, addressing challenges like nonlinear dynamics, occlusions, and motion style variations. Reconstruction aims to recover accurate 3D human body movements from visual inputs, often leveraging transformer-based architectures, diffusion models, and physical consistency losses to handle noise and complex poses. Motion generation synthesizes realistic and diverse motions from action labels, textual descriptions, or environmental constraints, with applications in robotics, gaming, and virtual avatars. Additionally, text-to-motion generation and human-object interaction modeling have gained attention, enabling fine-grained and context-aware motion synthesis for augmented reality and robotics. This review highlights key methodologies, datasets, challenges, and future research directions driving progress in these fields.

Title: CoRe: Coherency Regularization for Hierarchical Time Series

• Authors: Rares Cristian, Pavithra Harhsa, Georgia Perakis, Brian Quanz
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.15983
• Pdf URL: https://arxiv.org/pdf/2502.15983
• Copy Paste: [[2502.15983]] CoRe: Coherency Regularization for Hierarchical Time Series(https://arxiv.org/abs/2502.15983)
• Keywords: generative
• Abstract: Hierarchical time series forecasting presents unique challenges, particularly when dealing with noisy data that may not perfectly adhere to aggregation constraints. This paper introduces a novel approach to soft coherency in hierarchical time series forecasting using neural networks. We present a network coherency regularization method, which we denote as CoRe (Coherency Regularization), a technique that trains neural networks to produce forecasts that are inherently coherent across hierarchies, without strictly enforcing aggregation constraints. Our method offers several key advantages. (1) It provides theoretical guarantees on the coherency of forecasts, even for out-of-sample data. (2) It is adaptable to scenarios where data may contain errors or missing values, making it more robust than strict coherency methods. (3) It can be easily integrated into existing neural network architectures for time series forecasting. We demonstrate the effectiveness of our approach on multiple benchmark datasets, comparing it against state-of-the-art methods in both coherent and noisy data scenarios. Additionally, our method can be used within existing generative probabilistic forecasting frameworks to generate coherent probabilistic forecasts. Our results show improved generalization and forecast accuracy, particularly in the presence of data inconsistencies. On a variety of datasets, including both strictly hierarchically coherent and noisy data, our training method has either equal or better accuracy at all levels of the hierarchy while being strictly more coherent out-of-sample than existing soft-coherency methods.

Title: Mean-Shift Distillation for Diffusion Mode Seeking

• Authors: Vikas Thamizharasan, Nikitas Chatzis, Iliyan Georgiev, Matthew Fisher, Difan Liu, Nanxuan Zhao, Evangelos Kalogerakis, Michal Lukac
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.15989
• Pdf URL: https://arxiv.org/pdf/2502.15989
• Copy Paste: [[2502.15989]] Mean-Shift Distillation for Diffusion Mode Seeking(https://arxiv.org/abs/2502.15989)
• Keywords: diffusion
• Abstract: We present mean-shift distillation, a novel diffusion distillation technique that provides a provably good proxy for the gradient of the diffusion output distribution. This is derived directly from mean-shift mode seeking on the distribution, and we show that its extrema are aligned with the modes. We further derive an efficient product distribution sampling procedure to evaluate the gradient. Our method is formulated as a drop-in replacement for score distillation sampling (SDS), requiring neither model retraining nor extensive modification of the sampling procedure. We show that it exhibits superior mode alignment as well as improved convergence in both synthetic and practical setups, yielding higher-fidelity results when applied to both text-to-image and text-to-3D applications with Stable Diffusion.

Title: MedForge: Building Medical Foundation Models Like Open Source Software Development

• Authors: Zheling Tan, Kexin Ding, Jin Gao, Mu Zhou, Dimitris Metaxas, Shaoting Zhang, Dequan Wang
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16055
• Pdf URL: https://arxiv.org/pdf/2502.16055
• Copy Paste: [[2502.16055]] MedForge: Building Medical Foundation Models Like Open Source Software Development(https://arxiv.org/abs/2502.16055)
• Keywords: foundation model
• Abstract: Foundational models (FMs) have made significant strides in the healthcare domain. Yet the data silo challenge and privacy concern remain in healthcare systems, hindering safe medical data sharing and collaborative model development among institutions. The collection and curation of scalable clinical datasets increasingly become the bottleneck for training strong FMs. In this study, we propose Medical Foundation Models Merging (MedForge), a cooperative framework enabling a community-driven medical foundation model development, meanwhile preventing the information leakage of raw patient data and mitigating synchronization model development issues across clinical institutions. MedForge offers a bottom-up model construction mechanism by flexibly merging task-specific Low-Rank Adaptation (LoRA) modules, which can adapt to downstream tasks while retaining original model parameters. Through an asynchronous LoRA module integration scheme, the resulting composite model can progressively enhance its comprehensive performance on various clinical tasks. MedForge shows strong performance on multiple clinical datasets (e.g., breast cancer, lung cancer, and colon cancer) collected from different institutions. Our major findings highlight the value of collaborative foundation models in advancing multi-center clinical collaboration effectively and cohesively. Our code is publicly available at this https URL.

Title: Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming

• Authors: Rui Li, Peiyi Wang, Jingyuan Ma, Di Zhang, Lei Sha, Zhifang Sui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16109
• Pdf URL: https://arxiv.org/pdf/2502.16109
• Copy Paste: [[2502.16109]] Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming(https://arxiv.org/abs/2502.16109)
• Keywords: in-context
• Abstract: Large Language Models (LLMs) have gained increasing attention for their remarkable capacity, alongside concerns about safety arising from their potential to produce harmful content. Red teaming aims to find prompts that could elicit harmful responses from LLMs, and is essential to discover and mitigate safety risks before real-world deployment. However, manual red teaming is both time-consuming and expensive, rendering it unscalable. In this paper, we propose RTPE, a scalable evolution framework to evolve red teaming prompts across both breadth and depth dimensions, facilitating the automatic generation of numerous high-quality and diverse red teaming prompts. Specifically, in-breadth evolving employs a novel enhanced in-context learning method to create a multitude of quality prompts, whereas in-depth evolving applies customized transformation operations to enhance both content and form of prompts, thereby increasing diversity. Extensive experiments demonstrate that RTPE surpasses existing representative automatic red teaming methods on both attack success rate and diversity. In addition, based on 4,800 red teaming prompts created by RTPE, we further provide a systematic analysis of 8 representative LLMs across 8 sensitive topics.

Title: USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images

• Authors: Jiamu Wang, Jin Tae Kwak
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16160
• Pdf URL: https://arxiv.org/pdf/2502.16160
• Copy Paste: [[2502.16160]] USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images(https://arxiv.org/abs/2502.16160)
• Keywords: diffusion
• Abstract: In computational pathology, researchers often face challenges due to the scarcity of labeled pathology datasets. Data augmentation emerges as a crucial technique to mitigate this limitation. In this study, we introduce an efficient data augmentation method for pathology images, called USegMix. Given a set of pathology images, the proposed method generates a new, synthetic image in two phases. In the first phase, USegMix constructs a pool of tissue segments in an automated and unsupervised manner using superpixels and the Segment Anything Model (SAM). In the second phase, USegMix selects a candidate segment in a target image, replaces it with a similar segment from the segment pool, and blends them by using a pre-trained diffusion model. In this way, USegMix can generate diverse and realistic pathology images. We rigorously evaluate the effectiveness of USegMix on two pathology image datasets of colorectal and prostate cancers. The results demonstrate improvements in cancer classification performance, underscoring the substantial potential of USegMix for pathology image analysis.

Title: PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models

• Authors: Xinwei Liu, Xiaojun Jia, Yuan Xun, Hua Zhang, Xiaochun Cao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16167
• Pdf URL: https://arxiv.org/pdf/2502.16167
• Copy Paste: [[2502.16167]] PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.16167)
• Keywords: diffusion, generative
• Abstract: Diffusion models (DMs) have revolutionized data generation, particularly in text-to-image (T2I) synthesis. However, the widespread use of personalized generative models raises significant concerns regarding privacy violations and copyright infringement. To address these issues, researchers have proposed adversarial perturbation-based protection techniques. However, these methods have notable limitations, including insufficient robustness against data transformations and the inability to fully eliminate identifiable features of protected objects in the generated output. In this paper, we introduce PersGuard, a novel backdoor-based approach that prevents malicious personalization of specific images. Unlike traditional adversarial perturbation methods, PersGuard implant backdoor triggers into pre-trained T2I models, preventing the generation of customized outputs for designated protected images while allowing normal personalization for unprotected ones. Unfortunately, existing backdoor methods for T2I diffusion models fail to be applied to personalization scenarios due to the different backdoor objectives and the potential backdoor elimination during downstream fine-tuning processes. To address these, we propose three novel backdoor objectives specifically designed for personalization scenarios, coupled with backdoor retention loss engineered to resist downstream fine-tuning. These components are integrated into a unified optimization framework. Extensive experimental evaluations demonstrate PersGuard's effectiveness in preserving data privacy, even under challenging conditions including gray-box settings, multi-object protection, and facial identity scenarios. Our method significantly outperforms existing techniques, offering a more robust solution for privacy and copyright protection.

Title: IPO: Your Language Model is Secretly a Preference Classifier

• Authors: Shivank Garg, Ayush Singh, Shweta Singh, Paras Chopra
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16182
• Pdf URL: https://arxiv.org/pdf/2502.16182
• Copy Paste: [[2502.16182]] IPO: Your Language Model is Secretly a Preference Classifier(https://arxiv.org/abs/2502.16182)
• Keywords: generative
• Abstract: Reinforcement learning from human feedback (RLHF) has emerged as the primary method for aligning large language models (LLMs) with human preferences. While it enables LLMs to achieve human-level alignment, it often incurs significant computational and financial costs due to its reliance on training external reward models or human-labeled preferences. In this work, we propose \textbf{Implicit Preference Optimization (IPO)}, an alternative approach that leverages generative LLMs as preference classifiers, thereby reducing the dependence on external human feedback or reward models to obtain preferences. We conduct a comprehensive evaluation on the preference classification ability of LLMs using RewardBench, assessing models across different sizes, architectures, and training levels to validate our hypothesis. Furthermore, we investigate the self-improvement capabilities of LLMs by generating multiple responses for a given instruction and employing the model itself as a preference classifier for Direct Preference Optimization (DPO)-based training. Our findings demonstrate that models trained through IPO achieve performance comparable to those utilizing state-of-the-art reward models for obtaining preferences.

Title: Graph Self-Supervised Learning with Learnable Structural and Positional Encodings

• Authors: Asiri Wijesinghe, Hao Zhu, Piotr Koniusz
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16233
• Pdf URL: https://arxiv.org/pdf/2502.16233
• Copy Paste: [[2502.16233]] Graph Self-Supervised Learning with Learnable Structural and Positional Encodings(https://arxiv.org/abs/2502.16233)
• Keywords: self-supervised
• Abstract: Traditional Graph Self-Supervised Learning (GSSL) struggles to capture complex structural properties well. This limitation stems from two main factors: (1) the inadequacy of conventional Graph Neural Networks (GNNs) in representing sophisticated topological features, and (2) the focus of self-supervised learning solely on final graph representations. To address these issues, we introduce \emph{GenHopNet}, a GNN framework that integrates a $k$-hop message-passing scheme, enhancing its ability to capture local structural information without explicit substructure extraction. We theoretically demonstrate that \emph{GenHopNet} surpasses the expressiveness of the classical Weisfeiler-Lehman (WL) test for graph isomorphism. Furthermore, we propose a structural- and positional-aware GSSL framework that incorporates topological information throughout the learning process. This approach enables the learning of representations that are both sensitive to graph topology and invariant to specific structural and feature augmentations. Comprehensive experiments on graph classification datasets, including those designed to test structural sensitivity, show that our method consistently outperforms the existing approaches and maintains computational efficiency. Our work significantly advances GSSL's capability in distinguishing graphs with similar local structures but different global topologies.

Title: DiffFake: Exposing Deepfakes using Differential Anomaly Detection

• Authors: Sotirios Stamnas, Victor Sanchez
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16247
• Pdf URL: https://arxiv.org/pdf/2502.16247
• Copy Paste: [[2502.16247]] DiffFake: Exposing Deepfakes using Differential Anomaly Detection(https://arxiv.org/abs/2502.16247)
• Keywords: anomaly
• Abstract: Traditional deepfake detectors have dealt with the detection problem as a binary classification task. This approach can achieve satisfactory results in cases where samples of a given deepfake generation technique have been seen during training, but can easily fail with deepfakes generated by other techniques. In this paper, we propose DiffFake, a novel deepfake detector that approaches the detection problem as an anomaly detection task. Specifically, DiffFake learns natural changes that occur between two facial images of the same person by leveraging a differential anomaly detection framework. This is done by combining pairs of deep face embeddings and using them to train an anomaly detection model. We further propose to train a feature extractor on pseudo-deepfakes with global and local artifacts, to extract meaningful and generalizable features that can then be used to train the anomaly detection model. We perform extensive experiments on five different deepfake datasets and show that our method can match and sometimes even exceed the performance of state-of-the-art competitors.

Title: Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction

• Authors: Sarah Ball, Simeon Allmendinger, Frauke Kreuter, Niklas Kühl
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16280
• Pdf URL: https://arxiv.org/pdf/2502.16280
• Copy Paste: [[2502.16280]] Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction(https://arxiv.org/abs/2502.16280)
• Keywords: generative
• Abstract: Generative AI (GenAI) is increasingly used in survey contexts to simulate human preferences. While many research endeavors evaluate the quality of synthetic GenAI data by comparing model-generated responses to gold-standard survey results, fundamental questions about the validity and reliability of using LLMs as substitutes for human respondents remain. Our study provides a technical analysis of how demographic attributes and prompt variations influence latent opinion mappings in large language models (LLMs) and evaluates their suitability for survey-based predictions. Using 14 different models, we find that LLM-generated data fails to replicate the variance observed in real-world human responses, particularly across demographic subgroups. In the political space, persona-to-party mappings exhibit limited differentiation, resulting in synthetic data that lacks the nuanced distribution of opinions found in survey data. Moreover, we show that prompt sensitivity can significantly alter outputs for some models, further undermining the stability and predictiveness of LLM-based simulations. As a key contribution, we adapt a probe-based methodology that reveals how LLMs encode political affiliations in their latent space, exposing the systematic distortions introduced by these models. Our findings highlight critical limitations in AI-generated survey data, urging caution in its use for public opinion research, social science experimentation, and computational behavioral modeling.

Title: DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation

• Authors: Yuxuan Xiong, Yue Shi, Yishun Dou, Bingbing Ni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16302
• Pdf URL: https://arxiv.org/pdf/2502.16302
• Copy Paste: [[2502.16302]] DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation(https://arxiv.org/abs/2502.16302)
• Keywords: diffusion
• Abstract: Recently, denoising diffusion models have achieved promising results in 2D image generation and editing. Instruct-NeRF2NeRF (IN2N) introduces the success of diffusion into 3D scene editing through an "Iterative dataset update" (IDU) strategy. Though achieving fascinating results, IN2N suffers from problems of blurry backgrounds and trapping in local optima. The first problem is caused by IN2N's lack of efficient guidance for background maintenance, while the second stems from the interaction between image editing and NeRF training during IDU. In this work, we introduce DualNeRF to deal with these problems. We propose a dual-field representation to preserve features of the original scene and utilize them as additional guidance to the model for background maintenance during IDU. Moreover, a simulated annealing strategy is embedded into IDU to endow our model with the power of addressing local optima issues. A CLIP-based consistency indicator is used to further improve the editing quality by filtering out low-quality edits. Extensive experiments demonstrate that our method outperforms previous methods both qualitatively and quantitatively.

Title: Machine Learning-Based Cloud Computing Compliance Process Automation

• Authors: Yuqing Wang, Xiao Yang
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.16344
• Pdf URL: https://arxiv.org/pdf/2502.16344
• Copy Paste: [[2502.16344]] Machine Learning-Based Cloud Computing Compliance Process Automation(https://arxiv.org/abs/2502.16344)
• Keywords: anomaly
• Abstract: Cloud computing adoption across industries has revolutionized enterprise operations while introducing significant challenges in compliance management. Organizations must continuously meet evolving regulatory requirements such as GDPR and ISO 27001, yet traditional manual review processes have become increasingly inadequate for modern business scales. This paper presents a novel machine learning-based framework for automating cloud computing compliance processes, addressing critical challenges including resource-intensive manual reviews, extended compliance cycles, and delayed risk identification. Our proposed framework integrates multiple machine learning technologies, including BERT-based document processing (94.5% accuracy), One-Class SVM for anomaly detection (88.7% accuracy), and an improved CNN-LSTM architecture for sequential compliance data analysis (90.2% accuracy). Implementation results demonstrate significant improvements: reducing compliance process duration from 7 days to 1.5 days, improving accuracy from 78% to 93%, and decreasing manual effort by 73.3%. A real-world deployment at a major securities firm validated these results, processing 800,000 daily transactions with 94.2% accuracy in risk identification.

Title: Audio Visual Segmentation Through Text Embeddings

• Authors: Kyungbok Lee, You Zhang, Zhiyao Duan
• Subjects: cs.CV, cs.AI, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.16359
• Pdf URL: https://arxiv.org/pdf/2502.16359
• Copy Paste: [[2502.16359]] Audio Visual Segmentation Through Text Embeddings(https://arxiv.org/abs/2502.16359)
• Keywords: foundation model
• Abstract: The goal of Audio-Visual Segmentation (AVS) is to localize and segment the sounding source objects from the video frames. Researchers working on AVS suffer from limited datasets because hand-crafted annotation is expensive. Recent works attempt to overcome the challenge of limited data by leveraging the segmentation foundation model, SAM, prompting it with audio to enhance its ability to segment sounding source objects. While this approach alleviates the model's burden on understanding visual modality by utilizing pre-trained knowledge of SAM, it does not address the fundamental challenge of the limited dataset for learning audio-visual relationships. To address these limitations, we propose \textbf{AV2T-SAM}, a novel framework that bridges audio features with the text embedding space of pre-trained text-prompted SAM. Our method leverages multimodal correspondence learned from rich text-image paired datasets to enhance audio-visual alignment. Furthermore, we introduce a novel feature, $\mathbf{\textit{\textbf{f}}_{CLIP} \odot \textit{\textbf{f}}_{CLAP}}$, which emphasizes shared semantics of audio and visual modalities while filtering irrelevant noise. Experiments on the AVSBench dataset demonstrate state-of-the-art performance on both datasets of AVSBench. Our approach outperforms existing methods by effectively utilizing pretrained segmentation models and cross-modal semantic alignment.

Title: A generative approach to LLM harmfulness detection with special red flag tokens

• Authors: Sophie Xhonneux, David Dobre, Mehrnaz Mohfakhami, Leo Schwinn, Gauthier Gidel
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16366
• Pdf URL: https://arxiv.org/pdf/2502.16366
• Copy Paste: [[2502.16366]] A generative approach to LLM harmfulness detection with special red flag tokens(https://arxiv.org/abs/2502.16366)
• Keywords: generative
• Abstract: Most safety training methods for large language models (LLMs) based on fine-tuning rely on dramatically changing the output distribution of the model when faced with a harmful request, shifting it from an unsafe answer to a refusal to respond. These methods inherently compromise model capabilities and might make auto-regressive models vulnerable to attacks that make likely an initial token of affirmative response. To avoid that, we propose to expand the model's vocabulary with a special token we call red flag token () and propose to fine-tune the model to generate this token at any time harmful content is generated or about to be generated. This novel safety training method effectively augments LLMs into generative classifiers of harmfulness at all times during the conversation. This method offers several advantages: it enables the model to explicitly learn the concept of harmfulness while marginally affecting the generated distribution, thus maintaining the model's utility. It also evaluates each generated answer rather than just the input prompt and provides a stronger defence against sampling-based attacks. In addition, it simplifies the evaluation of the model's robustness and reduces correlated failures when combined with a classifier. We further show an increased robustness to long contexts, and supervised fine-tuning attacks.

Title: Concept Corrector: Erase concepts on the fly for text-to-image diffusion models

• Authors: Zheling Meng, Bo Peng, Xiaochuan Jin, Yueming Lyu, Wei Wang, Jing Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16368
• Pdf URL: https://arxiv.org/pdf/2502.16368
• Copy Paste: [[2502.16368]] Concept Corrector: Erase concepts on the fly for text-to-image diffusion models(https://arxiv.org/abs/2502.16368)
• Keywords: diffusion
• Abstract: Text-to-image diffusion models have demonstrated the underlying risk of generating various unwanted content, such as sexual elements. To address this issue, the task of concept erasure has been introduced, aiming to erase any undesired concepts that the models can generate. Previous methods, whether training-based or training-free, have primarily focused on the input side, i.e. texts. However, they often suffer from incomplete erasure due to limitations in the generalization from limited prompts to diverse image content. In this paper, motivated by the notion that concept erasure on the output side, i.e. generated images, may be more direct and effective, we propose to check concepts based on intermediate-generated images and correct them in the remainder of the generation process. Two key challenges are identified, i.e. determining the presence of target concepts during generation and replacing them on the fly. Leveraging the generation mechanism of diffusion models, we present the Concept Corrector, which incorporates the Generation Check Mechanism and the Concept Removal Attention. This method can identify the generated features associated with target concepts and replace them using pre-defined negative prompts, thereby achieving concept erasure. It requires no changes to model parameters and only relies on a given concept name and its replacement content. To the best of our knowledge, this is the first erasure method based on intermediate-generated images. The experiments on various concepts demonstrate its impressive erasure performance. Code: this https URL.

Title: An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Driving

• Authors: Tianchen Ji, Neeloy Chakraborty, Andre Schreiber, Katherine Driggs-Campbell
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.16389
• Pdf URL: https://arxiv.org/pdf/2502.16389
• Copy Paste: [[2502.16389]] An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Driving(https://arxiv.org/abs/2502.16389)
• Keywords: anomaly
• Abstract: As automated vehicles enter public roads, safety in a near-infinite number of driving scenarios becomes one of the major concerns for the widespread adoption of fully autonomous driving. The ability to detect anomalous situations outside of the operational design domain is a key component in self-driving cars, enabling us to mitigate the impact of abnormal ego behaviors and to realize trustworthy driving systems. On-road anomaly detection in egocentric videos remains a challenging problem due to the difficulties introduced by complex and interactive scenarios. We conduct a holistic analysis of common on-road anomaly patterns, from which we propose three unsupervised anomaly detection experts: a scene expert that focuses on frame-level appearances to detect abnormal scenes and unexpected scene motions; an interaction expert that models normal relative motions between two road participants and raises alarms whenever anomalous interactions emerge; and a behavior expert which monitors abnormal behaviors of individual objects by future trajectory prediction. To combine the strengths of all the modules, we propose an expert ensemble (Xen) using a Kalman filter, in which the final anomaly score is absorbed as one of the states and the observations are generated by the experts. Our experiments employ a novel evaluation protocol for realistic model performance, demonstrate superior anomaly detection performance than previous methods, and show that our framework has potential in classifying anomaly types using unsupervised learning on a large-scale on-road anomaly dataset.

Title: TrustChain: A Blockchain Framework for Auditing and Verifying Aggregators in Decentralized Federated Learning

• Authors: Ehsan Hallaji, Roozbeh Razavi-Far, Mehrdad Saif
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.16406
• Pdf URL: https://arxiv.org/pdf/2502.16406
• Copy Paste: [[2502.16406]] TrustChain: A Blockchain Framework for Auditing and Verifying Aggregators in Decentralized Federated Learning(https://arxiv.org/abs/2502.16406)
• Keywords: anomaly
• Abstract: The server-less nature of Decentralized Federated Learning (DFL) requires allocating the aggregation role to specific participants in each federated round. Current DFL architectures ensure the trustworthiness of the aggregator node upon selection. However, most of these studies overlook the possibility that the aggregating node may turn rogue and act maliciously after being nominated. To address this problem, this paper proposes a DFL structure, called TrustChain, that scores the aggregators before selection based on their past behavior and additionally audits them after the aggregation. To do this, the statistical independence between the client updates and the aggregated model is continuously monitored using the Hilbert-Schmidt Independence Criterion (HSIC). The proposed method relies on several principles, including blockchain, anomaly detection, and concept drift analysis. The designed structure is evaluated on several federated datasets and attack scenarios with different numbers of Byzantine nodes.

Title: A Survey on Industrial Anomalies Synthesis

• Authors: Xichen Xu, Yanshu Wang, Yawen Huang, Jiaqi Liu, Xiaoning Lei, Guoyang Xie, Guannan Jiang, Zhichao Lu
• Subjects: cs.CV, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.16412
• Pdf URL: https://arxiv.org/pdf/2502.16412
• Copy Paste: [[2502.16412]] A Survey on Industrial Anomalies Synthesis(https://arxiv.org/abs/2502.16412)
• Keywords: generative, anomaly
• Abstract: This paper comprehensively reviews anomaly synthesis methodologies. Existing surveys focus on limited techniques, missing an overall field view and understanding method interconnections. In contrast, our study offers a unified review, covering about 40 representative methods across Hand-crafted, Distribution-hypothesis-based, Generative models (GM)-based, and Vision-language models (VLM)-based synthesis. We introduce the first industrial anomaly synthesis (IAS) taxonomy. Prior works lack formal classification or use simplistic taxonomies, hampering structured comparisons and trend identification. Our taxonomy provides a fine-grained framework reflecting methodological progress and practical implications, grounding future research. Furthermore, we explore cross-modality synthesis and large-scale VLM. Previous surveys overlooked multimodal data and VLM in anomaly synthesis, limiting insights into their advantages. Our survey analyzes their integration, benefits, challenges, and prospects, offering a roadmap to boost IAS with multimodal learning. More resources are available at this https URL.

Title: TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation

• Authors: Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16414
• Pdf URL: https://arxiv.org/pdf/2502.16414
• Copy Paste: [[2502.16414]] TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation(https://arxiv.org/abs/2502.16414)
• Keywords: in-context
• Abstract: Large Language models (LLMs) have achieved encouraging results in tabular data generation. However, existing approaches require fine-tuning, which is computationally expensive. This paper explores an alternative: prompting a fixed LLM with in-context examples. We observe that using randomly selected in-context examples hampers the LLM's performance, resulting in sub-optimal generation quality. To address this, we propose a novel in-context learning framework: TabGen-ICL, to enhance the in-context learning ability of LLMs for tabular data generation. TabGen-ICL operates iteratively, retrieving a subset of real samples that represent the residual between currently generated samples and true data distributions. This approach serves two purposes: locally, it provides more effective in-context learning examples for the LLM in each iteration; globally, it progressively narrows the gap between generated and real data. Extensive experiments on five real-world tabular datasets demonstrate that TabGen-ICL significantly outperforms the random selection strategy. Specifically, it reduces the error rate by a margin of $3.5\%-42.2\%$ on fidelity metrics. We demonstrate for the first time that prompting a fixed LLM can yield high-quality synthetic tabular data. The code is provided in the \href{this https URL}{link}.

Title: High-resolution Rainy Image Synthesis: Learning from Rendering

• Authors: Kaibin Zhou, Shengjie Zhao, Hao Deng, Lin Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16421
• Pdf URL: https://arxiv.org/pdf/2502.16421
• Copy Paste: [[2502.16421]] High-resolution Rainy Image Synthesis: Learning from Rendering(https://arxiv.org/abs/2502.16421)
• Keywords: diffusion
• Abstract: Currently, there are few effective methods for synthesizing a mass of high-resolution rainy images in complex illumination conditions. However, these methods are essential for synthesizing large-scale high-quality paired rainy-clean image datasets, which can train deep learning-based single image rain removal models capable of generalizing to various illumination conditions. Therefore, we propose a practical two-stage learning-from-rendering pipeline for high-resolution rainy image synthesis. The pipeline combines the benefits of the realism of rendering-based methods and the high-efficiency of learning-based methods, providing the possibility of creating large-scale high-quality paired rainy-clean image datasets. In the rendering stage, we use a rendering-based method to create a High-resolution Rainy Image (HRI) dataset, which contains realistic high-resolution paired rainy-clean images of multiple scenes and various illumination conditions. In the learning stage, to learn illumination information from background images for high-resolution rainy image generation, we propose a High-resolution Rainy Image Generation Network (HRIGNet). HRIGNet is designed to introduce a guiding diffusion model in the Latent Diffusion Model, which provides additional guidance information for high-resolution image synthesis. In our experiments, HRIGNet is able to synthesize high-resolution rainy images up to 2048x1024 resolution. Rain removal experiments on real dataset validate that our method can help improve the robustness of deep derainers to real rainy images. To make our work reproducible, source codes and the dataset have been released at this https URL.

Title: Unified Prompt Attack Against Text-to-Image Generation Models

• Authors: Duo Peng, Qiuhong Ke, Mark He Huang, Ping Hu, Jun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16423
• Pdf URL: https://arxiv.org/pdf/2502.16423
• Copy Paste: [[2502.16423]] Unified Prompt Attack Against Text-to-Image Generation Models(https://arxiv.org/abs/2502.16423)
• Keywords: in-context
• Abstract: Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries--common in prior methods and easily detectable by API defenders--by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM's superiority in effectiveness, efficiency, naturalness, and low query detection rates.

Title: Sequence-level Large Language Model Training with Contrastive Preference Optimization

• Authors: Zhili Feng, Dhananjay Ram, Cole Hawkins, Aditya Rawal, Jinman Zhao, Sheng Zha
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16433
• Pdf URL: https://arxiv.org/pdf/2502.16433
• Copy Paste: [[2502.16433]] Sequence-level Large Language Model Training with Contrastive Preference Optimization(https://arxiv.org/abs/2502.16433)
• Keywords: self-supervised
• Abstract: The next token prediction loss is the dominant self-supervised training objective for large language models and has achieved promising results in a variety of downstream tasks. However, upon closer investigation of this objective, we find that it lacks an understanding of sequence-level signals, leading to a mismatch between training and inference processes. To bridge this gap, we introduce a contrastive preference optimization (CPO) procedure that can inject sequence-level information into the language model at any training stage without expensive human labeled data. Our experiments show that the proposed objective surpasses the next token prediction in terms of win rate in the instruction-following and text generation tasks.

Title: Iterative Flow Matching -- Path Correction and Gradual Refinement for Enhanced Generative Modeling

• Authors: Eldad Haber, Shadab Ahamed, Md. Shahriar Rahim Siddiqui, Niloufar Zakariaei, Moshe Eliasof
• Subjects: cs.LG, cs.AI, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.16445
• Pdf URL: https://arxiv.org/pdf/2502.16445
• Copy Paste: [[2502.16445]] Iterative Flow Matching -- Path Correction and Gradual Refinement for Enhanced Generative Modeling(https://arxiv.org/abs/2502.16445)
• Keywords: generative
• Abstract: Generative models for image generation are now commonly used for a wide variety of applications, ranging from guided image generation for entertainment to solving inverse problems. Nonetheless, training a generator is a non-trivial feat that requires fine-tuning and can lead to so-called hallucinations, that is, the generation of images that are unrealistic. In this work, we explore image generation using flow matching. We explain and demonstrate why flow matching can generate hallucinations, and propose an iterative process to improve the generation process. Our iterative process can be integrated into virtually $\textit{any}$ generative modeling technique, thereby enhancing the performance and robustness of image synthesis systems.

Title: Auxiliary Discrminator Sequence Generative Adversarial Networks (ADSeqGAN) for Few Sample Molecule Generation

• Authors: Haocheng Tang, Jing Long, Junmei Wang
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.16446
• Pdf URL: https://arxiv.org/pdf/2502.16446
• Copy Paste: [[2502.16446]] Auxiliary Discrminator Sequence Generative Adversarial Networks (ADSeqGAN) for Few Sample Molecule Generation(https://arxiv.org/abs/2502.16446)
• Keywords: generative
• Abstract: In this work, we introduce Auxiliary Discriminator Sequence Generative Adversarial Networks (ADSeqGAN), a novel approach for molecular generation in small-sample datasets. Traditional generative models often struggle with limited training data, particularly in drug discovery, where molecular datasets for specific therapeutic targets, such as nucleic acids binders and central nervous system (CNS) drugs, are scarce. ADSeqGAN addresses this challenge by integrating an auxiliary random forest classifier as an additional discriminator into the GAN framework, significantly improves molecular generation quality and class specificity. Our method incorporates pretrained generator and Wasserstein distance to enhance training stability and diversity. We evaluate ADSeqGAN on a dataset comprising nucleic acid-targeting and protein-targeting small molecules, demonstrating its superior ability to generate nucleic acid binders compared to baseline models such as SeqGAN, ORGAN, and MolGPT. Through an oversampling strategy, ADSeqGAN also significantly improves CNS drug generation, achieving a higher yield than traditional de novo models. Critical assessments, including docking simulations and molecular property analysis, confirm that ADSeqGAN-generated molecules exhibit strong binding affinities, enhanced chemical diversity, and improved synthetic feasibility. Overall, ADSeqGAN presents a novel framework for generative molecular design in data-scarce scenarios, offering potential applications in computational drug discovery. We have demonstrated the successful applications of ADSeqGAN in generating synthetic nucleic acid-targeting and CNS drugs in this work.

Title: Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control

• Authors: Jinbo Yan, Alan Zhao, Yixin Hu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16475
• Pdf URL: https://arxiv.org/pdf/2502.16475
• Copy Paste: [[2502.16475]] Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control(https://arxiv.org/abs/2502.16475)
• Keywords: diffusion
• Abstract: Single-image 3D generation has emerged as a prominent research topic, playing a vital role in virtual reality, 3D modeling, and digital content creation. However, existing methods face challenges such as a lack of multi-view geometric consistency and limited controllability during the generation process, which significantly restrict their usability. % To tackle these challenges, we introduce Dragen3D, a novel approach that achieves geometrically consistent and controllable 3D generation leveraging 3D Gaussian Splatting (3DGS). We introduce the Anchor-Gaussian Variational Autoencoder (Anchor-GS VAE), which encodes a point cloud and a single image into anchor latents and decode these latents into 3DGS, enabling efficient latent-space generation. To enable multi-view geometry consistent and controllable generation, we propose a Seed-Point-Driven strategy: first generate sparse seed points as a coarse geometry representation, then map them to anchor latents via the Seed-Anchor Mapping Module. Geometric consistency is ensured by the easily learned sparse seed points, and users can intuitively drag the seed points to deform the final 3DGS geometry, with changes propagated through the anchor latents. To the best of our knowledge, we are the first to achieve geometrically controllable 3D Gaussian generation and editing without relying on 2D diffusion priors, delivering comparable 3D generation quality to state-of-the-art methods.

Title: On Computational Limits of FlowAR Models: Expressivity and Efficiency

• Authors: Chengyue Gong, Yekun Ke, Xiaoyu Li, Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song
• Subjects: cs.LG, cs.AI, cs.CC, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16490
• Pdf URL: https://arxiv.org/pdf/2502.16490
• Copy Paste: [[2502.16490]] On Computational Limits of FlowAR Models: Expressivity and Efficiency(https://arxiv.org/abs/2502.16490)
• Keywords: generative
• Abstract: The expressive power and computational complexity of deep visual generative models, such as flow-based and autoregressive (AR) models, have gained considerable interest for their wide-ranging applications in generative tasks. However, the theoretical characterization of their expressiveness through the lens of circuit complexity remains underexplored, particularly for the state-of-the-art architecture like FlowAR proposed by [Ren et al., 2024], which integrates flow-based and autoregressive mechanisms. This gap limits our understanding of their inherent computational limits and practical efficiency. In this study, we address this gap by analyzing the circuit complexity of the FlowAR architecture. We demonstrate that when the largest feature map produced by the FlowAR model has dimensions $n \times n \times c$, the FlowAR model is simulable by a family of threshold circuits $\mathsf{TC}^0$, which have constant depth $O(1)$ and polynomial width $\mathrm{poly}(n)$. This is the first study to rigorously highlight the limitations in the expressive power of FlowAR models. Furthermore, we identify the conditions under which the FlowAR model computations can achieve almost quadratic time. To validate our theoretical findings, we present efficient model variant constructions based on low-rank approximations that align with the derived criteria. Our work provides a foundation for future comparisons with other generative paradigms and guides the development of more efficient and expressive implementations.

Title: Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Models

• Authors: Yuyi Huang, Runzhe Zhan, Derek F. Wong, Lidia S. Chao, Ailin Tao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16491
• Pdf URL: https://arxiv.org/pdf/2502.16491
• Copy Paste: [[2502.16491]] Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Models(https://arxiv.org/abs/2502.16491)
• Keywords: generative
• Abstract: Large language models (LLMs) have significantly influenced various industries but suffer from a critical flaw, the potential sensitivity of generating harmful content, which poses severe societal risks. We developed and tested novel attack strategies on popular LLMs to expose their vulnerabilities in generating inappropriate content. These strategies, inspired by psychological phenomena such as the "Priming Effect", "Safe Attention Shift", and "Cognitive Dissonance", effectively attack the models' guarding mechanisms. Our experiments achieved an attack success rate (ASR) of 100% on various open-source models, including Meta's Llama-3.2, Google's Gemma-2, Mistral's Mistral-NeMo, Falcon's Falcon-mamba, Apple's DCLM, Microsoft's Phi3, and Qwen's Qwen2.5, among others. Similarly, for closed-source models such as OpenAI's GPT-4o, Google's Gemini-1.5, and Claude-3.5, we observed an ASR of at least 95% on the AdvBench dataset, which represents the current state-of-the-art. This study underscores the urgent need to reassess the use of generative models in critical applications to mitigate potential adverse societal impacts.

Title: Pay Attention to Real World Perturbations! Natural Robustness Evaluation in Machine Reading Comprehension

• Authors: Yulong Wu, Viktor Schlegel, Riza Batista-Navarro
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16523
• Pdf URL: https://arxiv.org/pdf/2502.16523
• Copy Paste: [[2502.16523]] Pay Attention to Real World Perturbations! Natural Robustness Evaluation in Machine Reading Comprehension(https://arxiv.org/abs/2502.16523)
• Keywords: generative
• Abstract: As neural language models achieve human-comparable performance on Machine Reading Comprehension (MRC) and see widespread adoption, ensuring their robustness in real-world scenarios has become increasingly important. Current robustness evaluation research, though, primarily develops synthetic perturbation methods, leaving unclear how well they reflect real life scenarios. Considering this, we present a framework to automatically examine MRC models on naturally occurring textual perturbations, by replacing paragraph in MRC benchmarks with their counterparts based on available Wikipedia edit history. Such perturbation type is natural as its design does not stem from an arteficial generative process, inherently distinct from the previously investigated synthetic approaches. In a large-scale study encompassing SQUAD datasets and various model architectures we observe that natural perturbations result in performance degradation in pre-trained encoder language models. More worryingly, these state-of-the-art Flan-T5 and Large Language Models (LLMs) inherit these errors. Further experiments demonstrate that our findings generalise to natural perturbations found in other more challenging MRC benchmarks. In an effort to mitigate these errors, we show that it is possible to improve the robustness to natural perturbations by training on naturally or synthetically perturbed examples, though a noticeable gap still remains compared to performance on unperturbed data.

Title: SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition

• Authors: Feng Lu, Tong Jin, Xiangyuan Lan, Lijun Zhang, Yunpeng Liu, Yaowei Wang, Chun Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16601
• Pdf URL: https://arxiv.org/pdf/2502.16601
• Copy Paste: [[2502.16601]] SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition(https://arxiv.org/abs/2502.16601)
• Keywords: foundation model
• Abstract: Recent studies show that the visual place recognition (VPR) method using pre-trained visual foundation models can achieve promising performance. In our previous work, we propose a novel method to realize seamless adaptation of foundation models to VPR (SelaVPR). This method can produce both global and local features that focus on discriminative landmarks to recognize places for two-stage VPR by a parameter-efficient adaptation approach. Although SelaVPR has achieved competitive results, we argue that the previous adaptation is inefficient in training time and GPU memory usage, and the re-ranking paradigm is also costly in retrieval latency and storage usage. In pursuit of higher efficiency and better performance, we propose an extension of the SelaVPR, called SelaVPR++. Concretely, we first design a parameter-, time-, and memory-efficient adaptation method that uses lightweight multi-scale convolution (MultiConv) adapters to refine intermediate features from the frozen foundation backbone. This adaptation method does not back-propagate gradients through the backbone during training, and the MultiConv adapter facilitates feature interactions along the spatial axes and introduces proper local priors, thus achieving higher efficiency and better performance. Moreover, we propose an innovative re-ranking paradigm for more efficient VPR. Instead of relying on local features for re-ranking, which incurs huge overhead in latency and storage, we employ compact binary features for initial retrieval and robust floating-point (global) features for re-ranking. To obtain such binary features, we propose a similarity-constrained deep hashing method, which can be easily integrated into the VPR pipeline. Finally, we improve our training strategy and unify the training protocol of several common training datasets to merge them for better training of VPR models. Extensive experiments show that ......

Title: Can Large Vision-Language Models Detect Images Copyright Infringement from GenAI?

• Authors: Qipan Xu, Zhenting Wang, Xiaoxiao He, Ligong Han, Ruixiang Tang
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16618
• Pdf URL: https://arxiv.org/pdf/2502.16618
• Copy Paste: [[2502.16618]] Can Large Vision-Language Models Detect Images Copyright Infringement from GenAI?(https://arxiv.org/abs/2502.16618)
• Keywords: generative
• Abstract: Generative AI models, renowned for their ability to synthesize high-quality content, have sparked growing concerns over the improper generation of copyright-protected material. While recent studies have proposed various approaches to address copyright issues, the capability of large vision-language models (LVLMs) to detect copyright infringements remains largely unexplored. In this work, we focus on evaluating the copyright detection abilities of state-of-the-art LVLMs using a various set of image samples. Recognizing the absence of a comprehensive dataset that includes both IP-infringement samples and ambiguous non-infringement negative samples, we construct a benchmark dataset comprising positive samples that violate the copyright protection of well-known IP figures, as well as negative samples that resemble these figures but do not raise copyright concerns. This dataset is created using advanced prompt engineering techniques. We then evaluate leading LVLMs using our benchmark dataset. Our experimental results reveal that LVLMs are prone to overfitting, leading to the misclassification of some negative samples as IP-infringement cases. In the final section, we analyze these failure cases and propose potential solutions to mitigate the overfitting problem.

Title: Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines

• Authors: Xinwei Long, Zhiyuan Ma, Ermo Hua, Kaiyan Zhang, Biqing Qi, Bowen Zhou
• Subjects: cs.CV, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.16641
• Pdf URL: https://arxiv.org/pdf/2502.16641
• Copy Paste: [[2502.16641]] Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines(https://arxiv.org/abs/2502.16641)
• Keywords: generative
• Abstract: Retrieval-augmented generation (RAG) has emerged to address the knowledge-intensive visual question answering (VQA) task. Current methods mainly employ separate retrieval and generation modules to acquire external knowledge and generate answers, respectively. We propose ReAuSE, an alternative to the previous RAG model for the knowledge-based VQA task, which seamlessly integrates knowledge retriever into the generative multi-modal large language model, serving as a built-in search engine. Specifically, our model functions both as a generative retriever and an accurate answer generator. It not only helps retrieve documents from the knowledge base by producing identifiers for each document, but it also answers visual questions based on the retrieved documents. Furthermore, we propose a reinforced retrieval calibration module from relevance feedback to improve retrieval performance and align with the preferences for accurate answer generation. Extensive experiments on two representative OKVQA and A-OKVQA datasets demonstrate significant improvements ranging from 2.9\% to 9.6\% across all evaluation metrics when compared to strong baselines.

Title: MimeQA: Towards Socially-Intelligent Nonverbal Foundation Models

• Authors: Hengzhi Li, Megan Tjandrasuwita, Yi R. Fung, Armando Solar-Lezama, Paul Pu Liang
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16671
• Pdf URL: https://arxiv.org/pdf/2502.16671
• Copy Paste: [[2502.16671]] MimeQA: Towards Socially-Intelligent Nonverbal Foundation Models(https://arxiv.org/abs/2502.16671)
• Keywords: foundation model
• Abstract: Socially intelligent AI that can understand and interact seamlessly with humans in daily lives is increasingly important as AI becomes more closely integrated with peoples' daily activities. However, current works in artificial social reasoning all rely on language-only, or language-dominant approaches to benchmark and training models, resulting in systems that are improving in verbal communication but struggle with nonverbal social understanding. To address this limitation, we tap into a novel source of data rich in nonverbal and social interactions -- mime videos. Mimes refer to the art of expression through gesture and movement without spoken words, which presents unique challenges and opportunities in interpreting non-verbal social communication. We contribute a new dataset called MimeQA, obtained by sourcing 221 videos from YouTube, through rigorous annotation and verification, resulting in a benchmark with 101 videos and 806 question-answer pairs. Using MimeQA, we evaluate state-of-the-art video large language models (vLLMs) and find that their overall accuracy ranges from 15-30%. Our analysis reveals that vLLMs often fail to ground imagined objects and over-rely on the text prompt while ignoring subtle nonverbal interactions. Our data resources are released at this https URL to inspire future work in foundation models that embody true social intelligence capable of interpreting non-verbal human interactions.

Title: Code Summarization Beyond Function Level

• Authors: Vladimir Makharev, Vladimir Ivanov
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16704
• Pdf URL: https://arxiv.org/pdf/2502.16704
• Copy Paste: [[2502.16704]] Code Summarization Beyond Function Level(https://arxiv.org/abs/2502.16704)
• Keywords: in-context
• Abstract: Code summarization is a critical task in natural language processing and software engineering, which aims to generate concise descriptions of source code. Recent advancements have improved the quality of these summaries, enhancing code readability and maintainability. However, the content of a repository or a class has not been considered in function code summarization. This study investigated the effectiveness of code summarization models beyond the function level, exploring the impact of class and repository contexts on the summary quality. The study involved revising benchmarks for evaluating models at class and repository levels, assessing baseline models, and evaluating LLMs with in-context learning to determine the enhancement of summary quality with additional context. The findings revealed that the fine-tuned state-of-the-art CodeT5+ base model excelled in code summarization, while incorporating few-shot learning and retrieved code chunks from RAG significantly enhanced the performance of LLMs in this task. Notably, the Deepseek Coder 1.3B and Starcoder2 15B models demonstrated substantial improvements in metrics such as BLEURT, METEOR, and BLEU-4 at both class and repository levels. Repository-level summarization exhibited promising potential but necessitates significant computational resources and gains from the inclusion of structured context. Lastly, we employed the recent SIDE code summarization metric in our evaluation. This study contributes to refining strategies for prompt engineering, few-shot learning, and RAG, addressing gaps in benchmarks for code summarization at various levels. Finally, we publish all study details, code, datasets, and results of evaluation in the GitHub repository available at this https URL.

Title: DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories

• Authors: Hongzhe Cheng, Tianyou Zheng, Tianyi Zhang, Matthew Johnson-Roberson, Weiming Zhi
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16725
• Pdf URL: https://arxiv.org/pdf/2502.16725
• Copy Paste: [[2502.16725]] DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories(https://arxiv.org/abs/2502.16725)
• Keywords: diffusion
• Abstract: Out-of-Distribution(OOD) detection, a fundamental machine learning task aimed at identifying abnormal samples, traditionally requires model retraining for different inlier distributions. While recent research demonstrates the applicability of diffusion models to OOD detection, existing approaches are limited to Euclidean or latent image spaces. Our work extends OOD detection to trajectories in the Special Euclidean Group in 3D ($\mathbb{SE}(3)$), addressing a critical need in computer vision, robotics, and engineering applications that process object pose sequences in $\mathbb{SE}(3)$. We present $\textbf{D}$iffusion-based $\textbf{O}$ut-of-distribution detection on $\mathbb{SE}(3)$ ($\mathbf{DOSE3}$), a novel OOD framework that extends diffusion to a unified sample space of $\mathbb{SE}(3)$ pose sequences. Through extensive validation on multiple benchmark datasets, we demonstrate $\mathbf{DOSE3}$'s superior performance compared to state-of-the-art OOD detection frameworks.

Title: Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning

• Authors: Avinandan Bose, Laurent Lessard, Maryam Fazel, Krishnamurthy Dj Dvijotham
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16737
• Pdf URL: https://arxiv.org/pdf/2502.16737
• Copy Paste: [[2502.16737]] Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning(https://arxiv.org/abs/2502.16737)
• Keywords: foundation model
• Abstract: The rise of foundation models fine-tuned on human feedback from potentially untrusted users has increased the risk of adversarial data poisoning, necessitating the study of robustness of learning algorithms against such attacks. Existing research on provable certified robustness against data poisoning attacks primarily focuses on certifying robustness for static adversaries who modify a fraction of the dataset used to train the model before the training algorithm is applied. In practice, particularly when learning from human feedback in an online sense, adversaries can observe and react to the learning process and inject poisoned samples that optimize adversarial objectives better than when they are restricted to poisoning a static dataset once, before the learning algorithm is applied. Indeed, it has been shown in prior work that online dynamic adversaries can be significantly more powerful than static ones. We present a novel framework for computing certified bounds on the impact of dynamic poisoning, and use these certificates to design robust learning algorithms. We give an illustration of the framework for the mean estimation and binary classification problems and outline directions for extending this in further work. The code to implement our certificates and replicate our results is available at this https URL.

Title: Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model

• Authors: Yaxuan Huang, Xili Dai, Jianan Wang, Xianbiao Qi, Yixing Yuan, Xiangyu Yue
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16779
• Pdf URL: https://arxiv.org/pdf/2502.16779
• Copy Paste: [[2502.16779]] Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model(https://arxiv.org/abs/2502.16779)
• Keywords: foundation model
• Abstract: Room layout estimation from multiple-perspective images is poorly investigated due to the complexities that emerge from multi-view geometry, which requires muti-step solutions such as camera intrinsic and extrinsic estimation, image matching, and triangulation. However, in 3D reconstruction, the advancement of recent 3D foundation models such as DUSt3R has shifted the paradigm from the traditional multi-step structure-from-motion process to an end-to-end single-step approach. To this end, we introduce Plane-DUSt3R}, a novel method for multi-view room layout estimation leveraging the 3D foundation model DUSt3R. Plane-DUSt3R incorporates the DUSt3R framework and fine-tunes on a room layout dataset (Structure3D) with a modified objective to estimate structural planes. By generating uniform and parsimonious results, Plane-DUSt3R enables room layout estimation with only a single post-processing step and 2D detection results. Unlike previous methods that rely on single-perspective or panorama image, Plane-DUSt3R extends the setting to handle multiple-perspective images. Moreover, it offers a streamlined, end-to-end solution that simplifies the process and reduces error accumulation. Experimental results demonstrate that Plane-DUSt3R not only outperforms state-of-the-art methods on the synthetic dataset but also proves robust and effective on in the wild data with different image styles such as cartoon.

Title: Hierarchical Semantic Compression for Consistent Image Semantic Restoration

• Authors: Shengxi Li, Zifu Zhang, Mai Xu, Lai Jiang, Yufan Liu, Ce Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16799
• Pdf URL: https://arxiv.org/pdf/2502.16799
• Copy Paste: [[2502.16799]] Hierarchical Semantic Compression for Consistent Image Semantic Restoration(https://arxiv.org/abs/2502.16799)
• Keywords: generative
• Abstract: The emerging semantic compression has been receiving increasing research efforts most recently, capable of achieving high fidelity restoration during compression, even at extremely low bitrates. However, existing semantic compression methods typically combine standard pipelines with either pre-defined or high-dimensional semantics, thus suffering from deficiency in compression. To address this issue, we propose a novel hierarchical semantic compression (HSC) framework that purely operates within intrinsic semantic spaces from generative models, which is able to achieve efficient compression for consistent semantic restoration. More specifically, we first analyse the entropy models for the semantic compression, which motivates us to employ a hierarchical architecture based on a newly developed general inversion encoder. Then, we propose the feature compression network (FCN) and semantic compression network (SCN), such that the middle-level semantic feature and core semantics are hierarchically compressed to restore both accuracy and consistency of image semantics, via an entropy model progressively shared by channel-wise context. Experimental results demonstrate that the proposed HSC framework achieves the state-of-the-art performance on subjective quality and consistency for human vision, together with superior performances on machine vision tasks given compressed bitstreams. This essentially coincides with human visual system in understanding images, thus providing a new framework for future image/video compression paradigms. Our code shall be released upon acceptance.

Title: Fast, Accurate Manifold Denoising by Tunneling Riemannian Optimization

• Authors: Shiyu Wang, Mariam Avagyan, Yihan Shen, Arnaud Lamy, Tingran Wang, Szabolcs Márka, Zsuzsa Márka, John Wright
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2502.16819
• Pdf URL: https://arxiv.org/pdf/2502.16819
• Copy Paste: [[2502.16819]] Fast, Accurate Manifold Denoising by Tunneling Riemannian Optimization(https://arxiv.org/abs/2502.16819)
• Keywords: diffusion
• Abstract: Learned denoisers play a fundamental role in various signal generation (e.g., diffusion models) and reconstruction (e.g., compressed sensing) architectures, whose success derives from their ability to leverage low-dimensional structure in data. Existing denoising methods, however, either rely on local approximations that require a linear scan of the entire dataset or treat denoising as generic function approximation problems, often sacrificing efficiency and interpretability. We consider the problem of efficiently denoising a new noisy data point sampled from an unknown $d$-dimensional manifold $M \in \mathbb{R}^D$, using only noisy samples. This work proposes a framework for test-time efficient manifold denoising, by framing the concept of "learning-to-denoise" as "learning-to-optimize". We have two technical innovations: (i) online learning methods which learn to optimize over the manifold of clean signals using only noisy data, effectively "growing" an optimizer one sample at a time. (ii) mixed-order methods which guarantee that the learned optimizers achieve global optimality, ensuring both efficiency and near-optimal denoising performance. We corroborate these claims with theoretical analyses of both the complexity and denoising performance of mixed-order traversal. Our experiments on scientific manifolds demonstrate significantly improved complexity-performance tradeoffs compared to nearest neighbor search, which underpins existing provable denoising approaches based on exhaustive search.

Title: Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization

• Authors: Taeyoung Yun, Kiyoung Om, Jaewoo Lee, Sujin Yun, Jinkyoo Park
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.16824
• Pdf URL: https://arxiv.org/pdf/2502.16824
• Copy Paste: [[2502.16824]] Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization(https://arxiv.org/abs/2502.16824)
• Keywords: diffusion, generative
• Abstract: Optimizing high-dimensional and complex black-box functions is crucial in numerous scientific applications. While Bayesian optimization (BO) is a powerful method for sample-efficient optimization, it struggles with the curse of dimensionality and scaling to thousands of evaluations. Recently, leveraging generative models to solve black-box optimization problems has emerged as a promising framework. However, those methods often underperform compared to BO methods due to limited expressivity and difficulty of uncertainty estimation in high-dimensional spaces. To overcome these issues, we introduce \textbf{DiBO}, a novel framework for solving high-dimensional black-box optimization problems. Our method iterates two stages. First, we train a diffusion model to capture the data distribution and an ensemble of proxies to predict function values with uncertainty quantification. Second, we cast the candidate selection as a posterior inference problem to balance exploration and exploitation in high-dimensional spaces. Concretely, we fine-tune diffusion models to amortize posterior inference. Extensive experiments demonstrate that our method outperforms state-of-the-art baselines across various synthetic and real-world black-box optimization tasks. Our code is publicly available \href{this https URL}{here}

Title: A Novel Multi-Task Teacher-Student Architecture with Self-Supervised Pretraining for 48-Hour Vasoactive-Inotropic Trend Analysis in Sepsis Mortality Prediction

• Authors: Houji Jin, Negin Ashrafi, Kamiar Alaei, Elham Pishgar, Greg Placencia, Maryam Pishgar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16834
• Pdf URL: https://arxiv.org/pdf/2502.16834
• Copy Paste: [[2502.16834]] A Novel Multi-Task Teacher-Student Architecture with Self-Supervised Pretraining for 48-Hour Vasoactive-Inotropic Trend Analysis in Sepsis Mortality Prediction(https://arxiv.org/abs/2502.16834)
• Keywords: self-supervised
• Abstract: Sepsis is a major cause of ICU mortality, where early recognition and effective interventions are essential for improving patient outcomes. However, the vasoactive-inotropic score (VIS) varies dynamically with a patient's hemodynamic status, complicated by irregular medication patterns, missing data, and confounders, making sepsis prediction challenging. To address this, we propose a novel Teacher-Student multitask framework with self-supervised VIS pretraining via a Masked Autoencoder (MAE). The teacher model performs mortality classification and severity-score regression, while the student distills robust time-series representations, enhancing adaptation to heterogeneous VIS data. Compared to LSTM-based methods, our approach achieves an AUROC of 0.82 on MIMIC-IV 3.0 (9,476 patients), outperforming the baseline (0.74). SHAP analysis revealed that SOFA score (0.147) had the greatest impact on ICU mortality, followed by LODS (0.033), single marital status (0.031), and Medicaid insurance (0.023), highlighting the role of sociodemographic factors. SAPSII (0.020) also contributed significantly. These findings suggest that both clinical and social factors should be considered in ICU decision-making. Our novel multitask and distillation strategies enable earlier identification of high-risk patients, improving prediction accuracy and disease management, offering new tools for ICU decision support.

Title: REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction

• Authors: Omar Sharif, Joseph Gatto, Madhusudan Basak, Sarah M. Preum
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16838
• Pdf URL: https://arxiv.org/pdf/2502.16838
• Copy Paste: [[2502.16838]] REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction(https://arxiv.org/abs/2502.16838)
• Keywords: generative
• Abstract: Event argument extraction identifies arguments for predefined event roles in text. Traditional evaluations rely on exact match (EM), requiring predicted arguments to match annotated spans exactly. However, this approach fails for generative models like large language models (LLMs), which produce diverse yet semantically accurate responses. EM underestimates performance by disregarding valid variations, implicit arguments (unstated but inferable), and scattered arguments (distributed across a document). To bridge this gap, we introduce Reliable Evaluation framework for Generative event argument extraction (REGen), a framework that better aligns with human judgment. Across six datasets, REGen improves performance by an average of 23.93 F1 points over EM. Human validation further confirms REGen's effectiveness, achieving 87.67% alignment with human assessments of argument correctness.

Title: "Actionable Help" in Crises: A Novel Dataset and Resource-Efficient Models for Identifying Request and Offer Social Media Posts

• Authors: Rabindra Lamsal, Maria Rodriguez Read, Shanika Karunasekera, Muhammad Imran
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16839
• Pdf URL: https://arxiv.org/pdf/2502.16839
• Copy Paste: [[2502.16839]] "Actionable Help" in Crises: A Novel Dataset and Resource-Efficient Models for Identifying Request and Offer Social Media Posts(https://arxiv.org/abs/2502.16839)
• Keywords: generative
• Abstract: During crises, social media serves as a crucial coordination tool, but the vast influx of posts--from "actionable" requests and offers to generic content like emotional support, behavioural guidance, or outdated information--complicates effective classification. Although generative LLMs (Large Language Models) can address this issue with few-shot classification, their high computational demands limit real-time crisis response. While fine-tuning encoder-only models (e.g., BERT) is a popular choice, these models still exhibit higher inference times in resource-constrained environments. Moreover, although distilled variants (e.g., DistilBERT) exist, they are not tailored for the crisis domain. To address these challenges, we make two key contributions. First, we present CrisisHelpOffer, a novel dataset of 101k tweets collaboratively labelled by generative LLMs and validated by humans, specifically designed to distinguish actionable content from noise. Second, we introduce the first crisis-specific mini models optimized for deployment in resource-constrained settings. Across 13 crisis classification tasks, our mini models surpass BERT (also outperform or match the performance of RoBERTa, MPNet, and BERTweet), offering higher accuracy with significantly smaller sizes and faster speeds. The Medium model is 47% smaller with 3.8% higher accuracy at 3.5x speed, the Small model is 68% smaller with a 1.8% accuracy gain at 7.7x speed, and the Tiny model, 83% smaller, matches BERT's accuracy at 18.6x speed. All models outperform existing distilled variants, setting new benchmarks. Finally, as a case study, we analyze social media posts from a global crisis to explore help-seeking and assistance-offering behaviours in selected developing and developed countries.

Title: In-context learning of evolving data streams with tabular foundational models

• Authors: Afonso Lourenço, João Gama, Eric P. Xing, Goreti Marreiros
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16840
• Pdf URL: https://arxiv.org/pdf/2502.16840
• Copy Paste: [[2502.16840]] In-context learning of evolving data streams with tabular foundational models(https://arxiv.org/abs/2502.16840)
• Keywords: in-context
• Abstract: State-of-the-art data stream mining in supervised classification has traditionally relied on ensembles of incremental decision trees. However, the emergence of large tabular models, i.e., transformers designed for structured numerical data, marks a significant paradigm shift. These models move beyond traditional weight updates, instead employing in-context learning through prompt tuning. By using on-the-fly sketches to summarize unbounded streaming data, one can feed this information into a pre-trained model for efficient processing. This work bridges advancements from both areas, highlighting how transformers' implicit meta-learning abilities, pre-training on drifting natural data, and reliance on context optimization directly address the core challenges of adaptive learning in dynamic environments. Exploring real-time model adaptation, this research demonstrates that TabPFN, coupled with a simple sliding memory strategy, consistently outperforms ensembles of Hoeffding trees across all non-stationary benchmarks. Several promising research directions are outlined in the paper. The authors urge the community to explore these ideas, offering valuable opportunities to advance in-context stream learning.

Title: Fair Foundation Models for Medical Image Analysis: Challenges and Perspectives

• Authors: Dilermando Queiroz, Anderson Carlos, André Anjos, Lilian Berton
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16841
• Pdf URL: https://arxiv.org/pdf/2502.16841
• Copy Paste: [[2502.16841]] Fair Foundation Models for Medical Image Analysis: Challenges and Perspectives(https://arxiv.org/abs/2502.16841)
• Keywords: self-supervised, foundation model
• Abstract: Ensuring equitable Artificial Intelligence (AI) in healthcare demands systems that make unbiased decisions across all demographic groups, bridging technical innovation with ethical principles. Foundation Models (FMs), trained on vast datasets through self-supervised learning, enable efficient adaptation across medical imaging tasks while reducing dependency on labeled data. These models demonstrate potential for enhancing fairness, though significant challenges remain in achieving consistent performance across demographic groups. Our review indicates that effective bias mitigation in FMs requires systematic interventions throughout all stages of development. While previous approaches focused primarily on model-level bias mitigation, our analysis reveals that fairness in FMs requires integrated interventions throughout the development pipeline, from data documentation to deployment protocols. This comprehensive framework advances current knowledge by demonstrating how systematic bias mitigation, combined with policy engagement, can effectively address both technical and institutional barriers to equitable AI in healthcare. The development of equitable FMs represents a critical step toward democratizing advanced healthcare technologies, particularly for underserved populations and regions with limited medical infrastructure and computational resources.

Title: Mitigating Hallucinations in Diffusion Models through Adaptive Attention Modulation

• Authors: Trevine Oorloff, Yaser Yacoob, Abhinav Shrivastava
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16872
• Pdf URL: https://arxiv.org/pdf/2502.16872
• Copy Paste: [[2502.16872]] Mitigating Hallucinations in Diffusion Models through Adaptive Attention Modulation(https://arxiv.org/abs/2502.16872)
• Keywords: diffusion
• Abstract: Diffusion models, while increasingly adept at generating realistic images, are notably hindered by hallucinations -- unrealistic or incorrect features inconsistent with the trained data distribution. In this work, we propose Adaptive Attention Modulation (AAM), a novel approach to mitigate hallucinations by analyzing and modulating the self-attention mechanism in diffusion models. We hypothesize that self-attention during early denoising steps may inadvertently amplify or suppress features, contributing to hallucinations. To counter this, AAM introduces a temperature scaling mechanism within the softmax operation of the self-attention layers, dynamically modulating the attention distribution during inference. Additionally, AAM employs a masked perturbation technique to disrupt early-stage noise that may otherwise propagate into later stages as hallucinations. Extensive experiments demonstrate that AAM effectively reduces hallucinatory artifacts, enhancing both the fidelity and reliability of generated images. For instance, the proposed approach improves the FID score by 20.8% and reduces the percentage of hallucinated images by 12.9% (in absolute terms) on the Hands dataset.

Title: Unveiling Institution-Specific Bias in Pathology Foundation Models: Detriments, Causes, and Potential Solutions

• Authors: Weiping Lin, Shen Liu, Runchen Zhu, Liansheng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16889
• Pdf URL: https://arxiv.org/pdf/2502.16889
• Copy Paste: [[2502.16889]] Unveiling Institution-Specific Bias in Pathology Foundation Models: Detriments, Causes, and Potential Solutions(https://arxiv.org/abs/2502.16889)
• Keywords: foundation model
• Abstract: Pathology foundation models (PFMs) extract valuable discriminative features from images for downstream clinical tasks. PFMs have simplified the development of deep learning models, effectively leveraging prior knowledge to improve diagnostic accuracy in diverse scenarios. However, we find that PFMs sometimes struggle with certain challenges. Specifically, features extracted by PFMs are often contaminated by diagnosis-irrelevant information, i.e., institution-specific features associated with the images. This contamination can lead to spurious correlations, undermining the models' generalization ability when applied in real-world clinical settings. In this work, we first reveal the issue of feature contamination in PFMs, demonstrate the presence of institution-specific features, thoroughly investigate its negative impacts, analyze the underlying causes, and provide insights into potential solutions. Specifically, we find that institution-specific information is embedded in pathological images and can be readily captured by current PFMs. Through extensive experiments, we demonstrate the detrimental impact of this irrelevant information, particularly in out-of-distribution (OOD) settings, where reliance on contaminated features leads to significant performance degradation. This indicates that the models are being misled by non-diagnostic information. We further delve into the reasons PFMs extract such institution-specific information and validate our findings. Finally, we propose a simple yet effective solution to mitigate the influence of irrelevant information. This study is not intended to criticize existing PFMs, as they have indeed greatly advanced the development of computational pathology. our aim is to inspire future research to focus on innovative training strategies, rather than relying exclusively on scaling laws, to realize more generalized PFMs.

Title: Applying LLMs to Active Learning: Towards Cost-Efficient Cross-Task Text Classification without Manually Labeled Data

• Authors: Yejian Zhang, Shingo Takada
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.16892
• Pdf URL: https://arxiv.org/pdf/2502.16892
• Copy Paste: [[2502.16892]] Applying LLMs to Active Learning: Towards Cost-Efficient Cross-Task Text Classification without Manually Labeled Data(https://arxiv.org/abs/2502.16892)
• Keywords: generative
• Abstract: Machine learning-based classifiers have been used for text classification, such as sentiment analysis, news classification, and toxic comment classification. However, supervised machine learning models often require large amounts of labeled data for training, and manual annotation is both labor-intensive and requires domain-specific knowledge, leading to relatively high annotation costs. To address this issue, we propose an approach that integrates large language models (LLMs) into an active learning framework. Our approach combines the Robustly Optimized BERT Pretraining Approach (RoBERTa), Generative Pre-trained Transformer (GPT), and active learning, achieving high cross-task text classification performance without the need for any manually labeled data. Furthermore, compared to directly applying GPT for classification tasks, our approach retains over 93% of its classification performance while requiring only approximately 6% of the computational time and monetary cost, effectively balancing performance and resource efficiency. These findings provide new insights into the efficient utilization of LLMs and active learning algorithms in text classification tasks, paving the way for their broader application.

Title: Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinment

• Authors: Suchae Jeong, Inseong Choi, Youngsik Yun, Jihie Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.16902
• Pdf URL: https://arxiv.org/pdf/2502.16902
• Copy Paste: [[2502.16902]] Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinment(https://arxiv.org/abs/2502.16902)
• Keywords: diffusion
• Abstract: Text-to-Image models, including Stable Diffusion, have significantly improved in generating images that are highly semantically aligned with the given prompts. However, existing models may fail to produce appropriate images for the cultural concepts or objects that are not well known or underrepresented in western cultures, such as `hangari' (Korean utensil). In this paper, we propose a novel approach, Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (Culture-TRIP), which refines the prompt in order to improve the alignment of the image with such culture nouns in text-to-image models. Our approach (1) retrieves cultural contexts and visual details related to the culture nouns in the prompt and (2) iteratively refines and evaluates the prompt based on a set of cultural criteria and large language models. The refinement process utilizes the information retrieved from Wikipedia and the Web. Our user survey, conducted with 66 participants from eight different countries demonstrates that our proposed approach enhances the alignment between the images and the prompts. In particular, C-TRIP demonstrates improved alignment between the generated images and underrepresented culture nouns. Resource can be found at this https URL.

Title: Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model

• Authors: Kang Fu, Huiyu Duan, Zicheng Zhang, Xiaohong Liu, Xiongkuo Min, Jia Wang, Guangtao Zhai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16915
• Pdf URL: https://arxiv.org/pdf/2502.16915
• Copy Paste: [[2502.16915]] Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model(https://arxiv.org/abs/2502.16915)
• Keywords: diffusion
• Abstract: Recent advancements in text-to-image (T2I) generation have spurred the development of text-to-3D asset (T23DA) generation, leveraging pretrained 2D text-to-image diffusion models for text-to-3D asset synthesis. Despite the growing popularity of text-to-3D asset generation, its evaluation has not been well considered and studied. However, given the significant quality discrepancies among various text-to-3D assets, there is a pressing need for quality assessment models aligned with human subjective judgments. To tackle this challenge, we conduct a comprehensive study to explore the T23DA quality assessment (T23DAQA) problem in this work from both subjective and objective perspectives. Given the absence of corresponding databases, we first establish the largest text-to-3D asset quality assessment database to date, termed the AIGC-T23DAQA database. This database encompasses 969 validated 3D assets generated from 170 prompts via 6 popular text-to-3D asset generation models, and corresponding subjective quality ratings for these assets from the perspectives of quality, authenticity, and text-asset correspondence, respectively. Subsequently, we establish a comprehensive benchmark based on the AIGC-T23DAQA database, and devise an effective T23DAQA model to evaluate the generated 3D assets from the aforementioned three perspectives, respectively.

Title: MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection

• Authors: Farzad Beizaee, Gregory Lodygensky, Christian Desrosiers, Jose Dolz
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.16943
• Pdf URL: https://arxiv.org/pdf/2502.16943
• Copy Paste: [[2502.16943]] MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection(https://arxiv.org/abs/2502.16943)
• Keywords: diffusion, generative, anomaly
• Abstract: Unsupervised anomaly detection in brain images is crucial for identifying injuries and pathologies without access to labels. However, the accurate localization of anomalies in medical images remains challenging due to the inherent complexity and variability of brain structures and the scarcity of annotated abnormal data. To address this challenge, we propose a novel approach that incorporates masking within diffusion models, leveraging their generative capabilities to learn robust representations of normal brain anatomy. During training, our model processes only normal brain MRI scans and performs a forward diffusion process in the latent space that adds noise to the features of randomly-selected patches. Following a dual objective, the model learns to identify which patches are noisy and recover their original features. This strategy ensures that the model captures intricate patterns of normal brain structures while isolating potential anomalies as noise in the latent space. At inference, the model identifies noisy patches corresponding to anomalies and generates a normal counterpart for these patches by applying a reverse diffusion process. Our method surpasses existing unsupervised anomaly detection techniques, demonstrating superior performance in generating accurate normal counterparts and localizing anomalies. The code is available at hhttps://github.com/farzad-bz/MAD-AD.

Title: Autoregressive Image Generation Guided by Chains of Thought

• Authors: Miaomiao Cai, Guanjie Wang, Wei Li, Zhijun Tu, Hanting Chen, Shaohui Lin, Jie Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.16965
• Pdf URL: https://arxiv.org/pdf/2502.16965
• Copy Paste: [[2502.16965]] Autoregressive Image Generation Guided by Chains of Thought(https://arxiv.org/abs/2502.16965)
• Keywords: diffusion
• Abstract: In the field of autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases. However, directly applying LLMs to complex image generation can struggle with reconstructing the structure and details of the image, impacting the accuracy and stability of generation. Additionally, the 'next-token prediction' paradigm in the AR model does not align with the contextual scanning and logical reasoning processes involved in human visual perception, limiting effective image generation. Chain-of-Thought (CoT), as a key reasoning capability of LLMs, utilizes reasoning prompts to guide the model, improving reasoning performance on complex natural language process (NLP) tasks, enhancing accuracy and stability of generation, and helping the model maintain contextual coherence and logical consistency, similar to human reasoning. Inspired by CoT from the field of NLP, we propose autoregressive Image Generation with Thoughtful Reasoning (IGTR) to enhance autoregressive image generation. IGTR adds reasoning prompts without modifying the model structure or raster generation order. Specifically, we design specialized image-related reasoning prompts for AR image generation to simulate the human reasoning process, which enhances contextual reasoning by allowing the model to first perceive overall distribution information before generating the image, and improve generation stability by increasing the inference steps. Compared to the AR method without prompts, our method shows outstanding performance and achieves an approximate improvement of 20%.

Title: TraFlow: Trajectory Distillation on Pre-Trained Rectified Flow

• Authors: Zhangkai Wu, Xuhui Fan, Hongyu Wu, Longbing Cao
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.16972
• Pdf URL: https://arxiv.org/pdf/2502.16972
• Copy Paste: [[2502.16972]] TraFlow: Trajectory Distillation on Pre-Trained Rectified Flow(https://arxiv.org/abs/2502.16972)
• Keywords: diffusion
• Abstract: Majorities of distillation methods on pre-trained diffusion models or on pre-trained rectified flow, focus on either the distillation outputs or the trajectories between random noises and clean images to speed up sample generations from pre-trained models. In those trajectory-based distillation methods, consistency distillation requires the self-consistent trajectory projection to regulate the trajectory, which might avoid the common ODE approximation error {while still be concerning about sampling efficiencies}. At the same time, rectified flow distillations enforce straight trajectory for fast sampling, although an ODE solver is still required. In this work, we propose a trajectory distillation method, \modelname, that enjoys the benefits of both and enables few-step generations. TraFlow adopts the settings of consistency trajectory models, and further enforces the properties of self-consistency and straightness throughout the entire trajectory. These two properties are pursued by reaching a balance with following three targets: (1) reconstruct the output from pre-trained models; (2) learn the amount of changes by pre-trained models; (3) satisfy the self-consistency over its trajectory. Extensive experimental results have shown the effectiveness of our proposed method.

Title: Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization

• Authors: Zixuan Gong, Xiaolin Hu, Huayi Tang, Yong Liu
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.17024
• Pdf URL: https://arxiv.org/pdf/2502.17024
• Copy Paste: [[2502.17024]] Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization(https://arxiv.org/abs/2502.17024)
• Keywords: in-context
• Abstract: Large language models (LLMs) have demonstrated remarkable in-context learning (ICL) abilities. However, existing theoretical analysis of ICL primarily exhibits two limitations: (a) Limited i.i.d. Setting. Most studies focus on supervised function learning tasks where prompts are constructed with i.i.d. input-label pairs. This i.i.d. assumption diverges significantly from real language learning scenarios where prompt tokens are interdependent. (b) Lack of Emergence Explanation. Most literature answers what ICL does from an implicit optimization perspective but falls short in elucidating how ICL emerges and the impact of pre-training phase on ICL. In our paper, to extend (a), we adopt a more practical paradigm, auto-regressive next-token prediction (AR-NTP), which closely aligns with the actual training of language models. Specifically, within AR-NTP, we emphasize prompt token-dependency, which involves predicting each subsequent token based on the preceding sequence. To address (b), we formalize a systematic pre-training and ICL framework, highlighting the layer-wise structure of sequences and topics, alongside a two-level expectation. In conclusion, we present data-dependent, topic-dependent and optimization-dependent PAC-Bayesian generalization bounds for pre-trained LLMs, investigating that ICL emerges from the generalization of sequences and topics. Our theory is supported by experiments on numerical linear dynamic systems, synthetic GINC and real-world language datasets.

Title: PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance

• Authors: Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, Yangqiu Song
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.17041
• Pdf URL: https://arxiv.org/pdf/2502.17041
• Copy Paste: [[2502.17041]] PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance(https://arxiv.org/abs/2502.17041)
• Keywords: generative
• Abstract: Recent advancements in generative large language models (LLMs) have enabled wider applicability, accessibility, and flexibility. However, their reliability and trustworthiness are still in doubt, especially for concerns regarding individuals' data privacy. Great efforts have been made on privacy by building various evaluation benchmarks to study LLMs' privacy awareness and robustness from their generated outputs to their hidden representations. Unfortunately, most of these works adopt a narrow formulation of privacy and only investigate personally identifiable information (PII). In this paper, we follow the merit of the Contextual Integrity (CI) theory, which posits that privacy evaluation should not only cover the transmitted attributes but also encompass the whole relevant social context through private information flows. We present PrivaCI-Bench, a comprehensive contextual privacy evaluation benchmark targeted at legal compliance to cover well-annotated privacy and safety regulations, real court cases, privacy policies, and synthetic data built from the official toolkit to study LLMs' privacy and safety compliance. We evaluate the latest LLMs, including the recent reasoner models QwQ-32B and Deepseek R1. Our experimental results suggest that though LLMs can effectively capture key CI parameters inside a given context, they still require further advancements for privacy compliance.

Title: SpecDM: Hyperspectral Dataset Synthesis with Pixel-level Semantic Annotations

• Authors: Wendi Liu, Pei Yang, Wenhui Hong, Xiaoguang Mei, Jiayi Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17056
• Pdf URL: https://arxiv.org/pdf/2502.17056
• Copy Paste: [[2502.17056]] SpecDM: Hyperspectral Dataset Synthesis with Pixel-level Semantic Annotations(https://arxiv.org/abs/2502.17056)
• Keywords: diffusion, generative
• Abstract: In hyperspectral remote sensing field, some downstream dense prediction tasks, such as semantic segmentation (SS) and change detection (CD), rely on supervised learning to improve model performance and require a large amount of manually annotated data for training. However, due to the needs of specific equipment and special application scenarios, the acquisition and annotation of hyperspectral images (HSIs) are often costly and time-consuming. To this end, our work explores the potential of generative diffusion model in synthesizing HSIs with pixel-level annotations. The main idea is to utilize a two-stream VAE to learn the latent representations of images and corresponding masks respectively, learn their joint distribution during the diffusion model training, and finally obtain the image and mask through their respective decoders. To the best of our knowledge, it is the first work to generate high-dimensional HSIs with annotations. Our proposed approach can be applied in various kinds of dataset generation. We select two of the most widely used dense prediction tasks: semantic segmentation and change detection, and generate datasets suitable for these tasks. Experiments demonstrate that our synthetic datasets have a positive impact on the improvement of these downstream tasks.

Title: DUNIA: Pixel-Sized Embeddings via Cross-Modal Alignment for Earth Observation Applications

• Authors: Ibrahim Fayad, Max Zimmer, Martin Schwartz, Philippe Ciais, Fabian Gieseke, Gabriel Belouze, Sarah Brood, Aurelien De Truchis, Alexandre d'Aspremont
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17066
• Pdf URL: https://arxiv.org/pdf/2502.17066
• Copy Paste: [[2502.17066]] DUNIA: Pixel-Sized Embeddings via Cross-Modal Alignment for Earth Observation Applications(https://arxiv.org/abs/2502.17066)
• Keywords: self-supervised
• Abstract: Significant efforts have been directed towards adapting self-supervised multimodal learning for Earth observation applications. However, existing methods produce coarse patch-sized embeddings, limiting their effectiveness and integration with other modalities like LiDAR. To close this gap, we present DUNIA, an approach to learn pixel-sized embeddings through cross-modal alignment between images and full-waveform LiDAR data. As the model is trained in a contrastive manner, the embeddings can be directly leveraged in the context of a variety of environmental monitoring tasks in a zero-shot setting. In our experiments, we demonstrate the effectiveness of the embeddings for seven such tasks (canopy height mapping, fractional canopy cover, land cover mapping, tree species identification, plant area index, crop type classification, and per-pixel waveform-based vertical structure mapping). The results show that the embeddings, along with zero-shot classifiers, often outperform specialized supervised models, even in low data regimes. In the fine-tuning setting, we show strong low-shot capabilities with performances near or better than state-of-the-art on five out of six tasks.

Title: Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence

• Authors: Bolin Chen, Hanwei Zhu, Shanzhi Yin, Lingyu Zhu, Jie Chen, Ru-Ling Liao, Shiqi Wang, Yan Ye
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.17085
• Pdf URL: https://arxiv.org/pdf/2502.17085
• Copy Paste: [[2502.17085]] Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence(https://arxiv.org/abs/2502.17085)
• Keywords: generative
• Abstract: Generative model based compact video compression is typically operated within a relative narrow range of bitrates, and often with an emphasis on ultra-low rate applications. There has been an increasing consensus in the video communication industry that full bitrate coverage should be enabled by generative coding. However, this is an extremely difficult task, largely because generation and compression, although related, have distinct goals and trade-offs. The proposed Pleno-Generation (PGen) framework distinguishes itself through its exceptional capabilities in ensuring the robustness of video coding by utilizing a wider range of bandwidth for generation via bandwidth intelligence. In particular, we initiate our research of PGen with face video coding, and PGen offers a paradigm shift that prioritizes high-fidelity reconstruction over pursuing compact bitstream. The novel PGen framework leverages scalable representation and layered reconstruction for Generative Face Video Compression (GFVC), in an attempt to imbue the bitstream with intelligence in different granularity. Experimental results illustrate that the proposed PGen framework can facilitate existing GFVC algorithms to better deliver high-fidelity and faithful face videos. In addition, the proposed framework can allow a greater space of flexibility for coding applications and show superior RD performance with a much wider bitrate range in terms of various quality evaluations. Moreover, in comparison with the latest Versatile Video Coding (VVC) codec, the proposed scheme achieves competitive Bjøntegaard-delta-rate savings for perceptual-level evaluations.

Title: Enhancing Image Matting in Real-World Scenes with Mask-Guided Iterative Refinement

• Authors: Rui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17093
• Pdf URL: https://arxiv.org/pdf/2502.17093
• Copy Paste: [[2502.17093]] Enhancing Image Matting in Real-World Scenes with Mask-Guided Iterative Refinement(https://arxiv.org/abs/2502.17093)
• Keywords: self-supervised
• Abstract: Real-world image matting is essential for applications in content creation and augmented reality. However, it remains challenging due to the complex nature of scenes and the scarcity of high-quality datasets. To address these limitations, we introduce Mask2Alpha, an iterative refinement framework designed to enhance semantic comprehension, instance awareness, and fine-detail recovery in image matting. Our framework leverages self-supervised Vision Transformer features as semantic priors, strengthening contextual understanding in complex scenarios. To further improve instance differentiation, we implement a mask-guided feature selection module, enabling precise targeting of objects in multi-instance settings. Additionally, a sparse convolution-based optimization scheme allows Mask2Alpha to recover high-resolution details through progressive refinement,from low-resolution semantic passes to high-resolution sparse reconstructions. Benchmarking across various real-world datasets, Mask2Alpha consistently achieves state-of-the-art results, showcasing its effectiveness in accurate and efficient image matting.

Title: Improved Diffusion-based Generative Model with Better Adversarial Robustness

• Authors: Zekun Wang, Mingyang Yi, Shuchen Xue, Zhenguo Li, Ming Liu, Bing Qin, Zhi-Ming Ma
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17099
• Pdf URL: https://arxiv.org/pdf/2502.17099
• Copy Paste: [[2502.17099]] Improved Diffusion-based Generative Model with Better Adversarial Robustness(https://arxiv.org/abs/2502.17099)
• Keywords: diffusion, generative
• Abstract: Diffusion Probabilistic Models (DPMs) have achieved significant success in generative tasks. However, their training and sampling processes suffer from the issue of distribution mismatch. During the denoising process, the input data distributions differ between the training and inference stages, potentially leading to inaccurate data generation. To obviate this, we analyze the training objective of DPMs and theoretically demonstrate that this mismatch can be alleviated through Distributionally Robust Optimization (DRO), which is equivalent to performing robustness-driven Adversarial Training (AT) on DPMs. Furthermore, for the recently proposed Consistency Model (CM), which distills the inference process of the DPM, we prove that its training objective also encounters the mismatch issue. Fortunately, this issue can be mitigated by AT as well. Based on these insights, we propose to conduct efficient AT on both DPM and CM. Finally, extensive empirical studies validate the effectiveness of AT in diffusion-based models. The code is available at this https URL.

Title: Generative Models in Decision Making: A Survey

• Authors: Yinchuan Li, Xinyu Shao, Jianping Zhang, Haozhi Wang, Leo Maxime Brunswic, Kaiwen Zhou, Jiqian Dong, Kaiyang Guo, Xiu Li, Zhitang Chen, Jun Wang, Jianye Hao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17100
• Pdf URL: https://arxiv.org/pdf/2502.17100
• Copy Paste: [[2502.17100]] Generative Models in Decision Making: A Survey(https://arxiv.org/abs/2502.17100)
• Keywords: diffusion, generative
• Abstract: In recent years, the exceptional performance of generative models in generative tasks has sparked significant interest in their integration into decision-making processes. Due to their ability to handle complex data distributions and their strong model capacity, generative models can be effectively incorporated into decision-making systems by generating trajectories that guide agents toward high-reward state-action regions or intermediate sub-goals. This paper presents a comprehensive review of the application of generative models in decision-making tasks. We classify seven fundamental types of generative models: energy-based models, generative adversarial networks, variational autoencoders, normalizing flows, diffusion models, generative flow networks, and autoregressive models. Regarding their applications, we categorize their functions into three main roles: controllers, modelers and optimizers, and discuss how each role contributes to decision-making. Furthermore, we examine the deployment of these models across five critical real-world decision-making scenarios. Finally, we summarize the strengths and limitations of current approaches and propose three key directions for advancing next-generation generative directive models: high-performance algorithms, large-scale generalized decision-making models, and self-evolving and adaptive models.

Title: SFLD: Reducing the content bias for AI-generated Image Detection

• Authors: Seoyeon Gye, Junwon Ko, Hyounguk Shon, Minchan Kwon, Junmo Kim
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17105
• Pdf URL: https://arxiv.org/pdf/2502.17105
• Copy Paste: [[2502.17105]] SFLD: Reducing the content bias for AI-generated Image Detection(https://arxiv.org/abs/2502.17105)
• Keywords: diffusion, generative
• Abstract: Identifying AI-generated content is critical for the safe and ethical use of generative AI. Recent research has focused on developing detectors that generalize to unknown generators, with popular methods relying either on high-level features or low-level fingerprints. However, these methods have clear limitations: biased towards unseen content, or vulnerable to common image degradations, such as JPEG compression. To address these issues, we propose a novel approach, SFLD, which incorporates PatchShuffle to integrate high-level semantic and low-level textural information. SFLD applies PatchShuffle at multiple levels, improving robustness and generalization across various generative models. Additionally, current benchmarks face challenges such as low image quality, insufficient content preservation, and limited class diversity. In response, we introduce TwinSynths, a new benchmark generation methodology that constructs visually near-identical pairs of real and synthetic images to ensure high quality and content preservation. Our extensive experiments and analysis show that SFLD outperforms existing methods on detecting a wide variety of fake images sourced from GANs, diffusion models, and TwinSynths, demonstrating the state-of-the-art performance and generalization capabilities to novel generative models.

Title: Diffusion Models for Tabular Data: Challenges, Current Progress, and Future Directions

• Authors: Zhong Li, Qi Huang, Lincen Yang, Jiayang Shi, Zhao Yang, Niki van Stein, Thomas Bäck, Matthijs van Leeuwen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17119
• Pdf URL: https://arxiv.org/pdf/2502.17119
• Copy Paste: [[2502.17119]] Diffusion Models for Tabular Data: Challenges, Current Progress, and Future Directions(https://arxiv.org/abs/2502.17119)
• Keywords: diffusion, generative
• Abstract: In recent years, generative models have achieved remarkable performance across diverse applications, including image generation, text synthesis, audio creation, video generation, and data augmentation. Diffusion models have emerged as superior alternatives to Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) by addressing their limitations, such as training instability, mode collapse, and poor representation of multimodal distributions. This success has spurred widespread research interest. In the domain of tabular data, diffusion models have begun to showcase similar advantages over GANs and VAEs, achieving significant performance breakthroughs and demonstrating their potential for addressing unique challenges in tabular data modeling. However, while domains like images and time series have numerous surveys summarizing advancements in diffusion models, there remains a notable gap in the literature for tabular data. Despite the increasing interest in diffusion models for tabular data, there has been little effort to systematically review and summarize these developments. This lack of a dedicated survey limits a clear understanding of the challenges, progress, and future directions in this critical area. This survey addresses this gap by providing a comprehensive review of diffusion models for tabular data. Covering works from June 2015, when diffusion models emerged, to December 2024, we analyze nearly all relevant studies, with updates maintained in a \href{this https URL}{GitHub repository}. Assuming readers possess foundational knowledge of statistics and diffusion models, we employ mathematical formulations to deliver a rigorous and detailed review, aiming to promote developments in this emerging and exciting area.

Title: Adversarial Training for Defense Against Label Poisoning Attacks

• Authors: Melis Ilayda Bal, Volkan Cevher, Michael Muehlebach
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17121
• Pdf URL: https://arxiv.org/pdf/2502.17121
• Copy Paste: [[2502.17121]] Adversarial Training for Defense Against Label Poisoning Attacks(https://arxiv.org/abs/2502.17121)
• Keywords: foundation model
• Abstract: As machine learning models grow in complexity and increasingly rely on publicly sourced data, such as the human-annotated labels used in training large language models, they become more vulnerable to label poisoning attacks. These attacks, in which adversaries subtly alter the labels within a training dataset, can severely degrade model performance, posing significant risks in critical applications. In this paper, we propose FLORAL, a novel adversarial training defense strategy based on support vector machines (SVMs) to counter these threats. Utilizing a bilevel optimization framework, we cast the training process as a non-zero-sum Stackelberg game between an attacker, who strategically poisons critical training labels, and the model, which seeks to recover from such attacks. Our approach accommodates various model architectures and employs a projected gradient descent algorithm with kernel SVMs for adversarial training. We provide a theoretical analysis of our algorithm's convergence properties and empirically evaluate FLORAL's effectiveness across diverse classification tasks. Compared to robust baselines and foundation models such as RoBERTa, FLORAL consistently achieves higher robust accuracy under increasing attacker budgets. These results underscore the potential of FLORAL to enhance the resilience of machine learning models against label poisoning threats, thereby ensuring robust classification in adversarial settings.

Title: DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks

• Authors: Canyu Zhao, Mingyu Liu, Huanyi Zheng, Muzhi Zhu, Zhiyue Zhao, Hao Chen, Tong He, Chunhua Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17157
• Pdf URL: https://arxiv.org/pdf/2502.17157
• Copy Paste: [[2502.17157]] DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks(https://arxiv.org/abs/2502.17157)
• Keywords: diffusion
• Abstract: Our primary goal here is to create a good, generalist perception model that can tackle multiple tasks, within limits on computational resources and training data. To achieve this, we resort to text-to-image diffusion models pre-trained on billions of images. Our exhaustive evaluation metrics demonstrate that DICEPTION effectively tackles multiple perception tasks, achieving performance on par with state-of-the-art models. We achieve results on par with SAM-vit-h using only 0.06% of their data (e.g., 600K vs. 1B pixel-level annotated images). Inspired by Wang et al., DICEPTION formulates the outputs of various perception tasks using color encoding; and we show that the strategy of assigning random colors to different instances is highly effective in both entity segmentation and semantic segmentation. Unifying various perception tasks as conditional image generation enables us to fully leverage pre-trained text-to-image models. Thus, DICEPTION can be efficiently trained at a cost of orders of magnitude lower, compared to conventional models that were trained from scratch. When adapting our model to other tasks, it only requires fine-tuning on as few as 50 images and 1% of its parameters. DICEPTION provides valuable insights and a more promising solution for visual generalist models.

Title: A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis

• Authors: Yuli Wu, Fucheng Liu, Rüveyda Yilmaz, Henning Konermann, Peter Walter, Johannes Stegmaier
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17160
• Pdf URL: https://arxiv.org/pdf/2502.17160
• Copy Paste: [[2502.17160]] A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis(https://arxiv.org/abs/2502.17160)
• Keywords: generative
• Abstract: Fréchet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic data match real data in many image synthesis tasks, the primary goal in biomedical generative models is often to enrich training datasets ideally with corresponding annotations. For this purpose, the gold standard for evaluating generative models is to incorporate synthetic data into downstream task training, such as classification and segmentation, to pragmatically assess its performance. In this paper, we examine cases from retinal imaging modalities, including color fundus photography and optical coherence tomography, where FID and its related metrics misalign with task-specific evaluation goals in classification and segmentation. We highlight the limitations of using various metrics, represented by FID and its variants, as evaluation criteria for these applications and address their potential caveats in broader biomedical imaging modalities and downstream tasks.

Title: Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch

• Authors: Xueru Wen, Jie Lou, Zichao Li, Yaojie Lu, Xing Yu, Yuqiu Ji, Guohai Xu, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Debing Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.17173
• Pdf URL: https://arxiv.org/pdf/2502.17173
• Copy Paste: [[2502.17173]] Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch(https://arxiv.org/abs/2502.17173)
• Keywords: generative
• Abstract: Reward models (RMs) are crucial for aligning large language models (LLMs) with human preferences. However, most RM research is centered on English and relies heavily on synthetic resources, which leads to limited and less reliable datasets and benchmarks for Chinese. To address this gap, we introduce CheemsBench, a fully human-annotated RM evaluation benchmark within Chinese contexts, and CheemsPreference, a large-scale and diverse preference dataset annotated through human-machine collaboration to support Chinese RM training. We systematically evaluate open-source discriminative and generative RMs on CheemsBench and observe significant limitations in their ability to capture human preferences in Chinese scenarios. Additionally, based on CheemsPreference, we construct an RM that achieves state-of-the-art performance on CheemsBench, demonstrating the necessity of human supervision in RM training. Our findings reveal that scaled AI-generated data struggles to fully capture human preferences, emphasizing the importance of high-quality human supervision in RM development.

Title: Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation

• Authors: Baptiste Chopin, Tashvik Dhamija, Pranav Balaji, Yaohui Wang, Antitza Dantcheva
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17198
• Pdf URL: https://arxiv.org/pdf/2502.17198
• Copy Paste: [[2502.17198]] Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation(https://arxiv.org/abs/2502.17198)
• Keywords: diffusion
• Abstract: We propose Dimitra, a novel framework for audio-driven talking head generation, streamlined to learn lip motion, facial expression, as well as head pose motion. Specifically, we train a conditional Motion Diffusion Transformer (cMDT) by modeling facial motion sequences with 3D representation. We condition the cMDT with only two input signals, an audio-sequence, as well as a reference facial image. By extracting additional features directly from audio, Dimitra is able to increase quality and realism of generated videos. In particular, phoneme sequences contribute to the realism of lip motion, whereas text transcript to facial expression and head pose realism. Quantitative and qualitative experiments on two widely employed datasets, VoxCeleb2 and HDTF, showcase that Dimitra is able to outperform existing approaches for generating realistic talking heads imparting lip motion, facial expression, and head pose.

Title: VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing

• Authors: Xiangpeng Yang, Linchao Zhu, Hehe Fan, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17258
• Pdf URL: https://arxiv.org/pdf/2502.17258
• Copy Paste: [[2502.17258]] VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing(https://arxiv.org/abs/2502.17258)
• Keywords: diffusion
• Abstract: Recent advancements in diffusion models have significantly improved video generation and editing capabilities. However, multi-grained video editing, which encompasses class-level, instance-level, and part-level modifications, remains a formidable challenge. The major difficulties in multi-grained editing include semantic misalignment of text-to-region control and feature coupling within the diffusion model. To address these difficulties, we present VideoGrain, a zero-shot approach that modulates space-time (cross- and self-) attention mechanisms to achieve fine-grained control over video content. We enhance text-to-region control by amplifying each local prompt's attention to its corresponding spatial-disentangled region while minimizing interactions with irrelevant areas in cross-attention. Additionally, we improve feature separation by increasing intra-region awareness and reducing inter-region interference in self-attention. Extensive experiments demonstrate our method achieves state-of-the-art performance in real-world scenarios. Our code, data, and demos are available at this https URL

Title: A Closer Look at TabPFN v2: Strength, Limitation, and Extension

• Authors: Han-Jia Ye, Si-Yang Liu, Wei-Lun Chao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17361
• Pdf URL: https://arxiv.org/pdf/2502.17361
• Copy Paste: [[2502.17361]] A Closer Look at TabPFN v2: Strength, Limitation, and Extension(https://arxiv.org/abs/2502.17361)
• Keywords: foundation model, in-context
• Abstract: Tabular datasets are inherently heterogeneous, posing significant challenges for developing pre-trained foundation models. The recently introduced transformer-based Tabular Prior-data Fitted Network v2 (TabPFN v2) achieves unprecedented in-context learning accuracy across multiple tabular datasets, marking a pivotal advancement in tabular foundation models. In this paper, we comprehensively evaluate TabPFN v2 on over 300 datasets, confirming its exceptional generalization capabilities on small- to medium-scale tasks. Our analysis identifies randomized feature tokens as a key factor behind TabPFN v2's success, as they unify heterogeneous datasets into a fixed-dimensional representation, enabling more effective training and inference. To further understand TabPFN v2's predictions, we propose a leave-one-fold-out approach, transforming TabPFN v2 into a feature extractor and revealing its capability to simplify data distributions and boost accuracy. Lastly, to address TabPFN v2's limitations in high-dimensional, large-scale, and many-category tasks, we introduce a divide-and-conquer mechanism inspired by Chain-of-Thought prompting, enabling scalable inference. By uncovering the mechanisms behind TabPFN v2's success and introducing strategies to expand its applicability, this study provides key insights into the future of tabular foundation models.

Title: KV-Edit: Training-Free Image Editing for Precise Background Preservation

• Authors: Tianrui Zhu, Shiyi Zhang, Jiawei Shao, Yansong Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17363
• Pdf URL: https://arxiv.org/pdf/2502.17363
• Copy Paste: [[2502.17363]] KV-Edit: Training-Free Image Editing for Precise Background Preservation(https://arxiv.org/abs/2502.17363)
• Keywords: generative
• Abstract: Background consistency remains a significant challenge in image editing tasks. Despite extensive developments, existing works still face a trade-off between maintaining similarity to the original image and generating content that aligns with the target. Here, we propose KV-Edit, a training-free approach that uses KV cache in DiTs to maintain background consistency, where background tokens are preserved rather than regenerated, eliminating the need for complex mechanisms or expensive training, ultimately generating new content that seamlessly integrates with the background within user-provided regions. We further explore the memory consumption of the KV cache during editing and optimize the space complexity to $O(1)$ using an inversion-free method. Our approach is compatible with any DiT-based generative model without additional training. Experiments demonstrate that KV-Edit significantly outperforms existing approaches in terms of both background and image quality, even surpassing training-based methods. Project webpage is available at this https URL

Title: Large Language Models are Powerful EHR Encoders

• Authors: Stefan Hegselmann, Georg von Arnim, Tillmann Rheude, Noel Kronenberg, David Sontag, Gerhard Hindricks, Roland Eils, Benjamin Wild
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.17403
• Pdf URL: https://arxiv.org/pdf/2502.17403
• Copy Paste: [[2502.17403]] Large Language Models are Powerful EHR Encoders(https://arxiv.org/abs/2502.17403)
• Keywords: foundation model
• Abstract: Electronic Health Records (EHRs) offer rich potential for clinical prediction, yet their inherent complexity and heterogeneity pose significant challenges for traditional machine learning approaches. Domain-specific EHR foundation models trained on large collections of unlabeled EHR data have demonstrated promising improvements in predictive accuracy and generalization; however, their training is constrained by limited access to diverse, high-quality datasets and inconsistencies in coding standards and healthcare practices. In this study, we explore the possibility of using general-purpose Large Language Models (LLMs) based embedding methods as EHR encoders. By serializing patient records into structured Markdown text, transforming codes into human-readable descriptors, we leverage the extensive generalization capabilities of LLMs pretrained on vast public corpora, thereby bypassing the need for proprietary medical datasets. We systematically evaluate two state-of-the-art LLM-embedding models, GTE-Qwen2-7B-Instruct and LLM2Vec-Llama3.1-8B-Instruct, across 15 diverse clinical prediction tasks from the EHRSHOT benchmark, comparing their performance to an EHRspecific foundation model, CLIMBR-T-Base, and traditional machine learning baselines. Our results demonstrate that LLM-based embeddings frequently match or exceed the performance of specialized models, even in few-shot settings, and that their effectiveness scales with the size of the underlying LLM and the available context window. Overall, our findings demonstrate that repurposing LLMs for EHR encoding offers a scalable and effective approach for clinical prediction, capable of overcoming the limitations of traditional EHR modeling and facilitating more interoperable and generalizable healthcare applications.

Title: X-Dancer: Expressive Music to Human Dance Video Generation

• Authors: Zeyuan Chen, Hongyi Xu, Guoxian Song, You Xie, Chenxu Zhang, Xin Chen, Chao Wang, Di Chang, Linjie Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17414
• Pdf URL: https://arxiv.org/pdf/2502.17414
• Copy Paste: [[2502.17414]] X-Dancer: Expressive Music to Human Dance Video Generation(https://arxiv.org/abs/2502.17414)
• Keywords: diffusion
• Abstract: We present X-Dancer, a novel zero-shot music-driven image animation pipeline that creates diverse and long-range lifelike human dance videos from a single static image. As its core, we introduce a unified transformer-diffusion framework, featuring an autoregressive transformer model that synthesize extended and music-synchronized token sequences for 2D body, head and hands poses, which then guide a diffusion model to produce coherent and realistic dance video frames. Unlike traditional methods that primarily generate human motion in 3D, X-Dancer addresses data limitations and enhances scalability by modeling a wide spectrum of 2D dance motions, capturing their nuanced alignment with musical beats through readily available monocular videos. To achieve this, we first build a spatially compositional token representation from 2D human pose labels associated with keypoint confidences, encoding both large articulated body movements (e.g., upper and lower body) and fine-grained motions (e.g., head and hands). We then design a music-to-motion transformer model that autoregressively generates music-aligned dance pose token sequences, incorporating global attention to both musical style and prior motion context. Finally we leverage a diffusion backbone to animate the reference image with these synthesized pose tokens through AdaIN, forming a fully differentiable end-to-end framework. Experimental results demonstrate that X-Dancer is able to produce both diverse and characterized dance videos, substantially outperforming state-of-the-art methods in term of diversity, expressiveness and realism. Code and model will be available for research purposes.

Title: S4S: Solving for a Diffusion Model Solver

• Authors: Eric Frankel, Sitan Chen, Jerry Li, Pang Wei Koh, Lillian J. Ratliff, Sewoong Oh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.17423
• Pdf URL: https://arxiv.org/pdf/2502.17423
• Copy Paste: [[2502.17423]] S4S: Solving for a Diffusion Model Solver(https://arxiv.org/abs/2502.17423)
• Keywords: diffusion
• Abstract: Diffusion models (DMs) create samples from a data distribution by starting from random noise and iteratively solving a reverse-time ordinary differential equation (ODE). Because each step in the iterative solution requires an expensive neural function evaluation (NFE), there has been significant interest in approximately solving these diffusion ODEs with only a few NFEs without modifying the underlying model. However, in the few NFE regime, we observe that tracking the true ODE evolution is fundamentally impossible using traditional ODE solvers. In this work, we propose a new method that learns a good solver for the DM, which we call Solving for the Solver (S4S). S4S directly optimizes a solver to obtain good generation quality by learning to match the output of a strong teacher solver. We evaluate S4S on six different pre-trained DMs, including pixel-space and latent-space DMs for both conditional and unconditional sampling. In all settings, S4S uniformly improves the sample quality relative to traditional ODE solvers. Moreover, our method is lightweight, data-free, and can be plugged in black-box on top of any discretization schedule or architecture to improve performance. Building on top of this, we also propose S4S-Alt, which optimizes both the solver and the discretization schedule. By exploiting the full design space of DM solvers, with 5 NFEs, we achieve an FID of 3.73 on CIFAR10 and 13.26 on MS-COCO, representing a $1.5\times$ improvement over previous training-free ODE methods.

Title: GCC: Generative Color Constancy via Diffusing a Color Checker

• Authors: Chen-Wei Chang, Cheng-De Fan, Chia-Che Chang, Yi-Chen Lo, Yu-Chee Tseng, Jiun-Long Huang, Yu-Lun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17435
• Pdf URL: https://arxiv.org/pdf/2502.17435
• Copy Paste: [[2502.17435]] GCC: Generative Color Constancy via Diffusing a Color Checker(https://arxiv.org/abs/2502.17435)
• Keywords: diffusion, generative
• Abstract: Color constancy methods often struggle to generalize across different camera sensors due to varying spectral sensitivities. We present GCC, which leverages diffusion models to inpaint color checkers into images for illumination estimation. Our key innovations include (1) a single-step deterministic inference approach that inpaints color checkers reflecting scene illumination, (2) a Laplacian decomposition technique that preserves checker structure while allowing illumination-dependent color adaptation, and (3) a mask-based data augmentation strategy for handling imprecise color checker annotations. GCC demonstrates superior robustness in cross-camera scenarios, achieving state-of-the-art worst-25% error rates of 5.15° and 4.32° in bi-directional evaluations. These results highlight our method's stability and generalization capability across different camera characteristics without requiring sensor-specific training, making it a versatile solution for real-world applications.

Title: Fractal Generative Models

• Authors: Tianhong Li, Qinyi Sun, Lijie Fan, Kaiming He
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.17437
• Pdf URL: https://arxiv.org/pdf/2502.17437
• Copy Paste: [[2502.17437]] Fractal Generative Models(https://arxiv.org/abs/2502.17437)
• Keywords: generative
• Abstract: Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at this https URL.