2025-02-12

Title: Prompt-Aware Scheduling for Efficient Text-to-Image Inferencing System

• Authors: Shubham Agarwal, Saud Iqbal, Subrata Mitra
• Subjects: cs.LG, cs.DC, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06798
• Pdf URL: https://arxiv.org/pdf/2502.06798
• Copy Paste: [[2502.06798]] Prompt-Aware Scheduling for Efficient Text-to-Image Inferencing System(https://arxiv.org/abs/2502.06798)
• Keywords: generative
• Abstract: Traditional ML models utilize controlled approximations during high loads, employing faster, but less accurate models in a process called accuracy scaling. However, this method is less effective for generative text-to-image models due to their sensitivity to input prompts and performance degradation caused by large model loading overheads. This work introduces a novel text-to-image inference system that optimally matches prompts across multiple instances of the same model operating at various approximation levels to deliver high-quality images under high loads and fixed budgets.

Title: Efficient Diffusion Models: A Survey

• Authors: Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, Chaofan Tao, Yongfeng Huang, Ye Yuan, Mi Zhang
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06805
• Pdf URL: https://arxiv.org/pdf/2502.06805
• Copy Paste: [[2502.06805]] Efficient Diffusion Models: A Survey(https://arxiv.org/abs/2502.06805)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as powerful generative models capable of producing high-quality contents such as images, videos, and audio, demonstrating their potential to revolutionize digital content creation. However, these capabilities come at the cost of their significant computational resources and lengthy generation time, underscoring the critical need to develop efficient techniques for practical deployment. In this survey, we provide a systematic and comprehensive review of research on efficient diffusion models. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient diffusion model topics from algorithm-level, system-level, and framework perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at this https URL. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient diffusion model research and inspire them to contribute to this important and exciting field.

Title: Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models

• Authors: Shuting Wang, Haihong Tang, Zhicheng Dou, Chenyan Xiong
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06812
• Pdf URL: https://arxiv.org/pdf/2502.06812
• Copy Paste: [[2502.06812]] Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models(https://arxiv.org/abs/2502.06812)
• Keywords: diffusion
• Abstract: The emergence of diffusion models (DMs) has significantly improved the quality of text-to-video generation models (VGMs). However, current VGM optimization primarily emphasizes the global quality of videos, overlooking localized errors, which leads to suboptimal generation capabilities. To address this issue, we propose a post-training strategy for VGMs, HALO, which explicitly incorporates local feedback from a patch reward model, providing detailed and comprehensive training signals with the video reward model for advanced VGM optimization. To develop an effective patch reward model, we distill GPT-4o to continuously train our video reward model, which enhances training efficiency and ensures consistency between video and patch reward distributions. Furthermore, to harmoniously integrate patch rewards into VGM optimization, we introduce a granular DPO (Gran-DPO) algorithm for DMs, allowing collaborative use of both patch and video rewards during the optimization process. Experimental results indicate that our patch reward model aligns well with human annotations and HALO substantially outperforms the baselines across two evaluation methods. Further experiments quantitatively prove the existence of patch defects, and our proposed method could effectively alleviate this issue.

Title: Diffusion Instruction Tuning

• Authors: Chen Jin, Ryutaro Tanno, Amrutha Saseendran, Tom Diethe, Philip Teare
• Subjects: cs.LG, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06814
• Pdf URL: https://arxiv.org/pdf/2502.06814
• Copy Paste: [[2502.06814]] Diffusion Instruction Tuning(https://arxiv.org/abs/2502.06814)
• Keywords: diffusion
• Abstract: We introduce Lavender, a simple supervised fine-tuning (SFT) method that boosts the performance of advanced vision-language models (VLMs) by leveraging state-of-the-art image generation models such as Stable Diffusion. Specifically, Lavender aligns the text-vision attention in the VLM transformer with the equivalent used by Stable Diffusion during SFT, instead of adapting separate encoders. This alignment enriches the model's visual understanding and significantly boosts performance across in- and out-of-distribution tasks. Lavender requires just 0.13 million training examples, 2.5% of typical large-scale SFT datasets, and fine-tunes on standard hardware (8 GPUs) in a single day. It consistently improves state-of-the-art open-source multimodal LLMs (e.g., Llama-3.2-11B, MiniCPM-Llama3-v2.5), achieving up to 30% gains and a 68% boost on challenging out-of-distribution medical QA tasks. By efficiently transferring the visual expertise of image generators with minimal supervision, Lavender offers a scalable solution for more accurate vision-language systems. All code, training data, and models will be shared at this https URL.

Title: DeepCell: Multiview Representation Learning for Post-Mapping Netlists

• Authors: Zhengyuan Shi, Chengyu Ma, Ziyang Zheng, Lingfeng Zhou, Hongyang Pan, Wentao Jiang, Fan Yang, Xiaoyan Yang, Zhufei Chu, Qiang Xu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06816
• Pdf URL: https://arxiv.org/pdf/2502.06816
• Copy Paste: [[2502.06816]] DeepCell: Multiview Representation Learning for Post-Mapping Netlists(https://arxiv.org/abs/2502.06816)
• Keywords: self-supervised
• Abstract: Representation learning for post-mapping (PM) netlists is a critical challenge in Electronic Design Automation (EDA), driven by the diverse and complex nature of modern circuit designs. Existing approaches focus on intermediate representations like And-Inverter Graphs (AIGs), limiting their applicability to post-synthesis stages. We introduce DeepCell, a multiview representation learning framework that integrates structural and functional insights from both PM netlists and AIGs to learn rich, generalizable embeddings. At its core, DeepCell employs the novel Mask Circuit Modeling (MCM) mechanism, which refines PM netlist representations in a self-supervised manner using pretrained AIG encoders. DeepCell sets a new benchmark in PM netlist representation, outperforming existing methods in predictive accuracy and reconstruction fidelity. To validate its efficacy, we apply DeepCell to functional Engineering Change Orders (ECO), achieving significant reductions in patch generation costs and runtime while improving patch quality.

Title: Functional 3D Scene Synthesis through Human-Scene Optimization

• Authors: Yao Wei, Matteo Toso, Pietro Morerio, Michael Ying Yang, Alessio Del Bue
• Subjects: cs.LG, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06819
• Pdf URL: https://arxiv.org/pdf/2502.06819
• Copy Paste: [[2502.06819]] Functional 3D Scene Synthesis through Human-Scene Optimization(https://arxiv.org/abs/2502.06819)
• Keywords: diffusion, generative
• Abstract: This paper presents a novel generative approach that outputs 3D indoor environments solely from a textual description of the scene. Current methods often treat scene synthesis as a mere layout prediction task, leading to rooms with overlapping objects or overly structured scenes, with limited consideration of the practical usability of the generated environment. Instead, our approach is based on a simple, but effective principle: we condition scene synthesis to generate rooms that are usable by humans. This principle is implemented by synthesizing 3D humans that interact with the objects composing the scene. If this human-centric scene generation is viable, the room layout is functional and it leads to a more coherent 3D structure. To this end, we propose a novel method for functional 3D scene synthesis, which consists of reasoning, 3D assembling and optimization. We regard text guided 3D synthesis as a reasoning process by generating a scene graph via a graph diffusion network. Considering object functional co-occurrence, a new strategy is designed to better accommodate human-object interaction and avoidance, achieving human-aware 3D scene optimization. We conduct both qualitative and quantitative experiments to validate the effectiveness of our method in generating coherent 3D scene synthesis results.

Title: DiffListener: Discrete Diffusion Model for Listener Generation

• Authors: Siyeol Jung, Taehwan Kim
• Subjects: cs.LG, cs.CL, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06822
• Pdf URL: https://arxiv.org/pdf/2502.06822
• Copy Paste: [[2502.06822]] DiffListener: Discrete Diffusion Model for Listener Generation(https://arxiv.org/abs/2502.06822)
• Keywords: diffusion
• Abstract: The listener head generation (LHG) task aims to generate natural nonverbal listener responses based on the speaker's multimodal cues. While prior work either rely on limited modalities (e.g. audio and facial information) or employ autoregressive approaches which have limitations such as accumulating prediction errors. To address these limitations, we propose DiffListener, a discrete diffusion based approach for non-autoregressive listener head generation. Our model takes the speaker's facial information, audio, and text as inputs, additionally incorporating facial differential information to represent the temporal dynamics of expressions and movements. With this explicit modeling of facial dynamics, DiffListener can generate coherent reaction sequences in a non-autoregressive manner. Through comprehensive experiments, DiffListener demonstrates state-of-the-art performance in both quantitative and qualitative evaluations. The user study shows that DiffListener generates natural context-aware listener reactions that are well synchronized with the speaker. The code and demo videos are available in this https URL

Title: Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework

• Authors: Dongliang Zhou, Haijun Zhang, Kai Yang, Linlin Liu, Han Yan, Xiaofei Xu, Zhao Zhang, Shuicheng Yan
• Subjects: cs.LG, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06827
• Pdf URL: https://arxiv.org/pdf/2502.06827
• Copy Paste: [[2502.06827]] Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework(https://arxiv.org/abs/2502.06827)
• Keywords: generative
• Abstract: The field of fashion compatibility learning has attracted great attention from both the academic and industrial communities in recent years. Many studies have been carried out for fashion compatibility prediction, collocated outfit recommendation, artificial intelligence (AI)-enabled compatible fashion design, and related topics. In particular, AI-enabled compatible fashion design can be used to synthesize compatible fashion items or outfits in order to improve the design experience for designers or the efficacy of recommendations for customers. However, previous generative models for collocated fashion synthesis have generally focused on the image-to-image translation between fashion items of upper and lower clothing. In this paper, we propose a novel outfit generation framework, i.e., OutfitGAN, with the aim of synthesizing a set of complementary items to compose an entire outfit, given one extant fashion item and reference masks of target synthesized items. OutfitGAN includes a semantic alignment module, which is responsible for characterizing the mapping correspondence between the existing fashion items and the synthesized ones, to improve the quality of the synthesized images, and a collocation classification module, which is used to improve the compatibility of a synthesized outfit. In order to evaluate the performance of our proposed models, we built a large-scale dataset consisting of 20,000 fashion outfits. Extensive experimental results on this dataset show that our OutfitGAN can synthesize photo-realistic outfits and outperform state-of-the-art methods in terms of similarity, authenticity and compatibility measurements.

Title: TorchResist: Open-Source Differentiable Resist Simulator

• Authors: Zixiao Wang, Jieya Zhou, Su Zheng, Shuo Yin, Kaichao Liang, Shoubo Hu, Xiao Chen, Bei Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06838
• Pdf URL: https://arxiv.org/pdf/2502.06838
• Copy Paste: [[2502.06838]] TorchResist: Open-Source Differentiable Resist Simulator(https://arxiv.org/abs/2502.06838)
• Keywords: generative
• Abstract: Recent decades have witnessed remarkable advancements in artificial intelligence (AI), including large language models (LLMs), image and video generative models, and embodied AI systems. These advancements have led to an explosive increase in the demand for computational power, challenging the limits of Moore's Law. Optical lithography, a critical technology in semiconductor manufacturing, faces significant challenges due to its high costs. To address this, various lithography simulators have been developed. However, many of these simulators are limited by their inadequate photoresist modeling capabilities. This paper presents TorchResist, an open-source, differentiable photoresist this http URL employs an analytical approach to model the photoresist process, functioning as a white-box system with at most twenty interpretable parameters. Leveraging modern differentiable programming techniques and parallel computing on GPUs, TorchResist enables seamless co-optimization with other tools across multiple related tasks. Our experimental results demonstrate that TorchResist achieves superior accuracy and efficiency compared to existing solutions. The source code is publicly available.

Title: Self-Supervised Prompt Optimization

• Authors: Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Fengwei Teng, Jinhao Tu, Xinbing Liang, Sirui Hong, Chenglin Wu, Yuyu Luo
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06855
• Pdf URL: https://arxiv.org/pdf/2502.06855
• Copy Paste: [[2502.06855]] Self-Supervised Prompt Optimization(https://arxiv.org/abs/2502.06855)
• Keywords: self-supervised
• Abstract: Well-designed prompts are crucial for enhancing Large language models' (LLMs) reasoning capabilities while aligning their outputs with task requirements across diverse domains. However, manually designed prompts require expertise and iterative experimentation. While existing prompt optimization methods aim to automate this process, they rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. To address this, we propose Self-Supervised Prompt Optimization (SPO), a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without requiring external reference. Motivated by the observations that prompt quality manifests directly in LLM outputs and LLMs can effectively assess adherence to task requirements, we derive evaluation and optimization signals purely from output comparisons. Specifically, SPO selects superior prompts through pairwise output comparisons evaluated by an LLM evaluator, followed by an LLM optimizer that aligns outputs with task requirements. Extensive experiments demonstrate that SPO outperforms state-of-the-art prompt optimization methods, achieving comparable or superior results with significantly lower costs (e.g., 1.1% to 5.6% of existing methods) and fewer samples (e.g., three samples). The code is available at this https URL.

Title: LLM-Supported Natural Language to Bash Translation

• Authors: Finnian Westenfelder, Erik Hemberg, Miguel Tulla, Stephen Moskal, Una-May O'Reilly, Silviu Chiricescu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06858
• Pdf URL: https://arxiv.org/pdf/2502.06858
• Copy Paste: [[2502.06858]] LLM-Supported Natural Language to Bash Translation(https://arxiv.org/abs/2502.06858)
• Keywords: in-context
• Abstract: The Bourne-Again Shell (Bash) command-line interface for Linux systems has complex syntax and requires extensive specialized knowledge. Using the natural language to Bash command (NL2SH) translation capabilities of large language models (LLMs) for command composition circumvents these issues. However, the NL2SH performance of LLMs is difficult to assess due to inaccurate test data and unreliable heuristics for determining the functional equivalence of Bash commands. We present a manually verified test dataset of 600 instruction-command pairs and a training dataset of 40,939 pairs, increasing the size of previous datasets by 441% and 135%, respectively. Further, we present a novel functional equivalence heuristic that combines command execution with LLM evaluation of command outputs. Our heuristic can determine the functional equivalence of two Bash commands with 95% confidence, a 16% increase over previous heuristics. Evaluation of popular LLMs using our test dataset and heuristic demonstrates that parsing, in-context learning, in-weight learning, and constrained decoding can improve NL2SH accuracy by up to 32%. Our findings emphasize the importance of dataset quality, execution-based evaluation and translation method for advancing NL2SH translation. Our code is available at this https URL

Title: AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion

• Authors: Hsiao-Yuan Chin, I-Chao Shen, Yi-Ting Chiu, Bing-Yu Chen
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.06860
• Pdf URL: https://arxiv.org/pdf/2502.06860
• Copy Paste: [[2502.06860]] AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion(https://arxiv.org/abs/2502.06860)
• Keywords: diffusion
• Abstract: The ability to automatically complete a partial sketch that depicts a complex scene, e.g., "a woman chatting with a man in the park", is very useful. However, existing sketch generation methods create sketches from scratch; they do not complete a partial sketch in the style of the original. To address this challenge, we introduce AutoSketch, a styleaware vector sketch completion method that accommodates diverse sketch styles. Our key observation is that the style descriptions of a sketch in natural language preserve the style during automatic sketch completion. Thus, we use a pretrained vision-language model (VLM) to describe the styles of the partial sketches in natural language and replicate these styles using newly generated strokes. We initially optimize the strokes to match an input prompt augmented by style descriptions extracted from the VLM. Such descriptions allow the method to establish a diffusion prior in close alignment with that of the partial sketch. Next, we utilize the VLM to generate an executable style adjustment code that adjusts the strokes to conform to the desired style. We compare our method with existing methods across various sketch styles and prompts, performed extensive ablation studies and qualitative and quantitative evaluations, and demonstrate that AutoSketch can support various sketch scenarios.

Title: Poincaré Inequality for Local Log-Polyak-Lojasiewicz Measures : Non-asymptotic Analysis in Low-temperature Regime

• Authors: Yun Gong, Zebang Shen, Niao He
• Subjects: cs.LG, math.CA, math.FA, math.PR, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.06862
• Pdf URL: https://arxiv.org/pdf/2502.06862
• Copy Paste: [[2502.06862]] Poincaré Inequality for Local Log-Polyak-Lojasiewicz Measures : Non-asymptotic Analysis in Low-temperature Regime(https://arxiv.org/abs/2502.06862)
• Keywords: diffusion
• Abstract: Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such landscapes, we propose to study the class of \logPLmeasure\ measures $\mu_\epsilon \propto \exp(-V/\epsilon)$, where the potential $V$ satisfies a local Polyak-Łojasiewicz (PŁ) inequality, and its set of local minima is provably \emph{connected}. Notably, potentials in this class can exhibit local maxima and we characterize its optimal set S to be a compact $\mathcal{C}^2$ \emph{embedding submanifold} of $\mathbb{R}^d$ without boundary. The \emph{non-contractibility} of S distinguishes our function class from the classical convex setting topologically. Moreover, the embedding structure induces a naturally defined Laplacian-Beltrami operator on S, and we show that its first non-trivial eigenvalue provides an \emph{$\epsilon$-independent} lower bound for the \Poincare\ constant in the \Poincare\ inequality of $\mu_\epsilon$. As a direct consequence, Langevin dynamics with such non-convex potential $V$ and diffusion coefficient $\epsilon$ converges to its equilibrium $\mu_\epsilon$ at a rate of $\tilde{\mathcal{O}}(1/\epsilon)$, provided $\epsilon$ is sufficiently small. Here $\tilde{\mathcal{O}}$ hides logarithmic terms.

Title: BF-GAN: Development of an AI-driven Bubbly Flow Image Generation Model Using Generative Adversarial Networks

• Authors: Wen Zhou, Shuichiro Miwa, Yang Liu, Koji Okamoto
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06863
• Pdf URL: https://arxiv.org/pdf/2502.06863
• Copy Paste: [[2502.06863]] BF-GAN: Development of an AI-driven Bubbly Flow Image Generation Model Using Generative Adversarial Networks(https://arxiv.org/abs/2502.06863)
• Keywords: generative
• Abstract: A generative AI architecture called bubbly flow generative adversarial networks (BF-GAN) is developed, designed to generate realistic and high-quality bubbly flow images through physically conditioned inputs, jg and jf. Initially, 52 sets of bubbly flow experiments under varying conditions are conducted to collect 140,000 bubbly flow images with physical labels of jg and jf for training data. A multi-scale loss function is then developed, incorporating mismatch loss and pixel loss to enhance the generative performance of BF-GAN further. Regarding evaluative metrics of generative AI, the BF-GAN has surpassed conventional GAN. Physically, key parameters of bubbly flow generated by BF-GAN are extracted and compared with measurement values and empirical correlations, validating BF-GAN's generative performance. The comparative analysis demonstrate that the BF-GAN can generate realistic and high-quality bubbly flow images with any given jg and jf within the research scope. BF-GAN offers a generative AI solution for two-phase flow research, substantially lowering the time and cost required to obtain high-quality data. In addition, it can function as a benchmark dataset generator for bubbly flow detection and segmentation algorithms, enhancing overall productivity in this research domain. The BF-GAN model is available online (this https URL).

Title: FlavorDiffusion: Predicting Food Pairings and Chemical Interactions Using Diffusion Models

• Authors: Seo Jun Pyo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06871
• Pdf URL: https://arxiv.org/pdf/2502.06871
• Copy Paste: [[2502.06871]] FlavorDiffusion: Predicting Food Pairings and Chemical Interactions Using Diffusion Models(https://arxiv.org/abs/2502.06871)
• Keywords: diffusion
• Abstract: The study of food pairing has evolved beyond subjective expertise with the advent of machine learning. This paper presents FlavorDiffusion, a novel framework leveraging diffusion models to predict food-chemical interactions and ingredient pairings without relying on chromatography. By integrating graph-based embeddings, diffusion processes, and chemical property encoding, FlavorDiffusion addresses data imbalances and enhances clustering quality. Using a heterogeneous graph derived from datasets like Recipe1M and FlavorDB, our model demonstrates superior performance in reconstructing ingredient-ingredient relationships. The addition of a Chemical Structure Prediction (CSP) layer further refines the embedding space, achieving state-of-the-art NMI scores and enabling meaningful discovery of novel ingredient combinations. The proposed framework represents a significant step forward in computational gastronomy, offering scalable, interpretable, and chemically informed solutions for food science.

Title: WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication

• Authors: Tingting Yang, Ping Zhang, Mengfan Zheng, Yuxuan Shi, Liwen Jing, Jianbo Huang, Nan Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06877
• Pdf URL: https://arxiv.org/pdf/2502.06877
• Copy Paste: [[2502.06877]] WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication(https://arxiv.org/abs/2502.06877)
• Keywords: foundation model, generative
• Abstract: This paper introduces WirelessGPT, a pioneering foundation model specifically designed for multi-task learning in wireless communication and sensing. Specifically, WirelessGPT leverages large-scale wireless channel datasets for unsupervised pretraining and extracting universal channel representations, which captures complex spatiotemporal dependencies. In fact,this task-agnostic design adapts WirelessGPT seamlessly to a wide range of downstream tasks, using a unified representation with minimal fine-tuning. By unifying communication and sensing functionalities, WirelessGPT addresses the limitations of task-specific models, offering a scalable and efficient solution for integrated sensing and communication (ISAC). With an initial parameter size of around 80 million, WirelessGPT demonstrates significant improvements over conventional methods and smaller AI models, reducing reliance on large-scale labeled data. As the first foundation model capable of supporting diverse tasks across different domains, WirelessGPT establishes a new benchmark, paving the way for future advancements in multi-task wireless systems.

Title: Enabling Autoregressive Models to Fill In Masked Tokens

• Authors: Daniel Israel, Aditya Grover, Guy Van den Broeck
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.06901
• Pdf URL: https://arxiv.org/pdf/2502.06901
• Copy Paste: [[2502.06901]] Enabling Autoregressive Models to Fill In Masked Tokens(https://arxiv.org/abs/2502.06901)
• Keywords: diffusion
• Abstract: Historically, LLMs have been trained using either autoregressive (AR) or masked language modeling (MLM) objectives, with AR models gaining dominance in recent years. However, AR models are inherently incapable of masked infilling, which is the ability to predict masked tokens between past and future context. In contrast, MLM models suffer from intrinsic computational inefficiencies during both training and inference that hinder their scalability. This work introduces MARIA (Masked and Autoregressive Infilling Architecture), a novel approach that leverages the strengths of both paradigms to achieve state-of-the-art masked infilling performance. MARIA combines a pre-trained MLM and AR model by training a linear decoder that takes their concatenated hidden states as input. This minimal modification enables the AR model to perform infilling while retaining its inherent advantages in terms of faster inference with KV caching. Our results demonstrate that MARIA significantly outperforms existing methods, namely discrete diffusion models, on masked infilling tasks.

Title: Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training

• Authors: Deven Mahesh Mistry, Anooshka Bajaj, Yash Aggarwal, Sahaj Singh Maini, Zoran Tiganj
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.06902
• Pdf URL: https://arxiv.org/pdf/2502.06902
• Copy Paste: [[2502.06902]] Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training(https://arxiv.org/abs/2502.06902)
• Keywords: in-context
• Abstract: We investigate in-context temporal biases in attention heads and transformer outputs. Using cognitive science methodologies, we analyze attention scores and outputs of the GPT-2 models of varying sizes. Across attention heads, we observe effects characteristic of human episodic memory, including temporal contiguity, primacy and recency. Transformer outputs demonstrate a tendency toward in-context serial recall. Importantly, this effect is eliminated after the ablation of the induction heads, which are the driving force behind the contiguity effect. Our findings offer insights into how transformers organize information temporally during in-context learning, shedding light on their similarities and differences with human memory and learning.

Title: Can ChatGPT Diagnose Alzheimer's Disease?

• Authors: Quoc-Toan Nguyen, Linh Le, Xuan-The Tran, Thomas Do, Chin-Teng Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06907
• Pdf URL: https://arxiv.org/pdf/2502.06907
• Copy Paste: [[2502.06907]] Can ChatGPT Diagnose Alzheimer's Disease?(https://arxiv.org/abs/2502.06907)
• Keywords: generative
• Abstract: Can ChatGPT diagnose Alzheimer's Disease (AD)? AD is a devastating neurodegenerative condition that affects approximately 1 in 9 individuals aged 65 and older, profoundly impairing memory and cognitive function. This paper utilises 9300 electronic health records (EHRs) with data from Magnetic Resonance Imaging (MRI) and cognitive tests to address an intriguing question: As a general-purpose task solver, can ChatGPT accurately detect AD using EHRs? We present an in-depth evaluation of ChatGPT using a black-box approach with zero-shot and multi-shot methods. This study unlocks ChatGPT's capability to analyse MRI and cognitive test results, as well as its potential as a diagnostic tool for AD. By automating aspects of the diagnostic process, this research opens a transformative approach for the healthcare system, particularly in addressing disparities in resource-limited regions where AD specialists are scarce. Hence, it offers a foundation for a promising method for early detection, supporting individuals with timely interventions, which is paramount for Quality of Life (QoL).

Title: Foundation Models for Anomaly Detection: Vision and Challenges

• Authors: Jing Ren, Tao Tang, Hong Jia, Haytham Fayek, Xiaodong Li, Suyu Ma, Xiwei Xu, Feng Xia
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06911
• Pdf URL: https://arxiv.org/pdf/2502.06911
• Copy Paste: [[2502.06911]] Foundation Models for Anomaly Detection: Vision and Challenges(https://arxiv.org/abs/2502.06911)
• Keywords: foundation model, anomaly
• Abstract: As data continues to grow in volume and complexity across domains such as finance, manufacturing, and healthcare, effective anomaly detection is essential for identifying irregular patterns that may signal critical issues. Recently, foundation models (FMs) have emerged as a powerful tool for advancing anomaly detection. They have demonstrated unprecedented capabilities in enhancing anomaly identification, generating detailed data descriptions, and providing visual explanations. This survey presents the first comprehensive review of recent advancements in FM-based anomaly detection. We propose a novel taxonomy that classifies FMs into three categories based on their roles in anomaly detection tasks, i.e., as encoders, detectors, or interpreters. We provide a systematic analysis of state-of-the-art methods and discuss key challenges in leveraging FMs for improved anomaly detection. We also outline future research directions in this rapidly evolving field.

Title: Hyper Compressed Fine-Tuning of Large Foundation Models with Quantum Inspired Adapters

• Authors: Snehal Raj, Brian Coyle
• Subjects: cs.LG, cs.AI, eess.SP, quant-ph
• Abstract URL: https://arxiv.org/abs/2502.06916
• Pdf URL: https://arxiv.org/pdf/2502.06916
• Copy Paste: [[2502.06916]] Hyper Compressed Fine-Tuning of Large Foundation Models with Quantum Inspired Adapters(https://arxiv.org/abs/2502.06916)
• Keywords: foundation model
• Abstract: Fine-tuning pre-trained large foundation models for specific tasks has become increasingly challenging due to the computational and storage demands associated with full parameter updates. Parameter-Efficient Fine-Tuning (PEFT) methods address this issue by updating only a small subset of model parameters using adapter modules. In this work, we propose \emph{Quantum-Inspired Adapters}, a PEFT approach inspired by Hamming-weight preserving quantum circuits from quantum machine learning literature. These models can be both expressive and parameter-efficient by operating in a combinatorially large space while simultaneously preserving orthogonality in weight parameters. We test our proposed adapters by adapting large language models and large vision transformers on benchmark datasets. Our method can achieve 99.2\% of the performance of existing fine-tuning methods such LoRA with a 44x parameter compression on language understanding datasets like GLUE and VTAB. Compared to existing orthogonal fine-tuning methods such as OFT or BOFT, we achieve 98\% relative performance with 25x fewer parameters. This demonstrates competitive performance paired with a significant reduction in trainable parameters. Through ablation studies, we determine that combining multiple Hamming-weight orders with orthogonality and matrix compounding are essential for performant fine-tuning. Our findings suggest that Quantum-Inspired Adapters offer a promising direction for efficient adaptation of language and vision models in resource-constrained environments.

Title: Leveraging GPT-4o Efficiency for Detecting Rework Anomaly in Business Processes

• Authors: Mohammad Derakhshan, Paolo Ceravolo, Fatemeh Mohammadi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.06918
• Pdf URL: https://arxiv.org/pdf/2502.06918
• Copy Paste: [[2502.06918]] Leveraging GPT-4o Efficiency for Detecting Rework Anomaly in Business Processes(https://arxiv.org/abs/2502.06918)
• Keywords: anomaly
• Abstract: This paper investigates the effectiveness of GPT-4o-2024-08-06, one of the Large Language Models (LLM) from OpenAI, in detecting business process anomalies, with a focus on rework anomalies. In our study, we developed a GPT-4o-based tool capable of transforming event logs into a structured format and identifying reworked activities within business event logs. The analysis was performed on a synthetic dataset designed to contain rework anomalies but free of loops. To evaluate the anomaly detection capabilities of GPT 4o-2024-08-06, we used three prompting techniques: zero-shot, one-shot, and few-shot. These techniques were tested on different anomaly distributions, namely normal, uniform, and exponential, to identify the most effective approach for each case. The results demonstrate the strong performance of GPT-4o-2024-08-06. On our dataset, the model achieved 96.14% accuracy with one-shot prompting for the normal distribution, 97.94% accuracy with few-shot prompting for the uniform distribution, and 74.21% accuracy with few-shot prompting for the exponential distribution. These results highlight the model's potential as a reliable tool for detecting rework anomalies in event logs and how anomaly distribution and prompting strategy influence the model's performance.

Title: GAS: Generative Avatar Synthesis from a Single Image

• Authors: Yixing Lu, Junting Dong, Youngjoong Kwon, Qin Zhao, Bo Dai, Fernando De la Torre
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.06957
• Pdf URL: https://arxiv.org/pdf/2502.06957
• Copy Paste: [[2502.06957]] GAS: Generative Avatar Synthesis from a Single Image(https://arxiv.org/abs/2502.06957)
• Keywords: diffusion, generative
• Abstract: We introduce a generalizable and unified framework to synthesize view-consistent and temporally coherent avatars from a single image, addressing the challenging problem of single-image avatar generation. While recent methods employ diffusion models conditioned on human templates like depth or normal maps, they often struggle to preserve appearance information due to the discrepancy between sparse driving signals and the actual human subject, resulting in multi-view and temporal inconsistencies. Our approach bridges this gap by combining the reconstruction power of regression-based 3D human reconstruction with the generative capabilities of a diffusion model. The dense driving signal from the initial reconstructed human provides comprehensive conditioning, ensuring high-quality synthesis faithful to the reference appearance and structure. Additionally, we propose a unified framework that enables the generalization learned from novel pose synthesis on in-the-wild videos to naturally transfer to novel view synthesis. Our video-based diffusion model enhances disentangled synthesis with high-quality view-consistent renderings for novel views and realistic non-rigid deformations in novel pose animation. Results demonstrate the superior generalization ability of our method across in-domain and out-of-domain in-the-wild datasets. Project page: this https URL

Title: Model Diffusion for Certifiable Few-shot Transfer Learning

• Authors: Fady Rezk, Royson Lee, Henry Gouk, Timothy Hospedales, Minyoung Kim
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.06970
• Pdf URL: https://arxiv.org/pdf/2502.06970
• Copy Paste: [[2502.06970]] Model Diffusion for Certifiable Few-shot Transfer Learning(https://arxiv.org/abs/2502.06970)
• Keywords: diffusion, foundation model
• Abstract: In modern large-scale deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models (FMs) to new tasks via parameter-efficient fine-tuning (PEFT). However, while empirically effective, the resulting solutions lack generalisation guarantees to certify their accuracy - which may be required for ethical or legal reasons prior to deployment in high-importance applications. In this paper we develop a novel transfer learning approach that is designed to facilitate non-vacuous learning theoretic generalisation guarantees for downstream tasks, even in the low-shot regime. Specifically, we first use upstream tasks to train a distribution over PEFT parameters. We then learn the downstream task by a sample-and-evaluate procedure -- sampling plausible PEFTs from the trained diffusion model and selecting the one with the highest likelihood on the downstream data. Crucially, this confines our model hypothesis to a finite set of PEFT samples. In contrast to learning in the typical continuous hypothesis spaces of neural network weights, this facilitates tighter risk certificates. We instantiate our bound and show non-trivial generalization guarantees compared to existing learning approaches which lead to vacuous bounds in the low-shot regime.

Title: Investigating the Zone of Proximal Development of Language Models for In-Context Learning

• Authors: Peng Cui, Mrinmaya Sachan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.06990
• Pdf URL: https://arxiv.org/pdf/2502.06990
• Copy Paste: [[2502.06990]] Investigating the Zone of Proximal Development of Language Models for In-Context Learning(https://arxiv.org/abs/2502.06990)
• Keywords: in-context
• Abstract: In this paper, we introduce a learning analytics framework to analyze the in-context learning (ICL) behavior of large language models (LLMs) through the lens of the Zone of Proximal Development (ZPD), an established theory in educational psychology. ZPD delineates the space between what a learner is capable of doing unsupported and what the learner cannot do even with support. We adapt this concept to ICL, measuring the ZPD of LLMs based on model performance on individual examples with and without ICL. Furthermore, we propose an item response theory (IRT) model to predict the distribution of zones for LLMs. Our findings reveal a series of intricate and multifaceted behaviors of ICL, providing new insights into understanding and leveraging this technique. Finally, we demonstrate how our framework can enhance LLM in both inference and fine-tuning scenarios: (1) By predicting a model's zone of proximal development, we selectively apply ICL to queries that are most likely to benefit from demonstrations, achieving a better balance between inference cost and performance; (2) We propose a human-like curriculum for fine-tuning, which prioritizes examples within the model's ZPD. The curriculum results in improved performance, and we explain its effectiveness through an analysis of the training dynamics of LLMs.

Title: Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models

• Authors: Siddarth Venkatraman, Mohsin Hasan, Minsu Kim, Luca Scimeca, Marcin Sendera, Yoshua Bengio, Glen Berseth, Nikolay Malkin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.06999
• Pdf URL: https://arxiv.org/pdf/2502.06999
• Copy Paste: [[2502.06999]] Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models(https://arxiv.org/abs/2502.06999)
• Keywords: diffusion, generative
• Abstract: Any well-behaved generative model over a variable $\mathbf{x}$ can be expressed as a deterministic transformation of an exogenous ('outsourced') Gaussian noise variable $\mathbf{z}$: $\mathbf{x}=f_\theta(\mathbf{z})$. In such a model (e.g., a VAE, GAN, or continuous-time flow-based model), sampling of the target variable $\mathbf{x} \sim p_\theta(\mathbf{x})$ is straightforward, but sampling from a posterior distribution of the form $p(\mathbf{x}\mid\mathbf{y}) \propto p_\theta(\mathbf{x})r(\mathbf{x},\mathbf{y})$, where $r$ is a constraint function depending on an auxiliary variable $\mathbf{y}$, is generally intractable. We propose to amortize the cost of sampling from such posterior distributions with diffusion models that sample a distribution in the noise space ($\mathbf{z}$). These diffusion samplers are trained by reinforcement learning algorithms to enforce that the transformed samples $f_\theta(\mathbf{z})$ are distributed according to the posterior in the data space ($\mathbf{x}$). For many models and constraints of interest, the posterior in the noise space is smoother than the posterior in the data space, making it more amenable to such amortized inference. Our method enables conditional sampling under unconditional GAN, (H)VAE, and flow-based priors, comparing favorably both with current amortized and non-amortized inference methods. We demonstrate the proposed outsourced diffusion sampling in several experiments with large pretrained prior models: conditional image generation, reinforcement learning with human feedback, and protein structure generation.

Title: From Image to Video: An Empirical Study of Diffusion Representations

• Authors: Pedro Vélez, Luisa F. Polanía, Yi Yang, Chuhan Zhang, Rishab Kabra, Anurag Arnab, Mehdi S. M. Sajjadi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07001
• Pdf URL: https://arxiv.org/pdf/2502.07001
• Copy Paste: [[2502.07001]] From Image to Video: An Empirical Study of Diffusion Representations(https://arxiv.org/abs/2502.07001)
• Keywords: diffusion, generative
• Abstract: Diffusion models have revolutionized generative modeling, enabling unprecedented realism in image and video synthesis. This success has sparked interest in leveraging their representations for visual understanding tasks. While recent works have explored this potential for image generation, the visual understanding capabilities of video diffusion models remain largely uncharted. To address this gap, we systematically compare the same model architecture trained for video versus image generation, analyzing the performance of their latent representations on various downstream tasks including image classification, action recognition, depth estimation, and tracking. Results show that video diffusion models consistently outperform their image counterparts, though we find a striking range in the extent of this superiority. We further analyze features extracted from different layers and with varying noise levels, as well as the effect of model size and training budget on representation and generation quality. This work marks the first direct comparison of video and image diffusion objectives for visual understanding, offering insights into the role of temporal information in representation learning.

Title: Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC

• Authors: Siwei Meng, Yawei Luo, Ping Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07007
• Pdf URL: https://arxiv.org/pdf/2502.07007
• Copy Paste: [[2502.07007]] Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC(https://arxiv.org/abs/2502.07007)
• Keywords: generative
• Abstract: Recent advancements in AI-generated content have significantly improved the realism of 3D and 4D generation. However, most existing methods prioritize appearance consistency while neglecting underlying physical principles, leading to artifacts such as unrealistic deformations, unstable dynamics, and implausible objects interactions. Incorporating physics priors into generative models has become a crucial research direction to enhance structural integrity and motion realism. This survey provides a review of physics-aware generative methods, systematically analyzing how physical constraints are integrated into 3D and 4D generation. First, we examine recent works in incorporating physical priors into static and dynamic 3D generation, categorizing methods based on representation types, including vision-based, NeRF-based, and Gaussian Splatting-based approaches. Second, we explore emerging techniques in 4D generation, focusing on methods that model temporal dynamics with physical simulations. Finally, we conduct a comparative analysis of major methods, highlighting their strengths, limitations, and suitability for different materials and motion dynamics. By presenting an in-depth analysis of physics-grounded AIGC, this survey aims to bridge the gap between generative models and physical realism, providing insights that inspire future research in physically consistent content generation.

Title: Detecting Neurodegenerative Diseases using Frame-Level Handwriting Embeddings

• Authors: Sarah Laouedj, Yuzhe Wang, Jesus Villalba, Thomas Thebaud, Laureano Moro-Velazquez, Najim Dehak
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07025
• Pdf URL: https://arxiv.org/pdf/2502.07025
• Copy Paste: [[2502.07025]] Detecting Neurodegenerative Diseases using Frame-Level Handwriting Embeddings(https://arxiv.org/abs/2502.07025)
• Keywords: generative
• Abstract: In this study, we explored the use of spectrograms to represent handwriting signals for assessing neurodegenerative diseases, including 42 healthy controls (CTL), 35 subjects with Parkinson's Disease (PD), 21 with Alzheimer's Disease (AD), and 15 with Parkinson's Disease Mimics (PDM). We applied CNN and CNN-BLSTM models for binary classification using both multi-channel fixed-size and frame-based spectrograms. Our results showed that handwriting tasks and spectrogram channel combinations significantly impacted classification performance. The highest F1-score (89.8%) was achieved for AD vs. CTL, while PD vs. CTL reached 74.5%, and PD vs. PDM scored 77.97%. CNN consistently outperformed CNN-BLSTM. Different sliding window lengths were tested for constructing frame-based spectrograms. A 1-second window worked best for AD, longer windows improved PD classification, and window length had little effect on PD vs. PDM.

Title: Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment

• Authors: Kwanghee Choi, Eunjung Yeo, Kalvin Chang, Shinji Watanabe, David Mortensen
• Subjects: cs.CL, cs.AI, cs.LG, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.07029
• Pdf URL: https://arxiv.org/pdf/2502.07029
• Copy Paste: [[2502.07029]] Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment(https://arxiv.org/abs/2502.07029)
• Keywords: self-supervised
• Abstract: Allophony refers to the variation in the phonetic realization of a phoneme based on its phonetic environment. Modeling allophones is crucial for atypical pronunciation assessment, which involves distinguishing atypical from typical pronunciations. However, recent phoneme classifier-based approaches often simplify this by treating various realizations as a single phoneme, bypassing the complexity of modeling allophonic variation. Motivated by the acoustic modeling capabilities of frozen self-supervised speech model (S3M) features, we propose MixGoP, a novel approach that leverages Gaussian mixture models to model phoneme distributions with multiple subclusters. Our experiments show that MixGoP achieves state-of-the-art performance across four out of five datasets, including dysarthric and non-native speech. Our analysis further suggests that S3M features capture allophonic variation more effectively than MFCCs and Mel spectrograms, highlighting the benefits of integrating MixGoP with S3M features.

Title: Automated Consistency Analysis of LLMs

• Authors: Aditya Patwardhan, Vivek Vaidya, Ashish Kundu
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07036
• Pdf URL: https://arxiv.org/pdf/2502.07036
• Copy Paste: [[2502.07036]] Automated Consistency Analysis of LLMs(https://arxiv.org/abs/2502.07036)
• Keywords: generative
• Abstract: Generative AI (Gen AI) with large language models (LLMs) are being widely adopted across the industry, academia and government. Cybersecurity is one of the key sectors where LLMs can be and/or are already being used. There are a number of problems that inhibit the adoption of trustworthy Gen AI and LLMs in cybersecurity and such other critical areas. One of the key challenge to the trustworthiness and reliability of LLMs is: how consistent an LLM is in its responses? In this paper, we have analyzed and developed a formal definition of consistency of responses of LLMs. We have formally defined what is consistency of responses and then develop a framework for consistency evaluation. The paper proposes two approaches to validate consistency: self-validation, and validation across multiple LLMs. We have carried out extensive experiments for several LLMs such as GPT4oMini, GPT3.5, Gemini, Cohere, and Llama3, on a security benchmark consisting of several cybersecurity questions: informational and situational. Our experiments corroborate the fact that even though these LLMs are being considered and/or already being used for several cybersecurity tasks today, they are often inconsistent in their responses, and thus are untrustworthy and unreliable for cybersecurity.

Title: Contextual Thompson Sampling via Generation of Missing Data

• Authors: Kelly W. Zhang, Tiffany Tianhui Cai, Hongseok Namkoong, Daniel Russo
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07064
• Pdf URL: https://arxiv.org/pdf/2502.07064
• Copy Paste: [[2502.07064]] Contextual Thompson Sampling via Generation of Missing Data(https://arxiv.org/abs/2502.07064)
• Keywords: generative
• Abstract: We introduce a framework for Thompson sampling contextual bandit algorithms, in which the algorithm's ability to quantify uncertainty and make decisions depends on the quality of a generative model that is learned offline. Instead of viewing uncertainty in the environment as arising from unobservable latent parameters, our algorithm treats uncertainty as stemming from missing, but potentially observable, future outcomes. If these future outcomes were all observed, one could simply make decisions using an "oracle" policy fit on the complete dataset. Inspired by this conceptualization, at each decision-time, our algorithm uses a generative model to probabilistically impute missing future outcomes, fits a policy using the imputed complete dataset, and uses that policy to select the next action. We formally show that this algorithm is a generative formulation of Thompson Sampling and prove a state-of-the-art regret bound for it. Notably, our regret bound i) depends on the probabilistic generative model only through the quality of its offline prediction loss, and ii) applies to any method of fitting the "oracle" policy, which easily allows one to adapt Thompson sampling to decision-making settings with fairness and/or resource constraints.

Title: Likelihood-Free Estimation for Spatiotemporal Hawkes processes with missing data and application to predictive policing

• Authors: Pramit Das, Moulinath Banerjee, Yuekai Sun
• Subjects: cs.LG, stat.AP, stat.ME
• Abstract URL: https://arxiv.org/abs/2502.07111
• Pdf URL: https://arxiv.org/pdf/2502.07111
• Copy Paste: [[2502.07111]] Likelihood-Free Estimation for Spatiotemporal Hawkes processes with missing data and application to predictive policing(https://arxiv.org/abs/2502.07111)
• Keywords: generative
• Abstract: With the growing use of AI technology, many police departments use forecasting software to predict probable crime hotspots and allocate patrolling resources effectively for crime prevention. The clustered nature of crime data makes self-exciting Hawkes processes a popular modeling choice. However, one significant challenge in fitting such models is the inherent missingness in crime data due to non-reporting, which can bias the estimated parameters of the predictive model, leading to inaccurate downstream hotspot forecasts, often resulting in over or under-policing in various communities, especially the vulnerable ones. Our work introduces a Wasserstein Generative Adversarial Networks (WGAN) driven likelihood-free approach to account for unreported crimes in Spatiotemporal Hawkes models. We demonstrate through empirical analysis how this methodology improves the accuracy of parametric estimation in the presence of data missingness, leading to more reliable and efficient policing strategies.

Title: SAFE: Self-Supervised Anomaly Detection Framework for Intrusion Detection

• Authors: Elvin Li, Zhengli Shang, Onat Gungor, Tajana Rosing
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07119
• Pdf URL: https://arxiv.org/pdf/2502.07119
• Copy Paste: [[2502.07119]] SAFE: Self-Supervised Anomaly Detection Framework for Intrusion Detection(https://arxiv.org/abs/2502.07119)
• Keywords: self-supervised, anomaly
• Abstract: The proliferation of IoT devices has significantly increased network vulnerabilities, creating an urgent need for effective Intrusion Detection Systems (IDS). Machine Learning-based IDS (ML-IDS) offer advanced detection capabilities but rely on labeled attack data, which limits their ability to identify unknown threats. Self-Supervised Learning (SSL) presents a promising solution by using only normal data to detect patterns and anomalies. This paper introduces SAFE, a novel framework that transforms tabular network intrusion data into an image-like format, enabling Masked Autoencoders (MAEs) to learn robust representations of network behavior. The features extracted by the MAEs are then incorporated into a lightweight novelty detector, enhancing the effectiveness of anomaly detection. Experimental results demonstrate that SAFE outperforms the state-of-the-art anomaly detection method, Scale Learning-based Deep Anomaly Detection method (SLAD), by up to 26.2% and surpasses the state-of-the-art SSL-based network intrusion detection approach, Anomal-E, by up to 23.5% in F1-score.

Title: HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitrates

• Authors: Lei Lu, Yize Li, Yanzhi Wang, Wei Wang, Wei Jiang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.07160
• Pdf URL: https://arxiv.org/pdf/2502.07160
• Copy Paste: [[2502.07160]] HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitrates(https://arxiv.org/abs/2502.07160)
• Keywords: diffusion, generative
• Abstract: Image compression under ultra-low bitrates remains challenging for both conventional learned image compression (LIC) and generative vector-quantized (VQ) modeling. Conventional LIC suffers from severe artifacts due to heavy quantization, while generative VQ modeling gives poor fidelity due to the mismatch between learned generative priors and specific inputs. In this work, we propose Hybrid-Diffusion Image Compression (HDCompression), a dual-stream framework that utilizes both generative VQ-modeling and diffusion models, as well as conventional LIC, to achieve both high fidelity and high perceptual quality. Different from previous hybrid methods that directly use pre-trained LIC models to generate low-quality fidelity-preserving information from heavily quantized latent, we use diffusion models to extract high-quality complimentary fidelity information from the ground-truth input, which can enhance the system performance in several aspects: improving indices map prediction, enhancing the fidelity-preserving output of the LIC stream, and refining conditioned image reconstruction with VQ-latent correction. In addition, our diffusion model is based on a dense representative vector (DRV), which is lightweight with very simple sampling schedulers. Extensive experiments demonstrate that our HDCompression outperforms the previous conventional LIC, generative VQ-modeling, and hybrid frameworks in both quantitative metrics and qualitative visualization, providing balanced robust compression performance at ultra-low bitrates.

Title: Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion

• Authors: Xingpei Ma, Jiaran Cai, Yuansheng Guan, Shenneng Huang, Qiang Zhang, Shunsi Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07203
• Pdf URL: https://arxiv.org/pdf/2502.07203
• Copy Paste: [[2502.07203]] Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion(https://arxiv.org/abs/2502.07203)
• Keywords: diffusion
• Abstract: Recent diffusion-based talking face generation models have demonstrated impressive potential in synthesizing videos that accurately match a speech audio clip with a given reference identity. However, existing approaches still encounter significant challenges due to uncontrollable factors, such as inaccurate lip-sync, inappropriate head posture and the lack of fine-grained control over facial expressions. In order to introduce more face-guided conditions beyond speech audio clips, a novel two-stage training framework Playmate is proposed to generate more lifelike facial expressions and talking faces. In the first stage, we introduce a decoupled implicit 3D representation along with a meticulously designed motion-decoupled module to facilitate more accurate attribute disentanglement and generate expressive talking videos directly from audio cues. Then, in the second stage, we introduce an emotion-control module to encode emotion control information into the latent space, enabling fine-grained control over emotions and thereby achieving the ability to generate talking videos with desired emotion. Extensive experiments demonstrate that Playmate outperforms existing state-of-the-art methods in terms of video quality and lip-synchronization, and improves flexibility in controlling emotion and head pose. The code will be available at this https URL.

Title: Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models

• Authors: Xinren Zhang, Jiadong Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07211
• Pdf URL: https://arxiv.org/pdf/2502.07211
• Copy Paste: [[2502.07211]] Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models(https://arxiv.org/abs/2502.07211)
• Keywords: diffusion, generative
• Abstract: Dynamic resource allocation in mobile wireless networks involves complex, time-varying optimization problems, motivating the adoption of deep reinforcement learning (DRL). However, most existing works rely on pre-trained policies, overlooking dynamic environmental changes that rapidly invalidate the policies. Periodic retraining becomes inevitable but incurs prohibitive computational costs and energy consumption-critical concerns for resource-constrained wireless systems. We identify three root causes of inefficient retraining: high-dimensional state spaces, suboptimal action spaces exploration-exploitation trade-offs, and reward design limitations. To overcome these limitations, we propose Diffusion-based Deep Reinforcement Learning (D2RL), which leverages generative diffusion models (GDMs) to holistically enhance all three DRL components. Iterative refinement process and distribution modelling of GDMs enable (1) the generation of diverse state samples to improve environmental understanding, (2) balanced action space exploration to escape local optima, and (3) the design of discriminative reward functions that better evaluate action quality. Our framework operates in two modes: Mode I leverages GDMs to explore reward spaces and design discriminative reward functions that rigorously evaluate action quality, while Mode II synthesizes diverse state samples to enhance environmental understanding and generalization. Extensive experiments demonstrate that D2RL achieves faster convergence and reduced computational costs over conventional DRL methods for resource allocation in wireless communications while maintaining competitive policy performance. This work underscores the transformative potential of GDMs in overcoming fundamental DRL training bottlenecks for wireless networks, paving the way for practical, real-time deployments.

Title: MLLM4PUE: Toward Universal Embeddings in Computational Pathology through Multimodal LLMs

• Authors: Qifeng Zhou, Thao M. Dang, Wenliang Zhong, Yuzhi Guo, Hehuan Ma, Saiyang Na, Junzhou Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07221
• Pdf URL: https://arxiv.org/pdf/2502.07221
• Copy Paste: [[2502.07221]] MLLM4PUE: Toward Universal Embeddings in Computational Pathology through Multimodal LLMs(https://arxiv.org/abs/2502.07221)
• Keywords: foundation model
• Abstract: Pathology plays a critical role in diagnosing a wide range of diseases, yet existing approaches often rely heavily on task-specific models trained on extensive, well-labeled datasets. These methods face sustainability challenges due to the diversity of pathologies and the labor-intensive nature of data collection. To address these limitations, we highlight the need for universal multimodal embeddings that can support multiple downstream tasks. Previous approaches often involve fine-tuning CLIP-based models, which handle images and text separately, limiting their ability to capture complex multimodal relationships. Additionally, these models are evaluated across diverse datasets without a unified benchmark for assessing multimodal embeddings in pathology. To address these challenges, we propose MLLM4PUE, a novel framework that leverages Multimodal Large Language Models (MLLMs) to generate Pathology Universal Embeddings. The MLLM4PUE framework not only facilitates robust integration of images and text but also enhances understanding and fusion capabilities across various tasks. We further introduce the Pathology Multimodal Embedding Benchmark (PMEB), a comprehensive benchmark designed to assess the quality of pathology multimodal embeddings. PMEB comprises 15 original tasks drawn from 14 datasets, organized into three meta-tasks: retrieval, classification, and composed retrieval. Experimental results demonstrate the superiority of MLLM4PUE, illustrating MLLM-based models can effectively support a wide range of downstream tasks and unify the research direction for foundation models in pathology.

Title: CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models

• Authors: Sen Peng, Mingyue Wang, Jianfei He, Jijia Yang, Xiaohua Jia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07225
• Pdf URL: https://arxiv.org/pdf/2502.07225
• Copy Paste: [[2502.07225]] CAT: Contrastive Adversarial Training for Evaluating the Robustness of Protective Perturbations in Latent Diffusion Models(https://arxiv.org/abs/2502.07225)
• Keywords: diffusion
• Abstract: Latent diffusion models have recently demonstrated superior capabilities in many downstream image synthesis tasks. However, customization of latent diffusion models using unauthorized data can severely compromise the privacy and intellectual property rights of data owners. Adversarial examples as protective perturbations have been developed to defend against unauthorized data usage by introducing imperceptible noise to customization samples, preventing diffusion models from effectively learning them. In this paper, we first reveal that the primary reason adversarial examples are effective as protective perturbations in latent diffusion models is the distortion of their latent representations, as demonstrated through qualitative and quantitative experiments. We then propose the Contrastive Adversarial Training (CAT) utilizing adapters as an adaptive attack against these protection methods, highlighting their lack of robustness. Extensive experiments demonstrate that our CAT method significantly reduces the effectiveness of protective perturbations in customization configurations, urging the community to reconsider and enhance the robustness of existing protective perturbation methods. Code is available at \hyperlink{here}{this https URL}.

Title: DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization

• Authors: Xuefeng Liu, Songhao Jiang, Siyu Chen, Zhuoran Yang, Yuxin Chen, Ian Foster, Rick Stevens
• Subjects: cs.LG, cs.CL, q-bio.BM, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07237
• Pdf URL: https://arxiv.org/pdf/2502.07237
• Copy Paste: [[2502.07237]] DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization(https://arxiv.org/abs/2502.07237)
• Keywords: generative
• Abstract: Finetuning a Large Language Model (LLM) is crucial for generating results towards specific objectives. This research delves into the realm of drug optimization and introduce a novel reinforcement learning algorithm to finetune a drug optimization LLM-based generative model, enhancing the original drug across target objectives, while retains the beneficial chemical properties of the original drug. This work is comprised of two primary components: (1) DrugImprover: A framework tailored for improving robustness and efficiency in drug optimization. It includes a LLM designed for drug optimization and a novel Structured Policy Optimization (SPO) algorithm, which is theoretically grounded. This algorithm offers a unique perspective for fine-tuning the LLM-based generative model by aligning the improvement of the generated molecule with the input molecule under desired objectives. (2) A dataset of 1 million compounds, each with OEDOCK docking scores on 5 human proteins associated with cancer cells and 24 binding sites from SARS-CoV-2 virus. We conduct a comprehensive evaluation of SPO and demonstrate its effectiveness in improving the original drug across target properties. Our code and dataset will be publicly available at: this https URL.

Title: Diffusion Suction Grasping with Large-Scale Parcel Dataset

• Authors: Ding-Tao Huang, Xinyi He, Debei Hua, Dongfang Yu, En-Te Lin, Long Zeng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07238
• Pdf URL: https://arxiv.org/pdf/2502.07238
• Copy Paste: [[2502.07238]] Diffusion Suction Grasping with Large-Scale Parcel Dataset(https://arxiv.org/abs/2502.07238)
• Keywords: diffusion
• Abstract: While recent advances in object suction grasping have shown remarkable progress, significant challenges persist particularly in cluttered and complex parcel handling scenarios. Two fundamental limitations hinder current approaches: (1) the lack of a comprehensive suction grasp dataset tailored for parcel manipulation tasks, and (2) insufficient adaptability to diverse object characteristics including size variations, geometric complexity, and textural diversity. To address these challenges, we present Parcel-Suction-Dataset, a large-scale synthetic dataset containing 25 thousand cluttered scenes with 410 million precision-annotated suction grasp poses. This dataset is generated through our novel geometric sampling algorithm that enables efficient generation of optimal suction grasps incorporating both physical constraints and material properties. We further propose Diffusion-Suction, an innovative framework that reformulates suction grasp prediction as a conditional generation task through denoising diffusion probabilistic models. Our method iteratively refines random noise into suction grasp score maps through visual-conditioned guidance from point cloud observations, effectively learning spatial point-wise affordances from our synthetic dataset. Extensive experiments demonstrate that the simple yet efficient Diffusion-Suction achieves new state-of-the-art performance compared to previous models on both Parcel-Suction-Dataset and the public SuctionNet-1Billion benchmark.

Title: Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting

• Authors: Jiecheng Lu, Shihao Yang
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07244
• Pdf URL: https://arxiv.org/pdf/2502.07244
• Copy Paste: [[2502.07244]] Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting(https://arxiv.org/abs/2502.07244)
• Keywords: generative
• Abstract: Autoregressive attention-based time series forecasting (TSF) has drawn increasing interest, with mechanisms like linear attention sometimes outperforming vanilla attention. However, deeper Transformer architectures frequently misalign with autoregressive objectives, obscuring the underlying VAR structure embedded within linear attention and hindering their ability to capture the data generative processes in TSF. In this work, we first show that a single linear attention layer can be interpreted as a dynamic vector autoregressive (VAR) structure. We then explain that existing multi-layer Transformers have structural mismatches with the autoregressive forecasting objective, which impair interpretability and generalization ability. To address this, we show that by rearranging the MLP, attention, and input-output flow, multi-layer linear attention can also be aligned as a VAR model. Then, we propose Structural Aligned Mixture of VAR (SAMoVAR), a linear Transformer variant that integrates interpretable dynamic VAR weights for multivariate TSF. By aligning the Transformer architecture with autoregressive objectives, SAMoVAR delivers improved performance, interpretability, and computational efficiency, comparing to SOTA TSF models.

Title: GENERator: A Long-Context Generative Genomic Foundation Model

• Authors: Wei Wu, Qiuyi Li, Mingyang Li, Kun Fu, Fuli Feng, Jieping Ye, Hui Xiong, Zheng Wang
• Subjects: cs.CL, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2502.07272
• Pdf URL: https://arxiv.org/pdf/2502.07272
• Copy Paste: [[2502.07272]] GENERator: A Long-Context Generative Genomic Foundation Model(https://arxiv.org/abs/2502.07272)
• Keywords: foundation model, generative
• Abstract: Advancements in DNA sequencing technologies have significantly improved our ability to decode genomic sequences. However, the prediction and interpretation of these sequences remain challenging due to the intricate nature of genetic material. Large language models (LLMs) have introduced new opportunities for biological sequence analysis. Recent developments in genomic language models have underscored the potential of LLMs in deciphering DNA sequences. Nonetheless, existing models often face limitations in robustness and application scope, primarily due to constraints in model structure and training data scale. To address these limitations, we present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters. Trained on an expansive dataset comprising 386B bp of eukaryotic DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks. The model adheres to the central dogma of molecular biology, accurately generating protein-coding sequences that translate into proteins structurally analogous to known families. It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of promoter sequences with specific activity profiles. These capabilities position the GENERator as a pivotal tool for genomic research and biotechnological advancement, enhancing our ability to interpret and predict complex biological systems and enabling precise genomic interventions.

Title: Cost-Efficient Continual Learning with Sufficient Exemplar Memory

• Authors: Dongkyu Cho, Taesup Moon, Rumi Chunara, Kyunghyun Cho, Sungmin Cha
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07274
• Pdf URL: https://arxiv.org/pdf/2502.07274
• Copy Paste: [[2502.07274]] Cost-Efficient Continual Learning with Sufficient Exemplar Memory(https://arxiv.org/abs/2502.07274)
• Keywords: foundation model
• Abstract: Continual learning (CL) research typically assumes highly constrained exemplar memory resources. However, in many real-world scenarios-especially in the era of large foundation models-memory is abundant, while GPU computational costs are the primary bottleneck. In this work, we investigate CL in a novel setting where exemplar memory is ample (i.e., sufficient exemplar memory). Unlike prior methods designed for strict exemplar memory constraints, we propose a simple yet effective approach that directly operates in the model's weight space through a combination of weight resetting and averaging techniques. Our method achieves state-of-the-art performance while reducing the computational cost to a quarter or third of existing methods. These findings challenge conventional CL assumptions and provide a practical baseline for computationally efficient CL applications.

Title: Dataset Ownership Verification in Contrastive Pre-trained Models

• Authors: Yuechen Xie, Jie Song, Mengqi Xue, Haofei Zhang, Xingen Wang, Bingde Hu, Genlang Chen, Mingli Song
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07276
• Pdf URL: https://arxiv.org/pdf/2502.07276
• Copy Paste: [[2502.07276]] Dataset Ownership Verification in Contrastive Pre-trained Models(https://arxiv.org/abs/2502.07276)
• Keywords: self-supervised
• Abstract: High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concurrently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges as a crucial method in this domain, but existing approaches are often limited to supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a $p$-value markedly below $0.05$, surpassing all previous methodologies. Our code is available at this https URL.

Title: Articulate That Object Part (ATOP): 3D Part Articulation from Text and Motion Personalization

• Authors: Aditya Vora, Sauradip Nag, Hao Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07278
• Pdf URL: https://arxiv.org/pdf/2502.07278
• Copy Paste: [[2502.07278]] Articulate That Object Part (ATOP): 3D Part Articulation from Text and Motion Personalization(https://arxiv.org/abs/2502.07278)
• Keywords: diffusion
• Abstract: We present ATOP (Articulate That Object Part), a novel method based on motion personalization to articulate a 3D object with respect to a part and its motion as prescribed in a text prompt. Specifically, the text input allows us to tap into the power of modern-day video diffusion to generate plausible motion samples for the right object category and part. In turn, the input 3D object provides image prompting to personalize the generated video to that very object we wish to articulate. Our method starts with a few-shot finetuning for category-specific motion generation, a key first step to compensate for the lack of articulation awareness by current video diffusion models. For this, we finetune a pre-trained multi-view image generation model for controllable multi-view video generation, using a small collection of video samples obtained for the target object category. This is followed by motion video personalization that is realized by multi-view rendered images of the target 3D object. At last, we transfer the personalized video motion to the target 3D object via differentiable rendering to optimize part motion parameters by a score distillation sampling loss. We show that our method is capable of generating realistic motion videos and predict 3D motion parameters in a more accurate and generalizable way, compared to prior works.

Title: Exploratory Diffusion Policy for Unsupervised Reinforcement Learning

• Authors: Chengyang Ying, Huayu Chen, Xinning Zhou, Zhongkai Hao, Hang Su, Jun Zhu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.07279
• Pdf URL: https://arxiv.org/pdf/2502.07279
• Copy Paste: [[2502.07279]] Exploratory Diffusion Policy for Unsupervised Reinforcement Learning(https://arxiv.org/abs/2502.07279)
• Keywords: diffusion
• Abstract: Unsupervised reinforcement learning (RL) aims to pre-train agents by exploring states or skills in reward-free environments, facilitating the adaptation to downstream tasks. However, existing methods often overlook the fitting ability of pre-trained policies and struggle to handle the heterogeneous pre-training data, which are crucial for achieving efficient exploration and fast fine-tuning. To address this gap, we propose Exploratory Diffusion Policy (EDP), which leverages the strong expressive ability of diffusion models to fit the explored data, both boosting exploration and obtaining an efficient initialization for downstream tasks. Specifically, we estimate the distribution of collected data in the replay buffer with the diffusion policy and propose a score intrinsic reward, encouraging the agent to explore unseen states. For fine-tuning the pre-trained diffusion policy on downstream tasks, we provide both theoretical analyses and practical algorithms, including an alternating method of Q function optimization and diffusion policy distillation. Extensive experiments demonstrate the effectiveness of EDP in efficient exploration during pre-training and fast adaptation during fine-tuning.

Title: Generation of Drug-Induced Cardiac Reactions towards Virtual Clinical Trials

• Authors: Qian Shao, Bang Du, Zepeng Li, Qiyuan Chen, Hongxia Xu, Jimeng Sun, Jian Wu, Jintai Chen
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.07297
• Pdf URL: https://arxiv.org/pdf/2502.07297
• Copy Paste: [[2502.07297]] Generation of Drug-Induced Cardiac Reactions towards Virtual Clinical Trials(https://arxiv.org/abs/2502.07297)
• Keywords: diffusion, generative
• Abstract: Clinical trials are pivotal in cardiac drug development, yet they often fail due to inadequate efficacy and unexpected safety issues, leading to significant financial losses. Using in-silico trials to replace a part of physical clinical trials, e.g., leveraging advanced generative models to generate drug-influenced electrocardiograms (ECGs), seems an effective method to reduce financial risk and potential harm to trial participants. While existing generative models have demonstrated progress in ECG generation, they fall short in modeling drug reactions due to limited fidelity and inability to capture individualized drug response patterns. In this paper, we propose a Drug-Aware Diffusion Model (DADM), which could simulate individualized drug reactions while ensuring fidelity. To ensure fidelity, we construct a set of ordinary differential equations to provide external physical knowledge (EPK) of the realistic ECG morphology. The EPK is used to adaptively constrain the morphology of the generated ECGs through a dynamic cross-attention (DCA) mechanism. Furthermore, we propose an extension of ControlNet to incorporate demographic and drug data, simulating individual drug reactions. We compare DADM with the other eight state-of-the-art ECG generative models on two real-world databases covering 8 types of drug regimens. The results demonstrate that DADM can more accurately simulate drug-induced changes in ECGs, improving the accuracy by at least 5.79% and recall by 8%.

Title: Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving

• Authors: Xiang Li, Pengfei Li, Yupeng Zheng, Wei Sun, Yan Wang, Yilun Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07309
• Pdf URL: https://arxiv.org/pdf/2502.07309
• Copy Paste: [[2502.07309]] Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving(https://arxiv.org/abs/2502.07309)
• Keywords: self-supervised
• Abstract: Understanding world dynamics is crucial for planning in autonomous driving. Recent methods attempt to achieve this by learning a 3D occupancy world model that forecasts future surrounding scenes based on current observation. However, 3D occupancy labels are still required to produce promising results. Considering the high annotation cost for 3D outdoor scenes, we propose a semi-supervised vision-centric 3D occupancy world model, PreWorld, to leverage the potential of 2D labels through a novel two-stage training paradigm: the self-supervised pre-training stage and the fully-supervised fine-tuning stage. Specifically, during the pre-training stage, we utilize an attribute projection head to generate different attribute fields of a scene (e.g., RGB, density, semantic), thus enabling temporal supervision from 2D labels via volume rendering techniques. Furthermore, we introduce a simple yet effective state-conditioned forecasting module to recursively forecast future occupancy and ego trajectory in a direct manner. Extensive experiments on the nuScenes dataset validate the effectiveness and scalability of our method, and demonstrate that PreWorld achieves competitive performance across 3D occupancy prediction, 4D occupancy forecasting and motion planning tasks.

Title: Semantic to Structure: Learning Structural Representations for Infringement Detection

• Authors: Chuanwei Huang, Zexi Jia, Hongyan Fei, Yeshuang Zhu, Zhiqiang Yuan, Jinchao Zhang, Jie Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07323
• Pdf URL: https://arxiv.org/pdf/2502.07323
• Copy Paste: [[2502.07323]] Semantic to Structure: Learning Structural Representations for Infringement Detection(https://arxiv.org/abs/2502.07323)
• Keywords: diffusion
• Abstract: Structural information in images is crucial for aesthetic assessment, and it is widely recognized in the artistic field that imitating the structure of other works significantly infringes on creators' rights. The advancement of diffusion models has led to AI-generated content imitating artists' structural creations, yet effective detection methods are still lacking. In this paper, we define this phenomenon as "structural infringement" and propose a corresponding detection method. Additionally, we develop quantitative metrics and create manually annotated datasets for evaluation: the SIA dataset of synthesized data, and the SIR dataset of real data. Due to the current lack of datasets for structural infringement detection, we propose a new data synthesis strategy based on diffusion models and LLM, successfully training a structural infringement detection model. Experimental results show that our method can successfully detect structural infringements and achieve notable improvements on annotated test sets.

Title: Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution

• Authors: Hongyu An, Xinfeng Zhang, Shijie Zhao, Li Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07381
• Pdf URL: https://arxiv.org/pdf/2502.07381
• Copy Paste: [[2502.07381]] Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution(https://arxiv.org/abs/2502.07381)
• Keywords: diffusion
• Abstract: Due to limitations of storage and bandwidth, videos stored and transmitted on the Internet are usually low-quality with low-resolution and compression noise. Although video super-resolution (VSR) is an efficient technique to enhance video resolution, relatively VSR methods focus on compressed videos. Directly applying general VSR approaches leads to the failure of improving practical videos, especially when frames are highly compressed at a low bit rate. Recently, diffusion models have achieved superior performance in low-level visual tasks, and their high-realism generation capability enables them to be applied in VSR. To synthesize more compression-lost details and refine temporal consistency, we propose a novel Spatial Degradation-Aware and Temporal Consistent (SDATC) diffusion model for compressed VSR. Specifically, we introduce a distortion Control module (DCM) to modulate diffusion model inputs and guide the generation. Next, the diffusion model executes the denoising process for texture generation with fine-tuned spatial prompt-based compression-aware module (PCAM) and spatio-temporal attention module (STAM). PCAM extracts features to encode specific compression information dynamically. STAM extends the spatial attention mechanism to a spatio-temporal dimension for capturing temporal correlation. Extensive experimental results on benchmark datasets demonstrate the effectiveness of the proposed modules in enhancing compressed videos.

Title: FADE: Forecasting for Anomaly Detection on ECG

• Authors: Paula Ruiz-Barroso, Francisco M. Castro, José Miranda, Denisa-Andreea Constantinescu, David Atienza, Nicolás Guil
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07389
• Pdf URL: https://arxiv.org/pdf/2502.07389
• Copy Paste: [[2502.07389]] FADE: Forecasting for Anomaly Detection on ECG(https://arxiv.org/abs/2502.07389)
• Keywords: self-supervised, anomaly
• Abstract: Cardiovascular diseases, a leading cause of noncommunicable disease-related deaths, require early and accurate detection to improve patient outcomes. Taking advantage of advances in machine learning and deep learning, multiple approaches have been proposed in the literature to address the challenge of detecting ECG anomalies. Typically, these methods are based on the manual interpretation of ECG signals, which is time consuming and depends on the expertise of healthcare professionals. The objective of this work is to propose a deep learning system, FADE, designed for normal ECG forecasting and anomaly detection, which reduces the need for extensive labeled datasets and manual interpretation. FADE has been trained in a self-supervised manner with a novel morphological inspired loss function. Unlike conventional models that learn from labeled anomalous ECG waveforms, our approach predicts the future of normal ECG signals, thus avoiding the need for extensive labeled datasets. Using a novel distance function to compare forecasted ECG signals with actual sensor data, our method effectively identifies cardiac anomalies. Additionally, this approach can be adapted to new contexts through domain adaptation techniques. To evaluate our proposal, we performed a set of experiments using two publicly available datasets: MIT-BIH NSR and MIT-BIH Arrythmia. The results demonstrate that our system achieves an average accuracy of 83.84% in anomaly detection, while correctly classifying normal ECG signals with an accuracy of 85.46%. Our proposed approach exhibited superior performance in the early detection of cardiac anomalies in ECG signals, surpassing previous methods that predominantly identify a limited range of anomalies. FADE effectively detects both abnormal heartbeats and arrhythmias, offering significant advantages in healthcare through cost reduction or processing of large-scale ECG data.

Title: MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification

• Authors: Anh-Tien Nguyen, Duy Minh Ho Nguyen, Nghiem Tuong Diep, Trung Quoc Nguyen, Nhat Ho, Jacqueline Michelle Metsch, Miriam Cindy Maurer, Daniel Sonntag, Hanibal Bohnenberger, Anne-Christin Hauschild
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07409
• Pdf URL: https://arxiv.org/pdf/2502.07409
• Copy Paste: [[2502.07409]] MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification(https://arxiv.org/abs/2502.07409)
• Keywords: foundation model
• Abstract: Whole slide pathology image classification presents challenges due to gigapixel image sizes and limited annotation labels, hindering model generalization. This paper introduces a prompt learning method to adapt large vision-language models for few-shot pathology classification. We first extend the Prov-GigaPath vision foundation model, pre-trained on 1.3 billion pathology image tiles, into a vision-language model by adding adaptors and aligning it with medical text encoders via contrastive learning on 923K image-text pairs. The model is then used to extract visual features and text embeddings from few-shot annotations and fine-tunes with learnable prompt embeddings. Unlike prior methods that combine prompts with frozen features using prefix embeddings or self-attention, we propose multi-granular attention that compares interactions between learnable prompts with individual image patches and groups of them. This approach improves the model's ability to capture both fine-grained details and broader context, enhancing its recognition of complex patterns across sub-regions. To further improve accuracy, we leverage (unbalanced) optimal transport-based visual-text distance to secure model robustness by mitigating perturbations that might occur during the data augmentation process. Empirical experiments on lung, kidney, and breast pathology modalities validate the effectiveness of our approach; thereby, we surpass several of the latest competitors and consistently improve performance across diverse architectures, including CLIP, PLIP, and Prov-GigaPath integrated PLIP. We release our implementations and pre-trained models at this MGPATH.

Title: Towards a Foundation Model for Physics-Informed Neural Networks: Multi-PDE Learning with Active Sampling

• Authors: Keon Vin Park
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07425
• Pdf URL: https://arxiv.org/pdf/2502.07425
• Copy Paste: [[2502.07425]] Towards a Foundation Model for Physics-Informed Neural Networks: Multi-PDE Learning with Active Sampling(https://arxiv.org/abs/2502.07425)
• Keywords: foundation model
• Abstract: Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical laws into neural network training. However, traditional PINN models are typically designed for single PDEs, limiting their generalizability across different physical systems. In this work, we explore the potential of a foundation PINN model capable of solving multiple PDEs within a unified architecture. We investigate the efficacy of a single PINN framework trained on four distinct PDEs-the Simple Harmonic Oscillator (SHO), the 1D Heat Equation, the 1D Wave Equation, and the 2D Laplace Equation, demonstrating its ability to learn diverse physical dynamics. To enhance sample efficiency, we incorporate Active Learning (AL) using Monte Carlo (MC) Dropout-based uncertainty estimation, selecting the most informative training samples iteratively. We evaluate different active learning strategies, comparing models trained on 10%, 20%, 30%, 40%, and 50% of the full dataset, and analyze their impact on solution accuracy. Our results indicate that targeted uncertainty sampling significantly improves performance with fewer training samples, leading to efficient learning across multiple PDEs. This work highlights the feasibility of a generalizable PINN-based foundation model, capable of adapting to different physics-based problems without redesigning network architectures. Our findings suggest that multi-PDE PINNs with active learning can serve as an effective approach for reducing computational costs while maintaining high accuracy in physics-based deep learning applications.

Title: Optimizing Knowledge Distillation in Transformers: Enabling Multi-Head Attention without Alignment Barriers

• Authors: Zhaodong Bing, Linze Li, Jiajun Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07436
• Pdf URL: https://arxiv.org/pdf/2502.07436
• Copy Paste: [[2502.07436]] Optimizing Knowledge Distillation in Transformers: Enabling Multi-Head Attention without Alignment Barriers(https://arxiv.org/abs/2502.07436)
• Keywords: generative
• Abstract: Knowledge distillation (KD) in transformers often faces challenges due to misalignment in the number of attention heads between teacher and student models. Existing methods either require identical head counts or introduce projectors to bridge dimensional gaps, limiting flexibility and efficiency. We propose Squeezing-Heads Distillation (SHD), a novel approach that enables seamless knowledge transfer between models with varying head counts by compressing multi-head attention maps via efficient linear approximation. Unlike prior work, SHD eliminates alignment barriers without additional parameters or architectural modifications. Our method dynamically approximates the combined effect of multiple teacher heads into fewer student heads, preserving fine-grained attention patterns while reducing redundancy. Experiments across language (LLaMA, GPT) and vision (DiT, MDT) generative and vision (DeiT) discriminative tasks demonstrate SHD's effectiveness: it outperforms logit-based and feature-alignment KD baselines, achieving state-of-the-art results in image classification, image generation language fine-tuning, and language pre-training. The key innovations of flexible head compression, projector-free design, and linear-time complexity make SHD a versatile and scalable solution for distilling modern transformers. This work bridges a critical gap in KD, enabling efficient deployment of compact models without compromising performance.

Title: RusCode: Russian Cultural Code Benchmark for Text-to-Image Generation

• Authors: Viacheslav Vasilev, Julia Agafonova, Nikolai Gerasimenko, Alexander Kapitanov, Polina Mikhailova, Evelina Mironova, Denis Dimitrov
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07455
• Pdf URL: https://arxiv.org/pdf/2502.07455
• Copy Paste: [[2502.07455]] RusCode: Russian Cultural Code Benchmark for Text-to-Image Generation(https://arxiv.org/abs/2502.07455)
• Keywords: generative
• Abstract: Text-to-image generation models have gained popularity among users around the world. However, many of these models exhibit a strong bias toward English-speaking cultures, ignoring or misrepresenting the unique characteristics of other language groups, countries, and nationalities. The lack of cultural awareness can reduce the generation quality and lead to undesirable consequences such as unintentional insult, and the spread of prejudice. In contrast to the field of natural language processing, cultural awareness in computer vision has not been explored as extensively. In this paper, we strive to reduce this gap. We propose a RusCode benchmark for evaluating the quality of text-to-image generation containing elements of the Russian cultural code. To do this, we form a list of 19 categories that best represent the features of Russian visual culture. Our final dataset consists of 1250 text prompts in Russian and their translations into English. The prompts cover a wide range of topics, including complex concepts from art, popular culture, folk traditions, famous people's names, natural objects, scientific achievements, etc. We present the results of a human evaluation of the side-by-side comparison of Russian visual concepts representations using popular generative models.

Title: Less is More: Masking Elements in Image Condition Features Avoids Content Leakages in Style Transfer Diffusion Models

• Authors: Lin Zhu, Xinbing Wang, Chenghu Zhou, Qinying Gu, Nanyang Ye
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07466
• Pdf URL: https://arxiv.org/pdf/2502.07466
• Copy Paste: [[2502.07466]] Less is More: Masking Elements in Image Condition Features Avoids Content Leakages in Style Transfer Diffusion Models(https://arxiv.org/abs/2502.07466)
• Keywords: diffusion
• Abstract: Given a style-reference image as the additional image condition, text-to-image diffusion models have demonstrated impressive capabilities in generating images that possess the content of text prompts while adopting the visual style of the reference image. However, current state-of-the-art methods often struggle to disentangle content and style from style-reference images, leading to issues such as content leakages. To address this issue, we propose a masking-based method that efficiently decouples content from style without the need of tuning any model parameters. By simply masking specific elements in the style reference's image features, we uncover a critical yet under-explored principle: guiding with appropriately-selected fewer conditions (e.g., dropping several image feature elements) can efficiently avoid unwanted content flowing into the diffusion models, enhancing the style transfer performances of text-to-image diffusion models. In this paper, we validate this finding both theoretically and experimentally. Extensive experiments across various styles demonstrate the effectiveness of our masking-based method and support our theoretical results.

Title: Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More

• Authors: Xialie Zhuang, Zhikai Jia, Jianjin Li, Zhenyu Zhang, Li Shen, Zheng Cao, Shiwei Liu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07490
• Pdf URL: https://arxiv.org/pdf/2502.07490
• Copy Paste: [[2502.07490]] Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More(https://arxiv.org/abs/2502.07490)
• Keywords: in-context
• Abstract: Large Language Models (LLMs) are discovered to suffer from accurately retrieving key information. To address this, we propose Mask-Enhanced Autoregressive Prediction (MEAP), a simple yet effective training paradigm that seamlessly integrates Masked Language Modeling (MLM) into Next-Token Prediction (NTP) to enhance the latter's in-context retrieval capabilities. Specifically, MEAP first randomly masks a small fraction of input tokens and then directly performs the standard next-token prediction autoregressive using a decoder-only Transformer. MEAP eliminates the need for bidirectional attention or encoder-decoder architectures for MLM, incurring no additional computational overhead during pre-training or inference. Intensive experiments demonstrate that MEAP substantially outperforms NTP on key information retrieval and long-context reasoning tasks, while performing on par or better on commonsense reasoning tasks. The benefits of MEAP also extend to supervised fine-tuning, where it shows remarkable advantages in lost-in-the-middle scenarios, outperforming NTP by 11.77 percentage points. Our analysis indicates that MEAP's effectiveness arises from its ability to promote more distinguishable attention scores by concentrating on a reduced set of non-masked tokens. This mechanism improves the model's focus on task-relevant signals while mitigating the influence of peripheral context. These findings position MEAP as a promising training paradigm for large language models.

Title: Decentralized Entropy-Driven Ransomware Detection Using Autonomous Neural Graph Embeddings

• Authors: Ekaterina Starchenko, Hugo Bellinghamshire, David Pickering, Tristan Weatherspoon, Nathaniel Berkhamstead, Elizabeth Green, Magnus Rothschild
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.07498
• Pdf URL: https://arxiv.org/pdf/2502.07498
• Copy Paste: [[2502.07498]] Decentralized Entropy-Driven Ransomware Detection Using Autonomous Neural Graph Embeddings(https://arxiv.org/abs/2502.07498)
• Keywords: anomaly
• Abstract: The increasing sophistication of cyber threats has necessitated the development of advanced detection mechanisms capable of identifying and mitigating ransomware attacks with high precision and efficiency. A novel framework, termed Decentralized Entropy-Driven Detection (DED), is introduced, leveraging autonomous neural graph embeddings and entropy-based anomaly scoring to address the limitations of traditional methods. The framework operates on a distributed network of nodes, eliminating single points of failure and enhancing resilience against targeted attacks. Experimental results demonstrate its ability to achieve detection accuracy exceeding 95\%, with false positive rates maintained below 2\% across diverse ransomware variants. The integration of graph-based modeling and machine learning techniques enables the framework to capture complex system interactions, facilitating the identification of subtle anomalies indicative of ransomware activity. Comparative analysis against existing methods highlights its superior performance in terms of detection rates and computational efficiency. Case studies further validate its effectiveness in real-world scenarios, showcasing its ability to detect and mitigate ransomware attacks within minutes of their initiation. The proposed framework represents a significant step forward in cybersecurity, offering a scalable and adaptive solution to the growing challenge of ransomware detection.

Title: Diffusion-LAM: Probabilistic Limited Area Weather Forecasting with Diffusion

• Authors: Erik Larsson, Joel Oskarsson, Tomas Landelius, Fredrik Lindsten
• Subjects: cs.LG, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2502.07532
• Pdf URL: https://arxiv.org/pdf/2502.07532
• Copy Paste: [[2502.07532]] Diffusion-LAM: Probabilistic Limited Area Weather Forecasting with Diffusion(https://arxiv.org/abs/2502.07532)
• Keywords: diffusion
• Abstract: Machine learning methods have been shown to be effective for weather forecasting, based on the speed and accuracy compared to traditional numerical models. While early efforts primarily concentrated on deterministic predictions, the field has increasingly shifted toward probabilistic forecasting to better capture the forecast uncertainty. Most machine learning-based models have been designed for global-scale predictions, with only limited work targeting regional or limited area forecasting, which allows more specialized and flexible modeling for specific locations. This work introduces Diffusion-LAM, a probabilistic limited area weather model leveraging conditional diffusion. By conditioning on boundary data from surrounding regions, our approach generates forecasts within a defined area. Experimental results on the MEPS limited area dataset demonstrate the potential of Diffusion-LAM to deliver accurate probabilistic forecasts, highlighting its promise for limited-area weather prediction.

Title: O1 Embedder: Let Retrievers Think Before Action

• Authors: Ruin Yan, Zheng Liu, Defu Lian
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07555
• Pdf URL: https://arxiv.org/pdf/2502.07555
• Copy Paste: [[2502.07555]] O1 Embedder: Let Retrievers Think Before Action(https://arxiv.org/abs/2502.07555)
• Keywords: foundation model
• Abstract: The growing power of large language models (LLMs) has revolutionized how people access and utilize information. Notably, the LLMs excel at performing fine-grained data representation, which facilitates precise retrieval of information. They also generate high-quality answers based on external references, enabling the production of useful knowledge. The recent introduction of reasoning models, like OpenAI O1 and DeepSeek R1, marks another leap forward, highlighting LLMs' ability to think progressively before delivering final answers. This breakthrough significantly improves the ability to address complex tasks, e.g., coding and math proofs. Inspired by this progress, we aim to develop similar capabilities for retrieval models, which hold great promise for tackling critical challenges in the field, including multi-task retrieval, zero-shot retrieval, and tasks requiring intensive reasoning of complex relationships. With this motivation, we propose a novel approach called O1 Embedder, which generates useful thoughts for the input query before making retrieval for the target documents. To realize this objective, we conquer two technical difficulties. First, we design a data synthesis workflow, creating training signals for O1 Embedder by generating initial thoughts from an LLM-expert and subsequently refining them using a retrieval committee. Second, we optimize the training process, enabling a pre-trained model to be jointly fine-tuned to generate retrieval thoughts via behavior cloning and perform dense retrieval through contrastive learning. Our approach is evaluated by comprehensive experiments, where substantial improvements are achieved across 12 popular datasets, spanning both in-domain and out-of-domain scenarios. These results highlight O1 Embedder's remarkable accuracy and generalizability, paving the way for the development of next-generation IR foundation models.

Title: Automated Capability Discovery via Model Self-Exploration

• Authors: Cong Lu, Shengran Hu, Jeff Clune
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07577
• Pdf URL: https://arxiv.org/pdf/2502.07577
• Copy Paste: [[2502.07577]] Automated Capability Discovery via Model Self-Exploration(https://arxiv.org/abs/2502.07577)
• Keywords: foundation model
• Abstract: Foundation models have become general-purpose assistants, exhibiting diverse capabilities across numerous domains through training on web-scale data. It remains challenging to precisely characterize even a fraction of the full spectrum of capabilities and potential risks in any new model. Existing evaluation approaches often require significant human effort, and it is taking increasing effort to design ever harder challenges for more capable models. We introduce Automated Capability Discovery (ACD), a framework that designates one foundation model as a scientist to systematically propose open-ended tasks probing the abilities of a subject model (potentially itself). By combining frontier models with ideas from the field of open-endedness, ACD automatically and systematically uncovers both surprising capabilities and failures in the subject model. We demonstrate ACD across a range of foundation models (including the GPT, Claude, and Llama series), showing that it automatically reveals thousands of capabilities that would be challenging for any single team to uncover. We further validate our method's automated scoring with extensive human surveys, observing high agreement between model-generated and human evaluations. By leveraging foundation models' ability to both create tasks and self-evaluate, ACD is a significant step toward scalable, automated evaluation of novel AI systems. All code and evaluation logs are open-sourced at this https URL.

Title: Single-Step Consistent Diffusion Samplers

• Authors: Pascal Jutras-Dubé, Patrick Pynadath, Ruqi Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07579
• Pdf URL: https://arxiv.org/pdf/2502.07579
• Copy Paste: [[2502.07579]] Single-Step Consistent Diffusion Samplers(https://arxiv.org/abs/2502.07579)
• Keywords: diffusion
• Abstract: Sampling from unnormalized target distributions is a fundamental yet challenging task in machine learning and statistics. Existing sampling algorithms typically require many iterative steps to produce high-quality samples, leading to high computational costs that limit their practicality in time-sensitive or resource-constrained settings. In this work, we introduce consistent diffusion samplers, a new class of samplers designed to generate high-fidelity samples in a single step. We first develop a distillation algorithm to train a consistent diffusion sampler from a pretrained diffusion model without pre-collecting large datasets of samples. Our algorithm leverages incomplete sampling trajectories and noisy intermediate states directly from the diffusion process. We further propose a method to train a consistent diffusion sampler from scratch, fully amortizing exploration by training a single model that both performs diffusion sampling and skips intermediate steps using a self-consistency loss. Through extensive experiments on a variety of unnormalized distributions, we show that our approach yields high-fidelity samples using less than 1% of the network evaluations required by traditional diffusion samplers.

Title: Generative Modeling with Bayesian Sample Inference

• Authors: Marten Lienen, Marcel Kollovieh, Stephan Günnemann
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07580
• Pdf URL: https://arxiv.org/pdf/2502.07580
• Copy Paste: [[2502.07580]] Generative Modeling with Bayesian Sample Inference(https://arxiv.org/abs/2502.07580)
• Keywords: diffusion, generative
• Abstract: We derive a novel generative model from the simple act of Gaussian posterior inference. Treating the generated sample as an unknown variable to infer lets us formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to narrow down the unknown sample from a broad initial belief. In addition to a rigorous theoretical analysis, we establish a connection between our model and diffusion models and show that it includes Bayesian Flow Networks (BFNs) as a special case. In our experiments, we demonstrate improved performance over both BFNs and Variational Diffusion Models, achieving competitive likelihood scores on CIFAR10 and ImageNet.

Title: SEMU: Singular Value Decomposition for Efficient Machine Unlearning

• Authors: Marcin Sendera, Łukasz Struski, Kamil Książek, Kryspin Musiol, Jacek Tabor, Dawid Rymarczyk
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07587
• Pdf URL: https://arxiv.org/pdf/2502.07587
• Copy Paste: [[2502.07587]] SEMU: Singular Value Decomposition for Efficient Machine Unlearning(https://arxiv.org/abs/2502.07587)
• Keywords: generative
• Abstract: While the capabilities of generative foundational models have advanced rapidly in recent years, methods to prevent harmful and unsafe behaviors remain underdeveloped. Among the pressing challenges in AI safety, machine unlearning (MU) has become increasingly critical to meet upcoming safety regulations. Most existing MU approaches focus on altering the most significant parameters of the model. However, these methods often require fine-tuning substantial portions of the model, resulting in high computational costs and training instabilities, which are typically mitigated by access to the original training dataset. In this work, we address these limitations by leveraging Singular Value Decomposition (SVD) to create a compact, low-dimensional projection that enables the selective forgetting of specific data points. We propose Singular Value Decomposition for Efficient Machine Unlearning (SEMU), a novel approach designed to optimize MU in two key aspects. First, SEMU minimizes the number of model parameters that need to be modified, effectively removing unwanted knowledge while making only minimal changes to the model's weights. Second, SEMU eliminates the dependency on the original training dataset, preserving the model's previously acquired knowledge without additional data requirements. Extensive experiments demonstrate that SEMU achieves competitive performance while significantly improving efficiency in terms of both data usage and the number of modified parameters.

Title: Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models

• Authors: Jiacong Xu, Shao-Yuan Lo, Bardia Safaei, Vishal M. Patel, Isht Dwivedi
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07601
• Pdf URL: https://arxiv.org/pdf/2502.07601
• Copy Paste: [[2502.07601]] Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models(https://arxiv.org/abs/2502.07601)
• Keywords: anomaly
• Abstract: Zero-Shot Anomaly Detection (ZSAD) is an emerging AD paradigm. Unlike the traditional unsupervised AD setting that requires a large number of normal samples to train a model, ZSAD is more practical for handling data-restricted real-world scenarios. Recently, Multimodal Large Language Models (MLLMs) have shown revolutionary reasoning capabilities in various vision tasks. However, the reasoning of image abnormalities remains underexplored due to the lack of corresponding datasets and benchmarks. To facilitate research in AD & reasoning, we establish the first visual instruction tuning dataset, Anomaly-Instruct-125k, and the evaluation benchmark, VisA-D&R. Through investigation with our benchmark, we reveal that current MLLMs like GPT-4o cannot accurately detect and describe fine-grained anomalous details in images. To address this, we propose Anomaly-OneVision (Anomaly-OV), the first specialist visual assistant for ZSAD and reasoning. Inspired by human behavior in visual inspection, Anomaly-OV leverages a Look-Twice Feature Matching (LTFM) mechanism to adaptively select and emphasize abnormal visual tokens. Extensive experiments demonstrate that Anomaly-OV achieves significant improvements over advanced generalist models in both detection and reasoning. Extensions to medical and 3D AD are provided for future study. The link to our project page: this https URL

Title: Beyond Prompting: Time2Lang -- Bridging Time-Series Foundation Models and Large Language Models for Health Sensing

• Authors: Arvind Pillai, Dimitris Spathis, Subigya Nepal, Amanda C Collins, Daniel M Mackin, Michael V Heinz, Tess Z Griffin, Nicholas C Jacobson, Andrew Campbell
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.07608
• Pdf URL: https://arxiv.org/pdf/2502.07608
• Copy Paste: [[2502.07608]] Beyond Prompting: Time2Lang -- Bridging Time-Series Foundation Models and Large Language Models for Health Sensing(https://arxiv.org/abs/2502.07608)
• Keywords: foundation model
• Abstract: Large language models (LLMs) show promise for health applications when combined with behavioral sensing data. Traditional approaches convert sensor data into text prompts, but this process is prone to errors, computationally expensive, and requires domain expertise. These challenges are particularly acute when processing extended time series data. While time series foundation models (TFMs) have recently emerged as powerful tools for learning representations from temporal data, bridging TFMs and LLMs remains challenging. Here, we present Time2Lang, a framework that directly maps TFM outputs to LLM representations without intermediate text conversion. Our approach first trains on synthetic data using periodicity prediction as a pretext task, followed by evaluation on mental health classification tasks. We validate Time2Lang on two longitudinal wearable and mobile sensing datasets: daily depression prediction using step count data (17,251 days from 256 participants) and flourishing classification based on conversation duration (46 participants over 10 weeks). Time2Lang maintains near constant inference times regardless of input length, unlike traditional prompting methods. The generated embeddings preserve essential time-series characteristics such as auto-correlation. Our results demonstrate that TFMs and LLMs can be effectively integrated while minimizing information loss and enabling performance transfer across these distinct modeling paradigms. To our knowledge, we are the first to integrate a TFM and an LLM for health, thus establishing a foundation for future research combining general-purpose large models for complex healthcare tasks.

Title: Tractable Transformers for Flexible Conditional Generation

• Authors: Anji Liu, Xuejie Liu, Dayuan Zhao, Mathias Niepert, Yitao Liang, Guy Van den Broeck
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07616
• Pdf URL: https://arxiv.org/pdf/2502.07616
• Copy Paste: [[2502.07616]] Tractable Transformers for Flexible Conditional Generation(https://arxiv.org/abs/2502.07616)
• Keywords: diffusion, generative
• Abstract: Non-autoregressive (NAR) generative models are valuable because they can handle diverse conditional generation tasks in a more principled way than their autoregressive (AR) counterparts, which are constrained by sequential dependency requirements. Recent advancements in NAR models, such as diffusion language models, have demonstrated superior performance in unconditional generation compared to AR models (e.g., GPTs) of similar sizes. However, such improvements do not always lead to improved conditional generation performance. We show that a key reason for this gap is the difficulty in generalizing to conditional probability queries unseen during training. As a result, strong unconditional generation performance does not guarantee high-quality conditional generation. This paper proposes Tractable Transformers (Tracformer), a Transformer-based generative model that is more robust to different conditional generation tasks. Unlike existing models that rely solely on global contextual features derived from full inputs, Tracformers incorporate a sparse Transformer encoder to capture both local and global contextual information. This information is routed through a decoder for conditional generation. Empirical results demonstrate that Tracformers achieve state-of-the-art conditional generation performance on text modeling compared to recent diffusion and AR model baselines.

Title: Consistency Training with Physical Constraints

• Authors: Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin, Ming-Chih Lai, Chieh-Hsin Lai
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07636
• Pdf URL: https://arxiv.org/pdf/2502.07636
• Copy Paste: [[2502.07636]] Consistency Training with Physical Constraints(https://arxiv.org/abs/2502.07636)
• Keywords: diffusion, generative
• Abstract: We propose a physics-aware Consistency Training (CT) method that accelerates sampling in Diffusion Models with physical constraints. Our approach leverages a two-stage strategy: (1) learning the noise-to-data mapping via CT, and (2) incorporating physics constraints as a regularizer. Experiments on toy examples show that our method generates samples in a single step while adhering to the imposed constraints. This approach has the potential to efficiently solve partial differential equations (PDEs) using deep generative modeling.

Title: Matrix3D: Large Photogrammetry Model All-in-One

• Authors: Yuanxun Lu, Jingyang Zhang, Tian Fang, Jean-Daniel Nahmias, Yanghai Tsin, Long Quan, Xun Cao, Yao Yao, Shiwei Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07685
• Pdf URL: https://arxiv.org/pdf/2502.07685
• Copy Paste: [[2502.07685]] Matrix3D: Large Photogrammetry Model All-in-One(https://arxiv.org/abs/2502.07685)
• Keywords: diffusion
• Abstract: We present Matrix3D, a unified model that performs several photogrammetry subtasks, including pose estimation, depth prediction, and novel view synthesis using just the same model. Matrix3D utilizes a multi-modal diffusion transformer (DiT) to integrate transformations across several modalities, such as images, camera parameters, and depth maps. The key to Matrix3D's large-scale multi-modal training lies in the incorporation of a mask learning strategy. This enables full-modality model training even with partially complete data, such as bi-modality data of image-pose and image-depth pairs, thus significantly increases the pool of available training data. Matrix3D demonstrates state-of-the-art performance in pose estimation and novel view synthesis tasks. Additionally, it offers fine-grained control through multi-round interactions, making it an innovative tool for 3D content creation. Project page: this https URL.

Title: Magic 1-For-1: Generating One Minute Video Clips within One Minute

• Authors: Hongwei Yi, Shitong Shao, Tian Ye, Jiantong Zhao, Qingyu Yin, Michael Lingelbach, Li Yuan, Yonghong Tian, Enze Xie, Daquan Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07701
• Pdf URL: https://arxiv.org/pdf/2502.07701
• Copy Paste: [[2502.07701]] Magic 1-For-1: Generating One Minute Video Clips within One Minute(https://arxiv.org/abs/2502.07701)
• Keywords: diffusion, foundation model
• Abstract: In this technical report, we present Magic 1-For-1 (Magic141), an efficient video generation model with optimized memory consumption and inference latency. The key idea is simple: factorize the text-to-video generation task into two separate easier tasks for diffusion step distillation, namely text-to-image generation and image-to-video generation. We verify that with the same optimization algorithm, the image-to-video task is indeed easier to converge over the text-to-video task. We also explore a bag of optimization tricks to reduce the computational cost of training the image-to-video (I2V) models from three aspects: 1) model convergence speedup by using a multi-modal prior condition injection; 2) inference latency speed up by applying an adversarial step distillation, and 3) inference memory cost optimization with parameter sparsification. With those techniques, we are able to generate 5-second video clips within 3 seconds. By applying a test time sliding window, we are able to generate a minute-long video within one minute with significantly improved visual quality and motion dynamics, spending less than 1 second for generating 1 second video clips on average. We conduct a series of preliminary explorations to find out the optimal tradeoff between computational cost and video quality during diffusion step distillation and hope this could be a good foundation model for open-source explorations. The code and the model weights are available at this https URL.

Title: Near-Optimal Sample Complexity in Reward-Free Kernel-Based Reinforcement Learning

• Authors: Aya Kayal, Sattar Vakili, Laura Toni, Alberto Bernacchia
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07715
• Pdf URL: https://arxiv.org/pdf/2502.07715
• Copy Paste: [[2502.07715]] Near-Optimal Sample Complexity in Reward-Free Kernel-Based Reinforcement Learning(https://arxiv.org/abs/2502.07715)
• Keywords: generative
• Abstract: Reinforcement Learning (RL) problems are being considered under increasingly more complex structures. While tabular and linear models have been thoroughly explored, the analytical study of RL under nonlinear function approximation, especially kernel-based models, has recently gained traction for their strong representational capacity and theoretical tractability. In this context, we examine the question of statistical efficiency in kernel-based RL within the reward-free RL framework, specifically asking: how many samples are required to design a near-optimal policy? Existing work addresses this question under restrictive assumptions about the class of kernel functions. We first explore this question by assuming a generative model, then relax this assumption at the cost of increasing the sample complexity by a factor of H, the length of the episode. We tackle this fundamental problem using a broad class of kernels and a simpler algorithm compared to prior work. Our approach derives new confidence intervals for kernel ridge regression, specific to our RL setting, which may be of broader applicability. We further validate our theoretical findings through simulations.

Title: Making Language Models Robust Against Negation

• Authors: MohammadHossein Rezaei, Eduardo Blanco
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.07717
• Pdf URL: https://arxiv.org/pdf/2502.07717
• Copy Paste: [[2502.07717]] Making Language Models Robust Against Negation(https://arxiv.org/abs/2502.07717)
• Keywords: self-supervised
• Abstract: Negation has been a long-standing challenge for language models. Previous studies have shown that they struggle with negation in many natural language understanding tasks. In this work, we propose a self-supervised method to make language models more robust against negation. We introduce a novel task, Next Sentence Polarity Prediction (NSPP), and a variation of the Next Sentence Prediction (NSP) task. We show that BERT and RoBERTa further pre-trained on our tasks outperform the off-the-shelf versions on nine negation-related benchmarks. Most notably, our pre-training tasks yield between 1.8% and 9.1% improvement on CondaQA, a large question-answering corpus requiring reasoning over negation.

Title: Revisiting Non-Acyclic GFlowNets in Discrete Environments

• Authors: Nikita Morozov, Ian Maksimov, Daniil Tiapkin, Sergey Samsonov
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.07735
• Pdf URL: https://arxiv.org/pdf/2502.07735
• Copy Paste: [[2502.07735]] Revisiting Non-Acyclic GFlowNets in Discrete Environments(https://arxiv.org/abs/2502.07735)
• Keywords: generative
• Abstract: Generative Flow Networks (GFlowNets) are a family of generative models that learn to sample objects from a given probability distribution, potentially known up to a normalizing constant. Instead of working in the object space, GFlowNets proceed by sampling trajectories in an appropriately constructed directed acyclic graph environment, greatly relying on the acyclicity of the graph. In our paper, we revisit the theory that relaxes the acyclicity assumption and present a simpler theoretical framework for non-acyclic GFlowNets in discrete environments. Moreover, we provide various novel theoretical insights related to training with fixed backward policies, the nature of flow functions, and connections between entropy-regularized RL and non-acyclic GFlowNets, which naturally generalize the respective concepts and theoretical results from the acyclic setting. In addition, we experimentally re-examine the concept of loss stability in non-acyclic GFlowNet training, as well as validate our own theoretical findings.

Title: Advancing climate model interpretability: Feature attribution for Arctic melt anomalies

• Authors: Tolulope Ale, Nicole-Jeanne Schlegel, Vandana P. Janeja
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.07741
• Pdf URL: https://arxiv.org/pdf/2502.07741
• Copy Paste: [[2502.07741]] Advancing climate model interpretability: Feature attribution for Arctic melt anomalies(https://arxiv.org/abs/2502.07741)
• Keywords: anomaly
• Abstract: The focus of our work is improving the interpretability of anomalies in climate models and advancing our understanding of Arctic melt dynamics. The Arctic and Antarctic ice sheets are experiencing rapid surface melting and increased freshwater runoff, contributing significantly to global sea level rise. Understanding the mechanisms driving snowmelt in these regions is crucial. ERA5, a widely used reanalysis dataset in polar climate studies, offers extensive climate variables and global data assimilation. However, its snowmelt model employs an energy imbalance approach that may oversimplify the complexity of surface melt. In contrast, the Glacier Energy and Mass Balance (GEMB) model incorporates additional physical processes, such as snow accumulation, firn densification, and meltwater percolation/refreezing, providing a more detailed representation of surface melt dynamics. In this research, we focus on analyzing surface snowmelt dynamics of the Greenland Ice Sheet using feature attribution for anomalous melt events in ERA5 and GEMB models. We present a novel unsupervised attribution method leveraging counterfactual explanation method to analyze detected anomalies in ERA5 and GEMB. Our anomaly detection results are validated using MEaSUREs ground-truth data, and the attributions are evaluated against established feature ranking methods, including XGBoost, Shapley values, and Random Forest. Our attribution framework identifies the physics behind each model and the climate features driving melt anomalies. These findings demonstrate the utility of our attribution method in enhancing the interpretability of anomalies in climate models and advancing our understanding of Arctic melt dynamics.

Title: CausalGeD: Blending Causality and Diffusion for Spatial Gene Expression Generation

• Authors: Rabeya Tus Sadia, Md Atik Ahamed, Qiang Cheng
• Subjects: cs.CV, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2502.07751
• Pdf URL: https://arxiv.org/pdf/2502.07751
• Copy Paste: [[2502.07751]] CausalGeD: Blending Causality and Diffusion for Spatial Gene Expression Generation(https://arxiv.org/abs/2502.07751)
• Keywords: diffusion
• Abstract: The integration of single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) data is crucial for understanding gene expression in spatial context. Existing methods for such integration have limited performance, with structural similarity often below 60\%, We attribute this limitation to the failure to consider causal relationships between genes. We present CausalGeD, which combines diffusion and autoregressive processes to leverage these relationships. By generalizing the Causal Attention Transformer from image generation to gene expression data, our model captures regulatory mechanisms without predefined relationships. Across 10 tissue datasets, CausalGeD outperformed state-of-the-art baselines by 5- 32\% in key metrics, including Pearson's correlation and structural similarity, advancing both technical and biological insights.

Title: Direct Ascent Synthesis: Revealing Hidden Generative Capabilities in Discriminative Models

• Authors: Stanislav Fort, Jonathan Whitaker
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07753
• Pdf URL: https://arxiv.org/pdf/2502.07753
• Copy Paste: [[2502.07753]] Direct Ascent Synthesis: Revealing Hidden Generative Capabilities in Discriminative Models(https://arxiv.org/abs/2502.07753)
• Keywords: generative
• Abstract: We demonstrate that discriminative models inherently contain powerful generative capabilities, challenging the fundamental distinction between discriminative and generative architectures. Our method, Direct Ascent Synthesis (DAS), reveals these latent capabilities through multi-resolution optimization of CLIP model representations. While traditional inversion attempts produce adversarial patterns, DAS achieves high-quality image synthesis by decomposing optimization across multiple spatial scales (1x1 to 224x224), requiring no additional training. This approach not only enables diverse applications -- from text-to-image generation to style transfer -- but maintains natural image statistics ($1/f^2$ spectrum) and guides the generation away from non-robust adversarial patterns. Our results demonstrate that standard discriminative models encode substantially richer generative knowledge than previously recognized, providing new perspectives on model interpretability and the relationship between adversarial examples and natural image synthesis.

Title: Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection

• Authors: Anirudh Sundara Rajan, Yong Jae Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.07778
• Pdf URL: https://arxiv.org/pdf/2502.07778
• Copy Paste: [[2502.07778]] Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection(https://arxiv.org/abs/2502.07778)
• Keywords: generative
• Abstract: Detecting AI generated images is a challenging yet essential task. A primary difficulty arises from the detectors tendency to rely on spurious patterns, such as compression artifacts, which can influence its decisions. These issues often stem from specific patterns that the detector associates with the real data distribution, making it difficult to isolate the actual generative traces. We argue that an image should be classified as fake if and only if it contains artifacts introduced by the generative model. Based on this premise, we propose Stay Positive, an algorithm designed to constrain the detectors focus to generative artifacts while disregarding those associated with real data. Experimental results demonstrate that detectors trained with Stay Positive exhibit reduced susceptibility to spurious correlations, leading to improved generalization and robustness to post processing. Additionally, unlike detectors that associate artifacts with real images, those that focus purely on fake artifacts are better at detecting inpainted real images.

Title: MatSwap: Light-aware material transfers in images

• Authors: Ivan Lopes, Valentin Deschaintre, Yannick Hold-Geoffroy, Raoul de Charette
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.07784
• Pdf URL: https://arxiv.org/pdf/2502.07784
• Copy Paste: [[2502.07784]] MatSwap: Light-aware material transfers in images(https://arxiv.org/abs/2502.07784)
• Keywords: diffusion
• Abstract: We present MatSwap, a method to transfer materials to designated surfaces in an image photorealistically. Such a task is non-trivial due to the large entanglement of material appearance, geometry, and lighting in a photograph. In the literature, material editing methods typically rely on either cumbersome text engineering or extensive manual annotations requiring artist knowledge and 3D scene properties that are impractical to obtain. In contrast, we propose to directly learn the relationship between the input material -- as observed on a flat surface -- and its appearance within the scene, without the need for explicit UV mapping. To achieve this, we rely on a custom light- and geometry-aware diffusion model. We fine-tune a large-scale pre-trained text-to-image model for material transfer using our synthetic dataset, preserving its strong priors to ensure effective generalization to real images. As a result, our method seamlessly integrates a desired material into the target location in the photograph while retaining the identity of the scene. We evaluate our method on synthetic and real images and show that it compares favorably to recent work both qualitatively and quantitatively. We will release our code and data upon publication.

Title: Pippo: High-Resolution Multi-View Humans from a Single Image

• Authors: Yash Kant, Ethan Weber, Jin Kyu Kim, Rawal Khirodkar, Su Zhaoen, Julieta Martinez, Igor Gilitschenski, Shunsuke Saito, Timur Bagautdinov
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.07785
• Pdf URL: https://arxiv.org/pdf/2502.07785
• Copy Paste: [[2502.07785]] Pippo: High-Resolution Multi-View Humans from a Single Image(https://arxiv.org/abs/2502.07785)
• Keywords: diffusion, generative
• Abstract: We present Pippo, a generative model capable of producing 1K resolution dense turnaround videos of a person from a single casually clicked photo. Pippo is a multi-view diffusion transformer and does not require any additional inputs - e.g., a fitted parametric model or camera parameters of the input image. We pre-train Pippo on 3B human images without captions, and conduct multi-view mid-training and post-training on studio captured humans. During mid-training, to quickly absorb the studio dataset, we denoise several (up to 48) views at low-resolution, and encode target cameras coarsely using a shallow MLP. During post-training, we denoise fewer views at high-resolution and use pixel-aligned controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent generations. At inference, we propose an attention biasing technique that allows Pippo to simultaneously generate greater than 5 times as many views as seen during training. Finally, we also introduce an improved metric to evaluate 3D consistency of multi-view generations, and show that Pippo outperforms existing works on multi-view human generation from a single image.