2025-03-03

Title: Unifying Model Predictive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planning

• Authors: Yankai Li, Mo Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20476
• Pdf URL: https://arxiv.org/pdf/2502.20476
• Copy Paste: [[2502.20476]] Unifying Model Predictive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planning(https://arxiv.org/abs/2502.20476)
• Keywords: diffusion
• Abstract: Model Predictive Path Integral (MPPI) control, Reinforcement Learning (RL), and Diffusion Models have each demonstrated strong performance in trajectory optimization, decision-making, and motion planning. However, these approaches have traditionally been treated as distinct methodologies with separate optimization frameworks. In this work, we establish a unified perspective that connects MPPI, RL, and Diffusion Models through gradient-based optimization on the Gibbs measure. We first show that MPPI can be interpreted as performing gradient ascent on a smoothed energy function. We then demonstrate that Policy Gradient methods reduce to MPPI when treating policy parameters as control variables under a fixed initial state. Additionally, we establish that the reverse sampling process in diffusion models follows the same update rule as MPPI.

Title: Unified Kernel-Segregated Transpose Convolution Operation

• Authors: Vijay Srinivas Tida, Md Imran Hossen, Liqun Shan, Sai Venkatesh Chilukoti, Sonya Hsu, Xiali Hei
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20493
• Pdf URL: https://arxiv.org/pdf/2502.20493
• Copy Paste: [[2502.20493]] Unified Kernel-Segregated Transpose Convolution Operation(https://arxiv.org/abs/2502.20493)
• Keywords: generative
• Abstract: The optimization of the transpose convolution layer for deep learning applications is achieved with the kernel segregation mechanism. However, kernel segregation has disadvantages, such as computing extra elements to obtain the output feature map with odd dimensions while launching a thread. To mitigate this problem, we introduce a unified kernel segregation approach that limits the usage of memory and computational resources by employing one unified kernel to execute four sub-kernels. The findings reveal that the suggested approach achieves an average computational speedup of 2.03x (3.89x) when tested on specific datasets with an RTX 2070 GPU (Intel Xeon CPU). The ablation study shows an average computational speedup of 3.5x when evaluating the transpose convolution layers from well-known Generative Adversarial Networks (GANs). The implementation of the proposed method for the transpose convolution layers in the EB-GAN model demonstrates significant memory savings of up to 35 MB.

Title: CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understanding

• Authors: Yixiong Chen, Shawn Xu, Andrew Sellergren, Yossi Matias, Avinatan Hassidim, Shravya Shetty, Daniel Golden, Alan Yuille, Lin Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20509
• Pdf URL: https://arxiv.org/pdf/2502.20509
• Copy Paste: [[2502.20509]] CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understanding(https://arxiv.org/abs/2502.20509)
• Keywords: foundation model
• Abstract: Vision-language models have proven to be of great benefit for medical image analysis since they learn rich semantics from both images and reports. Prior efforts have focused on better alignment of image and text representations to enhance image understanding. However, though explicit reference to a prior image is common in Chest X-Ray (CXR) reports, aligning progression descriptions with the semantics differences in image pairs remains under-explored. In this work, we propose two components to address this issue. (1) A CXR report processing pipeline to extract temporal structure. It processes reports with a large language model (LLM) to separate the description and comparison contexts, and extracts fine-grained annotations from reports. (2) A contrastive captioner model for CXR, namely CoCa-CXR, to learn how to both describe images and their temporal progressions. CoCa-CXR incorporates a novel regional cross-attention module to identify local differences between paired CXR images. Extensive experiments show the superiority of CoCa-CXR on both progression analysis and report generation compared to previous methods. Notably, on MS-CXR-T progression classification, CoCa-CXR obtains 65.0% average testing accuracy on five pulmonary conditions, outperforming the previous state-of-the-art (SOTA) model BioViL-T by 4.8%. It also achieves a RadGraph F1 of 24.2% on MIMIC-CXR, which is comparable to the Med-Gemini foundation model.

Title: LISArD: Learning Image Similarity to Defend Against Gray-box Adversarial Attacks

• Authors: Joana C. Costa, Tiago Roxo, Hugo Proença, Pedro R. M. Inácio
• Subjects: cs.CV, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20562
• Pdf URL: https://arxiv.org/pdf/2502.20562
• Copy Paste: [[2502.20562]] LISArD: Learning Image Similarity to Defend Against Gray-box Adversarial Attacks(https://arxiv.org/abs/2502.20562)
• Keywords: generative
• Abstract: State-of-the-art defense mechanisms are typically evaluated in the context of white-box attacks, which is not realistic, as it assumes the attacker can access the gradients of the target network. To protect against this scenario, Adversarial Training (AT) and Adversarial Distillation (AD) include adversarial examples during the training phase, and Adversarial Purification uses a generative model to reconstruct all the images given to the classifier. This paper considers an even more realistic evaluation scenario: gray-box attacks, which assume that the attacker knows the architecture and the dataset used to train the target network, but cannot access its gradients. We provide empirical evidence that models are vulnerable to gray-box attacks and propose LISArD, a defense mechanism that does not increase computational and temporal costs but provides robustness against gray-box and white-box attacks without including AT. Our method approximates a cross-correlation matrix, created with the embeddings of perturbed and clean images, to a diagonal matrix while simultaneously conducting classification learning. Our results show that LISArD can effectively protect against gray-box attacks, can be used in multiple architectures, and carries over its resilience to the white-box scenario. Also, state-of-the-art AD models underperform greatly when removing AT and/or moving to gray-box settings, highlighting the lack of robustness from existing approaches to perform in various conditions (aside from white-box settings). All the source code is available at this https URL.

Title: InstaFace: Identity-Preserving Facial Editing with Single Image Inference

• Authors: MD Wahiduzzaman Khan, Mingshan Jia, Shaolin Zhang, En Yu, Kaska Musial-Gabrys
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20577
• Pdf URL: https://arxiv.org/pdf/2502.20577
• Copy Paste: [[2502.20577]] InstaFace: Identity-Preserving Facial Editing with Single Image Inference(https://arxiv.org/abs/2502.20577)
• Keywords: diffusion, generative
• Abstract: Facial appearance editing is crucial for digital avatars, AR/VR, and personalized content creation, driving realistic user experiences. However, preserving identity with generative models is challenging, especially in scenarios with limited data availability. Traditional methods often require multiple images and still struggle with unnatural face shifts, inconsistent hair alignment, or excessive smoothing effects. To overcome these challenges, we introduce a novel diffusion-based framework, InstaFace, to generate realistic images while preserving identity using only a single image. Central to InstaFace, we introduce an efficient guidance network that harnesses 3D perspectives by integrating multiple 3DMM-based conditionals without introducing additional trainable parameters. Moreover, to ensure maximum identity retention as well as preservation of background, hair, and other contextual features like accessories, we introduce a novel module that utilizes feature embeddings from a facial recognition model and a pre-trained vision-language model. Quantitative evaluations demonstrate that our method outperforms several state-of-the-art approaches in terms of identity preservation, photorealism, and effective control of pose, expression, and lighting.

Title: Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Inference

• Authors: Mingyuan Wu, Jize Jiang, Haozhen Zheng, Meitang Li, Zhaoheng Li, Beitong Tian, Bo Chen, Yongjoo Park, Minjia Zhang, Chengxiang Zhai, Klara Nahrstedt
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20587
• Pdf URL: https://arxiv.org/pdf/2502.20587
• Copy Paste: [[2502.20587]] Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Inference(https://arxiv.org/abs/2502.20587)
• Keywords: in-context
• Abstract: Vision Language Models (VLMs) have achieved remarkable success in a wide range of vision applications of increasing complexity and scales, yet choosing the right VLM model size involves a trade-off between response quality and cost. While smaller VLMs are cheaper to run, they typically produce responses only marginally better than random guessing on benchmarks such as MMMU. In this paper, we propose Cache of Thought (CoT), a master apprentice framework for collaborative inference between large and small VLMs. CoT manages high quality query results from large VLMs (master) in a cache, which are then selected via a novel multi modal retrieval and in-context learning to aid the performance of small VLMs (apprentice). We extensively evaluate CoT on various widely recognized and challenging general VQA benchmarks, and show that CoT increases overall VQA performance by up to 7.7% under the same budget, and specifically boosts the performance of apprentice VLMs by up to 36.6%.

Title: Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning

• Authors: Vicente Balmaseda, Bokun Wang, Ching-Long Lin, Tianbao Yang
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20612
• Pdf URL: https://arxiv.org/pdf/2502.20612
• Copy Paste: [[2502.20612]] Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning(https://arxiv.org/abs/2502.20612)
• Keywords: self-supervised
• Abstract: In self-supervised contrastive learning, negative pairs are typically constructed using an anchor image and a sample drawn from the entire dataset, excluding the anchor. However, this approach can result in the creation of negative pairs with similar semantics, referred to as "false negatives", leading to their embeddings being falsely pushed apart. To address this issue, we introduce GloFND, an optimization-based approach that automatically learns on the fly the threshold for each anchor data to identify its false negatives during training. In contrast to previous methods for false negative discovery, our approach globally detects false negatives across the entire dataset rather than locally within the mini-batch. Moreover, its per-iteration computation cost remains independent of the dataset size. Experimental results on image and image-text data demonstrate the effectiveness of the proposed method. Our implementation is available at this https URL .

Title: RTGen: Real-Time Generative Detection Transformer

• Authors: Chi Ruan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20622
• Pdf URL: https://arxiv.org/pdf/2502.20622
• Copy Paste: [[2502.20622]] RTGen: Real-Time Generative Detection Transformer(https://arxiv.org/abs/2502.20622)
• Keywords: generative
• Abstract: While open-vocabulary object detectors require predefined categories during inference, generative object detectors overcome this limitation by endowing the model with text generation capabilities. However, existing generative object detection methods directly append an autoregressive language model to an object detector to generate texts for each detected object. This straightforward design leads to structural redundancy and increased processing time. In this paper, we propose a Real-Time GENerative Detection Transformer (RTGen), a real-time generative object detector with a succinct encoder-decoder architecture. Specifically, we introduce a novel Region-Language Decoder (RL-Decoder), which innovatively integrates a non-autoregressive language model into the detection decoder, enabling concurrent processing of object and text information. With these efficient designs, RTGen achieves a remarkable inference speed of 60.41 FPS. Moreover, RTGen obtains 18.6 mAP on the LVIS dataset, outperforming the previous SOTA method by 3.5 mAP.

Title: SafeText: Safe Text-to-image Models via Aligning the Text Encoder

• Authors: Yuepeng Hu, Zhengyuan Jiang, Neil Zhenqiang Gong
• Subjects: cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20623
• Pdf URL: https://arxiv.org/pdf/2502.20623
• Copy Paste: [[2502.20623]] SafeText: Safe Text-to-image Models via Aligning the Text Encoder(https://arxiv.org/abs/2502.20623)
• Keywords: diffusion
• Abstract: Text-to-image models can generate harmful images when presented with unsafe prompts, posing significant safety and societal risks. Alignment methods aim to modify these models to ensure they generate only non-harmful images, even when exposed to unsafe prompts. A typical text-to-image model comprises two main components: 1) a text encoder and 2) a diffusion module. Existing alignment methods mainly focus on modifying the diffusion module to prevent harmful image generation. However, this often significantly impacts the model's behavior for safe prompts, causing substantial quality degradation of generated images. In this work, we propose SafeText, a novel alignment method that fine-tunes the text encoder rather than the diffusion module. By adjusting the text encoder, SafeText significantly alters the embedding vectors for unsafe prompts, while minimally affecting those for safe prompts. As a result, the diffusion module generates non-harmful images for unsafe prompts while preserving the quality of images for safe prompts. We evaluate SafeText on multiple datasets of safe and unsafe prompts, including those generated through jailbreak attacks. Our results show that SafeText effectively prevents harmful image generation with minor impact on the images for safe prompts, and SafeText outperforms six existing alignment methods. We will publish our code and data after paper acceptance.

Title: T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting

• Authors: Yifei Qian, Zhongliang Guo, Bowen Deng, Chun Tong Lei, Shuai Zhao, Chun Pong Lau, Xiaopeng Hong, Michael P. Pound
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20625
• Pdf URL: https://arxiv.org/pdf/2502.20625
• Copy Paste: [[2502.20625]] T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting(https://arxiv.org/abs/2502.20625)
• Keywords: diffusion
• Abstract: Zero-shot object counting aims to count instances of arbitrary object categories specified by text descriptions. Existing methods typically rely on vision-language models like CLIP, but often exhibit limited sensitivity to text prompts. We present T2ICount, a diffusion-based framework that leverages rich prior knowledge and fine-grained visual understanding from pretrained diffusion models. While one-step denoising ensures efficiency, it leads to weakened text sensitivity. To address this challenge, we propose a Hierarchical Semantic Correction Module that progressively refines text-image feature alignment, and a Representational Regional Coherence Loss that provides reliable supervision signals by leveraging the cross-attention maps extracted from the denosing U-Net. Furthermore, we observe that current benchmarks mainly focus on majority objects in images, potentially masking models' text sensitivity. To address this, we contribute a challenging re-annotated subset of FSC147 for better evaluation of text-guided counting ability. Extensive experiments demonstrate that our method achieves superior performance across different benchmarks. Code is available at this https URL.

Title: Are LLMs Ready for Practical Adoption for Assertion Generation?

• Authors: Vaishnavi Pulavarthi, Deeksha Nandal, Soham Dan, Debjit Pal
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20633
• Pdf URL: https://arxiv.org/pdf/2502.20633
• Copy Paste: [[2502.20633]] Are LLMs Ready for Practical Adoption for Assertion Generation?(https://arxiv.org/abs/2502.20633)
• Keywords: generative
• Abstract: Assertions have been the de facto collateral for simulation-based and formal verification of hardware designs for over a decade. The quality of hardware verification, i.e., detection and diagnosis of corner-case design bugs, is critically dependent on the quality of the assertions. With the onset of generative AI such as Transformers and Large-Language Models (LLMs), there has been a renewed interest in developing novel, effective, and scalable techniques of generating functional and security assertions from design source code. While there have been recent works that use commercial-of-the-shelf (COTS) LLMs for assertion generation, there is no comprehensive study in quantifying the effectiveness of LLMs in generating syntactically and semantically correct assertions. In this paper, we first discuss AssertionBench from our prior work, a comprehensive set of designs and assertions to quantify the goodness of a broad spectrum of COTS LLMs for the task of assertion generations from hardware design source code. Our key insight was that COTS LLMs are not yet ready for prime-time adoption for assertion generation as they generate a considerable fraction of syntactically and semantically incorrect assertions. Motivated by the insight, we propose AssertionLLM, a first of its kind LLM model, specifically fine-tuned for assertion generation. Our initial experimental results show that AssertionLLM considerably improves the semantic and syntactic correctness of the generated assertions over COTS LLMs.

Title: TractCloud-FOV: Deep Learning-based Robust Tractography Parcellation in Diffusion MRI with Incomplete Field of View

• Authors: Yuqian Chen, Leo Zekelman, Yui Lo, Suheyla Cetin-Karayumak, Tengfei Xue, Yogesh Rathi, Nikos Makris, Fan Zhang, Weidong Cai, Lauren J. O'Donnell
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20637
• Pdf URL: https://arxiv.org/pdf/2502.20637
• Copy Paste: [[2502.20637]] TractCloud-FOV: Deep Learning-based Robust Tractography Parcellation in Diffusion MRI with Incomplete Field of View(https://arxiv.org/abs/2502.20637)
• Keywords: diffusion
• Abstract: Tractography parcellation classifies streamlines reconstructed from diffusion MRI into anatomically defined fiber tracts for clinical and research applications. However, clinical scans often have incomplete fields of view (FOV) where brain regions are partially imaged, leading to partial or truncated fiber tracts. To address this challenge, we introduce TractCloud-FOV, a deep learning framework that robustly parcellates tractography under conditions of incomplete FOV. We propose a novel training strategy, FOV-Cut Augmentation (FOV-CA), in which we synthetically cut tractograms to simulate a spectrum of real-world inferior FOV cutoff scenarios. This data augmentation approach enriches the training set with realistic truncated streamlines, enabling the model to achieve superior generalization. We evaluate the proposed TractCloud-FOV on both synthetically cut tractography and two real-life datasets with incomplete FOV. TractCloud-FOV significantly outperforms several state-of-the-art methods on all testing datasets in terms of streamline classification accuracy, generalization ability, tract anatomical depiction, and computational efficiency. Overall, TractCloud-FOV achieves efficient and consistent tractography parcellation in diffusion MRI with incomplete FOV.

Title: Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models

• Authors: Yu Pan, Bingrong Dai, Jiahao Chen, Lin Wang, Yi Du, Jiao Liu
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.20650
• Pdf URL: https://arxiv.org/pdf/2502.20650
• Copy Paste: [[2502.20650]] Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models(https://arxiv.org/abs/2502.20650)
• Keywords: diffusion
• Abstract: In recent years, Diffusion Models (DMs) have demonstrated significant advances in the field of image generation. However, according to current research, DMs are vulnerable to backdoor attacks, which allow attackers to control the model's output by inputting data containing covert triggers, such as a specific patch or phrase. Existing defense strategies are well equipped to thwart such attacks through backdoor detection and trigger inversion because previous attack methods are constrained by limited input spaces and triggers defined by low-dimensional features. To bridge these gaps, we propose Gungnir, a novel method that enables attackers to activate the backdoor in DMs through hidden style triggers within input images. Our approach proposes using stylistic features as triggers for the first time and implements backdoor attacks successfully in image2image tasks by utilizing Reconstructing-Adversarial Noise (RAN) and Short-Term-Timesteps-Retention (STTR) of DMs. Meanwhile, experiments demonstrate that our method can easily bypass existing defense methods. Among existing DM main backdoor defense frameworks, our approach achieves a 0\% backdoor detection rate (BDR). Our codes are available at this https URL.

Title: Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA

• Authors: Ojonugwa Oluwafemi Ejiga Peter, Md Mahmudur Rahman, Fahmi Khalifa
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20667
• Pdf URL: https://arxiv.org/pdf/2502.20667
• Copy Paste: [[2502.20667]] Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA(https://arxiv.org/abs/2502.20667)
• Keywords: diffusion, generative
• Abstract: The MEDVQA-GI challenge addresses the integration of AI-driven text-to-image generative models in medical diagnostics, aiming to enhance diagnostic capabilities through synthetic image generation. Existing methods primarily focus on static image analysis and lack the dynamic generation of medical imagery from textual descriptions. This study intends to partially close this gap by introducing a novel approach based on fine-tuned generative models to generate dynamic, scalable, and precise images from textual descriptions. Particularly, our system integrates fine-tuned Stable Diffusion and DreamBooth models, as well as Low-Rank Adaptation (LORA), to generate high-fidelity medical images. The problem is around two sub-tasks namely: image synthesis (IS) and optimal prompt production (OPG). The former creates medical images via verbal prompts, whereas the latter provides prompts that produce high-quality images in specified categories. The study emphasizes the limitations of traditional medical image generation methods, such as hand sketching, constrained datasets, static procedures, and generic models. Our evaluation measures showed that Stable Diffusion surpasses CLIP and DreamBooth + LORA in terms of producing high-quality, diversified images. Specifically, Stable Diffusion had the lowest Fréchet Inception Distance (FID) scores (0.099 for single center, 0.064 for multi-center, and 0.067 for combined), indicating higher image quality. Furthermore, it had the highest average Inception Score (2.327 across all datasets), indicating exceptional diversity and quality. This advances the field of AI-powered medical diagnosis. Future research will concentrate on model refining, dataset augmentation, and ethical considerations for efficiently implementing these advances into clinical practice

Title: SciceVPR: Stable Cross-Image Correlation Enhanced Model for Visual Place Recognition

• Authors: Shanshan Wan, Yingmei Wei, Lai Kang, Tianrui Shen, Haixuan Wang, Yee-Hong Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20676
• Pdf URL: https://arxiv.org/pdf/2502.20676
• Copy Paste: [[2502.20676]] SciceVPR: Stable Cross-Image Correlation Enhanced Model for Visual Place Recognition(https://arxiv.org/abs/2502.20676)
• Keywords: foundation model
• Abstract: Visual Place Recognition (VPR) is a major challenge for robotics and autonomous systems, with the goal of predicting the location of an image based solely on its visual features. State-of-the-art (SOTA) models extract global descriptors using the powerful foundation model DINOv2 as backbone. These models either explore the cross-image correlation or propose a time-consuming two-stage re-ranking strategy to achieve better performance. However, existing works only utilize the final output of DINOv2, and the current cross-image correlation causes unstable retrieval results. To produce both discriminative and constant global descriptors, this paper proposes stable cross-image correlation enhanced model for VPR called SciceVPR. This model explores the full potential of DINOv2 in providing useful feature representations that implicitly encode valuable contextual knowledge. Specifically, SciceVPR first uses a multi-layer feature fusion module to capture increasingly detailed task-relevant channel and spatial information from the multi-layer output of DINOv2. Secondly, SciceVPR considers the invariant correlation between images within a batch as valuable knowledge to be distilled into the proposed self-enhanced encoder. In this way, SciceVPR can acquire fairly robust global features regardless of domain shifts (e.g., changes in illumination, weather and viewpoint between pictures taken in the same place). Experimental results demonstrate that the base variant, SciceVPR-B, outperforms SOTA one-stage methods with single input on multiple datasets with varying domain conditions. The large variant, SciceVPR-L, performs on par with SOTA two-stage models, scoring over 3% higher in Recall@1 compared to existing models on the challenging Tokyo24/7 dataset. Our code will be released at this https URL.

Title: STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding

• Authors: Aaryan Garg, Akash Kumar, Yogesh S Rawat
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20678
• Pdf URL: https://arxiv.org/pdf/2502.20678
• Copy Paste: [[2502.20678]] STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding(https://arxiv.org/abs/2502.20678)
• Keywords: foundation model
• Abstract: In this work we study Weakly Supervised Spatio-Temporal Video Grounding (WSTVG), a challenging task of localizing subjects spatio-temporally in videos using only textual queries and no bounding box supervision. Inspired by recent advances in vision-language foundation models, we investigate their utility for WSTVG, leveraging their zero-shot grounding capabilities. However, we find that a simple adaptation lacks essential spatio-temporal grounding abilities. To bridge this gap, we introduce Tubelet Referral Grounding (TRG), which connects textual queries to tubelets to enable spatio-temporal predictions. Despite its promise, TRG struggles with compositional action understanding and dense scene scenarios. To address these limitations, we propose STPro, a novel progressive learning framework with two key modules: (1) Sub-Action Temporal Curriculum Learning (SA-TCL), which incrementally builds compositional action understanding, and (2) Congestion-Guided Spatial Curriculum Learning (CG-SCL), which adapts the model to complex scenes by spatially increasing task difficulty. STPro achieves state-of-the-art results on three benchmark datasets, with improvements of 1.0% on VidSTG-Declarative and 3.0% on HCSTVG-v1.

Title: Diffusion Restoration Adapter for Real-World Image Restoration

• Authors: Hanbang Liang, Zhen Wang, Weihui Deng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20679
• Pdf URL: https://arxiv.org/pdf/2502.20679
• Copy Paste: [[2502.20679]] Diffusion Restoration Adapter for Real-World Image Restoration(https://arxiv.org/abs/2502.20679)
• Keywords: diffusion, generative
• Abstract: Diffusion models have demonstrated their powerful image generation capabilities, effectively fitting highly complex image distributions. These models can serve as strong priors for image restoration. Existing methods often utilize techniques like ControlNet to sample high quality images with low quality images from these priors. However, ControlNet typically involves copying a large part of the original network, resulting in a significantly large number of parameters as the prior scales up. In this paper, we propose a relatively lightweight Adapter that leverages the powerful generative capabilities of pretrained priors to achieve photo-realistic image restoration. The Adapters can be adapt to both denoising UNet and DiT, and performs excellent.

Title: Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformers

• Authors: Zixuan Gong, Jiaye Teng, Yong Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20681
• Pdf URL: https://arxiv.org/pdf/2502.20681
• Copy Paste: [[2502.20681]] Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformers(https://arxiv.org/abs/2502.20681)
• Keywords: in-context
• Abstract: Transformers may exhibit two-stage training dynamics during the real-world training process. For instance, when training GPT-2 on the Counterfact dataset, the answers progress from syntactically incorrect to syntactically correct to semantically correct. However, existing theoretical analyses hardly account for this two-stage phenomenon. In this paper, we theoretically demonstrate how such two-stage training dynamics occur in transformers. Specifically, we analyze the dynamics of transformers using feature learning techniques under in-context learning regimes, based on a disentangled two-type feature structure. Such disentanglement of feature structure is general in practice, e.g., natural languages contain syntax and semantics, and proteins contain primary and secondary structures. To our best known, this is the first rigorous result regarding a two-stage optimization process in transformers. Additionally, a corollary indicates that such a two-stage process is closely related to the spectral properties of the attention weights, which accords well with empirical findings.

Title: Glioma Classification using Multi-sequence MRI and Novel Wavelets-based Feature Fusion

• Authors: Kiranmayee Janardhan, Christy Bobby Thomas
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20715
• Pdf URL: https://arxiv.org/pdf/2502.20715
• Copy Paste: [[2502.20715]] Glioma Classification using Multi-sequence MRI and Novel Wavelets-based Feature Fusion(https://arxiv.org/abs/2502.20715)
• Keywords: diffusion
• Abstract: Glioma, a prevalent and heterogeneous tumor originating from the glial cells, can be differentiated as Low Grade Glioma (LGG) and High Grade Glioma (HGG) according to World Health Organization's norms. Classifying gliomas is essential for treatment protocols that depend extensively on subtype differentiation. For non-invasive glioma evaluation, Magnetic Resonance Imaging (MRI) offers vital information about the morphology and location of the the tumor. The versatility of MRI allows the classification of gliomas as LGG and HGG based on their texture, perfusion, and diffusion characteristics, and further for improving the diagnosis and providing tailored treatments. Nevertheless, the precise classification is complicated by tumor heterogeneity and overlapping radiomic characteristics. Thus, in this work, wavelet based novel fusion algorithm were implemented on multi-sequence T1, T1-contrast enhanced (T1CE), T2 and Fluid Attenuated Inversion Recovery (FLAIR) MRI images to compute the radiomics features. Furthermore, principal component analysis is applied to reduce the feature space and XGBoost, Support Vector Machine, and Random Forest Classifier are used for the classification. The result shows that the SVM algorithm performs comparatively well with an accuracy of 90.17%, precision of 91.04% and recall of 96.19%, F1-score of 93.53%, and AUC of 94.60% when implemented on BraTS 2018 dataset and with an accuracy of 91.34%, precision of 93.05% and recall of 96.13%, F1-score of 94.53%, and AUC of 93.71% for BraTS 2018 dataset. Thus, the proposed algorithm could be potentially implemented for the computer-aided diagnosis and grading system for gliomas.

Title: Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer

• Authors: Guanglin Zhou, Sebastiano Barbieri
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20719
• Pdf URL: https://arxiv.org/pdf/2502.20719
• Copy Paste: [[2502.20719]] Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer(https://arxiv.org/abs/2502.20719)
• Keywords: generative
• Abstract: Generating realistic synthetic electronic health records (EHRs) holds tremendous promise for accelerating healthcare research, facilitating AI model development and enhancing patient privacy. However, existing generative methods typically treat EHRs as flat sequences of discrete medical codes. This approach overlooks two critical aspects: the inherent hierarchical organization of clinical coding systems and the rich semantic context provided by code descriptions. Consequently, synthetic patient sequences often lack high clinical fidelity and have limited utility in downstream clinical tasks. In this paper, we propose the Hierarchy- and Semantics-Guided Transformer (HiSGT), a novel framework that leverages both hierarchical and semantic information for the generative process. HiSGT constructs a hierarchical graph to encode parent-child and sibling relationships among clinical codes and employs a graph neural network to derive hierarchy-aware embeddings. These are then fused with semantic embeddings extracted from a pre-trained clinical language model (e.g., ClinicalBERT), enabling the Transformer-based generator to more accurately model the nuanced clinical patterns inherent in real EHRs. Extensive experiments on the MIMIC-III and MIMIC-IV datasets demonstrate that HiSGT significantly improves the statistical alignment of synthetic data with real patient records, as well as supports robust downstream applications such as chronic disease classification. By addressing the limitations of conventional raw code-based generative models, HiSGT represents a significant step toward clinically high-fidelity synthetic data generation and a general paradigm suitable for interpretable medical code representation, offering valuable applications in data augmentation and privacy-preserving healthcare analytics.

Title: CADDreamer: CAD object Generation from Single-view Images

• Authors: Yuan Li, Cheng Lin, Yuan Liu, Xiaoxiao Long, Chenxu Zhang, Ningna Wang, Xin Li, Wenping Wang, Xiaohu Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20732
• Pdf URL: https://arxiv.org/pdf/2502.20732
• Copy Paste: [[2502.20732]] CADDreamer: CAD object Generation from Single-view Images(https://arxiv.org/abs/2502.20732)
• Keywords: diffusion, generative
• Abstract: Diffusion-based 3D generation has made remarkable progress in recent years. However, existing 3D generative models often produce overly dense and unstructured meshes, which stand in stark contrast to the compact, structured, and sharply-edged Computer-Aided Design (CAD) models crafted by human designers. To address this gap, we introduce CADDreamer, a novel approach for generating boundary representations (B-rep) of CAD objects from a single image. CADDreamer employs a primitive-aware multi-view diffusion model that captures both local geometric details and high-level structural semantics during the generation process. By encoding primitive semantics into the color domain, the method leverages the strong priors of pre-trained diffusion models to align with well-defined primitives. This enables the inference of multi-view normal maps and semantic maps from a single image, facilitating the reconstruction of a mesh with primitive labels. Furthermore, we introduce geometric optimization techniques and topology-preserving extraction methods to mitigate noise and distortion in the generated primitives. These enhancements result in a complete and seamless B-rep of the CAD model. Experimental results demonstrate that our method effectively recovers high-quality CAD objects from single-view images. Compared to existing 3D generation techniques, the B-rep models produced by CADDreamer are compact in representation, clear in structure, sharp in edges, and watertight in topology.

Title: Two-Stream Spatial-Temporal Transformer Framework for Person Identification via Natural Conversational Keypoints

• Authors: Masoumeh Chapariniya, Hossein Ranjbar, Teodora Vukovic, Sarah Ebling, Volker Dellwo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20803
• Pdf URL: https://arxiv.org/pdf/2502.20803
• Copy Paste: [[2502.20803]] Two-Stream Spatial-Temporal Transformer Framework for Person Identification via Natural Conversational Keypoints(https://arxiv.org/abs/2502.20803)
• Keywords: generative
• Abstract: In the age of AI-driven generative technologies, traditional biometric recognition systems face unprecedented challenges, particularly from sophisticated deepfake and face reenactment techniques. In this study, we propose a Two-Stream Spatial-Temporal Transformer Framework for person identification using upper body keypoints visible during online conversations, which we term conversational keypoints. Our framework processes both spatial relationships between keypoints and their temporal evolution through two specialized branches: a Spatial Transformer (STR) that learns distinctive structural patterns in keypoint configurations, and a Temporal Transformer (TTR) that captures sequential motion patterns. Using the state-of-the-art Sapiens pose estimator, we extract 133 keypoints (based on COCO-WholeBody format) representing facial features, head pose, and hand positions. The framework was evaluated on a dataset of 114 individuals engaged in natural conversations, achieving recognition accuracies of 80.12% for the spatial stream, 63.61% for the temporal stream. We then explored two fusion strategies: a shared loss function approach achieving 82.22% accuracy, and a feature-level fusion method that concatenates feature maps from both streams, significantly improving performance to 94.86%. By jointly modeling both static anatomical relationships and dynamic movement patterns, our approach learns comprehensive identity signatures that are more robust to spoofing than traditional appearance-based methods.

Title: Can We Simplify Slide-level Fine-tuning of Pathology Foundation Models?

• Authors: Jiawen Li, Jiali Hu, Qiehe Sun, Renao Yan, Minxi Ouyang, Tian Guan, Anjia Han, Chao He, Yonghong He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20823
• Pdf URL: https://arxiv.org/pdf/2502.20823
• Copy Paste: [[2502.20823]] Can We Simplify Slide-level Fine-tuning of Pathology Foundation Models?(https://arxiv.org/abs/2502.20823)
• Keywords: foundation model
• Abstract: The emergence of foundation models in computational pathology has transformed histopathological image analysis, with whole slide imaging (WSI) diagnosis being a core application. Traditionally, weakly supervised fine-tuning via multiple instance learning (MIL) has been the primary method for adapting foundation models to WSIs. However, in this work we present a key experimental finding: a simple nonlinear mapping strategy combining mean pooling and a multilayer perceptron, called SiMLP, can effectively adapt patch-level foundation models to slide-level tasks without complex MIL-based learning. Through extensive experiments across diverse downstream tasks, we demonstrate the superior performance of SiMLP with state-of-the-art methods. For instance, on a large-scale pan-cancer classification task, SiMLP surpasses popular MIL-based methods by 3.52%. Furthermore, SiMLP shows strong learning ability in few-shot classification and remaining highly competitive with slide-level foundation models pretrained on tens of thousands of slides. Finally, SiMLP exhibits remarkable robustness and transferability in lung cancer subtyping. Overall, our findings challenge the conventional MIL-based fine-tuning paradigm, demonstrating that a task-agnostic representation strategy alone can effectively adapt foundation models to WSI analysis. These insights offer a unique and meaningful perspective for future research in digital pathology, paving the way for more efficient and broadly applicable methodologies.

Title: Learning to Substitute Components for Compositional Generalization

• Authors: Zhaoyi Li, Gangwei Jiang, Chenwang Wu, Ying Wei, Defu Lian, Enhong Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20834
• Pdf URL: https://arxiv.org/pdf/2502.20834
• Copy Paste: [[2502.20834]] Learning to Substitute Components for Compositional Generalization(https://arxiv.org/abs/2502.20834)
• Keywords: in-context
• Abstract: Despite the rising prevalence of neural language models, recent empirical evidence suggests their deficiency in compositional generalization. One of the current de-facto solutions to this problem is compositional data augmentation, which aims to introduce additional compositional inductive bias. However, existing handcrafted augmentation strategies offer limited improvement when systematic generalization of neural language models requires multi-grained compositional bias (i.e., not limited to either lexical or structural biases alone) or when training sentences have an imbalanced difficulty distribution. To address these challenges, we first propose a novel compositional augmentation strategy called Component Substitution (CompSub), which enables multi-grained composition of substantial substructures across the entire training set. Furthermore, we introduce the Learning Component Substitution (LCS) framework. This framework empowers the learning of component substitution probabilities in CompSub in an end-to-end manner by maximizing the loss of neural language models, thereby prioritizing challenging compositions with elusive concepts and novel contexts. We extend the key ideas of CompSub and LCS to the recently emerging in-context learning scenarios of pre-trained large language models (LLMs), proposing the LCS-ICL algorithm to enhance the few-shot compositional generalization of state-of-the-art (SOTA) LLMs. Theoretically, we provide insights into why applying our algorithms to language models can improve compositional generalization performance. Empirically, our results on four standard compositional generalization benchmarks(SCAN, COGS, GeoQuery, and COGS-QL) demonstrate the superiority of CompSub, LCS, and LCS-ICL, with improvements of up to 66.5%, 10.3%, 1.4%, and 8.8%, respectively.

Title: A Fused Gromov-Wasserstein Approach to Subgraph Contrastive Learning

• Authors: Amadou S. Sangare, Nicolas Dunou, Jhony H. Giraldo, Fragkiskos D. Malliaros
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20885
• Pdf URL: https://arxiv.org/pdf/2502.20885
• Copy Paste: [[2502.20885]] A Fused Gromov-Wasserstein Approach to Subgraph Contrastive Learning(https://arxiv.org/abs/2502.20885)
• Keywords: self-supervised
• Abstract: Self-supervised learning has become a key method for training deep learning models when labeled data is scarce or unavailable. While graph machine learning holds great promise across various domains, the design of effective pretext tasks for self-supervised graph representation learning remains challenging. Contrastive learning, a popular approach in graph self-supervised learning, leverages positive and negative pairs to compute a contrastive loss function. However, current graph contrastive learning methods often struggle to fully use structural patterns and node similarities. To address these issues, we present a new method called Fused Gromov Wasserstein Subgraph Contrastive Learning (FOSSIL). Our model integrates node-level and subgraph-level contrastive learning, seamlessly combining a standard node-level contrastive loss with the Fused Gromov-Wasserstein distance. This combination helps our method capture both node features and graph structure together. Importantly, our approach works well with both homophilic and heterophilic graphs and can dynamically create views for generating positive and negative pairs. Through extensive experiments on benchmark graph datasets, we show that FOSSIL outperforms or achieves competitive performance compared to current state-of-the-art methods.

Title: DiffBrush:Just Painting the Art by Your Hands

• Authors: Jiaming Chu, Lei Jin, Tao Wang, Junliang Xing, Jian Zhao
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.20904
• Pdf URL: https://arxiv.org/pdf/2502.20904
• Copy Paste: [[2502.20904]] DiffBrush:Just Painting the Art by Your Hands(https://arxiv.org/abs/2502.20904)
• Keywords: diffusion
• Abstract: The rapid development of image generation and editing algorithms in recent years has enabled ordinary user to produce realistic images. However, the current AI painting ecosystem predominantly relies on text-driven diffusion models (T2I), which pose challenges in accurately capturing user requirements. Furthermore, achieving compatibility with other modalities incurs substantial training costs. To this end, we introduce DiffBrush, which is compatible with T2I models and allows users to draw and edit images. By manipulating and adapting the internal representation of the diffusion model, DiffBrush guides the model-generated images to converge towards the user's hand-drawn sketches for user's specific needs without additional training. DiffBrush achieves control over the color, semantic, and instance of objects in images by continuously guiding the latent and instance-level attention map during the denoising process of the diffusion model. Besides, we propose a latent regeneration, which refines the randomly sampled noise in the diffusion model, obtaining a better image generation layout. Finally, users only need to roughly draw the mask of the instance (acceptable colors) on the canvas, DiffBrush can naturally generate the corresponding instance at the corresponding location.

Title: Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry

• Authors: Ilya Koziev
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20931
• Pdf URL: https://arxiv.org/pdf/2502.20931
• Copy Paste: [[2502.20931]] Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry(https://arxiv.org/abs/2502.20931)
• Keywords: generative
• Abstract: Generative poetry systems require effective tools for data engineering and automatic evaluation, particularly to assess how well a poem adheres to versification rules, such as the correct alternation of stressed and unstressed syllables and the presence of rhymes. In this work, we introduce the Russian Poetry Scansion Tool library designed for stress mark placement in Russian-language syllabo-tonic poetry, rhyme detection, and identification of defects of poeticness. Additionally, we release RIFMA -- a dataset of poem fragments spanning various genres and forms, annotated with stress marks. This dataset can be used to evaluate the capability of modern large language models to accurately place stress marks in poetic texts. The published resources provide valuable tools for researchers and practitioners in the field of creative generative AI, facilitating advancements in the development and evaluation of generative poetry systems.

Title: Generative Uncertainty in Diffusion Models

• Authors: Metod Jazbec, Eliot Wong-Toi, Guoxuan Xia, Dan Zhang, Eric Nalisnick, Stephan Mandt
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20946
• Pdf URL: https://arxiv.org/pdf/2502.20946
• Copy Paste: [[2502.20946]] Generative Uncertainty in Diffusion Models(https://arxiv.org/abs/2502.20946)
• Keywords: diffusion, generative
• Abstract: Diffusion models have recently driven significant breakthroughs in generative modeling. While state-of-the-art models produce high-quality samples on average, individual samples can still be low quality. Detecting such samples without human inspection remains a challenging task. To address this, we propose a Bayesian framework for estimating generative uncertainty of synthetic samples. We outline how to make Bayesian inference practical for large, modern generative models and introduce a new semantic likelihood (evaluated in the latent space of a feature extractor) to address the challenges posed by high-dimensional sample spaces. Through our experiments, we demonstrate that the proposed generative uncertainty effectively identifies poor-quality samples and significantly outperforms existing uncertainty-based methods. Notably, our Bayesian framework can be applied post-hoc to any pretrained diffusion or flow matching model (via the Laplace approximation), and we propose simple yet effective techniques to minimize its computational overhead during sampling.

Title: Distribution Prototype Diffusion Learning for Open-set Supervised Anomaly Detection

• Authors: Fuyun Wang, Tong Zhang, Yuanzhi Wang, Yide Qiu, Xin Liu, Xu Guo, Zhen Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20981
• Pdf URL: https://arxiv.org/pdf/2502.20981
• Copy Paste: [[2502.20981]] Distribution Prototype Diffusion Learning for Open-set Supervised Anomaly Detection(https://arxiv.org/abs/2502.20981)
• Keywords: diffusion, anomaly
• Abstract: In Open-set Supervised Anomaly Detection (OSAD), the existing methods typically generate pseudo anomalies to compensate for the scarcity of observed anomaly samples, while overlooking critical priors of normal samples, leading to less effective discriminative boundaries. To address this issue, we propose a Distribution Prototype Diffusion Learning (DPDL) method aimed at enclosing normal samples within a compact and discriminative distribution space. Specifically, we construct multiple learnable Gaussian prototypes to create a latent representation space for abundant and diverse normal samples and learn a Schrödinger bridge to facilitate a diffusive transition toward these prototypes for normal samples while steering anomaly samples away. Moreover, to enhance inter-sample separation, we design a dispersion feature learning way in hyperspherical space, which benefits the identification of out-of-distribution anomalies. Experimental results demonstrate the effectiveness and superiority of our proposed DPDL, achieving state-of-the-art performance on 9 public datasets.

Title: UoR-NCL at SemEval-2025 Task 1: Using Generative LLMs and CLIP Models for Multilingual Multimodal Idiomaticity Representation

• Authors: Thanet Markchom, Tong Wu, Liting Huang, Huizhi Liang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20984
• Pdf URL: https://arxiv.org/pdf/2502.20984
• Copy Paste: [[2502.20984]] UoR-NCL at SemEval-2025 Task 1: Using Generative LLMs and CLIP Models for Multilingual Multimodal Idiomaticity Representation(https://arxiv.org/abs/2502.20984)
• Keywords: generative
• Abstract: SemEval-2025 Task 1 focuses on ranking images based on their alignment with a given nominal compound that may carry idiomatic meaning in both English and Brazilian Portuguese. To address this challenge, this work uses generative large language models (LLMs) and multilingual CLIP models to enhance idiomatic compound representations. LLMs generate idiomatic meanings for potentially idiomatic compounds, enriching their semantic interpretation. These meanings are then encoded using multilingual CLIP models, serving as representations for image ranking. Contrastive learning and data augmentation techniques are applied to fine-tune these embeddings for improved performance. Experimental results show that multimodal representations extracted through this method outperformed those based solely on the original nominal compounds. The fine-tuning approach shows promising outcomes but is less effective than using embeddings without fine-tuning. The source code used in this paper is available at this https URL.

Title: Soften the Mask: Adaptive Temporal Soft Mask for Efficient Dynamic Facial Expression Recognition

• Authors: Mengzhu Li, Quanxing Zha, Hongjun Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21004
• Pdf URL: https://arxiv.org/pdf/2502.21004
• Copy Paste: [[2502.21004]] Soften the Mask: Adaptive Temporal Soft Mask for Efficient Dynamic Facial Expression Recognition(https://arxiv.org/abs/2502.21004)
• Keywords: self-supervised
• Abstract: Dynamic Facial Expression Recognition (DFER) facilitates the understanding of psychological intentions through non-verbal communication. Existing methods struggle to manage irrelevant information, such as background noise and redundant semantics, which impacts both efficiency and effectiveness. In this work, we propose a novel supervised temporal soft masked autoencoder network for DFER, namely AdaTosk, which integrates a parallel supervised classification branch with the self-supervised reconstruction branch. The self-supervised reconstruction branch applies random binary hard mask to generate diverse training samples, encouraging meaningful feature representations in visible tokens. Meanwhile the classification branch employs an adaptive temporal soft mask to flexibly mask visible tokens based on their temporal significance. Its two key components, respectively of, class-agnostic and class-semantic soft masks, serve to enhance critical expression moments and reduce semantic redundancy over time. Extensive experiments conducted on widely-used benchmarks demonstrate that our AdaTosk remarkably reduces computational costs compared with current state-of-the-art methods while still maintaining competitive performance.

Title: When Unsupervised Domain Adaptation meets One-class Anomaly Detection: Addressing the Two-fold Unsupervised Curse by Leveraging Anomaly Scarcity

• Authors: Nesryne Mejri, Enjie Ghorbel, Anis Kacem, Pavel Chernakov, Niki Foteinopoulou, Djamila Aouada
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21022
• Pdf URL: https://arxiv.org/pdf/2502.21022
• Copy Paste: [[2502.21022]] When Unsupervised Domain Adaptation meets One-class Anomaly Detection: Addressing the Two-fold Unsupervised Curse by Leveraging Anomaly Scarcity(https://arxiv.org/abs/2502.21022)
• Keywords: anomaly
• Abstract: This paper introduces the first fully unsupervised domain adaptation (UDA) framework for unsupervised anomaly detection (UAD). The performance of UAD techniques degrades significantly in the presence of a domain shift, difficult to avoid in a real-world setting. While UDA has contributed to solving this issue in binary and multi-class classification, such a strategy is ill-posed in UAD. This might be explained by the unsupervised nature of the two tasks, namely, domain adaptation and anomaly detection. Herein, we first formulate this problem that we call the two-fold unsupervised curse. Then, we propose a pioneering solution to this curse, considered intractable so far, by assuming that anomalies are rare. Specifically, we leverage clustering techniques to identify a dominant cluster in the target feature space. Posed as the normal cluster, the latter is aligned with the source normal features. Concretely, given a one-class source set and an unlabeled target set composed mostly of normal data and some anomalies, we fit the source features within a hypersphere while jointly aligning them with the features of the dominant cluster from the target set. The paper provides extensive experiments and analysis on common adaptation benchmarks for anomaly detection, demonstrating the relevance of both the newly introduced paradigm and the proposed approach. The code will be made publicly available.

Title: Synthesizing Tabular Data Using Selectivity Enhanced Generative Adversarial Networks

• Authors: Youran Zhou, Jianzhong Qi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.21034
• Pdf URL: https://arxiv.org/pdf/2502.21034
• Copy Paste: [[2502.21034]] Synthesizing Tabular Data Using Selectivity Enhanced Generative Adversarial Networks(https://arxiv.org/abs/2502.21034)
• Keywords: generative
• Abstract: As E-commerce platforms face surging transactions during major shopping events like Black Friday, stress testing with synthesized data is crucial for resource planning. Most recent studies use Generative Adversarial Networks (GANs) to generate tabular data while ensuring privacy and machine learning utility. However, these methods overlook the computational demands of processing GAN-generated data, making them unsuitable for E-commerce stress testing. This thesis introduces a novel GAN-based approach incorporating query selectivity constraints, a key factor in database transaction processing. We integrate a pre-trained deep neural network to maintain selectivity consistency between real and synthetic data. Our method, tested on five real-world datasets, outperforms three state-of-the-art GANs and a VAE model, improving selectivity estimation accuracy by up to 20pct and machine learning utility by up to 6 pct.

Title: Synthesizing Individualized Aging Brains in Health and Disease with Generative Models and Parallel Transport

• Authors: Jingru Fu, Yuqi Zheng, Neel Dey, Daniel Ferreira, Rodrigo Moreno
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.21049
• Pdf URL: https://arxiv.org/pdf/2502.21049
• Copy Paste: [[2502.21049]] Synthesizing Individualized Aging Brains in Health and Disease with Generative Models and Parallel Transport(https://arxiv.org/abs/2502.21049)
• Keywords: generative
• Abstract: Simulating prospective magnetic resonance imaging (MRI) scans from a given individual brain image is challenging, as it requires accounting for canonical changes in aging and/or disease progression while also considering the individual brain's current status and unique characteristics. While current deep generative models can produce high-resolution anatomically accurate templates for population-wide studies, their ability to predict future aging trajectories for individuals remains limited, particularly in capturing subject-specific neuroanatomical variations over time. In this study, we introduce Individualized Brain Synthesis (InBrainSyn), a framework for synthesizing high-resolution subject-specific longitudinal MRI scans that simulate neurodegeneration in both Alzheimer's disease (AD) and normal aging. InBrainSyn uses a parallel transport algorithm to adapt the population-level aging trajectories learned by a generative deep template network, enabling individualized aging synthesis. As InBrainSyn uses diffeomorphic transformations to simulate aging, the synthesized images are topologically consistent with the original anatomy by design. We evaluated InBrainSyn both quantitatively and qualitatively on AD and healthy control cohorts from the Open Access Series of Imaging Studies - version 3 dataset. Experimentally, InBrainSyn can also model neuroanatomical transitions between normal aging and AD. An evaluation of an external set supports its generalizability. Overall, with only a single baseline scan, InBrainSyn synthesizes realistic 3D spatiotemporal T1w MRI scans, producing personalized longitudinal aging trajectories. The code for InBrainSyn is available at: this https URL.

Title: Detection of anomalies in cow activity using wavelet transform based features

• Authors: Valentin Guien, Violaine Antoine, Romain Lardy, Isabelle Veissier, Luis E C Rocha
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.21051
• Pdf URL: https://arxiv.org/pdf/2502.21051
• Copy Paste: [[2502.21051]] Detection of anomalies in cow activity using wavelet transform based features(https://arxiv.org/abs/2502.21051)
• Keywords: anomaly
• Abstract: In Precision Livestock Farming, detecting deviations from optimal or baseline values - i.e. anomalies in time series - is essential to allow undertaking corrective actions rapidly. Here we aim at detecting anomalies in 24h time series of cow activity, with a view to detect cases of disease or oestrus. Deviations must be distinguished from noise which can be very high in case of biological data. It is also important to detect the anomaly early, e.g. before a farmer would notice it visually. Here, we investigate the benefit of using wavelet transforms to denoise data and we assess the performance of an anomaly detection algorithm considering the timing of the detection. We developed features based on the comparisons between the wavelet transforms of the mean of the time series and the wavelet transforms of individual time series instances. We hypothesized that these features contribute to the detection of anomalies in periodic time series using a feature-based algorithm. We tested this hypothesis with two datasets representing cow activity, which typically follows a daily pattern but can deviate due to specific physiological or pathological conditions. We applied features derived from wavelet transform as well as statistical features in an Isolation Forest algorithm. We measured the distance of detection between the days annotated abnormal by animal caretakers days and the days predicted abnormal by the algorithm. The results show that wavelet-based features are among the features most contributing to anomaly detection. They also show that detections are close to the annotated days, and often precede it. In conclusion, using wavelet transforms on time series of cow activity data helps to detect anomalies related to specific cow states. The detection is often obtained on days that precede the day annotated by caretakers, which offer possibility to take corrective actions at an early stage.

Title: Spatial Reasoning with Denoising Models

• Authors: Christopher Wewer, Bart Pogodzinski, Bernt Schiele, Jan Eric Lenssen
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.21075
• Pdf URL: https://arxiv.org/pdf/2502.21075
• Copy Paste: [[2502.21075]] Spatial Reasoning with Denoising Models(https://arxiv.org/abs/2502.21075)
• Keywords: diffusion, generative
• Abstract: We introduce Spatial Reasoning Models (SRMs), a framework to perform reasoning over sets of continuous variables via denoising generative models. SRMs infer continuous representations on a set of unobserved variables, given observations on observed variables. Current generative models on spatial domains, such as diffusion and flow matching models, often collapse to hallucination in case of complex distributions. To measure this, we introduce a set of benchmark tasks that test the quality of complex reasoning in generative models and can quantify hallucination. The SRM framework allows to report key findings about importance of sequentialization in generation, the associated order, as well as the sampling strategies during training. It demonstrates, for the first time, that order of generation can successfully be predicted by the denoising network itself. Using these findings, we can increase the accuracy of specific reasoning tasks from <1% to >50%.

Title: Training-free and Adaptive Sparse Attention for Efficient Long Video Generation

• Authors: Yifei Xia, Suhan Ling, Fangcheng Fu, Yujie Wang, Huixia Li, Xuefeng Xiao, Bin Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21079
• Pdf URL: https://arxiv.org/pdf/2502.21079
• Copy Paste: [[2502.21079]] Training-free and Adaptive Sparse Attention for Efficient Long Video Generation(https://arxiv.org/abs/2502.21079)
• Keywords: diffusion
• Abstract: Generating high-fidelity long videos with Diffusion Transformers (DiTs) is often hindered by significant latency, primarily due to the computational demands of attention mechanisms. For instance, generating an 8-second 720p video (110K tokens) with HunyuanVideo takes about 600 PFLOPs, with around 500 PFLOPs consumed by attention computations. To address this issue, we propose AdaSpa, the first Dynamic Pattern and Online Precise Search sparse attention method. Firstly, to realize the Dynamic Pattern, we introduce a blockified pattern to efficiently capture the hierarchical sparsity inherent in DiTs. This is based on our observation that sparse characteristics of DiTs exhibit hierarchical and blockified structures between and within different modalities. This blockified approach significantly reduces the complexity of attention computation while maintaining high fidelity in the generated videos. Secondly, to enable Online Precise Search, we propose the Fused LSE-Cached Search with Head-adaptive Hierarchical Block Sparse Attention. This method is motivated by our finding that DiTs' sparse pattern and LSE vary w.r.t. inputs, layers, and heads, but remain invariant across denoising steps. By leveraging this invariance across denoising steps, it adapts to the dynamic nature of DiTs and allows for precise, real-time identification of sparse indices with minimal overhead. AdaSpa is implemented as an adaptive, plug-and-play solution and can be integrated seamlessly with existing DiTs, requiring neither additional fine-tuning nor a dataset-dependent profiling. Extensive experiments validate that AdaSpa delivers substantial acceleration across various models while preserving video quality, establishing itself as a robust and scalable approach to efficient video generation.

Title: Rare event modeling with self-regularized normalizing flows: what can we learn from a single failure?

• Authors: Charles Dawson, Van Tran, Max Z. Li, Chuchu Fan
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.21110
• Pdf URL: https://arxiv.org/pdf/2502.21110
• Copy Paste: [[2502.21110]] Rare event modeling with self-regularized normalizing flows: what can we learn from a single failure?(https://arxiv.org/abs/2502.21110)
• Keywords: generative
• Abstract: Increased deployment of autonomous systems in fields like transportation and robotics have seen a corresponding increase in safety-critical failures. These failures can be difficult to model and debug due to the relative lack of data: compared to tens of thousands of examples from normal operations, we may have only seconds of data leading up to the failure. This scarcity makes it challenging to train generative models of rare failure events, as existing methods risk either overfitting to noise in the limited failure dataset or underfitting due to an overly strong prior. We address this challenge with CalNF, or calibrated normalizing flows, a self-regularized framework for posterior learning from limited data. CalNF achieves state-of-the-art performance on data-limited failure modeling and inverse problems and enables a first-of-a-kind case study into the root causes of the 2022 Southwest Airlines scheduling crisis.

Title: Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models

• Authors: Ruta Binkyte, Ivaxi Sheth, Zhijing Jin, Muhammad Havaei, Bernhardt Schölkopf, Mario Fritz
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.21123
• Pdf URL: https://arxiv.org/pdf/2502.21123
• Copy Paste: [[2502.21123]] Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models(https://arxiv.org/abs/2502.21123)
• Keywords: foundation model
• Abstract: Ensuring trustworthiness in machine learning (ML) systems is crucial as they become increasingly embedded in high-stakes domains. This paper advocates for the integration of causal methods into machine learning to navigate the trade-offs among key principles of trustworthy ML, including fairness, privacy, robustness, accuracy, and explainability. While these objectives should ideally be satisfied simultaneously, they are often addressed in isolation, leading to conflicts and suboptimal solutions. Drawing on existing applications of causality in ML that successfully align goals such as fairness and accuracy or privacy and robustness, this paper argues that a causal approach is essential for balancing multiple competing objectives in both trustworthy ML and foundation models. Beyond highlighting these trade-offs, we examine how causality can be practically integrated into ML and foundation models, offering solutions to enhance their reliability and interpretability. Finally, we discuss the challenges, limitations, and opportunities in adopting causal frameworks, paving the way for more accountable and ethically sound AI systems.

Title: A Review on Generative AI For Text-To-Image and Image-To-Image Generation and Implications To Scientific Images

• Authors: Zineb Sordo, Eric Chagnon, Daniela Ushizima
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21151
• Pdf URL: https://arxiv.org/pdf/2502.21151
• Copy Paste: [[2502.21151]] A Review on Generative AI For Text-To-Image and Image-To-Image Generation and Implications To Scientific Images(https://arxiv.org/abs/2502.21151)
• Keywords: diffusion, generative
• Abstract: This review surveys the state-of-the-art in text-to-image and image-to-image generation within the scope of generative AI. We provide a comparative analysis of three prominent architectures: Variational Autoencoders, Generative Adversarial Networks and Diffusion Models. For each, we elucidate core concepts, architectural innovations, and practical strengths and limitations, particularly for scientific image understanding. Finally, we discuss critical open challenges and potential future research directions in this rapidly evolving field.

Title: Parallel-Learning of Invariant and Tempo-variant Attributes of Single-Lead Cardiac Signals: PLITA

• Authors: Adtian Atienza, Jakob E. Bardram, Sadasivan Puthusserypady
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.21162
• Pdf URL: https://arxiv.org/pdf/2502.21162
• Copy Paste: [[2502.21162]] Parallel-Learning of Invariant and Tempo-variant Attributes of Single-Lead Cardiac Signals: PLITA(https://arxiv.org/abs/2502.21162)
• Keywords: self-supervised
• Abstract: Wearable sensing devices, such as Holter monitors, will play a crucial role in the future of digital health. Unsupervised learning frameworks such as Self-Supervised Learning (SSL) are essential to map these single-lead electrocardiogram (ECG) signals with their anticipated clinical outcomes. These signals are characterized by a tempo-variant component whose patterns evolve through the recording and an invariant component with patterns that remain unchanged. However, existing SSL methods only drive the model to encode the invariant attributes, leading the model to neglect tempo-variant information which reflects subject-state changes through time. In this paper, we present Parallel-Learning of Invariant and Tempo-variant Attributes (PLITA), a novel SSL method designed for capturing both invariant and tempo-variant ECG attributes. The latter are captured by mandating closer representations in space for closer inputs on time. We evaluate both the capability of the method to learn the attributes of these two distinct kinds, as well as PLITA's performance compared to existing SSL methods for ECG analysis. PLITA performs significantly better in the set-ups where tempo-variant attributes play a major role.

Title: SYN-LUNGS: Towards Simulating Lung Nodules with Anatomy-Informed Digital Twins for AI Training

• Authors: Fakrul Islam Tushar, Lavsen Dahal, Cindy McCabe, Fong Chi Ho, Paul Segars, Ehsan Abadi, Kyle J. Lafata, Ehsan Samei, Joseph Y. Lo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.21187
• Pdf URL: https://arxiv.org/pdf/2502.21187
• Copy Paste: [[2502.21187]] SYN-LUNGS: Towards Simulating Lung Nodules with Anatomy-Informed Digital Twins for AI Training(https://arxiv.org/abs/2502.21187)
• Keywords: generative
• Abstract: AI models for lung cancer screening are limited by data scarcity, impacting generalizability and clinical applicability. Generative models address this issue but are constrained by training data variability. We introduce SYN-LUNGS, a framework for generating high-quality 3D CT images with detailed annotations. SYN-LUNGS integrates XCAT3 phantoms for digital twin generation, X-Lesions for nodule simulation (varying size, location, and appearance), and DukeSim for CT image formation with vendor and parameter variability. The dataset includes 3,072 nodule images from 1,044 simulated CT scans, with 512 lesions and 174 digital twins. Models trained on clinical + simulated data outperform clinical only models, achieving 10% improvement in detection, 2-9% in segmentation and classification, and enhanced this http URL incorporating anatomy-informed simulations, SYN-LUNGS provides a scalable approach for AI model development, particularly in rare disease representation and improving model reliability.

Title: Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought

• Authors: Jianhao Huang, Zixuan Wang, Jason D. Lee
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.21212
• Pdf URL: https://arxiv.org/pdf/2502.21212
• Copy Paste: [[2502.21212]] Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought(https://arxiv.org/abs/2502.21212)
• Keywords: in-context
• Abstract: Chain of Thought (CoT) prompting has been shown to significantly improve the performance of large language models (LLMs), particularly in arithmetic and reasoning tasks, by instructing the model to produce intermediate reasoning steps. Despite the remarkable empirical success of CoT and its theoretical advantages in enhancing expressivity, the mechanisms underlying CoT training remain largely unexplored. In this paper, we study the training dynamics of transformers over a CoT objective on an in-context weight prediction task for linear regression. We prove that while a one-layer linear transformer without CoT can only implement a single step of gradient descent (GD) and fails to recover the ground-truth weight vector, a transformer with CoT prompting can learn to perform multi-step GD autoregressively, achieving near-exact recovery. Furthermore, we show that the trained transformer effectively generalizes on the unseen data. With our technique, we also show that looped transformers significantly improve final performance compared to transformers without looping in the in-context learning of linear regression. Empirically, we demonstrate that CoT prompting yields substantial performance improvements.

Title: TimesBERT: A BERT-Style Foundation Model for Time Series Understanding

• Authors: Haoran Zhang, Yong Liu, Yunzhong Qiu, Haixuan Liu, Zhongyi Pei, Jianmin Wang, Mingsheng Long
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.21245
• Pdf URL: https://arxiv.org/pdf/2502.21245
• Copy Paste: [[2502.21245]] TimesBERT: A BERT-Style Foundation Model for Time Series Understanding(https://arxiv.org/abs/2502.21245)
• Keywords: foundation model, anomaly
• Abstract: Time series analysis is crucial in diverse scenarios. Beyond forecasting, considerable real-world tasks are categorized into classification, imputation, and anomaly detection, underscoring different capabilities termed time series understanding in this paper. While GPT-style models have been positioned as foundation models for time series forecasting, the BERT-style architecture, which has made significant advances in natural language understanding, has not been fully unlocked for time series understanding, possibly attributed to the undesirable dropout of essential elements of BERT. In this paper, inspired by the shared multi-granularity structure between multivariate time series and multisentence documents, we design TimesBERT to learn generic representations of time series including temporal patterns and variate-centric characteristics. In addition to a natural adaptation of masked modeling, we propose a parallel task of functional token prediction to embody vital multi-granularity structures. Our model is pre-trained on 260 billion time points across diverse domains. Leveraging multi-granularity representations, TimesBERT achieves state-of-the-art performance across four typical downstream understanding tasks, outperforming task-specific models and language pre-trained backbones, positioning it as a versatile foundation model for time series understanding.

Title: Foundation Models -- A Panacea for Artificial Intelligence in Pathology?

• Authors: Nita Mulliqi (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Anders Blilie (Department of Pathology, Stavanger University Hospital, Stavanger, Norway and Faculty of Health Sciences, University of Stavanger, Stavanger, Norway), Xiaoyi Ji (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Kelvin Szolnoky (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Henrik Olsson (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Sol Erika Boman (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden and Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden), Matteo Titus (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Geraldine Martinez Gonzalez (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Julia Anna Mielcarz (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Masi Valkonen (Institute of Biomedicine, University of Turku, Turku, Finland), Einar Gudlaugsson (Department of Pathology, Stavanger University Hospital, Stavanger, Norway), Svein R. Kjosavik (The General Practice and Care Coordination Research Group, Stavanger University Hospital, Norway and Department of Global Public Health and Primary Care, Faculty of Medicine, University of Bergen, Norway), José Asenjo (Department of Pathology, Synlab, Madrid, Spain), Marcello Gambacorta (Department of Pathology, Synlab, Brescia, Italy), Paolo Libretti (Department of Pathology, Synlab, Brescia, Italy), Marcin Braun (Department of Pathology, Chair of Oncology, Medical University of Lodz, Lodz, Poland), Radzislaw Kordek (Department of Pathology, Chair of Oncology, Medical University of Lodz, Lodz, Poland), Roman Łowicki (1st Department of Urology, Medical University of Lodz, Lodz, Poland), Kristina Hotakainen (Department of Clinical Chemistry and Hematology, University of Helsinki, Helsinki, Finland and Laboratory Services, Mehiläinen Oy, Helsinki, Finland), Päivi Väre (Department of Pathology, Mehiläinen Länsi-Pohja Hospital, Kemi, Finland), Bodil Ginnerup Pedersen (Department of Radiology, Aarhus University Hospital, Aarhus, Denmark and Department of Clinical Medicine, Aarhus University, Aarhus, Denmark), Karina Dalsgaard Sørensen (Department of Clinical Medicine, Aarhus University, Aarhus, Denmark and Department of Molecular Medicine, Aarhus University Hospital, Aarhus, Denmark), Benedicte Parm Ulhøi (Department of Pathology, Aarhus University Hospital, Aarhus, Denmark), Pekka Ruusuvuori (Institute of Biomedicine, University of Turku, Turku, Finland and InFLAMES Research Flagship, University of Turku, Turku, Finland and Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland), Brett Delahunt (Malaghan Institute of Medical Research, Wellington, New Zealand and Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden), Hemamali Samaratunga (Aquesta Uropathology and University of Queensland, QLD, Brisbane, Australia), Toyonori Tsuzuki (Department of Surgical Pathology, School of Medicine, Aichi Medical University, Nagoya, Japan), Emilius A.M. Janssen (Department of Pathology, Stavanger University Hospital, Stavanger, Norway and Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Stavanger, Norway and Institute for Biomedicine and Glycomics, Griffith University, Queensland, Australia), Lars Egevad (Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden), Martin Eklund (Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden), Kimmo Kartasalo (Department of Medical Epidemiology and Biostatistics, SciLifeLab, Karolinska Institutet, Stockholm, Sweden)
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.21264
• Pdf URL: https://arxiv.org/pdf/2502.21264
• Copy Paste: [[2502.21264]] Foundation Models -- A Panacea for Artificial Intelligence in Pathology?(https://arxiv.org/abs/2502.21264)
• Keywords: self-supervised, foundation model
• Abstract: The role of artificial intelligence (AI) in pathology has evolved from aiding diagnostics to uncovering predictive morphological patterns in whole slide images (WSIs). Recently, foundation models (FMs) leveraging self-supervised pre-training have been widely advocated as a universal solution for diverse downstream tasks. However, open questions remain about their clinical applicability and generalization advantages over end-to-end learning using task-specific (TS) models. Here, we focused on AI with clinical-grade performance for prostate cancer diagnosis and Gleason grading. We present the largest validation of AI for this task, using over 100,000 core needle biopsies from 7,342 patients across 15 sites in 11 countries. We compared two FMs with a fully end-to-end TS model in a multiple instance learning framework. Our findings challenge assumptions that FMs universally outperform TS models. While FMs demonstrated utility in data-scarce scenarios, their performance converged with - and was in some cases surpassed by - TS models when sufficient labeled training data were available. Notably, extensive task-specific training markedly reduced clinically significant misgrading, misdiagnosis of challenging morphologies, and variability across different WSI scanners. Additionally, FMs used up to 35 times more energy than the TS model, raising concerns about their sustainability. Our results underscore that while FMs offer clear advantages for rapid prototyping and research, their role as a universal solution for clinically applicable medical AI remains uncertain. For high-stakes clinical applications, rigorous validation and consideration of task-specific training remain critically important. We advocate for integrating the strengths of FMs and end-to-end learning to achieve robust and resource-efficient AI pathology solutions fit for clinical use.

Title: BAnG: Bidirectional Anchored Generation for Conditional RNA Design

• Authors: Roman Klypa, Alberto Bietti, Sergei Grudinin
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.21274
• Pdf URL: https://arxiv.org/pdf/2502.21274
• Copy Paste: [[2502.21274]] BAnG: Bidirectional Anchored Generation for Conditional RNA Design(https://arxiv.org/abs/2502.21274)
• Keywords: generative
• Abstract: Designing RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Existing computational approaches require a substantial amount of experimentally determined RNA sequences for each specific protein or a detailed knowledge of RNA structure, restricting their utility in practice. To address this limitation, we develop RNA-BAnG, a deep learning-based model designed to generate RNA sequences for protein interactions without these requirements. Central to our approach is a novel generative method, Bidirectional Anchored Generation (BAnG), which leverages the observation that protein-binding RNA sequences often contain functional binding motifs embedded within broader sequence contexts. We first validate our method on generic synthetic tasks involving similar localized motifs to those appearing in RNAs, demonstrating its benefits over existing generative approaches. We then evaluate our model on biological sequences, showing its effectiveness for conditional RNA sequence design given a binding protein.

Title: Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion

• Authors: Kulin Shah, Alkis Kalavasis, Adam R. Klivans, Giannis Daras
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.21278
• Pdf URL: https://arxiv.org/pdf/2502.21278
• Copy Paste: [[2502.21278]] Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion(https://arxiv.org/abs/2502.21278)
• Keywords: diffusion, generative
• Abstract: There is strong empirical evidence that the state-of-the-art diffusion modeling paradigm leads to models that memorize the training set, especially when the training set is small. Prior methods to mitigate the memorization problem often lead to a decrease in image quality. Is it possible to obtain strong and creative generative models, i.e., models that achieve high generation quality and low memorization? Despite the current pessimistic landscape of results, we make significant progress in pushing the trade-off between fidelity and memorization. We first provide theoretical evidence that memorization in diffusion models is only necessary for denoising problems at low noise scales (usually used in generating high-frequency details). Using this theoretical insight, we propose a simple, principled method to train the diffusion models using noisy data at large noise scales. We show that our method significantly reduces memorization without decreasing the image quality, for both text-conditional and unconditional models and for a variety of data availability settings.

Title: MIGE: A Unified Framework for Multimodal Instruction-Based Image Generation and Editing

• Authors: Xueyun Tian, Wei Li, Bingbing Xu, Yige Yuan, Yuanzhuo Wang, Huawei Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21291
• Pdf URL: https://arxiv.org/pdf/2502.21291
• Copy Paste: [[2502.21291]] MIGE: A Unified Framework for Multimodal Instruction-Based Image Generation and Editing(https://arxiv.org/abs/2502.21291)
• Keywords: diffusion
• Abstract: Despite significant progress in diffusion-based image generation, subject-driven generation and instruction-based editing remain challenging. Existing methods typically treat them separately, struggling with limited high-quality data and poor generalization. However, both tasks require capturing complex visual variations while maintaining consistency between inputs and outputs. Therefore, we propose MIGE, a unified framework that standardizes task representations using multimodal instructions. It treats subject-driven generation as creation on a blank canvas and instruction-based editing as modification of an existing image, establishing a shared input-output formulation. MIGE introduces a novel multimodal encoder that maps free-form multimodal instructions into a unified vision-language space, integrating visual and semantic features through a feature fusion this http URL unification enables joint training of both tasks, providing two key advantages: (1) Cross-Task Enhancement: By leveraging shared visual and semantic representations, joint training improves instruction adherence and visual consistency in both subject-driven generation and instruction-based editing. (2) Generalization: Learning in a unified format facilitates cross-task knowledge transfer, enabling MIGE to generalize to novel compositional tasks, including instruction-based subject-driven editing. Experiments show that MIGE excels in both subject-driven generation and instruction-based editing while setting a state-of-the-art in the new task of instruction-based subject-driven editing. Code and model have been publicly available at this https URL.

Title: Unsupervised Parameter Efficient Source-free Post-pretraining

• Authors: Abhishek Jha, Tinne Tuytelaars, Yuki M. Asano
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.21313
• Pdf URL: https://arxiv.org/pdf/2502.21313
• Copy Paste: [[2502.21313]] Unsupervised Parameter Efficient Source-free Post-pretraining(https://arxiv.org/abs/2502.21313)
• Keywords: self-supervised
• Abstract: Following the success in NLP, the best vision models are now in the billion parameter ranges. Adapting these large models to a target distribution has become computationally and economically prohibitive. Addressing this challenge, we introduce UpStep, an Unsupervised Parameter-efficient Source-free post-pretraining approach, designed to efficiently adapt a base model from a source domain to a target domain: i) we design a self-supervised training scheme to adapt a pretrained model on an unlabeled target domain in a setting where source domain data is unavailable. Such source-free setting comes with the risk of catastrophic forgetting, hence, ii) we propose center vector regularization (CVR), a set of auxiliary operations that minimize catastrophic forgetting and additionally reduces the computational cost by skipping backpropagation in 50\% of the training iterations. Finally iii) we perform this adaptation process in a parameter-efficient way by adapting the pretrained model through low-rank adaptation methods, resulting in a fraction of parameters to optimize. We utilize various general backbone architectures, both supervised and unsupervised, trained on Imagenet as our base model and adapt them to a diverse set of eight target domains demonstrating the adaptability and generalizability of our proposed approach.

Title: Raccoon: Multi-stage Diffusion Training with Coarse-to-Fine Curating Videos

• Authors: Zhiyu Tan, Junyan Wang, Hao Yang, Luozheng Qin, Hesen Chen, Qiang Zhou, Hao Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.21314
• Pdf URL: https://arxiv.org/pdf/2502.21314
• Copy Paste: [[2502.21314]] Raccoon: Multi-stage Diffusion Training with Coarse-to-Fine Curating Videos(https://arxiv.org/abs/2502.21314)
• Keywords: diffusion
• Abstract: Text-to-video generation has demonstrated promising progress with the advent of diffusion models, yet existing approaches are limited by dataset quality and computational resources. To address these limitations, this paper presents a comprehensive approach that advances both data curation and model design. We introduce CFC-VIDS-1M, a high-quality video dataset constructed through a systematic coarse-to-fine curation pipeline. The pipeline first evaluates video quality across multiple dimensions, followed by a fine-grained stage that leverages vision-language models to enhance text-video alignment and semantic richness. Building upon the curated dataset's emphasis on visual quality and temporal coherence, we develop RACCOON, a transformer-based architecture with decoupled spatial-temporal attention mechanisms. The model is trained through a progressive four-stage strategy designed to efficiently handle the complexities of video generation. Extensive experiments demonstrate that our integrated approach of high-quality data curation and efficient training strategy generates visually appealing and temporally coherent videos while maintaining computational efficiency. We will release our dataset, code, and models.