2025-04-07

Title: Synthesized Annotation Guidelines are Knowledge-Lite Boosters for Clinical Information Extraction

• Authors: Enshuo Hsu, Martin Ugbala, Krishna Kumar Kookal, Zouaidi Kawtar, Nicholas L. Rider, Muhammad F. Walji, Kirk Roberts
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.02871
• Pdf URL: https://arxiv.org/pdf/2504.02871
• Copy Paste: [[2504.02871]] Synthesized Annotation Guidelines are Knowledge-Lite Boosters for Clinical Information Extraction(https://arxiv.org/abs/2504.02871)
• Keywords: generative
• Abstract: Generative information extraction using large language models, particularly through few-shot learning, has become a popular method. Recent studies indicate that providing a detailed, human-readable guideline-similar to the annotation guidelines traditionally used for training human annotators can significantly improve performance. However, constructing these guidelines is both labor- and knowledge-intensive. Additionally, the definitions are often tailored to meet specific needs, making them highly task-specific and often non-reusable. Handling these subtle differences requires considerable effort and attention to detail. In this study, we propose a self-improving method that harvests the knowledge summarization and text generation capacity of LLMs to synthesize annotation guidelines while requiring virtually no human input. Our zero-shot experiments on the clinical named entity recognition benchmarks, 2012 i2b2 EVENT, 2012 i2b2 TIMEX, 2014 i2b2, and 2018 n2c2 showed 25.86%, 4.36%, 0.20%, and 7.75% improvements in strict F1 scores from the no-guideline baseline. The LLM-synthesized guidelines showed equivalent or better performance compared to human-written guidelines by 1.15% to 4.14% in most tasks. In conclusion, this study proposes a novel LLM self-improving method that requires minimal knowledge and human input and is applicable to multiple biomedical domains.

Title: Processes Matter: How ML/GAI Approaches Could Support Open Qualitative Coding of Online Discourse Datasets

• Authors: John Chen, Alexandros Lotsos, Grace Wang, Lexie Zhao, Bruce Sherin, Uri Wilensky, Michael Horn
• Subjects: cs.CL, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.02887
• Pdf URL: https://arxiv.org/pdf/2504.02887
• Copy Paste: [[2504.02887]] Processes Matter: How ML/GAI Approaches Could Support Open Qualitative Coding of Online Discourse Datasets(https://arxiv.org/abs/2504.02887)
• Keywords: generative
• Abstract: Open coding, a key inductive step in qualitative research, discovers and constructs concepts from human datasets. However, capturing extensive and nuanced aspects or "coding moments" can be challenging, especially with large discourse datasets. While some studies explore machine learning (ML)/Generative AI (GAI)'s potential for open coding, few evaluation studies exist. We compare open coding results by five recently published ML/GAI approaches and four human coders, using a dataset of online chat messages around a mobile learning software. Our systematic analysis reveals ML/GAI approaches' strengths and weaknesses, uncovering the complementary potential between humans and AI. Line-by-line AI approaches effectively identify content-based codes, while humans excel in interpreting conversational dynamics. We discussed how embedded analytical processes could shape the results of ML/GAI approaches. Instead of replacing humans in open coding, researchers should integrate AI with and according to their analytical processes, e.g., as parallel co-coders.

Title: A Practical Synthesis of Detecting AI-Generated Textual, Visual, and Audio Content

• Authors: Lele Cao
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.02898
• Pdf URL: https://arxiv.org/pdf/2504.02898
• Copy Paste: [[2504.02898]] A Practical Synthesis of Detecting AI-Generated Textual, Visual, and Audio Content(https://arxiv.org/abs/2504.02898)
• Keywords: diffusion, generative
• Abstract: Advances in AI-generated content have led to wide adoption of large language models, diffusion-based visual generators, and synthetic audio tools. However, these developments raise critical concerns about misinformation, copyright infringement, security threats, and the erosion of public trust. In this paper, we explore an extensive range of methods designed to detect and mitigate AI-generated textual, visual, and audio content. We begin by discussing motivations and potential impacts associated with AI-based content generation, including real-world risks and ethical dilemmas. We then outline detection techniques spanning observation-based strategies, linguistic and statistical analysis, model-based pipelines, watermarking and fingerprinting, as well as emergent ensemble approaches. We also present new perspectives on robustness, adaptation to rapidly improving generative architectures, and the critical role of human-in-the-loop verification. By surveying state-of-the-art research and highlighting case studies in academic, journalistic, legal, and industrial contexts, this paper aims to inform robust solutions and policymaking. We conclude by discussing open challenges, including adversarial transformations, domain generalization, and ethical concerns, thereby offering a holistic guide for researchers, practitioners, and regulators to preserve content authenticity in the face of increasingly sophisticated AI-generated media.

Title: Comparative Analysis of Deepfake Detection Models: New Approaches and Perspectives

• Authors: Matheus Martins Batista
• Subjects: cs.CV, cs.LG, stat.CO, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.02900
• Pdf URL: https://arxiv.org/pdf/2504.02900
• Copy Paste: [[2504.02900]] Comparative Analysis of Deepfake Detection Models: New Approaches and Perspectives(https://arxiv.org/abs/2504.02900)
• Keywords: generative
• Abstract: The growing threat posed by deepfake videos, capable of manipulating realities and disseminating misinformation, drives the urgent need for effective detection methods. This work investigates and compares different approaches for identifying deepfakes, focusing on the GenConViT model and its performance relative to other architectures present in the DeepfakeBenchmark. To contextualize the research, the social and legal impacts of deepfakes are addressed, as well as the technical fundamentals of their creation and detection, including digital image processing, machine learning, and artificial neural networks, with emphasis on Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs), and Transformers. The performance evaluation of the models was conducted using relevant metrics and new datasets established in the literature, such as WildDeep-fake and DeepSpeak, aiming to identify the most effective tools in the battle against misinformation and media manipulation. The obtained results indicated that GenConViT, after fine-tuning, exhibited superior performance in terms of accuracy (93.82%) and generalization capacity, surpassing other architectures in the DeepfakeBenchmark on the DeepSpeak dataset. This study contributes to the advancement of deepfake detection techniques, offering contributions to the development of more robust and effective solutions against the dissemination of false information.

Title: Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments

• Authors: Chenyu Zhang, Daniil Cherniavskii, Andrii Zadaianchuk, Antonios Tragoudaras, Antonios Vozikis, Thijmen Nijdam, Derck W. E. Prinzhorn, Mark Bodracska, Nicu Sebe, Efstratios Gavves
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.02918
• Pdf URL: https://arxiv.org/pdf/2504.02918
• Copy Paste: [[2504.02918]] Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments(https://arxiv.org/abs/2504.02918)
• Keywords: foundation model, generative
• Abstract: Recent advances in image and video generation raise hopes that these models possess world modeling capabilities, the ability to generate realistic, physically plausible videos. This could revolutionize applications in robotics, autonomous driving, and scientific simulation. However, before treating these models as world models, we must ask: Do they adhere to physical conservation laws? To answer this, we introduce Morpheus, a benchmark for evaluating video generation models on physical reasoning. It features 80 real-world videos capturing physical phenomena, guided by conservation laws. Since artificial generations lack ground truth, we assess physical plausibility using physics-informed metrics evaluated with respect to infallible conservation laws known per physical setting, leveraging advances in physics-informed neural networks and vision-language foundation models. Our findings reveal that even with advanced prompting and video conditioning, current models struggle to encode physical principles despite generating aesthetically pleasing videos. All data, leaderboard, and code are open-sourced at our project page.

Title: VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning

• Authors: Xianwei Zhuang, Yuxin Xie, Yufan Deng, Dongchao Yang, Liming Liang, Jinghan Ru, Yuguo Yin, Yuexian Zou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.02949
• Pdf URL: https://arxiv.org/pdf/2504.02949
• Copy Paste: [[2504.02949]] VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning(https://arxiv.org/abs/2504.02949)
• Keywords: generative
• Abstract: In this work, we present VARGPT-v1.1, an advanced unified visual autoregressive model that builds upon our previous framework VARGPT. The model preserves the dual paradigm of next-token prediction for visual understanding and next-scale generation for image synthesis. Specifically, VARGPT-v1.1 integrates: (1) a novel training strategy combining iterative visual instruction tuning with reinforcement learning through Direct Preference Optimization (DPO), (2) an expanded training corpus containing 8.3M visual-generative instruction pairs, (3) an upgraded language model backbone using Qwen2, (4) enhanced image generation resolution, and (5) emergent image editing capabilities without architectural modifications. These advancements enable VARGPT-v1.1 to achieve state-of-the-art performance in multimodal understanding and text-to-image instruction-following tasks, demonstrating significant improvements in both comprehension and generation metrics. Notably, through visual instruction tuning, the model acquires image editing functionality while maintaining architectural consistency with its predecessor, revealing the potential for unified visual understanding, generation, and editing. Our findings suggest that well-designed unified visual autoregressive models can effectively adopt flexible training strategies from large language models (LLMs), exhibiting promising scalability. The codebase and model weights are publicly available at this https URL.

Title: Improving log-based anomaly detection through learned adaptive filter

• Authors: Yiyuan Xiong, Shaofeng Cai
• Subjects: cs.LG, cs.DB
• Abstract URL: https://arxiv.org/abs/2504.02994
• Pdf URL: https://arxiv.org/pdf/2504.02994
• Copy Paste: [[2504.02994]] Improving log-based anomaly detection through learned adaptive filter(https://arxiv.org/abs/2504.02994)
• Keywords: anomaly
• Abstract: Log messages record important system runtime information and are useful for detecting anomalous behaviors and managing modern software systems. Many supervised and unsupervised learning methods have been proposed recently for log-based anomaly detection. State-of-the-art unsupervised methods predict the next log event given a log sequence and apply fixed configurations that use the same filter condition (i.e. k, the top k predicted log events will be regarded as normal next events) which leads to inferior performance in the detection stage because it sets one fixed k for all log sequences, which ignores the dynamic nature and variance in different log sequences. Recently, deep reinforcement learning (DRL) are widely applied to make intelligent decisions in a dynamic environment. In this work, we contend that it is necessary to apply adaptive filters for different log sequences. To achieve this, we propose a novel approach based on DRL to construct a learned adaptive filter and apply different normal/abnormal filter thresholds for different log sequences. We define the Markov Decision Process (MDP) and formulate the learned adaptive filter as a problem that can be solved by DRL. We evaluate the learned adaptive filter on two state-of-the-art log-based anomaly detection unsupervised approaches DeepLog and LogAnomaly in two datasets HDFS and BGL. Extensive experiments show that our approach outperforms the fixed configurations and achieves significantly better performance in log-based anomaly detection.

Title: Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning

• Authors: Bahareh Golchin, Banafsheh Rekabdar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.02999
• Pdf URL: https://arxiv.org/pdf/2504.02999
• Copy Paste: [[2504.02999]] Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning(https://arxiv.org/abs/2504.02999)
• Keywords: anomaly
• Abstract: A novel approach to detecting anomalies in time series data is presented in this paper. This approach is pivotal in domains such as data centers, sensor networks, and finance. Traditional methods often struggle with manual parameter tuning and cannot adapt to new anomaly types. Our method overcomes these limitations by integrating Deep Reinforcement Learning (DRL) with a Variational Autoencoder (VAE) and Active Learning. By incorporating a Long Short-Term Memory (LSTM) network, our approach models sequential data and its dependencies effectively, allowing for the detection of new anomaly classes with minimal labeled data. Our innovative DRL- VAE and Active Learning combination significantly improves existing methods, as shown by our evaluations on real-world datasets, enhancing anomaly detection techniques and advancing time series analysis.

Title: DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery

• Authors: Jing Gao, Ce Zheng, Laszlo A. Jeni, Zackory Erickson
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03006
• Pdf URL: https://arxiv.org/pdf/2504.03006
• Copy Paste: [[2504.03006]] DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery(https://arxiv.org/abs/2504.03006)
• Keywords: diffusion
• Abstract: In-bed human mesh recovery can be crucial and enabling for several healthcare applications, including sleep pattern monitoring, rehabilitation support, and pressure ulcer prevention. However, it is difficult to collect large real-world visual datasets in this domain, in part due to privacy and expense constraints, which in turn presents significant challenges for training and deploying deep learning models. Existing in-bed human mesh estimation methods often rely heavily on real-world data, limiting their ability to generalize across different in-bed scenarios, such as varying coverings and environmental settings. To address this, we propose a Sim-to-Real Transfer Framework for in-bed human mesh recovery from overhead depth images, which leverages large-scale synthetic data alongside limited or no real-world samples. We introduce a diffusion model that bridges the gap between synthetic data and real data to support generalization in real-world in-bed pose and body inference scenarios. Extensive experiments and ablation studies validate the effectiveness of our framework, demonstrating significant improvements in robustness and adaptability across diverse healthcare scenarios.

Title: Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization

• Authors: Junying Wang, Jingyuan Liu, Xin Sun, Krishna Kumar Singh, Zhixin Shu, He Zhang, Jimei Yang, Nanxuan Zhao, Tuanfeng Y. Wang, Simon S. Chen, Ulrich Neumann, Jae Shin Yoon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03011
• Pdf URL: https://arxiv.org/pdf/2504.03011
• Copy Paste: [[2504.03011]] Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization(https://arxiv.org/abs/2504.03011)
• Keywords: diffusion
• Abstract: This paper introduces Comprehensive Relighting, the first all-in-one approach that can both control and harmonize the lighting from an image or video of humans with arbitrary body parts from any scene. Building such a generalizable model is extremely challenging due to the lack of dataset, restricting existing image-based relighting models to a specific scenario (e.g., face or static human). To address this challenge, we repurpose a pre-trained diffusion model as a general image prior and jointly model the human relighting and background harmonization in the coarse-to-fine framework. To further enhance the temporal coherence of the relighting, we introduce an unsupervised temporal lighting model that learns the lighting cycle consistency from many real-world videos without any ground truth. In inference time, our temporal lighting module is combined with the diffusion models through the spatio-temporal feature blending algorithms without extra training; and we apply a new guided refinement as a post-processing to preserve the high-frequency details from the input image. In the experiments, Comprehensive Relighting shows a strong generalizability and lighting temporal coherence, outperforming existing image-based human relighting and harmonization methods.

Title: The Dual-Route Model of Induction

• Authors: Sheridan Feucht, Eric Todd, Byron Wallace, David Bau
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03022
• Pdf URL: https://arxiv.org/pdf/2504.03022
• Copy Paste: [[2504.03022]] The Dual-Route Model of Induction(https://arxiv.org/abs/2504.03022)
• Keywords: in-context
• Abstract: Prior work on in-context copying has shown the existence of induction heads, which attend to and promote individual tokens during copying. In this work we introduce a new type of induction head: concept-level induction heads, which copy entire lexical units instead of individual tokens. Concept induction heads learn to attend to the ends of multi-token words throughout training, working in parallel with token-level induction heads to copy meaningful text. We show that these heads are responsible for semantic tasks like word-level translation, whereas token induction heads are vital for tasks that can only be done verbatim, like copying nonsense tokens. These two "routes" operate independently: in fact, we show that ablation of token induction heads causes models to paraphrase where they would otherwise copy verbatim. In light of these findings, we argue that although token induction heads are vital for specific tasks, concept induction heads may be more broadly relevant for in-context learning.

Title: AD-GPT: Large Language Models in Alzheimer's Disease

• Authors: Ziyu Liu, Lintao Tang, Zeliang Sun, Zhengliang Liu, Yanjun Lyu, Wei Ruan, Yangshuang Xu, Liang Shan, Jiyoon Shin, Xiaohe Chen, Dajiang Zhu, Tianming Liu, Rongjie Liu, Chao Huang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03071
• Pdf URL: https://arxiv.org/pdf/2504.03071
• Copy Paste: [[2504.03071]] AD-GPT: Large Language Models in Alzheimer's Disease(https://arxiv.org/abs/2504.03071)
• Keywords: generative
• Abstract: Large language models (LLMs) have emerged as powerful tools for medical information retrieval, yet their accuracy and depth remain limited in specialized domains such as Alzheimer's disease (AD), a growing global health challenge. To address this gap, we introduce AD-GPT, a domain-specific generative pre-trained transformer designed to enhance the retrieval and analysis of AD-related genetic and neurobiological information. AD-GPT integrates diverse biomedical data sources, including potential AD-associated genes, molecular genetic information, and key gene variants linked to brain regions. We develop a stacked LLM architecture combining Llama3 and BERT, optimized for four critical tasks in AD research: (1) genetic information retrieval, (2) gene-brain region relationship assessment, (3) gene-AD relationship analysis, and (4) brain region-AD relationship mapping. Comparative evaluations against state-of-the-art LLMs demonstrate AD-GPT's superior precision and reliability across these tasks, underscoring its potential as a robust and specialized AI tool for advancing AD research and biomarker discovery.

Title: How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models

• Authors: Pascal Chang, Jingwei Tang, Markus Gross, Vinicius C. Azevedo
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03072
• Pdf URL: https://arxiv.org/pdf/2504.03072
• Copy Paste: [[2504.03072]] How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models(https://arxiv.org/abs/2504.03072)
• Keywords: diffusion
• Abstract: Video editing and generation methods often rely on pre-trained image-based diffusion models. During the diffusion process, however, the reliance on rudimentary noise sampling techniques that do not preserve correlations present in subsequent frames of a video is detrimental to the quality of the results. This either produces high-frequency flickering, or texture-sticking artifacts that are not amenable to post-processing. With this in mind, we propose a novel method for preserving temporal correlations in a sequence of noise samples. This approach is materialized by a novel noise representation, dubbed $\int$-noise (integral noise), that reinterprets individual noise samples as a continuously integrated noise field: pixel values do not represent discrete values, but are rather the integral of an underlying infinite-resolution noise over the pixel area. Additionally, we propose a carefully tailored transport method that uses $\int$-noise to accurately advect noise samples over a sequence of frames, maximizing the correlation between different frames while also preserving the noise properties. Our results demonstrate that the proposed $\int$-noise can be used for a variety of tasks, such as video restoration, surrogate rendering, and conditional video generation. See this https URL for video results.

Title: SLACK: Attacking LiDAR-based SLAM with Adversarial Point Injections

• Authors: Prashant Kumar, Dheeraj Vattikonda, Kshitij Madhav Bhat, Kunal Dargan, Prem Kalra
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03089
• Pdf URL: https://arxiv.org/pdf/2504.03089
• Copy Paste: [[2504.03089]] SLACK: Attacking LiDAR-based SLAM with Adversarial Point Injections(https://arxiv.org/abs/2504.03089)
• Keywords: generative
• Abstract: The widespread adoption of learning-based methods for the LiDAR makes autonomous vehicles vulnerable to adversarial attacks through adversarial \textit{point injections (PiJ)}. It poses serious security challenges for navigation and map generation. Despite its critical nature, no major work exists that studies learning-based attacks on LiDAR-based SLAM. Our work proposes SLACK, an end-to-end deep generative adversarial model to attack LiDAR scans with several point injections without deteriorating LiDAR quality. To facilitate SLACK, we design a novel yet simple autoencoder that augments contrastive learning with segmentation-based attention for precise reconstructions. SLACK demonstrates superior performance on the task of \textit{point injections (PiJ)} compared to the best baselines on KITTI and CARLA-64 dataset while maintaining accurate scan quality. We qualitatively and quantitatively demonstrate PiJ attacks using a fraction of LiDAR points. It severely degrades navigation and map quality without deteriorating the LiDAR scan quality.

Title: Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Models

• Authors: Xuran Ma, Yexin Liu, Yaofu Liu, Xianfeng Wu, Mingzhe Zheng, Zihao Wang, Ser-Nam Lim, Harry Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03140
• Pdf URL: https://arxiv.org/pdf/2504.03140
• Copy Paste: [[2504.03140]] Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Models(https://arxiv.org/abs/2504.03140)
• Keywords: diffusion
• Abstract: Recent advances in diffusion models have demonstrated remarkable capabilities in video generation. However, the computational intensity remains a significant challenge for practical applications. While feature caching has been proposed to reduce the computational burden of diffusion models, existing methods typically overlook the heterogeneous significance of individual blocks, resulting in suboptimal reuse and degraded output quality. To this end, we address this gap by introducing ProfilingDiT, a novel adaptive caching strategy that explicitly disentangles foreground and background-focused blocks. Through a systematic analysis of attention distributions in diffusion models, we reveal a key observation: 1) Most layers exhibit a consistent preference for either foreground or background regions. 2) Predicted noise shows low inter-step similarity initially, which stabilizes as denoising progresses. This finding inspires us to formulate a selective caching strategy that preserves full computation for dynamic foreground elements while efficiently caching static background features. Our approach substantially reduces computational overhead while preserving visual fidelity. Extensive experiments demonstrate that our framework achieves significant acceleration (e.g., 2.01 times speedup for Wan2.1) while maintaining visual fidelity across comprehensive quality metrics, establishing a viable method for efficient video generation.

Title: RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation

• Authors: Hanbo Bi, Yingchao Feng, Boyuan Tong, Mengyu Wang, Haichen Yu, Yongqiang Mao, Hao Chang, Wenhui Diao, Peijin Wang, Yue Yu, Hanyang Peng, Yehong Zhang, Kun Fu, Xian Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03166
• Pdf URL: https://arxiv.org/pdf/2504.03166
• Copy Paste: [[2504.03166]] RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation(https://arxiv.org/abs/2504.03166)
• Keywords: self-supervised, foundation model
• Abstract: The rapid advancement of foundation models has revolutionized visual representation learning in a self-supervised manner. However, their application in remote sensing (RS) remains constrained by a fundamental gap: existing models predominantly handle single or limited modalities, overlooking the inherently multi-modal nature of RS observations. Optical, synthetic aperture radar (SAR), and multi-spectral data offer complementary insights that significantly reduce the inherent ambiguity and uncertainty in single-source analysis. To bridge this gap, we introduce RingMoE, a unified multi-modal RS foundation model with 14.7 billion parameters, pre-trained on 400 million multi-modal RS images from nine satellites. RingMoE incorporates three key innovations: (1) A hierarchical Mixture-of-Experts (MoE) architecture comprising modal-specialized, collaborative, and shared experts, effectively modeling intra-modal knowledge while capturing cross-modal dependencies to mitigate conflicts between modal representations; (2) Physics-informed self-supervised learning, explicitly embedding sensor-specific radiometric characteristics into the pre-training objectives; (3) Dynamic expert pruning, enabling adaptive model compression from 14.7B to 1B parameters while maintaining performance, facilitating efficient deployment in Earth observation applications. Evaluated across 23 benchmarks spanning six key RS tasks (i.e., classification, detection, segmentation, tracking, change detection, and depth estimation), RingMoE outperforms existing foundation models and sets new SOTAs, demonstrating remarkable adaptability from single-modal to multi-modal scenarios. Beyond theoretical progress, it has been deployed and trialed in multiple sectors, including emergency response, land management, marine sciences, and urban planning.

Title: REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval

• Authors: Shabnam Choudhury, Yash Salunkhe, Sarthak Mehrotra, Biplab Banerjee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03169
• Pdf URL: https://arxiv.org/pdf/2504.03169
• Copy Paste: [[2504.03169]] REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval(https://arxiv.org/abs/2504.03169)
• Keywords: self-supervised, generative
• Abstract: The rapid expansion of remote sensing image archives demands the development of strong and efficient techniques for content-based image retrieval (RS-CBIR). This paper presents REJEPA (Retrieval with Joint-Embedding Predictive Architecture), an innovative self-supervised framework designed for unimodal RS-CBIR. REJEPA utilises spatially distributed context token encoding to forecast abstract representations of target tokens, effectively capturing high-level semantic features and eliminating unnecessary pixel-level details. In contrast to generative methods that focus on pixel reconstruction or contrastive techniques that depend on negative pairs, REJEPA functions within feature space, achieving a reduction in computational complexity of 40-60% when compared to pixel-reconstruction baselines like Masked Autoencoders (MAE). To guarantee strong and varied representations, REJEPA incorporates Variance-Invariance-Covariance Regularisation (VICReg), which prevents encoder collapse by promoting feature diversity and reducing redundancy. The method demonstrates an estimated enhancement in retrieval accuracy of 5.1% on BEN-14K (S1), 7.4% on BEN-14K (S2), 6.0% on FMoW-RGB, and 10.1% on FMoW-Sentinel compared to prominent SSL techniques, including CSMAE-SESD, Mask-VLM, SatMAE, ScaleMAE, and SatMAE++, on extensive RS benchmarks BEN-14K (multispectral and SAR data), FMoW-RGB and FMoW-Sentinel. Through effective generalisation across sensor modalities, REJEPA establishes itself as a sensor-agnostic benchmark for efficient, scalable, and precise RS-CBIR, addressing challenges like varying resolutions, high object density, and complex backgrounds with computational efficiency.

Title: MIMRS: A Survey on Masked Image Modeling in Remote Sensing

• Authors: Shabnam Choudhury, Akhil Vasim, Michael Schmitt, Biplab Banerjee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03181
• Pdf URL: https://arxiv.org/pdf/2504.03181
• Copy Paste: [[2504.03181]] MIMRS: A Survey on Masked Image Modeling in Remote Sensing(https://arxiv.org/abs/2504.03181)
• Keywords: self-supervised
• Abstract: Masked Image Modeling (MIM) is a self-supervised learning technique that involves masking portions of an image, such as pixels, patches, or latent representations, and training models to predict the missing information using the visible context. This approach has emerged as a cornerstone in self-supervised learning, unlocking new possibilities in visual understanding by leveraging unannotated data for pre-training. In remote sensing, MIM addresses challenges such as incomplete data caused by cloud cover, occlusions, and sensor limitations, enabling applications like cloud removal, multi-modal data fusion, and super-resolution. By synthesizing and critically analyzing recent advancements, this survey (MIMRS) is a pioneering effort to chart the landscape of mask image modeling in remote sensing. We highlight state-of-the-art methodologies, applications, and future research directions, providing a foundational review to guide innovation in this rapidly evolving field.

Title: On the Connection Between Diffusion Models and Molecular Dynamics

• Authors: Liam Harcombe, Timothy T. Duignan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03187
• Pdf URL: https://arxiv.org/pdf/2504.03187
• Copy Paste: [[2504.03187]] On the Connection Between Diffusion Models and Molecular Dynamics(https://arxiv.org/abs/2504.03187)
• Keywords: diffusion
• Abstract: Neural Network Potentials (NNPs) have emerged as a powerful tool for modelling atomic interactions with high accuracy and computational efficiency. Recently, denoising diffusion models have shown promise in NNPs by training networks to remove noise added to stable configurations, eliminating the need for force data during training. In this work, we explore the connection between noise and forces by providing a new, simplified mathematical derivation of their relationship. We also demonstrate how a denoising model can be implemented using a conventional MD software package interfaced with a standard NNP architecture. We demonstrate the approach by training a diffusion-based NNP to simulate a coarse-grained lithium chloride solution and employ data duplication to enhance model performance.

Title: Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation

• Authors: Xin Zhang, Robby T. Tan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03193
• Pdf URL: https://arxiv.org/pdf/2504.03193
• Copy Paste: [[2504.03193]] Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation(https://arxiv.org/abs/2504.03193)
• Keywords: foundation model
• Abstract: Vision Foundation Models (VFMs) and Vision-Language Models (VLMs) have gained traction in Domain Generalized Semantic Segmentation (DGSS) due to their strong generalization capabilities. However, existing DGSS methods often rely exclusively on either VFMs or VLMs, overlooking their complementary strengths. VFMs (e.g., DINOv2) excel at capturing fine-grained features, while VLMs (e.g., CLIP) provide robust text alignment but struggle with coarse granularity. Despite their complementary strengths, effectively integrating VFMs and VLMs with attention mechanisms is challenging, as the increased patch tokens complicate long-sequence modeling. To address this, we propose MFuser, a novel Mamba-based fusion framework that efficiently combines the strengths of VFMs and VLMs while maintaining linear scalability in sequence length. MFuser consists of two key components: MVFuser, which acts as a co-adapter to jointly fine-tune the two models by capturing both sequential and spatial dynamics; and MTEnhancer, a hybrid attention-Mamba module that refines text embeddings by incorporating image priors. Our approach achieves precise feature locality and strong text alignment without incurring significant computational overhead. Extensive experiments demonstrate that MFuser significantly outperforms state-of-the-art DGSS methods, achieving 68.20 mIoU on synthetic-to-real and 71.87 mIoU on real-to-real benchmarks. The code is available at this https URL.

Title: Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video

• Authors: Jiaxin Guo, Wenzhen Dong, Tianyu Huang, Hao Ding, Ziyi Wang, Haomin Kuang, Qi Dou, Yun-Hui Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03198
• Pdf URL: https://arxiv.org/pdf/2504.03198
• Copy Paste: [[2504.03198]] Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video(https://arxiv.org/abs/2504.03198)
• Keywords: self-supervised, foundation model
• Abstract: Reconstructing 3D scenes from monocular surgical videos can enhance surgeon's perception and therefore plays a vital role in various computer-assisted surgery tasks. However, achieving scale-consistent reconstruction remains an open challenge due to inherent issues in endoscopic videos, such as dynamic deformations and textureless surfaces. Despite recent advances, current methods either rely on calibration or instrument priors to estimate scale, or employ SfM-like multi-stage pipelines, leading to error accumulation and requiring offline optimization. In this paper, we present Endo3R, a unified 3D foundation model for online scale-consistent reconstruction from monocular surgical video, without any priors or extra optimization. Our model unifies the tasks by predicting globally aligned pointmaps, scale-consistent video depths, and camera parameters without any offline optimization. The core contribution of our method is expanding the capability of the recent pairwise reconstruction model to long-term incremental dynamic reconstruction by an uncertainty-aware dual memory mechanism. The mechanism maintains history tokens of both short-term dynamics and long-term spatial consistency. Notably, to tackle the highly dynamic nature of surgical scenes, we measure the uncertainty of tokens via Sampson distance and filter out tokens with high uncertainty. Regarding the scarcity of endoscopic datasets with ground-truth depth and camera poses, we further devise a self-supervised mechanism with a novel dynamics-aware flow loss. Abundant experiments on SCARED and Hamlyn datasets demonstrate our superior performance in zero-shot surgical video depth prediction and camera pose estimation with online efficiency. Project page: this https URL.

Title: PIONM: A Generalized Approach to Solving Density-Constrained Mean-Field Games Equilibrium under Modified Boundary Conditions

• Authors: Jinwei Liu, Wang Yao, Xiao Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03209
• Pdf URL: https://arxiv.org/pdf/2504.03209
• Copy Paste: [[2504.03209]] PIONM: A Generalized Approach to Solving Density-Constrained Mean-Field Games Equilibrium under Modified Boundary Conditions(https://arxiv.org/abs/2504.03209)
• Keywords: diffusion
• Abstract: Neural network-based methods are effective for solving equilibria in Mean-Field Games (MFGs), particularly in high-dimensional settings. However, solving the coupled partial differential equations (PDEs) in MFGs limits their applicability since solving coupled PDEs is computationally expensive. Additionally, modifying boundary conditions, such as the initial state distribution or terminal value function, necessitates extensive retraining, reducing scalability. To address these challenges, we propose a generalized framework, PIONM (Physics-Informed Neural Operator NF-MKV Net), which leverages physics-informed neural operators to solve MFGs equations. PIONM utilizes neural operators to compute MFGs equilibria for arbitrary boundary conditions. The method encodes boundary conditions as input features and trains the model to align them with density evolution, modeled using discrete-time normalizing flows. Once trained, the algorithm efficiently computes the density distribution at any time step for modified boundary condition, ensuring efficient adaptation to different boundary conditions in MFGs equilibria. Unlike traditional MFGs methods constrained by fixed coefficients, PIONM efficiently computes equilibria under varying boundary conditions, including obstacles, diffusion coefficients, initial densities, and terminal functions. PIONM can adapt to modified conditions while preserving density distribution constraints, demonstrating superior scalability and generalization capabilities compared to existing methods.

Title: FaR: Enhancing Multi-Concept Text-to-Image Diffusion via Concept Fusion and Localized Refinement

• Authors: Gia-Nghia Tran, Quang-Huy Che, Trong-Tai Dam Vu, Bich-Nga Pham, Vinh-Tiep Nguyen, Trung-Nghia Le, Minh-Triet Tran
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03292
• Pdf URL: https://arxiv.org/pdf/2504.03292
• Copy Paste: [[2504.03292]] FaR: Enhancing Multi-Concept Text-to-Image Diffusion via Concept Fusion and Localized Refinement(https://arxiv.org/abs/2504.03292)
• Keywords: diffusion
• Abstract: Generating multiple new concepts remains a challenging problem in the text-to-image task. Current methods often overfit when trained on a small number of samples and struggle with attribute leakage, particularly for class-similar subjects (e.g., two specific dogs). In this paper, we introduce Fuse-and-Refine (FaR), a novel approach that tackles these challenges through two key contributions: Concept Fusion technique and Localized Refinement loss function. Concept Fusion systematically augments the training data by separating reference subjects from backgrounds and recombining them into composite images to increase diversity. This augmentation technique tackles the overfitting problem by mitigating the narrow distribution of the limited training samples. In addition, Localized Refinement loss function is introduced to preserve subject representative attributes by aligning each concept's attention map to its correct region. This approach effectively prevents attribute leakage by ensuring that the diffusion model distinguishes similar subjects without mixing their attention maps during the denoising process. By fine-tuning specific modules at the same time, FaR balances the learning of new concepts with the retention of previously learned knowledge. Empirical results show that FaR not only prevents overfitting and attribute leakage while maintaining photorealism, but also outperforms other state-of-the-art methods.

Title: Multi-Flow: Multi-View-Enriched Normalizing Flows for Industrial Anomaly Detection

• Authors: Mathis Kruse, Bodo Rosenhahn
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03306
• Pdf URL: https://arxiv.org/pdf/2504.03306
• Copy Paste: [[2504.03306]] Multi-Flow: Multi-View-Enriched Normalizing Flows for Industrial Anomaly Detection(https://arxiv.org/abs/2504.03306)
• Keywords: anomaly
• Abstract: With more well-performing anomaly detection methods proposed, many of the single-view tasks have been solved to a relatively good degree. However, real-world production scenarios often involve complex industrial products, whose properties may not be fully captured by one single image. While normalizing flow based approaches already work well in single-camera scenarios, they currently do not make use of the priors in multi-view data. We aim to bridge this gap by using these flow-based models as a strong foundation and propose Multi-Flow, a novel multi-view anomaly detection method. Multi-Flow makes use of a novel multi-view architecture, whose exact likelihood estimation is enhanced by fusing information across different views. For this, we propose a new cross-view message-passing scheme, letting information flow between neighboring views. We empirically validate it on the real-world multi-view data set Real-IAD and reach a new state-of-the-art, surpassing current baselines in both image-wise and sample-wise anomaly detection tasks.

Title: Steerable Anatomical Shape Synthesis with Implicit Neural Representations

• Authors: Bram de Wilde, Max T. Rietberg, Guillaume Lajoinie, Jelmer M. Wolterink
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03313
• Pdf URL: https://arxiv.org/pdf/2504.03313
• Copy Paste: [[2504.03313]] Steerable Anatomical Shape Synthesis with Implicit Neural Representations(https://arxiv.org/abs/2504.03313)
• Keywords: generative
• Abstract: Generative modeling of anatomical structures plays a crucial role in virtual imaging trials, which allow researchers to perform studies without the costs and constraints inherent to in vivo and phantom studies. For clinical relevance, generative models should allow targeted control to simulate specific patient populations rather than relying on purely random sampling. In this work, we propose a steerable generative model based on implicit neural representations. Implicit neural representations naturally support topology changes, making them well-suited for anatomical structures with varying topology, such as the thyroid. Our model learns a disentangled latent representation, enabling fine-grained control over shape variations. Evaluation includes reconstruction accuracy and anatomical plausibility. Our results demonstrate that the proposed model achieves high-quality shape generation while enabling targeted anatomical modifications.

Title: QIRL: Boosting Visual Question Answering via Optimized Question-Image Relation Learning

• Authors: Quanxing Xu, Ling Zhou, Xian Zhong, Feifei Zhang, Rubing Huang, Chia-Wen Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03337
• Pdf URL: https://arxiv.org/pdf/2504.03337
• Copy Paste: [[2504.03337]] QIRL: Boosting Visual Question Answering via Optimized Question-Image Relation Learning(https://arxiv.org/abs/2504.03337)
• Keywords: self-supervised
• Abstract: Existing debiasing approaches in Visual Question Answering (VQA) primarily focus on enhancing visual learning, integrating auxiliary models, or employing data augmentation strategies. However, these methods exhibit two major drawbacks. First, current debiasing techniques fail to capture the superior relation between images and texts because prevalent learning frameworks do not enable models to extract deeper correlations from highly contrasting samples. Second, they do not assess the relevance between the input question and image during inference, as no prior work has examined the degree of input relevance in debiasing studies. Motivated by these limitations, we propose a novel framework, Optimized Question-Image Relation Learning (QIRL), which employs a generation-based self-supervised learning strategy. Specifically, two modules are introduced to address the aforementioned issues. The Negative Image Generation (NIG) module automatically produces highly irrelevant question-image pairs during training to enhance correlation learning, while the Irrelevant Sample Identification (ISI) module improves model robustness by detecting and filtering irrelevant inputs, thereby reducing prediction errors. Furthermore, to validate our concept of reducing output errors through filtering unrelated question-image inputs, we propose a specialized metric to evaluate the performance of the ISI module. Notably, our approach is model-agnostic and can be integrated with various VQA models. Extensive experiments on VQA-CPv2 and VQA-v2 demonstrate the effectiveness and generalization ability of our method. Among data augmentation strategies, our approach achieves state-of-the-art results.

Title: BitHEP -- The Limits of Low-Precision ML in HEP

• Authors: Claudius Krause, Daohan Wang, Ramon Winterhalder
• Subjects: cs.LG, hep-ex, hep-ph
• Abstract URL: https://arxiv.org/abs/2504.03387
• Pdf URL: https://arxiv.org/pdf/2504.03387
• Copy Paste: [[2504.03387]] BitHEP -- The Limits of Low-Precision ML in HEP(https://arxiv.org/abs/2504.03387)
• Keywords: generative
• Abstract: The increasing complexity of modern neural network architectures demands fast and memory-efficient implementations to mitigate computational bottlenecks. In this work, we evaluate the recently proposed BitNet architecture in HEP applications, assessing its performance in classification, regression, and generative modeling tasks. Specifically, we investigate its suitability for quark-gluon discrimination, SMEFT parameter estimation, and detector simulation, comparing its efficiency and accuracy to state-of-the-art methods. Our results show that while BitNet consistently performs competitively in classification tasks, its performance in regression and generation varies with the size and type of the network, highlighting key limitations and potential areas for improvement.

Title: Pyramid-based Mamba Multi-class Unsupervised Anomaly Detection

• Authors: Nasar Iqbal, Niki Martinel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03442
• Pdf URL: https://arxiv.org/pdf/2504.03442
• Copy Paste: [[2504.03442]] Pyramid-based Mamba Multi-class Unsupervised Anomaly Detection(https://arxiv.org/abs/2504.03442)
• Keywords: anomaly
• Abstract: Recent advances in convolutional neural networks (CNNs) and transformer-based methods have improved anomaly detection and localization, but challenges persist in precisely localizing small anomalies. While CNNs face limitations in capturing long-range dependencies, transformer architectures often suffer from substantial computational overheads. We introduce a state space model (SSM)-based Pyramidal Scanning Strategy (PSS) for multi-class anomaly detection and localization--a novel approach designed to address the challenge of small anomaly localization. Our method captures fine-grained details at multiple scales by integrating the PSS with a pre-trained encoder for multi-scale feature extraction and a feature-level synthetic anomaly generator. An improvement of $+1\%$ AP for multi-class anomaly localization and a +$1\%$ increase in AU-PRO on MVTec benchmark demonstrate our method's superiority in precise anomaly localization across diverse industrial scenarios. The code is available at this https URL Mamba.

Title: Optimizing Specific and Shared Parameters for Efficient Parameter Tuning

• Authors: Van-Anh Nguyen, Thanh-Toan Do, Mehrtash Harandi, Dinh Phung, Trung Le
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03450
• Pdf URL: https://arxiv.org/pdf/2504.03450
• Copy Paste: [[2504.03450]] Optimizing Specific and Shared Parameters for Efficient Parameter Tuning(https://arxiv.org/abs/2504.03450)
• Keywords: foundation model
• Abstract: Foundation models, with a vast number of parameters and pretraining on massive datasets, achieve state-of-the-art performance across various applications. However, efficiently adapting them to downstream tasks with minimal computational overhead remains a challenge. Parameter-Efficient Transfer Learning (PETL) addresses this by fine-tuning only a small subset of parameters while preserving pre-trained knowledge. In this paper, we propose SaS, a novel PETL method that effectively mitigates distributional shifts during fine-tuning. SaS integrates (1) a shared module that captures common statistical characteristics across layers using low-rank projections and (2) a layer-specific module that employs hypernetworks to generate tailored parameters for each layer. This dual design ensures an optimal balance between performance and parameter efficiency while introducing less than 0.05% additional parameters, making it significantly more compact than existing methods. Extensive experiments on diverse downstream tasks, few-shot settings and domain generalization demonstrate that SaS significantly enhances performance while maintaining superior parameter efficiency compared to existing methods, highlighting the importance of capturing both shared and layer-specific information in transfer learning. Code and data are available at this https URL.

Title: D-Garment: Physics-Conditioned Latent Diffusion for Dynamic Garment Deformations

• Authors: Antoine Dumoulin, Adnane Boukhayma, Laurence Boissieux, Bharath Bhushan Damodaran, Pierre Hellier, Stefanie Wuhrer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03468
• Pdf URL: https://arxiv.org/pdf/2504.03468
• Copy Paste: [[2504.03468]] D-Garment: Physics-Conditioned Latent Diffusion for Dynamic Garment Deformations(https://arxiv.org/abs/2504.03468)
• Keywords: diffusion, generative
• Abstract: Adjusting and deforming 3D garments to body shapes, body motion, and cloth material is an important problem in virtual and augmented reality. Applications are numerous, ranging from virtual change rooms to the entertainment and gaming industry. This problem is challenging as garment dynamics influence geometric details such as wrinkling patterns, which depend on physical input including the wearer's body shape and motion, as well as cloth material features. Existing work studies learning-based modeling techniques to generate garment deformations from example data, and physics-inspired simulators to generate realistic garment dynamics. We propose here a learning-based approach trained on data generated with a physics-based simulator. Compared to prior work, our 3D generative model learns garment deformations for loose cloth geometry, especially for large deformations and dynamic wrinkles driven by body motion and cloth material. Furthermore, the model can be efficiently fitted to observations captured using vision sensors. We propose to leverage the capability of diffusion models to learn fine-scale detail: we model the 3D garment in a 2D parameter space, and learn a latent diffusion model using this representation independent from the mesh resolution. This allows to condition global and local geometric information with body and material information. We quantitatively and qualitatively evaluate our method on both simulated data and data captured with a multi-view acquisition platform. Compared to strong baselines, our method is more accurate in terms of Chamfer distance.

Title: Dynamic Importance in Diffusion U-Net for Enhanced Image Synthesis

• Authors: Xi Wang, Ziqi He, Yang Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03471
• Pdf URL: https://arxiv.org/pdf/2504.03471
• Copy Paste: [[2504.03471]] Dynamic Importance in Diffusion U-Net for Enhanced Image Synthesis(https://arxiv.org/abs/2504.03471)
• Keywords: diffusion
• Abstract: Traditional diffusion models typically employ a U-Net architecture. Previous studies have unveiled the roles of attention blocks in the U-Net. However, they overlook the dynamic evolution of their importance during the inference process, which hinders their further exploitation to improve image applications. In this study, we first theoretically proved that, re-weighting the outputs of the Transformer blocks within the U-Net is a "free lunch" for improving the signal-to-noise ratio during the sampling process. Next, we proposed Importance Probe to uncover and quantify the dynamic shifts in importance of the Transformer blocks throughout the denoising process. Finally, we design an adaptive importance-based re-weighting schedule tailored to specific image generation and editing tasks. Experimental results demonstrate that, our approach significantly improves the efficiency of the inference process, and enhances the aesthetic quality of the samples with identity consistency. Our method can be seamlessly integrated into any U-Net-based architecture. Code: this https URL

Title: Multi-encoder nnU-Net outperforms Transformer models with self-supervised pretraining

• Authors: Seyedeh Sahar Taheri Otaghsara, Reza Rahmanzadeh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03474
• Pdf URL: https://arxiv.org/pdf/2504.03474
• Copy Paste: [[2504.03474]] Multi-encoder nnU-Net outperforms Transformer models with self-supervised pretraining(https://arxiv.org/abs/2504.03474)
• Keywords: self-supervised
• Abstract: This study addresses the essential task of medical image segmentation, which involves the automatic identification and delineation of anatomical structures and pathological regions in medical images. Accurate segmentation is crucial in radiology, as it aids in the precise localization of abnormalities such as tumors, thereby enabling effective diagnosis, treatment planning, and monitoring of disease progression. Specifically, the size, shape, and location of tumors can significantly influence clinical decision-making and therapeutic strategies, making accurate segmentation a key component of radiological workflows. However, challenges posed by variations in MRI modalities, image artifacts, and the scarcity of labeled data complicate the segmentation task and impact the performance of traditional models. To overcome these limitations, we propose a novel self-supervised learning Multi-encoder nnU-Net architecture designed to process multiple MRI modalities independently through separate encoders. This approach allows the model to capture modality-specific features before fusing them for the final segmentation, thus improving accuracy. Our Multi-encoder nnU-Net demonstrates exceptional performance, achieving a Dice Similarity Coefficient (DSC) of 93.72%, which surpasses that of other models such as vanilla nnU-Net, SegResNet, and Swin UNETR. By leveraging the unique information provided by each modality, the model enhances segmentation tasks, particularly in scenarios with limited annotated data. Evaluations highlight the effectiveness of this architecture in improving tumor segmentation outcomes.

Title: BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution

• Authors: Zihao He, Shengchuan Zhang, Runze Hu, Yunhang Shen, Yan Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03490
• Pdf URL: https://arxiv.org/pdf/2504.03490
• Copy Paste: [[2504.03490]] BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution(https://arxiv.org/abs/2504.03490)
• Keywords: diffusion
• Abstract: Super-resolution (SR) techniques are critical for enhancing image quality, particularly in scenarios where high-resolution imagery is essential yet limited by hardware constraints. Existing diffusion models for SR have relied predominantly on Gaussian models for noise generation, which often fall short when dealing with the complex and variable texture inherent in natural scenes. To address these deficiencies, we introduce the Bayesian Uncertainty Guided Diffusion Probabilistic Model (BUFF). BUFF distinguishes itself by incorporating a Bayesian network to generate high-resolution uncertainty masks. These masks guide the diffusion process, allowing for the adjustment of noise intensity in a manner that is both context-aware and adaptive. This novel approach not only enhances the fidelity of super-resolved images to their original high-resolution counterparts but also significantly mitigates artifacts and blurring in areas characterized by complex textures and fine details. The model demonstrates exceptional robustness against complex noise patterns and showcases superior adaptability in handling textures and edges within images. Empirical evidence, supported by visual results, illustrates the model's robustness, especially in challenging scenarios, and its effectiveness in addressing common SR issues such as blurring. Experimental evaluations conducted on the DIV2K dataset reveal that BUFF achieves a notable improvement, with a +0.61 increase compared to baseline in SSIM on BSD100, surpassing traditional diffusion approaches by an average additional +0.20dB PSNR gain. These findings underscore the potential of Bayesian methods in enhancing diffusion processes for SR, paving the way for future advancements in the field.

Title: Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomography

• Authors: Luis Barba, Johannes Kirschner, Tomas Aidukas, Manuel Guizar-Sicairos, Benjamín Béjar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.03491
• Pdf URL: https://arxiv.org/pdf/2504.03491
• Copy Paste: [[2504.03491]] Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomography(https://arxiv.org/abs/2504.03491)
• Keywords: diffusion, generative
• Abstract: We introduce Diffusion Active Learning, a novel approach that combines generative diffusion modeling with data-driven sequential experimental design to adaptively acquire data for inverse problems. Although broadly applicable, we focus on scientific computed tomography (CT) for experimental validation, where structured prior datasets are available, and reducing data requirements directly translates to shorter measurement times and lower X-ray doses. We first pre-train an unconditional diffusion model on domain-specific CT reconstructions. The diffusion model acts as a learned prior that is data-dependent and captures the structure of the underlying data distribution, which is then used in two ways: It drives the active learning process and also improves the quality of the reconstructions. During the active learning loop, we employ a variant of diffusion posterior sampling to generate conditional data samples from the posterior distribution, ensuring consistency with the current measurements. Using these samples, we quantify the uncertainty in the current estimate to select the most informative next measurement. Our results show substantial reductions in data acquisition requirements, corresponding to lower X-ray doses, while simultaneously improving image reconstruction quality across multiple real-world tomography datasets.

Title: LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders

• Authors: Ilan Naiman, Emanuel Ben-Baruch, Oron Anschel, Alon Shoshan, Igor Kviatkovsky, Manoj Aggarwal, Gerard Medioni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03501
• Pdf URL: https://arxiv.org/pdf/2504.03501
• Copy Paste: [[2504.03501]] LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders(https://arxiv.org/abs/2504.03501)
• Keywords: self-supervised
• Abstract: In this work, we introduce long-video masked-embedding autoencoders (LV-MAE), a self-supervised learning framework for long video representation. Our approach treats short- and long-span dependencies as two separate tasks. Such decoupling allows for a more intuitive video processing where short-span spatiotemporal primitives are first encoded and are then used to capture long-range dependencies across consecutive video segments. To achieve this, we leverage advanced off-the-shelf multimodal encoders to extract representations from short segments within the long video, followed by pre-training a masked-embedding autoencoder capturing high-level interactions across segments. LV-MAE is highly efficient to train and enables the processing of much longer videos by alleviating the constraint on the number of input frames. Furthermore, unlike existing methods that typically pre-train on short-video datasets, our approach offers self-supervised pre-training using long video samples (e.g., 20+ minutes video clips) at scale. Using LV-MAE representations, we achieve state-of-the-art results on three long-video benchmarks -- LVU, COIN, and Breakfast -- employing only a simple classification head for either attentive or linear probing. Finally, to assess LV-MAE pre-training and visualize its reconstruction quality, we leverage the video-language aligned space of short video representations to monitor LV-MAE through video-text retrieval.

Title: RANa: Retrieval-Augmented Navigation

• Authors: Gianluca Monaci, Rafael S. Rezende, Romain Deffayet, Gabriela Csurka, Guillaume Bono, Hervé Déjean, Stéphane Clinchant, Christian Wolf
• Subjects: cs.CV, cs.IR, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.03524
• Pdf URL: https://arxiv.org/pdf/2504.03524
• Copy Paste: [[2504.03524]] RANa: Retrieval-Augmented Navigation(https://arxiv.org/abs/2504.03524)
• Keywords: foundation model
• Abstract: Methods for navigation based on large-scale learning typically treat each episode as a new problem, where the agent is spawned with a clean memory in an unknown environment. While these generalization capabilities to an unknown environment are extremely important, we claim that, in a realistic setting, an agent should have the capacity of exploiting information collected during earlier robot operations. We address this by introducing a new retrieval-augmented agent, trained with RL, capable of querying a database collected from previous episodes in the same environment and learning how to integrate this additional context information. We introduce a unique agent architecture for the general navigation task, evaluated on ObjectNav, ImageNav and Instance-ImageNav. Our retrieval and context encoding methods are data-driven and heavily employ vision foundation models (FM) for both semantic and geometric understanding. We propose new benchmarks for these settings and we show that retrieval allows zero-shot transfer across tasks and environments while significantly improving performance.

Title: HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction via Gaussian Restoration

• Authors: Boyuan Wang, Runqi Ouyang, Xiaofeng Wang, Zheng Zhu, Guosheng Zhao, Chaojun Ni, Guan Huang, Lihong Liu, Xingang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03536
• Pdf URL: https://arxiv.org/pdf/2504.03536
• Copy Paste: [[2504.03536]] HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction via Gaussian Restoration(https://arxiv.org/abs/2504.03536)
• Keywords: generative
• Abstract: Single-image human reconstruction is vital for digital human modeling applications but remains an extremely challenging task. Current approaches rely on generative models to synthesize multi-view images for subsequent 3D reconstruction and animation. However, directly generating multiple views from a single human image suffers from geometric inconsistencies, resulting in issues like fragmented or blurred limbs in the reconstructed models. To tackle these limitations, we introduce \textbf{HumanDreamer-X}, a novel framework that integrates multi-view human generation and reconstruction into a unified pipeline, which significantly enhances the geometric consistency and visual fidelity of the reconstructed 3D models. In this framework, 3D Gaussian Splatting serves as an explicit 3D representation to provide initial geometry and appearance priority. Building upon this foundation, \textbf{HumanFixer} is trained to restore 3DGS renderings, which guarantee photorealistic results. Furthermore, we delve into the inherent challenges associated with attention mechanisms in multi-view human generation, and propose an attention modulation strategy that effectively enhances geometric details identity consistency across multi-view. Experimental results demonstrate that our approach markedly improves generation and reconstruction PSNR quality metrics by 16.45% and 12.65%, respectively, achieving a PSNR of up to 25.62 dB, while also showing generalization capabilities on in-the-wild data and applicability to various human reconstruction backbone models.

Title: Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing

• Authors: Mayank Kothyari, Sunita Sarawagi, Soumen Chakrabarti, Gaurav Arora, Srujana Merugu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.03541
• Pdf URL: https://arxiv.org/pdf/2504.03541
• Copy Paste: [[2504.03541]] Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing(https://arxiv.org/abs/2504.03541)
• Keywords: in-context
• Abstract: LLMs are increasingly used as seq2seq translators from natural language utterances to structured programs, a process called semantic interpretation. Unlike atomic labels or token sequences, programs are naturally represented as abstract syntax trees (ASTs). Such structured representation raises novel issues related to the design and selection of in-context examples (ICEs) presented to the LLM. We focus on decomposing the pool of available ICE trees into fragments, some of which may be better suited to solving the test instance. Next, we propose how to use (additional invocations of) an LLM with prompted syntax constraints to automatically map the fragments to corresponding utterances. Finally, we adapt and extend a recent method for diverse ICE selection to work with whole and fragmented ICE instances. We evaluate our system, SCUD4ICL, on popular diverse semantic parsing benchmarks, showing visible accuracy gains from our proposed decomposed diverse demonstration method. Benefits are particularly notable for smaller LLMs, ICE pools having larger labeled trees, and programs in lower resource languages.

Title: PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector

• Authors: Kaidong Li, Tianxiao Zhang, Kuan-Chuan Peng, Guanghui Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03563
• Pdf URL: https://arxiv.org/pdf/2504.03563
• Copy Paste: [[2504.03563]] PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector(https://arxiv.org/abs/2504.03563)
• Keywords: foundation model
• Abstract: 3D object detection is crucial for autonomous driving, leveraging both LiDAR point clouds for precise depth information and camera images for rich semantic information. Therefore, the multi-modal methods that combine both modalities offer more robust detection results. However, efficiently fusing LiDAR points and images remains challenging due to the domain gaps. In addition, the performance of many models is limited by the amount of high quality labeled data, which is expensive to create. The recent advances in foundation models, which use large-scale pre-training on different modalities, enable better multi-modal fusion. Combining the prompt engineering techniques for efficient training, we propose the Prompted Foundational 3D Detector (PF3Det), which integrates foundation model encoders and soft prompts to enhance LiDAR-camera feature fusion. PF3Det achieves the state-of-the-art results under limited training data, improving NDS by 1.19% and mAP by 2.42% on the nuScenes dataset, demonstrating its efficiency in 3D detection.

Title: AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing

• Authors: Niu Lian, Jun Li, Jinpeng Wang, Ruisheng Luo, Yaowei Wang, Shu-Tao Xia, Bin Chen
• Subjects: cs.CV, cs.IR, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.03587
• Pdf URL: https://arxiv.org/pdf/2504.03587
• Copy Paste: [[2504.03587]] AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing(https://arxiv.org/abs/2504.03587)
• Keywords: self-supervised
• Abstract: Self-Supervised Video Hashing (SSVH) compresses videos into hash codes for efficient indexing and retrieval using unlabeled training videos. Existing approaches rely on random frame sampling to learn video features and treat all frames equally. This results in suboptimal hash codes, as it ignores frame-specific information density and reconstruction difficulty. To address this limitation, we propose a new framework, termed AutoSSVH, that employs adversarial frame sampling with hash-based contrastive learning. Our adversarial sampling strategy automatically identifies and selects challenging frames with richer information for reconstruction, enhancing encoding capability. Additionally, we introduce a hash component voting strategy and a point-to-set (P2Set) hash-based contrastive objective, which help capture complex inter-video semantic relationships in the Hamming space and improve the discriminability of learned hash codes. Extensive experiments demonstrate that AutoSSVH achieves superior retrieval efficacy and efficiency compared to state-of-the-art approaches. Code is available at this https URL.

Title: Multimodal Diffusion Bridge with Attention-Based SAR Fusion for Satellite Image Cloud Removal

• Authors: Yuyang Hu, Suhas Lohit, Ulugbek S. Kamilov, Tim K. Marks
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03607
• Pdf URL: https://arxiv.org/pdf/2504.03607
• Copy Paste: [[2504.03607]] Multimodal Diffusion Bridge with Attention-Based SAR Fusion for Satellite Image Cloud Removal(https://arxiv.org/abs/2504.03607)
• Keywords: diffusion
• Abstract: Deep learning has achieved some success in addressing the challenge of cloud removal in optical satellite images, by fusing with synthetic aperture radar (SAR) images. Recently, diffusion models have emerged as powerful tools for cloud removal, delivering higher-quality estimation by sampling from cloud-free distributions, compared to earlier methods. However, diffusion models initiate sampling from pure Gaussian noise, which complicates the sampling trajectory and results in suboptimal performance. Also, current methods fall short in effectively fusing SAR and optical data. To address these limitations, we propose Diffusion Bridges for Cloud Removal, DB-CR, which directly bridges between the cloudy and cloud-free image distributions. In addition, we propose a novel multimodal diffusion bridge architecture with a two-branch backbone for multimodal image restoration, incorporating an efficient backbone and dedicated cross-modality fusion blocks to effectively extract and fuse features from synthetic aperture radar (SAR) and optical images. By formulating cloud removal as a diffusion-bridge problem and leveraging this tailored architecture, DB-CR achieves high-fidelity results while being computationally efficient. We evaluated DB-CR on the SEN12MS-CR cloud-removal dataset, demonstrating that it achieves state-of-the-art results.

Title: AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset

• Authors: Bingxiang He, Wenbin Zhang, Jiaxi Song, Cheng Qian, Zixuan Fu, Bowen Sun, Ning Ding, Haiwen Hong, Longtao Huang, Hui Xue, Ganqu Cui, Wanxiang Che, Zhiyuan Liu, Maosong Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.03612
• Pdf URL: https://arxiv.org/pdf/2504.03612
• Copy Paste: [[2504.03612]] AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset(https://arxiv.org/abs/2504.03612)
• Keywords: generative
• Abstract: Preference learning is critical for aligning large language models (LLMs) with human values, yet its success hinges on high-quality datasets comprising three core components: Preference \textbf{A}nnotations, \textbf{I}nstructions, and \textbf{R}esponse Pairs. Current approaches conflate these components, obscuring their individual impacts and hindering systematic optimization. In this work, we propose \textbf{AIR}, a component-wise analysis framework that systematically isolates and optimizes each component while evaluating their synergistic effects. Through rigorous experimentation, AIR reveals actionable principles: annotation simplicity (point-wise generative scoring), instruction inference stability (variance-based filtering across LLMs), and response pair quality (moderate margins + high absolute scores). When combined, these principles yield +5.3 average gains over baseline method, even with only 14k high-quality pairs. Our work shifts preference dataset design from ad hoc scaling to component-aware optimization, offering a blueprint for efficient, reproducible alignment.

Title: Autonomous and Self-Adapting System for Synthetic Media Detection and Attribution

• Authors: Aref Azizpour, Tai D. Nguyen, Matthew C. Stamm
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03615
• Pdf URL: https://arxiv.org/pdf/2504.03615
• Copy Paste: [[2504.03615]] Autonomous and Self-Adapting System for Synthetic Media Detection and Attribution(https://arxiv.org/abs/2504.03615)
• Keywords: generative
• Abstract: Rapid advances in generative AI have enabled the creation of highly realistic synthetic images, which, while beneficial in many domains, also pose serious risks in terms of disinformation, fraud, and other malicious applications. Current synthetic image identification systems are typically static, relying on feature representations learned from known generators; as new generative models emerge, these systems suffer from severe performance degradation. In this paper, we introduce the concept of an autonomous self-adaptive synthetic media identification system -- one that not only detects synthetic images and attributes them to known sources but also autonomously identifies and incorporates novel generators without human intervention. Our approach leverages an open-set identification strategy with an evolvable embedding space that distinguishes between known and unknown sources. By employing an unsupervised clustering method to aggregate unknown samples into high-confidence clusters and continuously refining its decision boundaries, our system maintains robust detection and attribution performance even as the generative landscape evolves. Extensive experiments demonstrate that our method significantly outperforms existing approaches, marking a crucial step toward universal, adaptable forensic systems in the era of rapidly advancing generative models.

Title: Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task

• Authors: Leonardo Ranaldi, Barry Haddow, Alexandra Birch
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.03616
• Pdf URL: https://arxiv.org/pdf/2504.03616
• Copy Paste: [[2504.03616]] Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task(https://arxiv.org/abs/2504.03616)
• Keywords: in-context
• Abstract: Retrieval-augmented generation (RAG) has become a cornerstone of contemporary NLP, enhancing large language models (LLMs) by allowing them to access richer factual contexts through in-context retrieval. While effective in monolingual settings, especially in English, its use in multilingual tasks remains unexplored. This paper investigates the effectiveness of RAG across multiple languages by proposing novel approaches for multilingual open-domain question-answering. We evaluate the performance of various multilingual RAG strategies, including question-translation (tRAG), which translates questions into English before retrieval, and Multilingual RAG (MultiRAG), where retrieval occurs directly across multiple languages. Our findings reveal that tRAG, while useful, suffers from limited coverage. In contrast, MultiRAG improves efficiency by enabling multilingual retrieval but introduces inconsistencies due to cross-lingual variations in the retrieved content. To address these issues, we propose Crosslingual RAG (CrossRAG), a method that translates retrieved documents into a common language (e.g., English) before generating the response. Our experiments show that CrossRAG significantly enhances performance on knowledge-intensive tasks, benefiting both high-resource and low-resource languages.

Title: VISTA-OCR: Towards generative and interactive end to end OCR models

• Authors: Laziz Hamdi, Amine Tamasna, Pascal Boisson, Thierry Paquet
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03621
• Pdf URL: https://arxiv.org/pdf/2504.03621
• Copy Paste: [[2504.03621]] VISTA-OCR: Towards generative and interactive end to end OCR models(https://arxiv.org/abs/2504.03621)
• Keywords: generative
• Abstract: We introduce \textbf{VISTA-OCR} (Vision and Spatially-aware Text Analysis OCR), a lightweight architecture that unifies text detection and recognition within a single generative model. Unlike conventional methods that require separate branches with dedicated parameters for text recognition and detection, our approach leverages a Transformer decoder to sequentially generate text transcriptions and their spatial coordinates in a unified branch. Built on an encoder-decoder architecture, VISTA-OCR is progressively trained, starting with the visual feature extraction phase, followed by multitask learning with multimodal token generation. To address the increasing demand for versatile OCR systems capable of advanced tasks, such as content-based text localization \ref{content_based_localization}, we introduce new prompt-controllable OCR tasks during this http URL enhance the model's capabilities, we built a new dataset composed of real-world examples enriched with bounding box annotations and synthetic samples. Although recent Vision Large Language Models (VLLMs) can efficiently perform these tasks, their high computational cost remains a barrier for practical deployment. In contrast, our VISTA$_{\text{omni}}$ variant processes both handwritten and printed documents with only 150M parameters, interactively, by prompting. Extensive experiments on multiple datasets demonstrate that VISTA-OCR achieves better performance compared to state-of-the-art specialized models on standard OCR tasks while showing strong potential for more sophisticated OCR applications, addressing the growing need for interactive OCR systems. All code and annotations for VISTA-OCR will be made publicly available upon acceptance.

Title: Quantifying the uncertainty of model-based synthetic image quality metrics

• Authors: Ciaran Bench, Spencer A. Thomas
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.03623
• Pdf URL: https://arxiv.org/pdf/2504.03623
• Copy Paste: [[2504.03623]] Quantifying the uncertainty of model-based synthetic image quality metrics(https://arxiv.org/abs/2504.03623)
• Keywords: diffusion
• Abstract: The quality of synthetically generated images (e.g. those produced by diffusion models) are often evaluated using information about image contents encoded by pretrained auxiliary models. For example, the Fréchet Inception Distance (FID) uses embeddings from an InceptionV3 model pretrained to classify ImageNet. The effectiveness of this feature embedding model has considerable impact on the trustworthiness of the calculated metric (affecting its suitability in several domains, including medical imaging). Here, uncertainty quantification (UQ) is used to provide a heuristic measure of the trustworthiness of the feature embedding model and an FID-like metric called the Fréchet Autoencoder Distance (FAED). We apply Monte Carlo dropout to a feature embedding model (convolutional autoencoder) to model the uncertainty in its embeddings. The distribution of embeddings for each input are then used to compute a distribution of FAED values. We express uncertainty as the predictive variance of the embeddings as well as the standard deviation of the computed FAED values. We find that their magnitude correlates with the extent to which the inputs are out-of-distribution to the model's training data, providing some validation of its ability to assess the trustworthiness of the FAED.