2025-04-25

Title: Bidirectional Mamba for Single-Cell Data: Efficient Context Learning with Biological Fidelity

• Authors: Cong Qi, Hanzhang Fang, Tianxing Hu, Siqi Jiang, Wei Zhi
• Subjects: cs.CL, cs.LG, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2504.16956
• Pdf URL: https://arxiv.org/pdf/2504.16956
• Copy Paste: [[2504.16956]] Bidirectional Mamba for Single-Cell Data: Efficient Context Learning with Biological Fidelity(https://arxiv.org/abs/2504.16956)
• Keywords: foundation model
• Abstract: Single-cell RNA sequencing (scRNA-seq) enables high-resolution analysis of cellular heterogeneity, but its complexity, which is marked by high dimensionality, sparsity, and batch effects, which poses major computational challenges. Transformer-based models have made significant advances in this domain but are often limited by their quadratic complexity and suboptimal handling of long-range dependencies. In this work, we introduce GeneMamba, a scalable and efficient foundation model for single-cell transcriptomics built on state space modeling. Leveraging the Bi-Mamba architecture, GeneMamba captures bidirectional gene context with linear-time complexity, offering substantial computational gains over transformer baselines. The model is pretrained on nearly 30 million cells and incorporates biologically informed objectives, including pathway-aware contrastive loss and rank-based gene encoding. We evaluate GeneMamba across diverse tasks, including multi-batch integration, cell type annotation, and gene-gene correlation, demonstrating strong performance, interpretability, and robustness. These results position GeneMamba as a practical and powerful alternative to transformer-based methods, advancing the development of biologically grounded, scalable tools for large-scale single-cell data analysis.

Title: Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applications

• Authors: Hossein Ahmadi, Sajjad Emdadi Mahdimahalleh, Arman Farahat, Banafsheh Saffari
• Subjects: cs.LG, cs.AI, cs.CV, eess.SP
• Abstract URL: https://arxiv.org/abs/2504.16972
• Pdf URL: https://arxiv.org/pdf/2504.16972
• Copy Paste: [[2504.16972]] Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applications(https://arxiv.org/abs/2504.16972)
• Keywords: self-supervised, anomaly
• Abstract: The rapid growth of unlabeled time-series data in domains such as wireless communications, radar, biomedical engineering, and the Internet of Things (IoT) has driven advancements in unsupervised learning. This review synthesizes recent progress in applying autoencoders and vision transformers for unsupervised signal analysis, focusing on their architectures, applications, and emerging trends. We explore how these models enable feature extraction, anomaly detection, and classification across diverse signal types, including electrocardiograms, radar waveforms, and IoT sensor data. The review highlights the strengths of hybrid architectures and self-supervised learning, while identifying challenges in interpretability, scalability, and domain generalization. By bridging methodological innovations and practical applications, this work offers a roadmap for developing robust, adaptive models for signal intelligence.

Title: Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation

• Authors: Luca Moroni, Giovanni Puccetti, Pere-Lluis Huguet Cabot, Andrei Stefan Bejgu, Edoardo Barba, Alessio Miaschi, Felice Dell'Orletta, Andrea Esuli, Roberto Navigli
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.17025
• Pdf URL: https://arxiv.org/pdf/2504.17025
• Copy Paste: [[2504.17025]] Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation(https://arxiv.org/abs/2504.17025)
• Keywords: generative
• Abstract: The number of pretrained Large Language Models (LLMs) is increasing steadily, though the majority are designed predominantly for the English language. While state-of-the-art LLMs can handle other languages, due to language contamination or some degree of multilingual pretraining data, they are not optimized for non-English languages, leading to inefficient encoding (high token "fertility") and slower inference speed. In this work, we thoroughly compare a variety of vocabulary adaptation techniques for optimizing English LLMs for the Italian language, and put forward Semantic Alignment Vocabulary Adaptation (SAVA), a novel method that leverages neural mapping for vocabulary substitution. SAVA achieves competitive performance across multiple downstream tasks, enhancing grounded alignment strategies. We adapt two LLMs: Mistral-7b-v0.1, reducing token fertility by 25\%, and Llama-3.1-8B, optimizing the vocabulary and reducing the number of parameters by 1 billion. We show that, following the adaptation of the vocabulary, these models can recover their performance with a relatively limited stage of continual training on the target language. Finally, we test the capabilities of the adapted models on various multi-choice and generative tasks.

Title: Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation

• Authors: Rahul Vishwakarma
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17058
• Pdf URL: https://arxiv.org/pdf/2504.17058
• Copy Paste: [[2504.17058]] Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation(https://arxiv.org/abs/2504.17058)
• Keywords: generative
• Abstract: The generation of high-quality synthetic data presents significant challenges in machine learning research, particularly regarding statistical fidelity and uncertainty quantification. Existing generative models produce compelling synthetic samples but lack rigorous statistical guarantees about their relation to the underlying data distribution, limiting their applicability in critical domains requiring robust error bounds. We address this fundamental limitation by presenting a novel framework that incorporates conformal prediction methodologies into Generative Adversarial Networks (GANs). By integrating multiple conformal prediction paradigms including Inductive Conformal Prediction (ICP), Mondrian Conformal Prediction, Cross-Conformal Prediction, and Venn-Abers Predictors, we establish distribution-free uncertainty quantification in generated samples. This approach, termed Conformalized GAN (cGAN), demonstrates enhanced calibration properties while maintaining the generative power of traditional GANs, producing synthetic data with provable statistical guarantees. We provide rigorous mathematical proofs establishing finite-sample validity guarantees and asymptotic efficiency properties, enabling the reliable application of synthetic data in high-stakes domains including healthcare, finance, and autonomous systems.

Title: PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation

• Authors: Xinqi Xiong, Andrea Dunn Beltran, Jun Myeong Choi, Marc Niethammer, Roni Sengupta
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17067
• Pdf URL: https://arxiv.org/pdf/2504.17067
• Copy Paste: [[2504.17067]] PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation(https://arxiv.org/abs/2504.17067)
• Keywords: diffusion
• Abstract: Accurate depth estimation enhances endoscopy navigation and diagnostics, but obtaining ground-truth depth in clinical settings is challenging. Synthetic datasets are often used for training, yet the domain gap limits generalization to real data. We propose a novel image-to-image translation framework that preserves structure while generating realistic textures from clinical data. Our key innovation integrates Stable Diffusion with ControlNet, conditioned on a latent representation extracted from a Per-Pixel Shading (PPS) map. PPS captures surface lighting effects, providing a stronger structural constraint than depth maps. Experiments show our approach produces more realistic translations and improves depth estimation over GAN-based MI-CycleGAN. Our code is publicly accessible at this https URL.

Title: In-Context Learning can distort the relationship between sequence likelihoods and biological fitness

• Authors: Pranav Kantroo, Günter P. Wagner, Benjamin B. Machta
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2504.17068
• Pdf URL: https://arxiv.org/pdf/2504.17068
• Copy Paste: [[2504.17068]] In-Context Learning can distort the relationship between sequence likelihoods and biological fitness(https://arxiv.org/abs/2504.17068)
• Keywords: in-context
• Abstract: Language models have emerged as powerful predictors of the viability of biological sequences. During training these models learn the rules of the grammar obeyed by sequences of amino acids or nucleotides. Once trained, these models can take a sequence as input and produce a likelihood score as an output; a higher likelihood implies adherence to the learned grammar and correlates with experimental fitness measurements. Here we show that in-context learning can distort the relationship between fitness and likelihood scores of sequences. This phenomenon most prominently manifests as anomalously high likelihood scores for sequences that contain repeated motifs. We use protein language models with different architectures trained on the masked language modeling objective for our experiments, and find transformer-based models to be particularly vulnerable to this effect. This behavior is mediated by a look-up operation where the model seeks the identity of the masked position by using the other copy of the repeated motif as a reference. This retrieval behavior can override the model's learned priors. This phenomenon persists for imperfectly repeated sequences, and extends to other kinds of biologically relevant features such as reversed complement motifs in RNA sequences that fold into hairpin structures.

Title: Conditional Diffusion-Based Retrieval of Atmospheric CO2 from Earth Observing Spectroscopy

• Authors: William R. Keely, Otto Lamminpää, Steffen Mauceri, Sean M. R. Crowell, Christopher W. O'Dell, Gregory R. McGarragh
• Subjects: cs.LG, astro-ph.IM
• Abstract URL: https://arxiv.org/abs/2504.17074
• Pdf URL: https://arxiv.org/pdf/2504.17074
• Copy Paste: [[2504.17074]] Conditional Diffusion-Based Retrieval of Atmospheric CO2 from Earth Observing Spectroscopy(https://arxiv.org/abs/2504.17074)
• Keywords: diffusion
• Abstract: Satellite-based estimates of greenhouse gas (GHG) properties from observations of reflected solar spectra are integral for understanding and monitoring complex terrestrial systems and their impact on the carbon cycle due to their near global coverage. Known as retrieval, making GHG concentration estimations from these observations is a non-linear Bayesian inverse problem, which is operationally solved using a computationally expensive algorithm called Optimal Estimation (OE), providing a Gaussian approximation to a non-Gaussian posterior. This leads to issues in solver algorithm convergence, and to unrealistically confident uncertainty estimates for the retrieved quantities. Upcoming satellite missions will provide orders of magnitude more data than the current constellation of GHG observers. Development of fast and accurate retrieval algorithms with robust uncertainty quantification is critical. Doing so stands to provide substantial climate impact of moving towards the goal of near continuous real-time global monitoring of carbon sources and sinks which is essential for policy making. To achieve this goal, we propose a diffusion-based approach to flexibly retrieve a Gaussian or non-Gaussian posterior, for NASA's Orbiting Carbon Observatory-2 spectrometer, while providing a substantial computational speed-up over the current operational state-of-the-art.

Title: Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection

• Authors: Jens Petersen, Davide Abati, Amirhossein Habibian, Auke Wiggers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17076
• Pdf URL: https://arxiv.org/pdf/2504.17076
• Copy Paste: [[2504.17076]] Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection(https://arxiv.org/abs/2504.17076)
• Keywords: generative
• Abstract: Generative image models are increasingly being used for training data augmentation in vision tasks. In the context of automotive object detection, methods usually focus on producing augmented frames that look as realistic as possible, for example by replacing real objects with generated ones. Others try to maximize the diversity of augmented frames, for example by pasting lots of generated objects onto existing backgrounds. Both perspectives pay little attention to the locations of objects in the scene. Frame layouts are either reused with little or no modification, or they are random and disregard realism entirely. In this work, we argue that optimal data augmentation should also include realistic augmentation of layouts. We introduce a scene-aware probabilistic location model that predicts where new objects can realistically be placed in an existing scene. By then inpainting objects in these locations with a generative model, we obtain much stronger augmentation performance than existing approaches. We set a new state of the art for generative data augmentation on two automotive object detection tasks, achieving up to $2.8\times$ higher gains than the best competing approach ($+1.4$ vs. $+0.5$ mAP boost). We also demonstrate significant improvements for instance segmentation.

Title: MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluation

• Authors: Chanhee Park, Hyeonseok Moon, Chanjun Park, Heuiseok Lim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17137
• Pdf URL: https://arxiv.org/pdf/2504.17137
• Copy Paste: [[2504.17137]] MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluation(https://arxiv.org/abs/2504.17137)
• Keywords: generative
• Abstract: Retrieval-Augmented Generation (RAG) has gained prominence as an effective method for enhancing the generative capabilities of Large Language Models (LLMs) through the incorporation of external knowledge. However, the evaluation of RAG systems remains a challenge, due to the intricate interplay between retrieval and generation components. This limitation has resulted in a scarcity of benchmarks that facilitate a detailed, component-specific assessment. In this work, we present MIRAGE, a Question Answering dataset specifically designed for RAG evaluation. MIRAGE consists of 7,560 curated instances mapped to a retrieval pool of 37,800 entries, enabling an efficient and precise evaluation of both retrieval and generation tasks. We also introduce novel evaluation metrics aimed at measuring RAG adaptability, encompassing dimensions such as noise vulnerability, context acceptability, context insensitivity, and context misinterpretation. Through comprehensive experiments across various retriever-LLM configurations, we provide new insights into the optimal alignment of model pairs and the nuanced dynamics within RAG systems. The dataset and evaluation code are publicly available, allowing for seamless integration and customization in diverse research settings\footnote{The MIRAGE code and data are available at this https URL.

Title: A Genealogy of Multi-Sensor Foundation Models in Remote Sensing

• Authors: Kevin Lane, Morteza Karimzadeh
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17177
• Pdf URL: https://arxiv.org/pdf/2504.17177
• Copy Paste: [[2504.17177]] A Genealogy of Multi-Sensor Foundation Models in Remote Sensing(https://arxiv.org/abs/2504.17177)
• Keywords: foundation model
• Abstract: Foundation models have garnered increasing attention for representation learning in remote sensing, primarily adopting approaches that have demonstrated success in computer vision with minimal domain-specific modification. However, the development and application of foundation models in this field are still burgeoning, as there are a variety of competing approaches that each come with significant benefits and drawbacks. This paper examines these approaches along with their roots in the computer vision field in order to characterize potential advantages and pitfalls while outlining future directions to further improve remote sensing-specific foundation models. We discuss the quality of the learned representations and methods to alleviate the need for massive compute resources. We place emphasis on the multi-sensor aspect of Earth observations, and the extent to which existing approaches leverage multiple sensors in training foundation models in relation to multi-modal foundation models. Finally, we identify opportunities for further harnessing the vast amounts of unlabeled, seasonal, and multi-sensor remote sensing observations.

Title: Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation

• Authors: Phillip Y. Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, Minhyuk Sung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17207
• Pdf URL: https://arxiv.org/pdf/2504.17207
• Copy Paste: [[2504.17207]] Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation(https://arxiv.org/abs/2504.17207)
• Keywords: foundation model
• Abstract: We present a framework for perspective-aware reasoning in vision-language models (VLMs) through mental imagery simulation. Perspective-taking, the ability to perceive an environment or situation from an alternative viewpoint, is a key benchmark for human-level visual understanding, essential for environmental interaction and collaboration with autonomous agents. Despite advancements in spatial reasoning within VLMs, recent research has shown that modern VLMs significantly lack perspective-aware reasoning capabilities and exhibit a strong bias toward egocentric interpretations. To bridge the gap between VLMs and human perception, we focus on the role of mental imagery, where humans perceive the world through abstracted representations that facilitate perspective shifts. Motivated by this, we propose a framework for perspective-aware reasoning, named Abstract Perspective Change (APC), that effectively leverages vision foundation models, such as object detection, segmentation, and orientation estimation, to construct scene abstractions and enable perspective transformations. Our experiments on synthetic and real-image benchmarks, compared with various VLMs, demonstrate significant improvements in perspective-aware reasoning with our framework, further outperforming fine-tuned spatial reasoning models and novel-view-synthesis-based approaches.

Title: Synthetic Power Flow Data Generation Using Physics-Informed Denoising Diffusion Probabilistic Models

• Authors: Junfei Wang, Darshana Upadhyay, Marzia Zaman, Pirathayini Srikantha
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17210
• Pdf URL: https://arxiv.org/pdf/2504.17210
• Copy Paste: [[2504.17210]] Synthetic Power Flow Data Generation Using Physics-Informed Denoising Diffusion Probabilistic Models(https://arxiv.org/abs/2504.17210)
• Keywords: diffusion, generative
• Abstract: Many data-driven modules in smart grid rely on access to high-quality power flow data; however, real-world data are often limited due to privacy and operational constraints. This paper presents a physics-informed generative framework based on Denoising Diffusion Probabilistic Models (DDPMs) for synthesizing feasible power flow data. By incorporating auxiliary training and physics-informed loss functions, the proposed method ensures that the generated data exhibit both statistical fidelity and adherence to power system feasibility. We evaluate the approach on the IEEE 14-bus and 30-bus benchmark systems, demonstrating its ability to capture key distributional properties and generalize to out-of-distribution scenarios. Comparative results show that the proposed model outperforms three baseline models in terms of feasibility, diversity, and accuracy of statistical features. This work highlights the potential of integrating generative modelling into data-driven power system applications.

Title: Enhancing Variational Autoencoders with Smooth Robust Latent Encoding

• Authors: Hyomin Lee, Minseon Kim, Sangwon Jang, Jongheon Jeong, Sung Ju Hwang
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.17219
• Pdf URL: https://arxiv.org/pdf/2504.17219
• Copy Paste: [[2504.17219]] Enhancing Variational Autoencoders with Smooth Robust Latent Encoding(https://arxiv.org/abs/2504.17219)
• Keywords: diffusion, generative
• Abstract: Variational Autoencoders (VAEs) have played a key role in scaling up diffusion-based generative models, as in Stable Diffusion, yet questions regarding their robustness remain largely underexplored. Although adversarial training has been an established technique for enhancing robustness in predictive models, it has been overlooked for generative models due to concerns about potential fidelity degradation by the nature of trade-offs between performance and robustness. In this work, we challenge this presumption, introducing Smooth Robust Latent VAE (SRL-VAE), a novel adversarial training framework that boosts both generation quality and robustness. In contrast to conventional adversarial training, which focuses on robustness only, our approach smooths the latent space via adversarial perturbations, promoting more generalizable representations while regularizing with originality representation to sustain original fidelity. Applied as a post-training step on pre-trained VAEs, SRL-VAE improves image robustness and fidelity with minimal computational overhead. Experiments show that SRL-VAE improves both generation quality, in image reconstruction and text-guided image editing, and robustness, against Nightshade attacks and image editing attacks. These results establish a new paradigm, showing that adversarial training, once thought to be detrimental to generative models, can instead enhance both fidelity and robustness.

Title: Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?

• Authors: Kaidong Feng, Zhu Sun, Jie Yang, Hui Fang, Xinghua Qu, Wenyuan Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.17220
• Pdf URL: https://arxiv.org/pdf/2504.17220
• Copy Paste: [[2504.17220]] Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?(https://arxiv.org/abs/2504.17220)
• Keywords: in-context
• Abstract: LLMs are increasingly explored for bundle generation, thanks to their reasoning capabilities and knowledge. However, deploying large-scale LLMs introduces significant efficiency challenges, primarily high computational costs during fine-tuning and inference due to their massive parameterization. Knowledge distillation (KD) offers a promising solution, transferring expertise from large teacher models to compact student models. This study systematically investigates knowledge distillation approaches for bundle generation, aiming to minimize computational demands while preserving performance. We explore three critical research questions: (1) how does the format of KD impact bundle generation performance? (2) to what extent does the quantity of distilled knowledge influence performance? and (3) how do different ways of utilizing the distilled knowledge affect performance? We propose a comprehensive KD framework that (i) progressively extracts knowledge (patterns, rules, deep thoughts); (ii) captures varying quantities of distilled knowledge through different strategies; and (iii) exploits complementary LLM adaptation techniques (in-context learning, supervised fine-tuning, combination) to leverage distilled knowledge in small student models for domain-specific adaptation and enhanced efficiency. Extensive experiments provide valuable insights into how knowledge format, quantity, and utilization methodologies collectively shape LLM-based bundle generation performance, exhibiting KD's significant potential for more efficient yet effective LLM-based bundle generation.

Title: Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusion

• Authors: Mengyu Qiao, Runze Tian, Yang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17223
• Pdf URL: https://arxiv.org/pdf/2504.17223
• Copy Paste: [[2504.17223]] Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusion(https://arxiv.org/abs/2504.17223)
• Keywords: generative
• Abstract: The rapid evolution of deep generative models poses a critical challenge to deepfake detection, as detectors trained on forgery-specific artifacts often suffer significant performance degradation when encountering unseen forgeries. While existing methods predominantly rely on spatial domain analysis, frequency domain operations are primarily limited to feature-level augmentation, leaving frequency-native artifacts and spatial-frequency interactions insufficiently exploited. To address this limitation, we propose a novel detection framework that integrates multi-scale spatial-frequency analysis for universal deepfake detection. Our framework comprises three key components: (1) a local spectral feature extraction pipeline that combines block-wise discrete cosine transform with cascaded multi-scale convolutions to capture subtle spectral artifacts; (2) a global spectral feature extraction pipeline utilizing scale-invariant differential accumulation to identify holistic forgery distribution patterns; and (3) a multi-stage cross-modal fusion mechanism that incorporates shallow-layer attention enhancement and deep-layer dynamic modulation to model spatial-frequency interactions. Extensive evaluations on widely adopted benchmarks demonstrate that our method outperforms state-of-the-art deepfake detection methods in both accuracy and generalizability.

Title: Targeted AMP generation through controlled diffusion with efficient embeddings

• Authors: Diogo Soares, Leon Hetzel, Paulina Szymczak, Fabian Theis, Stephan Günnemann, Ewa Szczurek
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2504.17247
• Pdf URL: https://arxiv.org/pdf/2504.17247
• Copy Paste: [[2504.17247]] Targeted AMP generation through controlled diffusion with efficient embeddings(https://arxiv.org/abs/2504.17247)
• Keywords: diffusion, generative
• Abstract: Deep learning-based antimicrobial peptide (AMP) discovery faces critical challenges such as low experimental hit rates as well as the need for nuanced controllability and efficient modeling of peptide properties. To address these challenges, we introduce OmegAMP, a framework that leverages a diffusion-based generative model with efficient low-dimensional embeddings, precise controllability mechanisms, and novel classifiers with drastically reduced false positive rates for candidate filtering. OmegAMP enables the targeted generation of AMPs with specific physicochemical properties, activity profiles, and species-specific effectiveness. Moreover, it maximizes sample diversity while ensuring faithfulness to the underlying data distribution during generation. We demonstrate that OmegAMP achieves state-of-the-art performance across all stages of the AMP discovery pipeline, significantly advancing the potential of computational frameworks in combating antimicrobial resistance.

Title: DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks

• Authors: Yinqi Li, Hong Chang, Ruibing Hou, Shiguang Shan, Xilin Chen
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.17253
• Pdf URL: https://arxiv.org/pdf/2504.17253
• Copy Paste: [[2504.17253]] DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks(https://arxiv.org/abs/2504.17253)
• Keywords: diffusion, generative
• Abstract: Diffusion models have shown remarkable progress in various generative tasks such as image and video generation. This paper studies the problem of leveraging pretrained diffusion models for performing discriminative tasks. Specifically, we extend the discriminative capability of pretrained frozen generative diffusion models from the classification task to the more complex object detection task, by "inverting" a pretrained layout-to-image diffusion model. To this end, a gradient-based discrete optimization approach for replacing the heavy prediction enumeration process, and a prior distribution model for making more accurate use of the Bayes' rule, are proposed respectively. Empirical results show that this method is on par with basic discriminative object detection baselines on COCO dataset. In addition, our method can greatly speed up the previous diffusion-based method for classification without sacrificing accuracy. Code and models are available at this https URL .

Title: Symbolic Representation for Any-to-Any Generative Tasks

• Authors: Jiaqi Chen, Xiaoye Zhu, Yue Wang, Tianyang Liu, Xinhui Chen, Ying Chen, Chak Tou Leong, Yifei Ke, Joseph Liu, Yiwen Yuan, Julian McAuley, Li-jia Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17261
• Pdf URL: https://arxiv.org/pdf/2504.17261
• Copy Paste: [[2504.17261]] Symbolic Representation for Any-to-Any Generative Tasks(https://arxiv.org/abs/2504.17261)
• Keywords: generative
• Abstract: We propose a symbolic generative task description language and a corresponding inference engine capable of representing arbitrary multimodal tasks as structured symbolic flows. Unlike conventional generative models that rely on large-scale training and implicit neural representations to learn cross-modal mappings, often at high computational cost and with limited flexibility, our framework introduces an explicit symbolic representation comprising three core primitives: functions, parameters, and topological logic. Leveraging a pre-trained language model, our inference engine maps natural language instructions directly to symbolic workflows in a training-free manner. Our framework successfully performs over 12 diverse multimodal generative tasks, demonstrating strong performance and flexibility without the need for task-specific tuning. Experiments show that our method not only matches or outperforms existing state-of-the-art unified models in content quality, but also offers greater efficiency, editability, and interruptibility. We believe that symbolic task representations provide a cost-effective and extensible foundation for advancing the capabilities of generative AI.

Title: Towards Generalized and Training-Free Text-Guided Semantic Manipulation

• Authors: Yu Hong, Xiao Cai, Pengpeng Zeng, Shuai Zhang, Jingkuan Song, Lianli Gao, Heng Tao Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17269
• Pdf URL: https://arxiv.org/pdf/2504.17269
• Copy Paste: [[2504.17269]] Towards Generalized and Training-Free Text-Guided Semantic Manipulation(https://arxiv.org/abs/2504.17269)
• Keywords: diffusion, generative
• Abstract: Text-guided semantic manipulation refers to semantically editing an image generated from a source prompt to match a target prompt, enabling the desired semantic changes (e.g., addition, removal, and style transfer) while preserving irrelevant contents. With the powerful generative capabilities of the diffusion model, the task has shown the potential to generate high-fidelity visual content. Nevertheless, existing methods either typically require time-consuming fine-tuning (inefficient), fail to accomplish multiple semantic manipulations (poorly extensible), and/or lack support for different modality tasks (limited generalizability). Upon further investigation, we find that the geometric properties of noises in the diffusion model are strongly correlated with the semantic changes. Motivated by this, we propose a novel $\textit{GTF}$ for text-guided semantic manipulation, which has the following attractive capabilities: 1) $\textbf{Generalized}$: our $\textit{GTF}$ supports multiple semantic manipulations (e.g., addition, removal, and style transfer) and can be seamlessly integrated into all diffusion-based methods (i.e., Plug-and-play) across different modalities (i.e., modality-agnostic); and 2) $\textbf{Training-free}$: $\textit{GTF}$ produces high-fidelity results via simply controlling the geometric relationship between noises without tuning or optimization. Our extensive experiments demonstrate the efficacy of our approach, highlighting its potential to advance the state-of-the-art in semantics manipulation.

Title: Class-Conditional Distribution Balancing for Group Robust Classification

• Authors: Miaoyun Zhao, Qiang Zhang, Chenrong Li
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17314
• Pdf URL: https://arxiv.org/pdf/2504.17314
• Copy Paste: [[2504.17314]] Class-Conditional Distribution Balancing for Group Robust Classification(https://arxiv.org/abs/2504.17314)
• Keywords: foundation model
• Abstract: Spurious correlations that lead models to correct predictions for the wrong reasons pose a critical challenge for robust real-world generalization. Existing research attributes this issue to group imbalance and addresses it by maximizing group-balanced or worst-group accuracy, which heavily relies on expensive bias annotations. A compromise approach involves predicting bias information using extensively pretrained foundation models, which requires large-scale data and becomes impractical for resource-limited rare domains. To address these challenges, we offer a novel perspective by reframing the spurious correlations as imbalances or mismatches in class-conditional distributions, and propose a simple yet effective robust learning method that eliminates the need for both bias annotations and predictions. With the goal of reducing the mutual information between spurious factors and label information, our method leverages a sample reweighting strategy to achieve class-conditional distribution balancing, which automatically highlights minority groups and classes, effectively dismantling spurious correlations and producing a debiased data distribution for classification. Extensive experiments and analysis demonstrate that our approach consistently delivers state-of-the-art performance, rivaling methods that rely on bias supervision.

Title: DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition

• Authors: Yiyan Xu, Wuqiang Zheng, Wenjie Wang, Fengbin Zhu, Xinting Hu, Yang Zhang, Fuli Feng, Tat-Seng Chua
• Subjects: cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.17349
• Pdf URL: https://arxiv.org/pdf/2504.17349
• Copy Paste: [[2504.17349]] DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition(https://arxiv.org/abs/2504.17349)
• Keywords: diffusion
• Abstract: Personalized image generation has emerged as a promising direction in multimodal content creation. It aims to synthesize images tailored to individual style preferences (e.g., color schemes, character appearances, layout) and semantic intentions (e.g., emotion, action, scene contexts) by leveraging user-interacted history images and multimodal instructions. Despite notable progress, existing methods -- whether based on diffusion models, large language models, or Large Multimodal Models (LMMs) -- struggle to accurately capture and fuse user style preferences and semantic intentions. In particular, the state-of-the-art LMM-based method suffers from the entanglement of visual features, leading to Guidance Collapse, where the generated images fail to preserve user-preferred styles or reflect the specified semantics. To address these limitations, we introduce DRC, a novel personalized image generation framework that enhances LMMs through Disentangled Representation Composition. DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively, to form user-specific latent instructions that guide image generation within LMMs. Specifically, it involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation. Extensive experiments on two benchmarks demonstrate that DRC shows competitive performance while effectively mitigating the guidance collapse issue, underscoring the importance of disentangled representation learning for controllable and effective personalized image generation.

Title: Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset

• Authors: Oussema Dhaouadi, Johannes Meier, Luca Wahl, Jacques Kaiser, Luca Scalerandi, Nick Wandelburg, Zhuolun Zhou, Nijanthan Berinpanathan, Holger Banzhaf, Daniel Cremers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17371
• Pdf URL: https://arxiv.org/pdf/2504.17371
• Copy Paste: [[2504.17371]] Highly Accurate and Diverse Traffic Data: The DeepScenario Open 3D Dataset(https://arxiv.org/abs/2504.17371)
• Keywords: generative
• Abstract: Accurate 3D trajectory data is crucial for advancing autonomous driving. Yet, traditional datasets are usually captured by fixed sensors mounted on a car and are susceptible to occlusion. Additionally, such an approach can precisely reconstruct the dynamic environment in the close vicinity of the measurement vehicle only, while neglecting objects that are further away. In this paper, we introduce the DeepScenario Open 3D Dataset (DSC3D), a high-quality, occlusion-free dataset of 6 degrees of freedom bounding box trajectories acquired through a novel monocular camera drone tracking pipeline. Our dataset includes more than 175,000 trajectories of 14 types of traffic participants and significantly exceeds existing datasets in terms of diversity and scale, containing many unprecedented scenarios such as complex vehicle-pedestrian interaction on highly populated urban streets and comprehensive parking maneuvers from entry to exit. DSC3D dataset was captured in five various locations in Europe and the United States and include: a parking lot, a crowded inner-city, a steep urban intersection, a federal highway, and a suburban intersection. Our 3D trajectory dataset aims to enhance autonomous driving systems by providing detailed environmental 3D representations, which could lead to improved obstacle interactions and safety. We demonstrate its utility across multiple applications including motion prediction, motion planning, scenario mining, and generative reactive traffic agents. Our interactive online visualization platform and the complete dataset are publicly available at this http URL, facilitating research in motion prediction, behavior modeling, and safety validation.

Title: Fine-tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Models

• Authors: Francesc Marti-Escofet, Benedikt Blumenstiel, Linus Scheibenreif, Paolo Fraccaro, Konrad Schindler
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17397
• Pdf URL: https://arxiv.org/pdf/2504.17397
• Copy Paste: [[2504.17397]] Fine-tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Models(https://arxiv.org/abs/2504.17397)
• Keywords: foundation model
• Abstract: Earth observation (EO) is crucial for monitoring environmental changes, responding to disasters, and managing natural resources. In this context, foundation models facilitate remote sensing image analysis to retrieve relevant geoinformation accurately and efficiently. However, as these models grow in size, fine-tuning becomes increasingly challenging due to the associated computational resources and costs, limiting their accessibility and scalability. Furthermore, full fine-tuning can lead to forgetting pre-trained features and even degrade model generalization. To address this, Parameter-Efficient Fine-Tuning (PEFT) techniques offer a promising solution. In this paper, we conduct extensive experiments with various foundation model architectures and PEFT techniques to evaluate their effectiveness on five different EO datasets. Our results provide a comprehensive comparison, offering insights into when and how PEFT methods support the adaptation of pre-trained geospatial models. We demonstrate that PEFT techniques match or even exceed full fine-tuning performance and enhance model generalisation to unseen geographic regions, while reducing training time and memory requirements. Additional experiments investigate the effect of architecture choices such as the decoder type or the use of metadata, suggesting UNet decoders and fine-tuning without metadata as the recommended configuration. We have integrated all evaluated foundation models and techniques into the open-source package TerraTorch to support quick, scalable, and cost-effective model adaptation.

Title: 3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models

• Authors: Min Wei, Chaohui Yu, Jingkai Zhou, Fan Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17414
• Pdf URL: https://arxiv.org/pdf/2504.17414
• Copy Paste: [[2504.17414]] 3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models(https://arxiv.org/abs/2504.17414)
• Keywords: diffusion
• Abstract: Video try-on replaces clothing in videos with target garments. Existing methods struggle to generate high-quality and temporally consistent results when handling complex clothing patterns and diverse body poses. We present 3DV-TON, a novel diffusion-based framework for generating high-fidelity and temporally consistent video try-on results. Our approach employs generated animatable textured 3D meshes as explicit frame-level guidance, alleviating the issue of models over-focusing on appearance fidelity at the expanse of motion coherence. This is achieved by enabling direct reference to consistent garment texture movements throughout video sequences. The proposed method features an adaptive pipeline for generating dynamic 3D guidance: (1) selecting a keyframe for initial 2D image try-on, followed by (2) reconstructing and animating a textured 3D mesh synchronized with original video poses. We further introduce a robust rectangular masking strategy that successfully mitigates artifact propagation caused by leaking clothing information during dynamic human and garment movements. To advance video try-on research, we introduce HR-VVT, a high-resolution benchmark dataset containing 130 videos with diverse clothing types and scenarios. Quantitative and qualitative results demonstrate our superior performance over existing methods. The project page is at this link this https URL

Title: ESDiff: Encoding Strategy-inspired Diffusion Model with Few-shot Learning for Color Image Inpainting

• Authors: Junyan Zhang, Yan Li, Mengxiao Geng, Liu Shi, Qiegen Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17524
• Pdf URL: https://arxiv.org/pdf/2504.17524
• Copy Paste: [[2504.17524]] ESDiff: Encoding Strategy-inspired Diffusion Model with Few-shot Learning for Color Image Inpainting(https://arxiv.org/abs/2504.17524)
• Keywords: diffusion
• Abstract: Image inpainting is a technique used to restore missing or damaged regions of an image. Traditional methods primarily utilize information from adjacent pixels for reconstructing missing areas, while they struggle to preserve complex details and structures. Simultaneously, models based on deep learning necessitate substantial amounts of training data. To address this challenge, an encoding strategy-inspired diffusion model with few-shot learning for color image inpainting is proposed in this paper. The main idea of this novel encoding strategy is the deployment of a "virtual mask" to construct high-dimensional objects through mutual perturbations between channels. This approach enables the diffusion model to capture diverse image representations and detailed features from limited training samples. Moreover, the encoding strategy leverages redundancy between channels, integrates with low-rank methods during iterative inpainting, and incorporates the diffusion model to achieve accurate information output. Experimental results indicate that our method exceeds current techniques in quantitative metrics, and the reconstructed images quality has been improved in aspects of texture and structural integrity, leading to more precise and coherent results.

Title: Text-to-Image Alignment in Denoising-Based Models through Step Selection

• Authors: Paul Grimal, Hervé Le Borgne, Olivier Ferret
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17525
• Pdf URL: https://arxiv.org/pdf/2504.17525
• Copy Paste: [[2504.17525]] Text-to-Image Alignment in Denoising-Based Models through Step Selection(https://arxiv.org/abs/2504.17525)
• Keywords: diffusion, generative
• Abstract: Visual generative AI models often encounter challenges related to text-image alignment and reasoning limitations. This paper presents a novel method for selectively enhancing the signal at critical denoising steps, optimizing image generation based on input semantics. Our approach addresses the shortcomings of early-stage signal modifications, demonstrating that adjustments made at later stages yield superior results. We conduct extensive experiments to validate the effectiveness of our method in producing semantically aligned images on Diffusion and Flow Matching model, achieving state-of-the-art performance. Our results highlight the importance of a judicious choice of sampling stage to improve performance and overall image alignment.

Title: HalluLens: LLM Hallucination Benchmark

• Authors: Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17550
• Pdf URL: https://arxiv.org/pdf/2504.17550
• Copy Paste: [[2504.17550]] HalluLens: LLM Hallucination Benchmark(https://arxiv.org/abs/2504.17550)
• Keywords: generative
• Abstract: Large language models (LLMs) often generate responses that deviate from user input or training data, a phenomenon known as "hallucination." These hallucinations undermine user trust and hinder the adoption of generative AI systems. Addressing hallucinations is essential for the advancement of LLMs. This paper introduces a comprehensive hallucination benchmark, incorporating both new extrinsic and existing intrinsic evaluation tasks, built upon clear taxonomy of hallucination. A major challenge in benchmarking hallucinations is the lack of a unified framework due to inconsistent definitions and categorizations. We disentangle LLM hallucination from "factuality," proposing a clear taxonomy that distinguishes between extrinsic and intrinsic hallucinations, to promote consistency and facilitate research. Extrinsic hallucinations, where the generated content is not consistent with the training data, are increasingly important as LLMs evolve. Our benchmark includes dynamic test set generation to mitigate data leakage and ensure robustness against such leakage. We also analyze existing benchmarks, highlighting their limitations and saturation. The work aims to: (1) establish a clear taxonomy of hallucinations, (2) introduce new extrinsic hallucination tasks, with data that can be dynamically regenerated to prevent saturation by leakage, (3) provide a comprehensive analysis of existing benchmarks, distinguishing them from factuality evaluations.

Title: Occlusion-Aware Self-Supervised Monocular Depth Estimation for Weak-Texture Endoscopic Images

• Authors: Zebo Huang, Yinghui Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17582
• Pdf URL: https://arxiv.org/pdf/2504.17582
• Copy Paste: [[2504.17582]] Occlusion-Aware Self-Supervised Monocular Depth Estimation for Weak-Texture Endoscopic Images(https://arxiv.org/abs/2504.17582)
• Keywords: self-supervised
• Abstract: We propose a self-supervised monocular depth estimation network tailored for endoscopic scenes, aiming to infer depth within the gastrointestinal tract from monocular images. Existing methods, though accurate, typically assume consistent illumination, which is often violated due to dynamic lighting and occlusions caused by GI motility. These variations lead to incorrect geometric interpretations and unreliable self-supervised signals, degrading depth reconstruction quality. To address this, we introduce an occlusion-aware self-supervised framework. First, we incorporate an occlusion mask for data augmentation, generating pseudo-labels by simulating viewpoint-dependent occlusion scenarios. This enhances the model's ability to learn robust depth features under partial visibility. Second, we leverage semantic segmentation guided by non-negative matrix factorization, clustering convolutional activations to generate pseudo-labels in texture-deprived regions, thereby improving segmentation accuracy and mitigating information loss from lighting changes. Experimental results on the SCARED dataset show that our method achieves state-of-the-art performance in self-supervised depth estimation. Additionally, evaluations on the Endo-SLAM and SERV-CT datasets demonstrate strong generalization across diverse endoscopic environments.

Title: TarDiff: Target-Oriented Diffusion Guidance for Synthetic Electronic Health Record Time Series Generation

• Authors: Bowen Deng, Chang Xu, Hao Li, Yuhao Huang, Min Hou, Jiang Bian
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17613
• Pdf URL: https://arxiv.org/pdf/2504.17613
• Copy Paste: [[2504.17613]] TarDiff: Target-Oriented Diffusion Guidance for Synthetic Electronic Health Record Time Series Generation(https://arxiv.org/abs/2504.17613)
• Keywords: diffusion
• Abstract: Synthetic Electronic Health Record (EHR) time-series generation is crucial for advancing clinical machine learning models, as it helps address data scarcity by providing more training data. However, most existing approaches focus primarily on replicating statistical distributions and temporal dependencies of real-world data. We argue that fidelity to observed data alone does not guarantee better model performance, as common patterns may dominate, limiting the representation of rare but important conditions. This highlights the need for generate synthetic samples to improve performance of specific clinical models to fulfill their target outcomes. To address this, we propose TarDiff, a novel target-oriented diffusion framework that integrates task-specific influence guidance into the synthetic data generation process. Unlike conventional approaches that mimic training data distributions, TarDiff optimizes synthetic samples by quantifying their expected contribution to improving downstream model performance through influence functions. Specifically, we measure the reduction in task-specific loss induced by synthetic samples and embed this influence gradient into the reverse diffusion process, thereby steering the generation towards utility-optimized data. Evaluated on six publicly available EHR datasets, TarDiff achieves state-of-the-art performance, outperforming existing methods by up to 20.4% in AUPRC and 18.4% in AUROC. Our results demonstrate that TarDiff not only preserves temporal fidelity but also enhances downstream model performance, offering a robust solution to data scarcity and class imbalance in healthcare analytics.

Title: Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models

• Authors: Julius Vetter, Manuel Gloeckler, Daniel Gedon, Jakob H. Macke
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17660
• Pdf URL: https://arxiv.org/pdf/2504.17660
• Copy Paste: [[2504.17660]] Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models(https://arxiv.org/abs/2504.17660)
• Keywords: foundation model
• Abstract: Simulation-based inference (SBI) offers a flexible and general approach to performing Bayesian inference: In SBI, a neural network is trained on synthetic data simulated from a model and used to rapidly infer posterior distributions for observed data. A key goal for SBI is to achieve accurate inference with as few simulations as possible, especially for expensive simulators. In this work, we address this challenge by repurposing recent probabilistic foundation models for tabular data: We show how tabular foundation models -- specifically TabPFN -- can be used as pre-trained autoregressive conditional density estimators for SBI. We propose Neural Posterior Estimation with Prior-data Fitted Networks (NPE-PF) and show that it is competitive with current SBI approaches in terms of accuracy for both benchmark tasks and two complex scientific inverse problems. Crucially, it often substantially outperforms them in terms of simulation efficiency, sometimes requiring orders of magnitude fewer simulations. NPE-PF eliminates the need for inference network selection, training, and hyperparameter tuning. We also show that it exhibits superior robustness to model misspecification and can be scaled to simulation budgets that exceed the context size limit of TabPFN. NPE-PF provides a new direction for SBI, where training-free, general-purpose inference models offer efficient, easy-to-use, and flexible solutions for a wide range of stochastic inverse problems.

Title: DiMeR: Disentangled Mesh Reconstruction Model

• Authors: Lutao Jiang, Jiantao Lin, Kanghao Chen, Wenhang Ge, Xin Yang, Yifan Jiang, Yuanhuiyi Lyu, Xu Zheng, Yingcong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17670
• Pdf URL: https://arxiv.org/pdf/2504.17670
• Copy Paste: [[2504.17670]] DiMeR: Disentangled Mesh Reconstruction Model(https://arxiv.org/abs/2504.17670)
• Keywords: generative
• Abstract: With the advent of large-scale 3D datasets, feed-forward 3D generative models, such as the Large Reconstruction Model (LRM), have gained significant attention and achieved remarkable success. However, we observe that RGB images often lead to conflicting training objectives and lack the necessary clarity for geometry reconstruction. In this paper, we revisit the inductive biases associated with mesh reconstruction and introduce DiMeR, a novel disentangled dual-stream feed-forward model for sparse-view mesh reconstruction. The key idea is to disentangle both the input and framework into geometry and texture parts, thereby reducing the training difficulty for each part according to the Principle of Occam's Razor. Given that normal maps are strictly consistent with geometry and accurately capture surface variations, we utilize normal maps as exclusive input for the geometry branch to reduce the complexity between the network's input and output. Moreover, we improve the mesh extraction algorithm to introduce 3D ground truth supervision. As for texture branch, we use RGB images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust capabilities across various tasks, including sparse-view reconstruction, single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR significantly outperforms previous methods, achieving over 30% improvement in Chamfer Distance on the GSO and OmniObject3D dataset.

Title: Energy Considerations of Large Language Model Inference and Efficiency Optimizations

• Authors: Jared Fernandez, Clara Na, Vashisth Tiwari, Yonatan Bisk, Sasha Luccioni, Emma Strubell
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17674
• Pdf URL: https://arxiv.org/pdf/2504.17674
• Copy Paste: [[2504.17674]] Energy Considerations of Large Language Model Inference and Efficiency Optimizations(https://arxiv.org/abs/2504.17674)
• Keywords: generative
• Abstract: As large language models (LLMs) scale in size and adoption, their computational and environmental costs continue to rise. Prior benchmarking efforts have primarily focused on latency reduction in idealized settings, often overlooking the diverse real-world inference workloads that shape energy use. In this work, we systematically analyze the energy implications of common inference efficiency optimizations across diverse Natural Language Processing (NLP) and generative Artificial Intelligence (AI) workloads, including conversational AI and code generation. We introduce a modeling approach that approximates real-world LLM workflows through a binning strategy for input-output token distributions and batch size variations. Our empirical analysis spans software frameworks, decoding strategies, GPU architectures, online and offline serving settings, and model parallelism configurations. We show that the effectiveness of inference optimizations is highly sensitive to workload geometry, software stack, and hardware accelerators, demonstrating that naive energy estimates based on FLOPs or theoretical GPU utilization significantly underestimate real-world energy consumption. Our findings reveal that the proper application of relevant inference efficiency optimizations can reduce total energy use by up to 73% from unoptimized baselines. These insights provide a foundation for sustainable LLM deployment and inform energy-efficient design strategies for future AI infrastructure.

Title: PICO: Reconstructing 3D People In Contact with Objects

• Authors: Alpár Cseke, Shashank Tripathi, Sai Kumar Dwivedi, Arjun Lakshmipathy, Agniv Chatterjee, Michael J. Black, Dimitrios Tzionas
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17695
• Pdf URL: https://arxiv.org/pdf/2504.17695
• Copy Paste: [[2504.17695]] PICO: Reconstructing 3D People In Contact with Objects(https://arxiv.org/abs/2504.17695)
• Keywords: foundation model
• Abstract: Recovering 3D Human-Object Interaction (HOI) from single color images is challenging due to depth ambiguities, occlusions, and the huge variation in object shape and appearance. Thus, past work requires controlled settings such as known object shapes and contacts, and tackles only limited object classes. Instead, we need methods that generalize to natural images and novel object classes. We tackle this in two main ways: (1) We collect PICO-db, a new dataset of natural images uniquely paired with dense 3D contact on both body and object meshes. To this end, we use images from the recent DAMON dataset that are paired with contacts, but these contacts are only annotated on a canonical 3D body. In contrast, we seek contact labels on both the body and the object. To infer these given an image, we retrieve an appropriate 3D object mesh from a database by leveraging vision foundation models. Then, we project DAMON's body contact patches onto the object via a novel method needing only 2 clicks per patch. This minimal human input establishes rich contact correspondences between bodies and objects. (2) We exploit our new dataset of contact correspondences in a novel render-and-compare fitting method, called PICO-fit, to recover 3D body and object meshes in interaction. PICO-fit infers contact for the SMPL-X body, retrieves a likely 3D object mesh and contact from PICO-db for that object, and uses the contact to iteratively fit the 3D body and object meshes to image evidence via optimization. Uniquely, PICO-fit works well for many object categories that no existing method can tackle. This is crucial to enable HOI understanding to scale in the wild. Our data and code are available at this https URL.

Title: Fault Diagnosis in New Wind Turbines using Knowledge from Existing Turbines by Generative Domain Adaptation

• Authors: Stefan Jonas, Angela Meyer
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2504.17709
• Pdf URL: https://arxiv.org/pdf/2504.17709
• Copy Paste: [[2504.17709]] Fault Diagnosis in New Wind Turbines using Knowledge from Existing Turbines by Generative Domain Adaptation(https://arxiv.org/abs/2504.17709)
• Keywords: generative, anomaly
• Abstract: Intelligent condition monitoring of wind turbines is essential for reducing downtimes. Machine learning models trained on wind turbine operation data are commonly used to detect anomalies and, eventually, operation faults. However, data-driven normal behavior models (NBMs) require a substantial amount of training data, as NBMs trained with scarce data may result in unreliable fault diagnosis. To overcome this limitation, we present a novel generative deep learning approach to make SCADA samples from one wind turbine lacking training data resemble SCADA data from wind turbines with representative training data. Through CycleGAN-based domain mapping, our method enables the application of an NBM trained on an existing wind turbine to one with severely limited data. We demonstrate our approach on field data mapping SCADA samples across 7 substantially different WTs. Our findings show significantly improved fault diagnosis in wind turbines with scarce data. Our method achieves the most similar anomaly scores to an NBM trained with abundant data, outperforming NBMs trained on scarce training data with improvements of +10.3% in F1-score when 1 month of training data is available and +16.8% when 2 weeks are available. The domain mapping approach outperforms conventional fine-tuning at all considered degrees of data scarcity, ranging from 1 to 8 weeks of training data. The proposed technique enables earlier and more reliable fault diagnosis in newly installed wind farms, demonstrating a novel and promising research direction to improve anomaly detection when faced with training data scarcity.

Title: Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields

• Authors: Zhuo He, Paul Henderson, Nicolas Pugeault
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17712
• Pdf URL: https://arxiv.org/pdf/2504.17712
• Copy Paste: [[2504.17712]] Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields(https://arxiv.org/abs/2504.17712)
• Keywords: generative
• Abstract: StyleGAN has demonstrated the ability of GANs to synthesize highly-realistic faces of imaginary people from random noise. One limitation of GAN-based image generation is the difficulty of controlling the features of the generated image, due to the strong entanglement of the low-dimensional latent space. Previous work that aimed to control StyleGAN with image or text prompts modulated sampling in W latent space, which is more expressive than Z latent space. However, W space still has restricted expressivity since it does not control the feature synthesis directly; also the feature embedding in W space requires a pre-training process to reconstruct the style signal, limiting its application. This paper introduces the concept of "generative fields" to explain the hierarchical feature synthesis in StyleGAN, inspired by the receptive fields of convolution neural networks (CNNs). Additionally, we propose a new image editing pipeline for StyleGAN using generative field theory and the channel-wise style latent space S, utilizing the intrinsic structural feature of CNNs to achieve disentangled control of feature synthesis at synthesis time.

Title: Interpretable Early Detection of Parkinson's Disease through Speech Analysis

• Authors: Lorenzo Simone, Mauro Giuseppe Camporeale, Vito Marco Rubino, Vincenzo Gervasi, Giovanni Dimauro
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.17739
• Pdf URL: https://arxiv.org/pdf/2504.17739
• Copy Paste: [[2504.17739]] Interpretable Early Detection of Parkinson's Disease through Speech Analysis(https://arxiv.org/abs/2504.17739)
• Keywords: generative
• Abstract: Parkinson's disease is a progressive neurodegenerative disorder affecting motor and non-motor functions, with speech impairments among its earliest symptoms. Speech impairments offer a valuable diagnostic opportunity, with machine learning advances providing promising tools for timely detection. In this research, we propose a deep learning approach for early Parkinson's disease detection from speech recordings, which also highlights the vocal segments driving predictions to enhance interpretability. This approach seeks to associate predictive speech patterns with articulatory features, providing a basis for interpreting underlying neuromuscular impairments. We evaluated our approach using the Italian Parkinson's Voice and Speech Database, containing 831 audio recordings from 65 participants, including both healthy individuals and patients. Our approach showed competitive classification performance compared to state-of-the-art methods, while providing enhanced interpretability by identifying key speech features influencing predictions.

Title: Conversational Assistants to support Heart Failure Patients: comparing a Neurosymbolic Architecture with ChatGPT

• Authors: Anuja Tayal, Devika Salunke, Barbara Di Eugenio, Paula Allen-Meares, Eulalia Puig Abril, Olga Garcia, Carolyn Dickens, Andrew Boyd
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.17753
• Pdf URL: https://arxiv.org/pdf/2504.17753
• Copy Paste: [[2504.17753]] Conversational Assistants to support Heart Failure Patients: comparing a Neurosymbolic Architecture with ChatGPT(https://arxiv.org/abs/2504.17753)
• Keywords: generative
• Abstract: Conversational assistants are becoming more and more popular, including in healthcare, partly because of the availability and capabilities of Large Language Models. There is a need for controlled, probing evaluations with real stakeholders which can highlight advantages and disadvantages of more traditional architectures and those based on generative AI. We present a within-group user study to compare two versions of a conversational assistant that allows heart failure patients to ask about salt content in food. One version of the system was developed in-house with a neurosymbolic architecture, and one is based on ChatGPT. The evaluation shows that the in-house system is more accurate, completes more tasks and is less verbose than the one based on ChatGPT; on the other hand, the one based on ChatGPT makes fewer speech errors and requires fewer clarifications to complete the task. Patients show no preference for one over the other.

Title: Step1X-Edit: A Practical Framework for General Image Editing

• Authors: Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, Guopeng Li, Yuang Peng, Quan Sun, Jingwei Wu, Yan Cai, Zheng Ge, Ranchen Ming, Lei Xia, Xianfang Zeng, Yibo Zhu, Binxing Jiao, Xiangyu Zhang, Gang Yu, Daxin Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17761
• Pdf URL: https://arxiv.org/pdf/2504.17761
• Copy Paste: [[2504.17761]] Step1X-Edit: A Practical Framework for General Image Editing(https://arxiv.org/abs/2504.17761)
• Keywords: diffusion
• Abstract: In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.

Title: Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models

• Authors: Xu Ma, Peize Sun, Haoyu Ma, Hao Tang, Chih-Yao Ma, Jialiang Wang, Kunpeng Li, Xiaoliang Dai, Yujun Shi, Xuan Ju, Yushi Hu, Artsiom Sanakoyeu, Felix Juefei-Xu, Ji Hou, Junjiao Tian, Tao Xu, Tingbo Hou, Yen-Cheng Liu, Zecheng He, Zijian He, Matt Feiszli, Peizhao Zhang, Peter Vajda, Sam Tsai, Yun Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17789
• Pdf URL: https://arxiv.org/pdf/2504.17789
• Copy Paste: [[2504.17789]] Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models(https://arxiv.org/abs/2504.17789)
• Keywords: diffusion
• Abstract: Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models. A primary limitation is the substantial number of image tokens required for AR models, which constrains both training and inference efficiency, as well as image resolution. To address this, we present Token-Shuffle, a novel yet simple method that reduces the number of image tokens in Transformer. Our key insight is the dimensional redundancy of visual vocabularies in Multimodal Large Language Models (MLLMs), where low-dimensional visual codes from visual encoder are directly mapped to high-dimensional language vocabularies. Leveraging this, we consider two key operations: token-shuffle, which merges spatially local tokens along channel dimension to decrease the input token number, and token-unshuffle, which untangles the inferred tokens after Transformer blocks to restore the spatial arrangement for output. Jointly training with textual prompts, our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis in a unified next-token prediction way while maintaining efficient training and inference. For the first time, we push the boundary of AR text-to-image generation to a resolution of 2048x2048 with gratifying generation performance. In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15. Exhaustive large-scale human evaluations also demonstrate our prominent image generation ability in terms of text-alignment, visual flaw, and visual appearance. We hope that Token-Shuffle can serve as a foundational design for efficient high-resolution image generation within MLLMs.

Title: LiDPM: Rethinking Point Diffusion for Lidar Scene Completion

• Authors: Tetiana Martyniuk, Gilles Puy, Alexandre Boulch, Renaud Marlet, Raoul de Charette
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.17791
• Pdf URL: https://arxiv.org/pdf/2504.17791
• Copy Paste: [[2504.17791]] LiDPM: Rethinking Point Diffusion for Lidar Scene Completion(https://arxiv.org/abs/2504.17791)
• Keywords: diffusion
• Abstract: Training diffusion models that work directly on lidar points at the scale of outdoor scenes is challenging due to the difficulty of generating fine-grained details from white noise over a broad field of view. The latest works addressing scene completion with diffusion models tackle this problem by reformulating the original DDPM as a local diffusion process. It contrasts with the common practice of operating at the level of objects, where vanilla DDPMs are currently used. In this work, we close the gap between these two lines of work. We identify approximations in the local diffusion formulation, show that they are not required to operate at the scene level, and that a vanilla DDPM with a well-chosen starting point is enough for completion. Finally, we demonstrate that our method, LiDPM, leads to better results in scene completion on SemanticKITTI. The project page is this https URL .