2025-04-11

Title: Secure Text Mail Encryption with Generative Adversarial Networks

• Authors: Alexej Schelle
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07140
• Pdf URL: https://arxiv.org/pdf/2504.07140
• Copy Paste: [[2504.07140]] Secure Text Mail Encryption with Generative Adversarial Networks(https://arxiv.org/abs/2504.07140)
• Keywords: generative
• Abstract: This work presents an encryption model based on Generative Adversarial Networks (GANs). Encryption of RTF-8 data is realized by dynamically generating decimal numbers that lead to the encryption and decryption of alphabetic strings in integer representation by simple addition rules, the modulus of the dimension of the considered alphabet. The binary numbers for the private dynamical keys correlate with the binary numbers of public reference keys from a mapping defined by the specific GAN configuration. For reversible encryption with bijective mapping between dynamic and reference keys as defined by the GAN encryptor with random combinations of NOT logical gates between bitwise subcomponents of the transmitted text signal, secure text encryption can be realized by transferring a GAN-encrypted public key with encrypted text from a sender to a receiver. Using the technique described above, secure text mail transfer can be realized from component-wise encryption of text mail strings with total key sizes of up to $10^{8}$ bits that define random decimal numbers obtained from the GAN. From the present model, we assert that encrypted texts can be transmitted more efficiently and securely than from RSA encryption, as long as users of the specific configuration of the GAN encryption model are unaware of the GAN encryptor circuit.

Title: Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning

• Authors: Ashutosh Chaubey, Xulang Guan, Mohammad Soleymani
• Subjects: cs.CV, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2504.07198
• Pdf URL: https://arxiv.org/pdf/2504.07198
• Copy Paste: [[2504.07198]] Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning(https://arxiv.org/abs/2504.07198)
• Keywords: in-context
• Abstract: The human face plays a central role in social communication, necessitating the use of performant computer vision tools for human-centered applications. We propose Face-LLaVA, a multimodal large language model for face-centered, in-context learning, including facial expression and attribute recognition. Additionally, Face-LLaVA is able to generate natural language descriptions that can be used for reasoning. Leveraging existing visual databases, we first developed FaceInstruct-1M, a face-centered database for instruction tuning MLLMs for face processing. We then developed a novel face-specific visual encoder powered by Face-Region Guided Cross-Attention that integrates face geometry with local visual features. We evaluated the proposed method across nine different datasets and five different face processing tasks, including facial expression recognition, action unit detection, facial attribute detection, age estimation and deepfake detection. Face-LLaVA achieves superior results compared to existing open-source MLLMs and competitive performance compared to commercial solutions. Our model output also receives a higher reasoning rating by GPT under a zero-shot setting across all the tasks. Both our dataset and model wil be released at this https URL to support future advancements in social AI and foundational vision-language research.

Title: Leveraging Machine Learning Techniques in Intrusion Detection Systems for Internet of Things

• Authors: Saeid Jamshidi, Amin Nikanjam, Nafi Kawser Wazed, Foutse Khomh
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2504.07220
• Pdf URL: https://arxiv.org/pdf/2504.07220
• Copy Paste: [[2504.07220]] Leveraging Machine Learning Techniques in Intrusion Detection Systems for Internet of Things(https://arxiv.org/abs/2504.07220)
• Keywords: generative
• Abstract: As the Internet of Things (IoT) continues to expand, ensuring the security of connected devices has become increasingly critical. Traditional Intrusion Detection Systems (IDS) often fall short in managing the dynamic and large-scale nature of IoT networks. This paper explores how Machine Learning (ML) and Deep Learning (DL) techniques can significantly enhance IDS performance in IoT environments. We provide a thorough overview of various IDS deployment strategies and categorize the types of intrusions common in IoT systems. A range of ML methods -- including Support Vector Machines, Naive Bayes, K-Nearest Neighbors, Decision Trees, and Random Forests -- are examined alongside advanced DL models such as LSTM, CNN, Autoencoders, RNNs, and Deep Belief Networks. Each technique is evaluated based on its accuracy, efficiency, and suitability for real-world IoT applications. We also address major challenges such as high false positive rates, data imbalance, encrypted traffic analysis, and the resource constraints of IoT devices. In addition, we highlight the emerging role of Generative AI and Large Language Models (LLMs) in improving threat detection, automating responses, and generating intelligent security policies. Finally, we discuss ethical and privacy concerns, underscoring the need for responsible and transparent implementation. This paper aims to provide a comprehensive framework for developing adaptive, intelligent, and secure IDS solutions tailored for the evolving landscape of IoT.

Title: Resource-efficient Inference with Foundation Model Programs

• Authors: Lunyiu Nie, Zhimin Ding, Kevin Yu, Marco Cheung, Chris Jermaine, Swarat Chaudhuri
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07247
• Pdf URL: https://arxiv.org/pdf/2504.07247
• Copy Paste: [[2504.07247]] Resource-efficient Inference with Foundation Model Programs(https://arxiv.org/abs/2504.07247)
• Keywords: foundation model
• Abstract: The inference-time resource costs of large language and vision models present a growing challenge in production deployments. We propose the use of foundation model programs, i.e., programs that can invoke foundation models with varying resource costs and performance, as an approach to this problem. Specifically, we present a method that translates a task into a program, then learns a policy for resource allocation that, on each input, selects foundation model "backends" for each program module. The policy uses smaller, cheaper backends to handle simpler subtasks, while allowing more complex subtasks to leverage larger, more capable models. We evaluate the method on two new "streaming" visual question-answering tasks in which a system answers a question on a sequence of inputs, receiving ground-truth feedback after each answer. Compared to monolithic multi-modal models, our implementation achieves up to 98% resource savings with minimal accuracy loss, demonstrating its potential for scalable and resource-efficient multi-modal inference.

Title: Objaverse++: Curated 3D Object Dataset with Quality Annotations

• Authors: Chendi Lin, Heshan Liu, Qunshu Lin, Zachary Bright, Shitao Tang, Yihui He, Minghao Liu, Ling Zhu, Cindy Le
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07334
• Pdf URL: https://arxiv.org/pdf/2504.07334
• Copy Paste: [[2504.07334]] Objaverse++: Curated 3D Object Dataset with Quality Annotations(https://arxiv.org/abs/2504.07334)
• Keywords: generative
• Abstract: This paper presents Objaverse++, a curated subset of Objaverse enhanced with detailed attribute annotations by human experts. Recent advances in 3D content generation have been driven by large-scale datasets such as Objaverse, which contains over 800,000 3D objects collected from the Internet. Although Objaverse represents the largest available 3D asset collection, its utility is limited by the predominance of low-quality models. To address this limitation, we manually annotate 10,000 3D objects with detailed attributes, including aesthetic quality scores, texture color classifications, multi-object composition flags, transparency characteristics, etc. Then, we trained a neural network capable of annotating the tags for the rest of the Objaverse dataset. Through experiments and a user study on generation results, we demonstrate that models pre-trained on our quality-focused subset achieve better performance than those trained on the larger dataset of Objaverse in image-to-3D generation tasks. In addition, by comparing multiple subsets of training data filtered by our tags, our results show that the higher the data quality, the faster the training loss converges. These findings suggest that careful curation and rich annotation can compensate for the raw dataset size, potentially offering a more efficient path to develop 3D generative models. We release our enhanced dataset of approximately 500,000 curated 3D models to facilitate further research on various downstream tasks in 3D computer vision. In the near future, we aim to extend our annotations to cover the entire Objaverse dataset.

Title: FLASH: Flexible Learning of Adaptive Sampling from History in Temporal Graph Neural Networks

• Authors: Or Feldman, Krishna Sri Ipsit Mantri, Carola-Bibiane Schönlieb, Chaim Baskin, Moshe Eliasof
• Subjects: cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2504.07337
• Pdf URL: https://arxiv.org/pdf/2504.07337
• Copy Paste: [[2504.07337]] FLASH: Flexible Learning of Adaptive Sampling from History in Temporal Graph Neural Networks(https://arxiv.org/abs/2504.07337)
• Keywords: self-supervised
• Abstract: Aggregating temporal signals from historic interactions is a key step in future link prediction on dynamic graphs. However, incorporating long histories is resource-intensive. Hence, temporal graph neural networks (TGNNs) often rely on historical neighbors sampling heuristics such as uniform sampling or recent neighbors selection. These heuristics are static and fail to adapt to the underlying graph structure. We introduce FLASH, a learnable and graph-adaptive neighborhood selection mechanism that generalizes existing heuristics. FLASH integrates seamlessly into TGNNs and is trained end-to-end using a self-supervised ranking loss. We provide theoretical evidence that commonly used heuristics hinders TGNNs performance, motivating our design. Extensive experiments across multiple benchmarks demonstrate consistent and significant performance improvements for TGNNs equipped with FLASH.

Title: Leveraging deep learning for plant disease identification: a bibliometric analysis in SCOPUS from 2018 to 2024

• Authors: Enow Takang Achuo Albert, Ngalle Hermine Bille, Ngonkeu Mangaptche Eddy Leonard
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07342
• Pdf URL: https://arxiv.org/pdf/2504.07342
• Copy Paste: [[2504.07342]] Leveraging deep learning for plant disease identification: a bibliometric analysis in SCOPUS from 2018 to 2024(https://arxiv.org/abs/2504.07342)
• Keywords: generative
• Abstract: This work aimed to present a bibliometric analysis of deep learning research for plant disease identification, with a special focus on generative modeling. A thorough analysis of SCOPUS-sourced bibliometric data from 253 documents was performed. Key performance metrics such as accuracy, precision, recall, and F1-score were analyzed for generative modeling. The findings highlighted significant contributions from some authors Too and Arnal Barbedo, whose works had notable citation counts, suggesting their influence on the academic community. Co-authorship networks revealed strong collaborative clusters, while keyword analysis identified emerging research gaps. This study highlights the role of collaboration and citation metrics in shaping research directions and enhancing the impact of scholarly work in applications of deep learning to plant disease identification. Future research should explore the methodologies of highly cited studies to inform best practices and policy-making.

Title: Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction

• Authors: Junyi Ma, Wentao Bao, Jingyi Xu, Guanzhong Sun, Xieyuanli Chen, Hesheng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07375
• Pdf URL: https://arxiv.org/pdf/2504.07375
• Copy Paste: [[2504.07375]] Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction(https://arxiv.org/abs/2504.07375)
• Keywords: diffusion
• Abstract: Predicting hand motion is critical for understanding human intentions and bridging the action space between human movements and robot manipulations. Existing hand trajectory prediction (HTP) methods forecast the future hand waypoints in 3D space conditioned on past egocentric observations. However, such models are only designed to accommodate 2D egocentric video inputs. There is a lack of awareness of multimodal environmental information from both 2D and 3D observations, hindering the further improvement of 3D HTP performance. In addition, these models overlook the synergy between hand movements and headset camera egomotion, either predicting hand trajectories in isolation or encoding egomotion only from past frames. To address these limitations, we propose novel diffusion models (MMTwin) for multimodal 3D hand trajectory prediction. MMTwin is designed to absorb multimodal information as input encompassing 2D RGB images, 3D point clouds, past hand waypoints, and text prompt. Besides, two latent diffusion models, the egomotion diffusion and the HTP diffusion as twins, are integrated into MMTwin to predict camera egomotion and future hand trajectories concurrently. We propose a novel hybrid Mamba-Transformer module as the denoising model of the HTP diffusion to better fuse multimodal features. The experimental results on three publicly available datasets and our self-recorded data demonstrate that our proposed MMTwin can predict plausible future 3D hand trajectories compared to the state-of-the-art baselines, and generalizes well to unseen environments. The code and pretrained models will be released at this https URL.

Title: Model Discrepancy Learning: Synthetic Faces Detection Based on Multi-Reconstruction

• Authors: Qingchao Jiang, Zhishuo Xu, Zhiying Zhu, Ning Chen, Haoyue Wang, Zhongjie Ba
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07382
• Pdf URL: https://arxiv.org/pdf/2504.07382
• Copy Paste: [[2504.07382]] Model Discrepancy Learning: Synthetic Faces Detection Based on Multi-Reconstruction(https://arxiv.org/abs/2504.07382)
• Keywords: generative
• Abstract: Advances in image generation enable hyper-realistic synthetic faces but also pose risks, thus making synthetic face detection crucial. Previous research focuses on the general differences between generated images and real images, often overlooking the discrepancies among various generative techniques. In this paper, we explore the intrinsic relationship between synthetic images and their corresponding generation technologies. We find that specific images exhibit significant reconstruction discrepancies across different generative methods and that matching generation techniques provide more accurate reconstructions. Based on this insight, we propose a Multi-Reconstruction-based detector. By reversing and reconstructing images using multiple generative models, we analyze the reconstruction differences among real, GAN-generated, and DM-generated images to facilitate effective differentiation. Additionally, we introduce the Asian Synthetic Face Dataset (ASFD), containing synthetic Asian faces generated with various GANs and DMs. This dataset complements existing synthetic face datasets. Experimental results demonstrate that our detector achieves exceptional performance, with strong generalization and robustness.

Title: ID-Booth: Identity-consistent Face Generation with Diffusion Models

• Authors: Darian Tomašević, Fadi Boutros, Chenhao Lin, Naser Damer, Vitomir Štruc, Peter Peer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07392
• Pdf URL: https://arxiv.org/pdf/2504.07392
• Copy Paste: [[2504.07392]] ID-Booth: Identity-consistent Face Generation with Diffusion Models(https://arxiv.org/abs/2504.07392)
• Keywords: diffusion, generative
• Abstract: Recent advances in generative modeling have enabled the generation of high-quality synthetic data that is applicable in a variety of domains, including face recognition. Here, state-of-the-art generative models typically rely on conditioning and fine-tuning of powerful pretrained diffusion models to facilitate the synthesis of realistic images of a desired identity. Yet, these models often do not consider the identity of subjects during training, leading to poor consistency between generated and intended identities. In contrast, methods that employ identity-based training objectives tend to overfit on various aspects of the identity, and in turn, lower the diversity of images that can be generated. To address these issues, we present in this paper a novel generative diffusion-based framework, called ID-Booth. ID-Booth consists of a denoising network responsible for data generation, a variational auto-encoder for mapping images to and from a lower-dimensional latent space and a text encoder that allows for prompt-based control over the generation procedure. The framework utilizes a novel triplet identity training objective and enables identity-consistent image generation while retaining the synthesis capabilities of pretrained diffusion models. Experiments with a state-of-the-art latent diffusion model and diverse prompts reveal that our method facilitates better intra-identity consistency and inter-identity separability than competing methods, while achieving higher image diversity. In turn, the produced data allows for effective augmentation of small-scale datasets and training of better-performing recognition models in a privacy-preserving manner. The source code for the ID-Booth framework is publicly available at this https URL.

Title: ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Method

• Authors: Dongqi Fu, Yada Zhu, Zhining Liu, Lecheng Zheng, Xiao Lin, Zihao Li, Liri Fang, Katherine Tieu, Onkar Bhardwaj, Kommy Weldemariam, Hanghang Tong, Hendrik Hamann, Jingrui He
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07394
• Pdf URL: https://arxiv.org/pdf/2504.07394
• Copy Paste: [[2504.07394]] ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Method(https://arxiv.org/abs/2504.07394)
• Keywords: generative
• Abstract: Climate science studies the structure and dynamics of Earth's climate system and seeks to understand how climate changes over time, where the data is usually stored in the format of time series, recording the climate features, geolocation, time attributes, etc. Recently, much research attention has been paid to the climate benchmarks. In addition to the most common task of weather forecasting, several pioneering benchmark works are proposed for extending the modality, such as domain-specific applications like tropical cyclone intensity prediction and flash flood damage estimation, or climate statement and confidence level in the format of natural language. To further motivate the artificial general intelligence development for climate science, in this paper, we first contribute a multi-modal climate benchmark, i.e., ClimateBench-M, which aligns (1) the time series climate data from ERA5, (2) extreme weather events data from NOAA, and (3) satellite image data from NASA HLS based on a unified spatial-temporal granularity. Second, under each data modality, we also propose a simple but strong generative method that could produce competitive performance in weather forecasting, thunderstorm alerts, and crop segmentation tasks in the proposed ClimateBench-M. The data and code of ClimateBench-M are publicly available at this https URL.

Title: LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models

• Authors: Beilong Tang, Bang Zeng, Ming Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07402
• Pdf URL: https://arxiv.org/pdf/2504.07402
• Copy Paste: [[2504.07402]] LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models(https://arxiv.org/abs/2504.07402)
• Keywords: generative
• Abstract: We propose LauraTSE, an Auto-Regressive Decoder-Only Language Model for Target Speaker Extraction (TSE) based on the LauraGPT backbone. It employs a small-scale auto-regressive decoder-only language model which takes the continuous representations for both the mixture and the reference speeches and produces the first few layers of the target speech's discrete codec representations. In addition, a one-step encoder-only language model reconstructs the sum of the predicted codec embeddings using both the mixture and the reference information. Our approach achieves superior or comparable performance to existing generative and discriminative TSE models. To the best of our knowledge, LauraTSE is the first single-task TSE model to leverage an auto-regressive decoder-only language model as the backbone.

Title: FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generation

• Authors: Linyan Huang, Haonan Lin, Yanning Zhou, Kaiwen Xiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07405
• Pdf URL: https://arxiv.org/pdf/2504.07405
• Copy Paste: [[2504.07405]] FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generation(https://arxiv.org/abs/2504.07405)
• Keywords: generative
• Abstract: With the rapid advancement of 2D generative models, preserving subject identity while enabling diverse editing has emerged as a critical research focus. Existing methods typically face inherent trade-offs between identity preservation and personalized manipulation. We introduce FlexIP, a novel framework that decouples these objectives through two dedicated components: a Personalization Adapter for stylistic manipulation and a Preservation Adapter for identity maintenance. By explicitly injecting both control mechanisms into the generative model, our framework enables flexible parameterized control during inference through dynamic tuning of the weight adapter. Experimental results demonstrate that our approach breaks through the performance limitations of conventional methods, achieving superior identity preservation while supporting more diverse personalized generation capabilities (Project Page: this https URL).

Title: Unifying and extending Diffusion Models through PDEs for solving Inverse Problems

• Authors: Agnimitra Dasgupta, Alexsander Marciano da Cunha, Ali Fardisi, Mehrnegar Aminy, Brianna Binder, Bryan Shaddy, Assad A Oberai
• Subjects: cs.LG, stat.CO, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.07437
• Pdf URL: https://arxiv.org/pdf/2504.07437
• Copy Paste: [[2504.07437]] Unifying and extending Diffusion Models through PDEs for solving Inverse Problems(https://arxiv.org/abs/2504.07437)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as powerful generative tools with applications in computer vision and scientific machine learning (SciML), where they have been used to solve large-scale probabilistic inverse problems. Traditionally, these models have been derived using principles of variational inference, denoising, statistical signal processing, and stochastic differential equations. In contrast to the conventional presentation, in this study we derive diffusion models using ideas from linear partial differential equations and demonstrate that this approach has several benefits that include a constructive derivation of the forward and reverse processes, a unified derivation of multiple formulations and sampling strategies, and the discovery of a new class of models. We also apply the conditional version of these models to solving canonical conditional density estimation problems and challenging inverse problems. These problems help establish benchmarks for systematically quantifying the performance of different formulations and sampling strategies in this study, and for future studies. Finally, we identify and implement a mechanism through which a single diffusion model can be applied to measurements obtained from multiple measurement operators. Taken together, the contents of this manuscript provide a new understanding and several new directions in the application of diffusion models to solving physics-based inverse problems.

Title: Learning Universal Features for Generalizable Image Forgery Localization

• Authors: Hengrun Zhao, Yunzhi Zhuge, Yifan Wang, Lijun Wang, Huchuan Lu, Yu Zeng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07462
• Pdf URL: https://arxiv.org/pdf/2504.07462
• Copy Paste: [[2504.07462]] Learning Universal Features for Generalizable Image Forgery Localization(https://arxiv.org/abs/2504.07462)
• Keywords: generative
• Abstract: In recent years, advanced image editing and generation methods have rapidly evolved, making detecting and locating forged image content increasingly challenging. Most existing image forgery detection methods rely on identifying the edited traces left in the image. However, because the traces of different forgeries are distinct, these methods can identify familiar forgeries included in the training data but struggle to handle unseen ones. In response, we present an approach for Generalizable Image Forgery Localization (GIFL). Once trained, our model can detect both seen and unseen forgeries, providing a more practical and efficient solution to counter false information in the era of generative AI. Our method focuses on learning general features from the pristine content rather than traces of specific forgeries, which are relatively consistent across different types of forgeries and therefore can be used as universal features to locate unseen forgeries. Additionally, as existing image forgery datasets are still dominated by traditional hand-crafted forgeries, we construct a new dataset consisting of images edited by various popular deep generative image editing methods to further encourage research in detecting images manipulated by deep generative models. Extensive experimental results show that the proposed approach outperforms state-of-the-art methods in the detection of unseen forgeries and also demonstrates competitive results for seen forgeries. The code and dataset are available at this https URL.

Title: Event Signal Filtering via Probability Flux Estimation

• Authors: Jinze Chen, Wei Zhai, Yang Cao, Bin Li, Zheng-Jun Zha
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07503
• Pdf URL: https://arxiv.org/pdf/2504.07503
• Copy Paste: [[2504.07503]] Event Signal Filtering via Probability Flux Estimation(https://arxiv.org/abs/2504.07503)
• Keywords: diffusion, generative
• Abstract: Events offer a novel paradigm for capturing scene dynamics via asynchronous sensing, but their inherent randomness often leads to degraded signal quality. Event signal filtering is thus essential for enhancing fidelity by reducing this internal randomness and ensuring consistent outputs across diverse acquisition conditions. Unlike traditional time series that rely on fixed temporal sampling to capture steady-state behaviors, events encode transient dynamics through polarity and event intervals, making signal modeling significantly more complex. To address this, the theoretical foundation of event generation is revisited through the lens of diffusion processes. The state and process information within events is modeled as continuous probability flux at threshold boundaries of the underlying irradiance diffusion. Building on this insight, a generative, online filtering framework called Event Density Flow Filter (EDFilter) is introduced. EDFilter estimates event correlation by reconstructing the continuous probability flux from discrete events using nonparametric kernel smoothing, and then resamples filtered events from this flux. To optimize fidelity over time, spatial and temporal kernels are employed in a time-varying optimization framework. A fast recursive solver with O(1) complexity is proposed, leveraging state-space models and lookup tables for efficient likelihood computation. Furthermore, a new real-world benchmark Rotary Event Dataset (RED) is released, offering microsecond-level ground truth irradiance for full-reference event filtering evaluation. Extensive experiments validate EDFilter's performance across tasks like event filtering, super-resolution, and direct event-based blob tracking. Significant gains in downstream applications such as SLAM and video reconstruction underscore its robustness and effectiveness.

Title: GPT Carry-On: Training Foundation Model for Customization Could Be Simple, Scalable and Affordable

• Authors: Jianqiao Wangni
• Subjects: cs.LG, cs.AI, cs.DC, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.07513
• Pdf URL: https://arxiv.org/pdf/2504.07513
• Copy Paste: [[2504.07513]] GPT Carry-On: Training Foundation Model for Customization Could Be Simple, Scalable and Affordable(https://arxiv.org/abs/2504.07513)
• Keywords: foundation model
• Abstract: Modern large language foundation models (LLM) have now entered the daily lives of millions of users. We ask a natural question whether it is possible to customize LLM for every user or every task. From system and industrial economy consideration, general continue-training or fine-tuning still require substantial computation and memory of training GPU nodes, whereas most inference nodes under deployment, possibly with lower-end GPUs, are configured to make forward pass fastest possible. We propose a framework to take full advantages of existing LLMs and systems of online service. We train an additional branch of transformer blocks on the final-layer embedding of pretrained LLMs, which is the base, then a carry-on module merge the base models to compose a customized LLM. We can mix multiple layers, or multiple LLMs specialized in different domains such as chat, coding, math, to form a new mixture of LLM that best fit a new task. As the base model don't need to update parameters, we are able to outsource most computation of the training job on inference nodes, and only train a lightweight carry-on on training nodes, where we consume less than 1GB GPU memory to train a 100M carry-on layer on 30B LLM. We tested Qwen and DeepSeek opensourced models for continue-pretraining and got faster loss convergence. We use it to improve solving math questions with extremely small computation and model size, with 1000 data samples of chain-of-thoughts, and as small as 1 MB parameters of two layer layer carry-on, and the results are promising.

Title: Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data

• Authors: Jose Cribeiro-Ramallo, Federico Matteucci, Paul Enciu, Alexander Jenke, Vadim Arzamasov, Thorsten Strufe, Klemens Böhm
• Subjects: cs.LG, cs.AI, math.ST
• Abstract URL: https://arxiv.org/abs/2504.07522
• Pdf URL: https://arxiv.org/pdf/2504.07522
• Copy Paste: [[2504.07522]] Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data(https://arxiv.org/abs/2504.07522)
• Keywords: generative
• Abstract: Outlier detection in high-dimensional tabular data is challenging since data is often distributed across multiple lower-dimensional subspaces -- a phenomenon known as the Multiple Views effect (MV). This effect led to a large body of research focused on mining such subspaces, known as subspace selection. However, as the precise nature of the MV effect was not well understood, traditional methods had to rely on heuristic-driven search schemes that struggle to accurately capture the true structure of the data. Properly identifying these subspaces is critical for unsupervised tasks such as outlier detection or clustering, where misrepresenting the underlying data structure can hinder the performance. We introduce Myopic Subspace Theory (MST), a new theoretical framework that mathematically formulates the Multiple Views effect and writes subspace selection as a stochastic optimization problem. Based on MST, we introduce V-GAN, a generative method trained to solve such an optimization problem. This approach avoids any exhaustive search over the feature space while ensuring that the intrinsic data structure is preserved. Experiments on 42 real-world datasets show that using V-GAN subspaces to build ensemble methods leads to a significant increase in one-class classification performance -- compared to existing subspace selection, feature selection, and embedding methods. Further experiments on synthetic data show that V-GAN identifies subspaces more accurately while scaling better than other relevant subspace selection methods. These results confirm the theoretical guarantees of our approach and also highlight its practical viability in high-dimensional settings.

Title: STeP: A General and Scalable Framework for Solving Video Inverse Problems with Spatiotemporal Diffusion Priors

• Authors: Bingliang Zhang, Zihui Wu, Berthy T. Feng, Yang Song, Yisong Yue, Katherine L. Bouman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07549
• Pdf URL: https://arxiv.org/pdf/2504.07549
• Copy Paste: [[2504.07549]] STeP: A General and Scalable Framework for Solving Video Inverse Problems with Spatiotemporal Diffusion Priors(https://arxiv.org/abs/2504.07549)
• Keywords: diffusion
• Abstract: We study how to solve general Bayesian inverse problems involving videos using diffusion model priors. While it is desirable to use a video diffusion prior to effectively capture complex temporal relationships, due to the computational and data requirements of training such a model, prior work has instead relied on image diffusion priors on single frames combined with heuristics to enforce temporal consistency. However, these approaches struggle with faithfully recovering the underlying temporal relationships, particularly for tasks with high temporal uncertainty. In this paper, we demonstrate the feasibility of practical and accessible spatiotemporal diffusion priors by fine-tuning latent video diffusion models from pretrained image diffusion models using limited videos in specific domains. Leveraging this plug-and-play spatiotemporal diffusion prior, we introduce a general and scalable framework for solving video inverse problems. We then apply our framework to two challenging scientific video inverse problems--black hole imaging and dynamic MRI. Our framework enables the generation of diverse, high-fidelity video reconstructions that not only fit observations but also recover multi-modal solutions. By incorporating a spatiotemporal diffusion prior, we significantly improve our ability to capture complex temporal relationships in the data while also enhancing spatial fidelity.

Title: Diffusion Transformers for Tabular Data Time Series Generation

• Authors: Fabrizio Garuti, Enver Sangineto, Simone Luetto, Lorenzo Forni, Rita Cucchiara
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07566
• Pdf URL: https://arxiv.org/pdf/2504.07566
• Copy Paste: [[2504.07566]] Diffusion Transformers for Tabular Data Time Series Generation(https://arxiv.org/abs/2504.07566)
• Keywords: diffusion
• Abstract: Tabular data generation has recently attracted a growing interest due to its different application scenarios. However, generating time series of tabular data, where each element of the series depends on the others, remains a largely unexplored domain. This gap is probably due to the difficulty of jointly solving different problems, the main of which are the heterogeneity of tabular data (a problem common to non-time-dependent approaches) and the variable length of a time series. In this paper, we propose a Diffusion Transformers (DiTs) based approach for tabular data series generation. Inspired by the recent success of DiTs in image and video generation, we extend this framework to deal with heterogeneous data and variable-length sequences. Using extensive experiments on six datasets, we show that the proposed approach outperforms previous work by a large margin.

Title: Benchmarking Image Embeddings for E-Commerce: Evaluating Off-the Shelf Foundation Models, Fine-Tuning Strategies and Practical Trade-offs

• Authors: Urszula Czerwinska, Cenk Bircanoglu, Jeremy Chamoux
• Subjects: cs.CV, cs.AI, cs.CE, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07567
• Pdf URL: https://arxiv.org/pdf/2504.07567
• Copy Paste: [[2504.07567]] Benchmarking Image Embeddings for E-Commerce: Evaluating Off-the Shelf Foundation Models, Fine-Tuning Strategies and Practical Trade-offs(https://arxiv.org/abs/2504.07567)
• Keywords: self-supervised, foundation model
• Abstract: We benchmark foundation models image embeddings for classification and retrieval in e-Commerce, evaluating their suitability for real-world applications. Our study spans embeddings from pre-trained convolutional and transformer models trained via supervised, self-supervised, and text-image contrastive learning. We assess full fine-tuning and transfer learning (top-tuning) on six diverse e-Commerce datasets: fashion, consumer goods, cars, food, and retail. Results show full fine-tuning consistently performs well, while text-image and self-supervised embeddings can match its performance with less training. While supervised embeddings remain stable across architectures, SSL and contrastive embeddings vary significantly, often benefiting from top-tuning. Top-tuning emerges as an efficient alternative to full fine-tuning, reducing computational costs. We also explore cross-tuning, noting its impact depends on dataset characteristics. Our findings offer practical guidelines for embedding selection and fine-tuning strategies, balancing efficiency and performance.

Title: On Model and Data Scaling for Skeleton-based Self-Supervised Gait Recognition

• Authors: Adrian Cosma, Andy Cǎtrunǎ, Emilian Rǎdoi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07598
• Pdf URL: https://arxiv.org/pdf/2504.07598
• Copy Paste: [[2504.07598]] On Model and Data Scaling for Skeleton-based Self-Supervised Gait Recognition(https://arxiv.org/abs/2504.07598)
• Keywords: self-supervised
• Abstract: Gait recognition from video streams is a challenging problem in computer vision biometrics due to the subtle differences between gaits and numerous confounding factors. Recent advancements in self-supervised pretraining have led to the development of robust gait recognition models that are invariant to walking covariates. While neural scaling laws have transformed model development in other domains by linking performance to data, model size, and compute, their applicability to gait remains unexplored. In this work, we conduct the first empirical study scaling on skeleton-based self-supervised gait recognition to quantify the effect of data quantity, model size and compute on downstream gait recognition performance. We pretrain multiple variants of GaitPT - a transformer-based architecture - on a dataset of 2.7 million walking sequences collected in the wild. We evaluate zero-shot performance across four benchmark datasets to derive scaling laws for data, model size, and compute. Our findings demonstrate predictable power-law improvements in performance with increased scale and confirm that data and compute scaling significantly influence downstream accuracy. We further isolate architectural contributions by comparing GaitPT with GaitFormer under controlled compute budgets. These results provide practical insights into resource allocation and performance estimation for real-world gait recognition systems.

Title: PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization

• Authors: Yang Jiao, Xiaodong Wang, Kai Yang
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07717
• Pdf URL: https://arxiv.org/pdf/2504.07717
• Copy Paste: [[2504.07717]] PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization(https://arxiv.org/abs/2504.07717)
• Keywords: anomaly
• Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of applications, e.g., medical question-answering, mathematical sciences, and code generation. However, they also exhibit inherent limitations, such as outdated knowledge and susceptibility to hallucinations. Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to address these issues, but it also introduces new vulnerabilities. Recent efforts have focused on the security of RAG-based LLMs, yet existing attack methods face three critical challenges: (1) their effectiveness declines sharply when only a limited number of poisoned texts can be injected into the knowledge database, (2) they lack sufficient stealth, as the attacks are often detectable by anomaly detection systems, which compromises their effectiveness, and (3) they rely on heuristic approaches to generate poisoned texts, lacking formal optimization frameworks and theoretic guarantees, which limits their effectiveness and applicability. To address these issues, we propose coordinated Prompt-RAG attack (PR-attack), a novel optimization-driven attack that introduces a small number of poisoned texts into the knowledge database while embedding a backdoor trigger within the prompt. When activated, the trigger causes the LLM to generate pre-designed responses to targeted queries, while maintaining normal behavior in other contexts. This ensures both high effectiveness and stealth. We formulate the attack generation process as a bilevel optimization problem leveraging a principled optimization framework to develop optimal poisoned texts and triggers. Extensive experiments across diverse LLMs and datasets demonstrate the effectiveness of PR-Attack, achieving a high attack success rate even with a limited number of poisoned texts and significantly improved stealth compared to existing methods.

Title: SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding

• Authors: Yangliu Hu, Zikai Song, Na Feng, Yawei Luo, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07745
• Pdf URL: https://arxiv.org/pdf/2504.07745
• Copy Paste: [[2504.07745]] SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding(https://arxiv.org/abs/2504.07745)
• Keywords: self-supervised
• Abstract: Video-based Large Language Models (Video-LLMs) have witnessed substantial advancements in recent years, propelled by the advancement in multi-modal LLMs. Although these models have demonstrated proficiency in providing the overall description of videos, they struggle with fine-grained understanding, particularly in aspects such as visual dynamics and video details inquiries. To tackle these shortcomings, we find that fine-tuning Video-LLMs on self-supervised fragment tasks, greatly improve their fine-grained video understanding abilities. Hence we propose two key contributions:(1) Self-Supervised Fragment Fine-Tuning (SF$^2$T), a novel effortless fine-tuning method, employs the rich inherent characteristics of videos for training, while unlocking more fine-grained understanding ability of Video-LLMs. Moreover, it relieves researchers from labor-intensive annotations and smartly circumvents the limitations of natural language, which often fails to capture the complex spatiotemporal variations in videos; (2) A novel benchmark dataset, namely FineVidBench, for rigorously assessing Video-LLMs' performance at both the scene and fragment levels, offering a comprehensive evaluation of their capabilities. We assessed multiple models and validated the effectiveness of SF$^2$T on them. Experimental results reveal that our approach improves their ability to capture and interpret spatiotemporal details.

Title: NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark

• Authors: Vladislav Mikhailov, Tita Enstad, David Samuel, Hans Christian Farsethås, Andrey Kutuzov, Erik Velldal, Lilja Øvrelid
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07749
• Pdf URL: https://arxiv.org/pdf/2504.07749
• Copy Paste: [[2504.07749]] NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark(https://arxiv.org/abs/2504.07749)
• Keywords: generative
• Abstract: This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of 24 high-quality human-created datasets -- of which five are created from scratch. In contrast to existing benchmarks for Norwegian, NorEval covers a broad spectrum of task categories targeting Norwegian language understanding and generation, establishes human baselines, and focuses on both of the official written standards of the Norwegian language: Bokmål and Nynorsk. All our datasets and a collection of over 100 human-written prompts are integrated into LM Evaluation Harness, ensuring flexible and reproducible evaluation. We describe the NorEval design and present the results of benchmarking 19 open-source pre-trained and instruction-tuned LMs for Norwegian in various scenarios. Our benchmark, evaluation framework, and annotation materials are publicly available.

Title: Exploring a Patch-Wise Approach for Privacy-Preserving Fake ID Detection

• Authors: Javier Muñoz-Haro, Ruben Tolosana, Ruben Vera-Rodriguez, Aythami Morales, Julian Fierrez
• Subjects: cs.CV, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.07761
• Pdf URL: https://arxiv.org/pdf/2504.07761
• Copy Paste: [[2504.07761]] Exploring a Patch-Wise Approach for Privacy-Preserving Fake ID Detection(https://arxiv.org/abs/2504.07761)
• Keywords: foundation model
• Abstract: In an increasingly digitalized world, verifying the authenticity of ID documents has become a critical challenge for real-life applications such as digital banking, crypto-exchanges, renting, etc. This study focuses on the topic of fake ID detection, covering several limitations in the field. In particular, no publicly available data from real ID documents exists, and most studies rely on proprietary in-house databases that are not available due to privacy reasons. In order to shed some light on this critical challenge that makes difficult to advance in the field, we explore a trade-off between privacy (i.e., amount of sensitive data available) and performance, proposing a novel patch-wise approach for privacy-preserving fake ID detection. Our proposed approach explores how privacy can be enhanced through: i) two levels of anonymization for an ID document (i.e., fully- and pseudo-anonymized), and ii) different patch size configurations, varying the amount of sensitive data visible in the patch image. Also, state-of-the-art methods such as Vision Transformers and Foundation Models are considered in the analysis. The experimental framework shows that, on an unseen database (DLC-2021), our proposal achieves 13.91% and 0% EERs at patch and ID document level, showing a good generalization to other databases. In addition to this exploration, another key contribution of our study is the release of the first publicly available database that contains 48,400 patches from both real and fake ID documents, along with the experimental framework and models, which will be available in our GitHub.

Title: Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations

• Authors: Yifan Ding, Arturas Aleksandrauskas, Amirhossein Ahmadian, Jonas Unger, Fredrik Lindsten, Gabriel Eilertsen
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07793
• Pdf URL: https://arxiv.org/pdf/2504.07793
• Copy Paste: [[2504.07793]] Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations(https://arxiv.org/abs/2504.07793)
• Keywords: diffusion, generative
• Abstract: Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning systems, particularly in safety-critical applications. Likelihood-based deep generative models have historically faced criticism for their unsatisfactory performance in OOD detection, often assigning higher likelihood to OOD data than in-distribution samples when applied to image data. In this work, we demonstrate that likelihood is not inherently flawed. Rather, several properties in the images space prohibit likelihood as a valid detection score. Given a sufficiently good likelihood estimator, specifically using the probability flow formulation of a diffusion model, we show that likelihood-based methods can still perform on par with state-of-the-art methods when applied in the representation space of pre-trained encoders. The code of our work can be found at $\href{this https URL}{\texttt{this https URL}}$.

Title: What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks

• Authors: Pavel Chizhov, Mattia Nee, Pierre-Carl Langlais, Ivan P. Yamshchikov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.07825
• Pdf URL: https://arxiv.org/pdf/2504.07825
• Copy Paste: [[2504.07825]] What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks(https://arxiv.org/abs/2504.07825)
• Keywords: generative
• Abstract: Common-sense reasoning is a key language model capability because it encapsulates not just specific factual knowledge but rather general language and world understanding. Measuring common-sense reasoning, therefore, is crucial for language models of different sizes and applications. One of the most widely used benchmarks for evaluating such capabilities is HellaSwag; however, in this paper, we show that it has severe construct validity issues. These issues range from basic ungrammaticality and numerous typos to misleading prompts or equally correct options. Furthermore, we show that if models are evaluated only on answer texts, or with "Lorem ipsum dolor..." instead of the question, more than 65% of model predictions remain the same, and this cannot be attributed merely to contamination. Since benchmark scores are an essential part of model selection in both research and commercial applications, these validity issues can have severe consequences. In particular, knowing that taking benchmark scores at face value is ubiquitous, inadequate evaluation leads to ill-informed decisions about models. In this paper, we thoroughly investigate critical validity issues posed by HellaSwag and illustrate them with various evaluations using generative language models of different sizes. We argue that this benchmark does not accurately measure common-sense reasoning and, therefore, should not be used for evaluation in its current state. Based on the results of our study, we propose requirements that should be met by future common-sense reasoning benchmarks. In addition, we release GoldenSwag, a corrected subset of HellaSwag, which, to our belief, facilitates acceptable common-sense reasoning evaluation.

Title: MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations

• Authors: Genglin Liu, Salman Rahman, Elisa Kreiss, Marzyeh Ghassemi, Saadia Gabriel
• Subjects: cs.CL, cs.AI, cs.SI
• Abstract URL: https://arxiv.org/abs/2504.07830
• Pdf URL: https://arxiv.org/pdf/2504.07830
• Copy Paste: [[2504.07830]] MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations(https://arxiv.org/abs/2504.07830)
• Keywords: generative
• Abstract: We present a novel, open-source social network simulation framework, MOSAIC, where generative language agents predict user behaviors such as liking, sharing, and flagging content. This simulation combines LLM agents with a directed social graph to analyze emergent deception behaviors and gain a better understanding of how users determine the veracity of online social content. By constructing user representations from diverse fine-grained personas, our system enables multi-agent simulations that model content dissemination and engagement dynamics at scale. Within this framework, we evaluate three different content moderation strategies with simulated misinformation dissemination, and we find that they not only mitigate the spread of non-factual content but also increase user engagement. In addition, we analyze the trajectories of popular content in our simulations, and explore whether simulation agents' articulated reasoning for their social interactions truly aligns with their collective engagement patterns. We open-source our simulation software to encourage further research within AI and social sciences.

Title: SAMJAM: Zero-Shot Video Scene Graph Generation for Egocentric Kitchen Videos

• Authors: Joshua Li, Fernando Jose Pena Cantu, Emily Yu, Alexander Wong, Yuchen Cui, Yuhao Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07867
• Pdf URL: https://arxiv.org/pdf/2504.07867
• Copy Paste: [[2504.07867]] SAMJAM: Zero-Shot Video Scene Graph Generation for Egocentric Kitchen Videos(https://arxiv.org/abs/2504.07867)
• Keywords: foundation model
• Abstract: Video Scene Graph Generation (VidSGG) is an important topic in understanding dynamic kitchen environments. Current models for VidSGG require extensive training to produce scene graphs. Recently, Vision Language Models (VLM) and Vision Foundation Models (VFM) have demonstrated impressive zero-shot capabilities in a variety of tasks. However, VLMs like Gemini struggle with the dynamics for VidSGG, failing to maintain stable object identities across frames. To overcome this limitation, we propose SAMJAM, a zero-shot pipeline that combines SAM2's temporal tracking with Gemini's semantic understanding. SAM2 also improves upon Gemini's object grounding by producing more accurate bounding boxes. In our method, we first prompt Gemini to generate a frame-level scene graph. Then, we employ a matching algorithm to map each object in the scene graph with a SAM2-generated or SAM2-propagated mask, producing a temporally-consistent scene graph in dynamic environments. Finally, we repeat this process again in each of the following frames. We empirically demonstrate that SAMJAM outperforms Gemini by 8.33% in mean recall on the EPIC-KITCHENS and EPIC-KITCHENS-100 datasets.

Title: DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows

• Authors: Mashrur M. Morshed, Vishnu Boddeti
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07894
• Pdf URL: https://arxiv.org/pdf/2504.07894
• Copy Paste: [[2504.07894]] DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows(https://arxiv.org/abs/2504.07894)
• Keywords: generative
• Abstract: Many real-world applications of flow-based generative models desire a diverse set of samples that cover multiple modes of the target distribution. However, the predominant approach for obtaining diverse sets is not sample-efficient, as it involves independently obtaining many samples from the source distribution and mapping them through the flow until the desired mode coverage is achieved. As an alternative to repeated sampling, we introduce DiverseFlow: a training-free approach to improve the diversity of flow models. Our key idea is to employ a determinantal point process to induce a coupling between the samples that drives diversity under a fixed sampling budget. In essence, DiverseFlow allows exploration of more variations in a learned flow model with fewer samples. We demonstrate the efficacy of our method for tasks where sample-efficient diversity is desirable, such as text-guided image generation with polysemous words, inverse problems like large-hole inpainting, and class-conditional image synthesis.

Title: Fast Adaptation with Behavioral Foundation Models

• Authors: Harshit Sikchi, Andrea Tirinzoni, Ahmed Touati, Yingchen Xu, Anssi Kanervisto, Scott Niekum, Amy Zhang, Alessandro Lazaric, Matteo Pirotta
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.07896
• Pdf URL: https://arxiv.org/pdf/2504.07896
• Copy Paste: [[2504.07896]] Fast Adaptation with Behavioral Foundation Models(https://arxiv.org/abs/2504.07896)
• Keywords: self-supervised, foundation model
• Abstract: Unsupervised zero-shot reinforcement learning (RL) has emerged as a powerful paradigm for pretraining behavioral foundation models (BFMs), enabling agents to solve a wide range of downstream tasks specified via reward functions in a zero-shot fashion, i.e., without additional test-time learning or planning. This is achieved by learning self-supervised task embeddings alongside corresponding near-optimal behaviors and incorporating an inference procedure to directly retrieve the latent task embedding and associated policy for any given reward function. Despite promising results, zero-shot policies are often suboptimal due to errors induced by the unsupervised training process, the embedding, and the inference procedure. In this paper, we focus on devising fast adaptation strategies to improve the zero-shot performance of BFMs in a few steps of online interaction with the environment while avoiding any performance drop during the adaptation process. Notably, we demonstrate that existing BFMs learn a set of skills containing more performant policies than those identified by their inference procedure, making them well-suited for fast adaptation. Motivated by this observation, we propose both actor-critic and actor-only fast adaptation strategies that search in the low-dimensional task-embedding space of the pre-trained BFM to rapidly improve the performance of its zero-shot policies on any downstream task. Notably, our approach mitigates the initial "unlearning" phase commonly observed when fine-tuning pre-trained RL models. We evaluate our fast adaptation strategies on top of four state-of-the-art zero-shot RL methods in multiple navigation and locomotion domains. Our results show that they achieve 10-40% improvement over their zero-shot performance in a few tens of episodes, outperforming existing baselines.

Title: Hodge Laplacians and Hodge Diffusion Maps

• Authors: Alvaro Almeida Gomez, Jorge Duque Franco
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07910
• Pdf URL: https://arxiv.org/pdf/2504.07910
• Copy Paste: [[2504.07910]] Hodge Laplacians and Hodge Diffusion Maps(https://arxiv.org/abs/2504.07910)
• Keywords: diffusion
• Abstract: We introduce Hodge Diffusion Maps, a novel manifold learning algorithm designed to analyze and extract topological information from high-dimensional data-sets. This method approximates the exterior derivative acting on differential forms, thereby providing an approximation of the Hodge Laplacian operator. Hodge Diffusion Maps extend existing non-linear dimensionality reduction techniques, including vector diffusion maps, as well as the theories behind diffusion maps and Laplacian Eigenmaps. Our approach captures higher-order topological features of the data-set by projecting it into lower-dimensional Euclidean spaces using the Hodge Laplacian. We develop a theoretical framework to estimate the approximation error of the exterior derivative, based on sample points distributed over a real manifold. Numerical experiments support and validate the proposed methodology.

Title: HoloPart: Generative 3D Part Amodal Segmentation

• Authors: Yunhan Yang, Yuan-Chen Guo, Yukun Huang, Zi-Xin Zou, Zhipeng Yu, Yangguang Li, Yan-Pei Cao, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07943
• Pdf URL: https://arxiv.org/pdf/2504.07943
• Copy Paste: [[2504.07943]] HoloPart: Generative 3D Part Amodal Segmentation(https://arxiv.org/abs/2504.07943)
• Keywords: diffusion, generative
• Abstract: 3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.

Title: GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces

• Authors: Hao Yu, Rupayan Mallick, Margrit Betke, Sarah Adel Bargal
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.07945
• Pdf URL: https://arxiv.org/pdf/2504.07945
• Copy Paste: [[2504.07945]] GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces(https://arxiv.org/abs/2504.07945)
• Keywords: diffusion
• Abstract: Cartoon avatars have been widely used in various applications, including social media, online tutoring, and gaming. However, existing cartoon avatar datasets and generation methods struggle to present highly expressive avatars with fine-grained facial expressions and are often inspired from real-world identities, raising privacy concerns. To address these challenges, we propose a novel framework, GenEAva, for generating high-quality cartoon avatars with fine-grained facial expressions. Our approach fine-tunes a state-of-the-art text-to-image diffusion model to synthesize highly detailed and expressive facial expressions. We then incorporate a stylization model that transforms these realistic faces into cartoon avatars while preserving both identity and expression. Leveraging this framework, we introduce the first expressive cartoon avatar dataset, GenEAva 1.0, specifically designed to capture 135 fine-grained facial expressions, featuring 13,230 expressive cartoon avatars with a balanced distribution across genders, racial groups, and age ranges. We demonstrate that our fine-tuned model generates more expressive faces than the state-of-the-art text-to-image diffusion model SDXL. We also verify that the cartoon avatars generated by our framework do not include memorized identities from fine-tuning data. The proposed framework and dataset provide a diverse and expressive benchmark for future research in cartoon avatar generation.

Title: Detect Anything 3D in the Wild

• Authors: Hanxue Zhang, Haoran Jiang, Qingsong Yao, Yanan Sun, Renrui Zhang, Hao Zhao, Hongyang Li, Hongzi Zhu, Zetong Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07958
• Pdf URL: https://arxiv.org/pdf/2504.07958
• Copy Paste: [[2504.07958]] Detect Anything 3D in the Wild(https://arxiv.org/abs/2504.07958)
• Keywords: foundation model
• Abstract: Despite the success of deep learning in close-set 3D object detection, existing approaches struggle with zero-shot generalization to novel objects and camera configurations. We introduce DetAny3D, a promptable 3D detection foundation model capable of detecting any novel object under arbitrary camera configurations using only monocular inputs. Training a foundation model for 3D detection is fundamentally constrained by the limited availability of annotated 3D data, which motivates DetAny3D to leverage the rich prior knowledge embedded in extensively pre-trained 2D foundation models to compensate for this scarcity. To effectively transfer 2D knowledge to 3D, DetAny3D incorporates two core modules: the 2D Aggregator, which aligns features from different 2D foundation models, and the 3D Interpreter with Zero-Embedding Mapping, which mitigates catastrophic forgetting in 2D-to-3D knowledge transfer. Experimental results validate the strong generalization of our DetAny3D, which not only achieves state-of-the-art performance on unseen categories and novel camera configurations, but also surpasses most competitors on in-domain data.DetAny3D sheds light on the potential of the 3D foundation model for diverse applications in real-world scenarios, e.g., rare object detection in autonomous driving, and demonstrates promise for further exploration of 3D-centric tasks in open-world settings. More visualization results can be found at DetAny3D project page.

Title: VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning

• Authors: Zhong-Yu Li, Ruoyi Du, Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao, Zhanyu Ma, Ming-Ming Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07960
• Pdf URL: https://arxiv.org/pdf/2504.07960
• Copy Paste: [[2504.07960]] VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning(https://arxiv.org/abs/2504.07960)
• Keywords: diffusion, generative, in-context
• Abstract: Recent progress in diffusion models significantly advances various image generation tasks. However, the current mainstream approach remains focused on building task-specific models, which have limited efficiency when supporting a wide range of different needs. While universal models attempt to address this limitation, they face critical challenges, including generalizable task instruction, appropriate task distributions, and unified architectural design. To tackle these challenges, we propose VisualCloze, a universal image generation framework, which supports a wide range of in-domain tasks, generalization to unseen ones, unseen unification of multiple tasks, and reverse generation. Unlike existing methods that rely on language-based task instruction, leading to task ambiguity and weak generalization, we integrate visual in-context learning, allowing models to identify tasks from visual demonstrations. Meanwhile, the inherent sparsity of visual task distributions hampers the learning of transferable knowledge across tasks. To this end, we introduce Graph200K, a graph-structured dataset that establishes various interrelated tasks, enhancing task density and transferable knowledge. Furthermore, we uncover that our unified image generation formulation shared a consistent objective with image infilling, enabling us to leverage the strong generative priors of pre-trained infilling models without modifying the architectures.

Title: Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction

• Authors: Zeren Jiang, Chuanxia Zheng, Iro Laina, Diane Larlus, Andrea Vedaldi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07961
• Pdf URL: https://arxiv.org/pdf/2504.07961
• Copy Paste: [[2504.07961]] Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction(https://arxiv.org/abs/2504.07961)
• Keywords: diffusion
• Abstract: We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic prior captured by such video models, Geo4D can be trained using only synthetic data while generalizing well to real data in a zero-shot manner. Geo4D predicts several complementary geometric modalities, namely point, depth, and ray maps. It uses a new multi-modal alignment algorithm to align and fuse these modalities, as well as multiple sliding windows, at inference time, thus obtaining robust and accurate 4D reconstruction of long videos. Extensive experiments across multiple benchmarks show that Geo4D significantly surpasses state-of-the-art video depth estimation methods, including recent methods such as MonST3R, which are also designed to handle dynamic scenes.

Title: PixelFlow: Pixel-Space Generative Models with Flow

• Authors: Shoufa Chen, Chongjian Ge, Shilong Zhang, Peize Sun, Ping Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.07963
• Pdf URL: https://arxiv.org/pdf/2504.07963
• Copy Paste: [[2504.07963]] PixelFlow: Pixel-Space Generative Models with Flow(https://arxiv.org/abs/2504.07963)
• Keywords: generative
• Abstract: We present PixelFlow, a family of image generation models that operate directly in the raw pixel space, in contrast to the predominant latent-space models. This approach simplifies the image generation process by eliminating the need for a pre-trained Variational Autoencoder (VAE) and enabling the whole model end-to-end trainable. Through efficient cascade flow modeling, PixelFlow achieves affordable computation cost in pixel space. It achieves an FID of 1.98 on 256$\times$256 ImageNet class-conditional image generation benchmark. The qualitative text-to-image results demonstrate that PixelFlow excels in image quality, artistry, and semantic control. We hope this new paradigm will inspire and open up new opportunities for next-generation visual generation models. Code and models are available at this https URL.

Title: C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing

• Authors: Zhongyang Li, Ziyue Li, Tianyi Zhou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.07964
• Pdf URL: https://arxiv.org/pdf/2504.07964
• Copy Paste: [[2504.07964]] C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing(https://arxiv.org/abs/2504.07964)
• Keywords: in-context
• Abstract: Mixture-of-Experts (MoE) Large Language Models (LLMs) suffer from severely sub-optimal expert pathways-our study reveals that naive expert selection learned from pretraining leaves a surprising 10-20% accuracy gap for improvement. Motivated by this observation, we develop a novel class of test-time optimization methods to re-weight or "re-mixing" the experts in different layers jointly for each test sample. Since the test sample's ground truth is unknown, we propose to optimize a surrogate objective defined by the sample's "successful neighbors" from a reference set of samples. We introduce three surrogates and algorithms based on mode-finding, kernel regression, and the average loss of similar reference samples/tasks. To reduce the cost of optimizing whole pathways, we apply our algorithms merely to the core experts' mixing weights in critical layers, which enjoy similar performance but save significant computation. This leads to "Critical-Layer, Core-Expert, Collaborative Pathway Optimization (C3PO)". We apply C3PO to two recent MoE LLMs and examine it on six widely-used benchmarks. It consistently improves the base model by 7-15% in accuracy and outperforms widely used test-time learning baselines, e.g., in-context learning and prompt/prefix tuning, by a large margin. Moreover, C3PO enables MoE LLMs with 1-3B active parameters to outperform LLMs of 7-9B parameters, hence improving MoE's advantages on efficiency. Our thorough ablation study further sheds novel insights on achieving test-time improvement on MoE.

Title: Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments

• Authors: Lorenz Linhardt, Tom Neuhäuser, Lenka Tětková, Oliver Eberle
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.07965
• Pdf URL: https://arxiv.org/pdf/2504.07965
• Copy Paste: [[2504.07965]] Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments(https://arxiv.org/abs/2504.07965)
• Keywords: generative
• Abstract: Small and mid-sized generative language models have gained increasing attention. Their size and availability make them amenable to being analyzed at a behavioral as well as a representational level, allowing investigations of how these levels interact. We evaluate 32 publicly available language models for their representational and behavioral alignment with human similarity judgments on a word triplet task. This provides a novel evaluation setting to probe semantic associations in language beyond common pairwise comparisons. We find that (1) even the representations of small language models can achieve human-level alignment, (2) instruction-tuned model variants can exhibit substantially increased agreement, (3) the pattern of alignment across layers is highly model dependent, and (4) alignment based on models' behavioral responses is highly dependent on model size, matching their representational alignment only for the largest evaluated models.