2025-02-19

Title: Ten Challenging Problems in Federated Foundation Models

• Authors: Tao Fan, Hanlin Gu, Xuemei Cao, Chee Seng Chan, Qian Chen, Yiqiang Chen, Yihui Feng, Yang Gu, Jiaxiang Geng, Bing Luo, Shuoling Liu, Win Kent Ong, Chao Ren, Jiaqi Shao, Chuan Sun, Xiaoli Tang, Hong Xi Tae, Yongxin Tong, Shuyue Wei, Fan Wu, Wei Xi, Mingcong Xu, He Yang, Xin Yang, Jiangpeng Yan, Hao Yu, Han Yu, Teng Zhang, Yifei Zhang, Xiaojin Zhang, Zhenzhe Zheng, Lixin Fan, Qiang Yang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12176
• Pdf URL: https://arxiv.org/pdf/2502.12176
• Copy Paste: [[2502.12176]] Ten Challenging Problems in Federated Foundation Models(https://arxiv.org/abs/2502.12176)
• Keywords: foundation model
• Abstract: Federated Foundation Models (FedFMs) represent a distributed learning paradigm that fuses general competences of foundation models as well as privacy-preserving capabilities of federated learning. This combination allows the large foundation models and the small local domain models at the remote clients to learn from each other in a teacher-student learning setting. This paper provides a comprehensive summary of the ten challenging problems inherent in FedFMs, encompassing foundational theory, utilization of private data, continual learning, unlearning, Non-IID and graph data, bidirectional knowledge transfer, incentive mechanism design, game mechanism design, model watermarking, and efficiency. The ten challenging problems manifest in five pivotal aspects: ``Foundational Theory," which aims to establish a coherent and unifying theoretical framework for FedFMs. ``Data," addressing the difficulties in leveraging domain-specific knowledge from private data while maintaining privacy; ``Heterogeneity," examining variations in data, model, and computational resources across clients; ``Security and Privacy," focusing on defenses against malicious attacks and model theft; and ``Efficiency," highlighting the need for improvements in training, communication, and parameter efficiency. For each problem, we offer a clear mathematical definition on the objective function, analyze existing methods, and discuss the key challenges and potential solutions. This in-depth exploration aims to advance the theoretical foundations of FedFMs, guide practical implementations, and inspire future research to overcome these obstacles, thereby enabling the robust, efficient, and privacy-preserving FedFMs in various real-world applications.

Title: Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation

• Authors: Seungjun Yu, Kisung Kim, Daejung Kim, Haewook Han, Jinhan Lee
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2502.12178
• Pdf URL: https://arxiv.org/pdf/2502.12178
• Copy Paste: [[2502.12178]] Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation(https://arxiv.org/abs/2502.12178)
• Keywords: diffusion
• Abstract: Diffusion-based models are recognized for their effectiveness in using real-world driving data to generate realistic and diverse traffic scenarios. These models employ guided sampling to incorporate specific traffic preferences and enhance scenario realism. However, guiding the sampling process to conform to traffic rules and preferences can result in deviations from real-world traffic priors and potentially leading to unrealistic behaviors. To address this challenge, we introduce a multi-guided diffusion model that utilizes a novel training strategy to closely adhere to traffic priors, even when employing various combinations of guides. This model adopts a multi-task learning framework, enabling a single diffusion model to process various guide inputs. For increased guided sampling precision, our model is fine-tuned using the Direct Preference Optimization (DPO) algorithm. This algorithm optimizes preferences based on guide scores, effectively navigating the complexities and challenges associated with the expensive and often non-differentiable gradient calculations during the guided sampling fine-tuning process. Evaluated using the nuScenes dataset our model provides a strong baseline for balancing realism, diversity and controllability in the traffic scenario generation.

Title: E2CB2former: Effecitve and Explainable Transformer for CB2 Receptor Ligand Activity Prediction

• Authors: Jiacheng Xie, Yingrui Ji, Linghuan Zeng, Xi Xiao, Gaofei Chen, Lijing Zhu, Joyanta Jyoti Mondal, Jiansheng Chen
• Subjects: cs.LG, cs.AI, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.12186
• Pdf URL: https://arxiv.org/pdf/2502.12186
• Copy Paste: [[2502.12186]] E2CB2former: Effecitve and Explainable Transformer for CB2 Receptor Ligand Activity Prediction(https://arxiv.org/abs/2502.12186)
• Keywords: generative
• Abstract: Accurate prediction of CB2 receptor ligand activity is pivotal for advancing drug discovery targeting this receptor, which is implicated in inflammation, pain management, and neurodegenerative conditions. Although conventional machine learning and deep learning techniques have shown promise, their limited interpretability remains a significant barrier to rational drug design. In this work, we introduce CB2former, a framework that combines a Graph Convolutional Network with a Transformer architecture to predict CB2 receptor ligand activity. By leveraging the Transformer's self attention mechanism alongside the GCN's structural learning capability, CB2former not only enhances predictive performance but also offers insights into the molecular features underlying receptor activity. We benchmark CB2former against diverse baseline models including Random Forest, Support Vector Machine, K Nearest Neighbors, Gradient Boosting, Extreme Gradient Boosting, Multilayer Perceptron, Convolutional Neural Network, and Recurrent Neural Network and demonstrate its superior performance with an R squared of 0.685, an RMSE of 0.675, and an AUC of 0.940. Moreover, attention weight analysis reveals key molecular substructures influencing CB2 receptor activity, underscoring the model's potential as an interpretable AI tool for drug discovery. This ability to pinpoint critical molecular motifs can streamline virtual screening, guide lead optimization, and expedite therapeutic development. Overall, our results showcase the transformative potential of advanced AI approaches exemplified by CB2former in delivering both accurate predictions and actionable molecular insights, thus fostering interdisciplinary collaboration and innovation in drug discovery.

Title: Boosting Generalization in Diffusion-Based Neural Combinatorial Solver via Energy-guided Sampling

• Authors: Haoyu Lei, Kaiwen Zhou, Yinchuan Li, Zhitang Chen, Farzan Farnia
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12188
• Pdf URL: https://arxiv.org/pdf/2502.12188
• Copy Paste: [[2502.12188]] Boosting Generalization in Diffusion-Based Neural Combinatorial Solver via Energy-guided Sampling(https://arxiv.org/abs/2502.12188)
• Keywords: diffusion
• Abstract: Diffusion-based Neural Combinatorial Optimization (NCO) has demonstrated effectiveness in solving NP-complete (NPC) problems by learning discrete diffusion models for solution generation, eliminating hand-crafted domain knowledge. Despite their success, existing NCO methods face significant challenges in both cross-scale and cross-problem generalization, and high training costs compared to traditional solvers. While recent studies have introduced training-free guidance approaches that leverage pre-defined guidance functions for zero-shot conditional generation, such methodologies have not been extensively explored in combinatorial optimization. To bridge this gap, we propose a general energy-guided sampling framework during inference time that enhances both the cross-scale and cross-problem generalization capabilities of diffusion-based NCO solvers without requiring additional training. We provide theoretical analysis that helps understanding the cross-problem transfer capability. Our experimental results demonstrate that a diffusion solver, trained exclusively on the Traveling Salesman Problem (TSP), can achieve competitive zero-shot solution generation on TSP variants, such as Prize Collecting TSP (PCTSP) and the Orienteering Problem (OP), through energy-guided sampling across different problem scales.

Title: Self-supervised Attribute-aware Dynamic Preference Ranking Alignment

• Authors: Hongyu Yang, Qi Zhao, Zhenhua hu, Rui Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12189
• Pdf URL: https://arxiv.org/pdf/2502.12189
• Copy Paste: [[2502.12189]] Self-supervised Attribute-aware Dynamic Preference Ranking Alignment(https://arxiv.org/abs/2502.12189)
• Keywords: self-supervised
• Abstract: Reinforcement Learning from Human Feedback and its variants excel in aligning with human intentions to generate helpful, harmless, and honest responses. However, most of them rely on costly human-annotated pairwise comparisons for supervised alignment, which is not suitable for list-level scenarios, such as community question answering. Additionally, human preferences are influenced by multiple intrinsic factors in responses, leading to decision-making inconsistencies. Therefore, we propose \textbf{Se}lf-supervised \textbf{A}ttribute-aware \textbf{d}ynamic \textbf{p}reference \textbf{ra}nking, called \shortname. \ It quantifies preference differences between responses based on Attribute-Perceptual Distance Factors (APDF) and dynamically determines the list-wise alignment order. Furthermore, it achieves fine-grained preference difference learning and enables precise alignment with the optimal one. We specifically constructed a challenging code preference dataset named StaCoCoQA, and introduced more cost-effective and scalable preference evaluation metrics: PrefHit and PrefRecall. Extensive experimental results show that SeAdpra exhibits superior performance and generalizability on both StaCoCoQA and preference datasets from eight popular domains.

Title: Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control

• Authors: Dom Huh, Prasant Mohapatra
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12198
• Pdf URL: https://arxiv.org/pdf/2502.12198
• Copy Paste: [[2502.12198]] Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control(https://arxiv.org/abs/2502.12198)
• Keywords: diffusion
• Abstract: Diffusion-based planning, learning, and control methods present a promising branch of powerful and expressive decision-making solutions. Given the growing interest, such methods have undergone numerous refinements over the past years. However, despite these advancements, existing methods are limited in their investigations regarding general methods for reward maximization within the decision-making process. In this work, we study extensions of fine-tuning approaches for control applications. Specifically, we explore extensions and various design choices for four fine-tuning approaches: reward alignment through reinforcement learning, direct preference optimization, supervised fine-tuning, and cascading diffusion. We optimize their usage to merge these independent efforts into one unified paradigm. We show the utility of such propositions in offline RL settings and demonstrate empirical improvements over a rich array of control tasks.

Title: An Interpretable Automated Mechanism Design Framework with Large Language Models

• Authors: Jiayuan Liu, Mingyu Guo, Vincent Conitzer
• Subjects: cs.LG, cs.AI, cs.GT, cs.NE
• Abstract URL: https://arxiv.org/abs/2502.12203
• Pdf URL: https://arxiv.org/pdf/2502.12203
• Copy Paste: [[2502.12203]] An Interpretable Automated Mechanism Design Framework with Large Language Models(https://arxiv.org/abs/2502.12203)
• Keywords: generative
• Abstract: Mechanism design has long been a cornerstone of economic theory, with traditional approaches relying on mathematical derivations. Recently, automated approaches, including differentiable economics with neural networks, have emerged for designing payments and allocations. While both analytical and automated methods have advanced the field, they each face significant weaknesses: mathematical derivations are not automated and often struggle to scale to complex problems, while automated and especially neural-network-based approaches suffer from limited interpretability. To address these challenges, we introduce a novel framework that reformulates mechanism design as a code generation task. Using large language models (LLMs), we generate heuristic mechanisms described in code and evolve them to optimize over some evaluation metrics while ensuring key design criteria (e.g., strategy-proofness) through a problem-specific fixing process. This fixing process ensures any mechanism violating the design criteria is adjusted to satisfy them, albeit with some trade-offs in performance metrics. These trade-offs are factored in during the LLM-based evolution process. The code generation capabilities of LLMs enable the discovery of novel and interpretable solutions, bridging the symbolic logic of mechanism design and the generative power of modern AI. Through rigorous experimentation, we demonstrate that LLM-generated mechanisms achieve competitive performance while offering greater interpretability compared to previous approaches. Notably, our framework can rediscover existing manually designed mechanisms and provide insights into neural-network based solutions through Programming-by-Example. These results highlight the potential of LLMs to not only automate but also enhance the transparency and scalability of mechanism design, ensuring safe deployment of the mechanisms in society.

Title: Predicting Depression in Screening Interviews from Interactive Multi-Theme Collaboration

• Authors: Xianbing Zhao, Yiqing Lyu, Di Wang, Buzhou Tang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12204
• Pdf URL: https://arxiv.org/pdf/2502.12204
• Copy Paste: [[2502.12204]] Predicting Depression in Screening Interviews from Interactive Multi-Theme Collaboration(https://arxiv.org/abs/2502.12204)
• Keywords: in-context
• Abstract: Automatic depression detection provides cues for early clinical intervention by clinicians. Clinical interviews for depression detection involve dialogues centered around multiple themes. Existing studies primarily design end-to-end neural network models to capture the hierarchical structure of clinical interview dialogues. However, these methods exhibit defects in modeling the thematic content of clinical interviews: 1) they fail to capture intra-theme and inter-theme correlation explicitly, and 2) they do not allow clinicians to intervene and focus on themes of interest. To address these issues, this paper introduces an interactive depression detection framework. This framework leverages in-context learning techniques to identify themes in clinical interviews and then models both intra-theme and inter-theme correlation. Additionally, it employs AI-driven feedback to simulate the interests of clinicians, enabling interactive adjustment of theme importance. PDIMC achieves absolute improvements of 35\% and 12\% compared to the state-of-the-art on the depression detection dataset DAIC-WOZ, which demonstrates the effectiveness of modeling theme correlation and incorporating interactive external feedback.

Title: PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN

• Authors: Jiayu Zhang, Zhiyu Zhu, Xinyi Wang, Silin Liao, Zhibo Jin, Flora D. Salim, Huaming Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12207
• Pdf URL: https://arxiv.org/pdf/2502.12207
• Copy Paste: [[2502.12207]] PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN(https://arxiv.org/abs/2502.12207)
• Keywords: generative
• Abstract: Deep neural networks have demonstrated remarkable performance across various domains. However, they are vulnerable to adversarial examples, which can lead to erroneous predictions. Generative Adversarial Networks (GANs) can leverage the generators and discriminators model to quickly produce high-quality adversarial examples. Since both modules train in a competitive and simultaneous manner, GAN-based algorithms like AdvGAN can generate adversarial examples with better transferability compared to traditional methods. However, the generation of perturbations is usually limited to a single iteration, preventing these examples from fully exploiting the potential of the methods. To tackle this issue, we introduce a novel approach named Progressive Auto-Regression AdvGAN (PAR-AdvGAN). It incorporates an auto-regressive iteration mechanism within a progressive generation network to craft adversarial examples with enhanced attack capability. We thoroughly evaluate our PAR-AdvGAN method with a large-scale experiment, demonstrating its superior performance over various state-of-the-art black-box adversarial attacks, as well as the original this http URL, PAR-AdvGAN significantly accelerates the adversarial example generation, i.e., achieving the speeds of up to 335.5 frames per second on Inception-v3 model, outperforming the gradient-based transferable attack algorithms. Our code is available at: this https URL

Title: On Creating a Causally Grounded Usable Rating Method for Assessing the Robustness of Foundation Models Supporting Time Series

• Authors: Kausik Lakkaraju, Rachneet Kaur, Parisa Zehtabi, Sunandita Patra, Siva Likitha Valluru, Zhen Zeng, Biplav Srivastava, Marco Valtorta
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12226
• Pdf URL: https://arxiv.org/pdf/2502.12226
• Copy Paste: [[2502.12226]] On Creating a Causally Grounded Usable Rating Method for Assessing the Robustness of Foundation Models Supporting Time Series(https://arxiv.org/abs/2502.12226)
• Keywords: foundation model
• Abstract: Foundation Models (FMs) have improved time series forecasting in various sectors, such as finance, but their vulnerability to input disturbances can hinder their adoption by stakeholders, such as investors and analysts. To address this, we propose a causally grounded rating framework to study the robustness of Foundational Models for Time Series (FMTS) with respect to input perturbations. We evaluate our approach to the stock price prediction problem, a well-studied problem with easily accessible public data, evaluating six state-of-the-art (some multi-modal) FMTS across six prominent stocks spanning three industries. The ratings proposed by our framework effectively assess the robustness of FMTS and also offer actionable insights for model selection and deployment. Within the scope of our study, we find that (1) multi-modal FMTS exhibit better robustness and accuracy compared to their uni-modal versions and, (2) FMTS pre-trained on time series forecasting task exhibit better robustness and forecasting accuracy compared to general-purpose FMTS pre-trained across diverse settings. Further, to validate our framework's usability, we conduct a user study showcasing FMTS prediction errors along with our computed ratings. The study confirmed that our ratings reduced the difficulty for users in comparing the robustness of different systems.

Title: Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions

• Authors: Sujan Sai Gannamaneni, Rohil Prakash Rao, Michael Mock, Maram Akila, Stefan Wrobel
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12360
• Pdf URL: https://arxiv.org/pdf/2502.12360
• Copy Paste: [[2502.12360]] Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions(https://arxiv.org/abs/2502.12360)
• Keywords: foundation model
• Abstract: Studying systematic weaknesses of DNNs has gained prominence in the last few years with the rising focus on building safe AI systems. Slice discovery methods (SDMs) are prominent algorithmic approaches for finding such systematic weaknesses. They identify top-k semantically coherent slices/subsets of data where a DNN-under-test has low performance. For being directly useful, e.g., as evidences in a safety argumentation, slices should be aligned with human-understandable (safety-relevant) dimensions, which, for example, are defined by safety and domain experts as parts of the operational design domain (ODD). While straightforward for structured data, the lack of semantic metadata makes these investigations challenging for unstructured data. Therefore, we propose a complete workflow which combines contemporary foundation models with algorithms for combinatorial search that consider structured data and DNN errors for finding systematic weaknesses in images. In contrast to existing approaches, ours identifies weak slices that are in line with predefined human-understandable dimensions. As the workflow includes foundation models, its intermediate and final results may not always be exact. Therefore, we build into our workflow an approach to address the impact of noisy metadata. We evaluate our approach w.r.t. its quality on four popular computer vision datasets, including autonomous driving datasets like Cityscapes, BDD100k, and RailSem19, while using multiple state-of-the-art models as DNNs-under-test.

Title: Positional Encoding in Transformer-Based Time Series Models: A Survey

• Authors: Habib Irani, Vangelis Metsis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12370
• Pdf URL: https://arxiv.org/pdf/2502.12370
• Copy Paste: [[2502.12370]] Positional Encoding in Transformer-Based Time Series Models: A Survey(https://arxiv.org/abs/2502.12370)
• Keywords: anomaly
• Abstract: Recent advancements in transformer-based models have greatly improved time series analysis, providing robust solutions for tasks such as forecasting, anomaly detection, and classification. A crucial element of these models is positional encoding, which allows transformers to capture the intrinsic sequential nature of time series data. This survey systematically examines existing techniques for positional encoding in transformer-based time series models. We investigate a variety of methods, including fixed, learnable, relative, and hybrid approaches, and evaluate their effectiveness in different time series classification tasks. Furthermore, we outline key challenges and suggest potential research directions to enhance positional encoding strategies. By delivering a comprehensive overview and quantitative benchmarking, this survey intends to assist researchers and practitioners in selecting and designing effective positional encoding methods for transformer-based time series models.

Title: DiffuRNN: Harnessing Diffusion Processes for Global Interactions

• Authors: Jacob Fein-Ashley
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12381
• Pdf URL: https://arxiv.org/pdf/2502.12381
• Copy Paste: [[2502.12381]] DiffuRNN: Harnessing Diffusion Processes for Global Interactions(https://arxiv.org/abs/2502.12381)
• Keywords: diffusion
• Abstract: Diffusion kernels capture global dependencies. We present DiffuRNN, a novel architecture that reinterprets sequential data processing as a unified diffusion process. Our model integrates adaptive diffusion modules with localized nonlinear updates and a diffusion-inspired attention mechanism. This design enables efficient global information propagation while preserving fine-grained temporal details. DiffuRNN overcomes the limitations of conventional recurrent and transformer models by allowing full parallelization across time steps and supporting robust multi-scale temporal representations. Experiments on benchmark sequence modeling tasks demonstrate that DiffuRNN delivers superior performance and scalability, setting a new standard for global interaction in sequential data.

Title: Achieving Upper Bound Accuracy of Joint Training in Continual Learning

• Authors: Saleh Momeni, Bing Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12388
• Pdf URL: https://arxiv.org/pdf/2502.12388
• Copy Paste: [[2502.12388]] Achieving Upper Bound Accuracy of Joint Training in Continual Learning(https://arxiv.org/abs/2502.12388)
• Keywords: foundation model
• Abstract: Continual learning has been an active research area in machine learning, focusing on incrementally learning a sequence of tasks. A key challenge is catastrophic forgetting (CF), and most research efforts have been directed toward mitigating this issue. However, a significant gap remains between the accuracy achieved by state-of-the-art continual learning algorithms and the ideal or upper-bound accuracy achieved by training all tasks together jointly. This gap has hindered or even prevented the adoption of continual learning in applications, as accuracy is often of paramount importance. Recently, another challenge, termed inter-task class separation (ICS), was also identified, which spurred a theoretical study into principled approaches for solving continual learning. Further research has shown that by leveraging the theory and the power of large foundation models, it is now possible to achieve upper-bound accuracy, which has been empirically validated using both text and image classification datasets. Continual learning is now ready for real-life applications. This paper surveys the main research leading to this achievement, justifies the approach both intuitively and from neuroscience research, and discusses insights gained.

Title: Reward-Safety Balance in Offline Safe RL via Diffusion Regularization

• Authors: Junyu Guo, Zhi Zheng, Donghao Ying, Ming Jin, Shangding Gu, Costas Spanos, Javad Lavaei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12391
• Pdf URL: https://arxiv.org/pdf/2502.12391
• Copy Paste: [[2502.12391]] Reward-Safety Balance in Offline Safe RL via Diffusion Regularization(https://arxiv.org/abs/2502.12391)
• Keywords: diffusion
• Abstract: Constrained reinforcement learning (RL) seeks high-performance policies under safety constraints. We focus on an offline setting where the agent has only a fixed dataset -- common in realistic tasks to prevent unsafe exploration. To address this, we propose Diffusion-Regularized Constrained Offline Reinforcement Learning (DRCORL), which first uses a diffusion model to capture the behavioral policy from offline data and then extracts a simplified policy to enable efficient inference. We further apply gradient manipulation for safety adaptation, balancing the reward objective and constraint satisfaction. This approach leverages high-quality offline data while incorporating safety requirements. Empirical results show that DRCORL achieves reliable safety performance, fast inference, and strong reward outcomes across robot learning tasks. Compared to existing safe offline RL methods, it consistently meets cost limits and performs well with the same hyperparameters, indicating practical applicability in real-world scenarios.

Title: Efficient Neural SDE Training using Wiener-Space Cubature

• Authors: Luke Snow, Vikram Krishnamurthy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12395
• Pdf URL: https://arxiv.org/pdf/2502.12395
• Copy Paste: [[2502.12395]] Efficient Neural SDE Training using Wiener-Space Cubature(https://arxiv.org/abs/2502.12395)
• Keywords: diffusion
• Abstract: A neural stochastic differential equation (SDE) is an SDE with drift and diffusion terms parametrized by neural networks. The training procedure for neural SDEs consists of optimizing the SDE vector field (neural network) parameters to minimize the expected value of an objective functional on infinite-dimensional path-space. Existing training techniques focus on methods to efficiently compute path-wise gradients of the objective functional with respect to these parameters, then pair this with Monte-Carlo simulation to estimate the expectation, and stochastic gradient descent to optimize. In this work we introduce a novel training technique which bypasses and improves upon Monte-Carlo simulation; we extend results in the theory of Wiener-space cubature to approximate the expected objective functional by a weighted sum of deterministic ODE solutions. This allows us to compute gradients by efficient ODE adjoint methods. Furthermore, we exploit a high-order recombination scheme to drastically reduce the number of ODE solutions necessary to achieve a reasonable approximation. We show that this Wiener-space cubature approach can surpass the O(1/sqrt(n)) rate of Monte-Carlo simulation, or the O(log(n)/n) rate of quasi-Monte-Carlo, to achieve a O(1/n) rate under reasonable assumptions.

Title: On the Robust Approximation of ASR Metrics

• Authors: Abdul Waheed, Hanin Atwany, Rita Singh, Bhiksha Raj
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12408
• Pdf URL: https://arxiv.org/pdf/2502.12408
• Copy Paste: [[2502.12408]] On the Robust Approximation of ASR Metrics(https://arxiv.org/abs/2502.12408)
• Keywords: foundation model
• Abstract: Recent advances in speech foundation models are largely driven by scaling both model size and data, enabling them to perform a wide range of tasks, including speech recognition. Traditionally, ASR models are evaluated using metrics like Word Error Rate (WER) and Character Error Rate (CER), which depend on ground truth labels. As a result of limited labeled data from diverse domains and testing conditions, the true generalization capabilities of these models beyond standard benchmarks remain unclear. Moreover, labeling data is both costly and time-consuming. To address this, we propose a novel label-free approach for approximating ASR performance metrics, eliminating the need for ground truth labels. Our method utilizes multimodal embeddings in a unified space for speech and transcription representations, combined with a high-quality proxy model to compute proxy metrics. These features are used to train a regression model to predict key ASR metrics like Word Error Rate (WER) and Character Error Rate (CER). We experiment with over 40 models across 14 datasets representing both standard and in-the-wild testing conditions. Our results show that we approximate the metrics within a single-digit absolute difference across all experimental configurations, outperforming the most recent baseline by more than 50\%.

Title: Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models

• Authors: Hanin Atwany, Abdul Waheed, Rita Singh, Monojit Choudhury, Bhiksha Raj
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12414
• Pdf URL: https://arxiv.org/pdf/2502.12414
• Copy Paste: [[2502.12414]] Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models(https://arxiv.org/abs/2502.12414)
• Keywords: foundation model
• Abstract: Speech foundation models trained at a massive scale, both in terms of model and data size, result in robust systems capable of performing multiple speech tasks, including automatic speech recognition (ASR). These models transcend language and domain barriers, yet effectively measuring their performance remains a challenge. Traditional metrics like word error rate (WER) and character error rate (CER) are commonly used to evaluate ASR performance but often fail to reflect transcription quality in critical contexts, particularly when detecting fabricated outputs. This phenomenon, known as hallucination, is especially concerning in high-stakes domains such as healthcare, legal, and aviation, where errors can have severe consequences. In our work, we address this gap by investigating hallucination in ASR models. We examine how factors such as distribution shifts, model size, and model architecture influence the hallucination error rate (HER), a metric we introduce to quantify hallucinations. Our analysis of 20 ASR models reveals \numinsights~key insights: (1) High WERs can mask low hallucination rates, while low WERs may conceal dangerous hallucinations. (2) Synthetic noise, both adversarial and common perturbations like white noise, pitch shift, and time stretching, increase HER. (3) Distribution shift correlates strongly with HER ($\alpha = 0.91$). Our findings highlight the importance of incorporating HER alongside traditional metrics like WER to better assess ASR model performance, particularly in high-stakes domains.

Title: Multi Image Super Resolution Modeling for Earth System Models

• Authors: Ehsan Zeraatkar, Salah A Faroughi, Jelena Tešić
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12427
• Pdf URL: https://arxiv.org/pdf/2502.12427
• Copy Paste: [[2502.12427]] Multi Image Super Resolution Modeling for Earth System Models(https://arxiv.org/abs/2502.12427)
• Keywords: generative
• Abstract: Super-resolution (SR) techniques are essential for improving Earth System Model (ESM) data's spatial resolution, which helps better understand complex environmental processes. This paper presents a new algorithm, ViFOR, which combines Vision Transformers (ViT) and Implicit Neural Representation Networks (INRs) to generate High-Resolution (HR) images from Low-Resolution (LR) inputs. ViFOR introduces a novel integration of Fourier-based activation functions within the Vision Transformer architecture, enabling it to effectively capture global context and high-frequency details critical for accurate SR reconstruction. The results show that ViFOR outperforms state-of-the-art methods such as ViT, Sinusoidal Representation Networks (SIREN), and SR Generative Adversarial Networks (SRGANs) based on metrics like Peak Signal-to-Noise Ratio (PSNR) and Mean Squared Error (MSE) both for global as well as the local imagery. ViFOR improves PSNR of up to 4.18 dB, 1.56 dB, and 1.73 dB over ViT for full images in the Source Temperature, Shortwave, and Longwave Flux.

Title: Multi-Attribute Steering of Language Models via Targeted Intervention

• Authors: Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12446
• Pdf URL: https://arxiv.org/pdf/2502.12446
• Copy Paste: [[2502.12446]] Multi-Attribute Steering of Language Models via Targeted Intervention(https://arxiv.org/abs/2502.12446)
• Keywords: generative
• Abstract: Inference-time intervention (ITI) has emerged as a promising method for steering large language model (LLM) behavior in a particular direction (e.g., improving helpfulness) by intervening on token representations without costly updates to the LLM's parameters. However, existing ITI approaches fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity. To address this, we introduce Multi-Attribute Targeted Steering (MAT-Steer), a novel steering framework designed for selective token-level intervention across multiple attributes. MAT-Steer learns steering vectors using an alignment objective that shifts the model's internal representations of undesirable outputs closer to those of desirable ones while enforcing sparsity and orthogonality among vectors for different attributes, thereby reducing inter-attribute conflicts. We evaluate MAT-Steer in two distinct settings: (i) on question answering (QA) tasks where we balance attributes like truthfulness, bias, and toxicity; (ii) on generative tasks where we simultaneously improve attributes like helpfulness, correctness, and coherence. MAT-Steer outperforms existing ITI and parameter-efficient finetuning approaches across both task types (e.g., 3% average accuracy gain across QA tasks and 55.82% win rate against the best ITI baseline).

Title: Not-So-Optimal Transport Flows for 3D Point Cloud Generation

• Authors: Ka-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, Arash Vahdat
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12456
• Pdf URL: https://arxiv.org/pdf/2502.12456
• Copy Paste: [[2502.12456]] Not-So-Optimal Transport Flows for 3D Point Cloud Generation(https://arxiv.org/abs/2502.12456)
• Keywords: diffusion, generative
• Abstract: Learning generative models of 3D point clouds is one of the fundamental problems in 3D generative learning. One of the key properties of point clouds is their permutation invariance, i.e., changing the order of points in a point cloud does not change the shape they represent. In this paper, we analyze the recently proposed equivariant OT flows that learn permutation invariant generative models for point-based molecular data and we show that these models scale poorly on large point clouds. Also, we observe learning (equivariant) OT flows is generally challenging since straightening flow trajectories makes the learned flow model complex at the beginning of the trajectory. To remedy these, we propose not-so-optimal transport flow models that obtain an approximate OT by an offline OT precomputation, enabling an efficient construction of OT pairs for training. During training, we can additionally construct a hybrid coupling by combining our approximate OT and independent coupling to make the target flow models easier to learn. In an extensive empirical study, we show that our proposed model outperforms prior diffusion- and flow-based approaches on a wide range of unconditional generation and shape completion on the ShapeNet benchmark.

Title: MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation

• Authors: Yutong Wang, Pengliang Ji, Chaoqun Yang, Kaixin Li, Ming Hu, Jiaoyang Li, Guillaume Sartoretti
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12468
• Pdf URL: https://arxiv.org/pdf/2502.12468
• Copy Paste: [[2502.12468]] MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation(https://arxiv.org/abs/2502.12468)
• Keywords: generative
• Abstract: The LLM-as-a-Judge paradigm shows promise for evaluating generative content but lacks reliability in reasoning-intensive scenarios, such as programming. Inspired by recent advances in reasoning models and shifts in scaling laws, we pioneer bringing test-time computation into LLM-as-a-Judge, proposing MCTS-Judge, a resource-efficient, System-2 thinking framework for code correctness evaluation. MCTS-Judge leverages Monte Carlo Tree Search (MCTS) to decompose problems into simpler, multi-perspective evaluations. Through a node-selection strategy that combines self-assessment based on historical actions in the current trajectory and the Upper Confidence Bound for Trees based on prior rollouts, MCTS-Judge balances global optimization and refinement of the current trajectory. We further designed a high-precision, unit-test-level reward mechanism to encourage the Large Language Model (LLM) to perform line-by-line analysis. Extensive experiments on three benchmarks and five LLMs demonstrate the effectiveness of MCTS-Judge, which improves the base model's accuracy from 41% to 80%, surpassing the o1-series models with 3x fewer tokens. Further evaluations validate the superiority of its reasoning trajectory in logic, analytics, thoroughness, and overall quality, while revealing the test-time scaling law of the LLM-as-a-Judge paradigm.

Title: Predicate Hierarchies Improve Few-Shot State Classification

• Authors: Emily Jin, Joy Hsu, Jiajun Wu
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.12481
• Pdf URL: https://arxiv.org/pdf/2502.12481
• Copy Paste: [[2502.12481]] Predicate Hierarchies Improve Few-Shot State Classification(https://arxiv.org/abs/2502.12481)
• Keywords: self-supervised
• Abstract: State classification of objects and their relations is core to many long-horizon tasks, particularly in robot planning and manipulation. However, the combinatorial explosion of possible object-predicate combinations, coupled with the need to adapt to novel real-world environments, makes it a desideratum for state classification models to generalize to novel queries with few examples. To this end, we propose PHIER, which leverages predicate hierarchies to generalize effectively in few-shot scenarios. PHIER uses an object-centric scene encoder, self-supervised losses that infer semantic relations between predicates, and a hyperbolic distance metric that captures hierarchical structure; it learns a structured latent space of image-predicate pairs that guides reasoning over state classification queries. We evaluate PHIER in the CALVIN and BEHAVIOR robotic environments and show that PHIER significantly outperforms existing methods in few-shot, out-of-distribution state classification, and demonstrates strong zero- and few-shot generalization from simulated to real-world tasks. Our results demonstrate that leveraging predicate hierarchies improves performance on state classification tasks with limited data.

Title: Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models

• Authors: Die Chen, Zhiwen Li, Cen Chen, Xiaodan Li, Jinyan Ye
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12527
• Pdf URL: https://arxiv.org/pdf/2502.12527
• Copy Paste: [[2502.12527]] Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.12527)
• Keywords: diffusion
• Abstract: Text-to-image (T2I) diffusion models have gained widespread application across various domains, demonstrating remarkable creative potential. However, the strong generalization capabilities of these models can inadvertently led they to generate NSFW content even with efforts on filtering NSFW content from the training dataset, posing risks to their safe deployment. While several concept erasure methods have been proposed to mitigate this issue, a comprehensive evaluation of their effectiveness remains absent. To bridge this gap, we present the first systematic investigation of concept erasure methods for NSFW content and its sub-themes in text-to-image diffusion models. At the task level, we provide a holistic evaluation of 11 state-of-the-art baseline methods with 14 variants. Specifically, we analyze these methods from six distinct assessment perspectives, including three conventional perspectives, i.e., erasure proportion, image quality, and semantic alignment, and three new perspectives, i.e., excessive erasure, the impact of explicit and implicit unsafe prompts, and robustness. At the tool level, we perform a detailed toxicity analysis of NSFW datasets and compare the performance of different NSFW classifiers, offering deeper insights into their performance alongside a compilation of comprehensive evaluation metrics. Our benchmark not only systematically evaluates concept erasure methods, but also delves into the underlying factors influencing their performance at the insight level. By synthesizing insights from various evaluation perspectives, we provide a deeper understanding of the challenges and opportunities in the field, offering actionable guidance and inspiration for advancing research and practical applications in concept erasure.

Title: Policy-to-Language: Train LLMs to Explain Decisions with Flow-Matching Generated Rewards

• Authors: Xinyi Yang, Liang Zeng, Heng Dong, Chao Yu, Xiaoran Wu, Huazhong Yang, Yu Wang, Milind Tambe, Tonghan Wang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12530
• Pdf URL: https://arxiv.org/pdf/2502.12530
• Copy Paste: [[2502.12530]] Policy-to-Language: Train LLMs to Explain Decisions with Flow-Matching Generated Rewards(https://arxiv.org/abs/2502.12530)
• Keywords: generative
• Abstract: As humans increasingly share environments with diverse agents powered by RL, LLMs, and beyond, the ability to explain their policies in natural language will be vital for reliable coexistence. In this paper, we build a model-agnostic explanation generator based on an LLM. The technical novelty is that the rewards for training this LLM are generated by a generative flow matching model. This model has a specially designed structure with a hidden layer merged with an LLM to harness the linguistic cues of explanations into generating appropriate rewards. Experiments on both RL and LLM tasks demonstrate that our method can generate dense and effective rewards while saving on expensive human feedback; it thus enables effective explanations and even improves the accuracy of the decisions in original tasks.

Title: DeltaDiff: A Residual-Guided Diffusion Model for Enhanced Image Super-Resolution

• Authors: Chao Yang, Yong Fan, Cheng Lu, Zhijing Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12567
• Pdf URL: https://arxiv.org/pdf/2502.12567
• Copy Paste: [[2502.12567]] DeltaDiff: A Residual-Guided Diffusion Model for Enhanced Image Super-Resolution(https://arxiv.org/abs/2502.12567)
• Keywords: diffusion
• Abstract: Recently, the application of diffusion models in super-resolution tasks has become a popular research direction. Existing work is focused on fully migrating diffusion models to SR tasks. The diffusion model is proposed in the field of image generation, so in order to make the generated results diverse, the diffusion model combines random Gaussian noise and distributed sampling to increase the randomness of the model. However, the essence of super-resolution tasks requires the model to generate high-resolution images with fidelity. Excessive addition of random factors can result in the model generating detailed information that does not belong to the HR image. To address this issue, we propose a new diffusion model called Deltadiff, which uses only residuals between images for diffusion, making the entire diffusion process more stable. The experimental results show that our method surpasses state-of-the-art models and generates results with better fidelity. Our code and model are publicly available at this https URL

Title: CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation

• Authors: Minghao Fu, Guo-Hua Wang, Liangfu Cao, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12579
• Pdf URL: https://arxiv.org/pdf/2502.12579
• Copy Paste: [[2502.12579]] CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation(https://arxiv.org/abs/2502.12579)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as a dominant approach for text-to-image generation. Key components such as the human preference alignment and classifier-free guidance play a crucial role in ensuring generation quality. However, their independent application in current text-to-image models continues to face significant challenges in achieving strong text-image alignment, high generation quality, and consistency with human aesthetic standards. In this work, we for the first time, explore facilitating the collaboration of human performance alignment and test-time sampling to unlock the potential of text-to-image models. Consequently, we introduce CHATS (Combining Human-Aligned optimization and Test-time Sampling), a novel generative framework that separately models the preferred and dispreferred distributions and employs a proxy-prompt-based sampling strategy to utilize the useful information contained in both distributions. We observe that CHATS exhibits exceptional data efficiency, achieving strong performance with only a small, high-quality funetuning dataset. Extensive experiments demonstrate that CHATS surpasses traditional preference alignment methods, setting new state-of-the-art across various standard benchmarks.

Title: Enhancing Semi-supervised Learning with Noisy Zero-shot Pseudolabels

• Authors: Jichan Chung, Irene Y. Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12584
• Pdf URL: https://arxiv.org/pdf/2502.12584
• Copy Paste: [[2502.12584]] Enhancing Semi-supervised Learning with Noisy Zero-shot Pseudolabels(https://arxiv.org/abs/2502.12584)
• Keywords: foundation model
• Abstract: Semi-supervised learning (SSL) leverages limited labeled data alongside abundant unlabeled data to address labeling costs in machine learning. While recent foundation models enable zero-shot inference, attempts to integrate these capabilities into SSL through pseudo-labeling have shown mixed results due to unreliable zero-shot predictions. We present ZMT (Zero-Shot Multi-Task Learning), a framework that jointly optimizes zero-shot pseudo-labels and unsupervised representation learning objectives from contemporary SSL approaches. Our method introduces a multi-task learning-based mechanism that incorporates pseudo-labels while ensuring robustness to varying pseudo-label quality. Experiments across 8 datasets in vision, language, and audio domains demonstrate that ZMT reduces error by up to 56% compared to traditional SSL methods, with particularly compelling results when pseudo-labels are noisy and unreliable. ZMT represents a significant step toward making semi-supervised learning more effective and accessible in resource-constrained environments.

Title: Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining

• Authors: Jinfan Hu, Zhiyuan You, Jinjin Gu, Kaiwen Zhu, Tianfan Xue, Chao Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12600
• Pdf URL: https://arxiv.org/pdf/2502.12600
• Copy Paste: [[2502.12600]] Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining(https://arxiv.org/abs/2502.12600)
• Keywords: generative
• Abstract: Generalization remains a significant challenge for low-level vision models, which often struggle with unseen degradations in real-world scenarios despite their success in controlled benchmarks. In this paper, we revisit the generalization problem in low-level vision models. Image deraining is selected as a case study due to its well-defined and easily decoupled structure, allowing for more effective observation and analysis. Through comprehensive experiments, we reveal that the generalization issue is not primarily due to limited network capacity but rather the failure of existing training strategies, which leads networks to overfit specific degradation patterns. Our findings show that guiding networks to focus on learning the underlying image content, rather than the degradation patterns, is key to improving generalization. We demonstrate that balancing the complexity of background images and degradations in the training data helps networks better fit the image distribution. Furthermore, incorporating content priors from pre-trained generative models significantly enhances generalization. Experiments on both image deraining and image denoising validate the proposed strategies. We believe the insights and solutions will inspire further research and improve the generalization of low-level vision models.

Title: S2C: Learning Noise-Resistant Differences for Unsupervised Change Detection in Multimodal Remote Sensing Images

• Authors: Lei Ding, Xibing Zuo, Danfeng Hong, Haitao Guo, Jun Lu, Zhihui Gong, Lorenzo Bruzzone
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12604
• Pdf URL: https://arxiv.org/pdf/2502.12604
• Copy Paste: [[2502.12604]] S2C: Learning Noise-Resistant Differences for Unsupervised Change Detection in Multimodal Remote Sensing Images(https://arxiv.org/abs/2502.12604)
• Keywords: foundation model
• Abstract: Unsupervised Change Detection (UCD) in multimodal Remote Sensing (RS) images remains a difficult challenge due to the inherent spatio-temporal complexity within data, and the heterogeneity arising from different imaging sensors. Inspired by recent advancements in Visual Foundation Models (VFMs) and Contrastive Learning (CL) methodologies, this research aims to develop CL methodologies to translate implicit knowledge in VFM into change representations, thus eliminating the need for explicit supervision. To this end, we introduce a Semantic-to-Change (S2C) learning framework for UCD in both homogeneous and multimodal RS images. Differently from existing CL methodologies that typically focus on learning multi-temporal similarities, we introduce a novel triplet learning strategy that explicitly models temporal differences, which are crucial to the CD task. Furthermore, random spatial and spectral perturbations are introduced during the training to enhance robustness to temporal noise. In addition, a grid sparsity regularization is defined to suppress insignificant changes, and an IoU-matching algorithm is developed to refine the CD results. Experiments on four benchmark CD datasets demonstrate that the proposed S2C learning framework achieves significant improvements in accuracy, surpassing current state-of-the-art by over 31\%, 9\%, 23\%, and 15\%, respectively. It also demonstrates robustness and sample efficiency, suitable for training and adaptation of various Visual Foundation Models (VFMs) or backbone neural networks. The relevant code will be available at: this http URL.

Title: Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions

• Authors: Leonardo Ranaldi, Marco Valentino, Alexander Polonsky, Andrè Freitas
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12616
• Pdf URL: https://arxiv.org/pdf/2502.12616
• Copy Paste: [[2502.12616]] Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions(https://arxiv.org/abs/2502.12616)
• Keywords: in-context
• Abstract: Chain-of-Though (CoT) represents a common strategy for reasoning in Large Language Models (LLMs) by decomposing complex tasks into intermediate inference steps. However, explanations generated via CoT are susceptible to content biases that negatively affect their robustness and faithfulness. To mitigate existing limitations, recent work has proposed using logical formalisms coupled with external symbolic solvers. However, fully symbolic approaches possess the bottleneck of requiring a complete translation from natural language to formal languages, a process that affects efficiency and flexibility. To achieve a trade-off, this paper investigates methods to disentangle content from logical reasoning without a complete formalisation. In particular, we present QuaSAR (for Quasi-Symbolic Abstract Reasoning), a variation of CoT that guides LLMs to operate at a higher level of abstraction via quasi-symbolic explanations. Our framework leverages the capability of LLMs to formalise only relevant variables and predicates, enabling the coexistence of symbolic elements with natural language. We show the impact of QuaSAR for in-context learning and for constructing demonstrations to improve the reasoning capabilities of smaller models. Our experiments show that quasi-symbolic abstractions can improve CoT-based methods by up to 8% accuracy, enhancing robustness and consistency on challenging adversarial variations on both natural language (i.e. MMLU-Redux) and symbolic reasoning tasks (i.e., GSM-Symbolic).

Title: Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport

• Authors: Mingyang Sun, Pengxiang Ding, Weinan Zhang, Donglin Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.12631
• Pdf URL: https://arxiv.org/pdf/2502.12631
• Copy Paste: [[2502.12631]] Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport(https://arxiv.org/abs/2502.12631)
• Keywords: diffusion
• Abstract: Diffusion policies have shown promise in learning complex behaviors from demonstrations, particularly for tasks requiring precise control and long-term planning. However, they face challenges in robustness when encountering distribution shifts. This paper explores improving diffusion-based imitation learning models through online interactions with the environment. We propose OTPR (Optimal Transport-guided score-based diffusion Policy for Reinforcement learning fine-tuning), a novel method that integrates diffusion policies with RL using optimal transport theory. OTPR leverages the Q-function as a transport cost and views the policy as an optimal transport map, enabling efficient and stable fine-tuning. Moreover, we introduce masked optimal transport to guide state-action matching using expert keypoints and a compatibility-based resampling strategy to enhance training stability. Experiments on three simulation tasks demonstrate OTPR's superior performance and robustness compared to existing methods, especially in complex and sparse-reward environments. In sum, OTPR provides an effective framework for combining IL and RL, achieving versatile and reliable policy learning. The code will be released at this https URL.

Title: MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation

• Authors: Sihyun Yu, Meera Hahn, Dan Kondratyuk, Jinwoo Shin, Agrim Gupta, José Lezama, Irfan Essa, David Ross, Jonathan Huang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12632
• Pdf URL: https://arxiv.org/pdf/2502.12632
• Copy Paste: [[2502.12632]] MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation(https://arxiv.org/abs/2502.12632)
• Keywords: diffusion
• Abstract: Diffusion models are successful for synthesizing high-quality videos but are limited to generating short clips (e.g., 2-10 seconds). Synthesizing sustained footage (e.g. over minutes) still remains an open research question. In this paper, we propose MALT Diffusion (using Memory-Augmented Latent Transformers), a new diffusion model specialized for long video generation. MALT Diffusion (or just MALT) handles long videos by subdividing them into short segments and doing segment-level autoregressive generation. To achieve this, we first propose recurrent attention layers that encode multiple segments into a compact memory latent vector; by maintaining this memory vector over time, MALT is able to condition on it and continuously generate new footage based on a long temporal context. We also present several training techniques that enable the model to generate frames over a long horizon with consistent quality and minimal degradation. We validate the effectiveness of MALT through experiments on long video benchmarks. We first perform extensive analysis of MALT in long-contextual understanding capability and stability using popular long video benchmarks. For example, MALT achieves an FVD score of 220.4 on 128-frame video generation on UCF-101, outperforming the previous state-of-the-art of 648.4. Finally, we explore MALT's capabilities in a text-to-video generation setting and show that it can produce long videos compared with recent techniques for long text-to-video generation.

Title: Spherical Dense Text-to-Image Synthesis

• Authors: Timon Winter, Stanislav Frolov, Brian Bernhard Moser, Andreas Dengel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12691
• Pdf URL: https://arxiv.org/pdf/2502.12691
• Copy Paste: [[2502.12691]] Spherical Dense Text-to-Image Synthesis(https://arxiv.org/abs/2502.12691)
• Keywords: diffusion
• Abstract: Recent advancements in text-to-image (T2I) have improved synthesis results, but challenges remain in layout control and generating omnidirectional panoramic images. Dense T2I (DT2I) and spherical T2I (ST2I) models address these issues, but so far no unified approach exists. Trivial approaches, like prompting a DT2I model to generate panoramas can not generate proper spherical distortions and seamless transitions at the borders. Our work shows that spherical dense text-to-image (SDT2I) can be achieved by integrating training-free DT2I approaches into finetuned panorama models. Specifically, we propose MultiStitchDiffusion (MSTD) and MultiPanFusion (MPF) by integrating MultiDiffusion into StitchDiffusion and PanFusion, respectively. Since no benchmark for SDT2I exists, we further construct Dense-Synthetic-View (DSynView), a new synthetic dataset containing spherical layouts to evaluate our models. Our results show that MSTD outperforms MPF across image quality as well as prompt- and layout adherence. MultiPanFusion generates more diverse images but struggles to synthesize flawless foreground objects. We propose bootstrap-coupling and turning off equirectangular perspective-projection attention in the foreground as an improvement of MPF.

Title: 3D Shape-to-Image Brownian Bridge Diffusion for Brain MRI Synthesis from Cortical Surfaces

• Authors: Fabian Bongratz, Yitong Li, Sama Elbaroudy, Christian Wachinger
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12742
• Pdf URL: https://arxiv.org/pdf/2502.12742
• Copy Paste: [[2502.12742]] 3D Shape-to-Image Brownian Bridge Diffusion for Brain MRI Synthesis from Cortical Surfaces(https://arxiv.org/abs/2502.12742)
• Keywords: diffusion
• Abstract: Despite recent advances in medical image generation, existing methods struggle to produce anatomically plausible 3D structures. In synthetic brain magnetic resonance images (MRIs), characteristic fissures are often missing, and reconstructed cortical surfaces appear scattered rather than densely convoluted. To address this issue, we introduce Cor2Vox, the first diffusion model-based method that translates continuous cortical shape priors to synthetic brain MRIs. To achieve this, we leverage a Brownian bridge process which allows for direct structured mapping between shape contours and medical images. Specifically, we adapt the concept of the Brownian bridge diffusion model to 3D and extend it to embrace various complementary shape representations. Our experiments demonstrate significant improvements in the geometric accuracy of reconstructed structures compared to previous voxel-based approaches. Moreover, Cor2Vox excels in image quality and diversity, yielding high variation in non-target structures like the skull. Finally, we highlight the capability of our approach to simulate cortical atrophy at the sub-voxel level. Our code is available at this https URL.

Title: Architect of the Bits World: Masked Autoregressive Modeling for Circuit Generation Guided by Truth Table

• Authors: Haoyuan Wu, Haisheng Zheng, Shoubo Hu, Zhuolun He, Bei Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12751
• Pdf URL: https://arxiv.org/pdf/2502.12751
• Copy Paste: [[2502.12751]] Architect of the Bits World: Masked Autoregressive Modeling for Circuit Generation Guided by Truth Table(https://arxiv.org/abs/2502.12751)
• Keywords: generative
• Abstract: Logic synthesis, a critical stage in electronic design automation (EDA), optimizes gate-level circuits to minimize power consumption and area occupancy in integrated circuits (ICs). Traditional logic synthesis tools rely on human-designed heuristics, often yielding suboptimal results. Although differentiable architecture search (DAS) has shown promise in generating circuits from truth tables, it faces challenges such as high computational complexity, convergence to local optima, and extensive hyperparameter tuning. Consequently, we propose a novel approach integrating conditional generative models with DAS for circuit generation. Our approach first introduces CircuitVQ, a circuit tokenizer trained based on our Circuit AutoEncoder We then develop CircuitAR, a masked autoregressive model leveraging CircuitVQ as the tokenizer. CircuitAR can generate preliminary circuit structures from truth tables, which guide DAS in producing functionally equivalent circuits. Notably, we observe the scalability and emergent capability in generating complex circuit structures of our CircuitAR models. Extensive experiments also show the superior performance of our method. This research bridges the gap between probabilistic generative models and precise circuit generation, offering a robust solution for logic synthesis.

Title: High-Fidelity Novel View Synthesis via Splatting-Guided Diffusion

• Authors: Xiang Zhang, Yang Zhang, Lukas Mehl, Markus Gross, Christopher Schroers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12752
• Pdf URL: https://arxiv.org/pdf/2502.12752
• Copy Paste: [[2502.12752]] High-Fidelity Novel View Synthesis via Splatting-Guided Diffusion(https://arxiv.org/abs/2502.12752)
• Keywords: diffusion
• Abstract: Despite recent advances in Novel View Synthesis (NVS), generating high-fidelity views from single or sparse observations remains a significant challenge. Existing splatting-based approaches often produce distorted geometry due to splatting errors. While diffusion-based methods leverage rich 3D priors to achieve improved geometry, they often suffer from texture hallucination. In this paper, we introduce SplatDiff, a pixel-splatting-guided video diffusion model designed to synthesize high-fidelity novel views from a single image. Specifically, we propose an aligned synthesis strategy for precise control of target viewpoints and geometry-consistent view synthesis. To mitigate texture hallucination, we design a texture bridge module that enables high-fidelity texture generation through adaptive feature fusion. In this manner, SplatDiff leverages the strengths of splatting and diffusion to generate novel views with consistent geometry and high-fidelity details. Extensive experiments verify the state-of-the-art performance of SplatDiff in single-view NVS. Additionally, without extra training, SplatDiff shows remarkable zero-shot performance across diverse tasks, including sparse-view NVS and stereo video conversion.

Title: One-bit Compressed Sensing using Generative Models

• Authors: Swatantra Kafle, Geethu Joseph, Pramod K. Varshney
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2502.12762
• Pdf URL: https://arxiv.org/pdf/2502.12762
• Copy Paste: [[2502.12762]] One-bit Compressed Sensing using Generative Models(https://arxiv.org/abs/2502.12762)
• Keywords: generative
• Abstract: This paper addresses the classical problem of one-bit compressed sensing using a deep learning-based reconstruction algorithm that leverages a trained generative model to enhance the signal reconstruction performance. The generator, a pre-trained neural network, learns to map from a low-dimensional latent space to a higher-dimensional set of sparse vectors. This generator is then used to reconstruct sparse vectors from their one-bit measurements by searching over its range. The presented algorithm provides an excellent reconstruction performance because the generative model can learn additional structural information about the signal beyond sparsity. Furthermore, we provide theoretical guarantees on the reconstruction accuracy and sample complexity of the algorithm. Through numerical experiments using three publicly available image datasets, MNIST, Fashion-MNIST, and Omniglot, we demonstrate the superior performance of the algorithm compared to other existing algorithms and show that our algorithm can recover both the amplitude and the direction of the signal from one-bit measurements.

Title: Mind the Gap: Aligning the Brain with Language Models Requires a Nonlinear and Multimodal Approach

• Authors: Danny Dongyeop Han, Yunju Cho, Jiook Cha, Jay-Yoon Lee
• Subjects: cs.CL, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2502.12771
• Pdf URL: https://arxiv.org/pdf/2502.12771
• Copy Paste: [[2502.12771]] Mind the Gap: Aligning the Brain with Language Models Requires a Nonlinear and Multimodal Approach(https://arxiv.org/abs/2502.12771)
• Keywords: self-supervised
• Abstract: Self-supervised language and audio models effectively predict brain responses to speech. However, traditional prediction models rely on linear mappings from unimodal features, despite the complex integration of auditory signals with linguistic and semantic information across widespread brain networks during speech comprehension. Here, we introduce a nonlinear, multimodal prediction model that combines audio and linguistic features from pre-trained models (e.g., LLAMA, Whisper). Our approach achieves a 17.2% and 17.9% improvement in prediction performance (unnormalized and normalized correlation) over traditional unimodal linear models, as well as a 7.7% and 14.4% improvement, respectively, over prior state-of-the-art models. These improvements represent a major step towards future robust in-silico testing and improved decoding performance. They also reveal how auditory and semantic information are fused in motor, somatosensory, and higher-level semantic regions, aligning with existing neurolinguistic theories. Overall, our work highlights the often neglected potential of nonlinear and multimodal approaches to brain modeling, paving the way for future studies to embrace these strategies in naturalistic neurolinguistics research.

Title: Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models

• Authors: Daiki Chijiwa, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, Susumu Takeuchi
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.12776
• Pdf URL: https://arxiv.org/pdf/2502.12776
• Copy Paste: [[2502.12776]] Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models(https://arxiv.org/abs/2502.12776)
• Keywords: foundation model
• Abstract: While foundation models have been exploited for various expert tasks through fine-tuning, any foundation model will become outdated due to its old knowledge or limited capability. Thus the underlying foundation model should be eventually replaced by new ones, which leads to repeated cost of fine-tuning these new models. Existing work addresses this problem by inference-time tuning, i.e., modifying the output probabilities from the new foundation model with the outputs from the old foundation model and its fine-tuned model, which involves an additional overhead in inference by the latter two models. In this paper, we propose a new fine-tuning principle, Portable Reward Tuning (PRT), that reduces the inference overhead by its nature, based on the reformulation of fine-tuning as the reward maximization. Specifically, instead of fine-tuning parameters of the foundation models, PRT trains the reward model explicitly through the same loss function as in fine-tuning. During inference, the reward model can be used with any foundation model (with the same set of vocabularies or labels) through the formulation of reward maximization. Experimental results, covering both vision and language models, demonstrate that the PRT-trained model can achieve comparable accuracy to the existing work of inference-time tuning, with less inference cost.

Title: RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models

• Authors: Tanqiu Jiang, Changjiang Li, Fenglong Ma, Ting Wang
• Subjects: cs.CR, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12794
• Pdf URL: https://arxiv.org/pdf/2502.12794
• Copy Paste: [[2502.12794]] RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models(https://arxiv.org/abs/2502.12794)
• Keywords: diffusion, generative
• Abstract: Differentially private diffusion models (DPDMs) harness the remarkable generative capabilities of diffusion models while enforcing differential privacy (DP) for sensitive data. However, existing DPDM training approaches often suffer from significant utility loss, large memory footprint, and expensive inference cost, impeding their practical uses. To overcome such limitations, we present RAPID: Retrieval Augmented PrIvate Diffusion model, a novel approach that integrates retrieval augmented generation (RAG) into DPDM training. Specifically, RAPID leverages available public data to build a knowledge base of sample trajectories; when training the diffusion model on private data, RAPID computes the early sampling steps as queries, retrieves similar trajectories from the knowledge base as surrogates, and focuses on training the later sampling steps in a differentially private manner. Extensive evaluation using benchmark datasets and models demonstrates that, with the same privacy guarantee, RAPID significantly outperforms state-of-the-art approaches by large margins in generative quality, memory footprint, and inference cost, suggesting that retrieval-augmented DP training represents a promising direction for developing future privacy-preserving generative models. The code is available at: this https URL

Title: MOLLM: Multi-Objective Large Language Model for Molecular Design -- Optimizing with Experts

• Authors: Nian Ran, Yue Wang, Richard Allmendinger
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12845
• Pdf URL: https://arxiv.org/pdf/2502.12845
• Copy Paste: [[2502.12845]] MOLLM: Multi-Objective Large Language Model for Molecular Design -- Optimizing with Experts(https://arxiv.org/abs/2502.12845)
• Keywords: in-context
• Abstract: Molecular design plays a critical role in advancing fields such as drug discovery, materials science, and chemical engineering. This work introduces the Multi-Objective Large Language Model for Molecular Design (MOLLM), a novel framework that combines domain-specific knowledge with the adaptability of Large Language Models to optimize molecular properties across multiple objectives. Leveraging in-context learning and multi-objective optimization, MOLLM achieves superior efficiency, innovation, and performance, significantly surpassing state-of-the-art (SOTA) methods. Recognizing the substantial impact of initial populations on evolutionary algorithms, we categorize them into three types: best initial, worst initial, and random initial, to ensure the initial molecules are the same for each method across experiments. Our results demonstrate that MOLLM consistently outperforms SOTA models in all of our experiments. We also provide extensive ablation studies to evaluate the superiority of our components.

Title: Multilingual European Language Models: Benchmarking Approaches and Challenges

• Authors: Fabio Barth, Georg Rehm
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12895
• Pdf URL: https://arxiv.org/pdf/2502.12895
• Copy Paste: [[2502.12895]] Multilingual European Language Models: Benchmarking Approaches and Challenges(https://arxiv.org/abs/2502.12895)
• Keywords: generative
• Abstract: The breakthrough of generative large language models (LLMs) that can solve different tasks through chat interaction has led to a significant increase in the use of general benchmarks to assess the quality or performance of these models beyond individual applications. There is also a need for better methods to evaluate and also to compare models due to the ever increasing number of new models published. However, most of the established benchmarks revolve around the English language. This paper analyses the benefits and limitations of current evaluation datasets, focusing on multilingual European benchmarks. We analyse seven multilingual benchmarks and identify four major challenges. Furthermore, we discuss potential solutions to enhance translation quality and mitigate cultural biases, including human-in-the-loop verification and iterative translation ranking. Our analysis highlights the need for culturally aware and rigorously validated benchmarks to assess the reasoning and question-answering capabilities of multilingual LLMs accurately.

Title: Probabilistic neural operators for functional uncertainty quantification

• Authors: Christopher Bülte, Philipp Scholl, Gitta Kutyniok
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12902
• Pdf URL: https://arxiv.org/pdf/2502.12902
• Copy Paste: [[2502.12902]] Probabilistic neural operators for functional uncertainty quantification(https://arxiv.org/abs/2502.12902)
• Keywords: generative
• Abstract: Neural operators aim to approximate the solution operator of a system of differential equations purely from data. They have shown immense success in modeling complex dynamical systems across various domains. However, the occurrence of uncertainties inherent in both model and data has so far rarely been taken into account\textemdash{}a critical limitation in complex, chaotic systems such as weather forecasting. In this paper, we introduce the probabilistic neural operator (PNO), a framework for learning probability distributions over the output function space of neural operators. PNO extends neural operators with generative modeling based on strictly proper scoring rules, integrating uncertainty information directly into the training process. We provide a theoretical justification for the approach and demonstrate improved performance in quantifying uncertainty across different domains and with respect to different baselines. Furthermore, PNO requires minimal adjustment to existing architectures, shows improved performance for most probabilistic prediction tasks, and leads to well-calibrated predictive distributions and adequate uncertainty representations even for long dynamical trajectories. Implementing our approach into large-scale models for physical applications can lead to improvements in corresponding uncertainty quantification and extreme event identification, ultimately leading to a deeper understanding of the prediction of such surrogate models.

Title: Lightweight Online Adaption for Time Series Foundation Model Forecasts

• Authors: Thomas L. Lee, William Toner, Rajkarn Singh, Artjom Joosem, Martin Asenov
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.12920
• Pdf URL: https://arxiv.org/pdf/2502.12920
• Copy Paste: [[2502.12920]] Lightweight Online Adaption for Time Series Foundation Model Forecasts(https://arxiv.org/abs/2502.12920)
• Keywords: foundation model
• Abstract: Foundation models (FMs) have emerged as a promising approach for time series forecasting. While effective, FMs typically remain fixed during deployment due to the high computational costs of learning them online. Consequently, deployed FMs fail to adapt their forecasts to current data characteristics, despite the availability of online feedback from newly arriving data. This raises the question of whether FM performance can be enhanced by the efficient usage of this feedback. We propose AdapTS to answer this question. AdapTS is a lightweight mechanism for the online adaption of FM forecasts in response to online feedback. AdapTS consists of two parts: a) the AdapTS-Forecaster which is used to learn the current data distribution; and b) the AdapTS-Weighter which is used to combine the forecasts of the FM and the AdapTS-Forecaster. We evaluate the performance of AdapTS in conjunction with several recent FMs across a suite of standard time series datasets. In all of our experiments we find that using AdapTS improves performance. This work demonstrates how efficient usage of online feedback can be used to improve FM forecasts.

Title: On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation

• Authors: Rune Birkmose, Nathan Mørkeberg Reece, Esben Hofstedt Norvin, Johannes Bjerva, Mike Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12923
• Pdf URL: https://arxiv.org/pdf/2502.12923
• Copy Paste: [[2502.12923]] On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation(https://arxiv.org/abs/2502.12923)
• Keywords: generative
• Abstract: This paper investigates whether Large Language Models (LLMs), fine-tuned on synthetic but domain-representative data, can perform the twofold task of (i) slot and intent detection and (ii) natural language response generation for a smart home assistant, while running solely on resource-limited, CPU-only edge hardware. We fine-tune LLMs to produce both JSON action calls and text responses. Our experiments show that 16-bit and 8-bit quantized variants preserve high accuracy on slot and intent detection and maintain strong semantic coherence in generated text, while the 4-bit model, while retaining generative fluency, suffers a noticeable drop in device-service classification accuracy. Further evaluations on noisy human (non-synthetic) prompts and out-of-domain intents confirm the models' generalization ability, obtaining around 80--86\% accuracy. While the average inference time is 5--6 seconds per query -- acceptable for one-shot commands but suboptimal for multi-turn dialogue -- our results affirm that an on-device LLM can effectively unify command interpretation and flexible response generation for home automation without relying on specialized hardware.

Title: Performance of Zero-Shot Time Series Foundation Models on Cloud Data

• Authors: William Toner, Thomas L. Lee, Artjom Joosen, Rajkarn Singh, Martin Asenov
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12944
• Pdf URL: https://arxiv.org/pdf/2502.12944
• Copy Paste: [[2502.12944]] Performance of Zero-Shot Time Series Foundation Models on Cloud Data(https://arxiv.org/abs/2502.12944)
• Keywords: foundation model
• Abstract: Time series foundation models (FMs) have emerged as a popular paradigm for zero-shot multi-domain forecasting. FMs are trained on numerous diverse datasets and claim to be effective forecasters across multiple different time series domains, including cloud data. In this work we investigate this claim, exploring the effectiveness of FMs on cloud data. We demonstrate that many well-known FMs fail to generate meaningful or accurate zero-shot forecasts in this setting. We support this claim empirically, showing that FMs are outperformed consistently by simple linear baselines. We also illustrate a number of interesting pathologies, including instances where FMs suddenly output seemingly erratic, random-looking forecasts. Our results suggest a widespread failure of FMs to model cloud data.

Title: Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compression

• Authors: Jaemoon Lee, Xiao Li, Liangji Zhu, Sanjay Ranka, Anand Rangarajan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12951
• Pdf URL: https://arxiv.org/pdf/2502.12951
• Copy Paste: [[2502.12951]] Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compression(https://arxiv.org/abs/2502.12951)
• Keywords: diffusion, generative
• Abstract: This paper proposes a new compression paradigm -- Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) -- for lossy scientific data compression. The framework is based on recent conditional diffusion (CD) generative models, and it consists of a conditional diffusion model, tensor correction, and error guarantee. Our diffusion model is a mixture of 3D conditioning and 2D denoising U-Net. The approach leverages a 3D block-based compressing module to address spatiotemporal correlations in structured scientific data. Then, the reverse diffusion process for 2D spatial data is conditioned on the ``slices'' of content latent variables produced by the compressing module. After training, the denoising decoder reconstructs the data with zero noise and content latent variables, and thus it is entirely deterministic. The reconstructed outputs of the CD model are further post-processed by our tensor correction and error guarantee steps to control and ensure a maximum error distortion, which is an inevitable requirement in lossy scientific data compression. Our experiments involving two datasets generated by climate and chemical combustion simulations show that our framework outperforms standard convolutional autoencoders and yields competitive compression quality with an existing scientific data compression algorithm.

Title: Does Training with Synthetic Data Truly Protect Privacy?

• Authors: Yunpeng Zhao, Jie Zhang
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12976
• Pdf URL: https://arxiv.org/pdf/2502.12976
• Copy Paste: [[2502.12976]] Does Training with Synthetic Data Truly Protect Privacy?(https://arxiv.org/abs/2502.12976)
• Keywords: diffusion
• Abstract: As synthetic data becomes increasingly popular in machine learning tasks, numerous methods--without formal differential privacy guarantees--use synthetic data for training. These methods often claim, either explicitly or implicitly, to protect the privacy of the original training data. In this work, we explore four different training paradigms: coreset selection, dataset distillation, data-free knowledge distillation, and synthetic data generated from diffusion models. While all these methods utilize synthetic data for training, they lead to vastly different conclusions regarding privacy preservation. We caution that empirical approaches to preserving data privacy require careful and rigorous evaluation; otherwise, they risk providing a false sense of privacy.

Title: Electron flow matching for generative reaction mechanism prediction obeying conservation laws

• Authors: Joonyoung F. Joung, Mun Hong Fong, Nicholas Casetti, Jordan P. Liles, Ne S. Dassanayake, Connor W. Coley
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.12979
• Pdf URL: https://arxiv.org/pdf/2502.12979
• Copy Paste: [[2502.12979]] Electron flow matching for generative reaction mechanism prediction obeying conservation laws(https://arxiv.org/abs/2502.12979)
• Keywords: generative
• Abstract: Central to our understanding of chemical reactivity is the principle of mass conservation, which is fundamental for ensuring physical consistency, balancing equations, and guiding reaction design. However, data-driven computational models for tasks such as reaction product prediction rarely abide by this most basic constraint. In this work, we recast the problem of reaction prediction as a problem of electron redistribution using the modern deep generative framework of flow matching. Our model, FlowER, overcomes limitations inherent in previous approaches by enforcing exact mass conservation, thereby resolving hallucinatory failure modes, recovering mechanistic reaction sequences for unseen substrate scaffolds, and generalizing effectively to out-of-domain reaction classes with extremely data-efficient fine-tuning. FlowER additionally enables estimation of thermodynamic or kinetic feasibility and manifests a degree of chemical intuition in reaction prediction tasks. This inherently interpretable framework represents a significant step in bridging the gap between predictive accuracy and mechanistic understanding in data-driven reaction outcome prediction.

Title: Towards Variational Flow Matching on General Geometries

• Authors: Olga Zaghen, Floor Eijkelboom, Alison Pouplin, Erik J. Bekkers
• Subjects: cs.LG, math.DG
• Abstract URL: https://arxiv.org/abs/2502.12981
• Pdf URL: https://arxiv.org/pdf/2502.12981
• Copy Paste: [[2502.12981]] Towards Variational Flow Matching on General Geometries(https://arxiv.org/abs/2502.12981)
• Keywords: generative
• Abstract: We introduce Riemannian Gaussian Variational Flow Matching (RG-VFM), an extension of Variational Flow Matching (VFM) that leverages Riemannian Gaussian distributions for generative modeling on structured manifolds. We derive a variational objective for probability flows on manifolds with closed-form geodesics, making RG-VFM comparable - though fundamentally different to Riemannian Flow Matching (RFM) in this geometric setting. Experiments on a checkerboard dataset wrapped on the sphere demonstrate that RG-VFM captures geometric structure more effectively than Euclidean VFM and baseline methods, establishing it as a robust framework for manifold-aware generative modeling.

Title: Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs

• Authors: Zixiao Wang, Duzhen Zhang, Ishita Agrawal, Shen Gao, Le Song, Xiuying Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.12988
• Pdf URL: https://arxiv.org/pdf/2502.12988
• Copy Paste: [[2502.12988]] Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs(https://arxiv.org/abs/2502.12988)
• Keywords: generative
• Abstract: Previous approaches to persona simulation large language models (LLMs) have typically relied on learning basic biographical information, or using limited role-play dialogue datasets to capture a character's responses. However, a holistic representation of an individual goes beyond surface-level facts or conversations to deeper thoughts and thinking. In this work, we introduce CharacterBot, a model designed to replicate both the linguistic patterns and distinctive thought processes of a character. Using Lu Xun, a renowned Chinese writer, as a case study, we propose four training tasks derived from his 17 essay collections. These include a pre-training task focused on mastering external linguistic structures and knowledge, as well as three fine-tuning tasks: multiple-choice question answering, generative question answering, and style transfer, each aligning the LLM with Lu Xun's internal ideation and writing style. To optimize learning across these tasks, we introduce a CharLoRA parameter updating mechanism, where a general linguistic style expert collaborates with other task-specific experts to better study both the language style and the understanding of deeper thoughts. We evaluate CharacterBot on three tasks for linguistic accuracy and opinion comprehension, demonstrating that it significantly outperforms the baselines on our adapted metrics. We hope that this work inspires future research on deep character persona simulation LLM.

Title: SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decomposition

• Authors: Rema Daher, Francisco Vasconcelos, Danail Stoyanov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.12994
• Pdf URL: https://arxiv.org/pdf/2502.12994
• Copy Paste: [[2502.12994]] SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decomposition(https://arxiv.org/abs/2502.12994)
• Keywords: self-supervised
• Abstract: Purpose: Visual 3D scene reconstruction can support colonoscopy navigation. It can help in recognising which portions of the colon have been visualised and characterising the size and shape of polyps. This is still a very challenging problem due to complex illumination variations, including abundant specular reflections. We investigate how to effectively decouple light and depth in this problem. Methods: We introduce a self-supervised model that simultaneously characterises the shape and lighting of the visualised colonoscopy scene. Our model estimates shading, albedo, depth, and specularities (SHADeS) from single images. Unlike previous approaches (IID), we use a non-Lambertian model that treats specular reflections as a separate light component. The implementation of our method is available at this https URL. Results: We demonstrate on real colonoscopy images (Hyper Kvasir) that previous models for light decomposition (IID) and depth estimation (MonoVIT, ModoDepth2) are negatively affected by specularities. In contrast, SHADeS can simultaneously produce light decomposition and depth maps that are robust to specular regions. We also perform a quantitative comparison on phantom data (C3VD) where we further demonstrate the robustness of our model. Conclusion: Modelling specular reflections improves depth estimation in colonoscopy. We propose an effective self-supervised approach that uses this insight to jointly estimate light decomposition and depth. Light decomposition has the potential to help with other problems, such as place recognition within the colon.

Title: A deep learning framework for efficient pathology image analysis

• Authors: Peter Neidlinger, Tim Lenz, Sebastian Foersch, Chiara M. L. Loeffler, Jan Clusmann, Marco Gustav, Lawrence A. Shaktah, Rupert Langer, Bastian Dislich, Lisa A. Boardman, Amy J. French, Ellen L. Goode, Andrea Gsur, Stefanie Brezina, Marc J. Gunter, Robert Steinfelder, Hans-Michael Behrens, Christoph Röcken, Tabitha Harrison, Ulrike Peters, Amanda I. Phipps, Giuseppe Curigliano, Nicola Fusco, Antonio Marra, Michael Hoffmeister, Hermann Brenner, Jakob Nikolas Kather
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13027
• Pdf URL: https://arxiv.org/pdf/2502.13027
• Copy Paste: [[2502.13027]] A deep learning framework for efficient pathology image analysis(https://arxiv.org/abs/2502.13027)
• Keywords: foundation model
• Abstract: Artificial intelligence (AI) has transformed digital pathology by enabling biomarker prediction from high-resolution whole slide images (WSIs). However, current methods are computationally inefficient, processing thousands of redundant tiles per WSI and requiring complex aggregator models. We introduce EAGLE (Efficient Approach for Guided Local Examination), a deep learning framework that emulates pathologists by selectively analyzing informative regions. EAGLE incorporates two foundation models: CHIEF for efficient tile selection and Virchow2 for extracting high-quality features. Benchmarking was conducted against leading slide- and tile-level foundation models across 31 tasks from four cancer types, spanning morphology, biomarker prediction and prognosis. EAGLE outperformed state-of-the-art foundation models by up to 23% and achieved the highest AUROC overall. It processed a slide in 2.27 seconds, reducing computational time by more than 99% compared to existing models. This efficiency enables real-time workflows, allows pathologists to validate all tiles which are used by the model during analysis, and eliminates dependence on high-performance computing, making AI-powered pathology more accessible. By reliably identifying meaningful regions and minimizing artifacts, EAGLE provides robust and interpretable outputs, supporting rapid slide searches, integration into multi-omics pipelines and emerging clinical foundation models.

Title: Personalized Image Generation with Deep Generative Models: A Decade Survey

• Authors: Yuxiang Wei, Yiheng Zheng, Yabo Zhang, Ming Liu, Zhilong Ji, Lei Zhang, Wangmeng Zuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13081
• Pdf URL: https://arxiv.org/pdf/2502.13081
• Copy Paste: [[2502.13081]] Personalized Image Generation with Deep Generative Models: A Decade Survey(https://arxiv.org/abs/2502.13081)
• Keywords: diffusion, generative
• Abstract: Recent advancements in generative models have significantly facilitated the development of personalized content creation. Given a small set of images with user-specific concept, personalized image generation allows to create images that incorporate the specified concept and adhere to provided text descriptions. Due to its wide applications in content creation, significant effort has been devoted to this field in recent years. Nonetheless, the technologies used for personalization have evolved alongside the development of generative models, with their distinct and interrelated components. In this survey, we present a comprehensive review of generalized personalized image generation across various generative models, including traditional GANs, contemporary text-to-image diffusion models, and emerging multi-model autoregressive models. We first define a unified framework that standardizes the personalization process across different generative models, encompassing three key components, i.e., inversion spaces, inversion methods, and personalization schemes. This unified framework offers a structured approach to dissecting and comparing personalization techniques across different generative architectures. Building upon this unified framework, we further provide an in-depth analysis of personalization techniques within each generative model, highlighting their unique contributions and innovations. Through comparative analysis, this survey elucidates the current landscape of personalized image generation, identifying commonalities and distinguishing features among existing methods. Finally, we discuss the open challenges in the field and propose potential directions for future research. We keep tracing related works at this https URL.

Title: Is Noise Conditioning Necessary for Denoising Generative Models?

• Authors: Qiao Sun, Zhicheng Jiang, Hanhong Zhao, Kaiming He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13129
• Pdf URL: https://arxiv.org/pdf/2502.13129
• Copy Paste: [[2502.13129]] Is Noise Conditioning Necessary for Denoising Generative Models?(https://arxiv.org/abs/2502.13129)
• Keywords: diffusion, generative
• Abstract: It is widely believed that noise conditioning is indispensable for denoising diffusion models to work successfully. This work challenges this belief. Motivated by research on blind image denoising, we investigate a variety of denoising-based generative models in the absence of noise conditioning. To our surprise, most models exhibit graceful degradation, and in some cases, they even perform better without noise conditioning. We provide a theoretical analysis of the error caused by removing noise conditioning and demonstrate that our analysis aligns with empirical observations. We further introduce a noise-unconditional model that achieves a competitive FID of 2.23 on CIFAR-10, significantly narrowing the gap to leading noise-conditional models. We hope our findings will inspire the community to revisit the foundations and formulations of denoising generative models.

Title: Magma: A Foundation Model for Multimodal AI Agents

• Authors: Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, Yuquan Deng, Lars Liden, Jianfeng Gao
• Subjects: cs.CV, cs.AI, cs.HC, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.13130
• Pdf URL: https://arxiv.org/pdf/2502.13130
• Copy Paste: [[2502.13130]] Magma: A Foundation Model for Multimodal AI Agents(https://arxiv.org/abs/2502.13130)
• Keywords: foundation model
• Abstract: We present Magma, a foundation model that serves multimodal AI agentic tasks in both the digital and physical worlds. Magma is a significant extension of vision-language (VL) models in that it not only retains the VL understanding ability (verbal intelligence) of the latter, but is also equipped with the ability to plan and act in the visual-spatial world (spatial-temporal intelligence) and complete agentic tasks ranging from UI navigation to robot manipulation. To endow the agentic capabilities, Magma is pretrained on large amounts of heterogeneous datasets spanning from images, videos to robotics data, where the actionable visual objects (e.g., clickable buttons in GUI) in images are labeled by Set-of-Mark (SoM) for action grounding, and the object movements (e.g., the trace of human hands or robotic arms) in videos are labeled by Trace-of-Mark (ToM) for action planning. Extensive experiments show that SoM and ToM reach great synergy and facilitate the acquisition of spatial-temporal intelligence for our Magma model, which is fundamental to a wide range of tasks as shown in Fig.1. In particular, Magma creates new state-of-the-art results on UI navigation and robotic manipulation tasks, outperforming previous models that are specifically tailored to these tasks. On image and video-related multimodal tasks, Magma also compares favorably to popular large multimodal models that are trained on much larger datasets. We make our model and code public for reproducibility at this https URL.

Title: AV-Flow: Transforming Text to Audio-Visual Human-like Interactions

• Authors: Aggelina Chatziagapi, Louis-Philippe Morency, Hongyu Gong, Michael Zollhoefer, Dimitris Samaras, Alexander Richard
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13133
• Pdf URL: https://arxiv.org/pdf/2502.13133
• Copy Paste: [[2502.13133]] AV-Flow: Transforming Text to Audio-Visual Human-like Interactions(https://arxiv.org/abs/2502.13133)
• Keywords: diffusion, generative
• Abstract: We introduce AV-Flow, an audio-visual generative model that animates photo-realistic 4D talking avatars given only text input. In contrast to prior work that assumes an existing speech signal, we synthesize speech and vision jointly. We demonstrate human-like speech synthesis, synchronized lip motion, lively facial expressions and head pose; all generated from just text characters. The core premise of our approach lies in the architecture of our two parallel diffusion transformers. Intermediate highway connections ensure communication between the audio and visual modalities, and thus, synchronized speech intonation and facial dynamics (e.g., eyebrow motion). Our model is trained with flow matching, leading to expressive results and fast inference. In case of dyadic conversations, AV-Flow produces an always-on avatar, that actively listens and reacts to the audio-visual input of a user. Through extensive experiments, we show that our method outperforms prior work, synthesizing natural-looking 4D talking avatars. Project page: this https URL

Title: Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions

• Authors: Taedong Yun, Eric Yang, Mustafa Safdari, Jong Ha Lee, Vaishnavi Vinod Kumar, S. Sara Mahdavi, Jonathan Amar, Derek Peyton, Reut Aharony, Andreas Michaelides, Logan Schneider, Isaac Galatzer-Levy, Yugang Jia, John Canny, Arthur Gretton, Maja Matarić
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13135
• Pdf URL: https://arxiv.org/pdf/2502.13135
• Copy Paste: [[2502.13135]] Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions(https://arxiv.org/abs/2502.13135)
• Keywords: generative
• Abstract: We present an end-to-end framework for generating synthetic users for evaluating interactive agents designed to encourage positive behavior changes, such as in health and lifestyle coaching. The synthetic users are grounded in health and lifestyle conditions, specifically sleep and diabetes management in this study, to ensure realistic interactions with the health coaching agent. Synthetic users are created in two stages: first, structured data are generated grounded in real-world health and lifestyle factors in addition to basic demographics and behavioral attributes; second, full profiles of the synthetic users are developed conditioned on the structured data. Interactions between synthetic users and the coaching agent are simulated using generative agent-based models such as Concordia, or directly by prompting a language model. Using two independently-developed agents for sleep and diabetes coaching as case studies, the validity of this framework is demonstrated by analyzing the coaching agent's understanding of the synthetic users' needs and challenges. Finally, through multiple blinded evaluations of user-coach interactions by human experts, we demonstrate that our synthetic users with health and behavioral attributes more accurately portray real human users with the same attributes, compared to generic synthetic users not grounded in such attributes. The proposed framework lays the foundation for efficient development of conversational agents through extensive, realistic, and grounded simulated interactions.