2025-02-04

Title: Efficient Client Selection in Federated Learning

• Authors: William Marfo, Deepak K. Tosh, Shirley V. Moore
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2502.00036
• Pdf URL: https://arxiv.org/pdf/2502.00036
• Copy Paste: [[2502.00036]] Efficient Client Selection in Federated Learning(https://arxiv.org/abs/2502.00036)
• Keywords: anomaly
• Abstract: Federated Learning (FL) enables decentralized machine learning while preserving data privacy. This paper proposes a novel client selection framework that integrates differential privacy and fault tolerance. The adaptive client selection adjusts the number of clients based on performance and system constraints, with noise added to protect privacy. Evaluated on the UNSW-NB15 and ROAD datasets for network anomaly detection, the method improves accuracy by 7% and reduces training time by 25% compared to baselines. Fault tolerance enhances robustness with minimal performance trade-offs.

Title: Bridging Contrastive Learning and Domain Adaptation: Theoretical Perspective and Practical Application

• Authors: Gonzalo Iñaki Quintana, Laurence Vancamberg, Vincent Jugnon, Agnès Desolneux, Mathilde Mougeot
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.00052
• Pdf URL: https://arxiv.org/pdf/2502.00052
• Copy Paste: [[2502.00052]] Bridging Contrastive Learning and Domain Adaptation: Theoretical Perspective and Practical Application(https://arxiv.org/abs/2502.00052)
• Keywords: self-supervised
• Abstract: This work studies the relationship between Contrastive Learning and Domain Adaptation from a theoretical perspective. The two standard contrastive losses, NT-Xent loss (Self-supervised) and Supervised Contrastive loss, are related to the Class-wise Mean Maximum Discrepancy (CMMD), a dissimilarity measure widely used for Domain Adaptation. Our work shows that minimizing the contrastive losses decreases the CMMD and simultaneously improves class-separability, laying the theoretical groundwork for the use of Contrastive Learning in the context of Domain Adaptation. Due to the relevance of Domain Adaptation in medical imaging, we focused the experiments on mammography images. Extensive experiments on three mammography datasets - synthetic patches, clinical (real) patches, and clinical (real) images - show improved Domain Adaptation, class-separability, and classification performance, when minimizing the Supervised Contrastive loss.

Title: AIN: The Arabic INclusive Large Multimodal Model

• Authors: Ahmed Heakl, Sara Ghaboura, Omkar Thawkar, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan
• Subjects: cs.CV, cs.AI, cs.CL, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00094
• Pdf URL: https://arxiv.org/pdf/2502.00094
• Copy Paste: [[2502.00094]] AIN: The Arabic INclusive Large Multimodal Model(https://arxiv.org/abs/2502.00094)
• Keywords: generative
• Abstract: Amid the swift progress of large language models (LLMs) and their evolution into large multimodal models (LMMs), significant strides have been made in high-resource languages such as English and Chinese. While Arabic LLMs have seen notable progress, Arabic LMMs remain largely unexplored, often narrowly focusing on a few specific aspects of the language and visual understanding. To bridge this gap, we introduce AIN-the Arabic Inclusive Multimodal Model-designed to excel across diverse domains. AIN is an English-Arabic bilingual LMM designed to excel in English and Arabic, leveraging carefully constructed 3.6 million high-quality Arabic-English multimodal data samples. AIN demonstrates state-of-the-art Arabic performance, while also possessing strong English-language visual capabilities. On the recent CAMEL-Bench benchmark comprising 38 sub-domains including, multi-image understanding, complex visual perception, handwritten document understanding, video understanding, medical imaging, plant diseases, and remote sensing-based land use understanding, our AIN demonstrates strong performance with the 7B model outperforming GPT-4o by an absolute gain of 3.4% averaged over eight domains and 38 sub-domains. AIN's superior capabilities position it as a significant step toward empowering Arabic speakers with advanced multimodal generative AI tools across diverse applications.

Title: ProtoSnap: Prototype Alignment for Cuneiform Signs

• Authors: Rachel Mikulinsky, Morris Alper, Shai Gordin, Enrique Jiménez, Yoram Cohen, Hadar Averbuch-Elor
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00129
• Pdf URL: https://arxiv.org/pdf/2502.00129
• Copy Paste: [[2502.00129]] ProtoSnap: Prototype Alignment for Cuneiform Signs(https://arxiv.org/abs/2502.00129)
• Keywords: generative
• Abstract: The cuneiform writing system served as the medium for transmitting knowledge in the ancient Near East for a period of over three thousand years. Cuneiform signs have a complex internal structure which is the subject of expert paleographic analysis, as variations in sign shapes bear witness to historical developments and transmission of writing and culture over time. However, prior automated techniques mostly treat sign types as categorical and do not explicitly model their highly varied internal configurations. In this work, we present an unsupervised approach for recovering the fine-grained internal configuration of cuneiform signs by leveraging powerful generative models and the appearance and structure of prototype font images as priors. Our approach, ProtoSnap, enforces structural consistency on matches found with deep image features to estimate the diverse configurations of cuneiform characters, snapping a skeleton-based template to photographed cuneiform signs. We provide a new benchmark of expert annotations and evaluate our method on this task. Our evaluation shows that our approach succeeds in aligning prototype skeletons to a wide variety of cuneiform signs. Moreover, we show that conditioning on structures produced by our method allows for generating synthetic data with correct structural configurations, significantly boosting the performance of cuneiform sign recognition beyond existing techniques, in particular over rare signs. Our code, data, and trained models are available at the project page: this https URL

Title: A Three-Branch Checks-and-Balances Frameworkfor Context-Aware Ethical Alignment of Large Language Models

• Authors: Edward Y. Chang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.00136
• Pdf URL: https://arxiv.org/pdf/2502.00136
• Copy Paste: [[2502.00136]] A Three-Branch Checks-and-Balances Frameworkfor Context-Aware Ethical Alignment of Large Language Models(https://arxiv.org/abs/2502.00136)
• Keywords: self-supervised
• Abstract: This paper introduces a three-branch checks-and-balances framework for ethical alignment of Large Language Models (LLMs), inspired by governmental systems. It implements three independent yet interacting components: LLMs as the executive branch for knowledge generation, DIKE as the legislative branch establishing ethical guardrails, and ERIS as the judicial branch for contextual interpretation. The adversarial DIKE-ERIS duality enables adaptation to diverse cultural contexts while upholding consistent ethical principles. This architecture addresses limitations of reinforcement learning with human feedback (RLHF) by providing interpretable, adaptable, and culturally-aware ethical reasoning. Through self-supervised learning and adversarial testing, our framework demonstrates how emotional modeling can guide linguistic behaviors toward ethical outcomes while preserving independence across knowledge generation, ethical oversight, and contextual interpretation.

Title: Designing Scheduling for Diffusion Models via Spectral Analysis

• Authors: Roi Benita, Michael Elad, Joseph Keshet
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00180
• Pdf URL: https://arxiv.org/pdf/2502.00180
• Copy Paste: [[2502.00180]] Designing Scheduling for Diffusion Models via Spectral Analysis(https://arxiv.org/abs/2502.00180)
• Keywords: diffusion
• Abstract: Diffusion models (DMs) have emerged as powerful tools for modeling complex data distributions and generating realistic new samples. Over the years, advanced architectures and sampling methods have been developed to make these models practically usable. However, certain synthesis process decisions still rely on heuristics without a solid theoretical foundation. In our work, we offer a novel analysis of the DM's inference process, introducing a comprehensive frequency response perspective. Specifically, by relying on Gaussianity and shift-invariance assumptions, we present the inference process as a closed-form spectral transfer function, capturing how the generated signal evolves in response to the initial noise. We demonstrate how the proposed analysis can be leveraged for optimizing the noise schedule, ensuring the best alignment with the original dataset's characteristics. Our results lead to scheduling curves that are dependent on the frequency content of the data, offering a theoretical justification for some of the heuristics taken by practitioners.

Title: Should You Use Your Large Language Model to Explore or Exploit?

• Authors: Keegan Harris, Aleksandrs Slivkins
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00225
• Pdf URL: https://arxiv.org/pdf/2502.00225
• Copy Paste: [[2502.00225]] Should You Use Your Large Language Model to Explore or Exploit?(https://arxiv.org/abs/2502.00225)
• Keywords: in-context
• Abstract: We evaluate the ability of the current generation of large language models (LLMs) to help a decision-making agent facing an exploration-exploitation tradeoff. We use LLMs to explore and exploit in silos in various (contextual) bandit tasks. We find that while the current LLMs often struggle to exploit, in-context mitigations may be used to substantially improve performance for small-scale tasks. However even then, LLMs perform worse than a simple linear regression. On the other hand, we find that LLMs do help at exploring large action spaces with inherent semantics, by suggesting suitable candidates to explore.

Title: Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms

• Authors: Yinuo Ren, Haoxuan Chen, Yuchen Zhu, Wei Guo, Yongxin Chen, Grant M. Rotskoff, Molei Tao, Lexing Ying
• Subjects: cs.LG, cs.CV, math.NA, physics.comp-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00234
• Pdf URL: https://arxiv.org/pdf/2502.00234
• Copy Paste: [[2502.00234]] Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms(https://arxiv.org/abs/2502.00234)
• Keywords: diffusion, generative
• Abstract: Discrete diffusion models have emerged as a powerful generative modeling framework for discrete data with successful applications spanning from text generation to image synthesis. However, their deployment faces challenges due to the high dimensionality of the state space, necessitating the development of efficient inference algorithms. Current inference approaches mainly fall into two categories: exact simulation and approximate methods such as $\tau$-leaping. While exact methods suffer from unpredictable inference time and redundant function evaluations, $\tau$-leaping is limited by its first-order accuracy. In this work, we advance the latter category by tailoring the first extension of high-order numerical inference schemes to discrete diffusion models, enabling larger step sizes while reducing error. We rigorously analyze the proposed schemes and establish the second-order accuracy of the $\theta$-trapezoidal method in KL divergence. Empirical evaluations on GPT-2 level text and ImageNet-level image generation tasks demonstrate that our method achieves superior sample quality compared to existing approaches under equivalent computational constraints.

Title: Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion

• Authors: Tianyuan Zou, Yang Liu, Peng Li, Yufei Xiong, Jianqing Zhang, Jingjing Liu, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00245
• Pdf URL: https://arxiv.org/pdf/2502.00245
• Copy Paste: [[2502.00245]] Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion(https://arxiv.org/abs/2502.00245)
• Keywords: generative
• Abstract: Substantial quantity and high quality are the golden rules of making a good training dataset with sample privacy protection equally important. Generating synthetic samples that resemble high-quality private data while ensuring Differential Privacy (DP), a formal privacy guarantee, promises scalability and practicality. However, existing methods relying on pre-trained models for data synthesis %that avoid fine-tuning large pre-trained generative models often struggle in data-deficient scenarios, suffering from limited sample size, inevitable generation noise and existing pre-trained model bias. To address these challenges, we propose a novel contrAstive private data Synthesis via Weighted multiple Pre-trained language models (PLM) framework, named as WASP. WASP utilizes limited private samples for more accurate private data distribution estimation via a Top-Q voting mechanism, and leverages low-quality synthetic samples for contrastive generation via collaboration among dynamically weighted multiple pre-trained this http URL experiments on 6 well-developed datasets with 6 open-source and 3 closed-source PLMs demonstrate the superiority of WASP in improving model performance over diverse downstream tasks. Code is available at this https URL.

Title: Regularized Langevin Dynamics for Combinatorial Optimization

• Authors: Shengyu Feng, Yiming Yang
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00277
• Pdf URL: https://arxiv.org/pdf/2502.00277
• Copy Paste: [[2502.00277]] Regularized Langevin Dynamics for Combinatorial Optimization(https://arxiv.org/abs/2502.00277)
• Keywords: generative
• Abstract: This work proposes a simple yet effective sampling framework for combinatorial optimization (CO). Our method builds on discrete Langevin dynamics (LD), an efficient gradient-guided generative algorithm. However, we observed that directly applying LD often leads to limited exploration. To overcome this limitation, we propose the Regularized Langevin Dynamics (RLD), which enforces an expected distance between the sampled and current solutions, effectively avoiding local minima. We develop two CO solvers on top of RLD, one based on simulated annealing (SA) and the other one based on neural network (NN). Empirical results on three classical CO problems demonstrate that both of our methods can achieve comparable or better performance against the previous state-of-the-art (SOTA) SA and NN-based solvers. In particular, our SA algorithm reduces the running time of the previous SOTA SA method by up to 80\%, while achieving equal or superior performance. In summary, RLD offers a promising framework for enhancing both traditional heuristics and NN models to solve CO problems.

Title: ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference

• Authors: Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Bo Li, Xuming Hu, Xiaowen Chu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00299
• Pdf URL: https://arxiv.org/pdf/2502.00299
• Copy Paste: [[2502.00299]] ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference(https://arxiv.org/abs/2502.00299)
• Keywords: in-context
• Abstract: To reduce memory costs in long-context inference with Large Language Models (LLMs), many recent works focus on compressing the key-value (KV) cache of different tokens. However, we identify that the previous KV cache compression methods measure token importance individually, neglecting the dependency between different tokens in the real-world language characterics. In light of this, we introduce ChunkKV, grouping the tokens in a chunk as a basic compressing unit, and retaining the most informative semantic chunks while discarding the less important ones. Furthermore, observing that ChunkKV exhibits higher similarity in the preserved indices across different layers, we propose layer-wise index reuse to further reduce computational overhead. We evaluated ChunkKV on cutting-edge long-context benchmarks including LongBench and Needle-In-A-HayStack, as well as the GSM8K and JailbreakV in-context learning benchmark. Our experiments with instruction tuning and multi-step reasoning (O1 and R1) LLMs, achieve up to 10\% performance improvement under aggressive compression ratios compared to existing methods.

Title: A Diffusion Model Translator for Efficient Image-to-Image Translation

• Authors: Mengfei Xia, Yu Zhou, Ran Yi, Yong-Jin Liu, Wenping Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00307
• Pdf URL: https://arxiv.org/pdf/2502.00307
• Copy Paste: [[2502.00307]] A Diffusion Model Translator for Efficient Image-to-Image Translation(https://arxiv.org/abs/2502.00307)
• Keywords: diffusion
• Abstract: Applying diffusion models to image-to-image translation (I2I) has recently received increasing attention due to its practical applications. Previous attempts inject information from the source image into each denoising step for an iterative refinement, thus resulting in a time-consuming implementation. We propose an efficient method that equips a diffusion model with a lightweight translator, dubbed a Diffusion Model Translator (DMT), to accomplish I2I. Specifically, we first offer theoretical justification that in employing the pioneering DDPM work for the I2I task, it is both feasible and sufficient to transfer the distribution from one domain to another only at some intermediate step. We further observe that the translation performance highly depends on the chosen timestep for domain transfer, and therefore propose a practical strategy to automatically select an appropriate timestep for a given task. We evaluate our approach on a range of I2I applications, including image stylization, image colorization, segmentation to image, and sketch to image, to validate its efficacy and general utility. The comparisons show that our DMT surpasses existing methods in both quality and efficiency. Code will be made publicly available.

Title: MonoDINO-DETR: Depth-Enhanced Monocular 3D Object Detection Using a Vision Foundation Model

• Authors: Jihyeok Kim, Seongwoo Moon, Sungwon Nah, David Hyunchul Shim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00315
• Pdf URL: https://arxiv.org/pdf/2502.00315
• Copy Paste: [[2502.00315]] MonoDINO-DETR: Depth-Enhanced Monocular 3D Object Detection Using a Vision Foundation Model(https://arxiv.org/abs/2502.00315)
• Keywords: foundation model
• Abstract: This paper proposes novel methods to enhance the performance of monocular 3D object detection models by leveraging the generalized feature extraction capabilities of a vision foundation model. Unlike traditional CNN-based approaches, which often suffer from inaccurate depth estimation and rely on multi-stage object detection pipelines, this study employs a Vision Transformer (ViT)-based foundation model as the backbone, which excels at capturing global features for depth estimation. It integrates a detection transformer (DETR) architecture to improve both depth estimation and object detection performance in a one-stage manner. Specifically, a hierarchical feature fusion block is introduced to extract richer visual features from the foundation model, further enhancing feature extraction capabilities. Depth estimation accuracy is further improved by incorporating a relative depth estimation model trained on large-scale data and fine-tuning it through transfer learning. Additionally, the use of queries in the transformer's decoder, which consider reference points and the dimensions of 2D bounding boxes, enhances recognition performance. The proposed model outperforms recent state-of-the-art methods, as demonstrated through quantitative and qualitative evaluations on the KITTI 3D benchmark and a custom dataset collected from high-elevation racing environments. Code is available at this https URL.

Title: From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation

• Authors: Xingchen Wan, Han Zhou, Ruoxi Sun, Hootan Nakhost, Ke Jiang, Sercan Ö. Arık
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00330
• Pdf URL: https://arxiv.org/pdf/2502.00330
• Copy Paste: [[2502.00330]] From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation(https://arxiv.org/abs/2502.00330)
• Keywords: in-context
• Abstract: Recent advances in long-context large language models (LLMs) have led to the emerging paradigm of many-shot in-context learning (ICL), where it is observed that scaling many more demonstrating examples beyond the conventional few-shot setup in the context can lead to performance benefits. However, despite its promise, it is unclear what aspects dominate the benefits and whether simply scaling to more examples is the most effective way of improving many-shot ICL. In this work, we first provide an analysis of the factors driving many-shot ICL, and we find that 1) many-shot performance can still be attributed to often a few disproportionately influential examples and 2) identifying such influential examples ("optimize") and using them as demonstrations to regenerate new examples ("generate") can lead to further improvements. Inspired by the findings, we propose BRIDGE, an algorithm that alternates between the optimize step with Bayesian optimization to discover the influential sets of examples and the generate step to reuse this set to expand the reasoning paths of the examples back to the many-shot regime automatically. On Gemini, Claude, and Mistral LLMs of different sizes, we show that BRIDGE to significant improvements across a diverse set of tasks, including symbolic reasoning, numerical reasoning, and code generation.

Title: BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution

• Authors: Kai Liu, Kaicheng Yang, Zheng Chen, Zhiteng Li, Yong Guo, Wenbo Li, Linghe Kong, Yulun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00333
• Pdf URL: https://arxiv.org/pdf/2502.00333
• Copy Paste: [[2502.00333]] BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution(https://arxiv.org/abs/2502.00333)
• Keywords: diffusion
• Abstract: While super-resolution (SR) methods based on diffusion models (DM) have demonstrated inspiring performance, their deployment is impeded due to the heavy request of memory and computation. Recent researchers apply two kinds of methods to compress or fasten the DM. One is to compress the DM into 1-bit, aka binarization, alleviating the storage and computation pressure. The other distills the multi-step DM into only one step, significantly speeding up inference process. Nonetheless, it remains impossible to deploy DM to resource-limited edge devices. To address this problem, we propose BiMaCoSR, which combines binarization and one-step distillation to obtain extreme compression and acceleration. To prevent the catastrophic collapse of the model caused by binarization, we proposed sparse matrix branch (SMB) and low rank matrixbranch (LRM). Both auxiliary branches pass the full-precision (FP) information but in different ways. SMB absorbs the extreme values and its output is high rank, carrying abundant FP information. Whereas, the design of LRMB is inspired by LoRA and is initialized with the top r SVD components, outputting low rank representation. The computation and storage overhead of our proposed branches can be safely ignored. Comprehensive comparison experiments are conducted to exhibit BiMaCoSR outperforms current state-of-the-art binarization methods and gains competitive performance compared with FP one-step model. BiMaCoSR achieves a 23.8x compression ratio and a 27.4x speedup ratio compared to FP counterpart. Our code and model are available at this https URL.

Title: Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves

• Authors: Anand Jerry George, Rodrigo Veiga, Nicolas Macris
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00336
• Pdf URL: https://arxiv.org/pdf/2502.00336
• Copy Paste: [[2502.00336]] Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves(https://arxiv.org/abs/2502.00336)
• Keywords: diffusion, generative
• Abstract: We derive asymptotically precise expressions for test and train errors of denoising score matching (DSM) in generative diffusion models. The score function is parameterized by random features neural networks, with the target distribution being $d$-dimensional standard Gaussian. We operate in a regime where the dimension $d$, number of data samples $n$, and number of features $p$ tend to infinity while keeping the ratios $\psi_n=\frac{n}{d}$ and $\psi_p=\frac{p}{d}$ fixed. By characterizing the test and train errors, we identify regimes of generalization and memorization in diffusion models. Furthermore, our work sheds light on the conditions enhancing either generalization or memorization. Consistent with prior empirical observations, our findings indicate that the model complexity ($p$) and the number of noise samples per data sample ($m$) used during DSM significantly influence generalization and memorization behaviors.

Title: Sampling in High-Dimensions using Stochastic Interpolants and Forward-Backward Stochastic Differential Equations

• Authors: Anand Jerry George, Nicolas Macris
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00355
• Pdf URL: https://arxiv.org/pdf/2502.00355
• Copy Paste: [[2502.00355]] Sampling in High-Dimensions using Stochastic Interpolants and Forward-Backward Stochastic Differential Equations(https://arxiv.org/abs/2502.00355)
• Keywords: diffusion
• Abstract: We present a class of diffusion-based algorithms to draw samples from high-dimensional probability distributions given their unnormalized densities. Ideally, our methods can transport samples from a Gaussian distribution to a specified target distribution in finite time. Our approach relies on the stochastic interpolants framework to define a time-indexed collection of probability densities that bridge a Gaussian distribution to the target distribution. Subsequently, we derive a diffusion process that obeys the aforementioned probability density at each time instant. Obtaining such a diffusion process involves solving certain Hamilton-Jacobi-Bellman PDEs. We solve these PDEs using the theory of forward-backward stochastic differential equations (FBSDE) together with machine learning-based methods. Through numerical experiments, we demonstrate that our algorithm can effectively draw samples from distributions that conventional methods struggle to handle.

Title: Exploring Representation-Aligned Latent Space for Better Generation

• Authors: Wanghan Xu, Xiaoyu Yue, Zidong Wang, Yao Teng, Wenlong Zhang, Xihui Liu, Luping Zhou, Wanli Ouyang, Lei Bai
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00359
• Pdf URL: https://arxiv.org/pdf/2502.00359
• Copy Paste: [[2502.00359]] Exploring Representation-Aligned Latent Space for Better Generation(https://arxiv.org/abs/2502.00359)
• Keywords: diffusion, generative
• Abstract: Generative models serve as powerful tools for modeling the real world, with mainstream diffusion models, particularly those based on the latent diffusion model paradigm, achieving remarkable progress across various tasks, such as image and video synthesis. Latent diffusion models are typically trained using Variational Autoencoders (VAEs), interacting with VAE latents rather than the real samples. While this generative paradigm speeds up training and inference, the quality of the generated outputs is limited by the latents' quality. Traditional VAE latents are often seen as spatial compression in pixel space and lack explicit semantic representations, which are essential for modeling the real world. In this paper, we introduce ReaLS (Representation-Aligned Latent Space), which integrates semantic priors to improve generation performance. Extensive experiments show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric. Furthermore, the enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.

Title: Shape from Semantics: 3D Shape Generation from Multi-View Semantics

• Authors: Liangchen Li, Caoliwen Wang, Yuqi Zhou, Bailin Deng, Juyong Zhang
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.00360
• Pdf URL: https://arxiv.org/pdf/2502.00360
• Copy Paste: [[2502.00360]] Shape from Semantics: 3D Shape Generation from Multi-View Semantics(https://arxiv.org/abs/2502.00360)
• Keywords: diffusion, generative
• Abstract: We propose ``Shape from Semantics'', which is able to create 3D models whose geometry and appearance match given semantics when observed from different views. Traditional ``Shape from X'' tasks usually use visual input (e.g., RGB images or depth maps) to reconstruct geometry, imposing strict constraints that limit creative explorations. As applications, works like Shadow Art and Wire Art often struggle to grasp the embedded semantics of their design through direct observation and rely heavily on specific setups for proper display. To address these limitations, our framework uses semantics as input, greatly expanding the design space to create objects that integrate multiple semantic elements and are easily discernible by observers. Considering that this task requires a rich imagination, we adopt various generative models and structure-to-detail pipelines. Specifically, we adopt multi-semantics Score Distillation Sampling (SDS) to distill 3D geometry and appearance from 2D diffusion models, ensuring that the initial shape is consistent with the semantic input. We then use image restoration and video generation models to add more details as supervision. Finally, we introduce neural signed distance field (SDF) representation to achieve detailed shape reconstruction. Our framework generates meshes with complex details, well-structured geometry, coherent textures, and smooth transitions, resulting in visually appealing and eye-catching designs. Project page: this https URL

Title: Soft Diffusion Actor-Critic: Efficient Online Reinforcement Learning for Diffusion Policy

• Authors: Haitong Ma, Tianyi Chen, Kai Wang, Na Li, Bo Dai
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00361
• Pdf URL: https://arxiv.org/pdf/2502.00361
• Copy Paste: [[2502.00361]] Soft Diffusion Actor-Critic: Efficient Online Reinforcement Learning for Diffusion Policy(https://arxiv.org/abs/2502.00361)
• Keywords: diffusion
• Abstract: Diffusion policies have achieved superior performance in imitation learning and offline reinforcement learning (RL) due to their rich expressiveness. However, the vanilla diffusion training procedure requires samples from target distribution, which is impossible in online RL since we cannot sample from the optimal policy, making training diffusion policies highly non-trivial in online RL. Backpropagating policy gradient through the diffusion process incurs huge computational costs and instability, thus being expensive and impractical. To enable efficient diffusion policy training for online RL, we propose Soft Diffusion Actor-Critic (SDAC), exploiting the viewpoint of diffusion models as noise-perturbed energy-based models. The proposed SDAC relies solely on the state-action value function as the energy functions to train diffusion policies, bypassing sampling from the optimal policy while maintaining lightweight computations. We conducted comprehensive comparisons on MuJoCo benchmarks. The empirical results show that SDAC outperforms all recent diffusion-policy online RLs on most tasks, and improves more than 120% over soft actor-critic on complex locomotion tasks such as Humanoid and Ant.

Title: NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning

• Authors: Zhixi Cai, Fucai Ke, Simindokht Jahangard, Maria Garcia de la Banda, Reza Haffari, Peter J. Stuckey, Hamid Rezatofighi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00372
• Pdf URL: https://arxiv.org/pdf/2502.00372
• Copy Paste: [[2502.00372]] NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning(https://arxiv.org/abs/2502.00372)
• Keywords: foundation model
• Abstract: Visual Grounding (VG) tasks, such as referring expression detection and segmentation tasks are important for linking visual entities to context, especially in complex reasoning tasks that require detailed query interpretation. This paper explores VG beyond basic perception, highlighting challenges for methods that require reasoning like human cognition. Recent advances in large language methods (LLMs) and Vision-Language methods (VLMs) have improved abilities for visual comprehension, contextual understanding, and reasoning. These methods are mainly split into end-to-end and compositional methods, with the latter offering more flexibility. Compositional approaches that integrate LLMs and foundation models show promising performance but still struggle with complex reasoning with language-based logical representations. To address these limitations, we propose NAVER, a compositional visual grounding method that integrates explicit probabilistic logic reasoning within a finite-state automaton, equipped with a self-correcting mechanism. This design improves robustness and interpretability in inference through explicit logic reasoning. Our results show that NAVER achieves SoTA performance comparing to recent end-to-end and compositional baselines. The code is available at this https URL .

Title: Scalable Framework for Classifying AI-Generated Content Across Modalities

• Authors: Anh-Kiet Duong, Petra Gomez-Krämer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00375
• Pdf URL: https://arxiv.org/pdf/2502.00375
• Copy Paste: [[2502.00375]] Scalable Framework for Classifying AI-Generated Content Across Modalities(https://arxiv.org/abs/2502.00375)
• Keywords: generative
• Abstract: The rapid growth of generative AI technologies has heightened the importance of effectively distinguishing between human and AI-generated content, as well as classifying outputs from diverse generative models. This paper presents a scalable framework that integrates perceptual hashing, similarity measurement, and pseudo-labeling to address these challenges. Our method enables the incorporation of new generative models without retraining, ensuring adaptability and robustness in dynamic scenarios. Comprehensive evaluations on the Defactify4 dataset demonstrate competitive performance in text and image classification tasks, achieving high accuracy across both distinguishing human and AI-generated content and classifying among generative methods. These results highlight the framework's potential for real-world applications as generative AI continues to evolve. Source codes are publicly available at this https URL.

Title: When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation

• Authors: Anna Min, Chenxu Hu, Yi Ren, Hang Zhao
• Subjects: cs.CL, cs.AI, cs.MM, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.00377
• Pdf URL: https://arxiv.org/pdf/2502.00377
• Copy Paste: [[2502.00377]] When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation(https://arxiv.org/abs/2502.00377)
• Keywords: self-supervised
• Abstract: Though end-to-end speech-to-text translation has been a great success, we argue that the cascaded speech-to-text translation model still has its place, which is usually criticized for the error propagation between automatic speech recognition (ASR) and machine translation (MT) models. In this paper, we explore the benefits of incorporating multiple candidates from ASR and self-supervised speech features into MT. Our analysis reveals that the primary cause of cascading errors stems from the increased divergence between similar samples in the speech domain when mapped to the text domain. By including multiple candidates and self-supervised speech features, our approach allows the machine translation model to choose the right words and ensure precise translation using various speech samples. This strategy minimizes error spread and takes advantage of large ASR and MT datasets, along with pre-trained ASR/MT models, while addressing associated issues.

Title: Masked Generative Nested Transformers with Decode Time Scaling

• Authors: Sahil Goyal, Debapriya Tula, Gagan Jain, Pradeep Shenoy, Prateek Jain, Sujoy Paul
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00382
• Pdf URL: https://arxiv.org/pdf/2502.00382
• Copy Paste: [[2502.00382]] Masked Generative Nested Transformers with Decode Time Scaling(https://arxiv.org/abs/2502.00382)
• Keywords: generative
• Abstract: Recent advances in visual generation have made significant strides in producing content of exceptional quality. However, most methods suffer from a fundamental problem - a bottleneck of inference computational efficiency. Most of these algorithms involve multiple passes over a transformer model to generate tokens or denoise inputs. However, the model size is kept consistent throughout all iterations, which makes it computationally expensive. In this work, we aim to address this issue primarily through two key ideas - (a) not all parts of the generation process need equal compute, and we design a decode time model scaling schedule to utilize compute effectively, and (b) we can cache and reuse some of the computation. Combining these two ideas leads to using smaller models to process more tokens while large models process fewer tokens. These different-sized models do not increase the parameter size, as they share parameters. We rigorously experiment with ImageNet256$\times$256 , UCF101, and Kinetics600 to showcase the efficacy of the proposed method for image/video generation and frame prediction. Our experiments show that with almost $3\times$ less compute than baseline, our model obtains competitive performance.

Title: Predictive modeling and anomaly detection in large-scale web portals through the CAWAL framework

• Authors: Ozkan Canay, Umit Kocabicak
• Subjects: cs.LG, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.00413
• Pdf URL: https://arxiv.org/pdf/2502.00413
• Copy Paste: [[2502.00413]] Predictive modeling and anomaly detection in large-scale web portals through the CAWAL framework(https://arxiv.org/abs/2502.00413)
• Keywords: anomaly
• Abstract: This study presents an approach that uses session and page view data collected through the CAWAL framework, enriched through specialized processes, for advanced predictive modeling and anomaly detection in web usage mining (WUM) applications. Traditional WUM methods often rely on web server logs, which limit data diversity and quality. Integrating application logs with web analytics, the CAWAL framework creates comprehensive session and page view datasets, providing a more detailed view of user interactions and effectively addressing these limitations. This integration enhances data diversity and quality while eliminating the preprocessing stage required in conventional WUM, leading to greater process efficiency. The enriched datasets, created by cross-integrating session and page view data, were applied to advanced machine learning models, such as Gradient Boosting and Random Forest, which are known for their effectiveness in capturing complex patterns and modeling non-linear relationships. These models achieved over 92% accuracy in predicting user behavior and significantly improved anomaly detection capabilities. The results show that this approach offers detailed insights into user behavior and system performance metrics, making it a reliable solution for improving large-scale web portals' efficiency, reliability, and scalability.

Title: Parameter Efficient Fine-Tuning of Segment Anything Model

• Authors: Carolin Teuber, Anwai Archit, Constantin Pape
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00418
• Pdf URL: https://arxiv.org/pdf/2502.00418
• Copy Paste: [[2502.00418]] Parameter Efficient Fine-Tuning of Segment Anything Model(https://arxiv.org/abs/2502.00418)
• Keywords: foundation model
• Abstract: Segmentation is an important analysis task for biomedical images, enabling the study of individual organelles, cells or organs. Deep learning has massively improved segmentation methods, but challenges remain in generalization to new conditions, requiring costly data annotation. Vision foundation models, such as Segment Anything Model (SAM), address this issue through broad segmentation capabilities. However, these models still require finetuning on annotated data, although with less annotations, to achieve optimal results for new conditions. As a downside, they require more computational resources. This makes parameter-efficient finetuning (PEFT) relevant for their application. We contribute the first comprehensive study of PEFT for SAM applied to biomedical segmentation by evaluating 9 PEFT methods on diverse datasets. We also provide an implementation of QLoRA for vision transformers and a new approach for resource-efficient finetuning of SAM. Our code is publicly available at this https URL.

Title: TeST-V: TEst-time Support-set Tuning for Zero-shot Video Classification

• Authors: Rui Yan, Jin Wang, Hongyu Qu, Xiaoyu Du, Dong Zhang, Jinhui Tang, Tieniu Tan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00426
• Pdf URL: https://arxiv.org/pdf/2502.00426
• Copy Paste: [[2502.00426]] TeST-V: TEst-time Support-set Tuning for Zero-shot Video Classification(https://arxiv.org/abs/2502.00426)
• Keywords: self-supervised
• Abstract: Recently, adapting Vision Language Models (VLMs) to zero-shot visual classification by tuning class embedding with a few prompts (Test-time Prompt Tuning, TPT) or replacing class names with generated visual samples (support-set) has shown promising results. However, TPT cannot avoid the semantic gap between modalities while the support-set cannot be tuned. To this end, we draw on each other's strengths and propose a novel framework namely TEst-time Support-set Tuning for zero-shot Video Classification (TEST-V). It first dilates the support-set with multiple prompts (Multi-prompting Support-set Dilation, MSD) and then erodes the support-set via learnable weights to mine key cues dynamically (Temporal-aware Support-set Erosion, TSE). Specifically, i) MSD expands the support samples for each class based on multiple prompts enquired from LLMs to enrich the diversity of the support-set. ii) TSE tunes the support-set with factorized learnable weights according to the temporal prediction consistency in a self-supervised manner to dig pivotal supporting cues for each class. $\textbf{TEST-V}$ achieves state-of-the-art results across four benchmarks and has good interpretability for the support-set dilation and erosion.

Title: CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Models

• Authors: Xinle Cheng, Zhuoming Chen, Zhihao Jia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00433
• Pdf URL: https://arxiv.org/pdf/2502.00433
• Copy Paste: [[2502.00433]] CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.00433)
• Keywords: diffusion, generative
• Abstract: Diffusion models have revolutionized generative tasks, especially in the domain of text-to-image synthesis; however, their iterative denoising process demands substantial computational resources. In this paper, we present a novel acceleration strategy that integrates token-level pruning with caching techniques to tackle this computational challenge. By employing noise relative magnitude, we identify significant token changes across denoising iterations. Additionally, we enhance token selection by incorporating spatial clustering and ensuring distributional balance. Our experiments demonstrate reveal a 50%-60% reduction in computational costs while preserving the performance of the model, thereby markedly increasing the efficiency of diffusion models. The code is available at this https URL

Title: SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models

• Authors: Chuc Man Duc, Hiromichi Fukui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00435
• Pdf URL: https://arxiv.org/pdf/2502.00435
• Copy Paste: [[2502.00435]] SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models(https://arxiv.org/abs/2502.00435)
• Keywords: self-supervised, foundation model
• Abstract: Foundation models refer to deep learning models pretrained on large unlabeled datasets through self-supervised algorithms. In the Earth science and remote sensing communities, there is growing interest in transforming the use of Earth observation data, including satellite and aerial imagery, through foundation models. Various foundation models have been developed for remote sensing, such as those for multispectral, high-resolution, and hyperspectral images, and have demonstrated superior performance on various downstream tasks compared to traditional supervised models. These models are evolving rapidly, with capabilities to handle multispectral, multitemporal, and multisensor data. Most studies use masked autoencoders in combination with Vision Transformers (ViTs) as the backbone for pretraining. While the models showed promising performance, ViTs face challenges, such as quadratic computational scaling with input length, which may limit performance on multiband and multitemporal data with long sequences. This research aims to address these challenges by proposing SatMamba, a new pretraining framework that combines masked autoencoders with State Space Model, offering linear computational scaling. Experiments on high-resolution imagery across various downstream tasks show promising results, paving the way for more efficient foundation models and unlocking the full potential of Earth observation data. The source code is available in this https URL.

Title: HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering

• Authors: Taiji Li, Hao Chen, Fei Yu, Yin Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00448
• Pdf URL: https://arxiv.org/pdf/2502.00448
• Copy Paste: [[2502.00448]] HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering(https://arxiv.org/abs/2502.00448)
• Keywords: foundation model
• Abstract: Despite the rapid growth of context length of large language models (LLMs) , LLMs still perform poorly in long document summarization. An important reason for this is that relevant information about an event is scattered throughout long documents, and the messy narrative order impairs the accurate understanding and utilization of LLMs for long documents. To address these issues, we propose a novel summary generation framework, called HERA. Specifically, we first segment a long document by its semantic structure and retrieve text segments about the same event, and finally reorder them to form the input context. We evaluate our approach on two long document summarization datasets. The experimental results show that HERA outperforms foundation models in ROUGE, BERTScore and faithfulness metrics, while HERA does not require additional fine-tuning and resources.

Title: Enhancing Memory and Imagination Consistency in Diffusion-based World Models via Linear-Time Sequence Modeling

• Authors: Jia-Hua Lee, Bor-Jiun Lin, Wei-Fang Sun, Chun-Yi Lee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00466
• Pdf URL: https://arxiv.org/pdf/2502.00466
• Copy Paste: [[2502.00466]] Enhancing Memory and Imagination Consistency in Diffusion-based World Models via Linear-Time Sequence Modeling(https://arxiv.org/abs/2502.00466)
• Keywords: diffusion
• Abstract: World models are crucial for enabling agents to simulate and plan within environments, yet existing approaches struggle with long-term dependencies and inconsistent predictions. We introduce EDELINE, a novel framework that integrates diffusion models with linear-time state space modelsto enhance memory retention and temporal consistency. EDELINE employs a recurrent embedding module based on Mamba SSMs for processing unbounded sequences, a unified architecture for joint reward and termination prediction, and dynamic loss harmonization to balance multi-task learning. Our results across multiple benchmarks demonstrate EDELINE's superiority and robustness over prior baselines in long-horizon tasks.

Title: Weak-to-Strong Diffusion with Reflection

• Authors: Lichen Bai, Masashi Sugiyama, Zeke Xie
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00473
• Pdf URL: https://arxiv.org/pdf/2502.00473
• Copy Paste: [[2502.00473]] Weak-to-Strong Diffusion with Reflection(https://arxiv.org/abs/2502.00473)
• Keywords: diffusion, generative
• Abstract: The goal of diffusion generative models is to align the learned distribution with the real data distribution through gradient score matching. However, inherent limitations in training data quality, modeling strategies, and architectural design lead to inevitable gap between generated outputs and real data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel framework that utilizes the estimated difference between existing weak and strong models (i.e., weak-to-strong difference) to approximate the gap between an ideal model and a strong model. By employing a reflective operation that alternates between denoising and inversion with weak-to-strong difference, we theoretically understand that W2SD steers latent variables along sampling trajectories toward regions of the real data distribution. W2SD is highly flexible and broadly applicable, enabling diverse improvements through the strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5, good experts vs. bad experts in MoE). Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence, achieving SOTA performance across various modalities (e.g., image, video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to 90% over the original results. Moreover, the performance gains achieved by W2SD markedly outweigh its additional computational overhead, while the cumulative improvements from different weak-to-strong difference further solidify its practical utility and deployability.

Title: A framework for river connectivity classification using temporal image processing and attention based neural networks

• Authors: Timothy James Becker, Derin Gezgin, Jun Yi He Wu, Mary Becker
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.00474
• Pdf URL: https://arxiv.org/pdf/2502.00474
• Copy Paste: [[2502.00474]] A framework for river connectivity classification using temporal image processing and attention based neural networks(https://arxiv.org/abs/2502.00474)
• Keywords: diffusion, generative
• Abstract: Measuring the connectivity of water in rivers and streams is essential for effective water resource management. Increased extreme weather events associated with climate change can result in alterations to river and stream connectivity. While traditional stream flow gauges are costly to deploy and limited to large river bodies, trail camera methods are a low-cost and easily deployed alternative to collect hourly data. Image capturing, however requires stream ecologists to manually curate (select and label) tens of thousands of images per year. To improve this workflow, we developed an automated instream trail camera image classification system consisting of three parts: (1) image processing, (2) image augmentation and (3) machine learning. The image preprocessing consists of seven image quality filters, foliage-based luma variance reduction, resizing and bottom-center cropping. Images are balanced using variable amount of generative augmentation using diffusion models and then passed to a machine learning classification model in labeled form. By using the vision transformer architecture and temporal image enhancement in our framework, we are able to increase the 75% base accuracy to 90% for a new unseen site image. We make use of a dataset captured and labeled by staff from the Connecticut Department of Energy and Environmental Protection between 2018-2020. Our results indicate that a combination of temporal image processing and attention-based models are effective at classifying unseen river connectivity images.

Title: Muti-Fidelity Prediction and Uncertainty Quantification with Laplace Neural Operators for Parametric Partial Differential Equations

• Authors: Haoyang Zheng, Guang Lin
• Subjects: cs.LG, math.NA, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2502.00550
• Pdf URL: https://arxiv.org/pdf/2502.00550
• Copy Paste: [[2502.00550]] Muti-Fidelity Prediction and Uncertainty Quantification with Laplace Neural Operators for Parametric Partial Differential Equations(https://arxiv.org/abs/2502.00550)
• Keywords: diffusion
• Abstract: Laplace Neural Operators (LNOs) have recently emerged as a promising approach in scientific machine learning due to the ability to learn nonlinear maps between functional spaces. However, this framework often requires substantial amounts of high-fidelity (HF) training data, which is often prohibitively expensive to acquire. To address this, we propose multi-fidelity Laplace Neural Operators (MF-LNOs), which combine a low-fidelity (LF) base model with parallel linear/nonlinear HF correctors and dynamic inter-fidelity weighting. This allows us to exploit correlations between LF and HF datasets and achieve accurate inference of quantities of interest even with sparse HF data. We further incorporate a modified replica exchange stochastic gradient Langevin algorithm, which enables a more effective posterior distribution estimation and uncertainty quantification in model predictions. Extensive validation across four canonical dynamical systems (the Lorenz system, Duffing oscillator, Burgers equation, and Brusselator reaction-diffusion system) demonstrates the framework's effectiveness. The results show significant improvements, with testing losses reduced by 40% to 80% compared to traditional approaches. This validates MF-LNO as a versatile tool for surrogate modeling in parametric PDEs, offering significant improvements in data efficiency and uncertainty-aware prediction.

Title: Optimal Sensor Placement in Power Transformers Using Physics-Informed Neural Networks

• Authors: Sirui Li, Federica Bragone, Matthieu Barreau, Tor Laneryd, Kateryna Morozovska
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2502.00552
• Pdf URL: https://arxiv.org/pdf/2502.00552
• Copy Paste: [[2502.00552]] Optimal Sensor Placement in Power Transformers Using Physics-Informed Neural Networks(https://arxiv.org/abs/2502.00552)
• Keywords: diffusion
• Abstract: Our work aims at simulating and predicting the temperature conditions inside a power transformer using Physics-Informed Neural Networks (PINNs). The predictions obtained are then used to determine the optimal placement for temperature sensors inside the transformer under the constraint of a limited number of sensors, enabling efficient performance monitoring. The method consists of combining PINNs with Mixed Integer Optimization Programming to obtain the optimal temperature reconstruction inside the transformer. First, we extend our PINN model for the thermal modeling of power transformers to solve the heat diffusion equation from 1D to 2D space. Finally, we construct an optimal sensor placement model inside the transformer that can be applied to problems in 1D and 2D.

Title: Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions

• Authors: Samiran Dey, Christopher R.S. Banerji, Partha Basuchowdhuri, Sanjoy K. Saha, Deepak Parashar, Tapabrata Chakraborti
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00568
• Pdf URL: https://arxiv.org/pdf/2502.00568
• Copy Paste: [[2502.00568]] Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions(https://arxiv.org/abs/2502.00568)
• Keywords: diffusion, generative
• Abstract: Emerging research has highlighted that artificial intelligence based multimodal fusion of digital pathology and transcriptomic features can improve cancer diagnosis (grading/subtyping) and prognosis (survival risk) prediction. However, such direct fusion for joint decision is impractical in real clinical settings, where histopathology is still the gold standard for diagnosis and transcriptomic tests are rarely requested, at least in the public healthcare system. With our novel diffusion based crossmodal generative AI model PathoGen, we show that genomic expressions synthesized from digital histopathology jointly predicts cancer grading and patient survival risk with high accuracy (state-of-the-art performance), certainty (through conformal coverage guarantee) and interpretability (through distributed attention maps). PathoGen code is available for open use by the research community through GitHub at this https URL.

Title: Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

• Authors: Stuart Armstrong, Matija Franklin, Connor Stevens, Rebecca Gorman
• Subjects: cs.CR, cs.AI, cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.00580
• Pdf URL: https://arxiv.org/pdf/2502.00580
• Copy Paste: [[2502.00580]] Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation(https://arxiv.org/abs/2502.00580)
• Keywords: generative
• Abstract: Recent work showed Best-of-N (BoN) jailbreaking using repeated use of random augmentations (such as capitalization, punctuation, etc) is effective against all major large language models (LLMs). We have found that $100\%$ of the BoN paper's successful jailbreaks (confidence interval $[99.65\%, 100.00\%]$) and $99.8\%$ of successful jailbreaks in our replication (confidence interval $[99.28\%, 99.98\%]$) were blocked with our Defense Against The Dark Prompts (DATDP) method. The DATDP algorithm works by repeatedly utilizing an evaluation LLM to evaluate a prompt for dangerous or manipulative behaviors--unlike some other approaches, DATDP also explicitly looks for jailbreaking attempts--until a robust safety rating is generated. This success persisted even when utilizing smaller LLMs to power the evaluation (Claude and LLaMa-3-8B-instruct proved almost equally capable). These results show that, though language models are sensitive to seemingly innocuous changes to inputs, they seem also capable of successfully evaluating the dangers of these inputs. Versions of DATDP can therefore be added cheaply to generative AI systems to produce an immediate significant increase in safety.

Title: Using Causality for Enhanced Prediction of Web Traffic Time Series

• Authors: Chang Tian, Mingzhe Xing, Zenglin Shi, Matthew B. Blaschko, Yinliang Yue, Marie-Francine Moens
• Subjects: cs.LG, cs.NI
• Abstract URL: https://arxiv.org/abs/2502.00612
• Pdf URL: https://arxiv.org/pdf/2502.00612
• Copy Paste: [[2502.00612]] Using Causality for Enhanced Prediction of Web Traffic Time Series(https://arxiv.org/abs/2502.00612)
• Keywords: anomaly
• Abstract: Predicting web service traffic has significant social value, as it can be applied to various practical scenarios, including but not limited to dynamic resource scaling, load balancing, system anomaly detection, service-level agreement compliance, and fraud detection. Web service traffic is characterized by frequent and drastic fluctuations over time and are influenced by heterogeneous web user behaviors, making accurate prediction a challenging task. Previous research has extensively explored statistical approaches, and neural networks to mine features from preceding service traffic time series for prediction. However, these methods have largely overlooked the causal relationships between services. Drawing inspiration from causality in ecological systems, we empirically recognize the causal relationships between web services. To leverage these relationships for improved web service traffic prediction, we propose an effective neural network module, CCMPlus, designed to extract causal relationship features across services. This module can be seamlessly integrated with existing time series models to consistently enhance the performance of web service traffic predictions. We theoretically justify that the causal correlation matrix generated by the CCMPlus module captures causal relationships among services. Empirical results on real-world datasets from Microsoft Azure, Alibaba Group, and Ant Group confirm that our method surpasses state-of-the-art approaches in Mean Squared Error (MSE) and Mean Absolute Error (MAE) for predicting service traffic time series. These findings highlight the efficacy of leveraging causal relationships for improved predictions.

Title: Zeroth-order Informed Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer

• Authors: Tao Ren, Zishi Zhang, Zehao Li, Jingyang Jiang, Shentao Qin, Guanghao Li, Yan Li, Yi Zheng, Xinping Li, Min Zhan, Yijie Peng
• Subjects: cs.CV, cs.AI, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.00639
• Pdf URL: https://arxiv.org/pdf/2502.00639
• Copy Paste: [[2502.00639]] Zeroth-order Informed Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer(https://arxiv.org/abs/2502.00639)
• Keywords: diffusion
• Abstract: The probabilistic diffusion model (DM), generating content by inferencing through a recursive chain structure, has emerged as a powerful framework for visual generation. After pre-training on enormous unlabeled data, the model needs to be properly aligned to meet requirements for downstream applications. How to efficiently align the foundation DM is a crucial task. Contemporary methods are either based on Reinforcement Learning (RL) or truncated Backpropagation (BP). However, RL and truncated BP suffer from low sample efficiency and biased gradient estimation respectively, resulting in limited improvement or, even worse, complete training failure. To overcome the challenges, we propose the Recursive Likelihood Ratio (RLR) optimizer, a zeroth-order informed fine-tuning paradigm for DM. The zeroth-order gradient estimator enables the computation graph rearrangement within the recursive diffusive chain, making the RLR's gradient estimator an unbiased one with the lower variance than other methods. We provide theoretical guarantees for the performance of the RLR. Extensive experiments are conducted on image and video generation tasks to validate the superiority of the RLR. Furthermore, we propose a novel prompt technique that is natural for the RLR to achieve a synergistic effect.

Title: EmoTalkingGaussian: Continuous Emotion-conditioned Talking Head Synthesis

• Authors: Junuk Cha, Seongro Yoon, Valeriya Strizhkova, Francois Bremond, Seungryul Baek
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00654
• Pdf URL: https://arxiv.org/pdf/2502.00654
• Copy Paste: [[2502.00654]] EmoTalkingGaussian: Continuous Emotion-conditioned Talking Head Synthesis(https://arxiv.org/abs/2502.00654)
• Keywords: self-supervised
• Abstract: 3D Gaussian splatting-based talking head synthesis has recently gained attention for its ability to render high-fidelity images with real-time inference speed. However, since it is typically trained on only a short video that lacks the diversity in facial emotions, the resultant talking heads struggle to represent a wide range of emotions. To address this issue, we propose a lip-aligned emotional face generator and leverage it to train our EmoTalkingGaussian model. It is able to manipulate facial emotions conditioned on continuous emotion values (i.e., valence and arousal); while retaining synchronization of lip movements with input audio. Additionally, to achieve the accurate lip synchronization for in-the-wild audio, we introduce a self-supervised learning method that leverages a text-to-speech network and a visual-audio synchronization network. We experiment our EmoTalkingGaussian on publicly available videos and have obtained better results than state-of-the-arts in terms of image quality (measured in PSNR, SSIM, LPIPS), emotion expression (measured in V-RMSE, A-RMSE, V-SA, A-SA, Emotion Accuracy), and lip synchronization (measured in LMD, Sync-E, Sync-C), respectively.

Title: High-Order Matching for One-Step Shortcut Diffusion Models

• Authors: Bo Chen, Chengyue Gong, Xiaoyu Li, Yingyu Liang, Zhizhou Sha, Zhenmei Shi, Zhao Song, Mingda Wan
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00688
• Pdf URL: https://arxiv.org/pdf/2502.00688
• Copy Paste: [[2502.00688]] High-Order Matching for One-Step Shortcut Diffusion Models(https://arxiv.org/abs/2502.00688)
• Keywords: diffusion, generative
• Abstract: One-step shortcut diffusion models [Frans, Hafner, Levine and Abbeel, ICLR 2025] have shown potential in vision generation, but their reliance on first-order trajectory supervision is fundamentally limited. The Shortcut model's simplistic velocity-only approach fails to capture intrinsic manifold geometry, leading to erratic trajectories, poor geometric alignment, and instability-especially in high-curvature regions. These shortcomings stem from its inability to model mid-horizon dependencies or complex distributional features, leaving it ill-equipped for robust generative modeling. In this work, we introduce HOMO (High-Order Matching for One-Step Shortcut Diffusion), a game-changing framework that leverages high-order supervision to revolutionize distribution transportation. By incorporating acceleration, jerk, and beyond, HOMO not only fixes the flaws of the Shortcut model but also achieves unprecedented smoothness, stability, and geometric precision. Theoretically, we prove that HOMO's high-order supervision ensures superior approximation accuracy, outperforming first-order methods. Empirically, HOMO dominates in complex settings, particularly in high-curvature regions where the Shortcut model struggles. Our experiments show that HOMO delivers smoother trajectories and better distributional alignment, setting a new standard for one-step generative models.

Title: Leveraging Large Language Models to Predict Antibody Biological Activity Against Influenza A Hemagglutinin

• Authors: Ella Barkan, Ibrahim Siddiqui, Kevin J. Cheng, Alex Golts, Yoel Shoshan, Jeffrey K. Weber, Yailin Campos Mota, Michal Ozery-Flato, Giuseppe A. Sautto
• Subjects: cs.LG, cs.AI, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.00694
• Pdf URL: https://arxiv.org/pdf/2502.00694
• Copy Paste: [[2502.00694]] Leveraging Large Language Models to Predict Antibody Biological Activity Against Influenza A Hemagglutinin(https://arxiv.org/abs/2502.00694)
• Keywords: foundation model
• Abstract: Monoclonal antibodies (mAbs) represent one of the most prevalent FDA-approved modalities for treating autoimmune diseases, infectious diseases, and cancers. However, discovery and development of therapeutic antibodies remains a time-consuming and expensive process. Recent advancements in machine learning (ML) and artificial intelligence (AI) have shown significant promise in revolutionizing antibody discovery and optimization. In particular, models that predict antibody biological activity enable in-silico evaluation of binding and functional properties; such models can prioritize antibodies with the highest likelihoods of success in costly and time-intensive laboratory testing procedures. We here explore an AI model for predicting the binding and receptor blocking activity of antibodies against influenza A hemagglutinin (HA) antigens. Our present model is developed with the MAMMAL framework for biologics discovery to predict antibody-antigen interactions using only sequence information. To evaluate the model's performance, we tested it under various data split conditions to mimic real-world scenarios. Our models achieved an AUROC $\geq$ 0.91 for predicting the activity of existing antibodies against seen HAs and an AUROC of 0.9 for unseen HAs. For novel antibody activity prediction, the AUROC was 0.73, which further declined to 0.63-0.66 under stringent constraints on similarity to existing antibodies. These results demonstrate the potential of AI foundation models to transform antibody design by reducing dependence on extensive laboratory testing and enabling more efficient prioritization of antibody candidates. Moreover, our findings emphasize the critical importance of diverse and comprehensive antibody datasets to improve the generalization of prediction models, particularly for novel antibody development.

Title: Model Provenance Testing for Large Language Models

• Authors: Ivica Nikolic, Teodora Baluta, Prateek Saxena
• Subjects: cs.CR, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00706
• Pdf URL: https://arxiv.org/pdf/2502.00706
• Copy Paste: [[2502.00706]] Model Provenance Testing for Large Language Models(https://arxiv.org/abs/2502.00706)
• Keywords: foundation model
• Abstract: Large language models are increasingly customized through fine-tuning and other adaptations, creating challenges in enforcing licensing terms and managing downstream impacts. Tracking model origins is crucial both for protecting intellectual property and for identifying derived models when biases or vulnerabilities are discovered in foundation models. We address this challenge by developing a framework for testing model provenance: Whether one model is derived from another. Our approach is based on the key observation that real-world model derivations preserve significant similarities in model outputs that can be detected through statistical analysis. Using only black-box access to models, we employ multiple hypothesis testing to compare model similarities against a baseline established by unrelated models. On two comprehensive real-world benchmarks spanning models from 30M to 4B parameters and comprising over 600 models, our tester achieves 90-95% precision and 80-90% recall in identifying derived models. These results demonstrate the viability of systematic provenance verification in production environments even when only API access is available.

Title: PhiP-G: Physics-Guided Text-to-3D Compositional Scene Generation

• Authors: Qixuan Li, Chao Wang, Zongjin He, Yan Peng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.00708
• Pdf URL: https://arxiv.org/pdf/2502.00708
• Copy Paste: [[2502.00708]] PhiP-G: Physics-Guided Text-to-3D Compositional Scene Generation(https://arxiv.org/abs/2502.00708)
• Keywords: diffusion
• Abstract: Text-to-3D asset generation has achieved significant optimization under the supervision of 2D diffusion priors. However, when dealing with compositional scenes, existing methods encounter several challenges: 1). failure to ensure that composite scene layouts comply with physical laws; 2). difficulty in accurately capturing the assets and relationships described in complex scene descriptions; 3). limited autonomous asset generation capabilities among layout approaches leveraging large language models (LLMs). To avoid these compromises, we propose a novel framework for compositional scene generation, PhiP-G, which seamlessly integrates generation techniques with layout guidance based on a world model. Leveraging LLM-based agents, PhiP-G analyzes the complex scene description to generate a scene graph, and integrating a multimodal 2D generation agent and a 3D Gaussian generation method for targeted assets creation. For the stage of layout, PhiP-G employs a physical pool with adhesion capabilities and a visual supervision agent, forming a world model for layout prediction and planning. Extensive experiments demonstrate that PhiP-G significantly enhances the generation quality and physical rationality of the compositional scenes. Notably, PhiP-G attains state-of-the-art (SOTA) performance in CLIP scores, achieves parity with the leading methods in generation quality as measured by the T$^3$Bench, and improves efficiency by 24x.

Title: Understanding and Mitigating the High Computational Cost in Path Data Diffusion

• Authors: Dingyuan Shi, Lulu Zhang, Yongxin Tong, Ke Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00725
• Pdf URL: https://arxiv.org/pdf/2502.00725
• Copy Paste: [[2502.00725]] Understanding and Mitigating the High Computational Cost in Path Data Diffusion(https://arxiv.org/abs/2502.00725)
• Keywords: diffusion
• Abstract: Advancements in mobility services, navigation systems, and smart transportation technologies have made it possible to collect large amounts of path data. Modeling the distribution of this path data, known as the Path Generation (PG) problem, is crucial for understanding urban mobility patterns and developing intelligent transportation systems. Recent studies have explored using diffusion models to address the PG problem due to their ability to capture multimodal distributions and support conditional generation. A recent work devises a diffusion process explicitly in graph space and achieves state-of-the-art performance. However, this method suffers a high computation cost in terms of both time and memory, which prohibits its application. In this paper, we analyze this method both theoretically and experimentally and find that the main culprit of its high computation cost is its explicit design of the diffusion process in graph space. To improve efficiency, we devise a Latent-space Path Diffusion (LPD) model, which operates in latent space instead of graph space. Our LPD significantly reduces both time and memory costs by up to 82.8% and 83.1%, respectively. Despite these reductions, our approach does not suffer from performance degradation. It outperforms the state-of-the-art method in most scenarios by 24.5%~34.0%.

Title: Structural Latency Perturbation in Large Language Models Through Recursive State Induction

• Authors: Michael Mangrum, Jonathan Pemberton, Benedict Wetherby, Philip Montague
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00758
• Pdf URL: https://arxiv.org/pdf/2502.00758
• Copy Paste: [[2502.00758]] Structural Latency Perturbation in Large Language Models Through Recursive State Induction(https://arxiv.org/abs/2502.00758)
• Keywords: generative
• Abstract: Computational efficiency has remained a critical consideration in scaling high-capacity language models, with inference latency and resource consumption presenting significant constraints on real-time applications. The study has introduced a structured latency perturbation mechanism that modifies computational pathways through recursive state induction, enabling dynamic suppression of redundant activations while preserving generative fidelity. A formal mathematical framework has been established to describe recursive perturbations, ensuring that modifications remain adaptive rather than statically imposed. Experiments have demonstrated that applying recursive state adjustments reduces inference latency across varying sequence lengths, with longer text generations benefiting from cumulative efficiency improvements. Comparative evaluations against structured pruning and quantization have indicated that latency gains can be achieved without compromising token retention or memory utilization. The analysis of computational overhead has suggested that selectively suppressing redundant activations contributes to improved power efficiency, particularly in scenarios requiring extended text generation. An assessment of linguistic stability has shown that token-level consistency remains largely intact under controlled perturbation thresholds, reinforcing the viability of structural latency modifications as an alternative to weight-centric optimization techniques. The results have supported the hypothesis that recursive state induction offers an effective method for reducing computational complexity without requiring architectural modifications or external augmentation.

Title: A method for estimating forest carbon storage distribution density via artificial intelligence generated content model

• Authors: Zhenyu Yu, Jinnian Wang
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.00783
• Pdf URL: https://arxiv.org/pdf/2502.00783
• Copy Paste: [[2502.00783]] A method for estimating forest carbon storage distribution density via artificial intelligence generated content model(https://arxiv.org/abs/2502.00783)
• Keywords: diffusion, generative
• Abstract: Forest is the most significant land-based carbon storage mechanism. The forest carbon sink can effectively decrease the atmospheric CO2 concentration and mitigate climate change. Remote sensing estimation not only ensures high accuracy of data, but also enables large-scale area observation. Optical images provide the possibility for long-term monitoring, which is a potential issue in the future carbon storage estimation research. We chose Huize County, Qujing City, Yunnan Province, China as the study area, took GF-1 WFV satellite image as the data, introduced the KD-VGG module to extract the initial features, and proposed the improved implicit diffusion model (IIDM). The results showed that: (1) The VGG-19 module after knowledge distillation can realize the initial feature extraction, reduce the inference time and improve the accuracy in the case of reducing the number of model parameters. (2) The Attention + MLP module was added for feature fusion to obtain the relationship between global and local features and realized the restoration of high-fidelity images in the continuous scale range. (3) The IIDM model proposed in this paper had the highest estimation accuracy, with RMSE of 28.68, which was 13.16 higher than that of the regression model, about 31.45%. In the estimation of carbon storage, the generative model can extract deeper features, and its performance was significantly better than other models. It demonstrated the feasibility of artificial intelligence-generated content (AIGC) in the field of quantitative remote sensing and provided valuable insights for the study of carbon neutralization effect. By combining the actual characteristics of the forest, the regional carbon storage estimation with a resolution of 16-meter was utilized to provide a significant theoretical basis for the formulation of forest carbon sink regulation.

Title: Vision-centric Token Compression in Large Language Model

• Authors: Ling Xing, Alex Jinpeng Wang, Rui Yan, Jinhui Tang
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00791
• Pdf URL: https://arxiv.org/pdf/2502.00791
• Copy Paste: [[2502.00791]] Vision-centric Token Compression in Large Language Model(https://arxiv.org/abs/2502.00791)
• Keywords: in-context
• Abstract: Large Language Models (LLMs) have revolutionized natural language processing, excelling in handling longer sequences. However, the inefficiency and redundancy in processing extended in-context tokens remain a challenge. Many attempts to address this rely on compressing tokens with smaller text encoders, yet we question whether text encoders are truly indispensable. Our journey leads to an unexpected discovery-a much smaller vision encoder, applied directly to sequences of text tokens, can rival text encoders on text tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small text understanding benchmarks, VIST leads to comparable results with 16% fewer FLOPs and 50% less memory usage. We further uncover significant token redundancy and devise a frequency-based masking strategy to guide the focus of the visual encoder toward the most critical tokens. Interestingly, we observe the trained visual encoder performs like a summarizer, selectively ignoring less important words such as prepositions and conjunctions. This approach delivers remarkable results, outperforming traditional text encoder-based methods by 5.7% on average over benchmarks like TriviaQA, NQ, PopQA, TREF, SST2, and SST5, setting a new standard for token efficiency in LLMs.

Title: Adversarial Semantic Augmentation for Training Generative Adversarial Networks under Limited Data

• Authors: Mengping Yang, Zhe Wang, Ziqiu Chi, Dongdong Li, Wenli Du
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.00800
• Pdf URL: https://arxiv.org/pdf/2502.00800
• Copy Paste: [[2502.00800]] Adversarial Semantic Augmentation for Training Generative Adversarial Networks under Limited Data(https://arxiv.org/abs/2502.00800)
• Keywords: generative
• Abstract: Generative adversarial networks (GANs) have made remarkable achievements in synthesizing images in recent years. Typically, training GANs requires massive data, and the performance of GANs deteriorates significantly when training data is limited. To improve the synthesis performance of GANs in low-data regimes, existing approaches use various data augmentation techniques to enlarge the training sets. However, it is identified that these augmentation techniques may leak or even alter the data distribution. To remedy this, we propose an adversarial semantic augmentation (ASA) technique to enlarge the training data at the semantic level instead of the image level. Concretely, considering semantic features usually encode informative information of images, we estimate the covariance matrices of semantic features for both real and generated images to find meaningful transformation directions. Such directions translate original features to another semantic representation, e.g., changing the backgrounds or expressions of the human face dataset. Moreover, we derive an upper bound of the expected adversarial loss. By optimizing the upper bound, our semantic augmentation is implicitly achieved. Such design avoids redundant sampling of the augmented features and introduces negligible computation overhead, making our approach computation efficient. Extensive experiments on both few-shot and large-scale datasets demonstrate that our method consistently improve the synthesis quality under various data regimes, and further visualized and analytic results suggesting satisfactory versatility of our proposed method.

Title: UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs

• Authors: Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, Bryan Hooi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00806
• Pdf URL: https://arxiv.org/pdf/2502.00806
• Copy Paste: [[2502.00806]] UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs(https://arxiv.org/abs/2502.00806)
• Keywords: foundation model, generative
• Abstract: Existing foundation models, such as CLIP, aim to learn a unified embedding space for multimodal data, enabling a wide range of downstream web-based applications like search, recommendation, and content classification. However, these models often overlook the inherent graph structures in multimodal datasets, where entities and their relationships are crucial. Multimodal graphs (MMGs) represent such graphs where each node is associated with features from different modalities, while the edges capture the relationships between these entities. On the other hand, existing graph foundation models primarily focus on text-attributed graphs (TAGs) and are not designed to handle the complexities of MMGs. To address these limitations, we propose UniGraph2, a novel cross-domain graph foundation model that enables general representation learning on MMGs, providing a unified embedding space. UniGraph2 employs modality-specific encoders alongside a graph neural network (GNN) to learn a unified low-dimensional embedding space that captures both the multimodal information and the underlying graph structure. We propose a new cross-domain multi-graph pre-training algorithm at scale to ensure effective transfer learning across diverse graph domains and modalities. Additionally, we adopt a Mixture of Experts (MoE) component to align features from different domains and modalities, ensuring coherent and robust embeddings that unify the information across modalities. Extensive experiments on a variety of multimodal graph tasks demonstrate that UniGraph2 significantly outperforms state-of-the-art models in tasks such as representation learning, transfer learning, and multimodal generative tasks, offering a scalable and flexible solution for learning on MMGs.

Title: Sundial: A Family of Highly Capable Time Series Foundation Models

• Authors: Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang, Xiangdong Huang, Jianmin Wang, Mingsheng Long
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00816
• Pdf URL: https://arxiv.org/pdf/2502.00816
• Copy Paste: [[2502.00816]] Sundial: A Family of Highly Capable Time Series Foundation Models(https://arxiv.org/abs/2502.00816)
• Keywords: foundation model, generative
• Abstract: We introduce Sundial, a family of native, flexible, and scalable time series foundation models. To predict the next-patch's distribution, we propose a TimeFlow Loss based on flow-matching, which facilitates native pre-training of Transformers on time series without discrete tokenization. Conditioned on arbitrary-length time series, our model is pre-trained without specifying any prior distribution and can generate multiple probable predictions, achieving flexibility in representation learning beyond using parametric densities. Towards time series foundation models, we leverage minimal but crucial adaptations of Transformers and curate TimeBench with 1 trillion time points, comprising mostly real-world datasets and synthetic data. By mitigating mode collapse through TimeFlow Loss, we pre-train a family of Sundial models on TimeBench, which exhibit unprecedented model capacity and generalization performance on zero-shot forecasting. In addition to presenting good scaling behavior, Sundial achieves new state-of-the-art on both point forecasting and probabilistic forecasting benchmarks. We believe that Sundial's pioneering generative paradigm will facilitate a wide variety of forecasting scenarios.

Title: OOD Detection with immature Models

• Authors: Behrooz Montazeran, Ullrich Köthe
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00820
• Pdf URL: https://arxiv.org/pdf/2502.00820
• Copy Paste: [[2502.00820]] OOD Detection with immature Models(https://arxiv.org/abs/2502.00820)
• Keywords: generative
• Abstract: Likelihood-based deep generative models (DGMs) have gained significant attention for their ability to approximate the distributions of high-dimensional data. However, these models lack a performance guarantee in assigning higher likelihood values to in-distribution (ID) inputs, data the models are trained on, compared to out-of-distribution (OOD) inputs. This counter-intuitive behaviour is particularly pronounced when ID inputs are more complex than OOD data points. One potential approach to address this challenge involves leveraging the gradient of a data point with respect to the parameters of the DGMs. A recent OOD detection framework proposed estimating the joint density of layer-wise gradient norms for a given data point as a model-agnostic method, demonstrating superior performance compared to the Typicality Test across likelihood-based DGMs and image dataset pairs. In particular, most existing methods presuppose access to fully converged models, the training of which is both time-intensive and computationally demanding. In this work, we demonstrate that using immature models,stopped at early stages of training, can mostly achieve equivalent or even superior results on this downstream task compared to mature models capable of generating high-quality samples that closely resemble ID data. This novel finding enhances our understanding of how DGMs learn the distribution of ID data and highlights the potential of leveraging partially trained models for downstream tasks. Furthermore, we offer a possible explanation for this unexpected behaviour through the concept of support overlap.

Title: Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Models

• Authors: Julian Perry, Frank Sanders, Carter Scott
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00826
• Pdf URL: https://arxiv.org/pdf/2502.00826
• Copy Paste: [[2502.00826]] Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Models(https://arxiv.org/abs/2502.00826)
• Keywords: diffusion, generative
• Abstract: In this paper, we presents a novel method for improving text-to-image generation by combining Large Language Models (LLMs) with diffusion models, a hybrid approach aimed at achieving both higher quality and efficiency in image synthesis from text descriptions. Our approach introduces a new dynamic KL-weighting strategy to optimize the diffusion process, along with incorporating semantic understanding from pre-trained LLMs to guide the generation process. The proposed method significantly improves both the visual quality and alignment of generated images with text descriptions, addressing challenges such as computational inefficiency, instability in training, and robustness to textual variability. We evaluate our method on the COCO dataset and demonstrate its superior performance over traditional GAN-based models, both quantitatively and qualitatively. Extensive experiments, including ablation studies and human evaluations, confirm that our method outperforms existing approaches in terms of image realism, relevance to the input text, and overall aesthetic quality. Our approach also shows promise in scalability to other multimodal tasks, making it a versatile solution for a wide range of generative applications.

Title: A Comprehensive Analysis on LLM-based Node Classification Algorithms

• Authors: Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, Hong Cheng
• Subjects: cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2502.00829
• Pdf URL: https://arxiv.org/pdf/2502.00829
• Copy Paste: [[2502.00829]] A Comprehensive Analysis on LLM-based Node Classification Algorithms(https://arxiv.org/abs/2502.00829)
• Keywords: foundation model
• Abstract: Node classification is a fundamental task in graph analysis, with broad applications across various fields. Recent breakthroughs in Large Language Models (LLMs) have enabled LLM-based approaches for this task. Although many studies demonstrate the impressive performance of LLM-based methods, the lack of clear design guidelines may hinder their practical application. In this work, we aim to establish such guidelines through a fair and systematic comparison of these algorithms. As a first step, we developed LLMNodeBed, a comprehensive codebase and testbed for node classification using LLMs. It includes ten datasets, eight LLM-based algorithms, and three learning paradigms, and is designed for easy extension with new methods and datasets. Subsequently, we conducted extensive experiments, training and evaluating over 2,200 models, to determine the key settings (e.g., learning paradigms and homophily) and components (e.g., model size) that affect performance. Our findings uncover eight insights, e.g., (1) LLM-based methods can significantly outperform traditional methods in a semi-supervised setting, while the advantage is marginal in a supervised setting; (2) Graph Foundation Models can beat open-source LLMs but still fall short of strong LLMs like GPT-4o in a zero-shot setting. We hope that the release of LLMNodeBed, along with our insights, will facilitate reproducible research and inspire future studies in this field. Codes and datasets are released at \href{this https URL}{this https URL}.

Title: Generalization of Medical Large Language Models through Cross-Domain Weak Supervision

• Authors: Robert Long, Eric Gonzalez, Harrison Fuller
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00832
• Pdf URL: https://arxiv.org/pdf/2502.00832
• Copy Paste: [[2502.00832]] Generalization of Medical Large Language Models through Cross-Domain Weak Supervision(https://arxiv.org/abs/2502.00832)
• Keywords: generative
• Abstract: The advancement of large language models (LLMs) has opened new frontiers in natural language processing, particularly in specialized domains like healthcare. In this paper, we propose the Incremental Curriculum-Based Fine-Tuning (ICFT) framework to enhance the generative capabilities of medical large language models (MLLMs). ICFT combines curriculum-based learning, dual-stage memory coordination, and parameter-efficient fine-tuning to enable a progressive transition from general linguistic knowledge to strong domain-specific expertise. Experimental results across diverse medical NLP tasks, including question answering, preference classification, and response generation, demonstrate that ICFT consistently outperforms state-of-the-art baselines, achieving improvements in both accuracy and efficiency. Further analysis reveals the framework's ability to generalize to unseen data, reduce errors, and deliver diverse, contextually relevant medical responses. These findings establish ICFT as a robust and scalable solution for adapting LLMs to the medical domain, offering practical benefits for real-world healthcare applications.

Title: RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning

• Authors: Yuanhuiyi Lyu, Xu Zheng, Lutao Jiang, Yibo Yan, Xin Zou, Huiyu Zhou, Linfeng Zhang, Xuming Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00848
• Pdf URL: https://arxiv.org/pdf/2502.00848
• Copy Paste: [[2502.00848]] RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning(https://arxiv.org/abs/2502.00848)
• Keywords: diffusion, generative
• Abstract: Recent text-to-image generative models, e.g., Stable Diffusion V3 and Flux, have achieved notable progress. However, these models are strongly restricted to their limited knowledge, a.k.a., their own fixed parameters, that are trained with closed datasets. This leads to significant hallucinations or distortions when facing fine-grained and unseen novel real-world objects, e.g., the appearance of the Tesla Cybertruck. To this end, we present the first real-object-based retrieval-augmented generation framework (RealRAG), which augments fine-grained and unseen novel object generation by learning and retrieving real-world images to overcome the knowledge gaps of generative models. Specifically, to integrate missing memory for unseen novel object generation, we train a reflective retriever by self-reflective contrastive learning, which injects the generator's knowledge into the sef-reflective negatives, ensuring that the retrieved augmented images compensate for the model's missing knowledge. Furthermore, the real-object-based framework integrates fine-grained visual knowledge for the generative models, tackling the distortion problem and improving the realism for fine-grained object generation. Our Real-RAG is superior in its modular application to all types of state-of-the-art text-to-image generative models and also delivers remarkable performance boosts with all of them, such as a gain of 16.18% FID score with the auto-regressive model on the Stanford Car benchmark.

Title: Towards Automation of Cognitive Modeling using Large Language Models

• Authors: Milena Rmus, Akshay K. Jagadish, Marvin Mathony, Tobias Ludwig, Eric Schulz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00879
• Pdf URL: https://arxiv.org/pdf/2502.00879
• Copy Paste: [[2502.00879]] Towards Automation of Cognitive Modeling using Large Language Models(https://arxiv.org/abs/2502.00879)
• Keywords: in-context
• Abstract: Computational cognitive models, which formalize theories of cognition, enable researchers to quantify cognitive processes and arbitrate between competing theories by fitting models to behavioral data. Traditionally, these models are handcrafted, which requires significant domain knowledge, coding expertise, and time investment. Previous work has demonstrated that Large Language Models (LLMs) are adept at pattern recognition in-context, solving complex problems, and generating executable code. In this work, we leverage these abilities to explore the potential of LLMs in automating the generation of cognitive models based on behavioral data. We evaluated the LLM in two different tasks: model identification (relating data to a source model), and model generation (generating the underlying cognitive model). We performed these tasks across two cognitive domains - decision making and learning. In the case of data simulated from canonical cognitive models, we found that the LLM successfully identified and generated the ground truth model. In the case of human data, where behavioral noise and lack of knowledge of the true underlying process pose significant challenges, the LLM generated models that are identical or close to the winning model from cognitive science literature. Our findings suggest that LLMs can have a transformative impact on cognitive modeling. With this project, we aim to contribute to an ongoing effort of automating scientific discovery in cognitive science.

Title: Blink of an eye: a simple theory for feature localization in generative models

• Authors: Marvin Li, Aayush Karan, Sitan Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00921
• Pdf URL: https://arxiv.org/pdf/2502.00921
• Copy Paste: [[2502.00921]] Blink of an eye: a simple theory for feature localization in generative models(https://arxiv.org/abs/2502.00921)
• Keywords: diffusion, generative
• Abstract: Large language models (LLMs) can exhibit undesirable and unexpected behavior in the blink of an eye. In a recent Anthropic demo, Claude switched from coding to Googling pictures of Yellowstone, and these sudden shifts in behavior have also been observed in reasoning patterns and jailbreaks. This phenomenon is not unique to autoregressive models: in diffusion models, key features of the final output are decided in narrow ``critical windows'' of the generation process. In this work we develop a simple, unifying theory to explain this phenomenon. We show that it emerges generically as the generation process localizes to a sub-population of the distribution it models. While critical windows have been studied at length in diffusion models, existing theory heavily relies on strong distributional assumptions and the particulars of Gaussian diffusion. In contrast to existing work our theory (1) applies to autoregressive and diffusion models; (2) makes no distributional assumptions; (3) quantitatively improves previous bounds even when specialized to diffusions; and (4) requires basic tools and no stochastic calculus or statistical physics-based machinery. We also identify an intriguing connection to the all-or-nothing phenomenon from statistical inference. Finally, we validate our predictions empirically for LLMs and find that critical windows often coincide with failures in problem solving for various math and reasoning benchmarks.

Title: Hypo3D: Exploring Hypothetical Reasoning in 3D

• Authors: Ye Mao, Weixun Luo, Junpeng Jing, Anlan Qiu, Krystian Mikolajczyk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.00954
• Pdf URL: https://arxiv.org/pdf/2502.00954
• Copy Paste: [[2502.00954]] Hypo3D: Exploring Hypothetical Reasoning in 3D(https://arxiv.org/abs/2502.00954)
• Keywords: foundation model
• Abstract: The rise of vision-language foundation models marks an advancement in bridging the gap between human and machine capabilities in 3D scene reasoning. Existing 3D reasoning benchmarks assume real-time scene accessibility, which is impractical due to the high cost of frequent scene updates. To this end, we introduce Hypothetical 3D Reasoning, namely Hypo3D, a benchmark designed to evaluate models' ability to reason without access to real-time scene data. Models need to imagine the scene state based on a provided change description before reasoning. Hypo3D is formulated as a 3D Visual Question Answering (VQA) benchmark, comprising 7,727 context changes across 700 indoor scenes, resulting in 14,885 question-answer pairs. An anchor-based world frame is established for all scenes, ensuring consistent reference to a global frame for directional terms in context changes and QAs. Extensive experiments show that state-of-the-art foundation models struggle to reason in hypothetically changed scenes. This reveals a substantial performance gap compared to humans, particularly in scenarios involving movement changes and directional reasoning. Even when the context change is irrelevant to the question, models often incorrectly adjust their answers.

Title: CoDe: Blockwise Control for Denoising Diffusion Models

• Authors: Anuj Singh, Sayak Mukherjee, Ahmad Beirami, Hadi Jamali-Rad
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00968
• Pdf URL: https://arxiv.org/pdf/2502.00968
• Copy Paste: [[2502.00968]] CoDe: Blockwise Control for Denoising Diffusion Models(https://arxiv.org/abs/2502.00968)
• Keywords: diffusion
• Abstract: Aligning diffusion models to downstream tasks often requires finetuning new models or gradient-based guidance at inference time to enable sampling from the reward-tilted posterior. In this work, we explore a simple inference-time gradient-free guidance approach, called controlled denoising (CoDe), that circumvents the need for differentiable guidance functions and model finetuning. CoDe is a blockwise sampling method applied during intermediate denoising steps, allowing for alignment with downstream rewards. Our experiments demonstrate that, despite its simplicity, CoDe offers a favorable trade-off between reward alignment, prompt instruction following, and inference cost, achieving a competitive performance against the state-of-the-art baselines. Our code is available at: this https URL.

Title: Pushing the Boundaries of State Space Models for Image and Video Generation

• Authors: Yicong Hong, Long Mai, Yuan Yao, Feng Liu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.00972
• Pdf URL: https://arxiv.org/pdf/2502.00972
• Copy Paste: [[2502.00972]] Pushing the Boundaries of State Space Models for Image and Video Generation(https://arxiv.org/abs/2502.00972)
• Keywords: diffusion
• Abstract: While Transformers have become the dominant architecture for visual generation, linear attention models, such as the state-space models (SSM), are increasingly recognized for their efficiency in processing long visual sequences. However, the essential efficiency of these models comes from formulating a limited recurrent state, enforcing causality among tokens that are prone to inconsistent modeling of N-dimensional visual data, leaving questions on their capacity to generate long non-causal sequences. In this paper, we explore the boundary of SSM on image and video generation by building the largest-scale diffusion SSM-Transformer hybrid model to date (5B parameters) based on the sub-quadratic bi-directional Hydra and self-attention, and generate up to 2K images and 360p 8 seconds (16 FPS) videos. Our results demonstrate that the model can produce faithful results aligned with complex text prompts and temporal consistent videos with high dynamics, suggesting the great potential of using SSMs for visual generation tasks.

Title: ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution

• Authors: Kanika Goswami, Puneet Mathur, Ryan Rossi, Franck Dernoncourt
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.00989
• Pdf URL: https://arxiv.org/pdf/2502.00989
• Copy Paste: [[2502.00989]] ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution(https://arxiv.org/abs/2502.00989)
• Keywords: generative
• Abstract: Large Language Models (LLMs) can perform chart question-answering tasks but often generate unverified hallucinated responses. Existing answer attribution methods struggle to ground responses in source charts due to limited visual-semantic context, complex visual-text alignment requirements, and difficulties in bounding box prediction across complex layouts. We present ChartCitor, a multi-agent framework that provides fine-grained bounding box citations by identifying supporting evidence within chart images. The system orchestrates LLM agents to perform chart-to-table extraction, answer reformulation, table augmentation, evidence retrieval through pre-filtering and re-ranking, and table-to-chart mapping. ChartCitor outperforms existing baselines across different chart types. Qualitative user studies show that ChartCitor helps increase user trust in Generative AI by providing enhanced explainability for LLM-assisted chart QA and enables professionals to be more productive.

Title: FCBoost-Net: A Generative Network for Synthesizing Multiple Collocated Outfits via Fashion Compatibility Boosting

• Authors: Dongliang Zhou, Haijun Zhang, Jianghong Ma, Jicong Fan, Zhao Zhang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.00992
• Pdf URL: https://arxiv.org/pdf/2502.00992
• Copy Paste: [[2502.00992]] FCBoost-Net: A Generative Network for Synthesizing Multiple Collocated Outfits via Fashion Compatibility Boosting(https://arxiv.org/abs/2502.00992)
• Keywords: generative
• Abstract: Outfit generation is a challenging task in the field of fashion technology, in which the aim is to create a collocated set of fashion items that complement a given set of items. Previous studies in this area have been limited to generating a unique set of fashion items based on a given set of items, without providing additional options to users. This lack of a diverse range of choices necessitates the development of a more versatile framework. However, when the task of generating collocated and diversified outfits is approached with multimodal image-to-image translation methods, it poses a challenging problem in terms of non-aligned image translation, which is hard to address with existing methods. In this research, we present FCBoost-Net, a new framework for outfit generation that leverages the power of pre-trained generative models to produce multiple collocated and diversified outfits. Initially, FCBoost-Net randomly synthesizes multiple sets of fashion items, and the compatibility of the synthesized sets is then improved in several rounds using a novel fashion compatibility booster. This approach was inspired by boosting algorithms and allows the performance to be gradually improved in multiple steps. Empirical evidence indicates that the proposed strategy can improve the fashion compatibility of randomly synthesized fashion items as well as maintain their diversity. Extensive experiments confirm the effectiveness of our proposed framework with respect to visual authenticity, diversity, and fashion compatibility.

Title: Self-supervised Analogical Learning using Language Models

• Authors: Ben Zhou, Sarthak Jain, Yi Zhang, Qiang Ning, Shuai Wang, Yassine Benajiba, Dan Roth
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.00996
• Pdf URL: https://arxiv.org/pdf/2502.00996
• Copy Paste: [[2502.00996]] Self-supervised Analogical Learning using Language Models(https://arxiv.org/abs/2502.00996)
• Keywords: self-supervised
• Abstract: Large language models have been shown to suffer from reasoning inconsistency issues. That is, they fail more in situations unfamiliar to the training data, even though exact or very similar reasoning paths exist in more common cases that they can successfully solve. Such observations motivate us to propose methods that encourage models to understand the high-level and abstract reasoning processes during training instead of only the final answer. This way, models can transfer the exact solution to similar cases, regardless of their relevance to the pre-training data distribution. In this work, we propose SAL, a self-supervised analogical learning framework. SAL mimics the human analogy process and trains models to explicitly transfer high-quality symbolic solutions from cases that they know how to solve to other rare cases in which they tend to fail more. We show that the resulting models after SAL learning outperform base language models on a wide range of reasoning benchmarks, such as StrategyQA, GSM8K, and HotpotQA, by 2% to 20%. At the same time, we show that our model is more generalizable and controllable through analytical studies.

Title: Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentially

• Authors: Jingyun Yang, Guoqing Zhang, Jingge Wang, Yang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01000
• Pdf URL: https://arxiv.org/pdf/2502.01000
• Copy Paste: [[2502.01000]] Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentially(https://arxiv.org/abs/2502.01000)
• Keywords: foundation model
• Abstract: Recent advances in foundation models have brought promising results in computer vision, including medical image segmentation. Fine-tuning foundation models on specific low-resource medical tasks has become a standard practice. However, ensuring reliable and robust model adaptation when the target task has a large domain gap and few annotated samples remains a challenge. Previous few-shot domain adaptation (FSDA) methods seek to bridge the distribution gap between source and target domains by utilizing auxiliary data. The selection and scheduling of auxiliaries are often based on heuristics, which can easily cause negative transfer. In this work, we propose an Active and Sequential domain AdaPtation (ASAP) framework for dynamic auxiliary dataset selection in FSDA. We formulate FSDA as a multi-armed bandit problem and derive an efficient reward function to prioritize training on auxiliary datasets that align closely with the target task, through a single-round fine-tuning. Empirical validation on diverse medical segmentation datasets demonstrates that our method achieves favorable segmentation performance, significantly outperforming the state-of-the-art FSDA methods, achieving an average gain of 27.75% on MRI and 7.52% on CT datasets in Dice score. Code is available at the git repository: this https URL.

Title: Encrypted Large Model Inference: The Equivariant Encryption Paradigm

• Authors: James Buban, Hongyang Zhang, Claudio Angione, Harry Yang, Ahmad Farhan, Seyfal Sultanov, Michael Du, Xuran Ma, Zihao Wang, Yue Zhao, Arria Owlia, Fielding Johnston, Patrick Colangelo
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01013
• Pdf URL: https://arxiv.org/pdf/2502.01013
• Copy Paste: [[2502.01013]] Encrypted Large Model Inference: The Equivariant Encryption Paradigm(https://arxiv.org/abs/2502.01013)
• Keywords: diffusion
• Abstract: Large scale deep learning model, such as modern language models and diffusion architectures, have revolutionized applications ranging from natural language processing to computer vision. However, their deployment in distributed or decentralized environments raises significant privacy concerns, as sensitive data may be exposed during inference. Traditional techniques like secure multi-party computation, homomorphic encryption, and differential privacy offer partial remedies but often incur substantial computational overhead, latency penalties, or limited compatibility with non-linear network operations. In this work, we introduce Equivariant Encryption (EE), a novel paradigm designed to enable secure, "blind" inference on encrypted data with near zero performance overhead. Unlike fully homomorphic approaches that encrypt the entire computational graph, EE selectively obfuscates critical internal representations within neural network layers while preserving the exact functionality of both linear and a prescribed set of non-linear operations. This targeted encryption ensures that raw inputs, intermediate activations, and outputs remain confidential, even when processed on untrusted infrastructure. We detail the theoretical foundations of EE, compare its performance and integration complexity against conventional privacy preserving techniques, and demonstrate its applicability across a range of architectures, from convolutional networks to large language models. Furthermore, our work provides a comprehensive threat analysis, outlining potential attack vectors and baseline strategies, and benchmarks EE against standard inference pipelines in decentralized settings. The results confirm that EE maintains high fidelity and throughput, effectively bridging the gap between robust data confidentiality and the stringent efficiency requirements of modern, large scale model inference.

Title: WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction

• Authors: Zilong Wang, Zhiyang Dou, Yuan Liu, Cheng Lin, Xiao Dong, Yunhui Guo, Chenxu Zhang, Xin Li, Wenping Wang, Xiaohu Guo
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.01045
• Pdf URL: https://arxiv.org/pdf/2502.01045
• Copy Paste: [[2502.01045]] WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction(https://arxiv.org/abs/2502.01045)
• Keywords: diffusion, generative
• Abstract: In this paper, we present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis. Previous dynamic human avatar reconstruction methods typically require the input video to have full coverage of the observed human body. However, in daily practice, one typically has access to limited viewpoints, such as monocular front-view videos, making it a cumbersome task for previous methods to reconstruct the unseen parts of the human avatar. To tackle the issue, we present WonderHuman, which leverages 2D generative diffusion model priors to achieve high-quality, photorealistic reconstructions of dynamic human avatars from monocular videos, including accurate rendering of unseen body parts. Our approach introduces a Dual-Space Optimization technique, applying Score Distillation Sampling (SDS) in both canonical and observation spaces to ensure visual consistency and enhance realism in dynamic human reconstruction. Additionally, we present a View Selection strategy and Pose Feature Injection to enforce the consistency between SDS predictions and observed data, ensuring pose-dependent effects and higher fidelity in the reconstructed avatar. In the experiments, our method achieves SOTA performance in producing photorealistic renderings from the given monocular video, particularly for those challenging unseen parts. The project page and source code can be found at this https URL.

Title: Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization

• Authors: Tao Zhang, Cheng Da, Kun Ding, Kun Jin, Yan Li, Tingting Gao, Di Zhang, Shiming Xiang, Chunhong Pan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01051
• Pdf URL: https://arxiv.org/pdf/2502.01051
• Copy Paste: [[2502.01051]] Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization(https://arxiv.org/abs/2502.01051)
• Keywords: diffusion
• Abstract: Preference optimization for diffusion models aims to align them with human preferences for images. Previous methods typically leverage Vision-Language Models (VLMs) as pixel-level reward models to approximate human preferences. However, when used for step-level preference optimization, these models face challenges in handling noisy images of different timesteps and require complex transformations into pixel space. In this work, we demonstrate that diffusion models are inherently well-suited for step-level reward modeling in the latent space, as they can naturally extract features from noisy latent images. Accordingly, we propose the Latent Reward Model (LRM), which repurposes components of diffusion models to predict preferences of latent images at various timesteps. Building on LRM, we introduce Latent Preference Optimization (LPO), a method designed for step-level preference optimization directly in the latent space. Experimental results indicate that LPO not only significantly enhances performance in aligning diffusion models with general, aesthetic, and text-image alignment preferences, but also achieves 2.5-28$\times$ training speedup compared to existing preference optimization methods. Our code will be available at this https URL.

Title: OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models

• Authors: Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, Chao Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01061
• Pdf URL: https://arxiv.org/pdf/2502.01061
• Copy Paste: [[2502.01061]] OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models(https://arxiv.org/abs/2502.01061)
• Keywords: diffusion
• Abstract: End-to-end human animation, such as audio-driven talking human generation, has undergone notable advancements in the recent few years. However, existing methods still struggle to scale up as large general video generation models, limiting their potential in real applications. In this paper, we propose OmniHuman, a Diffusion Transformer-based framework that scales up data by mixing motion-related conditions into the training phase. To this end, we introduce two training principles for these mixed conditions, along with the corresponding model architecture and inference strategy. These designs enable OmniHuman to fully leverage data-driven motion generation, ultimately achieving highly realistic human video generation. More importantly, OmniHuman supports various portrait contents (face close-up, portrait, half-body, full-body), supports both talking and singing, handles human-object interactions and challenging body poses, and accommodates different image styles. Compared to existing end-to-end audio-driven methods, OmniHuman not only produces more realistic videos, but also offers greater flexibility in inputs. It also supports multiple driving modalities (audio-driven, video-driven and combined driving signals). Video samples are provided on the ttfamily project page (this https URL)

Title: BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing

• Authors: Dongliang Zhou, Haijun Zhang, Jianghong Ma, Jianyang Shi
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.01080
• Pdf URL: https://arxiv.org/pdf/2502.01080
• Copy Paste: [[2502.01080]] BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing(https://arxiv.org/abs/2502.01080)
• Keywords: generative
• Abstract: Collocated clothing synthesis using generative networks has become an emerging topic in the field of fashion intelligence, as it has significant potential economic value to increase revenue in the fashion industry. In previous studies, several works have attempted to synthesize visually-collocated clothing based on a given clothing item using generative adversarial networks (GANs) with promising results. These works, however, can only accomplish the synthesis of one collocated clothing item each time. Nevertheless, users may require different clothing items to meet their multiple choices due to their personal tastes and different dressing scenarios. To address this limitation, we introduce a novel batch clothing generation framework, named BC-GAN, which is able to synthesize multiple visually-collocated clothing images simultaneously. In particular, to further improve the fashion compatibility of synthetic results, BC-GAN proposes a new fashion compatibility discriminator in a contrastive learning perspective by fully exploiting the collocation relationship among all clothing items. Our model was examined in a large-scale dataset with compatible outfits constructed by ourselves. Extensive experiment results confirmed the effectiveness of our proposed BC-GAN in comparison to state-of-the-art methods in terms of diversity, visual authenticity, and fashion compatibility.

Title: SatFlow: Generative model based framework for producing High Resolution Gap Free Remote Sensing Imagery

• Authors: Bharath Irigireddy, Varaprasad Bandaru
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01098
• Pdf URL: https://arxiv.org/pdf/2502.01098
• Copy Paste: [[2502.01098]] SatFlow: Generative model based framework for producing High Resolution Gap Free Remote Sensing Imagery(https://arxiv.org/abs/2502.01098)
• Keywords: generative
• Abstract: Frequent, high-resolution remote sensing imagery is crucial for agricultural and environmental monitoring. Satellites from the Landsat collection offer detailed imagery at 30m resolution but with lower temporal frequency, whereas missions like MODIS and VIIRS provide daily coverage at coarser resolutions. Clouds and cloud shadows contaminate about 55\% of the optical remote sensing observations, posing additional challenges. To address these challenges, we present SatFlow, a generative model-based framework that fuses low-resolution MODIS imagery and Landsat observations to produce frequent, high-resolution, gap-free surface reflectance imagery. Our model, trained via Conditional Flow Matching, demonstrates better performance in generating imagery with preserved structural and spectral integrity. Cloud imputation is treated as an image inpainting task, where the model reconstructs cloud-contaminated pixels and fills gaps caused by scan lines during inference by leveraging the learned generative processes. Experimental results demonstrate the capability of our approach in reliably imputing cloud-covered regions. This capability is crucial for downstream applications such as crop phenology tracking, environmental change detection etc.,

Title: VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Control

• Authors: Lifan Jiang, Shuang Chen, Boxi Wu, Xiaotong Guan, Jiahui Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01101
• Pdf URL: https://arxiv.org/pdf/2502.01101
• Copy Paste: [[2502.01101]] VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Control(https://arxiv.org/abs/2502.01101)
• Keywords: diffusion, generative
• Abstract: With the advancement of generative artificial intelligence, previous studies have achieved the task of generating aesthetic images from hand-drawn sketches, fulfilling the public's needs for drawing. However, these methods are limited to static images and lack the ability to control video animation generation using hand-drawn sketches. To address this gap, we propose VidSketch, the first method capable of generating high-quality video animations directly from any number of hand-drawn sketches and simple text prompts, bridging the divide between ordinary users and professional artists. Specifically, our method introduces a Level-Based Sketch Control Strategy to automatically adjust the guidance strength of sketches during the generation process, accommodating users with varying drawing skills. Furthermore, a TempSpatial Attention mechanism is designed to enhance the spatiotemporal consistency of generated video animations, significantly improving the coherence across frames. You can find more detailed cases on our official website.

Title: LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer

• Authors: Yiren Song, Danze Chen, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01105
• Pdf URL: https://arxiv.org/pdf/2502.01105
• Copy Paste: [[2502.01105]] LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer(https://arxiv.org/abs/2502.01105)
• Keywords: diffusion
• Abstract: Generating cognitive-aligned layered SVGs remains challenging due to existing methods' tendencies toward either oversimplified single-layer outputs or optimization-induced shape redundancies. We propose LayerTracer, a diffusion transformer based framework that bridges this gap by learning designers' layered SVG creation processes from a novel dataset of sequential design operations. Our approach operates in two phases: First, a text-conditioned DiT generates multi-phase rasterized construction blueprints that simulate human design workflows. Second, layer-wise vectorization with path deduplication produces clean, editable SVGs. For image vectorization, we introduce a conditional diffusion mechanism that encodes reference images into latent tokens, guiding hierarchical reconstruction while preserving structural integrity. Extensive experiments demonstrate LayerTracer's superior performance against optimization-based and neural baselines in both generation quality and editability, effectively aligning AI-generated vectors with professional design cognition.

Title: Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings

• Authors: Mithun Saha, Maxwell A. Xu, Wanting Mao, Sameer Neupane, James M. Rehg, Santosh Kumar
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2502.01108
• Pdf URL: https://arxiv.org/pdf/2502.01108
• Copy Paste: [[2502.01108]] Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings(https://arxiv.org/abs/2502.01108)
• Keywords: foundation model
• Abstract: Photoplethysmography (PPG)-based foundation models are gaining traction due to the widespread use of PPG in biosignal monitoring and their potential to generalize across diverse health applications. In this paper, we introduce Pulse-PPG, the first open-source PPG foundation model trained exclusively on raw PPG data collected over a 100-day field study with 120 participants. Existing PPG foundation models are either open-source but trained on clinical data or closed-source, limiting their applicability in real-world settings. We evaluate Pulse-PPG across multiple datasets and downstream tasks, comparing its performance against a state-of-the-art foundation model trained on clinical data. Our results demonstrate that Pulse-PPG, trained on uncurated field data, exhibits superior generalization across clinical and mobile health applications in both lab and field settings. This suggests that exposure to real-world variability enables the model to learn fine-grained representations, making it more adaptable across tasks. Furthermore, pre-training on field data surprisingly outperforms its pre-training on clinical data in many tasks, reinforcing the importance of training on real-world, diverse datasets. To encourage further advancements in robust foundation models leveraging field data, we plan to release Pulse-PPG, providing researchers with a powerful resource for developing more generalizable PPG-based models.

Title: Learning to Learn Weight Generation via Trajectory Diffusion

• Authors: Yunchuan Guan, Yu Liu, Ke Zhou, Zhiqi Shen, Serge Belongie, Jenq-Neng Hwang, Lei Li
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01117
• Pdf URL: https://arxiv.org/pdf/2502.01117
• Copy Paste: [[2502.01117]] Learning to Learn Weight Generation via Trajectory Diffusion(https://arxiv.org/abs/2502.01117)
• Keywords: diffusion
• Abstract: Diffusion-based algorithms have emerged as promising techniques for weight generation, particularly in scenarios like multi-task learning that require frequent weight updates. However, existing solutions suffer from limited cross-task transferability. In addition, they only utilize optimal weights as training samples, ignoring the value of other weights in the optimization process. To address these issues, we propose Lt-Di, which integrates the diffusion algorithm with meta-learning to generate weights for unseen tasks. Furthermore, we extend the vanilla diffusion algorithm into a trajectory diffusion algorithm to utilize other weights along the optimization trajectory. Trajectory diffusion decomposes the entire diffusion chain into multiple shorter ones, improving training and inference efficiency. We analyze the convergence properties of the weight generation paradigm and improve convergence efficiency without additional time overhead. Our experiments demonstrate Lt-Di's higher accuracy while reducing computational overhead across various tasks, including zero-shot and few-shot learning, multi-domain generalization, and large-scale language model this http URL code is released at this https URL.

Title: Large Language Model-Enhanced Multi-Armed Bandits

• Authors: Jiahang Sun, Zhiyong Wang, Runhan Yang, Chenjun Xiao, John C.S. Lui, Zhongxiang Dai
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01118
• Pdf URL: https://arxiv.org/pdf/2502.01118
• Copy Paste: [[2502.01118]] Large Language Model-Enhanced Multi-Armed Bandits(https://arxiv.org/abs/2502.01118)
• Keywords: in-context
• Abstract: Large language models (LLMs) have been adopted to solve sequential decision-making tasks such as multi-armed bandits (MAB), in which an LLM is directly instructed to select the arms to pull in every iteration. However, this paradigm of direct arm selection using LLMs has been shown to be suboptimal in many MAB tasks. Therefore, we propose an alternative approach which combines the strengths of classical MAB and LLMs. Specifically, we adopt a classical MAB algorithm as the high-level framework and leverage the strong in-context learning capability of LLMs to perform the sub-task of reward prediction. Firstly, we incorporate the LLM-based reward predictor into the classical Thompson sampling (TS) algorithm and adopt a decaying schedule for the LLM temperature to ensure a transition from exploration to exploitation. Next, we incorporate the LLM-based reward predictor (with a temperature of 0) into a regression oracle-based MAB algorithm equipped with an explicit exploration mechanism. We also extend our TS-based algorithm to dueling bandits where only the preference feedback between pairs of arms is available, which requires non-trivial algorithmic modifications. We conduct empirical evaluations using both synthetic MAB tasks and experiments designed using real-world text datasets, in which the results show that our algorithms consistently outperform previous baseline methods based on direct arm selection. Interestingly, we also demonstrate that in challenging tasks where the arms lack semantic meanings that can be exploited by the LLM, our approach achieves considerably better performance than LLM-based direct arm selection.

Title: FragmentNet: Adaptive Graph Fragmentation for Graph-to-Sequence Molecular Representation Learning

• Authors: Ankur Samanta, Rohan Gupta, Aditi Misra, Christian McIntosh Clarke, Jayakumar Rajadas
• Subjects: cs.LG, cs.AI, physics.chem-ph, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.01184
• Pdf URL: https://arxiv.org/pdf/2502.01184
• Copy Paste: [[2502.01184]] FragmentNet: Adaptive Graph Fragmentation for Graph-to-Sequence Molecular Representation Learning(https://arxiv.org/abs/2502.01184)
• Keywords: foundation model
• Abstract: Molecular property prediction uses molecular structure to infer chemical properties. Chemically interpretable representations that capture meaningful intramolecular interactions enhance the usability and effectiveness of these predictions. However, existing methods often rely on atom-based or rule-based fragment tokenization, which can be chemically suboptimal and lack scalability. We introduce FragmentNet, a graph-to-sequence foundation model with an adaptive, learned tokenizer that decomposes molecular graphs into chemically valid fragments while preserving structural connectivity. FragmentNet integrates VQVAE-GCN for hierarchical fragment embeddings, spatial positional encodings for graph serialization, global molecular descriptors, and a transformer. Pre-trained with Masked Fragment Modeling and fine-tuned on MoleculeNet tasks, FragmentNet outperforms models with similarly scaled architectures and datasets while rivaling larger state-of-the-art models requiring significantly more resources. This novel framework enables adaptive decomposition, serialization, and reconstruction of molecular graphs, facilitating fragment-based editing and visualization of property trends in learned embeddings - a powerful tool for molecular design and optimization.

Title: Dance recalibration for dance coherency with recurrent convolution block

• Authors: Seungho Eum, Ihjoon Cho, Junghyeon Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01190
• Pdf URL: https://arxiv.org/pdf/2502.01190
• Copy Paste: [[2502.01190]] Dance recalibration for dance coherency with recurrent convolution block(https://arxiv.org/abs/2502.01190)
• Keywords: diffusion, generative
• Abstract: With the recent advancements in generative AI such as GAN, Diffusion, and VAE, the use of generative AI for dance generation has seen significant progress and received considerable interest. In this study, We propose R-Lodge, an enhanced version of Lodge. R-Lodge incorporates Recurrent Sequential Representation Learning named Dance Recalibration to original coarse-to-fine long dance generation model. R-Lodge utilizes Dance Recalibration method using $N$ Dance Recalibration Block to address the lack of consistency in the coarse dance representation of the Lodge model. By utilizing this method, each generated dance motion incorporates a bit of information from the previous dance motions. We evaluate R-Lodge on FineDance dataset and the results show that R-Lodge enhances the consistency of the whole generated dance motions.

Title: One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection

• Authors: Yiyue Li, Shaoting Zhang, Kang Li, Qicheng Lao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01201
• Pdf URL: https://arxiv.org/pdf/2502.01201
• Copy Paste: [[2502.01201]] One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection(https://arxiv.org/abs/2502.01201)
• Keywords: anomaly
• Abstract: Traditional Anomaly Detection (AD) methods have predominantly relied on unsupervised learning from extensive normal data. Recent AD methods have evolved with the advent of large pre-trained vision-language models, enhancing few-shot anomaly detection capabilities. However, these latest AD methods still exhibit limitations in accuracy improvement. One contributing factor is their direct comparison of a query image's features with those of few-shot normal images. This direct comparison often leads to a loss of precision and complicates the extension of these techniques to more complex domains--an area that remains underexplored in a more refined and comprehensive manner. To address these limitations, we introduce the anomaly personalization method, which performs a personalized one-to-normal transformation of query images using an anomaly-free customized generation model, ensuring close alignment with the normal manifold. Moreover, to further enhance the stability and robustness of prediction results, we propose a triplet contrastive anomaly inference strategy, which incorporates a comprehensive comparison between the query and generated anomaly-free data pool and prompt information. Extensive evaluations across eleven datasets in three domains demonstrate our model's effectiveness compared to the latest AD methods. Additionally, our method has been proven to transfer flexibly to other AD methods, with the generated image data effectively improving the performance of other AD methods.

Title: Exploring Few-Shot Defect Segmentation in General Industrial Scenarios with Metric Learning and Vision Foundation Models

• Authors: Tongkun Liu, Bing Li, Xiao Jin, Yupeng Shi, Qiuying Li, Xiang Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01216
• Pdf URL: https://arxiv.org/pdf/2502.01216
• Copy Paste: [[2502.01216]] Exploring Few-Shot Defect Segmentation in General Industrial Scenarios with Metric Learning and Vision Foundation Models(https://arxiv.org/abs/2502.01216)
• Keywords: foundation model
• Abstract: Industrial defect segmentation is critical for manufacturing quality control. Due to the scarcity of training defect samples, few-shot semantic segmentation (FSS) holds significant value in this field. However, existing studies mostly apply FSS to tackle defects on simple textures, without considering more diverse scenarios. This paper aims to address this gap by exploring FSS in broader industrial products with various defect types. To this end, we contribute a new real-world dataset and reorganize some existing datasets to build a more comprehensive few-shot defect segmentation (FDS) benchmark. On this benchmark, we thoroughly investigate metric learning-based FSS methods, including those based on meta-learning and those based on Vision Foundation Models (VFMs). We observe that existing meta-learning-based methods are generally not well-suited for this task, while VFMs hold great potential. We further systematically study the applicability of various VFMs in this task, involving two paradigms: feature matching and the use of Segment Anything (SAM) models. We propose a novel efficient FDS method based on feature matching. Meanwhile, we find that SAM2 is particularly effective for addressing FDS through its video track mode. The contributed dataset and code will be available at: this https URL.

Title: A Framework for Double-Blind Federated Adaptation of Foundation Models

• Authors: Nurbek Tastan, Karthik Nandakumar
• Subjects: cs.LG, cs.CR, cs.CV, cs.DC
• Abstract URL: https://arxiv.org/abs/2502.01289
• Pdf URL: https://arxiv.org/pdf/2502.01289
• Copy Paste: [[2502.01289]] A Framework for Double-Blind Federated Adaptation of Foundation Models(https://arxiv.org/abs/2502.01289)
• Keywords: foundation model
• Abstract: The availability of foundational models (FMs) pre-trained on large-scale data has advanced the state-of-the-art in many computer vision tasks. While FMs have demonstrated good zero-shot performance on many image classification tasks, there is often scope for performance improvement by adapting the FM to the downstream task. However, the data that is required for this adaptation typically exists in silos across multiple entities (data owners) and cannot be collated at a central location due to regulations and privacy concerns. At the same time, a learning service provider (LSP) who owns the FM cannot share the model with the data owners due to proprietary reasons. In some cases, the data owners may not even have the resources to store such large FMs. Hence, there is a need for algorithms to adapt the FM in a double-blind federated manner, i.e., the data owners do not know the FM or each other's data, and the LSP does not see the data for the downstream tasks. In this work, we propose a framework for double-blind federated adaptation of FMs using fully homomorphic encryption (FHE). The proposed framework first decomposes the FM into a sequence of FHE-friendly blocks through knowledge distillation. The resulting FHE-friendly model is adapted for the downstream task via low-rank parallel adapters that can be learned without backpropagation through the FM. Since the proposed framework requires the LSP to share intermediate representations with the data owners, we design a privacy-preserving permutation scheme to prevent the data owners from learning the FM through model extraction attacks. Finally, a secure aggregation protocol is employed for federated learning of the low-rank parallel adapters. Empirical results on four datasets demonstrate the practical feasibility of the proposed framework.

Title: Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis

• Authors: Rupert Menneer, Christos Margadji, Sebastian W. Pattinson
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01309
• Pdf URL: https://arxiv.org/pdf/2502.01309
• Copy Paste: [[2502.01309]] Heterogeneous Image GNN: Graph-Conditioned Diffusion for Image Synthesis(https://arxiv.org/abs/2502.01309)
• Keywords: diffusion
• Abstract: We introduce a novel method for conditioning diffusion-based image synthesis models with heterogeneous graph data. Existing approaches typically incorporate conditioning variables directly into model architectures, either through cross-attention layers that attend to text latents or image concatenation that spatially restrict generation. However, these methods struggle to handle complex scenarios involving diverse, relational conditioning variables, which are more naturally represented as unstructured graphs. This paper presents Heterogeneous Image Graphs (HIG), a novel representation that models conditioning variables and target images as two interconnected graphs, enabling efficient handling of variable-length conditioning inputs and their relationships. We also propose a magnitude-preserving GNN that integrates the HIG into the existing EDM2 diffusion model using a ControlNet approach. Our approach improves upon the SOTA on a variety of conditioning inputs for the COCO-stuff and Visual Genome datasets, and showcases the ability to condition on graph attributes and relationships represented by edges in the HIG.

Title: A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers

• Authors: Roman Tarasov, Petr Mokrov, Milena Gazdieva, Evgeny Burnaev, Alexander Korotin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01310
• Pdf URL: https://arxiv.org/pdf/2502.01310
• Copy Paste: [[2502.01310]] A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers(https://arxiv.org/abs/2502.01310)
• Keywords: generative
• Abstract: Neural network based Optimal Transport (OT) is a recent and fruitful direction in the generative modeling community. It finds its applications in various fields such as domain translation, image super-resolution, computational biology and others. Among the existing approaches to OT, of considerable interest are adversarial minimax solvers based on semi-dual formulations of OT problems. While promising, these methods lack theoretical investigation from a statistical learning perspective. Our work fills this gap by establishing upper bounds on the generalization error of an approximate OT map recovered by the minimax quadratic OT solver. Importantly, the bounds we derive depend solely on some standard statistical and mathematical properties of the considered functional classes (neural networks). While our analysis focuses on the quadratic OT, we believe that similar bounds could be derived for more general OT formulations, paving the promising direction for future research.

Title: ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies

• Authors: Costin F. Ciusdel, Alex Serban, Tiziano Passerini
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01335
• Pdf URL: https://arxiv.org/pdf/2502.01335
• Copy Paste: [[2502.01335]] ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies(https://arxiv.org/abs/2502.01335)
• Keywords: self-supervised
• Abstract: While traditional self-supervised learning methods improve performance and robustness across various medical tasks, they rely on single-vector embeddings that may not capture fine-grained concepts such as anatomical structures or organs. The ability to identify such concepts and their characteristics without supervision has the potential to improve pre-training methods, and enable novel applications such as fine-grained image retrieval and concept-based outlier detection. In this paper, we introduce ConceptVAE, a novel pre-training framework that detects and disentangles fine-grained concepts from their style characteristics in a self-supervised manner. We present a suite of loss terms and model architecture primitives designed to discretise input data into a preset number of concepts along with their local style. We validate ConceptVAE both qualitatively and quantitatively, demonstrating its ability to detect fine-grained anatomical structures such as blood pools and septum walls from 2D cardiac echocardiographies. Quantitatively, ConceptVAE outperforms traditional self-supervised methods in tasks such as region-based instance retrieval, semantic segmentation, out-of-distribution detection, and object detection. Additionally, we explore the generation of in-distribution synthetic data that maintains the same concepts as the training data but with distinct styles, highlighting its potential for more calibrated data generation. Overall, our study introduces and validates a promising new pre-training technique based on concept-style disentanglement, opening multiple avenues for developing models for medical image analysis that are more interpretable and explainable than black-box approaches.

Title: Inverse Bridge Matching Distillation

• Authors: Nikita Gushchin, David Li, Daniil Selikhanovych, Evgeny Burnaev, Dmitry Baranchuk, Alexander Korotin
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01362
• Pdf URL: https://arxiv.org/pdf/2502.01362
• Copy Paste: [[2502.01362]] Inverse Bridge Matching Distillation(https://arxiv.org/abs/2502.01362)
• Keywords: diffusion
• Abstract: Learning diffusion bridge models is easy; making them fast and practical is an art. Diffusion bridge models (DBMs) are a promising extension of diffusion models for applications in image-to-image translation. However, like many modern diffusion and flow models, DBMs suffer from the problem of slow inference. To address it, we propose a novel distillation technique based on the inverse bridge matching formulation and derive the tractable objective to solve it in practice. Unlike previously developed DBM distillation techniques, the proposed method can distill both conditional and unconditional types of DBMs, distill models in a one-step generator, and use only the corrupted images for training. We evaluate our approach for both conditional and unconditional types of bridge matching on a wide set of setups, including super-resolution, JPEG restoration, sketch-to-image, and other tasks, and show that our distillation technique allows us to accelerate the inference of DBMs from 4x to 100x and even provide better generation quality than used teacher model depending on particular setup.

Title: Trajectory World Models for Heterogeneous Environments

• Authors: Shaofeng Yin, Jialong Wu, Siqiao Huang, Xingjian Su, Xu He, Jianye Hao, Mingsheng Long
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01366
• Pdf URL: https://arxiv.org/pdf/2502.01366
• Copy Paste: [[2502.01366]] Trajectory World Models for Heterogeneous Environments(https://arxiv.org/abs/2502.01366)
• Keywords: in-context
• Abstract: Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information. In this work, we explore pre-training world models for heterogeneous environments by addressing key transfer barriers in both data diversity and model flexibility. We introduce UniTraj, a unified dataset comprising over one million trajectories from 80 environments, designed to scale data while preserving critical diversity. Additionally, we propose TrajWorld, a novel architecture capable of flexibly handling varying sensor and actuator information and capturing environment dynamics in-context. Pre-training TrajWorld on UniTraj demonstrates significant improvements in transition prediction and achieves a new state-of-the-art for off-policy evaluation. To the best of our knowledge, this work, for the first time, demonstrates the transfer benefits of world models across heterogeneous and complex control environments.

Title: InfoBridge: Mutual Information estimation via Bridge Matching

• Authors: Sergei Kholkin, Ivan Butakov, Evgeny Burnaev, Nikita Gushchin, Alexander Korotin
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.01383
• Pdf URL: https://arxiv.org/pdf/2502.01383
• Copy Paste: [[2502.01383]] InfoBridge: Mutual Information estimation via Bridge Matching(https://arxiv.org/abs/2502.01383)
• Keywords: diffusion, generative
• Abstract: Diffusion bridge models have recently become a powerful tool in the field of generative modeling. In this work, we leverage their power to address another important problem in machine learning and information theory - the estimation of the mutual information (MI) between two random variables. We show that by using the theory of diffusion bridges, one can construct an unbiased estimator for data posing difficulties for conventional MI estimators. We showcase the performance of our estimator on a series of standard MI estimation benchmarks.

Title: Learning Traffic Anomalies from Generative Models on Real-Time Observations

• Authors: Fotis I. Giasemis, Alexandros Sopasakis
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01391
• Pdf URL: https://arxiv.org/pdf/2502.01391
• Copy Paste: [[2502.01391]] Learning Traffic Anomalies from Generative Models on Real-Time Observations(https://arxiv.org/abs/2502.01391)
• Keywords: generative
• Abstract: Accurate detection of traffic anomalies is crucial for effective urban traffic management and congestion mitigation. We use the Spatiotemporal Generative Adversarial Network (STGAN) framework combining Graph Neural Networks and Long Short-Term Memory networks to capture complex spatial and temporal dependencies in traffic data. We apply STGAN to real-time, minute-by-minute observations from 42 traffic cameras across Gothenburg, Sweden, collected over several months in 2020. The images are processed to compute a flow metric representing vehicle density, which serves as input for the model. Training is conducted on data from April to November 2020, and validation is performed on a separate dataset from November 14 to 23, 2020. Our results demonstrate that the model effectively detects traffic anomalies with high precision and low false positive rates. The detected anomalies include camera signal interruptions, visual artifacts, and extreme weather conditions affecting traffic flow.

Title: FourieRF: Few-Shot NeRFs via Progressive Fourier Frequency Control

• Authors: Diego Gomez, Bingchen Gong, Maks Ovsjanikov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01405
• Pdf URL: https://arxiv.org/pdf/2502.01405
• Copy Paste: [[2502.01405]] FourieRF: Few-Shot NeRFs via Progressive Fourier Frequency Control(https://arxiv.org/abs/2502.01405)
• Keywords: foundation model
• Abstract: In this work, we introduce FourieRF, a novel approach for achieving fast and high-quality reconstruction in the few-shot setting. Our method effectively parameterizes features through an explicit curriculum training procedure, incrementally increasing scene complexity during optimization. Experimental results show that the prior induced by our approach is both robust and adaptable across a wide variety of scenes, establishing FourieRF as a strong and versatile baseline for the few-shot rendering problem. While our approach significantly reduces artifacts, it may still lead to reconstruction errors in severely under-constrained scenarios, particularly where view occlusion leaves parts of the shape uncovered. In the future, our method could be enhanced by integrating foundation models to complete missing parts using large data-driven priors.

Title: Human Body Restoration with One-Step Diffusion Model and A New Benchmark

• Authors: Jue Gong, Jingkai Wang, Zheng Chen, Xing Liu, Hong Gu, Yulun Zhang, Xiaokang Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01411
• Pdf URL: https://arxiv.org/pdf/2502.01411
• Copy Paste: [[2502.01411]] Human Body Restoration with One-Step Diffusion Model and A New Benchmark(https://arxiv.org/abs/2502.01411)
• Keywords: diffusion
• Abstract: Human body restoration, as a specific application of image restoration, is widely applied in practice and plays a vital role across diverse fields. However, thorough research remains difficult, particularly due to the lack of benchmark datasets. In this study, we propose a high-quality dataset automated cropping and filtering (HQ-ACF) pipeline. This pipeline leverages existing object detection datasets and other unlabeled images to automatically crop and filter high-quality human images. Using this pipeline, we constructed a person-based restoration with sophisticated objects and natural activities (\emph{PERSONA}) dataset, which includes training, validation, and test sets. The dataset significantly surpasses other human-related datasets in both quality and content richness. Finally, we propose \emph{OSDHuman}, a novel one-step diffusion model for human body restoration. Specifically, we propose a high-fidelity image embedder (HFIE) as the prompt generator to better guide the model with low-quality human image information, effectively avoiding misleading prompts. Experimental results show that OSDHuman outperforms existing methods in both visual quality and quantitative metrics. The dataset and code will at this https URL.

Title: Categorical Schr\"odinger Bridge Matching

• Authors: Grigoriy Ksenofontov, Alexander Korotin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01416
• Pdf URL: https://arxiv.org/pdf/2502.01416
• Copy Paste: [[2502.01416]] Categorical Schr\"odinger Bridge Matching(https://arxiv.org/abs/2502.01416)
• Keywords: generative
• Abstract: The Schrödinger Bridge (SB) is a powerful framework for solving generative modeling tasks such as unpaired domain translation. Most SB-related research focuses on continuous data space $\mathbb{R}^{D}$ and leaves open theoretical and algorithmic questions about applying SB methods to discrete data, e.g, on finite spaces $\mathbb{S}^{D}$. Notable examples of such sets $\mathbb{S}$ are codebooks of vector-quantized (VQ) representations of modern autoencoders, tokens in texts, categories of atoms in molecules, etc. In this paper, we provide a theoretical and algorithmic foundation for solving SB in discrete spaces using the recently introduced Iterative Markovian Fitting (IMF) procedure. Specifically, we theoretically justify the convergence of discrete-time IMF (D-IMF) to SB in discrete spaces. This enables us to develop a practical computational algorithm for SB which we call Categorical Schrödinger Bridge Matching (CSBM). We show the performance of CSBM via a series of experiments with synthetic data and VQ representations of images.

Title: Towards Safer Chatbots: A Framework for Policy Compliance Evaluation of Custom GPTs

• Authors: David Rodriguez, William Seymour, Jose M. Del Alamo, Jose Such
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.01436
• Pdf URL: https://arxiv.org/pdf/2502.01436
• Copy Paste: [[2502.01436]] Towards Safer Chatbots: A Framework for Policy Compliance Evaluation of Custom GPTs(https://arxiv.org/abs/2502.01436)
• Keywords: generative
• Abstract: Large Language Models (LLMs) have gained unprecedented prominence, achieving widespread adoption across diverse domains and integrating deeply into society. The capability to fine-tune general-purpose LLMs, such as Generative Pre-trained Transformers (GPT), for specific tasks has facilitated the emergence of numerous Custom GPTs. These tailored models are increasingly made available through dedicated marketplaces, such as OpenAI's GPT Store. However, their black-box nature introduces significant safety and compliance risks. In this work, we present a scalable framework for the automated evaluation of Custom GPTs against OpenAI's usage policies, which define the permissible behaviors of these systems. Our framework integrates three core components: (1) automated discovery and data collection of models from the GPT store, (2) a red-teaming prompt generator tailored to specific policy categories and the characteristics of each target GPT, and (3) an LLM-as-a-judge technique to analyze each prompt-response pair for potential policy violations. We validate our framework with a manually annotated ground truth, and evaluate it through a large-scale study with 782 Custom GPTs across three categories: Romantic, Cybersecurity, and Academic GPTs. Our manual annotation process achieved an F1 score of 0.975 in identifying policy violations, confirming the reliability of the framework's assessments. The results reveal that 58.7% of the analyzed models exhibit indications of non-compliance, exposing weaknesses in the GPT store's review and approval processes. Furthermore, our findings indicate that a model's popularity does not correlate with compliance, and non-compliance issues largely stem from behaviors inherited from base models rather than user-driven customizations. We believe this approach is extendable to other chatbot platforms and policy domains, improving LLM-based systems safety.

Title: Improved Training Technique for Latent Consistency Models

• Authors: Quan Dao, Khanh Doan, Di Liu, Trung Le, Dimitris Metaxas
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01441
• Pdf URL: https://arxiv.org/pdf/2502.01441
• Copy Paste: [[2502.01441]] Improved Training Technique for Latent Consistency Models(https://arxiv.org/abs/2502.01441)
• Keywords: diffusion, generative
• Abstract: Consistency models are a new family of generative models capable of producing high-quality samples in either a single step or multiple steps. Recently, consistency models have demonstrated impressive performance, achieving results on par with diffusion models in the pixel space. However, the success of scaling consistency training to large-scale datasets, particularly for text-to-image and video generation tasks, is determined by performance in the latent space. In this work, we analyze the statistical differences between pixel and latent spaces, discovering that latent data often contains highly impulsive outliers, which significantly degrade the performance of iCT in the latent space. To address this, we replace Pseudo-Huber losses with Cauchy losses, effectively mitigating the impact of outliers. Additionally, we introduce a diffusion loss at early timesteps and employ optimal transport (OT) coupling to further enhance performance. Lastly, we introduce the adaptive scaling-$c$ scheduler to manage the robust training process and adopt Non-scaling LayerNorm in the architecture to better capture the statistics of the features and reduce outlier impact. With these strategies, we successfully train latent consistency models capable of high-quality sampling with one or two steps, significantly narrowing the performance gap between latent consistency and diffusion models. The implementation is released here: this https URL

Title: Generalization Error Analysis for Selective State-Space Models Through the Lens of Attention

• Authors: Arya Honarpisheh, Mustafa Bozdag, Mario Sznaier, Octavia Camps
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01473
• Pdf URL: https://arxiv.org/pdf/2502.01473
• Copy Paste: [[2502.01473]] Generalization Error Analysis for Selective State-Space Models Through the Lens of Attention(https://arxiv.org/abs/2502.01473)
• Keywords: foundation model
• Abstract: State-space models (SSMs) are a new class of foundation models that have emerged as a compelling alternative to Transformers and their attention mechanisms for sequence processing tasks. This paper provides a detailed theoretical analysis of selective SSMs, the core components of the Mamba and Mamba-2 architectures. We leverage the connection between selective SSMs and the self-attention mechanism to highlight the fundamental similarities between these models. Building on this connection, we establish a length independent covering number-based generalization bound for selective SSMs, providing a deeper understanding of their theoretical performance guarantees. We analyze the effects of state matrix stability and input-dependent discretization, shedding light on the critical role played by these factors in the generalization capabilities of selective SSMs. Finally, we empirically demonstrate the sequence length independence of the derived bounds on two tasks.

Title: MoireDB: Formula-generated Interference-fringe Image Dataset

• Authors: Yuto Matsuo, Ryo Hayamizu, Hirokatsu Kataoka, Akio Nakamura
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01490
• Pdf URL: https://arxiv.org/pdf/2502.01490
• Copy Paste: [[2502.01490]] MoireDB: Formula-generated Interference-fringe Image Dataset(https://arxiv.org/abs/2502.01490)
• Keywords: generative
• Abstract: Image recognition models have struggled to treat recognition robustness to real-world degradations. In this context, data augmentation methods like PixMix improve robustness but rely on generative arts and feature visualizations (FVis), which have copyright, drawing cost, and scalability issues. We propose MoireDB, a formula-generated interference-fringe image dataset for image augmentation enhancing robustness. MoireDB eliminates copyright concerns, reduces dataset assembly costs, and enhances robustness by leveraging illusory patterns. Experiments show that MoireDB augmented images outperforms traditional Fractal arts and FVis-based augmentations, making it a scalable and effective solution for improving model robustness against real-world degradations.

Title: End-to-end Training for Text-to-Image Synthesis using Dual-Text Embeddings

• Authors: Yeruru Asrar Ahmed, Anurag Mittal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01507
• Pdf URL: https://arxiv.org/pdf/2502.01507
• Copy Paste: [[2502.01507]] End-to-end Training for Text-to-Image Synthesis using Dual-Text Embeddings(https://arxiv.org/abs/2502.01507)
• Keywords: generative
• Abstract: Text-to-Image (T2I) synthesis is a challenging task that requires modeling complex interactions between two modalities ( i.e., text and image). A common framework adopted in recent state-of-the-art approaches to achieving such multimodal interactions is to bootstrap the learning process with pre-trained image-aligned text embeddings trained using contrastive loss. Furthermore, these embeddings are typically trained generically and reused across various synthesis models. In contrast, we explore an approach to learning text embeddings specifically tailored to the T2I synthesis network, trained in an end-to-end fashion. Further, we combine generative and contrastive training and use two embeddings, one optimized to enhance the photo-realism of the generated images, and the other seeking to capture text-to-image alignment. A comprehensive set of experiments on three text-to-image benchmark datasets (Oxford-102, Caltech-UCSD, and MS-COCO) reveal that having two separate embeddings gives better results than using a shared one and that such an approach performs favourably in comparison with methods that use text representations from a pre-trained text encoder trained using a discriminative approach. Finally, we demonstrate that such learned embeddings can be used in other contexts as well, such as text-to-image manipulation.

Title: BD-Diff: Generative Diffusion Model for Image Deblurring on Unknown Domains with Blur-Decoupled Learning

• Authors: Junhao Cheng, Wei-Ting Chen, Xi Lu, Ming-Hsuan Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01522
• Pdf URL: https://arxiv.org/pdf/2502.01522
• Copy Paste: [[2502.01522]] BD-Diff: Generative Diffusion Model for Image Deblurring on Unknown Domains with Blur-Decoupled Learning(https://arxiv.org/abs/2502.01522)
• Keywords: diffusion, generative
• Abstract: Generative diffusion models trained on large-scale datasets have achieved remarkable progress in image synthesis. In favor of their ability to supplement missing details and generate aesthetically pleasing contents, recent works have applied them to image deblurring tasks via training an adapter on blurry-sharp image pairs to provide structural conditions for restoration. However, acquiring substantial amounts of realistic paired data is challenging and costly in real-world scenarios. On the other hand, relying solely on synthetic data often results in overfitting, leading to unsatisfactory performance when confronted with unseen blur patterns. To tackle this issue, we propose BD-Diff, a generative-diffusion-based model designed to enhance deblurring performance on unknown domains by decoupling structural features and blur patterns through joint training on three specially designed tasks. We employ two Q-Formers as structural representations and blur patterns extractors separately. The features extracted by them will be used for the supervised deblurring task on synthetic data and the unsupervised blur-transfer task by leveraging unpaired blurred images from the target domain simultaneously. Furthermore, we introduce a reconstruction task to make the structural features and blur patterns complementary. This blur-decoupled learning process enhances the generalization capabilities of BD-Diff when encountering unknown domain blur patterns. Experiments on real-world datasets demonstrate that BD-Diff outperforms existing state-of-the-art methods in blur removal and structural preservation in various challenging scenarios. The codes will be released in this https URL

Title: The in-context inductive biases of vision-language models differ across modalities

• Authors: Kelsey Allen, Ishita Dasgupta, Eliza Kosoy, Andrew K. Lampinen
• Subjects: cs.CV, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01530
• Pdf URL: https://arxiv.org/pdf/2502.01530
• Copy Paste: [[2502.01530]] The in-context inductive biases of vision-language models differ across modalities(https://arxiv.org/abs/2502.01530)
• Keywords: foundation model, in-context
• Abstract: Inductive biases are what allow learners to make guesses in the absence of conclusive evidence. These biases have often been studied in cognitive science using concepts or categories -- e.g. by testing how humans generalize a new category from a few examples that leave the category boundary ambiguous. We use these approaches to study generalization in foundation models during in-context learning. Modern foundation models can condition on both vision and text, and differences in how they interpret and learn from these different modalities is an emerging area of study. Here, we study how their generalizations vary by the modality in which stimuli are presented, and the way the stimuli are described in text. We study these biases with three different experimental paradigms, across three different vision-language models. We find that the models generally show some bias towards generalizing according to shape over color. This shape bias tends to be amplified when the examples are presented visually. By contrast, when examples are presented in text, the ordering of adjectives affects generalization. However, the extent of these effects vary across models and paradigms. These results help to reveal how vision-language models represent different types of inputs in context, and may have practical implications for the use of vision-language models.

Title: Federated Learning with Discriminative Naive Bayes Classifier

• Authors: Pablo Torrijos, Juan C. Alfaro, José A. Gámez, José M. Puerta
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01532
• Pdf URL: https://arxiv.org/pdf/2502.01532
• Copy Paste: [[2502.01532]] Federated Learning with Discriminative Naive Bayes Classifier(https://arxiv.org/abs/2502.01532)
• Keywords: generative
• Abstract: Federated Learning has emerged as a promising approach to train machine learning models on decentralized data sources while preserving data privacy. This paper proposes a new federated approach for Naive Bayes (NB) classification, assuming discrete variables. Our approach federates a discriminative variant of NB, sharing meaningless parameters instead of conditional probability tables. Therefore, this process is more reliable against possible attacks. We conduct extensive experiments on 12 datasets to validate the efficacy of our approach, comparing federated and non-federated settings. Additionally, we benchmark our method against the generative variant of NB, which serves as a baseline for comparison. Our experimental results demonstrate the effectiveness of our method in achieving accurate classification.

Title: Transformers trained on proteins can learn to attend to Euclidean distance

• Authors: Isaac Ellmen, Constantin Schneider, Matthew I.J. Raybould, Charlotte M. Deane
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.01533
• Pdf URL: https://arxiv.org/pdf/2502.01533
• Copy Paste: [[2502.01533]] Transformers trained on proteins can learn to attend to Euclidean distance(https://arxiv.org/abs/2502.01533)
• Keywords: diffusion
• Abstract: While conventional Transformers generally operate on sequence data, they can be used in conjunction with structure models, typically SE(3)-invariant or equivariant graph neural networks (GNNs), for 3D applications such as protein structure modelling. These hybrids typically involve either (1) preprocessing/tokenizing structural features as input for Transformers or (2) taking Transformer embeddings and processing them within a structural representation. However, there is evidence that Transformers can learn to process structural information on their own, such as the AlphaFold3 structural diffusion model. In this work we show that Transformers can function independently as structure models when passed linear embeddings of coordinates. We first provide a theoretical explanation for how Transformers can learn to filter attention as a 3D Gaussian with learned variance. We then validate this theory using both simulated 3D points and in the context of masked token prediction for proteins. Finally, we show that pre-training protein Transformer encoders with structure improves performance on a downstream task, yielding better performance than custom structural models. Together, this work provides a basis for using standard Transformers as hybrid structure-language models.

Title: Unsupervised anomaly detection in large-scale estuarine acoustic telemetry data

• Authors: Siphendulwe Zaza, Marcellin Atemkeng, Taryn S. Murray, John David Filmalter, Paul D. Cowley
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01543
• Pdf URL: https://arxiv.org/pdf/2502.01543
• Copy Paste: [[2502.01543]] Unsupervised anomaly detection in large-scale estuarine acoustic telemetry data(https://arxiv.org/abs/2502.01543)
• Keywords: anomaly
• Abstract: Acoustic telemetry data plays a vital role in understanding the behaviour and movement of aquatic animals. However, these datasets, which often consist of millions of individual data points, frequently contain anomalous movements that pose significant challenges. Traditionally, anomalous movements are identified either manually or through basic statistical methods, approaches that are time-consuming and prone to high rates of unidentified anomalies in large datasets. This study focuses on the development of automated classifiers for a large telemetry dataset comprising detections from fifty acoustically tagged dusky kob monitored in the Breede Estuary, South Africa. Using an array of 16 acoustic receivers deployed throughout the estuary between 2016 and 2021, we collected over three million individual data points. We present detailed guidelines for data pre-processing, resampling strategies, labelling process, feature engineering, data splitting methodologies, and the selection and interpretation of machine learning and deep learning models for anomaly detection. Among the evaluated models, neural networks autoencoder (NN-AE) demonstrated superior performance, aided by our proposed threshold-finding algorithm. NN-AE achieved a high recall with no false normal (i.e., no misclassifications of anomalous movements as normal patterns), a critical factor in ensuring that no true anomalies are overlooked. In contrast, other models exhibited false normal fractions exceeding 0.9, indicating they failed to detect the majority of true anomalies; a significant limitation for telemetry studies where undetected anomalies can distort interpretations of movement patterns. While the NN-AE's performance highlights its reliability and robustness in detecting anomalies, it faced challenges in accurately learning normal movement patterns when these patterns gradually deviated from anomalous ones.

Title: Scalable Language Models with Posterior Inference of Latent Thought Vectors

• Authors: Deqian Kong, Minglu Zhao, Dehong Xu, Bo Pang, Shu Wang, Edouardo Honig, Zhangzhang Si, Chuan Li, Jianwen Xie, Sirui Xie, Ying Nian Wu
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.01567
• Pdf URL: https://arxiv.org/pdf/2502.01567
• Copy Paste: [[2502.01567]] Scalable Language Models with Posterior Inference of Latent Thought Vectors(https://arxiv.org/abs/2502.01567)
• Keywords: diffusion, in-context
• Abstract: We propose a novel family of language models, Latent-Thought Language Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. These latent thought vectors guide the autoregressive generation of ground tokens through a Transformer decoder. Training employs a dual-rate optimization process within the classical variational Bayes framework: fast learning of local variational parameters for the posterior distribution of latent vectors, and slow learning of global decoder parameters. Empirical studies reveal that LTMs possess additional scaling dimensions beyond traditional LLMs, yielding a structured design space. Higher sample efficiency can be achieved by increasing training compute per token, with further gains possible by trading model size for more inference steps. Designed based on these scaling properties, LTMs demonstrate superior sample and parameter efficiency compared to conventional autoregressive models and discrete diffusion models. They significantly outperform these counterparts in validation perplexity and zero-shot language modeling. Additionally, LTMs exhibit emergent few-shot in-context reasoning capabilities that scale with model and latent size, and achieve competitive performance in conditional and unconditional text generation.

Title: MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation

• Authors: Yiren Song, Cheng Liu, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01572
• Pdf URL: https://arxiv.org/pdf/2502.01572
• Copy Paste: [[2502.01572]] MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation(https://arxiv.org/abs/2502.01572)
• Keywords: diffusion, in-context
• Abstract: A hallmark of human intelligence is the ability to create complex artifacts through structured multi-step processes. Generating procedural tutorials with AI is a longstanding but challenging goal, facing three key obstacles: (1) scarcity of multi-task procedural datasets, (2) maintaining logical continuity and visual consistency between steps, and (3) generalizing across multiple domains. To address these challenges, we propose a multi-domain dataset covering 21 tasks with over 24,000 procedural sequences. Building upon this foundation, we introduce MakeAnything, a framework based on the diffusion transformer (DIT), which leverages fine-tuning to activate the in-context capabilities of DIT for generating consistent procedural sequences. We introduce asymmetric low-rank adaptation (LoRA) for image generation, which balances generalization capabilities and task-specific performance by freezing encoder parameters while adaptively tuning decoder layers. Additionally, our ReCraft model enables image-to-process generation through spatiotemporal consistency constraints, allowing static images to be decomposed into plausible creation sequences. Extensive experiments demonstrate that MakeAnything surpasses existing methods, setting new performance benchmarks for procedural generation tasks.

Title: Breaking Focus: Contextual Distraction Curse in Large Language Models

• Authors: Yue Huang, Yanbo Wang, Zixiang Xu, Chujie Gao, Siyuan Wu, Jiayi Ye, Xiuying Chen, Pin-Yu Chen, Xiangliang Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.01609
• Pdf URL: https://arxiv.org/pdf/2502.01609
• Copy Paste: [[2502.01609]] Breaking Focus: Contextual Distraction Curse in Large Language Models(https://arxiv.org/abs/2502.01609)
• Keywords: generative
• Abstract: Recent advances in Large Language Models (LLMs) have revolutionized generative systems, achieving excellent performance across diverse domains. Although these models perform well in controlled environments, their real-world applications frequently encounter inputs containing both essential and irrelevant details. Our investigation has revealed a critical vulnerability in LLMs, which we term Contextual Distraction Vulnerability (CDV). This phenomenon arises when models fail to maintain consistent performance on questions modified with semantically coherent but irrelevant context. To systematically investigate this vulnerability, we propose an efficient tree-based search methodology to automatically generate CDV examples. Our approach successfully generates CDV examples across four datasets, causing an average performance degradation of approximately 45% in state-of-the-art LLMs. To address this critical issue, we explore various mitigation strategies and find that post-targeted training approaches can effectively enhance model robustness against contextual distractions. Our findings highlight the fundamental nature of CDV as an ability-level challenge rather than a knowledge-level issue since models demonstrate the necessary knowledge by answering correctly in the absence of distractions. This calls the community's attention to address CDV during model development to ensure reliability. The code is available at this https URL.

Title: Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning

• Authors: Udita Ghosh, Dripta S. Raychaudhuri, Jiachen Li, Konstantinos Karydis, Amit Roy-Chowdhury
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01616
• Pdf URL: https://arxiv.org/pdf/2502.01616
• Copy Paste: [[2502.01616]] Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning(https://arxiv.org/abs/2502.01616)
• Keywords: self-supervised
• Abstract: Preference-based reinforcement learning (RL) offers a promising approach for aligning policies with human intent but is often constrained by the high cost of human feedback. In this work, we introduce PrefVLM, a framework that integrates Vision-Language Models (VLMs) with selective human feedback to significantly reduce annotation requirements while maintaining performance. Our method leverages VLMs to generate initial preference labels, which are then filtered to identify uncertain cases for targeted human annotation. Additionally, we adapt VLMs using a self-supervised inverse dynamics loss to improve alignment with evolving policies. Experiments on Meta-World manipulation tasks demonstrate that PrefVLM achieves comparable or superior success rates to state-of-the-art methods while using up to 2 x fewer human annotations. Furthermore, we show that adapted VLMs enable efficient knowledge transfer across tasks, further minimizing feedback needs. Our results highlight the potential of combining VLMs with selective human supervision to make preference-based RL more scalable and practical.

Title: MFP-VTON: Enhancing Mask-Free Person-to-Person Virtual Try-On via Diffusion Transformer

• Authors: Le Shen, Yanting Kang, Rong Huang, Zhijie Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.01626
• Pdf URL: https://arxiv.org/pdf/2502.01626
• Copy Paste: [[2502.01626]] MFP-VTON: Enhancing Mask-Free Person-to-Person Virtual Try-On via Diffusion Transformer(https://arxiv.org/abs/2502.01626)
• Keywords: diffusion, generative
• Abstract: The garment-to-person virtual try-on (VTON) task, which aims to generate fitting images of a person wearing a reference garment, has made significant strides. However, obtaining a standard garment is often more challenging than using the garment already worn by the person. To improve ease of use, we propose MFP-VTON, a Mask-Free framework for Person-to-Person VTON. Recognizing the scarcity of person-to-person data, we adapt a garment-to-person model and dataset to construct a specialized dataset for this task. Our approach builds upon a pretrained diffusion transformer, leveraging its strong generative capabilities. During mask-free model fine-tuning, we introduce a Focus Attention loss to emphasize the garment of the reference person and the details outside the garment of the target person. Experimental results demonstrate that our model excels in both person-to-person and garment-to-person VTON tasks, generating high-fidelity fitting images.

Title: SliderSpace: Decomposing the Visual Capabilities of Diffusion Models

• Authors: Rohit Gandikota, Zongze Wu, Richard Zhang, David Bau, Eli Shechtman, Nick Kolkin
• Subjects: cs.CV, cs.GR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.01639
• Pdf URL: https://arxiv.org/pdf/2502.01639
• Copy Paste: [[2502.01639]] SliderSpace: Decomposing the Visual Capabilities of Diffusion Models(https://arxiv.org/abs/2502.01639)
• Keywords: diffusion
• Abstract: We present SliderSpace, a framework for automatically decomposing the visual capabilities of diffusion models into controllable and human-understandable directions. Unlike existing control methods that require a user to specify attributes for each edit direction individually, SliderSpace discovers multiple interpretable and diverse directions simultaneously from a single text prompt. Each direction is trained as a low-rank adaptor, enabling compositional control and the discovery of surprising possibilities in the model's latent space. Through extensive experiments on state-of-the-art diffusion models, we demonstrate SliderSpace's effectiveness across three applications: concept decomposition, artistic style exploration, and diversity enhancement. Our quantitative evaluation shows that SliderSpace-discovered directions decompose the visual structure of model's knowledge effectively, offering insights into the latent capabilities encoded within diffusion models. User studies further validate that our method produces more diverse and useful variations compared to baselines. Our code, data and trained weights are available at this https URL