2025-07-15

Title: Physical Informed Neural Networks for modeling ocean pollutant

• Authors: Karishma Battina, Prathamesh Dinesh Joshi, Raj Abhijit Dandekar, Rajat Dandekar, Sreedath Panat
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.08834
• Pdf URL: https://arxiv.org/pdf/2507.08834
• Copy Paste: [[2507.08834]] Physical Informed Neural Networks for modeling ocean pollutant(https://arxiv.org/abs/2507.08834)
• Keywords: diffusion
• Abstract: Traditional numerical methods often struggle with the complexity and scale of modeling pollutant transport across vast and dynamic oceanic domains. This paper introduces a Physics-Informed Neural Network (PINN) framework to simulate the dispersion of pollutants governed by the 2D advection-diffusion equation. The model achieves physically consistent predictions by embedding physical laws and fitting to noisy synthetic data, generated via a finite difference method (FDM), directly into the neural network training process. This approach addresses challenges such as non-linear dynamics and the enforcement of boundary and initial conditions. Synthetic data sets, augmented with varying noise levels, are used to capture real-world variability. The training incorporates a hybrid loss function including PDE residuals, boundary/initial condition conformity, and a weighted data fit term. The approach takes advantage of the Julia language scientific computing ecosystem for high-performance simulations, offering a scalable and flexible alternative to traditional solvers

Title: wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models

• Authors: Xiaohang Tang, Rares Dolga, Sangwoong Yoon, Ilija Bogunovic
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.08838
• Pdf URL: https://arxiv.org/pdf/2507.08838
• Copy Paste: [[2507.08838]] wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models(https://arxiv.org/abs/2507.08838)
• Keywords: diffusion
• Abstract: Improving the reasoning capabilities of diffusion-based large language models (dLLMs) through reinforcement learning (RL) remains an open problem. The intractability of dLLMs likelihood function necessitates approximating the current, old, and reference policy likelihoods at each policy optimization step. This reliance introduces additional computational overhead and lead to potentially large bias -- particularly when approximation errors occur in the denominator of policy ratios used for importance sampling. To mitigate these issues, we introduce $\mathtt{wd1}$, a novel policy optimization approach that reformulates the objective as a weighted likelihood, requiring only a single approximation for the current parametrized policy likelihood. Experiments on widely used reasoning benchmarks demonstrate that $\mathtt{wd1}$, without supervised fine-tuning (SFT) or any supervised data, outperforms existing RL methods for dLLMs, achieving up to 16% higher accuracy. $\mathtt{wd1}$ delivers additional computational gains, including reduced training time and fewer function evaluations (NFEs) per gradient step. These findings, combined with the simplicity of method's implementation and R1-Zero-like training (no SFT), position $\mathtt{wd1}$ as a more effective and efficient method for applying RL to dLLMs reasoning.

Title: Clio-X: AWeb3 Solution for Privacy-Preserving AI Access to Digital Archives

• Authors: Victoria L. Lemieux, Rosa Gil, Faith Molosiwa, Qihong Zhou, Binming Li, Roberto Garcia, Luis De La Torre Cubillo, Zehua Wang
• Subjects: cs.CR, cs.AI, cs.CY, cs.DL
• Abstract URL: https://arxiv.org/abs/2507.08853
• Pdf URL: https://arxiv.org/pdf/2507.08853
• Copy Paste: [[2507.08853]] Clio-X: AWeb3 Solution for Privacy-Preserving AI Access to Digital Archives(https://arxiv.org/abs/2507.08853)
• Keywords: diffusion
• Abstract: As archives turn to artificial intelligence to manage growing volumes of digital records, privacy risks inherent in current AI data practices raise critical concerns about data sovereignty and ethical accountability. This paper explores how privacy-enhancing technologies (PETs) and Web3 architectures can support archives to preserve control over sensitive content while still being able to make it available for access by researchers. We present Clio-X, a decentralized, privacy-first Web3 digital solution designed to embed PETs into archival workflows and support AI-enabled reference and access. Drawing on a user evaluation of a medium-fidelity prototype, the study reveals both interest in the potential of the solution and significant barriers to adoption related to trust, system opacity, economic concerns, and governance. Using Rogers' Diffusion of Innovation theory, we analyze the sociotechnical dimensions of these barriers and propose a path forward centered on participatory design and decentralized governance through a Clio-X Decentralized Autonomous Organization. By integrating technical safeguards with community-based oversight, Clio-X offers a novel model to ethically deploy AI in cultural heritage contexts.

Title: Foundation models for time series forecasting: Application in conformal prediction

• Authors: Sami Achour, Yassine Bouher, Duong Nguyen, Nicolas Chesneau
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.08858
• Pdf URL: https://arxiv.org/pdf/2507.08858
• Copy Paste: [[2507.08858]] Foundation models for time series forecasting: Application in conformal prediction(https://arxiv.org/abs/2507.08858)
• Keywords: foundation model
• Abstract: The zero-shot capabilities of foundation models (FMs) for time series forecasting offer promising potentials in conformal prediction, as most of the available data can be allocated to calibration. This study compares the performance of Time Series Foundation Models (TSFMs) with traditional methods, including statistical models and gradient boosting, within a conformal prediction setting. Our findings highlight two key advantages of TSFMs. First, when the volume of data is limited, TSFMs provide more reliable conformalized prediction intervals than classic models, thanks to their superior predictive accuracy. Second, the calibration process is more stable because more data are used for calibration. Morever, the fewer data available, the more pronounced these benefits become, as classic models require a substantial amount of data for effective training. These results underscore the potential of foundation models in improving conformal prediction reliability in time series applications, particularly in data-constrained cases. All the code to reproduce the experiments is available.

Title: Next-Generation Travel Demand Modeling with a Generative Framework for Household Activity Coordination

• Authors: Xishun Liao, Haoxuan Ma, Yifan Liu, Yuxiang Wei, Brian Yueshuai He, Chris Stanford, Jiaqi Ma
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.08871
• Pdf URL: https://arxiv.org/pdf/2507.08871
• Copy Paste: [[2507.08871]] Next-Generation Travel Demand Modeling with a Generative Framework for Household Activity Coordination(https://arxiv.org/abs/2507.08871)
• Keywords: generative
• Abstract: Travel demand models are critical tools for planning, policy, and mobility system design. Traditional activity-based models (ABMs), although grounded in behavioral theories, often rely on simplified rules and assumptions, and are costly to develop and difficult to adapt across different regions. This paper presents a learning-based travel demand modeling framework that synthesizes household-coordinated daily activity patterns based on a household's socio-demographic profiles. The whole framework integrates population synthesis, coordinated activity generation, location assignment, and large-scale microscopic traffic simulation into a unified system. It is fully generative, data-driven, scalable, and transferable to other regions. A full-pipeline implementation is conducted in Los Angeles with a 10 million population. Comprehensive validation shows that the model closely replicates real-world mobility patterns and matches the performance of legacy ABMs with significantly reduced modeling cost and greater scalability. With respect to the SCAG ABM benchmark, the origin-destination matrix achieves a cosine similarity of 0.97, and the daily vehicle miles traveled (VMT) in the network yields a 0.006 Jensen-Shannon Divergence (JSD) and a 9.8% mean absolute percentage error (MAPE). When compared to real-world observations from Caltrans PeMS, the evaluation on corridor-level traffic speed and volume reaches a 0.001 JSD and a 6.11% MAPE.

Title: Beyond Scores: Proximal Diffusion Models

• Authors: Zhenghan Fang, Mateo Díaz, Sam Buchanan, Jeremias Sulam
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.08956
• Pdf URL: https://arxiv.org/pdf/2507.08956
• Copy Paste: [[2507.08956]] Beyond Scores: Proximal Diffusion Models(https://arxiv.org/abs/2507.08956)
• Keywords: diffusion, generative
• Abstract: Diffusion models have quickly become some of the most popular and powerful generative models for high-dimensional data. The key insight that enabled their development was the realization that access to the score -- the gradient of the log-density at different noise levels -- allows for sampling from data distributions by solving a reverse-time stochastic differential equation (SDE) via forward discretization, and that popular denoisers allow for unbiased estimators of this score. In this paper, we demonstrate that an alternative, backward discretization of these SDEs, using proximal maps in place of the score, leads to theoretical and practical benefits. We leverage recent results in proximal matching to learn proximal operators of the log-density and, with them, develop Proximal Diffusion Models (ProxDM). Theoretically, we prove that $\widetilde{O}(d/\sqrt{\varepsilon})$ steps suffice for the resulting discretization to generate an $\varepsilon$-accurate distribution w.r.t. the KL divergence. Empirically, we show that two variants of ProxDM achieve significantly faster convergence within just a few sampling steps compared to conventional score-matching methods.

Title: Theory-Informed Improvements to Classifier-Free Guidance for Discrete Diffusion Models

• Authors: Kevin Rojas, Ye He, Chieh-Hsin Lai, Yuta Takida, Yuki Mitsufuji, Molei Tao
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.08965
• Pdf URL: https://arxiv.org/pdf/2507.08965
• Copy Paste: [[2507.08965]] Theory-Informed Improvements to Classifier-Free Guidance for Discrete Diffusion Models(https://arxiv.org/abs/2507.08965)
• Keywords: diffusion
• Abstract: Classifier-Free Guidance (CFG) is a widely used technique for conditional generation and improving sample quality in continuous diffusion models, and recent works have extended it to discrete diffusion. This paper theoretically analyzes CFG in the context of masked discrete diffusion, focusing on the role of guidance schedules. Our analysis shows that high guidance early in sampling (when inputs are heavily masked) harms generation quality, while late-stage guidance has a larger effect. These findings provide a theoretical explanation for empirical observations in recent studies on guidance schedules. The analysis also reveals an imperfection of the current CFG implementations. These implementations can unintentionally cause imbalanced transitions, such as unmasking too rapidly during the early stages of generation, which degrades the quality of the resulting samples. To address this, we draw insight from the analysis and propose a novel classifier-free guidance mechanism empirically applicable to any discrete diffusion. Intuitively, our method smoothens the transport between the data distribution and the initial (masked/uniform) distribution, which results in improved sample quality. Remarkably, our method is achievable via a simple one-line code change. The efficacy of our method is empirically demonstrated with experiments on ImageNet (masked discrete diffusion) and QM9 (uniform discrete diffusion).

Title: Learning Diffusion Models with Flexible Representation Guidance

• Authors: Chenyu Wang, Cai Zhou, Sharut Gupta, Zongyu Lin, Stefanie Jegelka, Stephen Bates, Tommi Jaakkola
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.08980
• Pdf URL: https://arxiv.org/pdf/2507.08980
• Copy Paste: [[2507.08980]] Learning Diffusion Models with Flexible Representation Guidance(https://arxiv.org/abs/2507.08980)
• Keywords: diffusion
• Abstract: Diffusion models can be improved with additional guidance towards more effective representations of input. Indeed, prior empirical work has already shown that aligning internal representations of the diffusion model with those of pre-trained models improves generation quality. In this paper, we present a systematic framework for incorporating representation guidance into diffusion models. We provide alternative decompositions of denoising models along with their associated training criteria, where the decompositions determine when and how the auxiliary representations are incorporated. Guided by our theoretical insights, we introduce two new strategies for enhancing representation alignment in diffusion models. First, we pair examples with target representations either derived from themselves or arisen from different synthetic modalities, and subsequently learn a joint model over the multimodal pairs. Second, we design an optimal training curriculum that balances representation learning and data generation. Our experiments across image, protein sequence, and molecule generation tasks demonstrate superior performance as well as accelerated training. In particular, on the class-conditional ImageNet $256\times 256$ benchmark, our guidance results in $23.3$ times faster training than the original SiT-XL as well as four times speedup over the state-of-the-art method REPA. The code is available at this https URL.

Title: VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels

• Authors: Xiwei Xuan, Xiaoqi Wang, Wenbin He, Jorge Piazentin Ono, Liang Gou, Kwan-Liu Ma, Liu Ren
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09008
• Pdf URL: https://arxiv.org/pdf/2507.09008
• Copy Paste: [[2507.09008]] VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels(https://arxiv.org/abs/2507.09008)
• Keywords: foundation model
• Abstract: The advances in multi-modal foundation models (FMs) (e.g., CLIP and LLaVA) have facilitated the auto-labeling of large-scale datasets, enhancing model performance in challenging downstream tasks such as open-vocabulary object detection and segmentation. However, the quality of FM-generated labels is less studied as existing approaches focus more on data quantity over quality. This is because validating large volumes of data without ground truth presents a considerable challenge in practice. Existing methods typically rely on limited metrics to identify problematic data, lacking a comprehensive perspective, or apply human validation to only a small data fraction, failing to address the full spectrum of potential issues. To overcome these challenges, we introduce VISTA, a visual analytics framework that improves data quality to enhance the performance of multi-modal models. Targeting the complex and demanding domain of open-vocabulary image segmentation, VISTA integrates multi-phased data validation strategies with human expertise, enabling humans to identify, understand, and correct hidden issues within FM-generated labels. Through detailed use cases on two benchmark datasets and expert reviews, we demonstrate VISTA's effectiveness from both quantitative and qualitative perspectives.

Title: Multimodal Cardiovascular Risk Profiling Using Self-Supervised Learning of Polysomnography

• Authors: Zhengxiao He, Huayu Li, Geng Yuan, William D.S. Killgore, Stuart F. Quan, Chen X. Chen, Ao Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09009
• Pdf URL: https://arxiv.org/pdf/2507.09009
• Copy Paste: [[2507.09009]] Multimodal Cardiovascular Risk Profiling Using Self-Supervised Learning of Polysomnography(https://arxiv.org/abs/2507.09009)
• Keywords: self-supervised
• Abstract: Methods: We developed a self-supervised deep learning model that extracts meaningful patterns from multi-modal signals (Electroencephalography (EEG), Electrocardiography (ECG), and respiratory signals). The model was trained on data from 4,398 participants. Projection scores were derived by contrasting embeddings from individuals with and without CVD outcomes. External validation was conducted in an independent cohort with 1,093 participants. The source code is available on this https URL. Results: The projection scores revealed distinct and clinically meaningful patterns across modalities. ECG-derived features were predictive of both prevalent and incident cardiac conditions, particularly CVD mortality. EEG-derived features were predictive of incident hypertension and CVD mortality. Respiratory signals added complementary predictive value. Combining these projection scores with the Framingham Risk Score consistently improved predictive performance, achieving area under the curve values ranging from 0.607 to 0.965 across different outcomes. Findings were robustly replicated and validated in the external testing cohort. Conclusion: Our findings demonstrate that the proposed framework can generate individualized CVD risk scores directly from PSG data. The resulting projection scores have the potential to be integrated into clinical practice, enhancing risk assessment and supporting personalized care.

Title: Beyond vividness: Content analysis of induced hallucinations reveals the hidden structure of individual differences in visual imagery

• Authors: Ana Chkhaidze, Reshanne R. Reeder, Connor Gag, Anastasia Kiyonaga, Seana Coulson
• Subjects: cs.CL, q-bio.NC, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2507.09011
• Pdf URL: https://arxiv.org/pdf/2507.09011
• Copy Paste: [[2507.09011]] Beyond vividness: Content analysis of induced hallucinations reveals the hidden structure of individual differences in visual imagery(https://arxiv.org/abs/2507.09011)
• Keywords: generative
• Abstract: A rapidly alternating red and black display known as Ganzflicker induces visual hallucinations that reflect the generative capacity of the visual system. Recent proposals regarding the imagery spectrum, that is, differences in the visual system of individuals with absent imagery, typical imagery, and vivid imagery, suggest these differences should impact the complexity of other internally generated visual experiences. Here, we used tools from natural language processing to analyze free-text descriptions of hallucinations from over 4,000 participants, asking whether people with different imagery phenotypes see different things in their mind's eye during Ganzflicker-induced hallucinations. Strong imagers described complex, naturalistic content, while weak imagers reported simple geometric patterns. Embeddings from vision language models better captured these differences than text-only language models, and participants with stronger imagery used language with richer sensorimotor associations. These findings may reflect individual variation in coordination between early visual areas and higher-order regions relevant for the imagery spectrum.

Title: Behavioral Exploration: Learning to Explore via In-Context Adaptation

• Authors: Andrew Wagenmaker, Zhiyuan Zhou, Sergey Levine
• Subjects: cs.LG, cs.RO, eess.SY
• Abstract URL: https://arxiv.org/abs/2507.09041
• Pdf URL: https://arxiv.org/pdf/2507.09041
• Copy Paste: [[2507.09041]] Behavioral Exploration: Learning to Explore via In-Context Adaptation(https://arxiv.org/abs/2507.09041)
• Keywords: generative, in-context
• Abstract: Developing autonomous agents that quickly explore an environment and adapt their behavior online is a canonical challenge in robotics and machine learning. While humans are able to achieve such fast online exploration and adaptation, often acquiring new information and skills in only a handful of interactions, existing algorithmic approaches tend to rely on random exploration and slow, gradient-based behavior updates. How can we endow autonomous agents with such capabilities on par with humans? Taking inspiration from recent progress on both in-context learning and large-scale behavioral cloning, in this work we propose behavioral exploration: training agents to internalize what it means to explore and adapt in-context over the space of ``expert'' behaviors. To achieve this, given access to a dataset of expert demonstrations, we train a long-context generative model to predict expert actions conditioned on a context of past observations and a measure of how ``exploratory'' the expert's behaviors are relative to this context. This enables the model to not only mimic the behavior of an expert, but also, by feeding its past history of interactions into its context, to select different expert behaviors than what have been previously selected, thereby allowing for fast online adaptation and targeted, ``expert-like'' exploration. We demonstrate the effectiveness of our method in both simulated locomotion and manipulation settings, as well as on real-world robotic manipulation tasks, illustrating its ability to learn adaptive, exploratory behavior.

Title: Shortening the Trajectories: Identity-Aware Gaussian Approximation for Efficient 3D Molecular Generation

• Authors: Jingxiang Qu, Wenhan Gao, Yi Liu
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.09043
• Pdf URL: https://arxiv.org/pdf/2507.09043
• Copy Paste: [[2507.09043]] Shortening the Trajectories: Identity-Aware Gaussian Approximation for Efficient 3D Molecular Generation(https://arxiv.org/abs/2507.09043)
• Keywords: generative
• Abstract: Gaussian-based Probabilistic Generative Models (GPGMs) generate data by reversing a stochastic process that progressively corrupts samples with Gaussian noise. While these models have achieved state-of-the-art performance across diverse domains, their practical deployment remains constrained by the high computational cost of long generative trajectories, which often involve hundreds to thousands of steps during training and sampling. In this work, we introduce a theoretically grounded and empirically validated framework that improves generation efficiency without sacrificing training granularity or inference fidelity. Our key insight is that for certain data modalities, the noising process causes data to rapidly lose its identity and converge toward a Gaussian distribution. We analytically identify a characteristic step at which the data has acquired sufficient Gaussianity, and then replace the remaining generation trajectory with a closed-form Gaussian approximation. Unlike existing acceleration techniques that coarsening the trajectories by skipping steps, our method preserves the full resolution of learning dynamics while avoiding redundant stochastic perturbations between `Gaussian-like' distributions. Empirical results across multiple data modalities demonstrate substantial improvements in both sample quality and computational efficiency.

Title: Can Contrastive Learning Improve Class-Imbalanced Diffusion Model?

• Authors: Fang Chen, Alex Villa, Gongbo Liang, Xiaoyi Lu, Meng Tang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09052
• Pdf URL: https://arxiv.org/pdf/2507.09052
• Copy Paste: [[2507.09052]] Can Contrastive Learning Improve Class-Imbalanced Diffusion Model?(https://arxiv.org/abs/2507.09052)
• Keywords: diffusion
• Abstract: Training data for class-conditional image synthesis often exhibit a long-tailed distribution with limited images for tail classes. Such an imbalance causes mode collapse and reduces the diversity of synthesized images for tail classes. For class-conditional diffusion models trained on imbalanced data, we aim to improve the diversity of tail class images without compromising the fidelity and diversity of head class images. We achieve this by introducing two deceptively simple but highly effective contrastive loss functions. Firstly, we employ an unsupervised InfoNCE loss utilizing negative samples to increase the distance/dissimilarity among synthetic images, particularly for tail classes. To further enhance the diversity of tail classes, our second loss is an MSE loss that contrasts class-conditional generation with unconditional generation at large timesteps. This second loss makes the denoising process insensitive to class conditions for the initial steps, which enriches tail classes through knowledge sharing from head classes. Conditional-unconditional alignment has been shown to enhance the performance of long-tailed GAN. We are the first to adapt such alignment to diffusion models. We successfully leveraged contrastive learning for class-imbalanced diffusion models. Our contrastive learning framework is easy to implement and outperforms standard DDPM and alternative methods for class-imbalanced diffusion models across various datasets, including CIFAR10/100-LT, PlacesLT, TinyImageNetLT, and ImageNetLT.

Title: From Physics to Foundation Models: A Review of AI-Driven Quantitative Remote Sensing Inversion

• Authors: Zhenyu Yu, Mohd Yamani Idna Idris, Hua Wang, Pei Wang, Junyi Chen, Kun Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09081
• Pdf URL: https://arxiv.org/pdf/2507.09081
• Copy Paste: [[2507.09081]] From Physics to Foundation Models: A Review of AI-Driven Quantitative Remote Sensing Inversion(https://arxiv.org/abs/2507.09081)
• Keywords: self-supervised, foundation model
• Abstract: Quantitative remote sensing inversion aims to estimate continuous surface variables-such as biomass, vegetation indices, and evapotranspiration-from satellite observations, supporting applications in ecosystem monitoring, carbon accounting, and land management. With the evolution of remote sensing systems and artificial intelligence, traditional physics-based paradigms are giving way to data-driven and foundation model (FM)-based approaches. This paper systematically reviews the methodological evolution of inversion techniques, from physical models (e.g., PROSPECT, SCOPE, DART) to machine learning methods (e.g., deep learning, multimodal fusion), and further to foundation models (e.g., SatMAE, GFM, mmEarth). We compare the modeling assumptions, application scenarios, and limitations of each paradigm, with emphasis on recent FM advances in self-supervised pretraining, multi-modal integration, and cross-task adaptation. We also highlight persistent challenges in physical interpretability, domain generalization, limited supervision, and uncertainty quantification. Finally, we envision the development of next-generation foundation models for remote sensing inversion, emphasizing unified modeling capacity, cross-domain generalization, and physical interpretability.

Title: Taming generative video models for zero-shot optical flow extraction

• Authors: Seungwoo Kim, Khai Loong Aw, Klemen Kotar, Cristobal Eyzaguirre, Wanhee Lee, Yunong Liu, Jared Watrous, Stefan Stojanov, Juan Carlos Niebles, Jiajun Wu, Daniel L. K. Yamins
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09082
• Pdf URL: https://arxiv.org/pdf/2507.09082
• Copy Paste: [[2507.09082]] Taming generative video models for zero-shot optical flow extraction(https://arxiv.org/abs/2507.09082)
• Keywords: self-supervised, generative
• Abstract: Extracting optical flow from videos remains a core computer vision problem. Motivated by the success of large general-purpose models, we ask whether frozen self-supervised video models trained only for future frame prediction can be prompted, without fine-tuning, to output flow. Prior work reading out depth or illumination from video generators required fine-tuning, which is impractical for flow where labels are scarce and synthetic datasets suffer from a sim-to-real gap. Inspired by the Counterfactual World Model (CWM) paradigm, which can obtain point-wise correspondences by injecting a small tracer perturbation into a next-frame predictor and tracking its propagation, we extend this idea to generative video models. We explore several popular architectures and find that successful zero-shot flow extraction in this manner is aided by three model properties: (1) distributional prediction of future frames (avoiding blurry or noisy outputs); (2) factorized latents that treat each spatio-temporal patch independently; and (3) random-access decoding that can condition on any subset of future pixels. These properties are uniquely present in the recent Local Random Access Sequence (LRAS) architecture. Building on LRAS, we propose KL-tracing: a novel test-time procedure that injects a localized perturbation into the first frame, rolls out the model one step, and computes the Kullback-Leibler divergence between perturbed and unperturbed predictive distributions. Without any flow-specific fine-tuning, our method outperforms state-of-the-art models on real-world TAP-Vid DAVIS dataset (16.6% relative improvement for endpoint error) and synthetic TAP-Vid Kubric (4.7% relative improvement). Our results indicate that counterfactual prompting of controllable generative video models is a scalable and effective alternative to supervised or photometric-loss approaches for high-quality flow.

Title: On the Fragility of Multimodal Perception to Temporal Misalignment in Autonomous Driving

• Authors: Md Hasan Shahriar, Md Mohaimin Al Barat, Harshavardhan Sundar, Naren Ramakrishnan, Y. Thomas Hou, Wenjing Lou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09095
• Pdf URL: https://arxiv.org/pdf/2507.09095
• Copy Paste: [[2507.09095]] On the Fragility of Multimodal Perception to Temporal Misalignment in Autonomous Driving(https://arxiv.org/abs/2507.09095)
• Keywords: anomaly
• Abstract: Multimodal fusion (MMF) plays a critical role in the perception of autonomous driving, which primarily fuses camera and LiDAR streams for a comprehensive and efficient scene understanding. However, its strict reliance on precise temporal synchronization exposes it to new vulnerabilities. In this paper, we introduce DejaVu, a novel attack that exploits network-induced delays to create subtle temporal misalignments across sensor streams, severely degrading downstream MMF-based perception tasks. Our comprehensive attack analysis across different models and datasets reveals these sensors' task-specific imbalanced sensitivities: object detection is overly dependent on LiDAR inputs while object tracking is highly reliant on the camera inputs. Consequently, with a single-frame LiDAR delay, an attacker can reduce the car detection mAP by up to 88.5%, while with a three-frame camera delay, multiple object tracking accuracy (MOTA) for car drops by 73%. To detect such attacks, we propose AION, a defense patch that can work alongside the existing perception model to monitor temporal alignment through cross-modal temporal consistency. AION leverages multimodal shared representation learning and dynamic time warping to determine the path of temporal alignment and calculate anomaly scores based on the alignment. Our thorough evaluation of AION shows it achieves AUROC scores of 0.92-0.98 with low false positives across datasets and model architectures, demonstrating it as a robust and generalized defense against the temporal misalignment attacks.

Title: Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning

• Authors: Yiyang Chen, Shanshan Zhao, Lunhao Duan, Changxing Ding, Dacheng Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09102
• Pdf URL: https://arxiv.org/pdf/2507.09102
• Copy Paste: [[2507.09102]] Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning(https://arxiv.org/abs/2507.09102)
• Keywords: diffusion, self-supervised
• Abstract: Diffusion-based models, widely used in text-to-image generation, have proven effective in 2D representation learning. Recently, this framework has been extended to 3D self-supervised learning by constructing a conditional point generator for enhancing 3D representations. However, its performance remains constrained by the 3D diffusion model, which is trained on the available 3D datasets with limited size. We hypothesize that the robust capabilities of text-to-image diffusion models, particularly Stable Diffusion (SD), which is trained on large-scale datasets, can help overcome these limitations. To investigate this hypothesis, we propose PointSD, a framework that leverages the SD model for 3D self-supervised learning. By replacing the SD model's text encoder with a 3D encoder, we train a point-to-image diffusion model that allows point clouds to guide the denoising of rendered noisy images. With the trained point-to-image diffusion model, we use noise-free images as the input and point clouds as the condition to extract SD features. Next, we train a 3D backbone by aligning its features with these SD features, thereby facilitating direct semantic learning. Comprehensive experiments on downstream point cloud tasks and ablation studies demonstrate that the SD model can enhance point cloud self-supervised learning. Code is publicly available at this https URL.

Title: Hybrid Autoregressive-Diffusion Model for Real-Time Streaming Sign Language Production

• Authors: Maoxiao Ye, Xinfeng Ye, Mano Manoharan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09105
• Pdf URL: https://arxiv.org/pdf/2507.09105
• Copy Paste: [[2507.09105]] Hybrid Autoregressive-Diffusion Model for Real-Time Streaming Sign Language Production(https://arxiv.org/abs/2507.09105)
• Keywords: diffusion
• Abstract: Earlier Sign Language Production (SLP) models typically relied on autoregressive methods that generate output tokens one by one, which inherently provide temporal alignment. Although techniques like Teacher Forcing can prevent model collapse during training, they still cannot solve the problem of error accumulation during inference, since ground truth is unavailable at that stage. In contrast, more recent approaches based on diffusion models leverage step-by-step denoising to enable high-quality generation. However, the iterative nature of these models and the requirement to denoise entire sequences limit their applicability in real-time tasks like SLP. To address it, we apply a hybrid approach combining autoregressive and diffusion models to SLP for the first time, leveraging the strengths of both models in sequential dependency modeling and output refinement. To capture fine-grained body movements, we design a Multi-Scale Pose Representation module that separately extracts detailed features from distinct articulators and integrates them via a Multi-Scale Fusion module. Furthermore, we introduce a Confidence-Aware Causal Attention mechanism that utilizes joint-level confidence scores to dynamically guide the pose generation process, improving accuracy and robustness. Extensive experiments on the PHOENIX14T and How2Sign datasets demonstrate the effectiveness of our method in both generation quality and real-time streaming efficiency.

Title: SnapMoGen: Human Motion Generation from Expressive Texts

• Authors: Chuan Guo, Inwoo Hwang, Jian Wang, Bing Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09122
• Pdf URL: https://arxiv.org/pdf/2507.09122
• Copy Paste: [[2507.09122]] SnapMoGen: Human Motion Generation from Expressive Texts(https://arxiv.org/abs/2507.09122)
• Keywords: generative
• Abstract: Text-to-motion generation has experienced remarkable progress in recent years. However, current approaches remain limited to synthesizing motion from short or general text prompts, primarily due to dataset constraints. This limitation undermines fine-grained controllability and generalization to unseen prompts. In this paper, we introduce SnapMoGen, a new text-motion dataset featuring high-quality motion capture data paired with accurate, expressive textual annotations. The dataset comprises 20K motion clips totaling 44 hours, accompanied by 122K detailed textual descriptions averaging 48 words per description (vs. 12 words of HumanML3D). Importantly, these motion clips preserve original temporal continuity as they were in long sequences, facilitating research in long-term motion generation and blending. We also improve upon previous generative masked modeling approaches. Our model, MoMask++, transforms motion into multi-scale token sequences that better exploit the token capacity, and learns to generate all tokens using a single generative masked transformer. MoMask++ achieves state-of-the-art performance on both HumanML3D and SnapMoGen benchmarks. Additionally, we demonstrate the ability to process casual user prompts by employing an LLM to reformat inputs to align with the expressivity and narration style of SnapMoGen. Project webpage: this https URL

Title: Stable Score Distillation

• Authors: Haiming Zhu, Yangyang Xu, Chenshu Xu, Tingrui Shen, Wenxi Liu, Yong Du, Jun Yu, Shengfeng He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09168
• Pdf URL: https://arxiv.org/pdf/2507.09168
• Copy Paste: [[2507.09168]] Stable Score Distillation(https://arxiv.org/abs/2507.09168)
• Keywords: diffusion
• Abstract: Text-guided image and 3D editing have advanced with diffusion-based models, yet methods like Delta Denoising Score often struggle with stability, spatial control, and editing strength. These limitations stem from reliance on complex auxiliary structures, which introduce conflicting optimization signals and restrict precise, localized edits. We introduce Stable Score Distillation (SSD), a streamlined framework that enhances stability and alignment in the editing process by anchoring a single classifier to the source prompt. Specifically, SSD utilizes Classifier-Free Guidance (CFG) equation to achieves cross-prompt alignment, and introduces a constant term null-text branch to stabilize the optimization process. This approach preserves the original content's structure and ensures that editing trajectories are closely aligned with the source prompt, enabling smooth, prompt-specific modifications while maintaining coherence in surrounding regions. Additionally, SSD incorporates a prompt enhancement branch to boost editing strength, particularly for style transformations. Our method achieves state-of-the-art results in 2D and 3D editing tasks, including NeRF and text-driven style edits, with faster convergence and reduced complexity, providing a robust and efficient solution for text-guided editing.

Title: XiChen: An observation-scalable fully AI-driven global weather forecasting system with 4D variational knowledge

• Authors: Wuxin Wang, Weicheng Ni, Lilan Huang, Tao Hao, Ben Fei, Shuo Ma, Taikang Yuan, Yanlai Zhao, Kefeng Deng, Xiaoyong Li, Boheng Duan, Lei Bai, Kaijun Ren
• Subjects: cs.LG, cs.AI, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2507.09202
• Pdf URL: https://arxiv.org/pdf/2507.09202
• Copy Paste: [[2507.09202]] XiChen: An observation-scalable fully AI-driven global weather forecasting system with 4D variational knowledge(https://arxiv.org/abs/2507.09202)
• Keywords: foundation model
• Abstract: Recent advancements in Artificial Intelligence (AI) demonstrate significant potential to revolutionize weather forecasting. However, most AI-driven models rely on Numerical Weather Prediction (NWP) systems for initial condition preparation, which often consumes hours on supercomputers. Here we introduce XiChen, the first observation-scalable fully AI-driven global weather forecasting system, whose entire pipeline, from Data Assimilation (DA) to medium-range forecasting, can be accomplished within only 17 seconds. XiChen is built upon a foundation model that is pre-trained for weather forecasting. Meanwhile, this model is subsequently fine-tuned to serve as both observation operators and DA models, thereby scalably assimilating conventional and raw satellite observations. Furthermore, the integration of four-dimensional variational knowledge ensures that XiChen's DA and medium-range forecasting accuracy rivals that of operational NWP systems, amazingly achieving a skillful forecasting lead time exceeding 8.25 days. These findings demonstrate that XiChen holds strong potential toward fully AI-driven weather forecasting independent of NWP systems.

Title: Banzhida: Advancing Large Language Models for Tibetan with Curated Data and Continual Pre-Training

• Authors: Leiyu Pan, Bojian Xiong, Lei Yang, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Sangjee Dondrub, Caizang Tai, Haixing Zhao, Huaque Cairang, Suonan Cairang, Rou Te, Lengben Zhaxi, Gazang Zhaxi, Zhonglin Ye, Yuhui Zheng, Chunyan Peng, Secha Jia, Pema Tashi, Cizhen Jiacuo, Pema Dorjee, Hongkai Liu, Pema Yanggon, Tsehang Dorjee, Jiaxin Han, Qiongying Hu, Jilin Man, Huanke You, Yuqi Ren, Duo La, Deyi Xiong
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.09205
• Pdf URL: https://arxiv.org/pdf/2507.09205
• Copy Paste: [[2507.09205]] Banzhida: Advancing Large Language Models for Tibetan with Curated Data and Continual Pre-Training(https://arxiv.org/abs/2507.09205)
• Keywords: generative
• Abstract: Large language models have achieved remarkable progress across many languages. However, Tibetan, as a representative low-resource language, is particularly underrepresented in existing models due to the scarcity of high-quality training corpora. To address this gap, we curate the largest Tibetan pre-training corpus to date, aggregating data from diverse sources and applying a dedicated data cleaning and processing pipeline tailored for Tibetan. With the curated data, we continue pre/post-training a multilingual base model into Banzhida, a multilingual large language model that advances generative AI for Tibetan. To evaluate the Tibetan capabilities of the model, we create new high-quality Tibetan benchmarks, and complement them with existing public benchmarks. Experimental results demonstrate that Banzhida consistently and significantly outperforms both open-source models of similar scale and Tibetan-tailored models across a wide range of tasks.

Title: Capturing Unseen Spatial Extremes Through Knowledge-Informed Generative Modeling

• Authors: Xinyue Liu, Xiao Peng, Shuyue Yan, Yuntian Chen, Dongxiao Zhang, Zhixiao Niu, Hui-Min Wang, Xiaogang He
• Subjects: cs.LG, physics.ao-ph, physics.data-an, physics.geo-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.09211
• Pdf URL: https://arxiv.org/pdf/2507.09211
• Copy Paste: [[2507.09211]] Capturing Unseen Spatial Extremes Through Knowledge-Informed Generative Modeling(https://arxiv.org/abs/2507.09211)
• Keywords: generative
• Abstract: Observed records of climate extremes provide an incomplete picture of risk, missing "unseen" extremes that exceed historical bounds. In parallel, neglecting spatial dependence undervalues the risk of synchronized hazards that amplify impacts. To address these challenges, we develop DeepX-GAN (Dependence-Enhanced Embedding for Physical eXtremes - Generative Adversarial Network), a knowledge-informed deep generative model designed to better capture the spatial structure of rare extremes. The zero-shot generalizability of DeepX-GAN enables simulation of unseen extremes that fall outside historical experience yet remain statistically plausible. We define two types of unseen extremes: "checkmate" extremes that directly hit targets, and "stalemate" extremes that narrowly miss. These unrealized scenarios expose latent risks in fragile systems and may reinforce a false sense of resilience if overlooked. Near misses, in particular, can prompt either proactive adaptation or dangerous complacency, depending on how they are interpreted. Applying DeepX-GAN to the Middle East and North Africa (MENA), we find that these unseen extremes disproportionately affect regions with high vulnerability and low socioeconomic readiness, but differ in urgency and interpretation. Future warming could expand and redistribute these unseen extremes, with emerging exposure hotspots in Indo-Pakistan and Central Africa. This distributional shift highlights critical blind spots in conventional hazard planning and underscores the need to develop spatially adaptive policies that anticipate emergent risk hotspots rather than simply extrapolating from historical patterns.

Title: Warm Starts Accelerate Generative Modelling

• Authors: Jonas Scholz, Richard E. Turner
• Subjects: cs.LG, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.09212
• Pdf URL: https://arxiv.org/pdf/2507.09212
• Copy Paste: [[2507.09212]] Warm Starts Accelerate Generative Modelling(https://arxiv.org/abs/2507.09212)
• Keywords: diffusion, generative
• Abstract: Iterative generative models, like diffusion and flow-matching, create high-fidelity samples by progressively refining a noise vector into data. However, this process is notoriously slow, often requiring hundreds of function evaluations. We introduce the warm-start model, a simple, deterministic model that dramatically accelerates conditional generation by providing a better starting point. Instead of starting generation from an uninformed N(0, I) prior, our warm-start model predicts an informed prior N(mu, sigma), whose moments are conditioned on the input context. This "warm start" substantially reduces the distance the generative process must traverse, particularly when the conditioning information is strongly informative. On tasks like image inpainting, our method achieves results competitive with a 1000-step DDPM baseline using only 11 total function evaluations (1 for the warm start, 10 for generation). A simple conditional normalization trick makes our method compatible with any standard generative model and sampler without modification, allowing it to be combined with other efficient sampling techniques for further acceleration. Our implementation is available at this https URL.

Title: Stereo-based 3D Anomaly Object Detection for Autonomous Driving: A New Dataset and Baseline

• Authors: Shiyi Mu, Zichong Gu, Hanqi Lyu, Yilin Gao, Shugong Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09214
• Pdf URL: https://arxiv.org/pdf/2507.09214
• Copy Paste: [[2507.09214]] Stereo-based 3D Anomaly Object Detection for Autonomous Driving: A New Dataset and Baseline(https://arxiv.org/abs/2507.09214)
• Keywords: anomaly
• Abstract: 3D detection technology is widely used in the field of autonomous driving, with its application scenarios gradually expanding from enclosed highways to open conventional roads. For rare anomaly categories that appear on the road, 3D detection models trained on closed sets often misdetect or fail to detect anomaly objects. To address this risk, it is necessary to enhance the generalization ability of 3D detection models for targets of arbitrary shapes and to possess the capability to filter out anomalies. The generalization of 3D detection is limited by two factors: the coupled training of 2D and 3D, and the insufficient diversity in the scale distribution of training samples. This paper proposes a Stereo-based 3D Anomaly object Detection (S3AD) algorithm, which decouples the training strategy of 3D and 2D to release the generalization ability for arbitrary 3D foreground detection, and proposes an anomaly scoring algorithm based on foreground confidence prediction, achieving target-level anomaly scoring. In order to further verify and enhance the generalization of anomaly detection, we use a 3D rendering method to synthesize two augmented reality binocular stereo 3D detection datasets which named KITTI-AR. KITTI-AR extends upon KITTI by adding 97 new categories, totaling 6k pairs of stereo images. The KITTI-AR-ExD subset includes 39 common categories as extra training data to address the sparse sample distribution issue. Additionally, 58 rare categories form the KITTI-AR-OoD subset, which are not used in training to simulate zero-shot scenarios in real-world settings, solely for evaluating 3D anomaly detection. Finally, the performance of the algorithm and the dataset is verified in the experiments. (Code and dataset can be obtained at this https URL).

Title: Online Long-term Point Tracking in the Foundation Model Era

• Authors: Görkay Aydemir
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09217
• Pdf URL: https://arxiv.org/pdf/2507.09217
• Copy Paste: [[2507.09217]] Online Long-term Point Tracking in the Foundation Model Era(https://arxiv.org/abs/2507.09217)
• Keywords: foundation model
• Abstract: Point tracking aims to identify the same physical point across video frames and serves as a geometry-aware representation of motion. This representation supports a wide range of applications, from robotics to augmented reality, by enabling accurate modeling of dynamic environments. Most existing long-term tracking approaches operate in an offline setting, where future frames are available to refine predictions and recover from occlusions. However, real-world scenarios often demand online predictions: the model must operate causally, using only current and past frames. This constraint is critical in streaming video and embodied AI, where decisions must be made immediately based on past observations. Under such constraints, viewpoint invariance becomes essential. Visual foundation models, trained on diverse large-scale datasets, offer the potential for robust geometric representations. While they lack temporal reasoning on their own, they can be integrated into tracking pipelines to enrich spatial features. In this thesis, we address the problem of long-term point tracking in an online setting, where frames are processed sequentially without access to future information or sliding windows. We begin by evaluating the suitability of visual foundation models for this task and find that they can serve as useful initializations and be integrated into tracking pipelines. However, to enable long-term tracking in an online setting, a dedicated design is still required. In particular, maintaining coherence over time in this causal regime requires memory to propagate appearance and context across frames. To address this, we introduce Track-On, a transformer-based model that treats each tracked point as a query and processes video frames one at a time. Track-On sets a new state of the art across seven public benchmarks, demonstrating the feasibility of long-term tracking without future access.

Title: Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift

• Authors: Behraj Khan, Tahir Syed
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09222
• Pdf URL: https://arxiv.org/pdf/2507.09222
• Copy Paste: [[2507.09222]] Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift(https://arxiv.org/abs/2507.09222)
• Keywords: foundation model
• Abstract: Foundation models like CLIP and SAM have transformed computer vision and medical imaging via low-shot transfer learning. However, deployment of these models hindered by two key challenges: \textit{distribution shift} between training and test data, and \textit{confidence misalignment} that leads to overconfident incorrect predictions. These issues manifest differently in vision-language classification and medical segmentation tasks, yet existing solutions remain domain-specific. We propose \textit{StaRFM}, a unified framework addressing both challenges. It introduces a Fisher information penalty (FIP), extended to 3D medical data via patch-wise regularization, to reduce covariate shift in CLIP and SAM embeddings. Additionally, a confidence misalignment penalty (CMP), reformulated for voxel-level predictions, calibrates uncertainty in segmentation tasks. We theoretically derive PAC-Bayes bounds showing FIP controls generalization via the Fisher-Rao norm, while CMP minimizes calibration error through Brier score optimization. StaRFM shows consistent performance like \texttt{+}3.5\% accuracy and 28\% lower ECE on 19 vision datasets (e.g., ImageNet, Office-Home), 84.7\% DSC and 4.8mm HD95 in medical segmentation (e.g., BraTS, ATLAS), and 40\% lower cross-domain performance gap compared to prior benchmarking methods. The framework is plug-and-play, requiring minimal architectural changes for seamless integration with foundation models. Code and models will be released at this https URL

Title: EgoAnimate: Generating Human Animations from Egocentric top-down Views

• Authors: G. Kutay Türkoglu, Julian Tanke, Iheb Belgacem, Lev Markhasin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09230
• Pdf URL: https://arxiv.org/pdf/2507.09230
• Copy Paste: [[2507.09230]] EgoAnimate: Generating Human Animations from Egocentric top-down Views(https://arxiv.org/abs/2507.09230)
• Keywords: diffusion, generative
• Abstract: An ideal digital telepresence experience requires accurate replication of a person's body, clothing, and movements. To capture and transfer these movements into virtual reality, the egocentric (first-person) perspective can be adopted, which enables the use of a portable and cost-effective device without front-view cameras. However, this viewpoint introduces challenges such as occlusions and distorted body proportions. There are few works reconstructing human appearance from egocentric views, and none use a generative prior-based approach. Some methods create avatars from a single egocentric image during inference, but still rely on multi-view datasets during training. To our knowledge, this is the first study using a generative backbone to reconstruct animatable avatars from egocentric inputs. Based on Stable Diffusion, our method reduces training burden and improves generalizability. Inspired by methods such as SiTH and MagicMan, which perform 360-degree reconstruction from a frontal image, we introduce a pipeline that generates realistic frontal views from occluded top-down images using ControlNet and a Stable Diffusion backbone. Our goal is to convert a single top-down egocentric image into a realistic frontal representation and feed it into an image-to-motion model. This enables generation of avatar motions from minimal input, paving the way for more accessible and generalizable telepresence systems.

Title: ClaritySpeech: Dementia Obfuscation in Speech

• Authors: Dominika Woszczyk, Ranya Aloufi, Soteris Demetriou
• Subjects: cs.CL, cs.CR, cs.LG, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2507.09282
• Pdf URL: https://arxiv.org/pdf/2507.09282
• Copy Paste: [[2507.09282]] ClaritySpeech: Dementia Obfuscation in Speech(https://arxiv.org/abs/2507.09282)
• Keywords: generative
• Abstract: Dementia, a neurodegenerative disease, alters speech patterns, creating communication barriers and raising privacy concerns. Current speech technologies, such as automatic speech transcription (ASR), struggle with dementia and atypical speech, further challenging accessibility. This paper presents a novel dementia obfuscation in speech framework, ClaritySpeech, integrating ASR, text obfuscation, and zero-shot text-to-speech (TTS) to correct dementia-affected speech while preserving speaker identity in low-data environments without fine-tuning. Results show a 16% and 10% drop in mean F1 score across various adversarial settings and modalities (audio, text, fusion) for ADReSS and ADReSSo, respectively, maintaining 50% speaker similarity. We also find that our system improves WER (from 0.73 to 0.08 for ADReSS and 0.15 for ADReSSo) and speech quality from 1.65 to ~2.15, enhancing privacy and accessibility.

Title: Generative Latent Kernel Modeling for Blind Motion Deblurring

• Authors: Chenhao Ding, Jiangtao Zhang, Zongsheng Yue, Hui Wang, Qian Zhao, Deyu Meng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09285
• Pdf URL: https://arxiv.org/pdf/2507.09285
• Copy Paste: [[2507.09285]] Generative Latent Kernel Modeling for Blind Motion Deblurring(https://arxiv.org/abs/2507.09285)
• Keywords: generative
• Abstract: Deep prior-based approaches have demonstrated remarkable success in blind motion deblurring (BMD) recently. These methods, however, are often limited by the high non-convexity of the underlying optimization process in BMD, which leads to extreme sensitivity to the initial blur kernel. To address this issue, we propose a novel framework for BMD that leverages a deep generative model to encode the kernel prior and induce a better initialization for the blur kernel. Specifically, we pre-train a kernel generator based on a generative adversarial network (GAN) to aptly characterize the kernel's prior distribution, as well as a kernel initializer to provide a well-informed and high-quality starting point for kernel estimation. By combining these two components, we constrain the BMD solution within a compact latent kernel manifold, thus alleviating the aforementioned sensitivity for kernel initialization. Notably, the kernel generator and initializer are designed to be easily integrated with existing BMD methods in a plug-and-play manner, enhancing their overall performance. Furthermore, we extend our approach to tackle blind non-uniform motion deblurring without the need for additional priors, achieving state-of-the-art performance on challenging benchmark datasets. The source code is available at this https URL.

Title: AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning

• Authors: Zile Wang, Hao Yu, Jiabo Zhan, Chun Yuan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09308
• Pdf URL: https://arxiv.org/pdf/2507.09308
• Copy Paste: [[2507.09308]] AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning(https://arxiv.org/abs/2507.09308)
• Keywords: diffusion
• Abstract: Recent advances in latent diffusion models have achieved remarkable results in high-fidelity RGB image synthesis by leveraging pretrained VAEs to compress and reconstruct pixel data at low computational cost. However, the generation of transparent or layered content (RGBA image) remains largely unexplored, due to the lack of large-scale benchmarks. In this work, we propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds. We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel. The model is trained with a composite objective that combines alpha-blended pixel reconstruction, patch-level fidelity, perceptual consistency, and dual KL divergence constraints to ensure latent fidelity across both RGB and alpha representations. Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction. It also enables superior transparent image generation when fine-tuned within a latent diffusion framework. Our code, data, and models are released on this https URL for reproducibility.

Title: Dynamic Inter-Class Confusion-Aware Encoder for Audio-Visual Fusion in Human Activity Recognition

• Authors: Kaixuan Cong, Yifan Wang, Rongkun Xue, Yuyang Jiang, Yiming Feng, Jing Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09323
• Pdf URL: https://arxiv.org/pdf/2507.09323
• Copy Paste: [[2507.09323]] Dynamic Inter-Class Confusion-Aware Encoder for Audio-Visual Fusion in Human Activity Recognition(https://arxiv.org/abs/2507.09323)
• Keywords: self-supervised
• Abstract: Humans do not understand individual events in isolation; rather, they generalize concepts within classes and compare them to others. Existing audio-video pre-training paradigms only focus on the alignment of the overall audio-video modalities, without considering the reinforcement of distinguishing easily confused classes through cognitive induction and contrast during training. This paper proposes the Dynamic Inter-Class Confusion-Aware Encoder (DICCAE), an encoder that aligns audio-video representations at a fine-grained, category-level. DICCAE addresses category confusion by dynamically adjusting the confusion loss based on inter-class confusion degrees, thereby enhancing the model's ability to distinguish between similar activities. To further extend the application of DICCAE, we also introduce a novel training framework that incorporates both audio and video modalities, as well as their fusion. To mitigate the scarcity of audio-video data in the human activity recognition task, we propose a cluster-guided audio-video self-supervised pre-training strategy for DICCAE. DICCAE achieves near state-of-the-art performance on the VGGSound dataset, with a top-1 accuracy of 65.5%. We further evaluate its feature representation quality through extensive ablation studies, validating the necessity of each module.

Title: Simplifying Traffic Anomaly Detection with Video Foundation Models

• Authors: Svetlana Orlova, Tommie Kerssies, Brunó B. Englert, Gijs Dubbelman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09338
• Pdf URL: https://arxiv.org/pdf/2507.09338
• Copy Paste: [[2507.09338]] Simplifying Traffic Anomaly Detection with Video Foundation Models(https://arxiv.org/abs/2507.09338)
• Keywords: self-supervised, foundation model, anomaly
• Abstract: Recent methods for ego-centric Traffic Anomaly Detection (TAD) often rely on complex multi-stage or multi-representation fusion architectures, yet it remains unclear whether such complexity is necessary. Recent findings in visual perception suggest that foundation models, enabled by advanced pre-training, allow simple yet flexible architectures to outperform specialized designs. Therefore, in this work, we investigate an architecturally simple encoder-only approach using plain Video Vision Transformers (Video ViTs) and study how pre-training enables strong TAD performance. We find that: (i) strong pre-training enables simple encoder-only models to match or even surpass the performance of specialized state-of-the-art TAD methods, while also being significantly more efficient; (ii) although weakly- and fully-supervised pre-training are advantageous on standard benchmarks, we find them less effective for TAD. Instead, self-supervised Masked Video Modeling (MVM) provides the strongest signal; and (iii) Domain-Adaptive Pre-Training (DAPT) on unlabeled driving videos further improves downstream performance, without requiring anomalous examples. Our findings highlight the importance of pre-training and show that effective, efficient, and scalable TAD models can be built with minimal architectural complexity. We release our code, domain-adapted encoders, and fine-tuned models to support future work: this https URL.

Title: Meta-autoencoders: An approach to discovery and representation of relationships between dynamically evolving classes

• Authors: Assaf Marron, Smadar Szekely, Irun Cohen, David Harel
• Subjects: cs.LG, q-bio.PE
• Abstract URL: https://arxiv.org/abs/2507.09362
• Pdf URL: https://arxiv.org/pdf/2507.09362
• Copy Paste: [[2507.09362]] Meta-autoencoders: An approach to discovery and representation of relationships between dynamically evolving classes(https://arxiv.org/abs/2507.09362)
• Keywords: self-supervised
• Abstract: An autoencoder (AE) is a neural network that, using self-supervised training, learns a succinct parameterized representation, and a corresponding encoding and decoding process, for all instances in a given class. Here, we introduce the concept of a meta-autoencoder (MAE): an AE for a collection of autoencoders. Given a family of classes that differ from each other by the values of some parameters, and a trained AE for each class, an MAE for the family is a neural net that has learned a compact representation and associated encoder and decoder for the class-specific AEs. One application of this general concept is in research and modeling of natural evolution -- capturing the defining and the distinguishing properties across multiple species that are dynamically evolving from each other and from common ancestors. In this interim report we provide a constructive definition of MAEs, initial examples, and the motivating research directions in machine learning and biology.

Title: Geometric Generative Modeling with Noise-Conditioned Graph Networks

• Authors: Peter Pao-Huang, Mitchell Black, Xiaojie Qiu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09391
• Pdf URL: https://arxiv.org/pdf/2507.09391
• Copy Paste: [[2507.09391]] Geometric Generative Modeling with Noise-Conditioned Graph Networks(https://arxiv.org/abs/2507.09391)
• Keywords: generative
• Abstract: Generative modeling of graphs with spatial structure is essential across many applications from computer graphics to spatial genomics. Recent flow-based generative models have achieved impressive results by gradually adding and then learning to remove noise from these graphs. Existing models, however, use graph neural network architectures that are independent of the noise level, limiting their expressiveness. To address this issue, we introduce \textit{Noise-Conditioned Graph Networks} (NCGNs), a class of graph neural networks that dynamically modify their architecture according to the noise level during generation. Our theoretical and empirical analysis reveals that as noise increases, (1) graphs require information from increasingly distant neighbors and (2) graphs can be effectively represented at lower resolutions. Based on these insights, we develop Dynamic Message Passing (DMP), a specific instantiation of NCGNs that adapts both the range and resolution of message passing to the noise level. DMP consistently outperforms noise-independent architectures on a variety of domains including $3$D point clouds, spatiotemporal transcriptomics, and images. Code is available at this https URL.

Title: Scaling Laws for Optimal Data Mixtures

• Authors: Mustafa Shukor, Louis Bethune, Dan Busbridge, David Grangier, Enrico Fini, Alaaeldin El-Nouby, Pierre Ablin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09404
• Pdf URL: https://arxiv.org/pdf/2507.09404
• Copy Paste: [[2507.09404]] Scaling Laws for Optimal Data Mixtures(https://arxiv.org/abs/2507.09404)
• Keywords: foundation model
• Abstract: Large foundation models are typically trained on data from multiple domains, with the data mixture--the proportion of each domain used--playing a critical role in model performance. The standard approach to selecting this mixture relies on trial and error, which becomes impractical for large-scale pretraining. We propose a systematic method to determine the optimal data mixture for any target domain using scaling laws. Our approach accurately predicts the loss of a model of size $N$ trained with $D$ tokens and a specific domain weight vector $h$. We validate the universality of these scaling laws by demonstrating their predictive power in three distinct and large-scale settings: large language model (LLM), native multimodal model (NMM), and large vision models (LVM) pretraining. We further show that these scaling laws can extrapolate to new data mixtures and across scales: their parameters can be accurately estimated using a few small-scale training runs, and used to estimate the performance at larger scales and unseen domain weights. The scaling laws allow to derive the optimal domain weights for any target domain under a given training budget ($N$,$D$), providing a principled alternative to costly trial-and-error methods.

Title: Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers

• Authors: Santhosh Kumar Ravindran
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09406
• Pdf URL: https://arxiv.org/pdf/2507.09406
• Copy Paste: [[2507.09406]] Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers(https://arxiv.org/abs/2507.09406)
• Keywords: anomaly
• Abstract: Large language models (LLMs) aligned for safety through techniques like reinforcement learning from human feedback (RLHF) often exhibit emergent deceptive behaviors, where outputs appear compliant but subtly mislead or omit critical information. This paper introduces adversarial activation patching, a novel mechanistic interpretability framework that leverages activation patching as an adversarial tool to induce, detect, and mitigate such deception in transformer-based models. By sourcing activations from "deceptive" prompts and patching them into safe forward passes at specific layers, we simulate vulnerabilities and quantify deception rates. Through toy neural network simulations across multiple scenarios (e.g., 1000 trials per setup), we demonstrate that adversarial patching increases deceptive outputs to 23.9% from a 0% baseline, with layer-specific variations supporting our hypotheses. We propose six hypotheses, including transferability across models, exacerbation in multimodal settings, and scaling effects. An expanded literature review synthesizes over 20 key works in interpretability, deception, and adversarial attacks. Mitigation strategies, such as activation anomaly detection and robust fine-tuning, are detailed, alongside ethical considerations and future research directions. This work advances AI safety by highlighting patching's dual-use potential and provides a roadmap for empirical studies on large-scale models.

Title: Transformers Don't In-Context Learn Least Squares Regression

• Authors: Joshua Hill, Benjamin Eyre, Elliot Creager
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09440
• Pdf URL: https://arxiv.org/pdf/2507.09440
• Copy Paste: [[2507.09440]] Transformers Don't In-Context Learn Least Squares Regression(https://arxiv.org/abs/2507.09440)
• Keywords: in-context
• Abstract: In-context learning (ICL) has emerged as a powerful capability of large pretrained transformers, enabling them to solve new tasks implicit in example input-output pairs without any gradient updates. Despite its practical success, the mechanisms underlying ICL remain largely mysterious. In this work we study synthetic linear regression to probe how transformers implement learning at inference time. Previous works have demonstrated that transformers match the performance of learning rules such as Ordinary Least Squares (OLS) regression or gradient descent and have suggested ICL is facilitated in transformers through the learned implementation of one of these techniques. In this work, we demonstrate through a suite of out-of-distribution generalization experiments that transformers trained for ICL fail to generalize after shifts in the prompt distribution, a behaviour that is inconsistent with the notion of transformers implementing algorithms such as OLS. Finally, we highlight the role of the pretraining corpus in shaping ICL behaviour through a spectral analysis of the learned representations in the residual stream. Inputs from the same distribution as the training data produce representations with a unique spectral signature: inputs from this distribution tend to have the same top two singular vectors. This spectral signature is not shared by out-of-distribution inputs, and a metric characterizing the presence of this signature is highly correlated with low loss.

Title: La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching

• Authors: Tomas Geffner, Kieran Didi, Zhonglin Cao, Danny Reidenbach, Zuobai Zhang, Christian Dallago, Emine Kucukbenli, Karsten Kreis, Arash Vahdat
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2507.09466
• Pdf URL: https://arxiv.org/pdf/2507.09466
• Copy Paste: [[2507.09466]] La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching(https://arxiv.org/abs/2507.09466)
• Keywords: generative
• Abstract: Recently, many generative models for de novo protein structure design have emerged. Yet, only few tackle the difficult task of directly generating fully atomistic structures jointly with the underlying amino acid sequence. This is challenging, for instance, because the model must reason over side chains that change in length during generation. We introduce La-Proteina for atomistic protein design based on a novel partially latent protein representation: coarse backbone structure is modeled explicitly, while sequence and atomistic details are captured via per-residue latent variables of fixed dimensionality, thereby effectively side-stepping challenges of explicit side-chain representations. Flow matching in this partially latent space then models the joint distribution over sequences and full-atom structures. La-Proteina achieves state-of-the-art performance on multiple generation benchmarks, including all-atom co-designability, diversity, and structural validity, as confirmed through detailed structural analyses and evaluations. Notably, La-Proteina also surpasses previous models in atomistic motif scaffolding performance, unlocking critical atomistic structure-conditioned protein design tasks. Moreover, La-Proteina is able to generate co-designable proteins of up to 800 residues, a regime where most baselines collapse and fail to produce valid samples, demonstrating La-Proteina's scalability and robustness.

Title: Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challenges

• Authors: Yidong Jiang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09562
• Pdf URL: https://arxiv.org/pdf/2507.09562
• Copy Paste: [[2507.09562]] Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challenges(https://arxiv.org/abs/2507.09562)
• Keywords: foundation model
• Abstract: The Segment Anything Model (SAM) has revolutionized image segmentation through its innovative prompt-based approach, yet the critical role of prompt engineering in its success remains underexplored. This paper presents the first comprehensive survey focusing specifically on prompt engineering techniques for SAM and its variants. We systematically organize and analyze the rapidly growing body of work in this emerging field, covering fundamental methodologies, practical applications, and key challenges. Our review reveals how prompt engineering has evolved from simple geometric inputs to sophisticated multimodal approaches, enabling SAM's adaptation across diverse domains including medical imaging and remote sensing. We identify unique challenges in prompt optimization and discuss promising research directions. This survey fills an important gap in the literature by providing a structured framework for understanding and advancing prompt engineering in foundation models for segmentation.

Title: WordCraft: Interactive Artistic Typography with Attention Awareness and Noise Blending

• Authors: Zhe Wang, Jingbo Zhang, Tianyi Wei, Wanchao Su, Can Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09573
• Pdf URL: https://arxiv.org/pdf/2507.09573
• Copy Paste: [[2507.09573]] WordCraft: Interactive Artistic Typography with Attention Awareness and Noise Blending(https://arxiv.org/abs/2507.09573)
• Keywords: diffusion, generative
• Abstract: Artistic typography aims to stylize input characters with visual effects that are both creative and legible. Traditional approaches rely heavily on manual design, while recent generative models, particularly diffusion-based methods, have enabled automated character stylization. However, existing solutions remain limited in interactivity, lacking support for localized edits, iterative refinement, multi-character composition, and open-ended prompt interpretation. We introduce WordCraft, an interactive artistic typography system that integrates diffusion models to address these limitations. WordCraft features a training-free regional attention mechanism for precise, multi-region generation and a noise blending that supports continuous refinement without compromising visual quality. To support flexible, intent-driven generation, we incorporate a large language model to parse and structure both concrete and abstract user prompts. These components allow our framework to synthesize high-quality, stylized typography across single- and multi-character inputs across multiple languages, supporting diverse user-centered workflows. Our system significantly enhances interactivity in artistic typography synthesis, opening up creative possibilities for artists and designers.

Title: MENTOR: Efficient Multimodal-Conditioned Tuning for Autoregressive Vision Generation Models

• Authors: Haozhe Zhao, Zefan Cai, Shuzheng Si, Liang Chen, Jiuxiang Gu, Wen Xiao, Junjie Hu
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2507.09574
• Pdf URL: https://arxiv.org/pdf/2507.09574
• Copy Paste: [[2507.09574]] MENTOR: Efficient Multimodal-Conditioned Tuning for Autoregressive Vision Generation Models(https://arxiv.org/abs/2507.09574)
• Keywords: diffusion
• Abstract: Recent text-to-image models produce high-quality results but still struggle with precise visual control, balancing multimodal inputs, and requiring extensive training for complex multimodal image generation. To address these limitations, we propose MENTOR, a novel autoregressive (AR) framework for efficient Multimodal-conditioned Tuning for Autoregressive multimodal image generation. MENTOR combines an AR image generator with a two-stage training paradigm, enabling fine-grained, token-level alignment between multimodal inputs and image outputs without relying on auxiliary adapters or cross-attention modules. The two-stage training consists of: (1) a multimodal alignment stage that establishes robust pixel- and semantic-level alignment, followed by (2) a multimodal instruction tuning stage that balances the integration of multimodal inputs and enhances generation controllability. Despite modest model size, suboptimal base components, and limited training resources, MENTOR achieves strong performance on the DreamBench++ benchmark, outperforming competitive baselines in concept preservation and prompt following. Additionally, our method delivers superior image reconstruction fidelity, broad task adaptability, and improved training efficiency compared to diffusion-based methods. Dataset, code, and models are available at: this https URL

Title: Demystifying Flux Architecture

• Authors: Or Greenberg
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09595
• Pdf URL: https://arxiv.org/pdf/2507.09595
• Copy Paste: [[2507.09595]] Demystifying Flux Architecture(https://arxiv.org/abs/2507.09595)
• Keywords: diffusion
• Abstract: FLUX.1 is a diffusion-based text-to-image generation model developed by Black Forest Labs, designed to achieve faithful text-image alignment while maintaining high image quality and diversity. FLUX is considered state-of-the-art in text-to-image generation, outperforming popular models such as Midjourney, DALL-E 3, Stable Diffusion 3 (SD3), and SDXL. Although publicly available as open source, the authors have not released official technical documentation detailing the model's architecture or training setup. This report summarizes an extensive reverse-engineering effort aimed at demystifying FLUX's architecture directly from its source code, to support its adoption as a backbone for future research and development. This document is an unofficial technical report and is not published or endorsed by the original developers or their affiliated institutions.

Title: Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection

• Authors: Yilin Lu, Jianghang Lin, Linhuang Xie, Kai Zhao, Yansong Qu, Shengchuan Zhang, Liujuan Cao, Rongrong Ji
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09619
• Pdf URL: https://arxiv.org/pdf/2507.09619
• Copy Paste: [[2507.09619]] Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection(https://arxiv.org/abs/2507.09619)
• Keywords: diffusion, anomaly
• Abstract: Anomaly inspection plays a vital role in industrial manufacturing, but the scarcity of anomaly samples significantly limits the effectiveness of existing methods in tasks such as localization and classification. While several anomaly synthesis approaches have been introduced for data augmentation, they often struggle with low realism, inaccurate mask alignment, and poor generalization. To overcome these limitations, we propose Generate Aligned Anomaly (GAA), a region-guided, few-shot anomaly image-mask pair generation framework. GAA leverages the strong priors of a pretrained latent diffusion model to generate realistic, diverse, and semantically aligned anomalies using only a small number of samples. The framework first employs Localized Concept Decomposition to jointly model the semantic features and spatial information of anomalies, enabling flexible control over the type and location of anomalies. It then utilizes Adaptive Multi-Round Anomaly Clustering to perform fine-grained semantic clustering of anomaly concepts, thereby enhancing the consistency of anomaly representations. Subsequently, a region-guided mask generation strategy ensures precise alignment between anomalies and their corresponding masks, while a low-quality sample filtering module is introduced to further improve the overall quality of the generated samples. Extensive experiments on the MVTec AD and LOCO datasets demonstrate that GAA achieves superior performance in both anomaly synthesis quality and downstream tasks such as localization and classification.

Title: SpreadPy: A Python tool for modelling spreading activation and superdiffusion in cognitive multiplex networks

• Authors: Salvatore Citraro, Edith Haim, Alessandra Carini, Cynthia S. Q. Siew, Giulio Rossetti, Massimo Stella
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.09628
• Pdf URL: https://arxiv.org/pdf/2507.09628
• Copy Paste: [[2507.09628]] SpreadPy: A Python tool for modelling spreading activation and superdiffusion in cognitive multiplex networks(https://arxiv.org/abs/2507.09628)
• Keywords: diffusion
• Abstract: We introduce SpreadPy as a Python library for simulating spreading activation in cognitive single-layer and multiplex networks. Our tool is designed to perform numerical simulations testing structure-function relationships in cognitive processes. By comparing simulation results with grounded theories in knowledge modelling, SpreadPy enables systematic investigations of how activation dynamics reflect cognitive, psychological and clinical phenomena. We demonstrate the library's utility through three case studies: (1) Spreading activation on associative knowledge networks distinguishes students with high versus low math anxiety, revealing anxiety-related structural differences in conceptual organization; (2) Simulations of a creativity task show that activation trajectories vary with task difficulty, exposing how cognitive load modulates lexical access; (3) In individuals with aphasia, simulated activation patterns on lexical networks correlate with empirical error types (semantic vs. phonological) during picture-naming tasks, linking network structure to clinical impairments. SpreadPy's flexible framework allows researchers to model these processes using empirically derived or theoretical networks, providing mechanistic insights into individual differences and cognitive impairments. The library is openly available, supporting reproducible research in psychology, neuroscience, and education research.

Title: Prompt2DEM: High-Resolution DEMs for Urban and Open Environments from Global Prompts Using a Monocular Foundation Model

• Authors: Osher Rafaeli, Tal Svoray, Ariel Nahlieli
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.09681
• Pdf URL: https://arxiv.org/pdf/2507.09681
• Copy Paste: [[2507.09681]] Prompt2DEM: High-Resolution DEMs for Urban and Open Environments from Global Prompts Using a Monocular Foundation Model(https://arxiv.org/abs/2507.09681)
• Keywords: foundation model
• Abstract: High-resolution elevation estimations are essential to understand catchment and hillslope hydrology, study urban morphology and dynamics, and monitor the growth, decline, and mortality of terrestrial ecosystems. Various deep learning approaches (e.g., super-resolution techniques, monocular depth estimation) have been developed to create high-resolution Digital Elevation Models (DEMs). However, super-resolution techniques are limited by the upscaling factor, and monocular depth estimation lacks global elevation context, making its conversion to a seamless DEM restricted. The recently introduced technique of prompt-based monocular depth estimation has opened new opportunities to extract estimates of absolute elevation in a global context. We present here a framework for the estimation of high-resolution DEMs as a new paradigm for absolute global elevation mapping. It is exemplified using low-resolution Shuttle Radar Topography Mission (SRTM) elevation data as prompts and high-resolution RGB imagery from the National Agriculture Imagery Program (NAIP). The approach fine-tunes a vision transformer encoder with LiDAR-derived DEMs and employs a versatile prompting strategy, enabling tasks such as DEM estimation, void filling, and updating. Our framework achieves a 100x resolution gain (from 30-m to 30-cm), surpassing prior methods by an order of magnitude. Evaluations across three diverse U.S. landscapes show robust generalization, capturing urban structures and fine-scale terrain features with < 5 m MAE relative to LiDAR, improving over SRTM by up to 18%. Hydrological analysis confirms suitability for hazard and environmental studies. We demonstrate scalability by applying the framework to large regions in the U.S. and Israel. All code and pretrained models are publicly available at: this https URL.

Title: Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustness

• Authors: Md Mushfiqur Rahaman, Elliot Chang, Tasmiah Haque, Srinjoy Das
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.09687
• Pdf URL: https://arxiv.org/pdf/2507.09687
• Copy Paste: [[2507.09687]] Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustness(https://arxiv.org/abs/2507.09687)
• Keywords: generative
• Abstract: Text classification plays a pivotal role in edge computing applications like industrial monitoring, health diagnostics, and smart assistants, where low latency and high accuracy are both key requirements. Generative classifiers, in particular, have been shown to exhibit robustness to out-of-distribution and noisy data, which is an extremely critical consideration for deployment in such real-time edge environments. However, deploying such models on edge devices faces computational and memory constraints. Post Training Quantization (PTQ) reduces model size and compute costs without retraining, making it ideal for edge deployment. In this work, we present a comprehensive comparative study of generative and discriminative Long Short Term Memory (LSTM)-based text classification models with PTQ using the Brevitas quantization library. We evaluate both types of classifier models across multiple bitwidths and assess their robustness under regular and noisy input conditions. We find that while discriminative classifiers remain robust, generative ones are more sensitive to bitwidth, calibration data used during PTQ, and input noise during quantized inference. We study the influence of class imbalance in calibration data for both types of classifiers, comparing scenarios with evenly and unevenly distributed class samples including their effect on weight adjustments and activation profiles during PTQ. Using test statistics derived from nonparametric hypothesis testing, we identify that using class imbalanced data during calibration introduces insufficient weight adaptation at lower bitwidths for generative LSTM classifiers, thereby leading to degraded performance. This study underscores the role of calibration data in PTQ and when generative classifiers succeed or fail under noise, aiding deployment in edge environments.

Title: Continental scale habitat modelling with artificial intelligence and multimodal earth observation

• Authors: Sara Si-Moussi, Stephan Hennekens, Sander Mucher, Stan Los, Wilfried Thuiller
• Subjects: cs.LG, q-bio.PE, stat.AP
• Abstract URL: https://arxiv.org/abs/2507.09732
• Pdf URL: https://arxiv.org/pdf/2507.09732
• Copy Paste: [[2507.09732]] Continental scale habitat modelling with artificial intelligence and multimodal earth observation(https://arxiv.org/abs/2507.09732)
• Keywords: foundation model
• Abstract: Habitats integrate the abiotic conditions and biophysical structures that support biodiversity and sustain nature's contributions to people. As these ecosystems face mounting pressure from human activities, accurate, high-resolution habitat maps are essential for effective conservation and restoration. Yet current maps often fall short in thematic or spatial resolution because they must (1) model several mutually exclusive habitat types that co-occur across landscapes and (2) cope with severe class imbalance that complicate multi-class training. Here, we evaluated how high-resolution remote sensing (RS) data and Artificial Intelligence (AI) tools can improve habitat classification over large geographic extents at fine thematic resolution. Using vegetation plots from the European Vegetation Archive, we modelled Level 3 EUNIS habitats across Europe and assessed multiple modelling strategies against independent validation datasets. Strategies that exploited the hierarchical nature of habitat nomenclatures resolved classification ambiguities, especially in fragmented landscapes. Integrating multi-spectral (MSI) and synthetic aperture radar (SAR) imagery, particularly through Earth Observation Foundation models, enhanced within-formation discrimination and overall performance. Finally, ensemble machine learning that corrects class imbalance boosted accuracy further. Our methodological framework is transferable beyond Europe and adaptable to other classification systems. Future research should advance temporal modelling of dynamic habitats, extend to habitat segmentation and quality assessment, and exploit next-generation EO data paired with higher-quality in-situ observations.

Title: Universal Physics Simulation: A Foundational Diffusion Approach

• Authors: Bradley Camburn
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09733
• Pdf URL: https://arxiv.org/pdf/2507.09733
• Copy Paste: [[2507.09733]] Universal Physics Simulation: A Foundational Diffusion Approach(https://arxiv.org/abs/2507.09733)
• Keywords: diffusion
• Abstract: We present the first foundational AI model for universal physics simulation that learns physical laws directly from boundary-condition data without requiring a priori equation encoding. Traditional physics-informed neural networks (PINNs) and finite-difference methods necessitate explicit mathematical formulation of governing equations, fundamentally limiting their generalizability and discovery potential. Our sketch-guided diffusion transformer approach reimagines computational physics by treating simulation as a conditional generation problem, where spatial boundary conditions guide the synthesis of physically accurate steady-state solutions. By leveraging enhanced diffusion transformer architectures with novel spatial relationship encoding, our model achieves direct boundary-to-equilibrium mapping and is generalizable to diverse physics domains. Unlike sequential time-stepping methods that accumulate errors over iterations, our approach bypasses temporal integration entirely, directly generating steady-state solutions with SSIM > 0.8 while maintaining sub-pixel boundary accuracy. Our data-informed approach enables physics discovery through learned representations analyzable via Layer-wise Relevance Propagation (LRP), revealing emergent physical relationships without predetermined mathematical constraints. This work represents a paradigm shift from AI-accelerated physics to AI-discovered physics, establishing the first truly universal physics simulation framework.

Title: Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation

• Authors: Yu Lei, Bingde Liu, Qingsong Xie, Haonan Lu, Zhijie Deng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09748
• Pdf URL: https://arxiv.org/pdf/2507.09748
• Copy Paste: [[2507.09748]] Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation(https://arxiv.org/abs/2507.09748)
• Keywords: diffusion
• Abstract: Text-to-3D generation based on score distillation of pre-trained 2D diffusion models has gained increasing interest, with variational score distillation (VSD) as a remarkable example. VSD proves that vanilla score distillation can be improved by introducing an extra score-based model, which characterizes the distribution of images rendered from 3D models, to correct the distillation gradient. Despite the theoretical foundations, VSD, in practice, is likely to suffer from slow and sometimes ill-posed convergence. In this paper, we perform an in-depth investigation of the interplay between the introduced score model and the 3D model, and find that there exists a mismatching problem between LoRA and 3D distributions in practical implementation. We can simply adjust their optimization order to improve the generation quality. By doing so, the score model looks ahead to the current 3D state and hence yields more reasonable corrections. Nevertheless, naive lookahead VSD may suffer from unstable training in practice due to the potential over-fitting. To address this, we propose to use a linearized variant of the model for score distillation, giving rise to the Linearized Lookahead Variational Score Distillation ($L^2$-VSD). $L^2$-VSD can be realized efficiently with forward-mode autodiff functionalities of existing deep learning libraries. Extensive experiments validate the efficacy of $L^2$-VSD, revealing its clear superiority over prior score distillation-based methods. We also show that our method can be seamlessly incorporated into any other VSD-based text-to-3D framework.

Title: Do we need equivariant models for molecule generation?

• Authors: Ewa M. Nowara, Joshua Rackers, Patricia Suriana, Pan Kessel, Max Shen, Andrew Martin Watkins, Michael Maser
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2507.09753
• Pdf URL: https://arxiv.org/pdf/2507.09753
• Copy Paste: [[2507.09753]] Do we need equivariant models for molecule generation?(https://arxiv.org/abs/2507.09753)
• Keywords: generative
• Abstract: Deep generative models are increasingly used for molecular discovery, with most recent approaches relying on equivariant graph neural networks (GNNs) under the assumption that explicit equivariance is essential for generating high-quality 3D molecules. However, these models are complex, difficult to train, and scale poorly. We investigate whether non-equivariant convolutional neural networks (CNNs) trained with rotation augmentations can learn equivariance and match the performance of equivariant models. We derive a loss decomposition that separates prediction error from equivariance error, and evaluate how model size, dataset size, and training duration affect performance across denoising, molecule generation, and property prediction. To our knowledge, this is the first study to analyze learned equivariance in generative tasks.

Title: Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow

• Authors: Zhonglin Cao, Mario Geiger, Allan dos Santos Costa, Danny Reidenbach, Karsten Kreis, Tomas Geffner, Franco Pellegrini, Guoqing Zhou, Emine Kucukbenli
• Subjects: cs.LG, physics.chem-ph
• Abstract URL: https://arxiv.org/abs/2507.09785
• Pdf URL: https://arxiv.org/pdf/2507.09785
• Copy Paste: [[2507.09785]] Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow(https://arxiv.org/abs/2507.09785)
• Keywords: diffusion, generative
• Abstract: Fast and accurate generation of molecular conformers is desired for downstream computational chemistry and drug discovery tasks. Currently, training and sampling state-of-the-art diffusion or flow-based models for conformer generation require significant computational resources. In this work, we build upon flow-matching and propose two mechanisms for accelerating training and inference of generative models for 3D molecular conformer generation. For fast training, we introduce the SO(3)-Averaged Flow training objective, which leads to faster convergence to better generation quality compared to conditional optimal transport flow or Kabsch-aligned flow. We demonstrate that models trained using SO(3)-Averaged Flow can reach state-of-the-art conformer generation quality. For fast inference, we show that the reflow and distillation methods of flow-based models enable few-steps or even one-step molecular conformer generation with high quality. The training techniques proposed in this work show a path towards highly efficient molecular conformer generation with flow-based models.

Title: Generative Cognitive Diagnosis

• Authors: Jiatong Li, Qi Liu, Mengxiao Zhu
• Subjects: cs.LG, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2507.09831
• Pdf URL: https://arxiv.org/pdf/2507.09831
• Copy Paste: [[2507.09831]] Generative Cognitive Diagnosis(https://arxiv.org/abs/2507.09831)
• Keywords: generative
• Abstract: Cognitive diagnosis (CD) models latent cognitive states of human learners by analyzing their response patterns on diagnostic tests, serving as a crucial machine learning technique for educational assessment and evaluation. Traditional cognitive diagnosis models typically follow a transductive prediction paradigm that optimizes parameters to fit response scores and extract learner abilities. These approaches face significant limitations as they cannot perform instant diagnosis for new learners without computationally expensive retraining and produce diagnostic outputs with limited reliability. In this study, we introduces a novel generative diagnosis paradigm that fundamentally shifts CD from predictive to generative modeling, enabling inductive inference of cognitive states without parameter re-optimization. We propose two simple yet effective instantiations of this paradigm: Generative Item Response Theory (G-IRT) and Generative Neural Cognitive Diagnosis Model (G-NCDM), which achieve excellent performance improvements over traditional methods. The generative approach disentangles cognitive state inference from response prediction through a well-designed generation process that incorporates identifiability and monotonicity conditions. Extensive experiments on real-world datasets demonstrate the effectiveness of our methodology in addressing scalability and reliability challenges, especially $\times 100$ speedup for the diagnosis of new learners. Our framework opens new avenues for cognitive diagnosis applications in artificial intelligence, particularly for intelligent model evaluation and intelligent education systems. The code is available at this https URL.

Title: Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition

• Authors: Qinyuan Ye, Robin Jia, Xiang Ren
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09875
• Pdf URL: https://arxiv.org/pdf/2507.09875
• Copy Paste: [[2507.09875]] Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition(https://arxiv.org/abs/2507.09875)
• Keywords: in-context
• Abstract: Large language models demonstrate the intriguing ability to perform unseen tasks via in-context learning. However, it remains unclear what mechanisms inside the model drive such task-level generalization. In this work, we approach this question through the lens of off-by-one addition (i.e., 1+1=3, 2+2=5, 3+3=?), a two-step, counterfactual task with an unexpected +1 function as a second step. Leveraging circuit-style interpretability techniques such as path patching, we analyze the models' internal computations behind their notable performance and present three key findings. First, we uncover a function induction mechanism that explains the model's generalization from standard addition to off-by-one addition. This mechanism resembles the structure of the induction head mechanism found in prior work and elevates it to a higher level of abstraction. Second, we show that the induction of the +1 function is governed by multiple attention heads in parallel, each of which emits a distinct piece of the +1 function. Finally, we find that this function induction mechanism is reused in a broader range of tasks, including synthetic tasks such as shifted multiple-choice QA and algorithmic tasks such as base-8 addition. Overall, our findings offer deeper insights into how reusable and composable structures within language models enable task-level generalization.

Title: AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications

• Authors: Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, Chunfeng Song
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.09882
• Pdf URL: https://arxiv.org/pdf/2507.09882
• Copy Paste: [[2507.09882]] AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications(https://arxiv.org/abs/2507.09882)
• Keywords: self-supervised, foundation model
• Abstract: Non-invasive Brain-Computer Interfaces (BCI) offer a safe and accessible means of connecting the human brain to external devices, with broad applications in home and clinical settings to enhance human capabilities. However, the high noise level and limited task-specific data in non-invasive signals constrain decoding capabilities. Recently, the adoption of self-supervised pre-training is transforming the landscape of non-invasive BCI research, enabling the development of brain foundation models to capture generic neural representations from large-scale unlabeled electroencephalography (EEG) signals with substantial noises. However, despite these advances, the field currently lacks comprehensive, practical and extensible benchmarks to assess the utility of the public foundation models across diverse BCI tasks, hindering their widespread adoption. To address this challenge, we present AdaBrain-Bench, a large-scale standardized benchmark to systematically evaluate brain foundation models in widespread non-invasive BCI tasks. AdaBrain-Bench encompasses a diverse collection of representative BCI decoding datasets spanning 7 key applications. It introduces a streamlined task adaptation pipeline integrated with multi-dimensional evaluation metrics and a set of adaptation tools. The benchmark delivers an inclusive framework for assessing generalizability of brain foundation models across key transfer settings, including cross-subject, multi-subject, and few-shot scenarios. We leverage AdaBrain-Bench to evaluate a suite of publicly available brain foundation models and offer insights into practices for selecting appropriate models in various scenarios. We make our benchmark pipeline available to enable reproducible research and external use, offering a continuously evolving platform to foster progress toward robust and generalized neural decoding solutions.

Title: TolerantECG: A Foundation Model for Imperfect Electrocardiogram

• Authors: Huynh Nguyen Dang, Thang Pham, Ngan Le, Van Nguyen
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2507.09887
• Pdf URL: https://arxiv.org/pdf/2507.09887
• Copy Paste: [[2507.09887]] TolerantECG: A Foundation Model for Imperfect Electrocardiogram(https://arxiv.org/abs/2507.09887)
• Keywords: self-supervised, foundation model
• Abstract: The electrocardiogram (ECG) is an essential and effective tool for diagnosing heart diseases. However, its effectiveness can be compromised by noise or unavailability of one or more leads of the standard 12-lead recordings, resulting in diagnostic errors or uncertainty. To address these challenges, we propose TolerantECG, a foundation model for ECG signals that is robust to noise and capable of functioning with arbitrary subsets of the standard 12-lead ECG. TolerantECG training combines contrastive and self-supervised learning frameworks to jointly learn ECG signal representations alongside their corresponding knowledge-retrieval-based text report descriptions and corrupted or lead-missing signals. Comprehensive benchmarking results demonstrate that TolerantECG consistently ranks as the best or second-best performer across various ECG signal conditions and class levels in the PTB-XL dataset, and achieves the highest performance on the MIT-BIH Arrhythmia Database.

Title: IGD: Instructional Graphic Design with Multimodal Layer Generation

• Authors: Yadong Qu, Shancheng Fang, Yuxin Wang, Xiaorui Wang, Zhineng Chen, Hongtao Xie, Yongdong Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09910
• Pdf URL: https://arxiv.org/pdf/2507.09910
• Copy Paste: [[2507.09910]] IGD: Instructional Graphic Design with Multimodal Layer Generation(https://arxiv.org/abs/2507.09910)
• Keywords: diffusion
• Abstract: Graphic design visually conveys information and data by creating and combining text, images and graphics. Two-stage methods that rely primarily on layout generation lack creativity and intelligence, making graphic design still labor-intensive. Existing diffusion-based methods generate non-editable graphic design files at image level with poor legibility in visual text rendering, which prevents them from achieving satisfactory and practical automated graphic design. In this paper, we propose Instructional Graphic Designer (IGD) to swiftly generate multimodal layers with editable flexibility with only natural language instructions. IGD adopts a new paradigm that leverages parametric rendering and image asset generation. First, we develop a design platform and establish a standardized format for multi-scenario design files, thus laying the foundation for scaling up data. Second, IGD utilizes the multimodal understanding and reasoning capabilities of MLLM to accomplish attribute prediction, sequencing and layout of layers. It also employs a diffusion model to generate image content for assets. By enabling end-to-end training, IGD architecturally supports scalability and extensibility in complex graphic design tasks. The superior experimental results demonstrate that IGD offers a new solution for graphic design.

Title: Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios

• Authors: Siyue Yao, Mingjie Sun, Eng Gee Lim, Ran Yi, Baojiang Zhong, Moncef Gabbouj
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09915
• Pdf URL: https://arxiv.org/pdf/2507.09915
• Copy Paste: [[2507.09915]] Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios(https://arxiv.org/abs/2507.09915)
• Keywords: diffusion, generative
• Abstract: The scarcity of data in various scenarios, such as medical, industry and autonomous driving, leads to model overfitting and dataset imbalance, thus hindering effective detection and segmentation performance. Existing studies employ the generative models to synthesize more training samples to mitigate data scarcity. However, these synthetic samples are repetitive or simplistic and fail to provide "crucial information" that targets the downstream model's weaknesses. Additionally, these methods typically require separate training for different objects, leading to computational inefficiencies. To address these issues, we propose Crucial-Diff, a domain-agnostic framework designed to synthesize crucial samples. Our method integrates two key modules. The Scene Agnostic Feature Extractor (SAFE) utilizes a unified feature extractor to capture target information. The Weakness Aware Sample Miner (WASM) generates hard-to-detect samples using feedback from the detection results of downstream model, which is then fused with the output of SAFE module. Together, our Crucial-Diff framework generates diverse, high-quality training data, achieving a pixel-level AP of 83.63% and an F1-MAX of 78.12% on MVTec. On polyp dataset, Crucial-Diff reaches an mIoU of 81.64% and an mDice of 87.69%. Code will be released after acceptance.

Title: Iceberg: Enhancing HLS Modeling with Synthetic Data

• Authors: Zijian Ding, Tung Nguyen, Weikai Li, Aditya Grover, Yizhou Sun, Jason Cong
• Subjects: cs.LG, cs.AR
• Abstract URL: https://arxiv.org/abs/2507.09948
• Pdf URL: https://arxiv.org/pdf/2507.09948
• Copy Paste: [[2507.09948]] Iceberg: Enhancing HLS Modeling with Synthetic Data(https://arxiv.org/abs/2507.09948)
• Keywords: in-context
• Abstract: Deep learning-based prediction models for High-Level Synthesis (HLS) of hardware designs often struggle to generalize. In this paper, we study how to close the generalizability gap of these models through pretraining on synthetic data and introduce Iceberg, a synthetic data augmentation approach that expands both large language model (LLM)-generated programs and weak labels of unseen design configurations. Our weak label generation method is integrated with an in-context model architecture, enabling meta-learning from actual and proximate labels. Iceberg improves the geometric mean modeling accuracy by $86.4\%$ when adapt to six real-world applications with few-shot examples and achieves a $2.47\times$ and a $1.12\times$ better offline DSE performance when adapting to two different test datasets. Our open-sourced code is here: \href{this https URL}{this https URL}

Title: TextOmics-Guided Diffusion for Hit-like Molecular Generation

• Authors: Hang Yuan, Chen Li, Wenjun Ma, Yuncheng Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.09982
• Pdf URL: https://arxiv.org/pdf/2507.09982
• Copy Paste: [[2507.09982]] TextOmics-Guided Diffusion for Hit-like Molecular Generation(https://arxiv.org/abs/2507.09982)
• Keywords: diffusion, generative
• Abstract: Hit-like molecular generation with therapeutic potential is essential for target-specific drug discovery. However, the field lacks heterogeneous data and unified frameworks for integrating diverse molecular representations. To bridge this gap, we introduce TextOmics, a pioneering benchmark that establishes one-to-one correspondences between omics expressions and molecular textual descriptions. TextOmics provides a heterogeneous dataset that facilitates molecular generation through representations alignment. Built upon this foundation, we propose ToDi, a generative framework that jointly conditions on omics expressions and molecular textual descriptions to produce biologically relevant, chemically valid, hit-like molecules. ToDi leverages two encoders (OmicsEn and TextEn) to capture multi-level biological and semantic associations, and develops conditional diffusion (DiffGen) for controllable generation. Extensive experiments confirm the effectiveness of TextOmics and demonstrate ToDi outperforms existing state-of-the-art approaches, while also showcasing remarkable potential in zero-shot therapeutic molecular generation. Sources are available at: this https URL.

Title: Latent Diffusion Models with Masked AutoEncoders

• Authors: Junho Lee, Jeongwoo Shin, Hyungwook Choi, Joonseok Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.09984
• Pdf URL: https://arxiv.org/pdf/2507.09984
• Copy Paste: [[2507.09984]] Latent Diffusion Models with Masked AutoEncoders(https://arxiv.org/abs/2507.09984)
• Keywords: diffusion
• Abstract: In spite of remarkable potential of the Latent Diffusion Models (LDMs) in image generation, the desired properties and optimal design of the autoencoders have been underexplored. In this work, we analyze the role of autoencoders in LDMs and identify three key properties: latent smoothness, perceptual compression quality, and reconstruction quality. We demonstrate that existing autoencoders fail to simultaneously satisfy all three properties, and propose Variational Masked AutoEncoders (VMAEs), taking advantage of the hierarchical features maintained by Masked AutoEncoder. We integrate VMAEs into the LDM framework, introducing Latent Diffusion Models with Masked AutoEncoders (LDMAEs). Through comprehensive experiments, we demonstrate significantly enhanced image generation quality and computational efficiency.

Title: Leveraging Swin Transformer for enhanced diagnosis of Alzheimer's disease using multi-shell diffusion MRI

• Authors: Quentin Dessain, Nicolas Delinte, Bernard Hanseeuw, Laurence Dricot, Benoît Macq
• Subjects: cs.CV, q-bio.NC, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2507.09996
• Pdf URL: https://arxiv.org/pdf/2507.09996
• Copy Paste: [[2507.09996]] Leveraging Swin Transformer for enhanced diagnosis of Alzheimer's disease using multi-shell diffusion MRI(https://arxiv.org/abs/2507.09996)
• Keywords: diffusion
• Abstract: Objective: This study aims to support early diagnosis of Alzheimer's disease and detection of amyloid accumulation by leveraging the microstructural information available in multi-shell diffusion MRI (dMRI) data, using a vision transformer-based deep learning framework. Methods: We present a classification pipeline that employs the Swin Transformer, a hierarchical vision transformer model, on multi-shell dMRI data for the classification of Alzheimer's disease and amyloid presence. Key metrics from DTI and NODDI were extracted and projected onto 2D planes to enable transfer learning with ImageNet-pretrained models. To efficiently adapt the transformer to limited labeled neuroimaging data, we integrated Low-Rank Adaptation. We assessed the framework on diagnostic group prediction (cognitively normal, mild cognitive impairment, Alzheimer's disease dementia) and amyloid status classification. Results: The framework achieved competitive classification results within the scope of multi-shell dMRI-based features, with the best balanced accuracy of 95.2% for distinguishing cognitively normal individuals from those with Alzheimer's disease dementia using NODDI metrics. For amyloid detection, it reached 77.2% balanced accuracy in distinguishing amyloid-positive mild cognitive impairment/Alzheimer's disease dementia subjects from amyloid-negative cognitively normal subjects, and 67.9% for identifying amyloid-positive individuals among cognitively normal subjects. Grad-CAM-based explainability analysis identified clinically relevant brain regions, including the parahippocampal gyrus and hippocampus, as key contributors to model predictions. Conclusion: This study demonstrates the promise of diffusion MRI and transformer-based architectures for early detection of Alzheimer's disease and amyloid pathology, supporting biomarker-driven diagnostics in data-limited biomedical settings.

Title: (Almost) Free Modality Stitching of Foundation Models

• Authors: Jaisidh Singh, Diganta Misra, Boris Knyazev, Antonio Orvieto
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10015
• Pdf URL: https://arxiv.org/pdf/2507.10015
• Copy Paste: [[2507.10015]] (Almost) Free Modality Stitching of Foundation Models(https://arxiv.org/abs/2507.10015)
• Keywords: foundation model
• Abstract: Foundation multi-modal models are often designed by stitching of multiple existing pretrained uni-modal models: for example, an image classifier with an autoregressive text model. This stitching process is performed by training a connector module that aims to align the representation-representation or representation-input spaces of these uni-modal models. However, given the complexity of training such connectors on large scale web-based datasets coupled with the ever-increasing number of available pretrained uni-modal models, the task of uni-modal models selection and subsequent connector module training becomes computationally demanding. To address this under-studied critical problem, we propose Hypernetwork Model Alignment (Hyma), a novel all-in-one solution for optimal uni-modal model selection and connector training by leveraging hypernetworks. Specifically, our framework utilizes the parameter prediction capability of a hypernetwork to obtain jointly trained connector modules for $N \times M$ combinations of uni-modal models. In our experiments, Hyma reduces the optimal uni-modal model pair search cost by $10\times$ (averaged across all experiments), while matching the ranking and trained connector performance obtained via grid search across a suite of diverse multi-modal benchmarks.

Title: Memory-Efficient Personalization of Text-to-Image Diffusion Models via Selective Optimization Strategies

• Authors: Seokeon Choi, Sunghyun Park, Hyoungwoo Park, Jeongho Kim, Sungrack Yun
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10029
• Pdf URL: https://arxiv.org/pdf/2507.10029
• Copy Paste: [[2507.10029]] Memory-Efficient Personalization of Text-to-Image Diffusion Models via Selective Optimization Strategies(https://arxiv.org/abs/2507.10029)
• Keywords: diffusion
• Abstract: Memory-efficient personalization is critical for adapting text-to-image diffusion models while preserving user privacy and operating within the limited computational resources of edge devices. To this end, we propose a selective optimization framework that adaptively chooses between backpropagation on low-resolution images (BP-low) and zeroth-order optimization on high-resolution images (ZO-high), guided by the characteristics of the diffusion process. As observed in our experiments, BP-low efficiently adapts the model to target-specific features, but suffers from structural distortions due to resolution mismatch. Conversely, ZO-high refines high-resolution details with minimal memory overhead but faces slow convergence when applied without prior adaptation. By complementing both methods, our framework leverages BP-low for effective personalization while using ZO-high to maintain structural consistency, achieving memory-efficient and high-quality fine-tuning. To maximize the efficacy of both BP-low and ZO-high, we introduce a timestep-aware probabilistic function that dynamically selects the appropriate optimization strategy based on diffusion timesteps. This function mitigates the overfitting from BP-low at high timesteps, where structural information is critical, while ensuring ZO-high is applied more effectively as training progresses. Experimental results demonstrate that our method achieves competitive performance while significantly reducing memory consumption, enabling scalable, high-quality on-device personalization without increasing inference latency.

Title: Towards Applying Large Language Models to Complement Single-Cell Foundation Models

• Authors: Steven Palayew, Bo Wang, Gary Bader
• Subjects: cs.LG, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2507.10039
• Pdf URL: https://arxiv.org/pdf/2507.10039
• Copy Paste: [[2507.10039]] Towards Applying Large Language Models to Complement Single-Cell Foundation Models(https://arxiv.org/abs/2507.10039)
• Keywords: foundation model
• Abstract: Single-cell foundation models such as scGPT represent a significant advancement in single-cell omics, with an ability to achieve state-of-the-art performance on various downstream biological tasks. However, these models are inherently limited in that a vast amount of information in biology exists as text, which they are unable to leverage. There have therefore been several recent works that propose the use of LLMs as an alternative to single-cell foundation models, achieving competitive results. However, there is little understanding of what factors drive this performance, along with a strong focus on using LLMs as an alternative, rather than complementary approach to single-cell foundation models. In this study, we therefore investigate what biological insights contribute toward the performance of LLMs when applied to single-cell data, and introduce scMPT; a model which leverages synergies between scGPT, and single-cell representations from LLMs that capture these insights. scMPT demonstrates stronger, more consistent performance than either of its component models, which frequently have large performance gaps between each other across datasets. We also experiment with alternate fusion methods, demonstrating the potential of combining specialized reasoning models with scGPT to improve performance. This study ultimately showcases the potential for LLMs to complement single-cell foundation models and drive improvements in single-cell analysis.

Title: GeLaCo: An Evolutionary Approach to Layer Compression

• Authors: David Ponce, Thierry Etchegoyhen, Javier Del Ser
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.10059
• Pdf URL: https://arxiv.org/pdf/2507.10059
• Copy Paste: [[2507.10059]] GeLaCo: An Evolutionary Approach to Layer Compression(https://arxiv.org/abs/2507.10059)
• Keywords: generative
• Abstract: Large Language Models (LLM) have achieved remarkable performance across a large number of tasks, but face critical deployment and usage barriers due to substantial computational requirements. Model compression methods, which aim to reduce model size while preserving its capacity, are an important means to mitigate these issues. Promising approaches along these lines, such as structured pruning, typically require costly empirical search for optimal variants and may run the risk of ignoring better solutions. In this work we introduce GeLaCo, an evolutionary approach to LLM compression via layer collapse. Our approach supports an efficient exploration of the compression solution space via population-based search and a module-wise similarity fitness function capturing attention, feed-forward, and hidden state representations. GeLaCo also supports both single and multi-objective evolutionary compression search, establishing the first Pareto frontier along compression and quality axes. We evaluate GeLaCo solutions via both perplexity-based and generative evaluations over foundational and instruction-tuned models, outperforming state-of-the-art alternatives.

Title: Frequency Regulation for Exposure Bias Mitigation in Diffusion Models

• Authors: Meng Yu, Kun Zhan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10072
• Pdf URL: https://arxiv.org/pdf/2507.10072
• Copy Paste: [[2507.10072]] Frequency Regulation for Exposure Bias Mitigation in Diffusion Models(https://arxiv.org/abs/2507.10072)
• Keywords: diffusion, generative
• Abstract: Diffusion models exhibit impressive generative capabilities but are significantly impacted by exposure bias. In this paper, we make a key observation: the energy of the predicted noisy images decreases during the diffusion process. Building on this, we identify two important findings: 1) The reduction in energy follows distinct patterns in the low-frequency and high-frequency subbands; 2) This energy reduction results in amplitude variations between the network-reconstructed clean data and the real clean data. Based on the first finding, we introduce a frequency-domain regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands. Leveraging the second insight, we provide a more accurate analysis of exposure bias in the two subbands. Our method is training-free and plug-and-play, significantly improving the generative quality of various diffusion models and providing a robust solution to exposure bias across different model architectures. The source code is available at this https URL.

Title: FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text

• Authors: Bingchao Wang, Zhiwei Ning, Jianyu Ding, Xuanang Gao, Yin Li, Dongsheng Jiang, Jie Yang, Wei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10095
• Pdf URL: https://arxiv.org/pdf/2507.10095
• Copy Paste: [[2507.10095]] FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text(https://arxiv.org/abs/2507.10095)
• Keywords: diffusion
• Abstract: CLIP has shown promising performance across many short-text tasks in a zero-shot manner. However, limited by the input length of the text encoder, CLIP struggles on under-stream tasks with long-text inputs (>77 tokens). To remedy this issue, we propose FIX-CLIP which includes three novel modules: (1) A dual-branch training pipeline that aligns short and long texts with masked and raw images respectively, which boosts the long-text representation while preserving the short-text ability. (2) Multiple learnable regional prompts with unidirectional masks in Transformer layers for regional information extraction. (3) A hierarchical feature alignment module in the intermediate encoder layers to promote the consistency of multi-scale features. Furthermore, we collect 30M images and utilize existing MLLMs to synthesize long-text captions for training. Extensive experiments show that FIX-CLIP achieves state-of-the-art performance on both long-text and short-text retrieval benchmarks. For downstream applications, we reveal that FIX-CLIP's text encoder delivers promising performance in a plug-and-play manner for diffusion models with long-text input.

Title: DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation

• Authors: Ivan Martinović, Josip Šarić, Marin Oršić, Matej Kristan, Siniša Šegvić
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10118
• Pdf URL: https://arxiv.org/pdf/2507.10118
• Copy Paste: [[2507.10118]] DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation(https://arxiv.org/abs/2507.10118)
• Keywords: foundation model
• Abstract: Pixel-level annotation is expensive and time-consuming. Semi-supervised segmentation methods address this challenge by learning models on few labeled images alongside a large corpus of unlabeled images. Although foundation models could further account for label scarcity, effective mechanisms for their exploitation remain underexplored. We address this by devising a novel semi-supervised panoptic approach fueled by two dedicated foundation models. We enhance recognition by complementing unsupervised mask-transformer consistency with zero-shot classification of CLIP features. We enhance localization by class-agnostic decoder warm-up with respect to SAM pseudo-labels. The resulting decoupled enhancement of recognition and localization (DEARLi) particularly excels in the most challenging semi-supervised scenarios with large taxonomies and limited labeled data. Moreover, DEARLi outperforms the state of the art in semi-supervised semantic segmentation by a large margin while requiring 8x less GPU memory, in spite of being trained only for the panoptic objective. We observe 29.9 PQ and 38.9 mIoU on ADE20K with only 158 labeled images. The source code is available at this https URL.

Title: Task-Based Flexible Feature Distillation for LLMs

• Authors: Khouloud Saadi, Di Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.10155
• Pdf URL: https://arxiv.org/pdf/2507.10155
• Copy Paste: [[2507.10155]] Task-Based Flexible Feature Distillation for LLMs(https://arxiv.org/abs/2507.10155)
• Keywords: generative
• Abstract: Knowledge Distillation (KD) in general and feature distillation in particular are promising techniques for reducing the high computational demand of large language models (LLMs). However, traditional feature KD methods typically assume that the teacher and the student share the same hidden size, limiting the flexibility of the student's architecture. A common solution to this problem involves training a linear projector to align their feature spaces, but this introduces additional parameters that must be learned from scratch and often degrades performance on downstream tasks, especially in generative settings. To address this issue, in this work, we propose a novel task-based feature distillation method that enables knowledge transfer between teacher and student models with different hidden layer dimensions, without introducing any new parameters. Leveraging the insight that only a subset of LLM components contribute significantly to a specific downstream task, our approach identifies the most task-relevant hidden units in the teacher and directly distills their activations to the student. Our method is flexible and easily integrates with other distillation frameworks. Empirical results show consistent improvements over prior approaches across diverse tasks, including classification, instruction-following, and summarization, achieving up to a 3\% performance gain over the linear projection baseline.

Title: HASSLE: A Self-Supervised Learning Enhanced Hijacking Attack on Vertical Federated Learning

• Authors: Weiyang He, Chip-Hong Chang
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2507.10162
• Pdf URL: https://arxiv.org/pdf/2507.10162
• Copy Paste: [[2507.10162]] HASSLE: A Self-Supervised Learning Enhanced Hijacking Attack on Vertical Federated Learning(https://arxiv.org/abs/2507.10162)
• Keywords: self-supervised
• Abstract: Vertical Federated Learning (VFL) enables an orchestrating active party to perform a machine learning task by cooperating with passive parties that provide additional task-related features for the same training data entities. While prior research has leveraged the privacy vulnerability of VFL to compromise its integrity through a combination of label inference and backdoor attacks, their effectiveness is constrained by the low label inference precision and suboptimal backdoor injection conditions. To facilitate a more rigorous security evaluation on VFL without these limitations, we propose HASSLE, a hijacking attack framework composed of a gradient-direction-based label inference module and an adversarial embedding generation algorithm enhanced by self-supervised learning. HASSLE accurately identifies private samples associated with a targeted label using only a single known instance of that label. In the two-party scenario, it demonstrates strong performance with an attack success rate (ASR) of over 99% across four datasets, including both image and tabular modalities, and achieves 85% ASR on the more complex CIFAR-100 dataset. Evaluation of HASSLE against 8 potential defenses further highlights its significant threat while providing new insights into building a trustworthy VFL system.

Title: Minimizing the Pretraining Gap: Domain-aligned Text-Based Person Retrieval

• Authors: Shuyu Yang, Yaxiong Wang, Yongrui Li, Li Zhu, Zhedong Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10195
• Pdf URL: https://arxiv.org/pdf/2507.10195
• Copy Paste: [[2507.10195]] Minimizing the Pretraining Gap: Domain-aligned Text-Based Person Retrieval(https://arxiv.org/abs/2507.10195)
• Keywords: diffusion
• Abstract: In this work, we focus on text-based person retrieval, which aims to identify individuals based on textual descriptions. Given the significant privacy issues and the high cost associated with manual annotation, synthetic data has become a popular choice for pretraining models, leading to notable advancements. However, the considerable domain gap between synthetic pretraining datasets and real-world target datasets, characterized by differences in lighting, color, and viewpoint, remains a critical obstacle that hinders the effectiveness of the pretrain-finetune paradigm. To bridge this gap, we introduce a unified text-based person retrieval pipeline considering domain adaptation at both image and region levels. In particular, it contains two primary components, i.e., Domain-aware Diffusion (DaD) for image-level adaptation and Multi-granularity Relation Alignment (MRA) for region-level adaptation. As the name implies, Domain-aware Diffusion is to migrate the distribution of images from the pretraining dataset domain to the target real-world dataset domain, e.g., CUHK-PEDES. Subsequently, MRA performs a meticulous region-level alignment by establishing correspondences between visual regions and their descriptive sentences, thereby addressing disparities at a finer granularity. Extensive experiments show that our dual-level adaptation method has achieved state-of-the-art results on the CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets, outperforming existing methodologies. The dataset, model, and code are available at this https URL.

Title: From Wardrobe to Canvas: Wardrobe Polyptych LoRA for Part-level Controllable Human Image Generation

• Authors: Jeongho Kim, Sunghyun Park, Hyoungwoo Park, Sungrack Yun, Jaegul Choo, Seokeon Cho
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10217
• Pdf URL: https://arxiv.org/pdf/2507.10217
• Copy Paste: [[2507.10217]] From Wardrobe to Canvas: Wardrobe Polyptych LoRA for Part-level Controllable Human Image Generation(https://arxiv.org/abs/2507.10217)
• Keywords: diffusion
• Abstract: Recent diffusion models achieve personalization by learning specific subjects, allowing learned attributes to be integrated into generated images. However, personalized human image generation remains challenging due to the need for precise and consistent attribute preservation (e.g., identity, clothing details). Existing subject-driven image generation methods often require either (1) inference-time fine-tuning with few images for each new subject or (2) large-scale dataset training for generalization. Both approaches are computationally expensive and impractical for real-time applications. To address these limitations, we present Wardrobe Polyptych LoRA, a novel part-level controllable model for personalized human image generation. By training only LoRA layers, our method removes the computational burden at inference while ensuring high-fidelity synthesis of unseen subjects. Our key idea is to condition the generation on the subject's wardrobe and leverage spatial references to reduce information loss, thereby improving fidelity and consistency. Additionally, we introduce a selective subject region loss, which encourages the model to disregard some of reference images during training. Our loss ensures that generated images better align with text prompts while maintaining subject integrity. Notably, our Wardrobe Polyptych LoRA requires no additional parameters at the inference stage and performs generation using a single model trained on a few training samples. We construct a new dataset and benchmark tailored for personalized human image generation. Extensive experiments show that our approach significantly outperforms existing techniques in fidelity and consistency, enabling realistic and identity-preserving full-body synthesis.

Title: Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection

• Authors: Jinglun Li, Kaixun Jiang, Zhaoyu Chen, Bo Lin, Yao Tang, Weifeng Ge, Wenqiang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10225
• Pdf URL: https://arxiv.org/pdf/2507.10225
• Copy Paste: [[2507.10225]] Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection(https://arxiv.org/abs/2507.10225)
• Keywords: diffusion, foundation model
• Abstract: Pre-trained vision-language models have exhibited remarkable abilities in detecting out-of-distribution (OOD) samples. However, some challenging OOD samples, which lie close to in-distribution (InD) data in image feature space, can still lead to misclassification. The emergence of foundation models like diffusion models and multimodal large language models (MLLMs) offers a potential solution to this issue. In this work, we propose SynOOD, a novel approach that harnesses foundation models to generate synthetic, challenging OOD data for fine-tuning CLIP models, thereby enhancing boundary-level discrimination between InD and OOD samples. Our method uses an iterative in-painting process guided by contextual prompts from MLLMs to produce nuanced, boundary-aligned OOD samples. These samples are refined through noise adjustments based on gradients from OOD scores like the energy score, effectively sampling from the InD/OOD boundary. With these carefully synthesized images, we fine-tune the CLIP image encoder and negative label features derived from the text encoder to strengthen connections between near-boundary OOD samples and a set of negative labels. Finally, SynOOD achieves state-of-the-art performance on the large-scale ImageNet benchmark, with minimal increases in parameters and runtime. Our approach significantly surpasses existing methods, improving AUROC by 2.80% and reducing FPR95 by 11.13%. Codes are available in this https URL.

Title: Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?

• Authors: Despina Konstantinidou, Dimitrios Karageorgiou, Christos Koutlis, Olga Papadopoulou, Emmanouil Schinas, Symeon Papadopoulos
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10236
• Pdf URL: https://arxiv.org/pdf/2507.10236
• Copy Paste: [[2507.10236]] Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?(https://arxiv.org/abs/2507.10236)
• Keywords: generative
• Abstract: The rapid advancement of generative technologies presents both unprecedented creative opportunities and significant challenges, particularly in maintaining social trust and ensuring the integrity of digital information. Following these concerns, the challenge of AI-Generated Image Detection (AID) becomes increasingly critical. As these technologies become more sophisticated, the quality of AI-generated images has reached a level that can easily deceive even the most discerning observers. Our systematic evaluation highlights a critical weakness in current AI-Generated Image Detection models: while they perform exceptionally well on controlled benchmark datasets, they struggle significantly with real-world variations. To assess this, we introduce ITW-SM, a new dataset of real and AI-generated images collected from major social media platforms. In this paper, we identify four key factors that influence AID performance in real-world scenarios: backbone architecture, training data composition, pre-processing strategies and data augmentation combinations. By systematically analyzing these components, we shed light on their impact on detection efficacy. Our modifications result in an average AUC improvement of 26.87% across various AID models under real-world conditions.

Title: Kernel-Adaptive PI-ELMs for Forward and Inverse Problems in PDEs with Sharp Gradients

• Authors: Vikas Dwivedi, Balaji Srinivasan, Monica Sigovan, Bruno Sixou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10241
• Pdf URL: https://arxiv.org/pdf/2507.10241
• Copy Paste: [[2507.10241]] Kernel-Adaptive PI-ELMs for Forward and Inverse Problems in PDEs with Sharp Gradients(https://arxiv.org/abs/2507.10241)
• Keywords: diffusion
• Abstract: This paper introduces the Kernel Adaptive Physics-Informed Extreme Learning Machine (KAPI-ELM), an adaptive Radial Basis Function (RBF)-based extension of PI-ELM designed to solve both forward and inverse Partial Differential Equation (PDE) problems involving localized sharp gradients. While PI-ELMs outperform the traditional Physics-Informed Neural Networks (PINNs) in speed due to their single-shot, least square optimization, this advantage comes at a cost: their fixed, randomly initialized input layer limits their ability to capture sharp gradients. To overcome this limitation, we introduce a lightweight Bayesian Optimization (BO) framework that, instead of adjusting each input layer parameter individually as in traditional backpropagation, learns a small set of hyperparameters defining the statistical distribution from which the input weights are drawn. This novel distributional optimization strategy -- combining BO for input layer distributional parameters with least-squares optimization for output layer network parameters -- enables KAPI-ELM to preserve PI-ELM's speed while matching or exceeding the expressiveness of PINNs. We validate the proposed methodology on several challenging forward and inverse PDE benchmarks, including a 1D singularly perturbed convection-diffusion equation, a 2D Poisson equation with sharp localized sources, and a time-dependent advection equation. Notably, KAPI-ELM achieves state-of-the-art accuracy in both forward and inverse settings. In stiff PDE regimes, it matches or even outperforms advanced methods such as the Extended Theory of Functional Connections (XTFC), while requiring nearly an order of magnitude fewer tunable parameters. These results establish the potential of KAPI-ELM as a scalable, interpretable, and generalizable physics-informed learning framework, especially in stiff PDE regimes.

Title: Conditional Chemical Language Models are Versatile Tools in Drug Discovery

• Authors: Lu Zhu, Emmanuel Noutahi
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2507.10273
• Pdf URL: https://arxiv.org/pdf/2507.10273
• Copy Paste: [[2507.10273]] Conditional Chemical Language Models are Versatile Tools in Drug Discovery(https://arxiv.org/abs/2507.10273)
• Keywords: generative
• Abstract: Generative chemical language models (CLMs) have demonstrated strong capabilities in molecular design, yet their impact in drug discovery remains limited by the absence of reliable reward signals and the lack of interpretability in their outputs. We present SAFE-T, a generalist chemical modeling framework that conditions on biological context -- such as protein targets or mechanisms of action -- to prioritize and design molecules without relying on structural information or engineered scoring functions. SAFE-T models the conditional likelihood of fragment-based molecular sequences given a biological prompt, enabling principled scoring of molecules across tasks such as virtual screening, drug-target interaction prediction, and activity cliff detection. Moreover, it supports goal-directed generation by sampling from this learned distribution, aligning molecular design with biological objectives. In comprehensive zero-shot evaluations across predictive (LIT-PCBA, DAVIS, KIBA, ACNet) and generative (DRUG, PMO) benchmarks, SAFE-T consistently achieves performance comparable to or better than existing approaches while being significantly faster. Fragment-level attribution further reveals that SAFE-T captures known structure-activity relationships, supporting interpretable and biologically grounded design. Together with its computational efficiency, these results demonstrate that conditional generative CLMs can unify scoring and generation to accelerate early-stage drug discovery.

Title: Mind the Gap: Aligning Vision Foundation Models to Image Feature Matching

• Authors: Yuhan Liu, Jingwen Fu, Yang Wu, Kangyi Wu, Pengna Li, Jiayi Wu, Sanping Zhou, Jingmin Xin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10318
• Pdf URL: https://arxiv.org/pdf/2507.10318
• Copy Paste: [[2507.10318]] Mind the Gap: Aligning Vision Foundation Models to Image Feature Matching(https://arxiv.org/abs/2507.10318)
• Keywords: diffusion, foundation model, generative
• Abstract: Leveraging the vision foundation models has emerged as a mainstream paradigm that improves the performance of image feature matching. However, previous works have ignored the misalignment when introducing the foundation models into feature matching. The misalignment arises from the discrepancy between the foundation models focusing on single-image understanding and the cross-image understanding requirement of feature matching. Specifically, 1) the embeddings derived from commonly used foundation models exhibit discrepancies with the optimal embeddings required for feature matching; 2) lacking an effective mechanism to leverage the single-image understanding ability into cross-image understanding. A significant consequence of the misalignment is they struggle when addressing multi-instance feature matching problems. To address this, we introduce a simple but effective framework, called IMD (Image feature Matching with a pre-trained Diffusion model) with two parts: 1) Unlike the dominant solutions employing contrastive-learning based foundation models that emphasize global semantics, we integrate the generative-based diffusion models to effectively capture instance-level details. 2) We leverage the prompt mechanism in generative model as a natural tunnel, propose a novel cross-image interaction prompting module to facilitate bidirectional information interaction between image pairs. To more accurately measure the misalignment, we propose a new benchmark called IMIM, which focuses on multi-instance scenarios. Our proposed IMD establishes a new state-of-the-art in commonly evaluated benchmarks, and the superior improvement 12% in IMIM indicates our method efficiently mitigates the misalignment.

Title: MoCap-Impute: A Comprehensive Benchmark and Comparative Analysis of Imputation Methods for IMU-based Motion Capture Data

• Authors: Mahmoud Bekhit, Ahmad Salah, Ahmed Salim Alrawahi, Tarek Attia, Ahmed Ali, Esraa Eldesokey, Ahmed Fathalla
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10334
• Pdf URL: https://arxiv.org/pdf/2507.10334
• Copy Paste: [[2507.10334]] MoCap-Impute: A Comprehensive Benchmark and Comparative Analysis of Imputation Methods for IMU-based Motion Capture Data(https://arxiv.org/abs/2507.10334)
• Keywords: generative
• Abstract: Motion capture (MoCap) data from wearable Inertial Measurement Units (IMUs) is vital for applications in sports science, but its utility is often compromised by missing data. Despite numerous imputation techniques, a systematic performance evaluation for IMU-derived MoCap time-series data is lacking. We address this gap by conducting a comprehensive comparative analysis of statistical, machine learning, and deep learning imputation methods. Our evaluation considers three distinct contexts: univariate time-series, multivariate across subjects, and multivariate across kinematic angles. To facilitate this benchmark, we introduce the first publicly available MoCap dataset designed specifically for imputation, featuring data from 53 karate practitioners. We simulate three controlled missingness mechanisms: missing completely at random (MCAR), block missingness, and a novel value-dependent pattern at signal transition points. Our experiments, conducted on 39 kinematic variables across all subjects, reveal that multivariate imputation frameworks consistently outperform univariate approaches, particularly for complex missingness. For instance, multivariate methods achieve up to a 50% mean absolute error reduction (MAE from 10.8 to 5.8) compared to univariate techniques for transition point missingness. Advanced models like Generative Adversarial Imputation Networks (GAIN) and Iterative Imputers demonstrate the highest accuracy in these challenging scenarios. This work provides a critical baseline for future research and offers practical recommendations for improving the integrity and robustness of Mo-Cap data analysis.

Title: Text Embedding Knows How to Quantize Text-Guided Diffusion Models

• Authors: Hongjae Lee, Myungjun Son, Dongjea Kang, Seung-Won Jung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10340
• Pdf URL: https://arxiv.org/pdf/2507.10340
• Copy Paste: [[2507.10340]] Text Embedding Knows How to Quantize Text-Guided Diffusion Models(https://arxiv.org/abs/2507.10340)
• Keywords: diffusion
• Abstract: Despite the success of diffusion models in image generation tasks such as text-to-image, the enormous computational complexity of diffusion models limits their use in resource-constrained environments. To address this, network quantization has emerged as a promising solution for designing efficient diffusion models. However, existing diffusion model quantization methods do not consider input conditions, such as text prompts, as an essential source of information for quantization. In this paper, we propose a novel quantization method dubbed Quantization of Language-to-Image diffusion models using text Prompts (QLIP). QLIP leverages text prompts to guide the selection of bit precision for every layer at each time step. In addition, QLIP can be seamlessly integrated into existing quantization methods to enhance quantization efficiency. Our extensive experiments demonstrate the effectiveness of QLIP in reducing computational complexity and improving the quality of the generated images across various datasets.

Title: Parallel Sampling of Diffusion Models on $SO(3)$

• Authors: Yan-Ting Chen, Hao-Wei Chen, Tsu-Ching Hsiao, Chun-Yi Lee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10347
• Pdf URL: https://arxiv.org/pdf/2507.10347
• Copy Paste: [[2507.10347]] Parallel Sampling of Diffusion Models on $SO(3)$(https://arxiv.org/abs/2507.10347)
• Keywords: diffusion
• Abstract: In this paper, we design an algorithm to accelerate the diffusion process on the $SO(3)$ manifold. The inherently sequential nature of diffusion models necessitates substantial time for denoising perturbed data. To overcome this limitation, we proposed to adapt the numerical Picard iteration for the $SO(3)$ space. We demonstrate our algorithm on an existing method that employs diffusion models to address the pose ambiguity problem. Moreover, we show that this acceleration advantage occurs without any measurable degradation in task reward. The experiments reveal that our algorithm achieves a speed-up of up to 4.9$\times$, significantly reducing the latency for generating a single sample.

Title: Test-Time Canonicalization by Foundation Models for Robust Perception

• Authors: Utkarsh Singhal, Ryan Feng, Stella X. Yu, Atul Prakash
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10375
• Pdf URL: https://arxiv.org/pdf/2507.10375
• Copy Paste: [[2507.10375]] Test-Time Canonicalization by Foundation Models for Robust Perception(https://arxiv.org/abs/2507.10375)
• Keywords: foundation model
• Abstract: Real-world visual perception requires invariance to diverse transformations, yet current methods rely heavily on specialized architectures or training on predefined augmentations, limiting generalization. We propose FOCAL, a test-time, data-driven framework that achieves robust perception by leveraging internet-scale visual priors from foundation models. By generating and optimizing candidate transformations toward visually typical, "canonical" views, FOCAL enhances robustness without re-training or architectural changes. Our experiments demonstrate improved robustness of CLIP and SAM across challenging transformations, including 2D/3D rotations, illumination shifts (contrast and color), and day-night variations. We also highlight potential applications in active vision. Our approach challenges the assumption that transform-specific training is necessary, instead offering a scalable path to invariance. Our code is available at: this https URL.

Title: Stochastic Operator Network: A Stochastic Maximum Principle Based Approach to Operator Learning

• Authors: Ryan Bausback, Jingqiao Tang, Lu Lu, Feng Bao, Toan Huynh
• Subjects: cs.LG, math.PR
• Abstract URL: https://arxiv.org/abs/2507.10401
• Pdf URL: https://arxiv.org/pdf/2507.10401
• Copy Paste: [[2507.10401]] Stochastic Operator Network: A Stochastic Maximum Principle Based Approach to Operator Learning(https://arxiv.org/abs/2507.10401)
• Keywords: diffusion
• Abstract: We develop a novel framework for uncertainty quantification in operator learning, the Stochastic Operator Network (SON). SON combines the stochastic optimal control concepts of the Stochastic Neural Network (SNN) with the DeepONet. By formulating the branch net as an SDE and backpropagating through the adjoint BSDE, we replace the gradient of the loss function with the gradient of the Hamiltonian from Stohastic Maximum Principle in the SGD update. This allows SON to learn the uncertainty present in operators through its diffusion parameters. We then demonstrate the effectiveness of SON when replicating several noisy operators in 2D and 3D.

Title: CLA: Latent Alignment for Online Continual Self-Supervised Learning

• Authors: Giacomo Cignoni, Andrea Cossu, Alexandra Gomez-Villa, Joost van de Weijer, Antonio Carta
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10434
• Pdf URL: https://arxiv.org/pdf/2507.10434
• Copy Paste: [[2507.10434]] CLA: Latent Alignment for Online Continual Self-Supervised Learning(https://arxiv.org/abs/2507.10434)
• Keywords: self-supervised
• Abstract: Self-supervised learning (SSL) is able to build latent representations that generalize well to unseen data. However, only a few SSL techniques exist for the online CL setting, where data arrives in small minibatches, the model must comply with a fixed computational budget, and task boundaries are absent. We introduce Continual Latent Alignment (CLA), a novel SSL strategy for Online CL that aligns the representations learned by the current model with past representations to mitigate forgetting. We found that our CLA is able to speed up the convergence of the training process in the online scenario, outperforming state-of-the-art approaches under the same computational budget. Surprisingly, we also discovered that using CLA as a pretraining protocol in the early stages of pretraining leads to a better final performance when compared to a full i.i.d. pretraining.

Title: RefSTAR: Blind Facial Image Restoration with Reference Selection, Transfer, and Reconstruction

• Authors: Zhicun Yin, Junjie Chen, Ming Liu, Zhixin Wang, Fan Li, Renjing Pei, Xiaoming Li, Rynson W.H. Lau, Wangmeng Zuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.10470
• Pdf URL: https://arxiv.org/pdf/2507.10470
• Copy Paste: [[2507.10470]] RefSTAR: Blind Facial Image Restoration with Reference Selection, Transfer, and Reconstruction(https://arxiv.org/abs/2507.10470)
• Keywords: generative
• Abstract: Blind facial image restoration is highly challenging due to unknown complex degradations and the sensitivity of humans to faces. Although existing methods introduce auxiliary information from generative priors or high-quality reference images, they still struggle with identity preservation problems, mainly due to improper feature introduction on detailed textures. In this paper, we focus on effectively incorporating appropriate features from high-quality reference images, presenting a novel blind facial image restoration method that considers reference selection, transfer, and reconstruction (RefSTAR). In terms of selection, we construct a reference selection (RefSel) module. For training the RefSel module, we construct a RefSel-HQ dataset through a mask generation pipeline, which contains annotating masks for 10,000 ground truth-reference pairs. As for the transfer, due to the trivial solution in vanilla cross-attention operations, a feature fusion paradigm is designed to force the features from the reference to be integrated. Finally, we propose a reference image reconstruction mechanism that further ensures the presence of reference image features in the output image. The cycle consistency loss is also redesigned in conjunction with the mask. Extensive experiments on various backbone models demonstrate superior performance, showing better identity preservation ability and reference feature transfer quality. Source code, dataset, and pre-trained models are available at this https URL.

Title: Can You Detect the Difference?

• Authors: İsmail Tarım, Aytuğ Onan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.10475
• Pdf URL: https://arxiv.org/pdf/2507.10475
• Copy Paste: [[2507.10475]] Can You Detect the Difference?(https://arxiv.org/abs/2507.10475)
• Keywords: diffusion
• Abstract: The rapid advancement of large language models (LLMs) has raised concerns about reliably detecting AI-generated text. Stylometric metrics work well on autoregressive (AR) outputs, but their effectiveness on diffusion-based models is unknown. We present the first systematic comparison of diffusion-generated text (LLaDA) and AR-generated text (LLaMA) using 2 000 samples. Perplexity, burstiness, lexical diversity, readability, and BLEU/ROUGE scores show that LLaDA closely mimics human text in perplexity and burstiness, yielding high false-negative rates for AR-oriented detectors. LLaMA shows much lower perplexity but reduced lexical fidelity. Relying on any single metric fails to separate diffusion outputs from human writing. We highlight the need for diffusion-aware detectors and outline directions such as hybrid models, diffusion-specific stylometric signatures, and robust watermarking.

Title: BenchReAD: A systematic benchmark for retinal anomaly detection

• Authors: Chenyu Lian, Hong-Yu Zhou, Zhanli Hu, Jing Qin
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10492
• Pdf URL: https://arxiv.org/pdf/2507.10492
• Copy Paste: [[2507.10492]] BenchReAD: A systematic benchmark for retinal anomaly detection(https://arxiv.org/abs/2507.10492)
• Keywords: anomaly
• Abstract: Retinal anomaly detection plays a pivotal role in screening ocular and systemic diseases. Despite its significance, progress in the field has been hindered by the absence of a comprehensive and publicly available benchmark, which is essential for the fair evaluation and advancement of methodologies. Due to this limitation, previous anomaly detection work related to retinal images has been constrained by (1) a limited and overly simplistic set of anomaly types, (2) test sets that are nearly saturated, and (3) a lack of generalization evaluation, resulting in less convincing experimental setups. Furthermore, existing benchmarks in medical anomaly detection predominantly focus on one-class supervised approaches (training only with negative samples), overlooking the vast amounts of labeled abnormal data and unlabeled data that are commonly available in clinical practice. To bridge these gaps, we introduce a benchmark for retinal anomaly detection, which is comprehensive and systematic in terms of data and algorithm. Through categorizing and benchmarking previous methods, we find that a fully supervised approach leveraging disentangled representations of abnormalities (DRA) achieves the best performance but suffers from significant drops in performance when encountering certain unseen anomalies. Inspired by the memory bank mechanisms in one-class supervised learning, we propose NFM-DRA, which integrates DRA with a Normal Feature Memory to mitigate the performance degradation, establishing a new SOTA. The benchmark is publicly available at this https URL.

Title: Graph World Model

• Authors: Tao Feng, Yexin Wu, Guanyu Lin, Jiaxuan You
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10539
• Pdf URL: https://arxiv.org/pdf/2507.10539
• Copy Paste: [[2507.10539]] Graph World Model(https://arxiv.org/abs/2507.10539)
• Keywords: foundation model
• Abstract: World models (WMs) demonstrate strong capabilities in prediction, generation, and planning tasks. Existing WMs primarily focus on unstructured data and cannot leverage the ubiquitous structured data, often represented as graphs, in the digital world. While multiple graph foundation models have been proposed, they focus on graph learning tasks and cannot extend to diverse multi-modal data and interdisciplinary tasks. To address these challenges, we propose the Graph World Model (GWM), a world model that supports both unstructured and graph-structured states with multi-modal information and represents diverse tasks as actions. The core of a GWM is a generic message-passing algorithm to aggregate structured information, either over a unified multi-modal token space by converting multi-modal data into text (GWM-T) or a unified multi-modal embedding space by modality-specific encoders (GWM-E). Notably, GWM introduces action nodes to support diverse tasks, where action nodes are linked to other nodes via direct reference or similarity computation. Extensive experiments on six tasks from diverse domains, including multi-modal generation and matching, recommendation, graph prediction, multi-agent, retrieval-augmented generation, and planning and optimization, show that the same GWM outperforms or matches domain-specific baselines' performance, benefits from multi-hop structures, and demonstrates strong zero-shot/few-shot capabilities on unseen new tasks. Our code for GWM is released at this https URL.

Title: Self-supervised Learning on Camera Trap Footage Yields a Strong Universal Face Embedder

• Authors: Vladimir Iashin, Horace Lee, Dan Schofield, Andrew Zisserman
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.10552
• Pdf URL: https://arxiv.org/pdf/2507.10552
• Copy Paste: [[2507.10552]] Self-supervised Learning on Camera Trap Footage Yields a Strong Universal Face Embedder(https://arxiv.org/abs/2507.10552)
• Keywords: self-supervised
• Abstract: Camera traps are revolutionising wildlife monitoring by capturing vast amounts of visual data; however, the manual identification of individual animals remains a significant bottleneck. This study introduces a fully self-supervised approach to learning robust chimpanzee face embeddings from unlabeled camera-trap footage. Leveraging the DINOv2 framework, we train Vision Transformers on automatically mined face crops, eliminating the need for identity labels. Our method demonstrates strong open-set re-identification performance, surpassing supervised baselines on challenging benchmarks such as Bossou, despite utilising no labelled data during training. This work underscores the potential of self-supervised learning in biodiversity monitoring and paves the way for scalable, non-invasive population studies.