2025-07-10

Title: Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework

• Authors: Satyapriya Krishna, Ninareh Mehrabi, Abhinav Mohanty, Matteo Memelli, Vincent Ponzo, Payal Motwani, Rahul Gupta
• Subjects: cs.CR, cs.CY
• Abstract URL: https://arxiv.org/abs/2507.06260
• Pdf URL: https://arxiv.org/pdf/2507.06260
• Copy Paste: [[2507.06260]] Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework(https://arxiv.org/abs/2507.06260)
• Keywords: foundation model
• Abstract: Nova Premier is Amazon's most capable multimodal foundation model and teacher for model distillation. It processes text, images, and video with a one-million-token context window, enabling analysis of large codebases, 400-page documents, and 90-minute videos in a single prompt. We present the first comprehensive evaluation of Nova Premier's critical risk profile under the Frontier Model Safety Framework. Evaluations target three high-risk domains -- Chemical, Biological, Radiological & Nuclear (CBRN), Offensive Cyber Operations, and Automated AI R&D -- and combine automated benchmarks, expert red-teaming, and uplift studies to determine whether the model exceeds release thresholds. We summarize our methodology and report core findings. Based on this evaluation, we find that Nova Premier is safe for public release as per our commitments made at the 2025 Paris AI Safety Summit. We will continue to enhance our safety evaluation and mitigation pipelines as new risks and capabilities associated with frontier models are identified.

Title: Advancing Offline Handwritten Text Recognition: A Systematic Review of Data Augmentation and Generation Techniques

• Authors: Yassin Hussein Rassul, Aram M. Ahmed, Polla Fattah, Bryar A. Hassan, Arwaa W. Abdulkareem, Tarik A. Rashid, Joan Lu
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06275
• Pdf URL: https://arxiv.org/pdf/2507.06275
• Copy Paste: [[2507.06275]] Advancing Offline Handwritten Text Recognition: A Systematic Review of Data Augmentation and Generation Techniques(https://arxiv.org/abs/2507.06275)
• Keywords: diffusion, generative
• Abstract: Offline Handwritten Text Recognition (HTR) systems play a crucial role in applications such as historical document digitization, automatic form processing, and biometric authentication. However, their performance is often hindered by the limited availability of annotated training data, particularly for low-resource languages and complex scripts. This paper presents a comprehensive survey of offline handwritten data augmentation and generation techniques designed to improve the accuracy and robustness of HTR systems. We systematically examine traditional augmentation methods alongside recent advances in deep learning, including Generative Adversarial Networks (GANs), diffusion models, and transformer-based approaches. Furthermore, we explore the challenges associated with generating diverse and realistic handwriting samples, particularly in preserving script authenticity and addressing data scarcity. This survey follows the PRISMA methodology, ensuring a structured and rigorous selection process. Our analysis began with 1,302 primary studies, which were filtered down to 848 after removing duplicates, drawing from key academic sources such as IEEE Digital Library, Springer Link, Science Direct, and ACM Digital Library. By evaluating existing datasets, assessment metrics, and state-of-the-art methodologies, this survey identifies key research gaps and proposes future directions to advance the field of handwritten text generation across diverse linguistic and stylistic landscapes.

Title: DecoyDB: A Dataset for Graph Contrastive Learning in Protein-Ligand Binding Affinity Prediction

• Authors: Yupu Zhang, Zelin Xu, Tingsong Xiao, Gustavo Seabra, Yanjun Li, Chenglong Li, Zhe Jiang
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2507.06366
• Pdf URL: https://arxiv.org/pdf/2507.06366
• Copy Paste: [[2507.06366]] DecoyDB: A Dataset for Graph Contrastive Learning in Protein-Ligand Binding Affinity Prediction(https://arxiv.org/abs/2507.06366)
• Keywords: self-supervised
• Abstract: Predicting the binding affinity of protein-ligand complexes plays a vital role in drug discovery. Unfortunately, progress has been hindered by the lack of large-scale and high-quality binding affinity labels. The widely used PDBbind dataset has fewer than 20K labeled complexes. Self-supervised learning, especially graph contrastive learning (GCL), provides a unique opportunity to break the barrier by pre-training graph neural network models based on vast unlabeled complexes and fine-tuning the models on much fewer labeled complexes. However, the problem faces unique challenges, including a lack of a comprehensive unlabeled dataset with well-defined positive/negative complex pairs and the need to design GCL algorithms that incorporate the unique characteristics of such data. To fill the gap, we propose DecoyDB, a large-scale, structure-aware dataset specifically designed for self-supervised GCL on protein-ligand complexes. DecoyDB consists of high-resolution ground truth complexes (less than 2.5 Angstrom) and diverse decoy structures with computationally generated binding poses that range from realistic to suboptimal (negative pairs). Each decoy is annotated with a Root Mean Squared Deviation (RMSD) from the native pose. We further design a customized GCL framework to pre-train graph neural networks based on DecoyDB and fine-tune the models with labels from PDBbind. Extensive experiments confirm that models pre-trained with DecoyDB achieve superior accuracy, label efficiency, and generalizability.

Title: Bridging Data Gaps of Rare Conditions in ICU: A Multi-Disease Adaptation Approach for Clinical Prediction

• Authors: Mingcheng Zhu, Yu Liu, Zhiyao Luo, Tingting Zhu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06432
• Pdf URL: https://arxiv.org/pdf/2507.06432
• Copy Paste: [[2507.06432]] Bridging Data Gaps of Rare Conditions in ICU: A Multi-Disease Adaptation Approach for Clinical Prediction(https://arxiv.org/abs/2507.06432)
• Keywords: self-supervised
• Abstract: Artificial Intelligence has revolutionised critical care for common conditions. Yet, rare conditions in the intensive care unit (ICU), including recognised rare diseases and low-prevalence conditions in the ICU, remain underserved due to data scarcity and intra-condition heterogeneity. To bridge such gaps, we developed KnowRare, a domain adaptation-based deep learning framework for predicting clinical outcomes for rare conditions in the ICU. KnowRare mitigates data scarcity by initially learning condition-agnostic representations from diverse electronic health records through self-supervised pre-training. It addresses intra-condition heterogeneity by selectively adapting knowledge from clinically similar conditions with a developed condition knowledge graph. Evaluated on two ICU datasets across five clinical prediction tasks (90-day mortality, 30-day readmission, ICU mortality, remaining length of stay, and phenotyping), KnowRare consistently outperformed existing state-of-the-art models. Additionally, KnowRare demonstrated superior predictive performance compared to established ICU scoring systems, including APACHE IV and IV-a. Case studies further demonstrated KnowRare's flexibility in adapting its parameters to accommodate dataset-specific and task-specific characteristics, its generalisation to common conditions under limited data scenarios, and its rationality in selecting source conditions. These findings highlight KnowRare's potential as a robust and practical solution for supporting clinical decision-making and improving care for rare conditions in the ICU.

Title: FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models

• Authors: Qianyu Long, Qiyuan Wang, Christos Anagnostopoulos, Daning Bi
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2507.06449
• Pdf URL: https://arxiv.org/pdf/2507.06449
• Copy Paste: [[2507.06449]] FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models(https://arxiv.org/abs/2507.06449)
• Keywords: diffusion
• Abstract: Federated Learning (FL), as a distributed learning paradigm, trains models over distributed clients' data. FL is particularly beneficial for distributed training of Diffusion Models (DMs), which are high-quality image generators that require diverse data. However, challenges such as high communication costs and data heterogeneity persist in training DMs similar to training Transformers and Convolutional Neural Networks. Limited research has addressed these issues in FL environments. To address this gap and challenges, we introduce a novel approach, FedPhD, designed to efficiently train DMs in FL environments. FedPhD leverages Hierarchical FL with homogeneity-aware model aggregation and selection policy to tackle data heterogeneity while reducing communication costs. The distributed structured pruning of FedPhD enhances computational efficiency and reduces model storage requirements in clients. Our experiments across multiple datasets demonstrate that FedPhD achieves high model performance regarding Fréchet Inception Distance (FID) scores while reducing communication costs by up to $88\%$. FedPhD outperforms baseline methods achieving at least a $34\%$ improvement in FID, while utilizing only $56\%$ of the total computation and communication resources.

Title: Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models

• Authors: Arjun Banerjee, David Martinez, Camille Dang, Ethan Tam
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2507.06458
• Pdf URL: https://arxiv.org/pdf/2507.06458
• Copy Paste: [[2507.06458]] Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models(https://arxiv.org/abs/2507.06458)
• Keywords: generative
• Abstract: Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution.

Title: Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Models

• Authors: Aaron Dharna, Cong Lu, Jeff Clune
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06466
• Pdf URL: https://arxiv.org/pdf/2507.06466
• Copy Paste: [[2507.06466]] Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Models(https://arxiv.org/abs/2507.06466)
• Keywords: foundation model
• Abstract: Multi-agent interactions have long fueled innovation, from natural predator-prey dynamics to the space race. Self-play (SP) algorithms try to harness these dynamics by pitting agents against ever-improving opponents, thereby creating an implicit curriculum toward learning high-quality solutions. However, SP often fails to produce diverse solutions and can get stuck in locally optimal behaviors. We introduce Foundation-Model Self-Play (FMSP), a new direction that leverages the code-generation capabilities and vast knowledge of foundation models (FMs) to overcome these challenges by leaping across local optima in policy space. We propose a family of approaches: (1) \textbf{Vanilla Foundation-Model Self-Play (vFMSP)} continually refines agent policies via competitive self-play; (2) \textbf{Novelty-Search Self-Play (NSSP)} builds a diverse population of strategies, ignoring performance; and (3) the most promising variant, \textbf{Quality-Diveristy Self-Play (QDSP)}, creates a diverse set of high-quality policies by combining the diversity of NSSP and refinement of vFMSP. We evaluate FMSPs in Car Tag, a continuous-control pursuer-evader setting, and in Gandalf, a simple AI safety simulation in which an attacker tries to jailbreak an LLM's defenses. In Car Tag, FMSPs explore a wide variety of reinforcement learning, tree search, and heuristic-based methods, to name just a few. In terms of discovered policy quality, \ouralgo and vFMSP surpass strong human-designed strategies. In Gandalf, FMSPs can successfully automatically red-team an LLM, breaking through and jailbreaking six different, progressively stronger levels of defense. Furthermore, FMSPs can automatically proceed to patch the discovered vulnerabilities. Overall, FMSPs represent a promising new research frontier of improving self-play with foundation models, opening fresh paths toward more creative and open-ended strategy discovery

Title: FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning

• Authors: Huan Wang, Haoran Li, Huaming Chen, Jun Yan, Jiahua Shi, Jun Shen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06482
• Pdf URL: https://arxiv.org/pdf/2507.06482
• Copy Paste: [[2507.06482]] FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning(https://arxiv.org/abs/2507.06482)
• Keywords: diffusion, self-supervised
• Abstract: Federated learning aims at training models collaboratively across participants while protecting privacy. However, one major challenge for this paradigm is the data heterogeneity issue, where biased data preferences across multiple clients, harming the model's convergence and performance. In this paper, we first introduce powerful diffusion models into the federated learning paradigm and show that diffusion representations are effective steers during federated training. To explore the possibility of using diffusion representations in handling data heterogeneity, we propose a novel diffusion-inspired Federated paradigm with Diffusion Representation Collaboration, termed FedDifRC, leveraging meaningful guidance of diffusion models to mitigate data heterogeneity. The key idea is to construct text-driven diffusion contrasting and noise-driven diffusion regularization, aiming to provide abundant class-related semantic information and consistent convergence signals. On the one hand, we exploit the conditional feedback from the diffusion model for different text prompts to build a text-driven contrastive learning strategy. On the other hand, we introduce a noise-driven consistency regularization to align local instances with diffusion denoising representations, constraining the optimization region in the feature space. In addition, FedDifRC can be extended to a self-supervised scheme without relying on any labeled data. We also provide a theoretical analysis for FedDifRC to ensure convergence under non-convex objectives. The experiments on different scenarios validate the effectiveness of FedDifRC and the efficiency of crucial components.

Title: Concept Unlearning by Modeling Key Steps of Diffusion Process

• Authors: Chaoshuo Zhang, Chenhao Lin, Zhengyu Zhao, Le Yang, Qian Wang, Chao Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06526
• Pdf URL: https://arxiv.org/pdf/2507.06526
• Copy Paste: [[2507.06526]] Concept Unlearning by Modeling Key Steps of Diffusion Process(https://arxiv.org/abs/2507.06526)
• Keywords: diffusion, generative
• Abstract: Text-to-image diffusion models (T2I DMs), represented by Stable Diffusion, which generate highly realistic images based on textual input, have been widely used. However, their misuse poses serious security risks. While existing concept unlearning methods aim to mitigate these risks, they struggle to balance unlearning effectiveness with generative this http URL overcome this limitation, we innovatively propose the Key Step Concept Unlearning (KSCU) method, which ingeniously capitalizes on the unique stepwise sampling characteristic inherent in diffusion models during the image generation process. Unlike conventional approaches that treat all denoising steps equally, KSCU strategically focuses on pivotal steps with the most influence over the final outcome by dividing key steps for different concept unlearning tasks and fine-tuning the model only at those steps. This targeted approach reduces the number of parameter updates needed for effective unlearning, while maximizing the retention of the model's generative this http URL extensive benchmark experiments, we demonstrate that KSCU effectively prevents T2I DMs from generating undesirable images while better retaining the model's generative this http URL code will be released.

Title: Transferable Parasitic Estimation via Graph Contrastive Learning and Label Rebalancing in AMS Circuits

• Authors: Shan Shen, Shenglu Hua, Jiajun Zou, Jiawei Liu, Jianwang Zhai, Chuan Shi, Wenjian Yu
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2507.06535
• Pdf URL: https://arxiv.org/pdf/2507.06535
• Copy Paste: [[2507.06535]] Transferable Parasitic Estimation via Graph Contrastive Learning and Label Rebalancing in AMS Circuits(https://arxiv.org/abs/2507.06535)
• Keywords: self-supervised
• Abstract: Graph representation learning on Analog-Mixed Signal (AMS) circuits is crucial for various downstream tasks, e.g., parasitic estimation. However, the scarcity of design data, the unbalanced distribution of labels, and the inherent diversity of circuit implementations pose significant challenges to learning robust and transferable circuit representations. To address these limitations, we propose CircuitGCL, a novel graph contrastive learning framework that integrates representation scattering and label rebalancing to enhance transferability across heterogeneous circuit graphs. CircuitGCL employs a self-supervised strategy to learn topology-invariant node embeddings through hyperspherical representation scattering, eliminating dependency on large-scale data. Simultaneously, balanced mean squared error (MSE) and softmax cross-entropy (bsmCE) losses are introduced to mitigate label distribution disparities between circuits, enabling robust and transferable parasitic estimation. Evaluated on parasitic capacitance estimation (edge-level task) and ground capacitance classification (node-level task) across TSMC 28nm AMS designs, CircuitGCL outperforms all state-of-the-art (SOTA) methods, with the $R^2$ improvement of $33.64\% \sim 44.20\%$ for edge regression and F1-score gain of $0.9\times \sim 2.1\times$ for node classification. Our code is available at \href{this https URL}{here}.

Title: Token Bottleneck: One Token to Remember Dynamics

• Authors: Taekyung Kim, Dongyoon Han, Byeongho Heo, Jeongeun Park, Sangdoo Yun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06543
• Pdf URL: https://arxiv.org/pdf/2507.06543
• Copy Paste: [[2507.06543]] Token Bottleneck: One Token to Remember Dynamics(https://arxiv.org/abs/2507.06543)
• Keywords: self-supervised
• Abstract: Deriving compact and temporally aware visual representations from dynamic scenes is essential for successful execution of sequential scene understanding tasks such as visual tracking and robotic manipulation. In this paper, we introduce Token Bottleneck (ToBo), a simple yet intuitive self-supervised learning pipeline that squeezes a scene into a bottleneck token and predicts the subsequent scene using minimal patches as hints. The ToBo pipeline facilitates the learning of sequential scene representations by conservatively encoding the reference scene into a compact bottleneck token during the squeeze step. In the expansion step, we guide the model to capture temporal dynamics by predicting the target scene using the bottleneck token along with few target patches as hints. This design encourages the vision backbone to embed temporal dependencies, thereby enabling understanding of dynamic transitions across scenes. Extensive experiments in diverse sequential tasks, including video label propagation and robot manipulation in simulated environments demonstrate the superiority of ToBo over baselines. Moreover, deploying our pre-trained model on physical robots confirms its robustness and effectiveness in real-world environments. We further validate the scalability of ToBo across different model scales.

Title: Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attribution

• Authors: Yonghyun Park, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Naoki Murata, Wei-Hsiang Liao, Woosung Choi, Kin Wai Cheuk, Junghyun Koo, Yuki Mitsufuji
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06547
• Pdf URL: https://arxiv.org/pdf/2507.06547
• Copy Paste: [[2507.06547]] Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attribution(https://arxiv.org/abs/2507.06547)
• Keywords: diffusion, generative
• Abstract: While diffusion models excel at image generation, their growing adoption raises critical concerns around copyright issues and model transparency. Existing attribution methods identify training examples influencing an entire image, but fall short in isolating contributions to specific elements, such as styles or objects, that matter most to stakeholders. To bridge this gap, we introduce \emph{concept-level attribution} via a novel method called \emph{Concept-TRAK}. Concept-TRAK extends influence functions with two key innovations: (1) a reformulated diffusion training loss based on diffusion posterior sampling, enabling robust, sample-specific attribution; and (2) a concept-aware reward function that emphasizes semantic relevance. We evaluate Concept-TRAK on the AbC benchmark, showing substantial improvements over prior methods. Through diverse case studies--ranging from identifying IP-protected and unsafe content to analyzing prompt engineering and compositional learning--we demonstrate how concept-level attribution yields actionable insights for responsible generative AI development and governance.

Title: Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis

• Authors: Srihari K B, Pushpak Bhattacharyya
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2507.06571
• Pdf URL: https://arxiv.org/pdf/2507.06571
• Copy Paste: [[2507.06571]] Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis(https://arxiv.org/abs/2507.06571)
• Keywords: diffusion, generative
• Abstract: We propose a unified food-domain QA framework that combines a large-scale multimodal knowledge graph (MMKG) with generative AI. Our MMKG links 13,000 recipes, 3,000 ingredients, 140,000 relations, and 14,000 images. We generate 40,000 QA pairs using 40 templates and LLaVA/DeepSeek augmentation. Joint fine-tuning of Meta LLaMA 3.1-8B and Stable Diffusion 3.5-Large improves BERTScore by 16.2\%, reduces FID by 37.8\%, and boosts CLIP alignment by 31.1\%. Diagnostic analyses-CLIP-based mismatch detection (35.2\% to 7.3\%) and LLaVA-driven hallucination checks-ensure factual and visual fidelity. A hybrid retrieval-generation strategy achieves 94.1\% accurate image reuse and 85\% adequacy in synthesis. Our results demonstrate that structured knowledge and multimodal generation together enhance reliability and diversity in food QA.

Title: MOST: Motion Diffusion Model for Rare Text via Temporal Clip Banzhaf Interaction

• Authors: Yin Wang, Mu li, Zhiying Leng, Frederick W. B. Li, Xiaohui Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06590
• Pdf URL: https://arxiv.org/pdf/2507.06590
• Copy Paste: [[2507.06590]] MOST: Motion Diffusion Model for Rare Text via Temporal Clip Banzhaf Interaction(https://arxiv.org/abs/2507.06590)
• Keywords: diffusion
• Abstract: We introduce MOST, a novel motion diffusion model via temporal clip Banzhaf interaction, aimed at addressing the persistent challenge of generating human motion from rare language prompts. While previous approaches struggle with coarse-grained matching and overlook important semantic cues due to motion redundancy, our key insight lies in leveraging fine-grained clip relationships to mitigate these issues. MOST's retrieval stage presents the first formulation of its kind - temporal clip Banzhaf interaction - which precisely quantifies textual-motion coherence at the clip level. This facilitates direct, fine-grained text-to-motion clip matching and eliminates prevalent redundancy. In the generation stage, a motion prompt module effectively utilizes retrieved motion clips to produce semantically consistent movements. Extensive evaluations confirm that MOST achieves state-of-the-art text-to-motion retrieval and generation performance by comprehensively addressing previous challenges, as demonstrated through quantitative and qualitative results highlighting its effectiveness, especially for rare prompts.

Title: Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation

• Authors: Anshuk Uppal, Yuhta Takida, Chieh-Hsin Lai, Yuki Mitsufuji
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06613
• Pdf URL: https://arxiv.org/pdf/2507.06613
• Copy Paste: [[2507.06613]] Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation(https://arxiv.org/abs/2507.06613)
• Keywords: diffusion, generative
• Abstract: Disentangled and interpretable latent representations in generative models typically come at the cost of generation quality. The $\beta$-VAE framework introduces a hyperparameter $\beta$ to balance disentanglement and reconstruction quality, where setting $\beta > 1$ introduces an information bottleneck that favors disentanglement over sharp, accurate reconstructions. To address this trade-off, we propose a novel generative modeling framework that leverages a range of $\beta$ values to learn multiple corresponding latent representations. First, we obtain a slew of representations by training a single variational autoencoder (VAE), with a new loss function that controls the information retained in each latent representation such that the higher $\beta$ value prioritize disentanglement over reconstruction fidelity. We then, introduce a non-linear diffusion model that smoothly transitions latent representations corresponding to different $\beta$ values. This model denoises towards less disentangled and more informative representations, ultimately leading to (almost) lossless representations, enabling sharp reconstructions. Furthermore, our model supports sample generation without input images, functioning as a standalone generative model. We evaluate our framework in terms of both disentanglement and generation quality. Additionally, we observe smooth transitions in the latent spaces with respect to changes in $\beta$, facilitating consistent manipulation of generated outputs.

Title: Prevention of Overfitting on Mesh-Structured Data Regressions with a Modified Laplace Operator

• Authors: Enda D.V. Bigarella
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06631
• Pdf URL: https://arxiv.org/pdf/2507.06631
• Copy Paste: [[2507.06631]] Prevention of Overfitting on Mesh-Structured Data Regressions with a Modified Laplace Operator(https://arxiv.org/abs/2507.06631)
• Keywords: diffusion
• Abstract: This document reports on a method for detecting and preventing overfitting on data regressions, herein applied to mesh-like data structures. The mesh structure allows for the straightforward computation of the Laplace-operator second-order derivatives in a finite-difference fashion for noiseless data. Derivatives of the training data are computed on the original training mesh to serve as a true label of the entropy of the training data. Derivatives of the trained data are computed on a staggered mesh to identify oscillations in the interior of the original training mesh cells. The loss of the Laplace-operator derivatives is used for hyperparameter optimisation, achieving a reduction of unwanted oscillation through the minimisation of the entropy of the trained model. In this setup, testing does not require the splitting of points from the training data, and training is thus directly performed on all available training points. The Laplace operator applied to the trained data on a staggered mesh serves as a surrogate testing metric based on diffusion properties.

Title: EXAONE Path 2.0: Pathology Foundation Model with End-to-End Supervision

• Authors: Myungjang Pyeon, Janghyeon Lee, Minsoo Lee, Juseung Yun, Hwanil Choi, Jonghyun Kim, Jiwon Kim, Yi Hu, Jongseong Jang, Soonyoung Lee
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06639
• Pdf URL: https://arxiv.org/pdf/2507.06639
• Copy Paste: [[2507.06639]] EXAONE Path 2.0: Pathology Foundation Model with End-to-End Supervision(https://arxiv.org/abs/2507.06639)
• Keywords: self-supervised, foundation model
• Abstract: In digital pathology, whole-slide images (WSIs) are often difficult to handle due to their gigapixel scale, so most approaches train patch encoders via self-supervised learning (SSL) and then aggregate the patch-level embeddings via multiple instance learning (MIL) or slide encoders for downstream tasks. However, patch-level SSL may overlook complex domain-specific features that are essential for biomarker prediction, such as mutation status and molecular characteristics, as SSL methods rely only on basic augmentations selected for natural image domains on small patch-level area. Moreover, SSL methods remain less data efficient than fully supervised approaches, requiring extensive computational resources and datasets to achieve competitive performance. To address these limitations, we present EXAONE Path 2.0, a pathology foundation model that learns patch-level representations under direct slide-level supervision. Using only 37k WSIs for training, EXAONE Path 2.0 achieves state-of-the-art average performance across 10 biomarker prediction tasks, demonstrating remarkable data efficiency.

Title: Deep Disentangled Representation Network for Treatment Effect Estimation

• Authors: Hui Meng, Keping Yang, Xuyu Peng, Bo Zheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06650
• Pdf URL: https://arxiv.org/pdf/2507.06650
• Copy Paste: [[2507.06650]] Deep Disentangled Representation Network for Treatment Effect Estimation(https://arxiv.org/abs/2507.06650)
• Keywords: generative
• Abstract: Estimating individual-level treatment effect from observational data is a fundamental problem in causal inference and has attracted increasing attention in the fields of education, healthcare, and public this http URL this work, we concentrate on the study of disentangled representation methods that have shown promising outcomes by decomposing observed covariates into instrumental, confounding, and adjustment factors. However, most of the previous work has primarily revolved around generative models or hard decomposition methods for covariates, which often struggle to guarantee the attainment of precisely disentangled factors. In order to effectively model different causal relationships, we propose a novel treatment effect estimation algorithm that incorporates a mixture of experts with multi-head attention and a linear orthogonal regularizer to softly decompose the pre-treatment variables, and simultaneously eliminates selection bias via importance sampling re-weighting techniques. We conduct extensive experiments on both public semi-synthetic and real-world production datasets. The experimental results clearly demonstrate that our algorithm outperforms the state-of-the-art methods focused on individual treatment effects.

Title: Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior

• Authors: Juncheng Mu, Chengwei Ren, Weixiang Zhang, Liang Pan, Xiao-Ping Zhang, Yue Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06651
• Pdf URL: https://arxiv.org/pdf/2507.06651
• Copy Paste: [[2507.06651]] Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior(https://arxiv.org/abs/2507.06651)
• Keywords: diffusion
• Abstract: Learning cross-modal correspondences is essential for image-to-point cloud (I2P) registration. Existing methods achieve this mostly by utilizing metric learning to enforce feature alignment across modalities, disregarding the inherent modality gap between image and point data. Consequently, this paradigm struggles to ensure accurate cross-modal correspondences. To this end, inspired by the cross-modal generation success of recent large diffusion models, we propose Diff$^2$I2P, a fully Differentiable I2P registration framework, leveraging a novel and effective Diffusion prior for bridging the modality gap. Specifically, we propose a Control-Side Score Distillation (CSD) technique to distill knowledge from a depth-conditioned diffusion model to directly optimize the predicted transformation. However, the gradients on the transformation fail to backpropagate onto the cross-modal features due to the non-differentiability of correspondence retrieval and PnP solver. To this end, we further propose a Deformable Correspondence Tuning (DCT) module to estimate the correspondences in a differentiable way, followed by the transformation estimation using a differentiable PnP solver. With these two designs, the Diffusion model serves as a strong prior to guide the cross-modal feature learning of image and point cloud for forming robust correspondences, which significantly improves the registration. Extensive experimental results demonstrate that Diff$^2$I2P consistently outperforms SoTA I2P registration methods, achieving over 7% improvement in registration recall on the 7-Scenes benchmark.

Title: Enhancing Diffusion Model Stability for Image Restoration via Gradient Management

• Authors: Hongjie Wu, Mingqin Zhang, Linchao He, Ji-Zhe Zhou, Jiancheng Lv
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06656
• Pdf URL: https://arxiv.org/pdf/2507.06656
• Copy Paste: [[2507.06656]] Enhancing Diffusion Model Stability for Image Restoration via Gradient Management(https://arxiv.org/abs/2507.06656)
• Keywords: diffusion, generative
• Abstract: Diffusion models have shown remarkable promise for image restoration by leveraging powerful priors. Prominent methods typically frame the restoration problem within a Bayesian inference framework, which iteratively combines a denoising step with a likelihood guidance step. However, the interactions between these two components in the generation process remain underexplored. In this paper, we analyze the underlying gradient dynamics of these components and identify significant instabilities. Specifically, we demonstrate conflicts between the prior and likelihood gradient directions, alongside temporal fluctuations in the likelihood gradient itself. We show that these instabilities disrupt the generative process and compromise restoration performance. To address these issues, we propose Stabilized Progressive Gradient Diffusion (SPGD), a novel gradient management technique. SPGD integrates two synergistic components: (1) a progressive likelihood warm-up strategy to mitigate gradient conflicts; and (2) adaptive directional momentum (ADM) smoothing to reduce fluctuations in the likelihood gradient. Extensive experiments across diverse restoration tasks demonstrate that SPGD significantly enhances generation stability, leading to state-of-the-art performance in quantitative metrics and visually superior results. Code is available at \href{this https URL}{here}.

Title: MK-Pose: Category-Level Object Pose Estimation via Multimodal-Based Keypoint Learning

• Authors: Yifan Yang, Peili Song, Enfan Lan, Dong Liu, Jingtai Liu
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2507.06662
• Pdf URL: https://arxiv.org/pdf/2507.06662
• Copy Paste: [[2507.06662]] MK-Pose: Category-Level Object Pose Estimation via Multimodal-Based Keypoint Learning(https://arxiv.org/abs/2507.06662)
• Keywords: self-supervised
• Abstract: Category-level object pose estimation, which predicts the pose of objects within a known category without prior knowledge of individual instances, is essential in applications like warehouse automation and manufacturing. Existing methods relying on RGB images or point cloud data often struggle with object occlusion and generalization across different instances and categories. This paper proposes a multimodal-based keypoint learning framework (MK-Pose) that integrates RGB images, point clouds, and category-level textual descriptions. The model uses a self-supervised keypoint detection module enhanced with attention-based query generation, soft heatmap matching and graph-based relational modeling. Additionally, a graph-enhanced feature fusion module is designed to integrate local geometric information and global context. MK-Pose is evaluated on CAMERA25 and REAL275 dataset, and is further tested for cross-dataset capability on HouseCat6D dataset. The results demonstrate that MK-Pose outperforms existing state-of-the-art methods in both IoU and average precision without shape priors. Codes will be released at \href{this https URL}{this https URL}.

Title: Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis

• Authors: Hao Tang, Ling Shao, Zhenyu Zhang, Luc Van Gool, Nicu Sebe
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06689
• Pdf URL: https://arxiv.org/pdf/2507.06689
• Copy Paste: [[2507.06689]] Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis(https://arxiv.org/abs/2507.06689)
• Keywords: self-supervised
• Abstract: We propose a novel spatial-temporal graph Mamba (STG-Mamba) for the music-guided dance video synthesis task, i.e., to translate the input music to a dance video. STG-Mamba consists of two translation mappings: music-to-skeleton translation and skeleton-to-video translation. In the music-to-skeleton translation, we introduce a novel spatial-temporal graph Mamba (STGM) block to effectively construct skeleton sequences from the input music, capturing dependencies between joints in both the spatial and temporal dimensions. For the skeleton-to-video translation, we propose a novel self-supervised regularization network to translate the generated skeletons, along with a conditional image, into a dance video. Lastly, we collect a new skeleton-to-video translation dataset from the Internet, containing 54,944 video clips. Extensive experiments demonstrate that STG-Mamba achieves significantly better results than existing methods.

Title: MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport

• Authors: Mahshid Shiri, Cigdem Beyan, Vittorio Murino
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06733
• Pdf URL: https://arxiv.org/pdf/2507.06733
• Copy Paste: [[2507.06733]] MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport(https://arxiv.org/abs/2507.06733)
• Keywords: anomaly
• Abstract: Medical anomaly detection (AD) is challenging due to diverse imaging modalities, anatomical variations, and limited labeled data. We propose a novel approach combining visual adapters and prompt learning with Partial Optimal Transport (POT) and contrastive learning (CL) to improve CLIP's adaptability to medical images, particularly for AD. Unlike standard prompt learning, which often yields a single representation, our method employs multiple prompts aligned with local features via POT to capture subtle abnormalities. CL further enforces intra-class cohesion and inter-class separation. Our method achieves state-of-the-art results in few-shot, zero-shot, and cross-dataset scenarios without synthetic data or memory banks. The code is available at this https URL.

Title: DIFFUMA: High-Fidelity Spatio-Temporal Video Prediction via Dual-Path Mamba and Diffusion Enhancement

• Authors: Xinyu Xie, Weifeng Cao, Jun Shi, Yangyang Hu, Hui Liang, Wanyong Liang, Xiaoliang Qian
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06738
• Pdf URL: https://arxiv.org/pdf/2507.06738
• Copy Paste: [[2507.06738]] DIFFUMA: High-Fidelity Spatio-Temporal Video Prediction via Dual-Path Mamba and Diffusion Enhancement(https://arxiv.org/abs/2507.06738)
• Keywords: diffusion
• Abstract: Spatio-temporal video prediction plays a pivotal role in critical domains, ranging from weather forecasting to industrial automation. However, in high-precision industrial scenarios such as semiconductor manufacturing, the absence of specialized benchmark datasets severely hampers research on modeling and predicting complex processes. To address this challenge, we make a twofold this http URL, we construct and release the Chip Dicing Lane Dataset (CHDL), the first public temporal image dataset dedicated to the semiconductor wafer dicing process. Captured via an industrial-grade vision system, CHDL provides a much-needed and challenging benchmark for high-fidelity process modeling, defect detection, and digital twin this http URL, we propose DIFFUMA, an innovative dual-path prediction architecture specifically designed for such fine-grained dynamics. The model captures global long-range temporal context through a parallel Mamba module, while simultaneously leveraging a diffusion module, guided by temporal features, to restore and enhance fine-grained spatial details, effectively combating feature degradation. Experiments demonstrate that on our CHDL benchmark, DIFFUMA significantly outperforms existing methods, reducing the Mean Squared Error (MSE) by 39% and improving the Structural Similarity (SSIM) from 0.926 to a near-perfect 0.988. This superior performance also generalizes to natural phenomena datasets. Our work not only delivers a new state-of-the-art (SOTA) model but, more importantly, provides the community with an invaluable data resource to drive future research in industrial AI.

Title: PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI

• Authors: Haitham S. Al-Sinani, Chris J. Mitchell
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2507.06742
• Pdf URL: https://arxiv.org/pdf/2507.06742
• Copy Paste: [[2507.06742]] PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI(https://arxiv.org/abs/2507.06742)
• Keywords: generative
• Abstract: Ethical hacking today relies on highly skilled practitioners executing complex sequences of commands, which is inherently time-consuming, difficult to scale, and prone to human error. To help mitigate these limitations, we previously introduced 'PenTest++', an AI-augmented system combining automation with generative AI supporting ethical hacking workflows. However, a key limitation of PenTest++ was its lack of support for privilege escalation, a crucial element of ethical hacking. In this paper we present 'PenTest2.0', a substantial evolution of PenTest++ supporting automated privilege escalation driven entirely by Large Language Model reasoning. It also incorporates several significant enhancements: 'Retrieval-Augmented Generation', including both one-line and offline modes; 'Chain-of-Thought' prompting for intermediate reasoning; persistent 'PenTest Task Trees' to track goal progression across turns; and the optional integration of human-authored hints. We describe how it operates, present a proof-of-concept prototype, and discuss its benefits and limitations. We also describe application of the system to a controlled Linux target, showing it can carry out multi-turn, adaptive privilege escalation. We explain the rationale behind its core design choices, and provide comprehensive testing results and cost analysis. Our findings indicate that 'PenTest2.0' represents a meaningful step toward practical, scalable, AI-automated penetration testing, whilst highlighting the shortcomings of generative AI systems, particularly their sensitivity to prompt structure, execution context, and semantic drift, reinforcing the need for further research and refinement in this emerging space. Keywords: AI, Ethical Hacking, Privilege Escalation, GenAI, ChatGPT, LLM (Large Language Model), HITL (Human-in-the-Loop)

Title: Robust Deep Network Learning of Nonlinear Regression Tasks by Parametric Leaky Exponential Linear Units (LELUs) and a Diffusion Metric

• Authors: Enda D.V. Bigarella
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06765
• Pdf URL: https://arxiv.org/pdf/2507.06765
• Copy Paste: [[2507.06765]] Robust Deep Network Learning of Nonlinear Regression Tasks by Parametric Leaky Exponential Linear Units (LELUs) and a Diffusion Metric(https://arxiv.org/abs/2507.06765)
• Keywords: diffusion
• Abstract: This document proposes a parametric activation function (ac.f.) aimed at improving multidimensional nonlinear data regression. It is a established knowledge that nonlinear ac.f.'s are required for learning nonlinear datasets. This work shows that smoothness and gradient properties of the ac.f. further impact the performance of large neural networks in terms of overfitting and sensitivity to model parameters. Smooth but vanishing-gradient ac.f.'s such as ELU or SiLU have limited performance and non-smooth ac.f.'s such as RELU and Leaky-RELU further impart discontinuity in the trained model. Improved performance is demonstrated with a smooth "Leaky Exponential Linear Unit", with non-zero gradient that can be trained. A novel diffusion-loss metric is also proposed to gauge the performance of the trained models in terms of overfitting.

Title: Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining: Method, Evaluation and Applications

• Authors: Seonwu Kim, Yohan Na, Kihun Kim, Hanhee Cho, Geun Lim, Mintae Kim, Seongik Park, Ki Hyun Kim, Youngsub Han, Byoung-Ki Jeon
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.06795
• Pdf URL: https://arxiv.org/pdf/2507.06795
• Copy Paste: [[2507.06795]] Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining: Method, Evaluation and Applications(https://arxiv.org/abs/2507.06795)
• Keywords: foundation model
• Abstract: The emergence of open-source large language models (LLMs) has expanded opportunities for enterprise applications; however, many organizations still lack the infrastructure to deploy and maintain large-scale models. As a result, small LLMs (sLLMs) have become a practical alternative, despite their inherent performance limitations. While Domain Adaptive Continual Pretraining (DACP) has been previously explored as a method for domain adaptation, its utility in commercial applications remains under-examined. In this study, we validate the effectiveness of applying a DACP-based recipe across diverse foundation models and service domains. Through extensive experiments and real-world evaluations, we demonstrate that DACP-applied sLLMs achieve substantial gains in target domain performance while preserving general capabilities, offering a cost-efficient and scalable solution for enterprise-level deployment.

Title: GreenHyperSpectra: A multi-source hyperspectral dataset for global vegetation trait prediction

• Authors: Eya Cherif (1, 2 and 3), Arthur Ouaknine (3 and 4), Luke A. Brown (5), Phuong D. Dao (6, 7 and 8), Kyle R. Kovach (9), Bing Lu (10), Daniel Mederer (1), Hannes Feilhauer (1, 2, 12 and 13), Teja Kattenborn (11 and 12), David Rolnick (3 and 4) ((1) Institute for Earth System Science and Remote Sensing, Leipzig University, Germany, (2) Center for Scalable Data Analytics and Artificial Intelligence (this http URL), Leipzig University, Germany, (3) Mila Quebec AI Institute, Canada, (4) McGill University, Canada, (5) School of Science, Engineering and Environment, University of Salford, UK, (6) Department of Agricultural Biology, Colorado State University, USA, (7) Graduate Degree Program in Ecology, Colorado State University, USA, (8) School of Global Environmental Sustainability, Colorado State University, USA, (9) Department of Forest and Wildlife Ecology, University of Wisconsin, USA, (10) Department of Geography, Simon Fraser University, Canada, (11) Chair of Sensor-based Geoinformatics (geosense), University of Freiburg, Germany, (12) German Centre for Integrative Biodiversity Research (iDiv), Halle-Jena-Leipzig, Germany, (13) Helmholtz-Centre for Environmental Research (UFZ), Leipzig, Germany)
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.06806
• Pdf URL: https://arxiv.org/pdf/2507.06806
• Copy Paste: [[2507.06806]] GreenHyperSpectra: A multi-source hyperspectral dataset for global vegetation trait prediction(https://arxiv.org/abs/2507.06806)
• Keywords: self-supervised
• Abstract: Plant traits such as leaf carbon content and leaf mass are essential variables in the study of biodiversity and climate change. However, conventional field sampling cannot feasibly cover trait variation at ecologically meaningful spatial scales. Machine learning represents a valuable solution for plant trait prediction across ecosystems, leveraging hyperspectral data from remote sensing. Nevertheless, trait prediction from hyperspectral data is challenged by label scarcity and substantial domain shifts (\eg across sensors, ecological distributions), requiring robust cross-domain methods. Here, we present GreenHyperSpectra, a pretraining dataset encompassing real-world cross-sensor and cross-ecosystem samples designed to benchmark trait prediction with semi- and self-supervised methods. We adopt an evaluation framework encompassing in-distribution and out-of-distribution scenarios. We successfully leverage GreenHyperSpectra to pretrain label-efficient multi-output regression models that outperform the state-of-the-art supervised baseline. Our empirical analyses demonstrate substantial improvements in learning spectral representations for trait prediction, establishing a comprehensive methodological framework to catalyze research at the intersection of representation learning and plant functional traits assessment. All code and data are available at: this https URL.

Title: Democratizing High-Fidelity Co-Speech Gesture Video Generation

• Authors: Xu Yang, Shaoli Huang, Shenbo Xie, Xuelin Chen, Yifei Liu, Changxing Ding
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06812
• Pdf URL: https://arxiv.org/pdf/2507.06812
• Copy Paste: [[2507.06812]] Democratizing High-Fidelity Co-Speech Gesture Video Generation(https://arxiv.org/abs/2507.06812)
• Keywords: diffusion
• Abstract: Co-speech gesture video generation aims to synthesize realistic, audio-aligned videos of speakers, complete with synchronized facial expressions and body gestures. This task presents challenges due to the significant one-to-many mapping between audio and visual content, further complicated by the scarcity of large-scale public datasets and high computational demands. We propose a lightweight framework that utilizes 2D full-body skeletons as an efficient auxiliary condition to bridge audio signals with visual outputs. Our approach introduces a diffusion model conditioned on fine-grained audio segments and a skeleton extracted from the speaker's reference image, predicting skeletal motions through skeleton-audio feature fusion to ensure strict audio coordination and body shape consistency. The generated skeletons are then fed into an off-the-shelf human video generation model with the speaker's reference image to synthesize high-fidelity videos. To democratize research, we present CSG-405-the first public dataset with 405 hours of high-resolution videos across 71 speech types, annotated with 2D skeletons and diverse speaker demographics. Experiments show that our method exceeds state-of-the-art approaches in visual quality and synchronization while generalizing across speakers and contexts.

Title: Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generation

• Authors: Tao Feng, Xianbing Zhao, Zhenhua Chen, Tien Tsin Wong, Hamid Rezatofighi, Gholamreza Haffari, Lizhen Qu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06830
• Pdf URL: https://arxiv.org/pdf/2507.06830
• Copy Paste: [[2507.06830]] Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generation(https://arxiv.org/abs/2507.06830)
• Keywords: diffusion
• Abstract: Recent advances in diffusion-based and autoregressive video generation models have achieved remarkable visual realism. However, these models typically lack accurate physical alignment, failing to replicate real-world dynamics in object motion. This limitation arises primarily from their reliance on learned statistical correlations rather than capturing mechanisms adhering to physical laws. To address this issue, we introduce a novel framework that integrates symbolic regression (SR) and trajectory-guided image-to-video (I2V) models for physics-grounded video forecasting. Our approach extracts motion trajectories from input videos, uses a retrieval-based pre-training mechanism to enhance symbolic regression, and discovers equations of motion to forecast physically accurate future trajectories. These trajectories then guide video generation without requiring fine-tuning of existing models. Evaluated on scenarios in Classical Mechanics, including spring-mass, pendulums, and projectile motions, our method successfully recovers ground-truth analytical equations and improves the physical alignment of generated videos over baseline methods.

Title: DiffSpectra: Molecular Structure Elucidation from Spectra using Diffusion Models

• Authors: Liang Wang, Yu Rong, Tingyang Xu, Zhenyi Zhong, Zhiyuan Liu, Pengju Wang, Deli Zhao, Qiang Liu, Shu Wu, Liang Wang
• Subjects: cs.LG, cs.AI, cs.CE, physics.chem-ph, q-bio.MN
• Abstract URL: https://arxiv.org/abs/2507.06853
• Pdf URL: https://arxiv.org/pdf/2507.06853
• Copy Paste: [[2507.06853]] DiffSpectra: Molecular Structure Elucidation from Spectra using Diffusion Models(https://arxiv.org/abs/2507.06853)
• Keywords: diffusion, generative
• Abstract: Molecular structure elucidation from spectra is a foundational problem in chemistry, with profound implications for compound identification, synthesis, and drug development. Traditional methods rely heavily on expert interpretation and lack scalability. Pioneering machine learning methods have introduced retrieval-based strategies, but their reliance on finite libraries limits generalization to novel molecules. Generative models offer a promising alternative, yet most adopt autoregressive SMILES-based architectures that overlook 3D geometry and struggle to integrate diverse spectral modalities. In this work, we present DiffSpectra, a generative framework that directly infers both 2D and 3D molecular structures from multi-modal spectral data using diffusion models. DiffSpectra formulates structure elucidation as a conditional generation process. Its denoising network is parameterized by Diffusion Molecule Transformer, an SE(3)-equivariant architecture that integrates topological and geometric information. Conditioning is provided by SpecFormer, a transformer-based spectral encoder that captures intra- and inter-spectral dependencies from multi-modal spectra. Extensive experiments demonstrate that DiffSpectra achieves high accuracy in structure elucidation, recovering exact structures with 16.01% top-1 accuracy and 96.86% top-20 accuracy through sampling. The model benefits significantly from 3D geometric modeling, SpecFormer pre-training, and multi-modal conditioning. These results highlight the effectiveness of spectrum-conditioned diffusion modeling in addressing the challenge of molecular structure elucidation. To our knowledge, DiffSpectra is the first framework to unify multi-modal spectral reasoning and joint 2D/3D generative modeling for de novo molecular structure elucidation.

Title: Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement

• Authors: Qiyuan Dai, Hanzhuo Huang, Yu Wu, Sibei Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06928
• Pdf URL: https://arxiv.org/pdf/2507.06928
• Copy Paste: [[2507.06928]] Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement(https://arxiv.org/abs/2507.06928)
• Keywords: self-supervised
• Abstract: Generalized Category Discovery (GCD) aims to recognize unlabeled images from known and novel classes by distinguishing novel classes from known ones, while also transferring knowledge from another set of labeled images with known classes. Existing GCD methods rely on self-supervised vision transformers such as DINO for representation learning. However, focusing solely on the global representation of the DINO CLS token introduces an inherent trade-off between discriminability and generalization. In this paper, we introduce an adaptive part discovery and learning method, called APL, which generates consistent object parts and their correspondences across different similar images using a set of shared learnable part queries and DINO part priors, without requiring any additional annotations. More importantly, we propose a novel all-min contrastive loss to learn discriminative yet generalizable part representation, which adaptively highlights discriminative object parts to distinguish similar categories for enhanced discriminability while simultaneously sharing other parts to facilitate knowledge transfer for improved generalization. Our APL can easily be incorporated into different GCD frameworks by replacing their CLS token feature with our part representations, showing significant enhancements on fine-grained datasets.

Title: What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models

• Authors: Keyon Vafa, Peter G. Chang, Ashesh Rambachan, Sendhil Mullainathan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2507.06952
• Pdf URL: https://arxiv.org/pdf/2507.06952
• Copy Paste: [[2507.06952]] What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models(https://arxiv.org/abs/2507.06952)
• Keywords: foundation model
• Abstract: Foundation models are premised on the idea that sequence prediction can uncover deeper domain understanding, much like how Kepler's predictions of planetary motion later led to the discovery of Newtonian mechanics. However, evaluating whether these models truly capture deeper structure remains a challenge. We develop a technique for evaluating foundation models that examines how they adapt to synthetic datasets generated from some postulated world model. Our technique measures whether the foundation model's inductive bias aligns with the world model, and so we refer to it as an inductive bias probe. Across multiple domains, we find that foundation models can excel at their training tasks yet fail to develop inductive biases towards the underlying world model when adapted to new tasks. We particularly find that foundation models trained on orbital trajectories consistently fail to apply Newtonian mechanics when adapted to new physics tasks. Further analysis reveals that these models behave as if they develop task-specific heuristics that fail to generalize.

Title: Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting

• Authors: Fei Teng, Kai Luo, Sheng Wu, Siyu Li, Pujun Guo, Jiale Wei, Kunyu Peng, Jiaming Zhang, Kailun Yang
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2507.06971
• Pdf URL: https://arxiv.org/pdf/2507.06971
• Copy Paste: [[2507.06971]] Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting(https://arxiv.org/abs/2507.06971)
• Keywords: diffusion
• Abstract: Panoramic perception holds significant potential for autonomous driving, enabling vehicles to acquire a comprehensive 360° surround view in a single shot. However, autonomous driving is a data-driven task. Complete panoramic data acquisition requires complex sampling systems and annotation pipelines, which are time-consuming and labor-intensive. Although existing street view generation models have demonstrated strong data regeneration capabilities, they can only learn from the fixed data distribution of existing datasets and cannot achieve high-quality, controllable panoramic generation. In this paper, we propose the first panoramic generation method Percep360 for autonomous driving. Percep360 enables coherent generation of panoramic data with control signals based on the stitched panoramic data. Percep360 focuses on two key aspects: coherence and controllability. Specifically, to overcome the inherent information loss caused by the pinhole sampling process, we propose the Local Scenes Diffusion Method (LSDM). LSDM reformulates the panorama generation as a spatially continuous diffusion process, bridging the gaps between different data distributions. Additionally, to achieve the controllable generation of panoramic images, we propose a Probabilistic Prompting Method (PPM). PPM dynamically selects the most relevant control cues, enabling controllable panoramic image generation. We evaluate the effectiveness of the generated images from three perspectives: image quality assessment (i.e., no-reference and with reference), controllability, and their utility in real-world Bird's Eye View (BEV) segmentation. Notably, the generated data consistently outperforms the original stitched images in no-reference quality metrics and enhances downstream perception models. The source code will be publicly available at this https URL.

Title: A Principled Framework for Multi-View Contrastive Learning

• Authors: Panagiotis Koromilas, Efthymios Georgiou, Giorgos Bouritsas, Theodoros Giannakopoulos, Mihalis A. Nicolaou, Yannis Panagakis
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2507.06979
• Pdf URL: https://arxiv.org/pdf/2507.06979
• Copy Paste: [[2507.06979]] A Principled Framework for Multi-View Contrastive Learning(https://arxiv.org/abs/2507.06979)
• Keywords: self-supervised
• Abstract: Contrastive Learning (CL), a leading paradigm in Self-Supervised Learning (SSL), typically relies on pairs of data views generated through augmentation. While multiple augmentations per instance (more than two) improve generalization in supervised learning, current CL methods handle additional views suboptimally by simply aggregating different pairwise objectives. This approach suffers from four critical limitations: (L1) it utilizes multiple optimization terms per data point resulting to conflicting objectives, (L2) it fails to model all interactions across views and data points, (L3) it inherits fundamental limitations (e.g. alignment-uniformity coupling) from pairwise CL losses, and (L4) it prevents fully realizing the benefits of increased view multiplicity observed in supervised settings. We address these limitations through two novel loss functions: MV-InfoNCE, which extends InfoNCE to incorporate all possible view interactions simultaneously in one term per data point, and MV-DHEL, which decouples alignment from uniformity across views while scaling interaction complexity with view multiplicity. Both approaches are theoretically grounded - we prove they asymptotically optimize for alignment of all views and uniformity, providing principled extensions to multi-view contrastive learning. Our empirical results on ImageNet1K and three other datasets demonstrate that our methods consistently outperform existing multi-view approaches and effectively scale with increasing view multiplicity. We also apply our objectives to multimodal data and show that, in contrast to other contrastive objectives, they can scale beyond just two modalities. Most significantly, ablation studies reveal that MV-DHEL with five or more views effectively mitigates dimensionality collapse by fully utilizing the embedding space, thereby delivering multi-view benefits observed in supervised learning.

Title: Exact Evaluation of the Accuracy of Diffusion Models for Inverse Problems with Gaussian Data Distributions

• Authors: Emile Pierret, Bruno Galerne
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2507.07008
• Pdf URL: https://arxiv.org/pdf/2507.07008
• Copy Paste: [[2507.07008]] Exact Evaluation of the Accuracy of Diffusion Models for Inverse Problems with Gaussian Data Distributions(https://arxiv.org/abs/2507.07008)
• Keywords: diffusion
• Abstract: Used as priors for Bayesian inverse problems, diffusion models have recently attracted considerable attention in the literature. Their flexibility and high variance enable them to generate multiple solutions for a given task, such as inpainting, super-resolution, and deblurring. However, several unresolved questions remain about how well they perform. In this article, we investigate the accuracy of these models when applied to a Gaussian data distribution for deblurring. Within this constrained context, we are able to precisely analyze the discrepancy between the theoretical resolution of inverse problems and their resolution obtained using diffusion models by computing the exact Wasserstein distance between the distribution of the diffusion model sampler and the ideal distribution of solutions to the inverse problem. Our findings allow for the comparison of different algorithms from the literature.

Title: Integrating Pathology Foundation Models and Spatial Transcriptomics for Cellular Decomposition from Histology Images

• Authors: Yutong Sun, Sichen Zhu, Peng Qiu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.07013
• Pdf URL: https://arxiv.org/pdf/2507.07013
• Copy Paste: [[2507.07013]] Integrating Pathology Foundation Models and Spatial Transcriptomics for Cellular Decomposition from Histology Images(https://arxiv.org/abs/2507.07013)
• Keywords: foundation model
• Abstract: The rapid development of digital pathology and modern deep learning has facilitated the emergence of pathology foundation models that are expected to solve general pathology problems under various disease conditions in one unified model, with or without fine-tuning. In parallel, spatial transcriptomics has emerged as a transformative technology that enables the profiling of gene expression on hematoxylin and eosin (H&E) stained histology images. Spatial transcriptomics unlocks the unprecedented opportunity to dive into existing histology images at a more granular, cellular level. In this work, we propose a lightweight and training-efficient approach to predict cellular composition directly from H&E-stained histology images by leveraging information-enriched feature embeddings extracted from pre-trained pathology foundation models. By training a lightweight multi-layer perceptron (MLP) regressor on cell-type abundances derived via cell2location, our method efficiently distills knowledge from pathology foundation models and demonstrates the ability to accurately predict cell-type compositions from histology images, without physically performing the costly spatial transcriptomics. Our method demonstrates competitive performance compared to existing methods such as Hist2Cell, while significantly reducing computational complexity.

Title: Self-Supervised Learning at the Edge: The Cost of Labeling

• Authors: Roberto Pereira, Fernanda Famá, Asal Rangrazi, Marco Miozzo, Charalampos Kalalas, Paolo Dini
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2507.07033
• Pdf URL: https://arxiv.org/pdf/2507.07033
• Copy Paste: [[2507.07033]] Self-Supervised Learning at the Edge: The Cost of Labeling(https://arxiv.org/abs/2507.07033)
• Keywords: self-supervised
• Abstract: Contrastive learning (CL) has recently emerged as an alternative to traditional supervised machine learning solutions by enabling rich representations from unstructured and unlabeled data. However, CL and, more broadly, self-supervised learning (SSL) methods often demand a large amount of data and computational resources, posing challenges for deployment on resource-constrained edge devices. In this work, we explore the feasibility and efficiency of SSL techniques for edge-based learning, focusing on trade-offs between model performance and energy efficiency. In particular, we analyze how different SSL techniques adapt to limited computational, data, and energy budgets, evaluating their effectiveness in learning robust representations under resource-constrained settings. Moreover, we also consider the energy costs involved in labeling data and assess how semi-supervised learning may assist in reducing the overall energy consumed to train CL models. Through extensive experiments, we demonstrate that tailored SSL strategies can achieve competitive performance while reducing resource consumption by up to 4X, underscoring their potential for energy-efficient learning at the edge.

Title: Discrete Diffusion Models for Language Generation

• Authors: Ashen Weligalle
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2507.07050
• Pdf URL: https://arxiv.org/pdf/2507.07050
• Copy Paste: [[2507.07050]] Discrete Diffusion Models for Language Generation(https://arxiv.org/abs/2507.07050)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as a powerful class of generative models, achieving state-of-the-art results in continuous data domains such as image and video generation. Their core mechanism involves a forward diffusion process that gradually transforms structured data into a Gaussian-like distribution, followed by a learned reverse process to reconstruct the data. While successful in continuous modalities, applying this framework to discrete data-particularly natural language-remains challenging due to token dependency complexities and the lack of a defined generation this http URL thesis investigates the feasibility and performance of discrete diffusion models for natural language generation. Specifically, we evaluate the Discrete Denoising Diffusion Probabilistic Model (D3PM) and compare it with traditional autoregressive (AR) language models. To assess generative performance, we use Bits Per Token (BPT), Negative Log-Likelihood (NLL), Perplexity (PPL), and Batch Processing Speed. Results show the best-performing D3PM model achieves a BPT of 5.72, with a mean of 8.05. The AR model outperforms in compression with a lower mean BPT of 4.59, but D3PM achieves higher processing speed, reaching up to 3.97 batches per sec., indicating potential for parallel this http URL evaluations were conducted under consistent conditions-generating 100,000 tokens per model with a fixed batch size of four-for fair comparison. This research presents a detailed analysis of diffusion-based vs. autoregressive models, highlighting trade-offs in generative quality and efficiency. Findings emphasize both the promise and limitations of diffusion models for discrete data, supporting future work in non-autoregressive language generation.

Title: LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing

• Authors: Jiahao Chen, junhao li, Yiming Wang, Zhe Ma, Yi Jiang, Chunyi Zhou, Qingming Li, Tianyu Du, Shouling Ji
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.07056
• Pdf URL: https://arxiv.org/pdf/2507.07056
• Copy Paste: [[2507.07056]] LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing(https://arxiv.org/abs/2507.07056)
• Keywords: diffusion, generative
• Abstract: The proliferation of Low-Rank Adaptation (LoRA) models has democratized personalized text-to-image generation, enabling users to share lightweight models (e.g., personal portraits) on platforms like Civitai and Liblib. However, this "share-and-play" ecosystem introduces critical risks: benign LoRAs can be weaponized by adversaries to generate harmful content (e.g., political, defamatory imagery), undermining creator rights and platform safety. Existing defenses like concept-erasure methods focus on full diffusion models (DMs), neglecting LoRA's unique role as a modular adapter and its vulnerability to adversarial prompt engineering. To bridge this gap, we propose LoRAShield, the first data-free editing framework for securing LoRA models against misuse. Our platform-driven approach dynamically edits and realigns LoRA's weight subspace via adversarial optimization and semantic augmentation. Experimental results demonstrate that LoRAShield achieves remarkable effectiveness, efficiency, and robustness in blocking malicious generations without sacrificing the functionality of the benign task. By shifting the defense to platforms, LoRAShield enables secure, scalable sharing of personalized models, a critical step toward trustworthy generative ecosystems.

Title: Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models

• Authors: Tiezheng Zhang, Yitong Li, Yu-cheng Chou, Jieneng Chen, Alan Yuille, Chen Wei, Junfei Xiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2507.07104
• Pdf URL: https://arxiv.org/pdf/2507.07104
• Copy Paste: [[2507.07104]] Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models(https://arxiv.org/abs/2507.07104)
• Keywords: diffusion
• Abstract: Building state-of-the-art Vision-Language Models (VLMs) with strong captioning capabilities typically necessitates training on billions of high-quality image-text pairs, requiring millions of GPU hours. This paper introduces the Vision-Language-Vision (VLV) auto-encoder framework, which strategically leverages key pretrained components: a vision encoder, the decoder of a Text-to-Image (T2I) diffusion model, and subsequently, a Large Language Model (LLM). Specifically, we establish an information bottleneck by regularizing the language representation space, achieved through freezing the pretrained T2I diffusion decoder. Our VLV pipeline effectively distills knowledge from the text-conditioned diffusion model using continuous embeddings, demonstrating comprehensive semantic understanding via high-quality reconstructions. Furthermore, by fine-tuning a pretrained LLM to decode the intermediate language representations into detailed descriptions, we construct a state-of-the-art (SoTA) captioner comparable to leading models like GPT-4o and Gemini 2.0 Flash. Our method demonstrates exceptional cost-efficiency and significantly reduces data requirements; by primarily utilizing single-modal images for training and maximizing the utility of existing pretrained models (image encoder, T2I diffusion model, and LLM), it circumvents the need for massive paired image-text datasets, keeping the total training expenditure under $1,000 USD.

Title: Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractor

• Authors: Vatsal Agarwal, Matthew Gwilliam, Gefen Kohavi, Eshan Verma, Daniel Ulbricht, Abhinav Shrivastava
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2507.07106
• Pdf URL: https://arxiv.org/pdf/2507.07106
• Copy Paste: [[2507.07106]] Towards Multimodal Understanding via Stable Diffusion as a Task-Aware Feature Extractor(https://arxiv.org/abs/2507.07106)
• Keywords: diffusion
• Abstract: Recent advances in multimodal large language models (MLLMs) have enabled image-based question-answering capabilities. However, a key limitation is the use of CLIP as the visual encoder; while it can capture coarse global information, it often can miss fine-grained details that are relevant to the input query. To address these shortcomings, this work studies whether pre-trained text-to-image diffusion models can serve as instruction-aware visual encoders. Through an analysis of their internal representations, we find diffusion features are both rich in semantics and can encode strong image-text alignment. Moreover, we find that we can leverage text conditioning to focus the model on regions relevant to the input question. We then investigate how to align these features with large language models and uncover a leakage phenomenon, where the LLM can inadvertently recover information from the original diffusion prompt. We analyze the causes of this leakage and propose a mitigation strategy. Based on these insights, we explore a simple fusion strategy that utilizes both CLIP and conditional diffusion features. We evaluate our approach on both general VQA and specialized MLLM benchmarks, demonstrating the promise of diffusion models for visual understanding, particularly in vision-centric tasks that require spatial and compositional reasoning. Our project page can be found this https URL.