2025-02-14

Title: Few-shot_LLM_Synthetic_Data_with_Distribution_Matching

• Authors: Jiyuan Ren, Zhaocheng Du, Zhihao Wen, Qinglin Jia, Sunhao Dai, Chuhan Wu, Zhenhua Dong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08661
• Pdf URL: https://arxiv.org/pdf/2502.08661
• Copy Paste: [[2502.08661]] Few-shot_LLM_Synthetic_Data_with_Distribution_Matching(https://arxiv.org/abs/2502.08661)
• Keywords: in-context
• Abstract: As large language models (LLMs) advance, their ability to perform in-context learning and few-shot language generation has improved significantly. This has spurred using LLMs to produce high-quality synthetic data to enhance the performance of smaller models like online retrievers or weak LLMs. However, LLM-generated synthetic data often differs from the real data in key language attributes (e.g., styles, tones, content proportions, etc.). As a result, mixing these synthetic data directly with real data may distort the original data distribution, potentially hindering performance improvements. To solve this, we introduce SynAlign: a synthetic data generation and filtering framework based on key attribute distribution matching. Before generation, SynAlign employs an uncertainty tracker surrogated by the Gaussian Process model to iteratively select data clusters distinct from selected ones as demonstrations for new data synthesis, facilitating the efficient exploration diversity of the real data. Then, a latent attribute reasoning method is employed: the LLM summarizes linguistic attributes of demonstrations and then synthesizes new data based on them. This approach facilitates synthesizing diverse data with linguistic attributes that appear in real this http URL generation, the Maximum Mean Discrepancy is used as the objective function to learn the sampling weight of each synthetic data, ensuring distribution matching with the real data. Our experiments on multiple text prediction tasks show significant performance improvements. We also conducted an online A/B test on an online retriever to demonstrate SynAlign's effectiveness.

Title: Hallucination, Monofacts, and Miscalibration: An Empirical Investigation

• Authors: Muqing Miao, Michael Kearns
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08666
• Pdf URL: https://arxiv.org/pdf/2502.08666
• Copy Paste: [[2502.08666]] Hallucination, Monofacts, and Miscalibration: An Empirical Investigation(https://arxiv.org/abs/2502.08666)
• Keywords: in-context
• Abstract: Recent theoretical work by [Kalai and Vempala 2024] proves that a particular notion of hallucination rate in LLMs must be lower bounded by the training data monofact rate (related to the classical Good-Turing missing mass estimator) minus model miscalibration. Through systematic experiments with n-gram models and in-context learning with LLMs, we empirically investigate and validate this theory by examining how different underlying data distributions affect the monofact rate and a model's tendency to hallucinate. We then vary model miscalibration through controlled upweighting of training samples while holding monofact rates constant, allowing us to isolate miscalibration's reduction effect on hallucination. These findings suggest that both the distribution of fact frequencies in training data and the calibration-hallucination trade-off are inherent to probabilistic language generation. Our results also suggest that current practices of aggressive deduplication in training data may need to be reconsidered, as selective duplication could serve as a principled mechanism for reducing hallucination.

Title: Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation

• Authors: Hoigi Seo, Wongi Jeong, Jae-sun Seo, Se Young Chun
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08690
• Pdf URL: https://arxiv.org/pdf/2502.08690
• Copy Paste: [[2502.08690]] Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation(https://arxiv.org/abs/2502.08690)
• Keywords: diffusion
• Abstract: Large-scale text encoders in text-to-image (T2I) diffusion models have demonstrated exceptional performance in generating high-quality images from textual prompts. Unlike denoising modules that rely on multiple iterative steps, text encoders require only a single forward pass to produce text embeddings. However, despite their minimal contribution to total inference time and floating-point operations (FLOPs), text encoders demand significantly higher memory usage, up to eight times more than denoising modules. To address this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet effective pruning strategy specifically designed for text encoders in T2I diffusion models. Skrr exploits the inherent redundancy in transformer blocks by selectively skipping or reusing certain layers in a manner tailored for T2I tasks, thereby reducing memory consumption without compromising performance. Extensive experiments demonstrate that Skrr maintains image quality comparable to the original model even under high sparsity levels, outperforming existing blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory efficiency while preserving performance across multiple evaluation metrics, including the FID, CLIP, DreamSim, and GenEval scores.

Title: Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics

• Authors: Sebastian Sanokowski, Wilhelm Berghammer, Martin Ennemoser, Haoyu Peter Wang, Sepp Hochreiter, Sebastian Lehner
• Subjects: cs.LG, cond-mat.stat-mech, cs.AI, physics.comp-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08696
• Pdf URL: https://arxiv.org/pdf/2502.08696
• Copy Paste: [[2502.08696]] Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics(https://arxiv.org/abs/2502.08696)
• Keywords: diffusion, generative
• Abstract: Learning to sample from complex unnormalized distributions over discrete domains emerged as a promising research direction with applications in statistical physics, variational inference, and combinatorial optimization. Recent work has demonstrated the potential of diffusion models in this domain. However, existing methods face limitations in memory scaling and thus the number of attainable diffusion steps since they require backpropagation through the entire generative process. To overcome these limitations we introduce two novel training methods for discrete diffusion samplers, one grounded in the policy gradient theorem and the other one leveraging Self-Normalized Neural Importance Sampling (SN-NIS). These methods yield memory-efficient training and achieve state-of-the-art results in unsupervised combinatorial optimization. Numerous scientific applications additionally require the ability of unbiased sampling. We introduce adaptations of SN-NIS and Neural Markov Chain Monte Carlo that enable for the first time the application of discrete diffusion models to this problem. We validate our methods on Ising model benchmarks and find that they outperform popular autoregressive approaches. Our work opens new avenues for applying diffusion models to a wide range of scientific applications in discrete domains that were hitherto restricted to exact likelihood models.

Title: HistoSmith: Single-Stage Histology Image-Label Generation via Conditional Latent Diffusion for Enhanced Cell Segmentation and Classification

• Authors: Valentina Vadori, Jean-Marie Graïc, Antonella Peruffo, Livio Finos, Ujwala Kiran Chaudhari, Enrico Grisan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08754
• Pdf URL: https://arxiv.org/pdf/2502.08754
• Copy Paste: [[2502.08754]] HistoSmith: Single-Stage Histology Image-Label Generation via Conditional Latent Diffusion for Enhanced Cell Segmentation and Classification(https://arxiv.org/abs/2502.08754)
• Keywords: diffusion
• Abstract: Precise segmentation and classification of cell instances are vital for analyzing the tissue microenvironment in histology images, supporting medical diagnosis, prognosis, treatment planning, and studies of brain cytoarchitecture. However, the creation of high-quality annotated datasets for training remains a major challenge. This study introduces a novel single-stage approach (HistoSmith) for generating image-label pairs to augment histology datasets. Unlike state-of-the-art methods that utilize diffusion models with separate components for label and image generation, our approach employs a latent diffusion model to learn the joint distribution of cellular layouts, classification masks, and histology images. This model enables tailored data generation by conditioning on user-defined parameters such as cell types, quantities, and tissue types. Trained on the Conic H&E histopathology dataset and the Nissl-stained CytoDArk0 dataset, the model generates realistic and diverse labeled samples. Experimental results demonstrate improvements in cell instance segmentation and classification, particularly for underrepresented cell types like neutrophils in the Conic dataset. These findings underscore the potential of our approach to address data scarcity challenges.

Title: Cluster and Predict Latents Patches for Improved Masked Image Modeling

• Authors: Timothée Darcet, Federico Baldassarre, Maxime Oquab, Julien Mairal, Piotr Bojanowski
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08769
• Pdf URL: https://arxiv.org/pdf/2502.08769
• Copy Paste: [[2502.08769]] Cluster and Predict Latents Patches for Improved Masked Image Modeling(https://arxiv.org/abs/2502.08769)
• Keywords: self-supervised
• Abstract: Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this paper, we systematically analyze target representations, loss functions, and architectures, to introduce CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2. We release all our code and models.

Title: If Multi-Agent Debate is the Answer, What is the Question?

• Authors: Hangfan Zhang, Zhiyao Cui, Xinrun Wang, Qiaosheng Zhang, Zhen Wang, Dinghao Wu, Shuyue Hu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08788
• Pdf URL: https://arxiv.org/pdf/2502.08788
• Copy Paste: [[2502.08788]] If Multi-Agent Debate is the Answer, What is the Question?(https://arxiv.org/abs/2502.08788)
• Keywords: foundation model
• Abstract: Multi-agent debate (MAD) has emerged as a promising approach to enhance the factual accuracy and reasoning quality of large language models (LLMs) by engaging multiple agents in iterative discussions during inference. Despite its potential, we argue that current MAD research suffers from critical shortcomings in evaluation practices, including limited dataset overlap and inconsistent baselines, raising significant concerns about generalizability. Correspondingly, this paper presents a systematic evaluation of five representative MAD methods across nine benchmarks using four foundational models. Surprisingly, our findings reveal that MAD methods fail to reliably outperform simple single-agent baselines such as Chain-of-Thought and Self-Consistency, even when consuming additional inference-time computation. From our analysis, we found that model heterogeneity can significantly improve MAD frameworks. We propose Heter-MAD enabling a single LLM agent to access the output from heterogeneous foundation models, which boosts the performance of current MAD frameworks. Finally, we outline potential directions for advancing MAD, aiming to spark a broader conversation and inspire future work in this area.

Title: Deep EEG Super-Resolution: Upsampling EEG Spatial Resolution with Generative Adversarial Networks

• Authors: Isaac Corley, Yufei Huang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08803
• Pdf URL: https://arxiv.org/pdf/2502.08803
• Copy Paste: [[2502.08803]] Deep EEG Super-Resolution: Upsampling EEG Spatial Resolution with Generative Adversarial Networks(https://arxiv.org/abs/2502.08803)
• Keywords: generative
• Abstract: Electroencephalography (EEG) activity contains a wealth of information about what is happening within the human brain. Recording more of this data has the potential to unlock endless future applications. However, the cost of EEG hardware is increasingly expensive based upon the number of EEG channels being recorded simultaneously. We combat this problem in this paper by proposing a novel deep EEG super-resolution (SR) approach based on Generative Adversarial Networks (GANs). This approach can produce high spatial resolution EEG data from low resolution samples, by generating channel-wise upsampled data to effectively interpolate numerous missing channels, thus reducing the need for expensive EEG equipment. We tested the performance using an EEG dataset from a mental imagery task. Our proposed GAN model provided 10^4 fold and 10^2 fold reduction in mean-squared error (MSE) and mean-absolute error (MAE), respectively, over the baseline bicubic interpolation method. We further validate our method by training a classifier on the original classification task, which displayed minimal loss in accuracy while using the super-resolved data. The proposed SR EEG by GAN is a promising approach to improve the spatial resolution of low density EEG headsets.

Title: A First-order Generative Bilevel Optimization Framework for Diffusion Models

• Authors: Quan Xiao, Hui Yuan, A F M Saif, Gaowen Liu, Ramana Kompella, Mengdi Wang, Tianyi Chen
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08808
• Pdf URL: https://arxiv.org/pdf/2502.08808
• Copy Paste: [[2502.08808]] A First-order Generative Bilevel Optimization Framework for Diffusion Models(https://arxiv.org/abs/2502.08808)
• Keywords: diffusion, generative
• Abstract: Diffusion models, which iteratively denoise data samples to synthesize high-quality outputs, have achieved empirical success across domains. However, optimizing these models for downstream tasks often involves nested bilevel structures, such as tuning hyperparameters for fine-tuning tasks or noise schedules in training dynamics, where traditional bilevel methods fail due to the infinite-dimensional probability space and prohibitive sampling costs. We formalize this challenge as a generative bilevel optimization problem and address two key scenarios: (1) fine-tuning pre-trained models via an inference-only lower-level solver paired with a sample-efficient gradient estimator for the upper level, and (2) training diffusion models from scratch with noise schedule optimization by reparameterizing the lower-level problem and designing a computationally tractable gradient estimator. Our first-order bilevel framework overcomes the incompatibility of conventional bilevel methods with diffusion processes, offering theoretical grounding and computational practicality. Experiments demonstrate that our method outperforms existing fine-tuning and hyperparameter search baselines.

Title: DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps

• Authors: Jocelyn Dzuong
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08821
• Pdf URL: https://arxiv.org/pdf/2502.08821
• Copy Paste: [[2502.08821]] DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps(https://arxiv.org/abs/2502.08821)
• Keywords: diffusion, generative
• Abstract: The recent surge in advanced generative models, such as diffusion models and generative adversarial networks (GANs), has led to an alarming rise in AI-generated images across various domains on the web. While such technologies offer benefits such as democratizing artistic creation, they also pose challenges in misinformation, digital forgery, and authenticity verification. Additionally, the uncredited use of AI-generated images in media and marketing has sparked significant backlash from online communities. In response to this, we introduce DejAIvu, a Chrome Web extension that combines real-time AI-generated image detection with saliency-based explainability while users browse the web. Using an ONNX-optimized deep learning model, DejAIvu automatically analyzes images on websites such as Google Images, identifies AI-generated content using model inference, and overlays a saliency heatmap to highlight AI-related artifacts. Our approach integrates efficient in-browser inference, gradient-based saliency analysis, and a seamless user experience, ensuring that AI detection is both transparent and interpretable. We also evaluate DejAIvu across multiple pretrained architectures and benchmark datasets, demonstrating high accuracy and low latency, making it a practical and deployable tool for enhancing AI image accountability. The code for this system can be found at this https URL.

Title: $\mathsf{CSMAE~}$:~Cataract Surgical Masked Autoencoder (MAE) based Pre-training

• Authors: Nisarg A. Shah, Wele Gedara Chaminda Bandara, Shameema Skider, S. Swaroop Vedula, Vishal M. Patel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08822
• Pdf URL: https://arxiv.org/pdf/2502.08822
• Copy Paste: [[2502.08822]] $\mathsf{CSMAE~}$:~Cataract Surgical Masked Autoencoder (MAE) based Pre-training(https://arxiv.org/abs/2502.08822)
• Keywords: self-supervised
• Abstract: Automated analysis of surgical videos is crucial for improving surgical training, workflow optimization, and postoperative assessment. We introduce a CSMAE, Masked Autoencoder (MAE)-based pretraining approach, specifically developed for Cataract Surgery video analysis, where instead of randomly selecting tokens for masking, they are selected based on the spatiotemporal importance of the token. We created a large dataset of cataract surgery videos to improve the model's learning efficiency and expand its robustness in low-data regimes. Our pre-trained model can be easily adapted for specific downstream tasks via fine-tuning, serving as a robust backbone for further analysis. Through rigorous testing on a downstream step-recognition task on two Cataract Surgery video datasets, D99 and Cataract-101, our approach surpasses current state-of-the-art self-supervised pretraining and adapter-based transfer learning methods by a significant margin. This advancement not only demonstrates the potential of our MAE-based pretraining in the field of surgical video analysis but also sets a new benchmark for future research.

Title: A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective

• Authors: Wangyang Ying, Cong Wei, Nanxu Gong, Xinyuan Wang, Haoyue Bai, Arun Vignesh Malarkkan, Sixun Dong, Dongjie Wang, Denghui Zhang, Yanjie Fu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08828
• Pdf URL: https://arxiv.org/pdf/2502.08828
• Copy Paste: [[2502.08828]] A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective(https://arxiv.org/abs/2502.08828)
• Keywords: generative
• Abstract: Tabular data is one of the most widely used data formats across various domains such as bioinformatics, healthcare, and marketing. As artificial intelligence moves towards a data-centric perspective, improving data quality is essential for enhancing model performance in tabular data-driven applications. This survey focuses on data-driven tabular data optimization, specifically exploring reinforcement learning (RL) and generative approaches for feature selection and feature generation as fundamental techniques for refining data spaces. Feature selection aims to identify and retain the most informative attributes, while feature generation constructs new features to better capture complex data patterns. We systematically review existing generative methods for tabular data engineering, analyzing their latest advancements, real-world applications, and respective strengths and limitations. This survey emphasizes how RL-based and generative techniques contribute to the automation and intelligence of feature engineering. Finally, we summarize the existing challenges and discuss future research directions, aiming to provide insights that drive continued innovation in this field.

Title: A Reversible Solver for Diffusion SDEs

• Authors: Zander W. Blasingame, Chen Liu
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08834
• Pdf URL: https://arxiv.org/pdf/2502.08834
• Copy Paste: [[2502.08834]] A Reversible Solver for Diffusion SDEs(https://arxiv.org/abs/2502.08834)
• Keywords: diffusion
• Abstract: Diffusion models have quickly become the state-of-the-art for generation tasks across many different data modalities. An important ability of diffusion models is the ability to encode samples from the data distribution back into the sampling prior distribution. This is useful for performing alterations to real data samples along with guided generation via the continuous adjoint equations. We propose an algebraically reversible solver for diffusion SDEs that can exactly invert real data samples into the prior distribution.

Title: Hierarchical Entropy Disruption for Ransomware Detection: A Computationally-Driven Framework

• Authors: Hayden Srynn, Gilbert Pomeroy, Florence Lytton, Godfrey Ashcombe, Valentine Harcourt, Duncan Pettigrew
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.08843
• Pdf URL: https://arxiv.org/pdf/2502.08843
• Copy Paste: [[2502.08843]] Hierarchical Entropy Disruption for Ransomware Detection: A Computationally-Driven Framework(https://arxiv.org/abs/2502.08843)
• Keywords: anomaly
• Abstract: The rapid evolution of encryption-based threats has rendered conventional detection mechanisms increasingly ineffective against sophisticated attack strategies. Monitoring entropy variations across hierarchical system levels offers an alternative approach to identifying unauthorized data modifications without relying on static signatures. A framework leveraging hierarchical entropy disruption was introduced to analyze deviations in entropy distributions, capturing behavioral anomalies indicative of malicious encryption operations. Evaluating the framework across multiple ransomware variants demonstrated its capability to achieve high detection accuracy while maintaining minimal computational overhead. Entropy distributions across different system directories revealed that encryption activities predominantly targeted user-accessible files, aligning with observed attacker strategies. Detection latency analysis indicated that early-stage identification was feasible, mitigating potential data loss before critical system impact occurred. The framework's ability to operate efficiently in real-time environments was validated through an assessment of resource utilization, confirming a balanced trade-off between detection precision and computational efficiency. Comparative benchmarking against established detection methods highlighted the limitations of conventional approaches in identifying novel ransomware variants, whereas entropy-based anomaly detection provided resilience against obfuscation techniques.

Title: A Systematic Evaluation of Generative Models on Tabular Transportation Data

• Authors: Chengen Wang, Alvaro Cardenas, Gurcan Comert, Murat Kantarcioglu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08856
• Pdf URL: https://arxiv.org/pdf/2502.08856
• Copy Paste: [[2502.08856]] A Systematic Evaluation of Generative Models on Tabular Transportation Data(https://arxiv.org/abs/2502.08856)
• Keywords: generative
• Abstract: The sharing of large-scale transportation data is beneficial for transportation planning and policymaking. However, it also raises significant security and privacy concerns, as the data may include identifiable personal information, such as individuals' home locations. To address these concerns, synthetic data generation based on real transportation data offers a promising solution that allows privacy protection while potentially preserving data utility. Although there are various synthetic data generation techniques, they are often not tailored to the unique characteristics of transportation data, such as the inherent structure of transportation networks formed by all trips in the datasets. In this paper, we use New York City taxi data as a case study to conduct a systematic evaluation of the performance of widely used tabular data generative models. In addition to traditional metrics such as distribution similarity, coverage, and privacy preservation, we propose a novel graph-based metric tailored specifically for transportation data. This metric evaluates the similarity between real and synthetic transportation networks, providing potentially deeper insights into their structural and functional alignment. We also introduced an improved privacy metric to address the limitations of the commonly-used one. Our experimental results reveal that existing tabular data generative models often fail to perform as consistently as claimed in the literature, particularly when applied to transportation data use cases. Furthermore, our novel graph metric reveals a significant gap between synthetic and real data. This work underscores the potential need to develop generative models specifically tailored to take advantage of the unique characteristics of emerging domains, such as transportation.

Title: Generative AI for Internet of Things Security: Challenges and Opportunities

• Authors: Yan Lin Aung, Ivan Christian, Ye Dong, Xiaodong Ye, Sudipta Chattopadhyay, Jianying Zhou
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08886
• Pdf URL: https://arxiv.org/pdf/2502.08886
• Copy Paste: [[2502.08886]] Generative AI for Internet of Things Security: Challenges and Opportunities(https://arxiv.org/abs/2502.08886)
• Keywords: generative
• Abstract: As Generative AI (GenAI) continues to gain prominence and utility across various sectors, their integration into the realm of Internet of Things (IoT) security evolves rapidly. This work delves into an examination of the state-of-the-art literature and practical applications on how GenAI could improve and be applied in the security landscape of IoT. Our investigation aims to map the current state of GenAI implementation within IoT security, exploring their potential to fortify security measures further. Through the compilation, synthesis, and analysis of the latest advancements in GenAI technologies applied to IoT, this paper not only introduces fresh insights into the field, but also lays the groundwork for future research directions. It explains the prevailing challenges within IoT security, discusses the effectiveness of GenAI in addressing these issues, and identifies significant research gaps through MITRE Mitigations. Accompanied with three case studies, we provide a comprehensive overview of the progress and future prospects of GenAI applications in IoT security. This study serves as a foundational resource to improve IoT security through the innovative application of GenAI, thus contributing to the broader discourse on IoT security and technology integration.

Title: Diffusion Models Through a Global Lens: Are They Culturally Inclusive?

• Authors: Zahra Bayramli, Ayhan Suleymanzade, Na Min An, Huzama Ahmad, Eunsu Kim, Junyeong Park, James Thorne, Alice Oh
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08914
• Pdf URL: https://arxiv.org/pdf/2502.08914
• Copy Paste: [[2502.08914]] Diffusion Models Through a Global Lens: Are They Culturally Inclusive?(https://arxiv.org/abs/2502.08914)
• Keywords: diffusion, generative
• Abstract: Text-to-image diffusion models have recently enabled the creation of visually compelling, detailed images from textual prompts. However, their ability to accurately represent various cultural nuances remains an open question. In our work, we introduce CultDiff benchmark, evaluating state-of-the-art diffusion models whether they can generate culturally specific images spanning ten countries. We show that these models often fail to generate cultural artifacts in architecture, clothing, and food, especially for underrepresented country regions, by conducting a fine-grained analysis of different similarity aspects, revealing significant disparities in cultural relevance, description fidelity, and realism compared to real-world reference images. With the collected human evaluations, we develop a neural-based image-image similarity metric, namely, CultDiff-S, to predict human judgment on real and generated images with cultural artifacts. Our work highlights the need for more inclusive generative AI systems and equitable dataset representation over a wide range of cultures.

Title: Dynamic watermarks in images generated by diffusion models

• Authors: Yunzhuo Chen, Naveed Akhtar, Nur Al Hasan Haldar, Ajmal Mian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08927
• Pdf URL: https://arxiv.org/pdf/2502.08927
• Copy Paste: [[2502.08927]] Dynamic watermarks in images generated by diffusion models(https://arxiv.org/abs/2502.08927)
• Keywords: diffusion
• Abstract: High-fidelity text-to-image diffusion models have revolutionized visual content generation, but their widespread use raises significant ethical concerns, including intellectual property protection and the misuse of synthetic media. To address these challenges, we propose a novel multi-stage watermarking framework for diffusion models, designed to establish copyright and trace generated images back to their source. Our multi-stage watermarking technique involves embedding: (i) a fixed watermark that is localized in the diffusion model's learned noise distribution and, (ii) a human-imperceptible, dynamic watermark in generates images, leveraging a fine-tuned decoder. By leveraging the Structural Similarity Index Measure (SSIM) and cosine similarity, we adapt the watermark's shape and color to the generated content while maintaining robustness. We demonstrate that our method enables reliable source verification through watermark classification, even when the dynamic watermark is adjusted for content-specific variations. Source model verification is enabled through watermark classification. o support further research, we generate a dataset of watermarked images and introduce a methodology to evaluate the statistical impact of watermarking on generated this http URL, we rigorously test our framework against various attack scenarios, demonstrating its robustness and minimal impact on image quality. Our work advances the field of AI-generated content security by providing a scalable solution for model ownership verification and misuse prevention.

Title: The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding

• Authors: Mo Yu, Lemao Liu, Junjie Wu, Tsz Ting Chung, Shunchi Zhang, Jiangnan Li, Dit-Yan Yeung, Jie Zhou
• Subjects: cs.CL, cs.AI, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08946
• Pdf URL: https://arxiv.org/pdf/2502.08946
• Copy Paste: [[2502.08946]] The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding(https://arxiv.org/abs/2502.08946)
• Keywords: in-context
• Abstract: In a systematic way, we investigate a widely asked question: Do LLMs really understand what they say?, which relates to the more familiar term Stochastic Parrot. To this end, we propose a summative assessment over a carefully designed physical concept understanding task, PhysiCo. Our task alleviates the memorization issue via the usage of grid-format inputs that abstractly describe physical phenomena. The grids represents varying levels of understanding, from the core phenomenon, application examples to analogies to other abstract patterns in the grid world. A comprehensive study on our task demonstrates: (1) state-of-the-art LLMs, including GPT-4o, o1 and Gemini 2.0 flash thinking, lag behind humans by ~40%; (2) the stochastic parrot phenomenon is present in LLMs, as they fail on our grid task but can describe and recognize the same concepts well in natural language; (3) our task challenges the LLMs due to intrinsic difficulties rather than the unfamiliar grid format, as in-context learning and fine-tuning on same formatted data added little to their performance.

Title: Self-Supervised Graph Contrastive Pretraining for Device-level Integrated Circuits

• Authors: Sungyoung Lee, Ziyi Wang, Seunggeun Kim, Taekyun Lee, David Z. Pan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08949
• Pdf URL: https://arxiv.org/pdf/2502.08949
• Copy Paste: [[2502.08949]] Self-Supervised Graph Contrastive Pretraining for Device-level Integrated Circuits(https://arxiv.org/abs/2502.08949)
• Keywords: self-supervised
• Abstract: Self-supervised graph representation learning has driven significant advancements in domains such as social network analysis, molecular design, and electronics design automation (EDA). However, prior works in EDA have mainly focused on the representation of gate-level digital circuits, failing to capture analog and mixed-signal circuits. To address this gap, we introduce DICE: Device-level Integrated Circuits Encoder, the first self-supervised pretrained graph neural network (GNN) model for any circuit expressed at the device level. DICE is a message-passing neural network (MPNN) trained through graph contrastive learning, and its pretraining process is simulation-free, incorporating two novel data augmentation techniques. Experimental results demonstrate that DICE achieves substantial performance gains across three downstream tasks, underscoring its effectiveness for both analog and digital circuits.

Title: Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning

• Authors: Hyundong Cho, Karishma Sharma, Nicolaas Jedema, Leonardo F. R. Ribeiro, Alessandro Moschitti, Ravi Krishnan, Jonathan May
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08972
• Pdf URL: https://arxiv.org/pdf/2502.08972
• Copy Paste: [[2502.08972]] Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning(https://arxiv.org/abs/2502.08972)
• Keywords: in-context
• Abstract: Language models are aligned to the collective voice of many, resulting in generic outputs that do not align with specific users' styles. In this work, we present Trial-Error-Explain In-Context Learning (TICL), a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user. TICL iteratively expands an in-context learning prompt via a trial-error-explain process, adding model-generated negative samples and explanations that provide fine-grained guidance towards a specific user's style. TICL achieves favorable win rates on pairwise comparisons with LLM-as-a-judge up to 91.5% against the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles. Both lexical and qualitative analyses show that the negative samples and explanations enable language models to learn stylistic context more effectively and overcome the bias towards structural and formal phrases observed in their zero-shot outputs. By front-loading inference compute to create a user-specific in-context learning prompt that does not require extra generation steps at test time, TICL presents a novel yet simple approach for personalized alignment.

Title: What exactly has TabPFN learned to do?

• Authors: Calvin McCarter
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08978
• Pdf URL: https://arxiv.org/pdf/2502.08978
• Copy Paste: [[2502.08978]] What exactly has TabPFN learned to do?(https://arxiv.org/abs/2502.08978)
• Keywords: in-context
• Abstract: TabPFN [Hollmann et al., 2023], a Transformer model pretrained to perform in-context learning on fresh tabular classification problems, was presented at the last ICLR conference. To better understand its behavior, we treat it as a black-box function approximator generator and observe its generated function approximations on a varied selection of training datasets. Exploring its learned inductive biases in this manner, we observe behavior that is at turns either brilliant or baffling. We conclude this post with thoughts on how these results might inform the development, evaluation, and application of prior-data fitted networks (PFNs) in the future.

Title: Task Generalization With AutoRegressive Compositional Structure: Can Learning From $\d$ Tasks Generalize to $\d^{T}$ Tasks?

• Authors: Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen, Hongzhou Lin, Jingzhao Zhang, Mikhail Belkin
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08991
• Pdf URL: https://arxiv.org/pdf/2502.08991
• Copy Paste: [[2502.08991]] Task Generalization With AutoRegressive Compositional Structure: Can Learning From $\d$ Tasks Generalize to $\d^{T}$ Tasks?(https://arxiv.org/abs/2502.08991)
• Keywords: in-context
• Abstract: Large language models (LLMs) exhibit remarkable task generalization, solving tasks they were never explicitly trained on with only a few demonstrations. This raises a fundamental question: When can learning from a small set of tasks generalize to a large task family? In this paper, we investigate task generalization through the lens of AutoRegressive Compositional (ARC) structure, where each task is a composition of $T$ operations, and each operation is among a finite family of $\d$ subtasks. This yields a total class of size~\( \d^\TT \). We first show that generalization to all \( \d^\TT \) tasks is theoretically achievable by training on only \( \tilde{O}(\d) \) tasks. Empirically, we demonstrate that Transformers achieve such exponential task generalization on sparse parity functions via in-context learning (ICL) and Chain-of-Thought (CoT) reasoning. We further demonstrate this generalization in arithmetic and language translation, extending beyond parity functions.

Title: Privacy-Preserving Hybrid Ensemble Model for Network Anomaly Detection: Balancing Security and Data Protection

• Authors: Shaobo Liu, Zihao Zhao, Weijie He, Jiren Wang, Jing Peng, Haoyuan Ma
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09001
• Pdf URL: https://arxiv.org/pdf/2502.09001
• Copy Paste: [[2502.09001]] Privacy-Preserving Hybrid Ensemble Model for Network Anomaly Detection: Balancing Security and Data Protection(https://arxiv.org/abs/2502.09001)
• Keywords: anomaly
• Abstract: Privacy-preserving network anomaly detection has become an essential area of research due to growing concerns over the protection of sensitive data. Traditional anomaly de- tection models often prioritize accuracy while neglecting the critical aspect of privacy. In this work, we propose a hybrid ensemble model that incorporates privacy-preserving techniques to address both detection accuracy and data protection. Our model combines the strengths of several machine learning algo- rithms, including K-Nearest Neighbors (KNN), Support Vector Machines (SVM), XGBoost, and Artificial Neural Networks (ANN), to create a robust system capable of identifying network anomalies while ensuring privacy. The proposed approach in- tegrates advanced preprocessing techniques that enhance data quality and address the challenges of small sample sizes and imbalanced datasets. By embedding privacy measures into the model design, our solution offers a significant advancement over existing methods, ensuring both enhanced detection performance and strong privacy safeguards.

Title: Typhoon T1: An Open Thai Reasoning Model

• Authors: Pittawat Taveekitworachai, Potsawee Manakul, Kasima Tharnpipitchai, Kunat Pipatanakul
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09042
• Pdf URL: https://arxiv.org/pdf/2502.09042
• Copy Paste: [[2502.09042]] Typhoon T1: An Open Thai Reasoning Model(https://arxiv.org/abs/2502.09042)
• Keywords: generative
• Abstract: This paper introduces Typhoon T1, an open effort to develop an open Thai reasoning model. A reasoning model is a relatively new type of generative model built on top of large language models (LLMs). A reasoning model generates a long chain of thought before arriving at a final answer, an approach found to improve performance on complex tasks. However, details on developing such a model are limited, especially for reasoning models that can generate traces in a low-resource language. Typhoon T1 presents an open effort that dives into the details of developing a reasoning model in a more cost-effective way by leveraging supervised fine-tuning using open datasets, instead of reinforcement learning. This paper shares the details about synthetic data generation and training, as well as our dataset and model weights. Additionally, we provide insights gained from developing a reasoning model that generalizes across domains and is capable of generating reasoning traces in a low-resource language, using Thai as an example. We hope this open effort provides a foundation for further research in this field.

Title: Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model

• Authors: Shiryu Ueno, Yoshikazu Hayashi, Shunsuke Nakatsuka, Yusei Yamada, Hiroaki Aizawa, Kunihito Kato
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09057
• Pdf URL: https://arxiv.org/pdf/2502.09057
• Copy Paste: [[2502.09057]] Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model(https://arxiv.org/abs/2502.09057)
• Keywords: in-context
• Abstract: We propose general visual inspection model using Vision-Language Model~(VLM) with few-shot images of non-defective or defective products, along with explanatory texts that serve as inspection criteria. Although existing VLM exhibit high performance across various tasks, they are not trained on specific tasks such as visual inspection. Thus, we construct a dataset consisting of diverse images of non-defective and defective products collected from the web, along with unified formatted output text, and fine-tune VLM. For new products, our method employs In-Context Learning, which allows the model to perform inspections with an example of non-defective or defective image and the corresponding explanatory texts with visual prompts. This approach eliminates the need to collect a large number of training samples and re-train the model for each product. The experimental results show that our method achieves high performance, with MCC of 0.804 and F1-score of 0.950 on MVTec AD in a one-shot manner. Our code is available at~this https URL.

Title: StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models

• Authors: Zichong Chen, Shijin Wang, Yang Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09064
• Pdf URL: https://arxiv.org/pdf/2502.09064
• Copy Paste: [[2502.09064]] StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.09064)
• Keywords: diffusion
• Abstract: Synthesizing visually impressive images that seamlessly align both text prompts and specific artistic styles remains a significant challenge in Text-to-Image (T2I) diffusion models. This paper introduces StyleBlend, a method designed to learn and apply style representations from a limited set of reference images, enabling content synthesis of both text-aligned and stylistically coherent. Our approach uniquely decomposes style into two components, composition and texture, each learned through different strategies. We then leverage two synthesis branches, each focusing on a corresponding style component, to facilitate effective style blending through shared features without affecting content generation. StyleBlend addresses the common issues of text misalignment and weak style representation that previous methods have struggled with. Extensive qualitative and quantitative comparisons demonstrate the superiority of our approach.

Title: Unsupervised Anomaly Detection on Implicit Shape representations for Sarcopenia Detection

• Authors: Louise Piecuch, Jeremie Huet (MD), Antoine Frouin (PT), Antoine Nordez, Anne-Sophie Boureau (MD), Diana Mateus
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09088
• Pdf URL: https://arxiv.org/pdf/2502.09088
• Copy Paste: [[2502.09088]] Unsupervised Anomaly Detection on Implicit Shape representations for Sarcopenia Detection(https://arxiv.org/abs/2502.09088)
• Keywords: anomaly
• Abstract: Sarcopenia is an age-related progressive loss of muscle mass and strength that significantly impacts daily life. A commonly studied criterion for characterizing the muscle mass has been the combination of 3D imaging and manual segmentations. In this paper, we instead study the muscles' shape. We rely on an implicit neural representation (INR) to model normal muscle shapes. We then introduce an unsupervised anomaly detection method to identify sarcopenic muscles based on the reconstruction error of the implicit model. Relying on a conditional INR with an auto-decoding strategy, we also learn a latent representation of the muscles that clearly separates normal from abnormal muscles in an unsupervised fashion. Experimental results on a dataset of 103 segmented volumes indicate that our double anomaly detection strategy effectively discriminates sarcopenic and non-sarcopenic muscles.

Title: Finite-Time Analysis of Discrete-Time Stochastic Interpolants

• Authors: Yuhao Liu, Yu Chen, Rui Hu, Longbo Huang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09130
• Pdf URL: https://arxiv.org/pdf/2502.09130
• Copy Paste: [[2502.09130]] Finite-Time Analysis of Discrete-Time Stochastic Interpolants(https://arxiv.org/abs/2502.09130)
• Keywords: generative
• Abstract: The stochastic interpolant framework offers a powerful approach for constructing generative models based on ordinary differential equations (ODEs) or stochastic differential equations (SDEs) to transform arbitrary data distributions. However, prior analyses of this framework have primarily focused on the continuous-time setting, assuming a perfect solution of the underlying equations. In this work, we present the first discrete-time analysis of the stochastic interpolant framework, where we introduce an innovative discrete-time sampler and derive a finite-time upper bound on its distribution estimation error. Our result provides a novel quantification of how different factors, including the distance between source and target distributions and estimation accuracy, affect the convergence rate and also offers a new principled way to design efficient schedules for convergence acceleration. Finally, numerical experiments are conducted on the discrete-time sampler to corroborate our theoretical findings.

Title: Replay-free Online Continual Learning with Self-Supervised MultiPatches

• Authors: Giacomo Cignoni, Andrea Cossu, Alex Gomez-Villa, Joost van de Weijer, Antonio Carta
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09140
• Pdf URL: https://arxiv.org/pdf/2502.09140
• Copy Paste: [[2502.09140]] Replay-free Online Continual Learning with Self-Supervised MultiPatches(https://arxiv.org/abs/2502.09140)
• Keywords: self-supervised
• Abstract: Online Continual Learning (OCL) methods train a model on a non-stationary data stream where only a few examples are available at a time, often leveraging replay strategies. However, usage of replay is sometimes forbidden, especially in applications with strict privacy regulations. Therefore, we propose Continual MultiPatches (CMP), an effective plug-in for existing OCL self-supervised learning strategies that avoids the use of replay samples. CMP generates multiple patches from a single example and projects them into a shared feature space, where patches coming from the same example are pushed together without collapsing into a single point. CMP surpasses replay and other SSL-based strategies on OCL streams, challenging the role of replay as a go-to solution for self-supervised OCL.

Title: Regularization can make diffusion models more efficient

• Authors: Mahsa Taheri, Johannes Lederer
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09151
• Pdf URL: https://arxiv.org/pdf/2502.09151
• Copy Paste: [[2502.09151]] Regularization can make diffusion models more efficient(https://arxiv.org/abs/2502.09151)
• Keywords: diffusion, generative
• Abstract: Diffusion models are one of the key architectures of generative AI. Their main drawback, however, is the computational costs. This study indicates that the concept of sparsity, well known especially in statistics, can provide a pathway to more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity can reduce the input dimension's influence on the computational complexity to that of a much smaller intrinsic dimension of the data. Our empirical findings confirm that inducing sparsity can indeed lead to better samples at a lower cost.

Title: E-MD3C: Taming Masked Diffusion Transformers for Efficient Zero-Shot Object Customization

• Authors: Trung X. Pham, Zhang Kang, Ji Woo Hong, Xuran Zheng, Chang D. Yoo
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09164
• Pdf URL: https://arxiv.org/pdf/2502.09164
• Copy Paste: [[2502.09164]] E-MD3C: Taming Masked Diffusion Transformers for Efficient Zero-Shot Object Customization(https://arxiv.org/abs/2502.09164)
• Keywords: diffusion
• Abstract: We propose E-MD3C ($\underline{E}$fficient $\underline{M}$asked $\underline{D}$iffusion Transformer with Disentangled $\underline{C}$onditions and $\underline{C}$ompact $\underline{C}$ollector), a highly efficient framework for zero-shot object image customization. Unlike prior works reliant on resource-intensive Unet architectures, our approach employs lightweight masked diffusion transformers operating on latent patches, offering significantly improved computational efficiency. The framework integrates three core components: (1) an efficient masked diffusion transformer for processing autoencoder latents, (2) a disentangled condition design that ensures compactness while preserving background alignment and fine details, and (3) a learnable Conditions Collector that consolidates multiple inputs into a compact representation for efficient denoising and learning. E-MD3C outperforms the existing approach on the VITON-HD dataset across metrics such as PSNR, FID, SSIM, and LPIPS, demonstrating clear advantages in parameters, memory efficiency, and inference speed. With only $\frac{1}{4}$ of the parameters, our Transformer-based 468M model delivers $2.5\times$ faster inference and uses $\frac{2}{3}$ of the GPU memory compared to an 1720M Unet-based latent diffusion model.

Title: LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data

• Authors: Peer Nagy, Sascha Frey, Kang Li, Bidipta Sarkar, Svitlana Vyetrenko, Stefan Zohren, Ani Calinescu, Jakob Foerster
• Subjects: cs.LG, cs.CE, q-fin.CP, q-fin.TR
• Abstract URL: https://arxiv.org/abs/2502.09172
• Pdf URL: https://arxiv.org/pdf/2502.09172
• Copy Paste: [[2502.09172]] LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data(https://arxiv.org/abs/2502.09172)
• Keywords: generative
• Abstract: While financial data presents one of the most challenging and interesting sequence modelling tasks due to high noise, heavy tails, and strategic interactions, progress in this area has been hindered by the lack of consensus on quantitative evaluation paradigms. To address this, we present LOB-Bench, a benchmark, implemented in python, designed to evaluate the quality and realism of generative message-by-order data for limit order books (LOB) in the LOBSTER format. Our framework measures distributional differences in conditional and unconditional statistics between generated and real LOB data, supporting flexible multivariate statistical evaluation. The benchmark also includes features commonly used LOB statistics such as spread, order book volumes, order imbalance, and message inter-arrival times, along with scores from a trained discriminator network. Lastly, LOB-Bench contains "market impact metrics", i.e. the cross-correlations and price response functions for specific events in the data. We benchmark generative autoregressive state-space models, a (C)GAN, as well as a parametric LOB model and find that the autoregressive GenAI approach beats traditional model classes.

Title: Two-Stage Representation Learning for Analyzing Movement Behavior Dynamics in People Living with Dementia

• Authors: Jin Cui, Alexander Capstick, Payam Barnaghi, Gregory Scott
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09173
• Pdf URL: https://arxiv.org/pdf/2502.09173
• Copy Paste: [[2502.09173]] Two-Stage Representation Learning for Analyzing Movement Behavior Dynamics in People Living with Dementia(https://arxiv.org/abs/2502.09173)
• Keywords: self-supervised
• Abstract: In remote healthcare monitoring, time series representation learning reveals critical patient behavior patterns from high-frequency data. This study analyzes home activity data from individuals living with dementia by proposing a two-stage, self-supervised learning approach tailored to uncover low-rank structures. The first stage converts time-series activities into text sequences encoded by a pre-trained language model, providing a rich, high-dimensional latent state space using a PageRank-based method. This PageRank vector captures latent state transitions, effectively compressing complex behaviour data into a succinct form that enhances interpretability. This low-rank representation not only enhances model interpretability but also facilitates clustering and transition analysis, revealing key behavioral patterns correlated with clinicalmetrics such as MMSE and ADAS-COG scores. Our findings demonstrate the framework's potential in supporting cognitive status prediction, personalized care interventions, and large-scale health monitoring.

Title: On the Importance of Embedding Norms in Self-Supervised Learning

• Authors: Andrew Draganov, Sharvaree Vadgama, Sebastian Damrich, Jan Niklas Böhm, Lucas Maes, Dmitry Kobak, Erik Bekkers
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09252
• Pdf URL: https://arxiv.org/pdf/2502.09252
• Copy Paste: [[2502.09252]] On the Importance of Embedding Norms in Self-Supervised Learning(https://arxiv.org/abs/2502.09252)
• Keywords: self-supervised
• Abstract: Self-supervised learning (SSL) allows training data representations without a supervised signal and has become an important paradigm in machine learning. Most SSL methods employ the cosine similarity between embedding vectors and hence effectively embed data on a hypersphere. While this seemingly implies that embedding norms cannot play any role in SSL, a few recent works have suggested that embedding norms have properties related to network convergence and confidence. In this paper, we resolve this apparent contradiction and systematically establish the embedding norm's role in SSL training. Using theoretical analysis, simulations, and experiments, we show that embedding norms (i) govern SSL convergence rates and (ii) encode network confidence, with smaller norms corresponding to unexpected samples. Additionally, we show that manipulating embedding norms can have large effects on convergence speed. Our findings demonstrate that SSL embedding norms are integral to understanding and optimizing network behavior.

Title: AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detection

• Authors: Hezhe Qiao, Chaoxi Niu, Ling Chen, Guansong Pang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09254
• Pdf URL: https://arxiv.org/pdf/2502.09254
• Copy Paste: [[2502.09254]] AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detection(https://arxiv.org/abs/2502.09254)
• Keywords: foundation model, anomaly
• Abstract: Graph anomaly detection (GAD) aims to identify abnormal nodes that differ from the majority of the nodes in a graph, which has been attracting significant attention in recent years. Existing generalist graph models have achieved remarkable success in different graph tasks but struggle to generalize to the GAD task. This limitation arises from their difficulty in learning generalized knowledge for capturing the inherently infrequent, irregular and heterogeneous abnormality patterns in graphs from different domains. To address this challenge, we propose AnomalyGFM, a GAD-oriented graph foundation model that supports zero-shot inference and few-shot prompt tuning for GAD in diverse graph datasets. One key insight is that graph-agnostic representations for normal and abnormal classes are required to support effective zero/few-shot GAD across different graphs. Motivated by this, AnomalyGFM is pre-trained to align data-independent, learnable normal and abnormal class prototypes with node representation residuals (i.e., representation deviation of a node from its neighbors). The residual features essentially project the node information into a unified feature space where we can effectively measure the abnormality of nodes from different graphs in a consistent way. This provides a driving force for the learning of graph-agnostic, discriminative prototypes for the normal and abnormal classes, which can be used to enable zero-shot GAD on new graphs, including very large-scale graphs. If there are few-shot labeled normal nodes available in the new graphs, AnomalyGFM can further support prompt tuning to leverage these nodes for better adaptation. Comprehensive experiments on 11 widely-used GAD datasets with real anomalies, demonstrate that AnomalyGFM significantly outperforms state-of-the-art competing methods under both zero- and few-shot GAD settings.

Title: ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimization

• Authors: Onat Şahin, Mohammad Altillawi, George Eskandar, Carlos Carbone, Ziyuan Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09278
• Pdf URL: https://arxiv.org/pdf/2502.09278
• Copy Paste: [[2502.09278]] ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimization(https://arxiv.org/abs/2502.09278)
• Keywords: diffusion
• Abstract: Recent advances in diffusion models have significantly improved 3D generation, enabling the use of assets generated from an image for embodied AI simulations. However, the one-to-many nature of the image-to-3D problem limits their use due to inconsistent content and quality across views. Previous models optimize a 3D model by sampling views from a view-conditioned diffusion prior, but diffusion models cannot guarantee view consistency. Instead, we present ConsistentDreamer, where we first generate a set of fixed multi-view prior images and sample random views between them with another diffusion model through a score distillation sampling (SDS) loss. Thereby, we limit the discrepancies between the views guided by the SDS loss and ensure a consistent rough shape. In each iteration, we also use our generated multi-view prior images for fine-detail reconstruction. To balance between the rough shape and the fine-detail optimizations, we introduce dynamic task-dependent weights based on homoscedastic uncertainty, updated automatically in each iteration. Additionally, we employ opacity, depth distortion, and normal alignment losses to refine the surface for mesh extraction. Our method ensures better view consistency and visual quality compared to the state-of-the-art.

Title: SparQLe: Speech Queries to Text Translation Through LLMs

• Authors: Amirbek Djanibekov, Hanan Aldarmaki
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09284
• Pdf URL: https://arxiv.org/pdf/2502.09284
• Copy Paste: [[2502.09284]] SparQLe: Speech Queries to Text Translation Through LLMs(https://arxiv.org/abs/2502.09284)
• Keywords: self-supervised
• Abstract: With the growing influence of Large Language Models (LLMs), there is increasing interest in integrating speech representations with them to enable more seamless multi-modal processing and speech understanding. This study introduces a novel approach that leverages self-supervised speech representations in combination with instruction-tuned LLMs for speech-to-text translation. The proposed approach leverages a modality adapter to align extracted speech features with instruction-tuned LLMs using English-language data. Our experiments demonstrate that this method effectively preserves the semantic content of the input speech and serves as an effective bridge between self-supervised speech models and instruction-tuned LLMs, offering a promising solution for various speech understanding applications.

Title: When do neural networks learn world models?

• Authors: Tianren Zhang, Guanyu Chen, Feng Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09297
• Pdf URL: https://arxiv.org/pdf/2502.09297
• Copy Paste: [[2502.09297]] When do neural networks learn world models?(https://arxiv.org/abs/2502.09297)
• Keywords: self-supervised
• Abstract: Humans develop world models that capture the underlying generation process of data. Whether neural networks can learn similar world models remains an open problem. In this work, we provide the first theoretical results for this problem, showing that in a multi-task setting, models with a low-degree bias provably recover latent data-generating variables under mild assumptions -- even if proxy tasks involve complex, non-linear functions of the latents. However, such recovery is also sensitive to model architecture. Our analysis leverages Boolean models of task solutions via the Fourier-Walsh transform and introduces new techniques for analyzing invertible Boolean transforms, which may be of independent interest. We illustrate the algorithmic implications of our results and connect them to related research areas, including self-supervised learning, out-of-distribution generalization, and the linear representation hypothesis in large language models.

Title: A Benchmark for Crime Surveillance Video Analysis with Large Models

• Authors: Haoran Chen, Dong Yi, Moyan Cao, Chensen Huang, Guibo Zhu, Jinqiao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09325
• Pdf URL: https://arxiv.org/pdf/2502.09325
• Copy Paste: [[2502.09325]] A Benchmark for Crime Surveillance Video Analysis with Large Models(https://arxiv.org/abs/2502.09325)
• Keywords: anomaly
• Abstract: Anomaly analysis in surveillance videos is a crucial topic in computer vision. In recent years, multimodal large language models (MLLMs) have outperformed task-specific models in various domains. Although MLLMs are particularly versatile, their abilities to understand anomalous concepts and details are insufficiently studied because of the outdated benchmarks of this field not providing MLLM-style QAs and efficient algorithms to assess the model's open-ended text responses. To fill this gap, we propose a benchmark for crime surveillance video analysis with large models denoted as UCVL, including 1,829 videos and reorganized annotations from the UCF-Crime and UCF-Crime Annotation datasets. We design six types of questions and generate diverse QA pairs. Then we develop detailed instructions and use OpenAI's GPT-4o for accurate assessment. We benchmark eight prevailing MLLMs ranging from 0.5B to 40B parameters, and the results demonstrate the reliability of this bench. Moreover, we finetune LLaVA-OneVision on UCVL's training set. The improvement validates our data's high quality for video anomaly analysis.

Title: Graph Diffusion Network for Drug-Gene Prediction

• Authors: Jiayang Wu, Wensheng Gan, Philip S. Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09335
• Pdf URL: https://arxiv.org/pdf/2502.09335
• Copy Paste: [[2502.09335]] Graph Diffusion Network for Drug-Gene Prediction(https://arxiv.org/abs/2502.09335)
• Keywords: diffusion
• Abstract: Predicting drug-gene associations is crucial for drug development and disease treatment. While graph neural networks (GNN) have shown effectiveness in this task, they face challenges with data sparsity and efficient contrastive learning implementation. We introduce a graph diffusion network for drug-gene prediction (GDNDGP), a framework that addresses these limitations through two key innovations. First, it employs meta-path-based homogeneous graph learning to capture drug-drug and gene-gene relationships, ensuring similar entities share embedding spaces. Second, it incorporates a parallel diffusion network that generates hard negative samples during training, eliminating the need for exhaustive negative sample retrieval. Our model achieves superior performance on the DGIdb 4.0 dataset and demonstrates strong generalization capability on tripartite drug-gene-disease networks. Results show significant improvements over existing methods in drug-gene prediction tasks, particularly in handling complex heterogeneous relationships. The source code is publicly available at this https URL.

Title: Machine learning for modelling unstructured grid data in computational physics: a review

• Authors: Sibo Cheng, Marc Bocquet, Weiping Ding, Tobias Sebastian Finn, Rui Fu, Jinlong Fu, Yike Guo, Eleda Johnson, Siyi Li, Che Liu, Eric Newton Moro, Jie Pan, Matthew Piggott, Cesar Quilodran, Prakhar Sharma, Kun Wang, Dunhui Xiao, Xiao Xue, Yong Zeng, Mingrui Zhang, Hao Zhou, Kewei Zhu, Rossella Arcucci
• Subjects: cs.LG, cs.CE, physics.data-an, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2502.09346
• Pdf URL: https://arxiv.org/pdf/2502.09346
• Copy Paste: [[2502.09346]] Machine learning for modelling unstructured grid data in computational physics: a review(https://arxiv.org/abs/2502.09346)
• Keywords: generative
• Abstract: Unstructured grid data are essential for modelling complex geometries and dynamics in computational physics. Yet, their inherent irregularity presents significant challenges for conventional machine learning (ML) techniques. This paper provides a comprehensive review of advanced ML methodologies designed to handle unstructured grid data in high-dimensional dynamical systems. Key approaches discussed include graph neural networks, transformer models with spatial attention mechanisms, interpolation-integrated ML methods, and meshless techniques such as physics-informed neural networks. These methodologies have proven effective across diverse fields, including fluid dynamics and environmental simulations. This review is intended as a guidebook for computational scientists seeking to apply ML approaches to unstructured grid data in their domains, as well as for ML researchers looking to address challenges in computational physics. It places special focus on how ML methods can overcome the inherent limitations of traditional numerical techniques and, conversely, how insights from computational physics can inform ML development. To support benchmarking, this review also provides a summary of open-access datasets of unstructured grid data in computational physics. Finally, emerging directions such as generative models with unstructured data, reinforcement learning for mesh generation, and hybrid physics-data-driven paradigms are discussed to inspire future advancements in this evolving field.

Title: Galileo: Learning Global and Local Features in Pretrained Remote Sensing Models

• Authors: Gabriel Tseng, Anthony Fuller, Marlena Reil, Henry Herzog, Patrick Beukema, Favyen Bastani, James R. Green, Evan Shelhamer, Hannah Kerner, David Rolnick
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09356
• Pdf URL: https://arxiv.org/pdf/2502.09356
• Copy Paste: [[2502.09356]] Galileo: Learning Global and Local Features in Pretrained Remote Sensing Models(https://arxiv.org/abs/2502.09356)
• Keywords: self-supervised
• Abstract: From crop mapping to flood detection, machine learning in remote sensing has a wide range of societally beneficial applications. The commonalities between remote sensing data in these applications present an opportunity for pretrained machine learning models tailored to remote sensing to reduce the labeled data and effort required to solve individual tasks. However, such models must be: (i) flexible enough to ingest input data of varying sensor modalities and shapes (i.e., of varying spatial and temporal dimensions), and (ii) able to model Earth surface phenomena of varying scales and types. To solve this gap, we present Galileo, a family of pretrained remote sensing models designed to flexibly process multimodal remote sensing data. We also introduce a novel and highly effective self-supervised learning approach to learn both large- and small-scale features, a challenge not addressed by previous models. Our Galileo models obtain state-of-the-art results across diverse remote sensing tasks.

Title: LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)

• Authors: Junsu Kim, Jaeyeon Kim, Ernest K. Ryu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09376
• Pdf URL: https://arxiv.org/pdf/2502.09376
• Copy Paste: [[2502.09376]] LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)(https://arxiv.org/abs/2502.09376)
• Keywords: foundation model
• Abstract: Low-rank adaptation (LoRA) has become a standard approach for fine-tuning large foundation models. However, our theoretical understanding of LoRA remains limited as prior analyses of LoRA's training dynamics either rely on linearization arguments or consider highly simplified setups. In this work, we analyze the LoRA loss landscape without such restrictive assumptions. We define two regimes: a ``special regime'', which includes idealized setups where linearization arguments hold, and a ``generic regime'' representing more realistic setups where linearization arguments do not hold. In the generic regime, we show that LoRA training converges to a global minimizer with low rank and small magnitude, or a qualitatively distinct solution with high rank and large magnitude. Finally, we argue that the zero-initialization and weight decay in LoRA training induce an implicit bias toward the low-rank, small-magnitude region of the parameter space -- where global minima lie -- thus shedding light on why LoRA training usually succeeds in finding global minima.

Title: APT-LLM: Embedding-Based Anomaly Detection of Cyber Advanced Persistent Threats Using Large Language Models

• Authors: Sidahmed Benabderrahmane, Petko Valtchev, James Cheney, Talal Rahwan
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09385
• Pdf URL: https://arxiv.org/pdf/2502.09385
• Copy Paste: [[2502.09385]] APT-LLM: Embedding-Based Anomaly Detection of Cyber Advanced Persistent Threats Using Large Language Models(https://arxiv.org/abs/2502.09385)
• Keywords: anomaly
• Abstract: Advanced Persistent Threats (APTs) pose a major cybersecurity challenge due to their stealth and ability to mimic normal system behavior, making detection particularly difficult in highly imbalanced datasets. Traditional anomaly detection methods struggle to effectively differentiate APT-related activities from benign processes, limiting their applicability in real-world scenarios. This paper introduces APT-LLM, a novel embedding-based anomaly detection framework that integrates large language models (LLMs) -- BERT, ALBERT, DistilBERT, and RoBERTa -- with autoencoder architectures to detect APTs. Unlike prior approaches, which rely on manually engineered features or conventional anomaly detection models, APT-LLM leverages LLMs to encode process-action provenance traces into semantically rich embeddings, capturing nuanced behavioral patterns. These embeddings are analyzed using three autoencoder architectures -- Baseline Autoencoder (AE), Variational Autoencoder (VAE), and Denoising Autoencoder (DAE) -- to model normal process behavior and identify anomalies. The best-performing model is selected for comparison against traditional methods. The framework is evaluated on real-world, highly imbalanced provenance trace datasets from the DARPA Transparent Computing program, where APT-like attacks constitute as little as 0.004\% of the data across multiple operating systems (Android, Linux, BSD, and Windows) and attack scenarios. Results demonstrate that APT-LLM significantly improves detection performance under extreme imbalance conditions, outperforming existing anomaly detection methods and highlighting the effectiveness of LLM-based feature extraction in cybersecurity.

Title: ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation

• Authors: Rotem Shalev-Arkushin, Rinon Gal, Amit H. Bermano, Ohad Fried
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.09411
• Pdf URL: https://arxiv.org/pdf/2502.09411
• Copy Paste: [[2502.09411]] ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation(https://arxiv.org/abs/2502.09411)
• Keywords: diffusion
• Abstract: Diffusion models enable high-quality and diverse visual content synthesis. However, they struggle to generate rare or unseen concepts. To address this challenge, we explore the usage of Retrieval-Augmented Generation (RAG) with image generation models. We propose ImageRAG, a method that dynamically retrieves relevant images based on a given text prompt, and uses them as context to guide the generation process. Prior approaches that used retrieved images to improve generation, trained models specifically for retrieval-based generation. In contrast, ImageRAG leverages the capabilities of existing image conditioning models, and does not require RAG-specific training. Our approach is highly adaptable and can be applied across different model types, showing significant improvement in generating rare and fine-grained concepts using different base models. Our project page is available at: this https URL

Title: Redistribute Ensemble Training for Mitigating Memorization in Diffusion Models

• Authors: Xiaoliu Guan, Yu Wu, Huayang Huang, Xiao Liu, Jiaxu Miao, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09434
• Pdf URL: https://arxiv.org/pdf/2502.09434
• Copy Paste: [[2502.09434]] Redistribute Ensemble Training for Mitigating Memorization in Diffusion Models(https://arxiv.org/abs/2502.09434)
• Keywords: diffusion
• Abstract: Diffusion models, known for their tremendous ability to generate high-quality samples, have recently raised concerns due to their data memorization behavior, which poses privacy risks. Recent methods for memory mitigation have primarily addressed the issue within the context of the text modality in cross-modal generation tasks, restricting their applicability to specific conditions. In this paper, we propose a novel method for diffusion models from the perspective of visual modality, which is more generic and fundamental for mitigating memorization. Directly exposing visual data to the model increases memorization risk, so we design a framework where models learn through proxy model parameters instead. Specially, the training dataset is divided into multiple shards, with each shard training a proxy model, then aggregated to form the final model. Additionally, practical analysis of training losses illustrates that the losses for easily memorable images tend to be obviously lower. Thus, we skip the samples with abnormally low loss values from the current mini-batch to avoid memorizing. However, balancing the need to skip memorization-prone samples while maintaining sufficient training data for high-quality image generation presents a key challenge. Thus, we propose IET-AGC+, which redistributes highly memorizable samples between shards, to mitigate these samples from over-skipping. Furthermore, we dynamically augment samples based on their loss values to further reduce memorization. Extensive experiments and analysis on four datasets show that our method successfully reduces memory capacity while maintaining performance. Moreover, we fine-tune the pre-trained diffusion models, e.g., Stable Diffusion, and decrease the memorization score by 46.7\%, demonstrating the effectiveness of our method. Code is available in: this https URL.

Title: PenTest++: Elevating Ethical Hacking with AI and Automation

• Authors: Haitham S. Al-Sinani, Chris J. Mitchell
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09484
• Pdf URL: https://arxiv.org/pdf/2502.09484
• Copy Paste: [[2502.09484]] PenTest++: Elevating Ethical Hacking with AI and Automation(https://arxiv.org/abs/2502.09484)
• Keywords: generative
• Abstract: Traditional ethical hacking relies on skilled professionals and time-intensive command management, which limits its scalability and efficiency. To address these challenges, we introduce PenTest++, an AI-augmented system that integrates automation with generative AI (GenAI) to optimise ethical hacking workflows. Developed in a controlled virtual environment, PenTest++ streamlines critical penetration testing tasks, including reconnaissance, scanning, enumeration, exploitation, and documentation, while maintaining a modular and adaptable design. The system balances automation with human oversight, ensuring informed decision-making at key stages, and offers significant benefits such as enhanced efficiency, scalability, and adaptability. However, it also raises ethical considerations, including privacy concerns and the risks of AI-generated inaccuracies (hallucinations). This research underscores the potential of AI-driven systems like PenTest++ to complement human expertise in cybersecurity by automating routine tasks, enabling professionals to focus on strategic decision-making. By incorporating robust ethical safeguards and promoting ongoing refinement, PenTest++ demonstrates how AI can be responsibly harnessed to address operational and ethical challenges in the evolving cybersecurity landscape.

Title: EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

• Authors: Theodoros Kouzelis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09509
• Pdf URL: https://arxiv.org/pdf/2502.09509
• Copy Paste: [[2502.09509]] EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling(https://arxiv.org/abs/2502.09509)
• Keywords: generative
• Abstract: Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: this https URL.

Title: Diffusion Models for Molecules: A Survey of Methods and Tasks

• Authors: Liang Wang, Chao Song, Zhiyuan Liu, Yu Rong, Qiang Liu, Shu Wu, Liang Wang
• Subjects: cs.LG, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.09511
• Pdf URL: https://arxiv.org/pdf/2502.09511
• Copy Paste: [[2502.09511]] Diffusion Models for Molecules: A Survey of Methods and Tasks(https://arxiv.org/abs/2502.09511)
• Keywords: diffusion, generative
• Abstract: Generative tasks about molecules, including but not limited to molecule generation, are crucial for drug discovery and material design, and have consistently attracted significant attention. In recent years, diffusion models have emerged as an impressive class of deep generative models, sparking extensive research and leading to numerous studies on their application to molecular generative tasks. Despite the proliferation of related work, there remains a notable lack of up-to-date and systematic surveys in this area. Particularly, due to the diversity of diffusion model formulations, molecular data modalities, and generative task types, the research landscape is challenging to navigate, hindering understanding and limiting the area's growth. To address this, this paper conducts a comprehensive survey of diffusion model-based molecular generative methods. We systematically review the research from the perspectives of methodological formulations, data modalities, and task types, offering a novel taxonomy. This survey aims to facilitate understanding and further flourishing development in this area. The relevant papers are summarized at: this https URL.

Title: SQ-GAN: Semantic Image Communications Using Masked Vector Quantization

• Authors: Francesco Pezone, Sergio Barbarossa, Giuseppe Caire
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.09520
• Pdf URL: https://arxiv.org/pdf/2502.09520
• Copy Paste: [[2502.09520]] SQ-GAN: Semantic Image Communications Using Masked Vector Quantization(https://arxiv.org/abs/2502.09520)
• Keywords: generative
• Abstract: This work introduces Semantically Masked VQ-GAN (SQ-GAN), a novel approach integrating generative models to optimize image compression for semantic/task-oriented communications. SQ-GAN employs off-the-shelf semantic semantic segmentation and a new specifically developed semantic-conditioned adaptive mask module (SAMM) to selectively encode semantically significant features of the images. SQ-GAN outperforms state-of-the-art image compression schemes such as JPEG2000 and BPG across multiple metrics, including perceptual quality and semantic segmentation accuracy on the post-decoding reconstructed image, at extreme low compression rates expressed in bits per pixel.

Title: Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages

• Authors: Shreyan Biswas, Alexander Erlei, Ujwal Gadiraju
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.09532
• Pdf URL: https://arxiv.org/pdf/2502.09532
• Copy Paste: [[2502.09532]] Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages(https://arxiv.org/abs/2502.09532)
• Keywords: generative
• Abstract: Recent advances in generative AI have precipitated a proliferation of novel writing assistants. These systems typically rely on multilingual large language models (LLMs), providing globalized workers the ability to revise or create diverse forms of content in different languages. However, there is substantial evidence indicating that the performance of multilingual LLMs varies between languages. Users who employ writing assistance for multiple languages are therefore susceptible to disparate output quality. Importantly, recent research has shown that people tend to generalize algorithmic errors across independent tasks, violating the behavioral axiom of choice independence. In this paper, we analyze whether user utilization of novel writing assistants in a charity advertisement writing task is affected by the AI's performance in a second language. Furthermore, we quantify the extent to which these patterns translate into the persuasiveness of generated charity advertisements, as well as the role of peoples' beliefs about LLM utilization in their donation choices. Our results provide evidence that writers who engage with an LLM-based writing assistant violate choice independence, as prior exposure to a Spanish LLM reduces subsequent utilization of an English LLM. While these patterns do not affect the aggregate persuasiveness of the generated advertisements, people's beliefs about the source of an advertisement (human versus AI) do. In particular, Spanish-speaking female participants who believed that they read an AI-generated advertisement strongly adjusted their donation behavior downwards. Furthermore, people are generally not able to adequately differentiate between human-generated and LLM-generated ads. Our work has important implications for the design, development, integration, and adoption of multilingual LLMs as assistive agents -- particularly in writing tasks.

Title: Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

• Authors: Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09533
• Pdf URL: https://arxiv.org/pdf/2502.09533
• Copy Paste: [[2502.09533]] Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model(https://arxiv.org/abs/2502.09533)
• Keywords: diffusion
• Abstract: Recent advances in conditional diffusion models have shown promise for generating realistic TalkingFace videos, yet challenges persist in achieving consistent head movement, synchronized facial expressions, and accurate lip synchronization over extended generations. To address these, we introduce the \textbf{M}otion-priors \textbf{C}onditional \textbf{D}iffusion \textbf{M}odel (\textbf{MCDM}), which utilizes both archived and current clip motion priors to enhance motion prediction and ensure temporal consistency. The model consists of three key elements: (1) an archived-clip motion-prior that incorporates historical frames and a reference frame to preserve identity and context; (2) a present-clip motion-prior diffusion model that captures multimodal causality for accurate predictions of head movements, lip sync, and expressions; and (3) a memory-efficient temporal attention mechanism that mitigates error accumulation by dynamically storing and updating motion features. We also release the \textbf{TalkingFace-Wild} dataset, a multilingual collection of over 200 hours of footage across 10 languages. Experimental results demonstrate the effectiveness of MCDM in maintaining identity and motion continuity for long-term TalkingFace generation. Code, models, and datasets will be publicly available.

Title: Diffusing DeBias: a Recipe for Turning a Bug into a Feature

• Authors: Massimiliano Ciranni, Vito Paolo Pastore, Roberto Di Via, Enzo Tartaglione, Francesca Odone, Vittorio Murino
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09564
• Pdf URL: https://arxiv.org/pdf/2502.09564
• Copy Paste: [[2502.09564]] Diffusing DeBias: a Recipe for Turning a Bug into a Feature(https://arxiv.org/abs/2502.09564)
• Keywords: diffusion
• Abstract: Deep learning model effectiveness in classification tasks is often challenged by the quality and quantity of training data which, whenever containing strong spurious correlations between specific attributes and target labels, can result in unrecoverable biases in model predictions. Tackling these biases is crucial in improving model generalization and trust, especially in real-world scenarios. This paper presents Diffusing DeBias (DDB), a novel approach acting as a plug-in for common methods in model debiasing while exploiting the inherent bias-learning tendency of diffusion models. Our approach leverages conditional diffusion models to generate synthetic bias-aligned images, used to train a bias amplifier model, to be further employed as an auxiliary method in different unsupervised debiasing approaches. Our proposed method, which also tackles the common issue of training set memorization typical of this type of tech- niques, beats current state-of-the-art in multiple benchmark datasets by significant margins, demonstrating its potential as a versatile and effective tool for tackling dataset bias in deep learning applications.

Title: Zero-shot generation of synthetic neurosurgical data with large language models

• Authors: Austin A. Barr, Eddie Guo, Emre Sezgin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09566
• Pdf URL: https://arxiv.org/pdf/2502.09566
• Copy Paste: [[2502.09566]] Zero-shot generation of synthetic neurosurgical data with large language models(https://arxiv.org/abs/2502.09566)
• Keywords: generative
• Abstract: Clinical data is fundamental to advance neurosurgical research, but access is often constrained by data availability, small sample sizes, privacy regulations, and resource-intensive preprocessing and de-identification procedures. Synthetic data offers a potential solution to challenges associated with accessing and using real-world data (RWD). This study aims to evaluate the capability of zero-shot generation of synthetic neurosurgical data with a large language model (LLM), GPT-4o, by benchmarking with the conditional tabular generative adversarial network (CTGAN). Synthetic datasets were compared to real-world neurosurgical data to assess fidelity (means, proportions, distributions, and bivariate correlations), utility (ML classifier performance on RWD), and privacy (duplication of records from RWD). The GPT-4o-generated datasets matched or exceeded CTGAN performance, despite no fine-tuning or access to RWD for pre-training. Datasets demonstrated high univariate and bivariate fidelity to RWD without directly exposing any real patient records, even at amplified sample size. Training an ML classifier on GPT-4o-generated data and testing on RWD for a binary prediction task showed an F1 score (0.706) with comparable performance to training on the CTGAN data (0.705) for predicting postoperative functional status deterioration. GPT-4o demonstrated a promising ability to generate high-fidelity synthetic neurosurgical data. These findings also indicate that data synthesized with GPT-4o can effectively augment clinical data with small sample sizes, and train ML models for prediction of neurosurgical outcomes. Further investigation is necessary to improve the preservation of distributional characteristics and boost classifier performance.

Title: DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra

• Authors: Montgomery Bohde, Mrunali Manjrekar, Runzhong Wang, Shuiwang Ji, Connor W. Coley
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.09571
• Pdf URL: https://arxiv.org/pdf/2502.09571
• Copy Paste: [[2502.09571]] DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra(https://arxiv.org/abs/2502.09571)
• Keywords: diffusion, generative
• Abstract: Mass spectrometry plays a fundamental role in elucidating the structures of unknown molecules and subsequent scientific discoveries. One formulation of the structure elucidation task is the conditional $\textit{de novo}$ generation of molecular structure given a mass spectrum. Toward a more accurate and efficient scientific discovery pipeline for small molecules, we present DiffMS, a formula-restricted encoder-decoder generative network that achieves state-of-the-art performance on this task. The encoder utilizes a transformer architecture and models mass spectra domain knowledge such as peak formulae and neutral losses, and the decoder is a discrete graph diffusion model restricted by the heavy-atom composition of a known chemical formula. To develop a robust decoder that bridges latent embeddings and molecular structures, we pretrain the diffusion decoder with fingerprint-structure pairs, which are available in virtually infinite quantities, compared to structure-spectrum pairs that number in the tens of thousands. Extensive experiments on established benchmarks show that DiffMS outperforms existing models on $\textit{de novo}$ molecule generation. We provide several ablations to demonstrate the effectiveness of our diffusion and pretraining approaches and show consistent performance scaling with increasing pretraining dataset size. DiffMS code is publicly available at this https URL.

Title: Rolling Ahead Diffusion for Traffic Scene Simulation

• Authors: Yunpeng Liu, Matthew Niedoba, William Harvey, Adam Scibior, Berend Zwartsenberg, Frank Wood
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.09587
• Pdf URL: https://arxiv.org/pdf/2502.09587
• Copy Paste: [[2502.09587]] Rolling Ahead Diffusion for Traffic Scene Simulation(https://arxiv.org/abs/2502.09587)
• Keywords: diffusion
• Abstract: Realistic driving simulation requires that NPCs not only mimic natural driving behaviors but also react to the behavior of other simulated agents. Recent developments in diffusion-based scenario generation focus on creating diverse and realistic traffic scenarios by jointly modelling the motion of all the agents in the scene. However, these traffic scenarios do not react when the motion of agents deviates from their modelled trajectories. For example, the ego-agent can be controlled by a stand along motion planner. To produce reactive scenarios with joint scenario models, the model must regenerate the scenario at each timestep based on new observations in a Model Predictive Control (MPC) fashion. Although reactive, this method is time-consuming, as one complete possible future for all NPCs is generated per simulation step. Alternatively, one can utilize an autoregressive model (AR) to predict only the immediate next-step future for all NPCs. Although faster, this method lacks the capability for advanced planning. We present a rolling diffusion based traffic scene generation model which mixes the benefits of both methods by predicting the next step future and simultaneously predicting partially noised further future steps at the same time. We show that such model is efficient compared to diffusion model based AR, achieving a beneficial compromise between reactivity and computational efficiency.

Title: SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

• Authors: Yung-Sung Chuang, Benjamin Cohen-Wang, Shannon Zejiang Shen, Zhaofeng Wu, Hu Xu, Xi Victoria Lin, James Glass, Shang-Wen Li, Wen-tau Yih
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09604
• Pdf URL: https://arxiv.org/pdf/2502.09604
• Copy Paste: [[2502.09604]] SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models(https://arxiv.org/abs/2502.09604)
• Keywords: self-supervised
• Abstract: We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks.

Title: Score-of-Mixture Training: Training One-Step Generative Models Made Simple

• Authors: Tejas Jayashankar, J. Jon Ryu, Gregory Wornell
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09609
• Pdf URL: https://arxiv.org/pdf/2502.09609
• Copy Paste: [[2502.09609]] Score-of-Mixture Training: Training One-Step Generative Models Made Simple(https://arxiv.org/abs/2502.09609)
• Keywords: diffusion, generative
• Abstract: We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen-Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64x64 show that SMT/SMD are competitive with and can even outperform existing methods.

Title: Designing a Conditional Prior Distribution for Flow-Based Generative Models

• Authors: Noam Issachar, Mohammad Salama, Raanan Fattal, Sagie Benaim
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09611
• Pdf URL: https://arxiv.org/pdf/2502.09611
• Copy Paste: [[2502.09611]] Designing a Conditional Prior Distribution for Flow-Based Generative Models(https://arxiv.org/abs/2502.09611)
• Keywords: generative
• Abstract: Flow-based generative models have recently shown impressive performance for conditional generation tasks, such as text-to-image generation. However, current methods transform a general unimodal noise distribution to a specific mode of the target data distribution. As such, every point in the initial source distribution can be mapped to every point in the target distribution, resulting in long average paths. To this end, in this work, we tap into a non-utilized property of conditional flow-based models: the ability to design a non-trivial prior distribution. Given an input condition, such as a text prompt, we first map it to a point lying in data space, representing an ``average" data point with the minimal average distance to all data points of the same conditional mode (e.g., class). We then utilize the flow matching formulation to map samples from a parametric distribution centered around this point to the conditional target distribution. Experimentally, our method significantly improves training times and generation efficiency (FID, KID and CLIP alignment scores) compared to baselines, producing high quality samples using fewer sampling steps.

Title: RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assets

• Authors: Isabella Liu, Zhan Xu, Wang Yifan, Hao Tan, Zexiang Xu, Xiaolong Wang, Hao Su, Zifan Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09615
• Pdf URL: https://arxiv.org/pdf/2502.09615
• Copy Paste: [[2502.09615]] RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assets(https://arxiv.org/abs/2502.09615)
• Keywords: diffusion
• Abstract: We present RigAnything, a novel autoregressive transformer-based model, which makes 3D assets rig-ready by probabilistically generating joints, skeleton topologies, and assigning skinning weights in a template-free manner. Unlike most existing auto-rigging methods, which rely on predefined skeleton template and are limited to specific categories like humanoid, RigAnything approaches the rigging problem in an autoregressive manner, iteratively predicting the next joint based on the global input shape and the previous prediction. While autoregressive models are typically used to generate sequential data, RigAnything extends their application to effectively learn and represent skeletons, which are inherently tree structures. To achieve this, we organize the joints in a breadth-first search (BFS) order, enabling the skeleton to be defined as a sequence of 3D locations and the parent index. Furthermore, our model improves the accuracy of position prediction by leveraging diffusion modeling, ensuring precise and consistent placement of joints within the hierarchy. This formulation allows the autoregressive model to efficiently capture both spatial and hierarchical relationships within the skeleton. Trained end-to-end on both RigNet and Objaverse datasets, RigAnything demonstrates state-of-the-art performance across diverse object types, including humanoids, quadrupeds, marine creatures, insects, and many more, surpassing prior methods in quality, robustness, generalizability, and efficiency. Please check our website for more details: this https URL.

Title: Exploring the Potential of Encoder-free Architectures in 3D LMMs

• Authors: Yiwen Tang, Zoey Guo, Zhuhao Wang, Ray Zhang, Qizhi Chen, Junli Liu, Delin Qu, Zhigang Wang, Dong Wang, Xuelong Li, Bin Zhao
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09620
• Pdf URL: https://arxiv.org/pdf/2502.09620
• Copy Paste: [[2502.09620]] Exploring the Potential of Encoder-free Architectures in 3D LMMs(https://arxiv.org/abs/2502.09620)
• Keywords: self-supervised
• Abstract: Encoder-free architectures have been preliminarily explored in the 2D visual domain, yet it remains an open question whether they can be effectively applied to 3D understanding scenarios. In this paper, we present the first comprehensive investigation into the potential of encoder-free architectures to overcome the challenges of encoder-based 3D Large Multimodal Models (LMMs). These challenges include the failure to adapt to varying point cloud resolutions and the point features from the encoder not meeting the semantic needs of Large Language Models (LLMs). We identify key aspects for 3D LMMs to remove the encoder and enable the LLM to assume the role of the 3D encoder: 1) We propose the LLM-embedded Semantic Encoding strategy in the pre-training stage, exploring the effects of various point cloud self-supervised losses. And we present the Hybrid Semantic Loss to extract high-level semantics. 2) We introduce the Hierarchical Geometry Aggregation strategy in the instruction tuning stage. This incorporates inductive bias into the LLM early layers to focus on the local details of the point clouds. To the end, we present the first Encoder-free 3D LMM, ENEL. Our 7B model rivals the current state-of-the-art model, ShapeLLM-13B, achieving 55.0%, 50.92%, and 42.7% on the classification, captioning, and VQA tasks, respectively. Our results demonstrate that the encoder-free architecture is highly promising for replacing encoder-based architectures in the field of 3D understanding. The code is released at this https URL

Title: Theoretical Benefit and Limitation of Diffusion Language Model

• Authors: Guhao Feng, Yihan Geng, Jian Guan, Wei Wu, Liwei Wang, Di He
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09622
• Pdf URL: https://arxiv.org/pdf/2502.09622
• Copy Paste: [[2502.09622]] Theoretical Benefit and Limitation of Diffusion Language Model(https://arxiv.org/abs/2502.09622)
• Keywords: diffusion
• Abstract: Diffusion language models have emerged as a promising approach for text generation. One would naturally expect this method to be an efficient replacement for autoregressive models since multiple tokens can be sampled in parallel during each diffusion step. However, its efficiency-accuracy trade-off is not yet well understood. In this paper, we present a rigorous theoretical analysis of a widely used type of diffusion language model, the Masked Diffusion Model (MDM), and find that its effectiveness heavily depends on the target evaluation metric. Under mild conditions, we prove that when using perplexity as the metric, MDMs can achieve near-optimal perplexity in sampling steps regardless of sequence length, demonstrating that efficiency can be achieved without sacrificing performance. However, when using the sequence error rate--which is important for understanding the "correctness" of a sequence, such as a reasoning chain--we show that the required sampling steps must scale linearly with sequence length to obtain "correct" sequences, thereby eliminating MDM's efficiency advantage over autoregressive models. Our analysis establishes the first theoretical foundation for understanding the benefits and limitations of MDMs. All theoretical findings are supported by empirical studies.