2025-05-21

Title: ProdRev: A DNN framework for empowering customers using generative pre-trained transformers

• Authors: Aakash Gupta, Nataraj Das
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.13491
• Pdf URL: https://arxiv.org/pdf/2505.13491
• Copy Paste: [[2505.13491]] ProdRev: A DNN framework for empowering customers using generative pre-trained transformers(https://arxiv.org/abs/2505.13491)
• Keywords: generative
• Abstract: Following the pandemic, customers, preference for using e-commerce has accelerated. Since much information is available in multiple reviews (sometimes running in thousands) for a single product, it can create decision paralysis for the buyer. This scenario disempowers the consumer, who cannot be expected to go over so many reviews since its time consuming and can confuse them. Various commercial tools are available, that use a scoring mechanism to arrive at an adjusted score. It can alert the user to potential review manipulations. This paper proposes a framework that fine-tunes a generative pre-trained transformer to understand these reviews better. Furthermore, using "common-sense" to make better decisions. These models have more than 13 billion parameters. To fine-tune the model for our requirement, we use the curie engine from generative pre-trained transformer (GPT3). By using generative models, we are introducing abstractive summarization. Instead of using a simple extractive method of summarizing the reviews. This brings out the true relationship between the reviews and not simply copy-paste. This introduces an element of "common sense" for the user and helps them to quickly make the right decisions. The user is provided the pros and cons of the processed reviews. Thus the user/customer can take their own decisions.

Title: Optimizing DDoS Detection in SDNs Through Machine Learning Models

• Authors: Md. Ehsanul Haque, Amran Hossain, Md. Shafiqul Alam, Ahsan Habib Siam, Sayed Md Fazle Rabbi, Md. Muntasir Rahman
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.13493
• Pdf URL: https://arxiv.org/pdf/2505.13493
• Copy Paste: [[2505.13493]] Optimizing DDoS Detection in SDNs Through Machine Learning Models(https://arxiv.org/abs/2505.13493)
• Keywords: generative
• Abstract: The emergence of Software-Defined Networking (SDN) has changed the network structure by separating the control plane from the data plane. However, this innovation has also increased susceptibility to DDoS attacks. Existing detection techniques are often ineffective due to data imbalance and accuracy issues; thus, a considerable research gap exists regarding DDoS detection methods suitable for SDN contexts. This research attempts to detect DDoS attacks more effectively using machine learning algorithms: RF, SVC, KNN, MLP, and XGB. For this purpose, both balanced and imbalanced datasets have been used to measure the performance of the models in terms of accuracy and AUC. Based on the analysis, we can say that RF and XGB had the perfect score, 1.0000, in the accuracy and AUC, but since XGB ended with the lowest Brier Score which indicates the highest reliability. MLP achieved an accuracy of 99.93%, SVC an accuracy of 97.65% and KNN an accuracy of 97.87%, which was the next best performers after RF and XGB. These results are consistent with the validity of SDNs as a platform for RF and XGB techniques in detecting DDoS attacks and highlights the importance of balanced datasets for improving detection against generative cyber attacks that are continually evolving.

Title: SPIEDiff: robust learning of long-time macroscopic dynamics from short-time particle simulations with quantified epistemic uncertainty

• Authors: Zequn He, Celia Reina
• Subjects: cs.LG, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2505.13501
• Pdf URL: https://arxiv.org/pdf/2505.13501
• Copy Paste: [[2505.13501]] SPIEDiff: robust learning of long-time macroscopic dynamics from short-time particle simulations with quantified epistemic uncertainty(https://arxiv.org/abs/2505.13501)
• Keywords: diffusion
• Abstract: The data-driven discovery of long-time macroscopic dynamics and thermodynamics of dissipative systems with particle fidelity is hampered by significant obstacles. These include the strong time-scale limitations inherent to particle simulations, the non-uniqueness of the thermodynamic potentials and operators from given macroscopic dynamics, and the need for efficient uncertainty quantification. This paper introduces Statistical-Physics Informed Epistemic Diffusion Models (SPIEDiff), a machine learning framework designed to overcome these limitations in the context of purely dissipative systems by leveraging statistical physics, conditional diffusion models, and epinets. We evaluate the proposed framework on stochastic Arrhenius particle processes and demonstrate that SPIEDiff can accurately uncover both thermodynamics and kinetics, while enabling reliable long-time macroscopic predictions using only short-time particle simulation data. SPIEDiff can deliver accurate predictions with quantified uncertainty in minutes, drastically reducing the computational demand compared to direct particle simulations, which would take days or years in the examples considered. Overall, SPIEDiff offers a robust and trustworthy pathway for the data-driven discovery of thermodynamic models.

Title: Federated Low-Rank Adaptation for Foundation Models: A Survey

• Authors: Yiyuan Yang, Guodong Long, Qinghua Lu, Liming Zhu, Jing Jiang, Chengqi Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.13502
• Pdf URL: https://arxiv.org/pdf/2505.13502
• Copy Paste: [[2505.13502]] Federated Low-Rank Adaptation for Foundation Models: A Survey(https://arxiv.org/abs/2505.13502)
• Keywords: foundation model
• Abstract: Effectively leveraging private datasets remains a significant challenge in developing foundation models. Federated Learning (FL) has recently emerged as a collaborative framework that enables multiple users to fine-tune these models while mitigating data privacy risks. Meanwhile, Low-Rank Adaptation (LoRA) offers a resource-efficient alternative for fine-tuning foundation models by dramatically reducing the number of trainable parameters. This survey examines how LoRA has been integrated into federated fine-tuning for foundation models, an area we term FedLoRA, by focusing on three key challenges: distributed learning, heterogeneity, and efficiency. We further categorize existing work based on the specific methods used to address each challenge. Finally, we discuss open research questions and highlight promising directions for future investigation, outlining the next steps for advancing FedLoRA.

Title: Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models

• Authors: Shuxun Wang, Qingyu Yin, Chak Tou Leong, Qiang Zhang, Linyi Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13514
• Pdf URL: https://arxiv.org/pdf/2505.13514
• Copy Paste: [[2505.13514]] Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models(https://arxiv.org/abs/2505.13514)
• Keywords: in-context
• Abstract: Repetition curse is a phenomenon where Large Language Models (LLMs) generate repetitive sequences of tokens or cyclic sequences. While the repetition curse has been widely observed, its underlying mechanisms remain poorly understood. In this work, we investigate the role of induction heads--a specific type of attention head known for their ability to perform in-context learning--in driving this repetitive behavior. Specifically, we focus on the "toxicity" of induction heads, which we define as their tendency to dominate the model's output logits during repetition, effectively excluding other attention heads from contributing to the generation process. Our findings have important implications for the design and training of LLMs. By identifying induction heads as a key driver of the repetition curse, we provide a mechanistic explanation for this phenomenon and suggest potential avenues for mitigation. We also propose a technique with attention head regularization that could be employed to reduce the dominance of induction heads during generation, thereby promoting more diverse and coherent outputs.

Title: Zero-Shot Forecasting Mortality Rates: A Global Study

• Authors: Gabor Petnehazi, Laith Al Shaggah, Jozsef Gall, Bernadett Aradi
• Subjects: cs.LG, q-fin.RM, stat.AP
• Abstract URL: https://arxiv.org/abs/2505.13521
• Pdf URL: https://arxiv.org/pdf/2505.13521
• Copy Paste: [[2505.13521]] Zero-Shot Forecasting Mortality Rates: A Global Study(https://arxiv.org/abs/2505.13521)
• Keywords: foundation model
• Abstract: This study explores the potential of zero-shot time series forecasting, an innovative approach leveraging pre-trained foundation models, to forecast mortality rates without task-specific fine-tuning. We evaluate two state-of-the-art foundation models, TimesFM and CHRONOS, alongside traditional and machine learning-based methods across three forecasting horizons (5, 10, and 20 years) using data from 50 countries and 111 age groups. In our investigations, zero-shot models showed varying results: while CHRONOS delivered competitive shorter-term forecasts, outperforming traditional methods like ARIMA and the Lee-Carter model, TimesFM consistently underperformed. Fine-tuning CHRONOS on mortality data significantly improved long-term accuracy. A Random Forest model, trained on mortality data, achieved the best overall performance. These findings underscore the potential of zero-shot forecasting while highlighting the need for careful model selection and domain-specific adaptation.

Title: OMGPT: A Sequence Modeling Framework for Data-driven Operational Decision Making

• Authors: Hanzhao Wang, Guanting Chen, Kalyan Talluri, Xiaocheng Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13580
• Pdf URL: https://arxiv.org/pdf/2505.13580
• Copy Paste: [[2505.13580]] OMGPT: A Sequence Modeling Framework for Data-driven Operational Decision Making(https://arxiv.org/abs/2505.13580)
• Keywords: generative
• Abstract: We build a Generative Pre-trained Transformer (GPT) model from scratch to solve sequential decision making tasks arising in contexts of operations research and management science which we call OMGPT. We first propose a general sequence modeling framework to cover several operational decision making tasks as special cases, such as dynamic pricing, inventory management, resource allocation, and queueing control. Under the framework, all these tasks can be viewed as a sequential prediction problem where the goal is to predict the optimal future action given all the historical information. Then we train a transformer-based neural network model (OMGPT) as a natural and powerful architecture for sequential modeling. This marks a paradigm shift compared to the existing methods for these OR/OM tasks in that (i) the OMGPT model can take advantage of the huge amount of pre-trained data; (ii) when tackling these problems, OMGPT does not assume any analytical model structure and enables a direct and rich mapping from the history to the future actions. Either of these two aspects, to the best of our knowledge, is not achieved by any existing method. We establish a Bayesian perspective to theoretically understand the working mechanism of the OMGPT on these tasks, which relates its performance with the pre-training task diversity and the divergence between the testing task and pre-training tasks. Numerically, we observe a surprising performance of the proposed model across all the above tasks.

Title: Self-Supervised Learning for Image Segmentation: A Comprehensive Survey

• Authors: Thangarajah Akilan, Nusrat Jahan, Wandong Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.13584
• Pdf URL: https://arxiv.org/pdf/2505.13584
• Copy Paste: [[2505.13584]] Self-Supervised Learning for Image Segmentation: A Comprehensive Survey(https://arxiv.org/abs/2505.13584)
• Keywords: self-supervised
• Abstract: Supervised learning demands large amounts of precisely annotated data to achieve promising results. Such data curation is labor-intensive and imposes significant overhead regarding time and costs. Self-supervised learning (SSL) partially overcomes these limitations by exploiting vast amounts of unlabeled data and creating surrogate (pretext or proxy) tasks to learn useful representations without manual labeling. As a result, SSL has become a powerful machine learning (ML) paradigm for solving several practical downstream computer vision problems, such as classification, detection, and segmentation. Image segmentation is the cornerstone of many high-level visual perception applications, including medical imaging, intelligent transportation, agriculture, and surveillance. Although there is substantial research potential for developing advanced algorithms for SSL-based semantic segmentation, a comprehensive study of existing methodologies is essential to trace advances and guide emerging researchers. This survey thoroughly investigates over 150 recent image segmentation articles, particularly focusing on SSL. It provides a practical categorization of pretext tasks, downstream tasks, and commonly used benchmark datasets for image segmentation research. It concludes with key observations distilled from a large body of literature and offers future directions to make this research field more accessible and comprehensible for readers.

Title: Incentivizing Truthful Language Models via Peer Elicitation Games

• Authors: Baiting Chen, Tong Zhu, Jiale Han, Lexin Li, Gang Li, Xiaowu Dai
• Subjects: cs.LG, cs.GT
• Abstract URL: https://arxiv.org/abs/2505.13636
• Pdf URL: https://arxiv.org/pdf/2505.13636
• Copy Paste: [[2505.13636]] Incentivizing Truthful Language Models via Peer Elicitation Games(https://arxiv.org/abs/2505.13636)
• Keywords: generative
• Abstract: Large Language Models (LLMs) have demonstrated strong generative capabilities but remain prone to inconsistencies and hallucinations. We introduce Peer Elicitation Games (PEG), a training-free, game-theoretic framework for aligning LLMs through a peer elicitation mechanism involving a generator and multiple discriminators instantiated from distinct base models. Discriminators interact in a peer evaluation setting, where rewards are computed using a determinant-based mutual information score that provably incentivizes truthful reporting without requiring ground-truth labels. We establish theoretical guarantees showing that each agent, via online learning, achieves sublinear regret in the sense their cumulative performance approaches that of the best fixed truthful strategy in hindsight. Moreover, we prove last-iterate convergence to a truthful Nash equilibrium, ensuring that the actual policies used by agents converge to stable and truthful behavior over time. Empirical evaluations across multiple benchmarks demonstrate significant improvements in factual accuracy. These results position PEG as a practical approach for eliciting truthful behavior from LLMs without supervision or fine-tuning.

Title: Self-Reinforced Graph Contrastive Learning

• Authors: Chou-Ying Hsieh, Chun-Fu Jang, Cheng-En Hsieh, Qian-Hui Chen, Sy-Yen Kuo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13650
• Pdf URL: https://arxiv.org/pdf/2505.13650
• Copy Paste: [[2505.13650]] Self-Reinforced Graph Contrastive Learning(https://arxiv.org/abs/2505.13650)
• Keywords: self-supervised
• Abstract: Graphs serve as versatile data structures in numerous real-world domains-including social networks, molecular biology, and knowledge graphs-by capturing intricate relational information among entities. Among graph-based learning techniques, Graph Contrastive Learning (GCL) has gained significant attention for its ability to derive robust, self-supervised graph representations through the contrasting of positive and negative sample pairs. However, a critical challenge lies in ensuring high-quality positive pairs so that the intrinsic semantic and structural properties of the original graph are preserved rather than distorted. To address this issue, we propose SRGCL (Self-Reinforced Graph Contrastive Learning), a novel framework that leverages the model's own encoder to dynamically evaluate and select high-quality positive pairs. We designed a unified positive pair generator employing multiple augmentation strategies, and a selector guided by the manifold hypothesis to maintain the underlying geometry of the latent space. By adopting a probabilistic mechanism for selecting positive pairs, SRGCL iteratively refines its assessment of pair quality as the encoder's representational power improves. Extensive experiments on diverse graph-level classification tasks demonstrate that SRGCL, as a plug-in module, consistently outperforms state-of-the-art GCL methods, underscoring its adaptability and efficacy across various domains.

Title: Unsupervised anomaly detection in MeV ultrafast electron diffraction

• Authors: Mariana A. Fazio, Salvador Sosa Güitron, Marcus Babzien, Mikhail Fedurin, Junjie Li, Mark Palmer, Sandra S. Biedron, Manel Martinez-Ramon
• Subjects: cs.LG, physics.ins-det
• Abstract URL: https://arxiv.org/abs/2505.13702
• Pdf URL: https://arxiv.org/pdf/2505.13702
• Copy Paste: [[2505.13702]] Unsupervised anomaly detection in MeV ultrafast electron diffraction(https://arxiv.org/abs/2505.13702)
• Keywords: anomaly
• Abstract: This study focus in the construction of an unsupervised anomaly detection methodology to detect faulty images in MUED. We believe that unsupervised techniques are the best choice for our purposes because the data used to train the detector does not need to be manually labeled, and instead, the machine is intended to detect by itself the anomalies in the dataset, which liberates the user of tedious, time-consuming initial image examination. The structure must, additionally, provide the user with some measure of uncertainty in the detection, so the user can take decisions based on this measure.

Title: Improving Compositional Generation with Diffusion Models Using Lift Scores

• Authors: Chenning Yu, Sicun Gao
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.13740
• Pdf URL: https://arxiv.org/pdf/2505.13740
• Copy Paste: [[2505.13740]] Improving Compositional Generation with Diffusion Models Using Lift Scores(https://arxiv.org/abs/2505.13740)
• Keywords: diffusion
• Abstract: We introduce a novel resampling criterion using lift scores, for improving compositional generation in diffusion models. By leveraging the lift scores, we evaluate whether generated samples align with each single condition and then compose the results to determine whether the composed prompt is satisfied. Our key insight is that lift scores can be efficiently approximated using only the original diffusion model, requiring no additional training or external modules. We develop an optimized variant that achieves relatively lower computational overhead during inference while maintaining effectiveness. Through extensive experiments, we demonstrate that lift scores significantly improved the condition alignment for compositional generation across 2D synthetic data, CLEVR position tasks, and text-to-image synthesis. Our code is available at this http URL.

Title: Panda: A pretrained forecast model for universal representation of chaotic dynamics

• Authors: Jeffrey Lai, Anthony Bao, William Gilpin
• Subjects: cs.LG, nlin.CD, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.13755
• Pdf URL: https://arxiv.org/pdf/2505.13755
• Copy Paste: [[2505.13755]] Panda: A pretrained forecast model for universal representation of chaotic dynamics(https://arxiv.org/abs/2505.13755)
• Keywords: foundation model
• Abstract: Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or neuronal activity. Prior efforts comprise either specialized models trained separately on individual time series, or foundation models trained on vast time series databases with little underlying dynamical structure. Motivated by dynamical systems theory, we present Panda, Patched Attention for Nonlinear DynAmics. We train Panda on a novel synthetic, extensible dataset of $2 \times 10^4$ chaotic dynamical systems that we discover using an evolutionary algorithm. Trained purely on simulated data, Panda exhibits emergent properties: zero-shot forecasting of unseen real world chaotic systems, and nonlinear resonance patterns in cross-channel attention heads. Despite having been trained only on low-dimensional ordinary differential equations, Panda spontaneously develops the ability to predict partial differential equations without retraining. We demonstrate a neural scaling law for differential equations, underscoring the potential of pretrained models for probing abstract mathematical domains like nonlinear dynamics.

Title: Scalable Autoregressive 3D Molecule Generation

• Authors: Austin H. Cheng, Chong Sun, Alán Aspuru-Guzik
• Subjects: cs.LG, physics.chem-ph
• Abstract URL: https://arxiv.org/abs/2505.13791
• Pdf URL: https://arxiv.org/pdf/2505.13791
• Copy Paste: [[2505.13791]] Scalable Autoregressive 3D Molecule Generation(https://arxiv.org/abs/2505.13791)
• Keywords: diffusion, generative
• Abstract: Generative models of 3D molecular structure play a rapidly growing role in the design and simulation of molecules. Diffusion models currently dominate the space of 3D molecule generation, while autoregressive models have trailed behind. In this work, we present Quetzal, a simple but scalable autoregressive model that builds molecules atom-by-atom in 3D. Treating each molecule as an ordered sequence of atoms, Quetzal combines a causal transformer that predicts the next atom's discrete type with a smaller Diffusion MLP that models the continuous next-position distribution. Compared to existing autoregressive baselines, Quetzal achieves substantial improvements in generation quality and is competitive with the performance of state-of-the-art diffusion models. In addition, by reducing the number of expensive forward passes through a dense transformer, Quetzal enables significantly faster generation speed, as well as exact divergence-based likelihood computation. Finally, without any architectural changes, Quetzal natively handles variable-size tasks like hydrogen decoration and scaffold completion. We hope that our work motivates a perspective on scalability and generality for generative modelling of 3D molecules.

Title: Physics-Driven Local-Whole Elastic Deformation Modeling for Point Cloud Representation Learning

• Authors: Zhongyu Chen, Rong Zhao, Xie Han, Xindong Guo, Song Wang, Zherui Qiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.13812
• Pdf URL: https://arxiv.org/pdf/2505.13812
• Copy Paste: [[2505.13812]] Physics-Driven Local-Whole Elastic Deformation Modeling for Point Cloud Representation Learning(https://arxiv.org/abs/2505.13812)
• Keywords: self-supervised
• Abstract: Existing point cloud representation learning tend to learning the geometric distribution of objects through data-driven approaches, emphasizing structural features while overlooking the relationship between the local information and the whole structure. Local features reflect the fine-grained variations of an object, while the whole structure is determined by the interaction and combination of these local features, collectively defining the object's shape. In real-world, objects undergo elastic deformation under external forces, and this deformation gradually affects the whole structure through the propagation of forces from local regions, thereby altering the object's geometric properties. Inspired by this, we propose a physics-driven self-supervised learning method for point cloud representation, which captures the relationship between parts and the whole by constructing a local-whole force propagation mechanism. Specifically, we employ a dual-task encoder-decoder framework, integrating the geometric modeling capability of implicit fields with physics-driven elastic deformation. The encoder extracts features from the point cloud and its tetrahedral mesh representation, capturing both geometric and physical properties. These features are then fed into two decoders: one learns the whole geometric shape of the point cloud through an implicit field, while the other predicts local deformations using two specifically designed physics information loss functions, modeling the deformation relationship between local and whole shapes. Experimental results show that our method outperforms existing approaches in object classification, few-shot learning, and segmentation, demonstrating its effectiveness.

Title: EfficientLLM: Efficiency in Large Language Models

• Authors: Zhengqing Yuan, Weixiang Sun, Yixin Liu, Huichi Zhou, Rong Zhou, Yiyang Li, Zheyuan Zhang, Wei Song, Yue Huang, Haolong Jia, Keerthiram Murugesan, Yu Wang, Lifang He, Jianfeng Gao, Lichao Sun, Yanfang Ye
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13840
• Pdf URL: https://arxiv.org/pdf/2505.13840
• Copy Paste: [[2505.13840]] EfficientLLM: Efficiency in Large Language Models(https://arxiv.org/abs/2505.13840)
• Keywords: diffusion, foundation model
• Abstract: Large Language Models (LLMs) have driven significant progress, yet their growing parameter counts and context windows incur prohibitive compute, energy, and monetary costs. We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale. Conducted on a production-class cluster (48xGH200, 8xH200 GPUs), our study systematically explores three key axes: (1) architecture pretraining (efficient attention variants: MQA, GQA, MLA, NSA; sparse Mixture-of-Experts (MoE)), (2) fine-tuning (parameter-efficient methods: LoRA, RSLoRA, DoRA), and (3) inference (quantization methods: int4, float16). We define six fine-grained metrics (Memory Utilization, Compute Utilization, Latency, Throughput, Energy Consumption, Compression Rate) to capture hardware saturation, latency-throughput balance, and carbon cost. Evaluating over 100 model-technique pairs (0.5B-72B parameters), we derive three core insights: (i) Efficiency involves quantifiable trade-offs: no single method is universally optimal; e.g., MoE reduces FLOPs and improves accuracy but increases VRAM by 40%, while int4 quantization cuts memory/energy by up to 3.9x at a 3-5% accuracy drop. (ii) Optima are task- and scale-dependent: MQA offers optimal memory-latency trade-offs for constrained devices, MLA achieves lowest perplexity for quality-critical tasks, and RSLoRA surpasses LoRA efficiency only beyond 14B parameters. (iii) Techniques generalize across modalities: we extend evaluations to Large Vision Models (Stable Diffusion 3.5, Wan 2.1) and Vision-Language Models (Qwen2.5-VL), confirming effective transferability. By open-sourcing datasets, evaluation pipelines, and leaderboards, EfficientLLM provides essential guidance for researchers and engineers navigating the efficiency-performance landscape of next-generation foundation models.

Title: Domain Gating Ensemble Networks for AI-Generated Text Detection

• Authors: Arihant Tripathi, Liam Dugan, Charis Gao, Maggie Huan, Emma Jin, Peter Zhang, David Zhang, Julia Zhao, Chris Callison-Burch
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.13855
• Pdf URL: https://arxiv.org/pdf/2505.13855
• Copy Paste: [[2505.13855]] Domain Gating Ensemble Networks for AI-Generated Text Detection(https://arxiv.org/abs/2505.13855)
• Keywords: generative
• Abstract: As state-of-the-art language models continue to improve, the need for robust detection of machine-generated text becomes increasingly critical. However, current state-of-the-art machine text detectors struggle to adapt to new unseen domains and generative models. In this paper we present DoGEN (Domain Gating Ensemble Networks), a technique that allows detectors to adapt to unseen domains by ensembling a set of domain expert detector models using weights from a domain classifier. We test DoGEN on a wide variety of domains from leading benchmarks and find that it achieves state-of-the-art performance on in-domain detection while outperforming models twice its size on out-of-domain detection. We release our code and trained models to assist in future research in domain-adaptive AI detection.

Title: Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model

• Authors: Peisong Niu, Ziqing Ma, Tian Zhou, Weiqi Chen, Lefei Shen, Rong Jin, Liang Sun
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13873
• Pdf URL: https://arxiv.org/pdf/2505.13873
• Copy Paste: [[2505.13873]] Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model(https://arxiv.org/abs/2505.13873)
• Keywords: self-supervised
• Abstract: Weather forecasting has long posed a significant challenge for humanity. While recent AI-based models have surpassed traditional numerical weather prediction (NWP) methods in global forecasting tasks, overfitting remains a critical issue due to the limited availability of real-world weather data spanning only a few decades. Unlike fields like computer vision or natural language processing, where data abundance can mitigate overfitting, weather forecasting demands innovative strategies to address this challenge with existing data. In this paper, we explore pre-training methods for weather forecasting, finding that selecting an appropriately challenging pre-training task introduces locality bias, effectively mitigating overfitting and enhancing performance. We introduce Baguan, a novel data-driven model for medium-range weather forecasting, built on a Siamese Autoencoder pre-trained in a self-supervised manner and fine-tuned for different lead times. Experimental results show that Baguan outperforms traditional methods, delivering more accurate forecasts. Additionally, the pre-trained Baguan demonstrates robust overfitting control and excels in downstream tasks, such as subseasonal-to-seasonal (S2S) modeling and regional forecasting, after fine-tuning.

Title: Exploring Causes of Representational Similarity in Machine Learning Models

• Authors: Zeyu Michael Li, Hung Anh Vu, Damilola Awofisayo, Emily Wenger
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.13899
• Pdf URL: https://arxiv.org/pdf/2505.13899
• Copy Paste: [[2505.13899]] Exploring Causes of Representational Similarity in Machine Learning Models(https://arxiv.org/abs/2505.13899)
• Keywords: generative
• Abstract: Numerous works have noted significant similarities in how machine learning models represent the world, even across modalities. Although much effort has been devoted to uncovering properties and metrics on which these models align, surprisingly little work has explored causes of this similarity. To advance this line of inquiry, this work explores how two possible causal factors -- dataset overlap and task overlap -- influence downstream model similarity. The exploration of dataset overlap is motivated by the reality that large-scale generative AI models are often trained on overlapping datasets of scraped internet data, while the exploration of task overlap seeks to substantiate claims from a recent work, the Platonic Representation Hypothesis, that task similarity may drive model similarity. We evaluate the effects of both factors through a broad set of experiments. We find that both positively correlate with higher representational similarity and that combining them provides the strongest effect. Our code and dataset are published.

Title: Cross-Domain Diffusion with Progressive Alignment for Efficient Adaptive Retrieval

• Authors: Junyu Luo, Yusheng Zhao, Xiao Luo, Zhiping Xiao, Wei Ju, Li Shen, Dacheng Tao, Ming Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.13907
• Pdf URL: https://arxiv.org/pdf/2505.13907
• Copy Paste: [[2505.13907]] Cross-Domain Diffusion with Progressive Alignment for Efficient Adaptive Retrieval(https://arxiv.org/abs/2505.13907)
• Keywords: diffusion
• Abstract: Unsupervised efficient domain adaptive retrieval aims to transfer knowledge from a labeled source domain to an unlabeled target domain, while maintaining low storage cost and high retrieval efficiency. However, existing methods typically fail to address potential noise in the target domain, and directly align high-level features across domains, thus resulting in suboptimal retrieval performance. To address these challenges, we propose a novel Cross-Domain Diffusion with Progressive Alignment method (COUPLE). This approach revisits unsupervised efficient domain adaptive retrieval from a graph diffusion perspective, simulating cross-domain adaptation dynamics to achieve a stable target domain adaptation process. First, we construct a cross-domain relationship graph and leverage noise-robust graph flow diffusion to simulate the transfer dynamics from the source domain to the target domain, identifying lower noise clusters. We then leverage the graph diffusion results for discriminative hash code learning, effectively learning from the target domain while reducing the negative impact of noise. Furthermore, we employ a hierarchical Mixup operation for progressive domain alignment, which is performed along the cross-domain random walk paths. Utilizing target domain discriminative hash learning and progressive domain alignment, COUPLE enables effective domain adaptive hash learning. Extensive experiments demonstrate COUPLE's effectiveness on competitive benchmarks.

Title: Blind Restoration of High-Resolution Ultrasound Video

• Authors: Chu Chen, Kangning Cui, Pasquale Cascarano, Wei Tang, Elena Loli Piccolomini, Raymond H. Chan
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.13915
• Pdf URL: https://arxiv.org/pdf/2505.13915
• Copy Paste: [[2505.13915]] Blind Restoration of High-Resolution Ultrasound Video(https://arxiv.org/abs/2505.13915)
• Keywords: self-supervised
• Abstract: Ultrasound imaging is widely applied in clinical practice, yet ultrasound videos often suffer from low signal-to-noise ratios (SNR) and limited resolutions, posing challenges for diagnosis and analysis. Variations in equipment and acquisition settings can further exacerbate differences in data distribution and noise levels, reducing the generalizability of pre-trained models. This work presents a self-supervised ultrasound video super-resolution algorithm called Deep Ultrasound Prior (DUP). DUP employs a video-adaptive optimization process of a neural network that enhances the resolution of given ultrasound videos without requiring paired training data while simultaneously removing noise. Quantitative and visual evaluations demonstrate that DUP outperforms existing super-resolution algorithms, leading to substantial improvements for downstream applications.

Title: RLVR-World: Training World Models with Reinforcement Learning

• Authors: Jialong Wu, Shaofeng Yin, Ningya Feng, Mingsheng Long
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.13934
• Pdf URL: https://arxiv.org/pdf/2505.13934
• Copy Paste: [[2505.13934]] RLVR-World: Training World Models with Reinforcement Learning(https://arxiv.org/abs/2505.13934)
• Keywords: generative
• Abstract: World models predict state transitions in response to actions and are increasingly developed across diverse modalities. However, standard training objectives such as maximum likelihood estimation (MLE) often misalign with task-specific goals of world models, i.e., transition prediction metrics like accuracy or perceptual quality. In this paper, we present RLVR-World, a unified framework that leverages reinforcement learning with verifiable rewards (RLVR) to directly optimize world models for such metrics. Despite formulating world modeling as autoregressive prediction of tokenized sequences, RLVR-World evaluates metrics of decoded predictions as verifiable rewards. We demonstrate substantial performance gains on both language- and video-based world models across domains, including text games, web navigation, and robot manipulation. Our work indicates that, beyond recent advances in reasoning language models, RLVR offers a promising post-training paradigm for enhancing the utility of generative models more broadly.

Title: Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting

• Authors: Bao-Ngoc Dao, Quang Nguyen, Luyen Ngo Dinh, Minh Le, Nam Le, Linh Ngo Van
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.13944
• Pdf URL: https://arxiv.org/pdf/2505.13944
• Copy Paste: [[2505.13944]] Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting(https://arxiv.org/abs/2505.13944)
• Keywords: generative
• Abstract: Memory-based approaches have shown strong performance in Continual Relation Extraction (CRE). However, storing examples from previous tasks increases memory usage and raises privacy concerns. Recently, prompt-based methods have emerged as a promising alternative, as they do not rely on storing past samples. Despite this progress, current prompt-based techniques face several core challenges in CRE, particularly in accurately identifying task identities and mitigating catastrophic forgetting. Existing prompt selection strategies often suffer from inaccuracies, lack robust mechanisms to prevent forgetting in shared parameters, and struggle to handle both cross-task and within-task variations. In this paper, we propose WAVE++, a novel approach inspired by the connection between prefix-tuning and mixture of experts. Specifically, we introduce task-specific prompt pools that enhance flexibility and adaptability across diverse tasks while avoiding boundary-spanning risks; this design more effectively captures variations within each task and across tasks. To further refine relation classification, we incorporate label descriptions that provide richer, more global context, enabling the model to better distinguish among different relations. We also propose a training-free mechanism to improve task prediction during inference. Moreover, we integrate a generative model to consolidate prior knowledge within the shared parameters, thereby removing the need for explicit data storage. Extensive experiments demonstrate that WAVE++ outperforms state-of-the-art prompt-based and rehearsal-based methods, offering a more robust solution for continual relation extraction. Our code is publicly available at this https URL.

Title: CSAGC-IDS: A Dual-Module Deep Learning Network Intrusion Detection Model for Complex and Imbalanced Data

• Authors: Yifan Zeng
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14027
• Pdf URL: https://arxiv.org/pdf/2505.14027
• Copy Paste: [[2505.14027]] CSAGC-IDS: A Dual-Module Deep Learning Network Intrusion Detection Model for Complex and Imbalanced Data(https://arxiv.org/abs/2505.14027)
• Keywords: generative
• Abstract: As computer networks proliferate, the gravity of network intrusions has escalated, emphasizing the criticality of network intrusion detection systems for safeguarding security. While deep learning models have exhibited promising results in intrusion detection, they face challenges in managing high-dimensional, complex traffic patterns and imbalanced data categories. This paper presents CSAGC-IDS, a network intrusion detection model based on deep learning techniques. CSAGC-IDS integrates SC-CGAN, a self-attention-enhanced convolutional conditional generative adversarial network that generates high-quality data to mitigate class imbalance. Furthermore, CSAGC-IDS integrates CSCA-CNN, a convolutional neural network enhanced through cost sensitive learning and channel attention mechanism, to extract features from complex traffic data for precise detection. Experiments conducted on the NSL-KDD dataset. CSAGC-IDS achieves an accuracy of 84.55% and an F1-score of 84.52% in five-class classification task, and an accuracy of 91.09% and an F1 score of 92.04% in binary classification this http URL, this paper provides an interpretability analysis of the proposed model, using SHAP and LIME to explain the decision-making mechanisms of the model.

Title: OmniStyle: Filtering High Quality Style Transfer Data at Scale

• Authors: Ye Wang, Ruiqi Liu, Jiang Lin, Fei Liu, Zili Yi, Yilin Wang, Rui Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14028
• Pdf URL: https://arxiv.org/pdf/2505.14028
• Copy Paste: [[2505.14028]] OmniStyle: Filtering High Quality Style Transfer Data at Scale(https://arxiv.org/abs/2505.14028)
• Keywords: diffusion
• Abstract: In this paper, we introduce OmniStyle-1M, a large-scale paired style transfer dataset comprising over one million content-style-stylized image triplets across 1,000 diverse style categories, each enhanced with textual descriptions and instruction prompts. We show that OmniStyle-1M can not only enable efficient and scalable of style transfer models through supervised training but also facilitate precise control over target stylization. Especially, to ensure the quality of the dataset, we introduce OmniFilter, a comprehensive style transfer quality assessment framework, which filters high-quality triplets based on content preservation, style consistency, and aesthetic appeal. Building upon this foundation, we propose OmniStyle, a framework based on the Diffusion Transformer (DiT) architecture designed for high-quality and efficient style transfer. This framework supports both instruction-guided and image-guided style transfer, generating high resolution outputs with exceptional detail. Extensive qualitative and quantitative evaluations demonstrate OmniStyle's superior performance compared to existing approaches, highlighting its efficiency and versatility. OmniStyle-1M and its accompanying methodologies provide a significant contribution to advancing high-quality style transfer, offering a valuable resource for the research community.

Title: Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening

• Authors: Guoming Li, Jian Yang, Yifan Chen
• Subjects: cs.LG, eess.SP, math.NA
• Abstract URL: https://arxiv.org/abs/2505.14033
• Pdf URL: https://arxiv.org/pdf/2505.14033
• Copy Paste: [[2505.14033]] Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening(https://arxiv.org/abs/2505.14033)
• Keywords: anomaly
• Abstract: Filtering-based graph neural networks (GNNs) constitute a distinct class of GNNs that employ graph filters to handle graph-structured data, achieving notable success in various graph-related tasks. Conventional methods adopt a graph-wise filtering paradigm, imposing a uniform filter across all nodes, yet recent findings suggest that this rigid paradigm struggles with heterophilic graphs. To overcome this, recent works have introduced node-wise filtering, which assigns distinct filters to individual nodes, offering enhanced adaptability. However, a fundamental gap remains: a comprehensive framework unifying these two strategies is still absent, limiting theoretical insights into the filtering paradigms. Moreover, through the lens of Contextual Stochastic Block Model, we reveal that a synthesis of graph-wise and node-wise filtering provides a sufficient solution for classification on graphs exhibiting both homophily and heterophily, suggesting the risk of excessive parameterization and potential overfitting with node-wise filtering. To address the limitations, this paper introduces Coarsening-guided Partition-wise Filtering (CPF). CPF innovates by performing filtering on node partitions. The method begins with structure-aware partition-wise filtering, which filters node partitions obtained via graph coarsening algorithms, and then performs feature-aware partition-wise filtering, refining node embeddings via filtering on clusters produced by $k$-means clustering over features. In-depth analysis is conducted for each phase of CPF, showing its superiority over other paradigms. Finally, benchmark node classification experiments, along with a real-world graph anomaly detection application, validate CPF's efficacy and practical utility.

Title: Adaptive Cyclic Diffusion for Inference Scaling

• Authors: Gyubin Lee, Truong Nhat Nguyen Bao, Jaesik Yoon, Dongwoo Lee, Minsu Kim, Yoshua Bengio, Sungjin Ahn
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14036
• Pdf URL: https://arxiv.org/pdf/2505.14036
• Copy Paste: [[2505.14036]] Adaptive Cyclic Diffusion for Inference Scaling(https://arxiv.org/abs/2505.14036)
• Keywords: diffusion, generative
• Abstract: Diffusion models have demonstrated strong generative capabilities across domains ranging from image synthesis to complex reasoning tasks. However, most inference-time scaling methods rely on fixed denoising schedules, limiting their ability to allocate computation based on instance difficulty or task-specific demands adaptively. We introduce the challenge of adaptive inference-time scaling-dynamically adjusting computational effort during inference-and propose Adaptive Bi-directional Cyclic Diffusion (ABCD), a flexible, search-based inference framework. ABCD refines outputs through bi-directional diffusion cycles while adaptively controlling exploration depth and termination. It comprises three components: Cyclic Diffusion Search, Automatic Exploration-Exploitation Balancing, and Adaptive Thinking Time. Experiments show that ABCD improves performance across diverse tasks while maintaining computational efficiency.

Title: Adversarially Pretrained Transformers may be Universally Robust In-Context Learners

• Authors: Soichiro Kumano, Hiroshi Kera, Toshihiko Yamasaki
• Subjects: cs.LG, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.14042
• Pdf URL: https://arxiv.org/pdf/2505.14042
• Copy Paste: [[2505.14042]] Adversarially Pretrained Transformers may be Universally Robust In-Context Learners(https://arxiv.org/abs/2505.14042)
• Keywords: foundation model, in-context
• Abstract: Adversarial training is one of the most effective adversarial defenses, but it incurs a high computational cost. In this study, we show that transformers adversarially pretrained on diverse tasks can serve as robust foundation models and eliminate the need for adversarial training in downstream tasks. Specifically, we theoretically demonstrate that through in-context learning, a single adversarially pretrained transformer can robustly generalize to multiple unseen tasks without any additional training, i.e., without any parameter updates. This robustness stems from the model's focus on robust features and its resistance to attacks that exploit non-predictive features. Besides these positive findings, we also identify several limitations. Under certain conditions (though unrealistic), no universally robust single-layer transformers exist. Moreover, robust transformers exhibit an accuracy--robustness trade-off and require a large number of in-context demonstrations. The code is available at this https URL.

Title: Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts

• Authors: Xi Chen, Shen Yan, Juelin Zhu, Chen Chen, Yu Liu, Maojun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14088
• Pdf URL: https://arxiv.org/pdf/2505.14088
• Copy Paste: [[2505.14088]] Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts(https://arxiv.org/abs/2505.14088)
• Keywords: foundation model
• Abstract: We introduce Land-MoE, a novel approach for multispectral land cover classification (MLCC). Spectral shift, which emerges from disparities in sensors and geospatial conditions, poses a significant challenge in this domain. Existing methods predominantly rely on domain adaptation and generalization strategies, often utilizing small-scale models that exhibit limited performance. In contrast, Land-MoE addresses these issues by hierarchically inserting a Frequency-aware Mixture of Low-rank Token Experts, to fine-tune Vision Foundation Models (VFMs) in a parameter-efficient manner. Specifically, Land-MoE comprises two key modules: the mixture of low-rank token experts (MoLTE) and frequency-aware filters (FAF). MoLTE leverages rank-differentiated tokens to generate diverse feature adjustments for individual instances within multispectral images. By dynamically combining learnable low-rank token experts of varying ranks, it enhances the robustness against spectral shifts. Meanwhile, FAF conducts frequency-domain modulation on the refined features. This process enables the model to effectively capture frequency band information that is strongly correlated with semantic essence, while simultaneously suppressing frequency noise irrelevant to the task. Comprehensive experiments on MLCC tasks involving cross-sensor and cross-geospatial setups demonstrate that Land-MoE outperforms existing methods by a large margin. Additionally, the proposed approach has also achieved state-of-the-art performance in domain generalization semantic segmentation tasks of RGB remote sensing images.

Title: Unlocking the Power of SAM 2 for Few-Shot Segmentation

• Authors: Qianxiong Xu, Lanyun Zhu, Xuanyi Liu, Guosheng Lin, Cheng Long, Ziyue Li, Rui Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14100
• Pdf URL: https://arxiv.org/pdf/2505.14100
• Copy Paste: [[2505.14100]] Unlocking the Power of SAM 2 for Few-Shot Segmentation(https://arxiv.org/abs/2505.14100)
• Keywords: foundation model
• Abstract: Few-Shot Segmentation (FSS) aims to learn class-agnostic segmentation on few classes to segment arbitrary classes, but at the risk of overfitting. To address this, some methods use the well-learned knowledge of foundation models (e.g., SAM) to simplify the learning process. Recently, SAM 2 has extended SAM by supporting video segmentation, whose class-agnostic matching ability is useful to FSS. A simple idea is to encode support foreground (FG) features as memory, with which query FG features are matched and fused. Unfortunately, the FG objects in different frames of SAM 2's video data are always the same identity, while those in FSS are different identities, i.e., the matching step is incompatible. Therefore, we design Pseudo Prompt Generator to encode pseudo query memory, matching with query features in a compatible way. However, the memories can never be as accurate as the real ones, i.e., they are likely to contain incomplete query FG, and some unexpected query background (BG) features, leading to wrong segmentation. Hence, we further design Iterative Memory Refinement to fuse more query FG features into the memory, and devise a Support-Calibrated Memory Attention to suppress the unexpected query BG features in memory. Extensive experiments have been conducted on PASCAL-5$^i$ and COCO-20$^i$ to validate the effectiveness of our design, e.g., the 1-shot mIoU can be 4.2\% better than the best baseline.

Title: MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations

• Authors: Ernests Lavrinovics, Russa Biswas, Katja Hose, Johannes Bjerva
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14101
• Pdf URL: https://arxiv.org/pdf/2505.14101
• Copy Paste: [[2505.14101]] MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations(https://arxiv.org/abs/2505.14101)
• Keywords: generative
• Abstract: Large Language Models (LLMs) have inherent limitations of faithfulness and factuality, commonly referred to as hallucinations. Several benchmarks have been developed that provide a test bed for factuality evaluation within the context of English-centric datasets, while relying on supplementary informative context like web links or text passages but ignoring the available structured factual resources. To this end, Knowledge Graphs (KGs) have been identified as a useful aid for hallucination mitigation, as they provide a structured way to represent the facts about entities and their relations with minimal linguistic overhead. We bridge the lack of KG paths and multilinguality for factual language modeling within the existing hallucination evaluation benchmarks and propose a KG-based multilingual, multihop benchmark called \textbf{MultiHal} framed for generative text evaluation. As part of our data collection pipeline, we mined 140k KG-paths from open-domain KGs, from which we pruned noisy KG-paths, curating a high-quality subset of 25.9k. Our baseline evaluation shows an absolute scale increase by approximately 0.12 to 0.36 points for the semantic similarity score in KG-RAG over vanilla QA across multiple languages and multiple models, demonstrating the potential of KG integration. We anticipate MultiHal will foster future research towards several graph-based hallucination mitigation and fact-checking tasks.

Title: Hunyuan-Game: Industrial-grade Intelligent Game Creation Model

• Authors: Ruihuang Li, Caijin Zhou, Shoujian Zheng, Jianxiang Lu, Jiabin Huang, Comi Chen, Junshu Tang, Guangzheng Xu, Jiale Tao, Hongmei Wang, Donghao Li, Wenqing Yu, Senbo Wang, Zhimin Li, Yetshuan Shi, Haoyu Yang, Yukun Wang, Wenxun Dai, Jiaqi Li, Linqing Wang, Qixun Wang, Zhiyong Xu, Yingfang Zhang, Jiangfeng Xiong, Weijie Kong, Chao Zhang, Hongxin Zhang, Qiaoling Zheng, Weiting Guo, Xinchi Deng, Yixuan Li, Renjia Wei, Yulin Jian, Duojun Huang, Xuhua Ren, Sihuan Lin, Yifu Sun, Yuan Zhou, Joey Wang, Qin Lin, Jingmiao Yu, Jihong Zhang, Caesar Zhong, Di Wang, Yuhong Liu, Linus, Jie Jiang, Longhuang Wu, Shuai Shao, Qinglin Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14135
• Pdf URL: https://arxiv.org/pdf/2505.14135
• Copy Paste: [[2505.14135]] Hunyuan-Game: Industrial-grade Intelligent Game Creation Model(https://arxiv.org/abs/2505.14135)
• Keywords: generative
• Abstract: Intelligent game creation represents a transformative advancement in game development, utilizing generative artificial intelligence to dynamically generate and enhance game content. Despite notable progress in generative models, the comprehensive synthesis of high-quality game assets, including both images and videos, remains a challenging frontier. To create high-fidelity game content that simultaneously aligns with player preferences and significantly boosts designer efficiency, we present Hunyuan-Game, an innovative project designed to revolutionize intelligent game production. Hunyuan-Game encompasses two primary branches: image generation and video generation. The image generation component is built upon a vast dataset comprising billions of game images, leading to the development of a group of customized image generation models tailored for game scenarios: (1) General Text-to-Image Generation. (2) Game Visual Effects Generation, involving text-to-effect and reference image-based game visual effect generation. (3) Transparent Image Generation for characters, scenes, and game visual effects. (4) Game Character Generation based on sketches, black-and-white images, and white models. The video generation component is built upon a comprehensive dataset of millions of game and anime videos, leading to the development of five core algorithmic models, each targeting critical pain points in game development and having robust adaptation to diverse game video scenarios: (1) Image-to-Video Generation. (2) 360 A/T Pose Avatar Video Synthesis. (3) Dynamic Illustration Generation. (4) Generative Video Super-Resolution. (5) Interactive Game Video Generation. These image and video generation models not only exhibit high-level aesthetic expression but also deeply integrate domain-specific knowledge, establishing a systematic understanding of diverse game and anime art styles.

Title: FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning

• Authors: Marvin Alles, Nutan Chen, Patrick van der Smagt, Botond Cseke
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.14139
• Pdf URL: https://arxiv.org/pdf/2505.14139
• Copy Paste: [[2505.14139]] FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning(https://arxiv.org/abs/2505.14139)
• Keywords: diffusion
• Abstract: The use of guidance to steer sampling toward desired outcomes has been widely explored within diffusion models, especially in applications such as image and trajectory generation. However, incorporating guidance during training remains relatively underexplored. In this work, we introduce energy-guided flow matching, a novel approach that enhances the training of flow models and eliminates the need for guidance at inference time. We learn a conditional velocity field corresponding to the flow policy by approximating an energy-guided probability path as a Gaussian path. Learning guided trajectories is appealing for tasks where the target distribution is defined by a combination of data and an energy function, as in reinforcement learning. Diffusion-based policies have recently attracted attention for their expressive power and ability to capture multi-modal action distributions. Typically, these policies are optimized using weighted objectives or by back-propagating gradients through actions sampled by the policy. As an alternative, we propose FlowQ, an offline reinforcement learning algorithm based on energy-guided flow matching. Our method achieves competitive performance while the policy training time is constant in the number of flow sampling steps.

Title: ReactDiff: Latent Diffusion for Facial Reaction Generation

• Authors: Jiaming Li, Sheng Wang, Xin Wang, Yitao Zhu, Honglin Xiong, Zixu Zhuang, Qian Wang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.14151
• Pdf URL: https://arxiv.org/pdf/2505.14151
• Copy Paste: [[2505.14151]] ReactDiff: Latent Diffusion for Facial Reaction Generation(https://arxiv.org/abs/2505.14151)
• Keywords: diffusion
• Abstract: Given the audio-visual clip of the speaker, facial reaction generation aims to predict the listener's facial reactions. The challenge lies in capturing the relevance between video and audio while balancing appropriateness, realism, and diversity. While prior works have mostly focused on uni-modal inputs or simplified reaction mappings, recent approaches such as PerFRDiff have explored multi-modal inputs and the one-to-many nature of appropriate reaction mappings. In this work, we propose the Facial Reaction Diffusion (ReactDiff) framework that uniquely integrates a Multi-Modality Transformer with conditional diffusion in the latent space for enhanced reaction generation. Unlike existing methods, ReactDiff leverages intra- and inter-class attention for fine-grained multi-modal interaction, while the latent diffusion process between the encoder and decoder enables diverse yet contextually appropriate outputs. Experimental results demonstrate that ReactDiff significantly outperforms existing approaches, achieving a facial reaction correlation of 0.26 and diversity score of 0.094 while maintaining competitive realism. The code is open-sourced at \href{this https URL}{github}.

Title: Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search

• Authors: Songhao Wu, Quan Tu, Hong Liu, Jia Xu, Zhongyi Liu, Guannan Zhang, Ran Wang, Xiuying Chen, Rui Yan
• Subjects: cs.CV, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2505.14156
• Pdf URL: https://arxiv.org/pdf/2505.14156
• Copy Paste: [[2505.14156]] Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search(https://arxiv.org/abs/2505.14156)
• Keywords: self-supervised, generative
• Abstract: Session search involves a series of interactive queries and actions to fulfill user's complex information need. Current strategies typically prioritize sequential modeling for deep semantic understanding, overlooking the graph structure in interactions. While some approaches focus on capturing structural information, they use a generalized representation for documents, neglecting the word-level semantic modeling. In this paper, we propose Symbolic Graph Ranker (SGR), which aims to take advantage of both text-based and graph-based approaches by leveraging the power of recent Large Language Models (LLMs). Concretely, we first introduce a set of symbolic grammar rules to convert session graph into text. This allows integrating session history, interaction process, and task instruction seamlessly as inputs for the LLM. Moreover, given the natural discrepancy between LLMs pre-trained on textual corpora, and the symbolic language we produce using our graph-to-text grammar, our objective is to enhance LLMs' ability to capture graph structures within a textual format. To achieve this, we introduce a set of self-supervised symbolic learning tasks including link prediction, node content generation, and generative contrastive learning, to enable LLMs to capture the topological information from coarse-grained to fine-grained. Experiment results and comprehensive analysis on two benchmark datasets, AOL and Tiangong-ST, confirm the superiority of our approach. Our paradigm also offers a novel and effective methodology that bridges the gap between traditional search strategies and modern LLMs.

Title: LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer

• Authors: Changgu Chen, Xiaoyan Yang, Junwei Shu, Changbo Wang, Yang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14167
• Pdf URL: https://arxiv.org/pdf/2505.14167
• Copy Paste: [[2505.14167]] LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer(https://arxiv.org/abs/2505.14167)
• Keywords: diffusion, generative
• Abstract: In recent years, large-scale pre-trained diffusion transformer models have made significant progress in video generation. While current DiT models can produce high-definition, high-frame-rate, and highly diverse videos, there is a lack of fine-grained control over the video content. Controlling the motion of subjects in videos using only prompts is challenging, especially when it comes to describing complex movements. Further, existing methods fail to control the motion in image-to-video generation, as the subject in the reference image often differs from the subject in the reference video in terms of initial position, size, and shape. To address this, we propose the Leveraging Motion Prior (LMP) framework for zero-shot video generation. Our framework harnesses the powerful generative capabilities of pre-trained diffusion transformers to enable motion in the generated videos to reference user-provided motion videos in both text-to-video and image-to-video generation. To this end, we first introduce a foreground-background disentangle module to distinguish between moving subjects and backgrounds in the reference video, preventing interference in the target video generation. A reweighted motion transfer module is designed to allow the target video to reference the motion from the reference video. To avoid interference from the subject in the reference video, we propose an appearance separation module to suppress the appearance of the reference subject in the target video. We annotate the DAVIS dataset with detailed prompts for our experiments and design evaluation metrics to validate the effectiveness of our method. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in generation quality, prompt-video consistency, and control capability. Our homepage is available at this https URL

Title: $α$-GAN by Rényi Cross Entropy

• Authors: Ni Ding, Miao Qiao, Jiaxing Xu, Yiping Ke, Xiaoyu Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14190
• Pdf URL: https://arxiv.org/pdf/2505.14190
• Copy Paste: [[2505.14190]] $α$-GAN by Rényi Cross Entropy(https://arxiv.org/abs/2505.14190)
• Keywords: generative
• Abstract: This paper proposes $\alpha$-GAN, a generative adversarial network using Rényi measures. The value function is formulated, by Rényi cross entropy, as an expected certainty measure incurred by the discriminator's soft decision as to where the sample is from, true population or the generator. The discriminator tries to maximize the Rényi certainty about sample source, while the generator wants to reduce it by injecting fake samples. This forms a min-max problem with the solution parameterized by the Rényi order $\alpha$. This $\alpha$-GAN reduces to vanilla GAN at $\alpha = 1$, where the value function is exactly the binary cross entropy. The optimization of $\alpha$-GAN is over probability (vector) space. It is shown that the gradient is exponentially enlarged when Rényi order is in the range $\alpha \in (0,1)$. This makes convergence faster, which is verified by experimental results. A discussion shows that choosing $\alpha \in (0,1)$ may be able to solve some common problems, e.g., vanishing gradient. A following observation reveals that this range has not been fully explored in the existing Rényi version GANs.

Title: Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment

• Authors: Yang Hu, Runchen Wang, Stephen Chong Zhao, Xuhui Zhan, Do Hun Kim, Mark Wallace, David A. Tovar
• Subjects: cs.CV, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2505.14204
• Pdf URL: https://arxiv.org/pdf/2505.14204
• Copy Paste: [[2505.14204]] Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment(https://arxiv.org/abs/2505.14204)
• Keywords: self-supervised
• Abstract: We introduce Perceptual-Initialization (PI), a paradigm shift in visual representation learning that incorporates human perceptual structure during the initialization phase rather than as a downstream fine-tuning step. By integrating human-derived triplet embeddings from the NIGHTS dataset to initialize a CLIP vision encoder, followed by self-supervised learning on YFCC15M, our approach demonstrates significant zero-shot performance improvements, without any task-specific fine-tuning, across 29 zero shot classification and 2 retrieval benchmarks. On ImageNet-1K, zero-shot gains emerge after approximately 15 epochs of pretraining. Benefits are observed across datasets of various scales, with improvements manifesting at different stages of the pretraining process depending on dataset characteristics. Our approach consistently enhances zero-shot top-1 accuracy, top-5 accuracy, and retrieval recall (e.g., R@1, R@5) across these diverse evaluation tasks, without requiring any adaptation to target domains. These findings challenge the conventional wisdom of using human-perceptual data primarily for fine-tuning and demonstrate that embedding human perceptual structure during early representation learning yields more capable and vision-language aligned systems that generalize immediately to unseen tasks. Our work shows that "beginning with you", starting with human perception, provides a stronger foundation for general-purpose vision-language intelligence.

Title: Challenges and Limitations in the Synthetic Generation of mHealth Sensor Data

• Authors: Flavio Di Martino, Franca Delmastro
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14206
• Pdf URL: https://arxiv.org/pdf/2505.14206
• Copy Paste: [[2505.14206]] Challenges and Limitations in the Synthetic Generation of mHealth Sensor Data(https://arxiv.org/abs/2505.14206)
• Keywords: diffusion, generative
• Abstract: The widespread adoption of mobile sensors has the potential to provide massive and heterogeneous time series data, driving Artificial Intelligence applications in mHealth. However, data collection remains limited due to stringent ethical regulations, privacy concerns, and other constraints, hindering progress in the field. Synthetic data generation, particularly through Generative Adversarial Networks and Diffusion Models, has emerged as a promising solution to address both data scarcity and privacy issues. Yet, these models are often limited to short-term, unimodal signal patterns. This paper presents a systematic evaluation of state-of-the-art generative models for time series synthesis, with a focus on their ability to jointly handle multi-modality, long-range dependencies, and conditional generation-key challenges in the mHealth domain. To ensure a fair comparison, we introduce a novel evaluation framework designed to measure both the intrinsic quality of synthetic data and its utility in downstream predictive tasks. Our findings reveal critical limitations in the existing approaches, particularly in maintaining cross-modal consistency, preserving temporal coherence, and ensuring robust performance in train-on-synthetic, test-on-real, and data augmentation scenarios. Finally, we present our future research directions to enhance synthetic time series generation and improve the applicability of generative models in mHealth.

Title: Mechanistic Fine-tuning for In-context Learning

• Authors: Hakaze Cho, Peng Luo, Mariko Kato, Rin Kaenbyou, Naoya Inoue
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14233
• Pdf URL: https://arxiv.org/pdf/2505.14233
• Copy Paste: [[2505.14233]] Mechanistic Fine-tuning for In-context Learning(https://arxiv.org/abs/2505.14233)
• Keywords: in-context
• Abstract: In-context Learning (ICL) utilizes structured demonstration-query inputs to induce few-shot learning on Language Models (LMs), which are not originally pre-trained on ICL-style data. To bridge the gap between ICL and pre-training, some approaches fine-tune LMs on large ICL-style datasets by an end-to-end paradigm with massive computational costs. To reduce such costs, in this paper, we propose Attention Behavior Fine-Tuning (ABFT), utilizing the previous findings on the inner mechanism of ICL, building training objectives on the attention scores instead of the final outputs, to force the attention scores to focus on the correct label tokens presented in the context and mitigate attention scores from the wrong label tokens. Our experiments on 9 modern LMs and 8 datasets empirically find that ABFT outperforms in performance, robustness, unbiasedness, and efficiency, with only around 0.01% data cost compared to the previous methods. Moreover, our subsequent analysis finds that the end-to-end training objective contains the ABFT objective, suggesting the implicit bias of ICL-style data to the emergence of induction heads. Our work demonstrates the possibility of controlling specific module sequences within LMs to improve their behavior, opening up the future application of mechanistic interpretability.

Title: Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization

• Authors: Yuanyuan Chang, Yinghua Yao, Tao Qin, Mengmeng Wang, Ivor Tsang, Guang Dai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14254
• Pdf URL: https://arxiv.org/pdf/2505.14254
• Copy Paste: [[2505.14254]] Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization(https://arxiv.org/abs/2505.14254)
• Keywords: diffusion
• Abstract: Text-to-image diffusion models have emerged as powerful tools for high-quality image generation and editing. Many existing approaches rely on text prompts as editing guidance. However, these methods are constrained by the need for manual prompt crafting, which can be time-consuming, introduce irrelevant details, and significantly limit editing performance. In this work, we propose optimizing semantic embeddings guided by attribute classifiers to steer text-to-image models toward desired edits, without relying on text prompts or requiring any training or fine-tuning of the diffusion model. We utilize classifiers to learn precise semantic embeddings at the dataset level. The learned embeddings are theoretically justified as the optimal representation of attribute semantics, enabling disentangled and accurate edits. Experiments further demonstrate that our method achieves high levels of disentanglement and strong generalization across different domains of data.

Title: Think-J: Learning to Think for Generative LLM-as-a-Judge

• Authors: Hui Huang, Yancheng He, Hongli Zhou, Rui Zhang, Wei Liu, Weixun Wang, Wenbo Su, Bo Zheng, Jiaheng Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14268
• Pdf URL: https://arxiv.org/pdf/2505.14268
• Copy Paste: [[2505.14268]] Think-J: Learning to Think for Generative LLM-as-a-Judge(https://arxiv.org/abs/2505.14268)
• Keywords: generative
• Abstract: LLM-as-a-Judge refers to the automatic modeling of preferences for responses generated by Large Language Models (LLMs), which is of significant importance for both LLM evaluation and reward modeling. Although generative LLMs have made substantial progress in various tasks, their performance as LLM-Judge still falls short of expectations. In this work, we propose Think-J, which improves generative LLM-as-a-Judge by learning how to think. We first utilized a small amount of curated data to develop the model with initial judgment thinking capabilities. Subsequently, we optimize the judgment thinking traces based on reinforcement learning (RL). We propose two methods for judgment thinking optimization, based on offline and online RL, respectively. The offline RL requires training a critic model to construct positive and negative examples for learning. The online method defines rule-based reward as feedback for optimization. Experimental results showed that our approach can significantly enhance the evaluation capability of generative LLM-Judge, surpassing both generative and classifier-based LLM-Judge without requiring extra human annotations.

Title: FAID: Fine-grained AI-generated Text Detection using Multi-task Auxiliary and Multi-level Contrastive Learning

• Authors: Minh Ngoc Ta, Dong Cao Van, Duc-Anh Hoang, Minh Le-Anh, Truong Nguyen, My Anh Tran Nguyen, Yuxia Wang, Preslav Nakov, Sang Dinh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14271
• Pdf URL: https://arxiv.org/pdf/2505.14271
• Copy Paste: [[2505.14271]] FAID: Fine-grained AI-generated Text Detection using Multi-task Auxiliary and Multi-level Contrastive Learning(https://arxiv.org/abs/2505.14271)
• Keywords: generative
• Abstract: The growing collaboration between humans and AI models in generative tasks has introduced new challenges in distinguishing between human-written, AI-generated, and human-AI collaborative texts. In this work, we collect a multilingual, multi-domain, multi-generator dataset FAIDSet. We further introduce a fine-grained detection framework FAID to classify text into these three categories, meanwhile identifying the underlying AI model family. Unlike existing binary classifiers, FAID is built to capture both authorship and model-specific characteristics. Our method combines multi-level contrastive learning with multi-task auxiliary classification to learn subtle stylistic cues. By modeling AI families as distinct stylistic entities, FAID offers improved interpretability. We incorporate an adaptation to address distributional shifts without retraining for unseen data. Experimental results demonstrate that FAID outperforms several baseline approaches, particularly enhancing the generalization accuracy on unseen domains and new AI models. It provide a potential solution for improving transparency and accountability in AI-assisted writing.

Title: Towards Generating Realistic Underwater Images

• Authors: Abdul-Kazeem Shamba
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.14296
• Pdf URL: https://arxiv.org/pdf/2505.14296
• Copy Paste: [[2505.14296]] Towards Generating Realistic Underwater Images(https://arxiv.org/abs/2505.14296)
• Keywords: generative
• Abstract: This paper explores the use of contrastive learning and generative adversarial networks for generating realistic underwater images from synthetic images with uniform lighting. We investigate the performance of image translation models for generating realistic underwater images using the VAROS dataset. Two key evaluation metrics, Fréchet Inception Distance (FID) and Structural Similarity Index Measure (SSIM), provide insights into the trade-offs between perceptual quality and structural preservation. For paired image translation, pix2pix achieves the best FID scores due to its paired supervision and PatchGAN discriminator, while the autoencoder model attains the highest SSIM, suggesting better structural fidelity despite producing blurrier outputs. Among unpaired methods, CycleGAN achieves a competitive FID score by leveraging cycle-consistency loss, whereas CUT, which replaces cycle-consistency with contrastive learning, attains higher SSIM, indicating improved spatial similarity retention. Notably, incorporating depth information into CUT results in the lowest overall FID score, demonstrating that depth cues enhance realism. However, the slight decrease in SSIM suggests that depth-aware learning may introduce structural variations.

Title: A MIND for Reasoning: Meta-learning for In-context Deduction

• Authors: Leonardo Bertolazzi, Manuel Vargas Guzmán, Raffaella Bernardi, Maciej Malicki, Jakub Szymanik
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14313
• Pdf URL: https://arxiv.org/pdf/2505.14313
• Copy Paste: [[2505.14313]] A MIND for Reasoning: Meta-learning for In-context Deduction(https://arxiv.org/abs/2505.14313)
• Keywords: in-context
• Abstract: Large language models (LLMs) are increasingly evaluated on formal tasks, where strong reasoning abilities define the state of the art. However, their ability to generalize to out-of-distribution problems remains limited. In this paper, we investigate how LLMs can achieve a systematic understanding of deductive rules. Our focus is on the task of identifying the appropriate subset of premises within a knowledge base needed to derive a given hypothesis. To tackle this challenge, we propose Meta-learning for In-context Deduction (MIND), a novel few-shot meta-learning fine-tuning approach. The goal of MIND is to enable models to generalize more effectively to unseen knowledge bases and to systematically apply inference rules. Our results show that MIND significantly improves generalization in small LMs ranging from 1.5B to 7B parameters. The benefits are especially pronounced in smaller models and low-data settings. Remarkably, small models fine-tuned with MIND outperform state-of-the-art LLMs, such as GPT-4o and o3-mini, on this task.

Title: Handloom Design Generation Using Generative Networks

• Authors: Rajat Kanti Bhattacharjee, Meghali Nandi, Amrit Jha, Gunajit Kalita, Ferdous Ahmed Barbhuiya
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14330
• Pdf URL: https://arxiv.org/pdf/2505.14330
• Copy Paste: [[2505.14330]] Handloom Design Generation Using Generative Networks(https://arxiv.org/abs/2505.14330)
• Keywords: generative
• Abstract: This paper proposes deep learning techniques of generating designs for clothing, focused on handloom fabric and discusses the associated challenges along with its application. The capability of generative neural network models in understanding artistic designs and synthesizing those is not yet explored well. In this work, multiple methods are employed incorporating the current state of the art generative models and style transfer algorithms to study and observe their performance for the task. The results are then evaluated through user score. This work also provides a new dataset NeuralLoom for the task of the design generation.

Title: QA-prompting: Improving Summarization with Large Language Models using Question-Answering

• Authors: Neelabh Sinha
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14347
• Pdf URL: https://arxiv.org/pdf/2505.14347
• Copy Paste: [[2505.14347]] QA-prompting: Improving Summarization with Large Language Models using Question-Answering(https://arxiv.org/abs/2505.14347)
• Keywords: in-context
• Abstract: Language Models (LMs) have revolutionized natural language processing, enabling high-quality text generation through prompting and in-context learning. However, models often struggle with long-context summarization due to positional biases, leading to suboptimal extraction of critical information. There are techniques to improve this with fine-tuning, pipelining, or using complex techniques, which have their own challenges. To solve these challenges, we propose QA-prompting - a simple prompting method for summarization that utilizes question-answering as an intermediate step prior to summary generation. Our method extracts key information and enriches the context of text to mitigate positional biases and improve summarization in a single LM call per task without requiring fine-tuning or pipelining. Experiments on multiple datasets belonging to different domains using ten state-of-the-art pre-trained models demonstrate that QA-prompting outperforms baseline and other state-of-the-art methods, achieving up to 29% improvement in ROUGE scores. This provides an effective and scalable solution for summarization and highlights the importance of domain-specific question selection for optimal performance.

Title: Vid2World: Crafting Video Diffusion Models to Interactive World Models

• Authors: Siqiao Huang, Jialong Wu, Qixing Zhou, Shangchen Miao, Mingsheng Long
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14357
• Pdf URL: https://arxiv.org/pdf/2505.14357
• Copy Paste: [[2505.14357]] Vid2World: Crafting Video Diffusion Models to Interactive World Models(https://arxiv.org/abs/2505.14357)
• Keywords: diffusion
• Abstract: World models, which predict transitions based on history observation and action sequences, have shown great promise in improving data efficiency for sequential decision making. However, existing world models often require extensive domain-specific training and still produce low-fidelity, coarse predictions, limiting their applicability in complex environments. In contrast, video diffusion models trained on large, internet-scale datasets have demonstrated impressive capabilities in generating high-quality videos that capture diverse real-world dynamics. In this work, we present Vid2World, a general approach for leveraging and transferring pre-trained video diffusion models into interactive world models. To bridge the gap, Vid2World performs casualization of a pre-trained video diffusion model by crafting its architecture and training objective to enable autoregressive generation. Furthermore, it introduces a causal action guidance mechanism to enhance action controllability in the resulting interactive world model. Extensive experiments in robot manipulation and game simulation domains show that our method offers a scalable and effective approach for repurposing highly capable video diffusion models to interactive world models.

Title: Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable

• Authors: Ruoxin Chen, Junwei Xi, Zhiyuan Yan, Ke-Yue Zhang, Shuang Wu, Jingyi Xie, Xu Chen, Lei Xu, Isabel Guan, Taiping Yao, Shouhong Ding
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14359
• Pdf URL: https://arxiv.org/pdf/2505.14359
• Copy Paste: [[2505.14359]] Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable(https://arxiv.org/abs/2505.14359)
• Keywords: generative
• Abstract: Existing detectors are often trained on biased datasets, leading to the possibility of overfitting on non-causal image attributes that are spuriously correlated with real/synthetic labels. While these biased features enhance performance on the training data, they result in substantial performance degradation when applied to unbiased datasets. One common solution is to perform dataset alignment through generative reconstruction, matching the semantic content between real and synthetic images. However, we revisit this approach and show that pixel-level alignment alone is insufficient. The reconstructed images still suffer from frequency-level misalignment, which can perpetuate spurious correlations. To illustrate, we observe that reconstruction models tend to restore the high-frequency details lost in real images (possibly due to JPEG compression), inadvertently creating a frequency-level misalignment, where synthetic images appear to have richer high-frequency content than real ones. This misalignment leads to models associating high-frequency features with synthetic labels, further reinforcing biased cues. To resolve this, we propose Dual Data Alignment (DDA), which aligns both the pixel and frequency domains. Moreover, we introduce two new test sets: DDA-COCO, containing DDA-aligned synthetic images for testing detector performance on the most aligned dataset, and EvalGEN, featuring the latest generative models for assessing detectors under new generative architectures such as visual auto-regressive generators. Finally, our extensive evaluations demonstrate that a detector trained exclusively on DDA-aligned MSCOCO could improve across 8 diverse benchmarks by a non-trivial margin, showing a +7.2% on in-the-wild benchmarks, highlighting the improved generalizability of unbiased detectors.

Title: Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives

• Authors: Xingxing Weng, Chao Pang, Gui-Song Xia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14361
• Pdf URL: https://arxiv.org/pdf/2505.14361
• Copy Paste: [[2505.14361]] Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives(https://arxiv.org/abs/2505.14361)
• Keywords: foundation model, generative
• Abstract: Vision-language modeling (VLM) aims to bridge the information gap between images and natural language. Under the new paradigm of first pre-training on massive image-text pairs and then fine-tuning on task-specific data, VLM in the remote sensing domain has made significant progress. The resulting models benefit from the absorption of extensive general knowledge and demonstrate strong performance across a variety of remote sensing data analysis tasks. Moreover, they are capable of interacting with users in a conversational manner. In this paper, we aim to provide the remote sensing community with a timely and comprehensive review of the developments in VLM using the two-stage paradigm. Specifically, we first cover a taxonomy of VLM in remote sensing: contrastive learning, visual instruction tuning, and text-conditioned image generation. For each category, we detail the commonly used network architecture and pre-training objectives. Second, we conduct a thorough review of existing works, examining foundation models and task-specific adaptation methods in contrastive-based VLM, architectural upgrades, training strategies and model capabilities in instruction-based VLM, as well as generative foundation models with their representative downstream applications. Third, we summarize datasets used for VLM pre-training, fine-tuning, and evaluation, with an analysis of their construction methodologies (including image sources and caption generation) and key properties, such as scale and task adaptability. Finally, we conclude this survey with insights and discussions on future research directions: cross-modal representation alignment, vague requirement comprehension, explanation-driven model reliability, continually scalable model capabilities, and large-scale datasets featuring richer modalities and greater challenges.

Title: Byte Pair Encoding for Efficient Time Series Forecasting

• Authors: Leon Götz, Marcel Kollovieh, Stephan Günnemann, Leo Schwinn
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14411
• Pdf URL: https://arxiv.org/pdf/2505.14411
• Copy Paste: [[2505.14411]] Byte Pair Encoding for Efficient Time Series Forecasting(https://arxiv.org/abs/2505.14411)
• Keywords: foundation model
• Abstract: Existing time series tokenization methods predominantly encode a constant number of samples into individual tokens. This inflexible approach can generate excessive tokens for even simple patterns like extended constant values, resulting in substantial computational overhead. Inspired by the success of byte pair encoding, we propose the first pattern-centric tokenization scheme for time series analysis. Based on a discrete vocabulary of frequent motifs, our method merges samples with underlying patterns into tokens, compressing time series adaptively. Exploiting our finite set of motifs and the continuous properties of time series, we further introduce conditional decoding as a lightweight yet powerful post-hoc optimization method, which requires no gradient computation and adds no computational overhead. On recent time series foundation models, our motif-based tokenization improves forecasting performance by 36% and boosts efficiency by 1990% on average. Conditional decoding further reduces MSE by up to 44%. In an extensive analysis, we demonstrate the adaptiveness of our tokenization to diverse temporal patterns, its generalization to unseen data, and its meaningful token representations capturing distinct time series properties, including statistical moments and trends.

Title: Diving into the Fusion of Monocular Priors for Generalized Stereo Matching

• Authors: Chengtang Yao, Lidong Yu, Zhidan Liu, Jiaxi Zeng, Yuwei Wu, Yunde Jia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14414
• Pdf URL: https://arxiv.org/pdf/2505.14414
• Copy Paste: [[2505.14414]] Diving into the Fusion of Monocular Priors for Generalized Stereo Matching(https://arxiv.org/abs/2505.14414)
• Keywords: foundation model
• Abstract: The matching formulation makes it naturally hard for the stereo matching to handle ill-posed regions like occlusions and non-Lambertian surfaces. Fusing monocular priors has been proven helpful for ill-posed matching, but the biased monocular prior learned from small stereo datasets constrains the generalization. Recently, stereo matching has progressed by leveraging the unbiased monocular prior from the vision foundation model (VFM) to improve the generalization in ill-posed regions. We dive into the fusion process and observe three main problems limiting the fusion of the VFM monocular prior. The first problem is the misalignment between affine-invariant relative monocular depth and absolute depth of disparity. Besides, when we use the monocular feature in an iterative update structure, the over-confidence in the disparity update leads to local optima results. A direct fusion of a monocular depth map could alleviate the local optima problem, but noisy disparity results computed at the first several iterations will misguide the fusion. In this paper, we propose a binary local ordering map to guide the fusion, which converts the depth map into a binary relative format, unifying the relative and absolute depth representation. The computed local ordering map is also used to re-weight the initial disparity update, resolving the local optima and noisy problem. In addition, we formulate the final direct fusion of monocular depth to the disparity as a registration problem, where a pixel-wise linear regression module can globally and adaptively align them. Our method fully exploits the monocular prior to support stereo matching results effectively and efficiently. We significantly improve the performance from the experiments when generalizing from SceneFlow to Middlebury and Booster datasets while barely reducing the efficiency.

Title: Table Foundation Models: on knowledge pre-training for tabular learning

• Authors: Myung Jun Kim, Félix Lefebvre, Gaëtan Brison, Alexandre Perez-Lebel, Gaël Varoquaux
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14415
• Pdf URL: https://arxiv.org/pdf/2505.14415
• Copy Paste: [[2505.14415]] Table Foundation Models: on knowledge pre-training for tabular learning(https://arxiv.org/abs/2505.14415)
• Keywords: foundation model
• Abstract: Table foundation models bring high hopes to data science: pre-trained on tabular data to embark knowledge or priors, they should facilitate downstream tasks on tables. One specific challenge is that of data semantics: numerical entries take their meaning from context, e.g., column name. Pre-trained neural networks that jointly model column names and table entries have recently boosted prediction accuracy. While these models outline the promises of world knowledge to interpret table values, they lack the convenience of popular foundation models in text or vision. Indeed, they must be fine-tuned to bring benefits, come with sizeable computation costs, and cannot easily be reused or combined with other architectures. Here we introduce TARTE, a foundation model that transforms tables to knowledge-enhanced vector representations using the string to capture semantics. Pre-trained on large relational data, TARTE yields representations that facilitate subsequent learning with little additional cost. These representations can be fine-tuned or combined with other learners, giving models that push the state-of-the-art prediction performance and improve the prediction/computation performance trade-off. Specialized to a task or a domain, TARTE gives domain-specific representations that facilitate further learning. Our study demonstrates an effective approach to knowledge pre-training for tabular learning.

Title: RefiDiff: Refinement-Aware Diffusion for Efficient Missing Data Imputation

• Authors: Md Atik Ahamed, Qiang Ye, Qiang Cheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14451
• Pdf URL: https://arxiv.org/pdf/2505.14451
• Copy Paste: [[2505.14451]] RefiDiff: Refinement-Aware Diffusion for Efficient Missing Data Imputation(https://arxiv.org/abs/2505.14451)
• Keywords: diffusion
• Abstract: Missing values in high-dimensional, mixed-type datasets pose significant challenges for data imputation, particularly under Missing Not At Random (MNAR) mechanisms. Existing methods struggle to integrate local and global data characteristics, limiting performance in MNAR and high-dimensional settings. We propose an innovative framework, RefiDiff, combining local machine learning predictions with a novel Mamba-based denoising network capturing interrelationships among distant features and samples. Our approach leverages pre-refinement for initial warm-up imputations and post-refinement to polish results, enhancing stability and accuracy. By encoding mixed-type data into unified tokens, RefiDiff enables robust imputation without architectural or hyperparameter tuning. RefiDiff outperforms state-of-the-art (SOTA) methods across missing-value settings, excelling in MNAR with a 4x faster training time than SOTA DDPM-based approaches. Extensive evaluations on nine real-world datasets demonstrate its robustness, scalability, and effectiveness in handling complex missingness patterns.

Title: CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation

• Authors: Chihan Huang, Hao Tang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14455
• Pdf URL: https://arxiv.org/pdf/2505.14455
• Copy Paste: [[2505.14455]] CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation(https://arxiv.org/abs/2505.14455)
• Keywords: diffusion
• Abstract: Although autoregressive models have dominated language modeling in recent years, there has been a growing interest in exploring alternative paradigms to the conventional next-token prediction framework. Diffusion-based language models have emerged as a compelling alternative due to their powerful parallel generation capabilities and inherent editability. However, these models are often constrained by fixed-length generation. A promising direction is to combine the strengths of both paradigms, segmenting sequences into blocks, modeling autoregressive dependencies across blocks while leveraging discrete diffusion to estimate the conditional distribution within each block given the preceding context. Nevertheless, their practical application is often hindered by two key limitations: rigid fixed-length outputs and a lack of flexible control mechanisms. In this work, we address the critical limitations of fixed granularity and weak controllability in current large diffusion language models. We propose CtrlDiff, a dynamic and controllable semi-autoregressive framework that adaptively determines the size of each generation block based on local semantics using reinforcement learning. Furthermore, we introduce a classifier-guided control mechanism tailored to discrete diffusion, which significantly reduces computational overhead while facilitating efficient post-hoc conditioning without retraining. Extensive experiments demonstrate that CtrlDiff sets a new standard among hybrid diffusion models, narrows the performance gap to state-of-the-art autoregressive approaches, and enables effective conditional text generation across diverse tasks.

Title: Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

• Authors: Tong Li, Jiachuan Wang, Yongqi Zhang, Shuangyin Li, Lei Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14471
• Pdf URL: https://arxiv.org/pdf/2505.14471
• Copy Paste: [[2505.14471]] Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning(https://arxiv.org/abs/2505.14471)
• Keywords: self-supervised
• Abstract: Citation classification, which identifies the intention behind academic citations, is pivotal for scholarly analysis. Previous works suggest fine-tuning pretrained language models (PLMs) on citation classification datasets, reaping the reward of the linguistic knowledge they gained during pretraining. However, directly fine-tuning for citation classification is challenging due to labeled data scarcity, contextual noise, and spurious keyphrase correlations. In this paper, we present a novel framework, Citss, that adapts the PLMs to overcome these challenges. Citss introduces self-supervised contrastive learning to alleviate data scarcity, and is equipped with two specialized strategies to obtain the contrastive pairs: sentence-level cropping, which enhances focus on target citations within long contexts, and keyphrase perturbation, which mitigates reliance on specific keyphrases. Compared with previous works that are only designed for encoder-based PLMs, Citss is carefully developed to be compatible with both encoder-based PLMs and decoder-based LLMs, to embrace the benefits of enlarged pretraining. Experiments with three benchmark datasets with both encoder-based PLMs and decoder-based LLMs demonstrate our superiority compared to the previous state of the art. Our code is available at: this http URL

Title: Enhancing Interpretability of Sparse Latent Representations with Class Information

• Authors: Farshad Sangari Abiz, Reshad Hosseini, Babak N. Araabi
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14476
• Pdf URL: https://arxiv.org/pdf/2505.14476
• Copy Paste: [[2505.14476]] Enhancing Interpretability of Sparse Latent Representations with Class Information(https://arxiv.org/abs/2505.14476)
• Keywords: generative
• Abstract: Variational Autoencoders (VAEs) are powerful generative models for learning latent representations. Standard VAEs generate dispersed and unstructured latent spaces by utilizing all dimensions, which limits their interpretability, especially in high-dimensional spaces. To address this challenge, Variational Sparse Coding (VSC) introduces a spike-and-slab prior distribution, resulting in sparse latent representations for each input. These sparse representations, characterized by a limited number of active dimensions, are inherently more interpretable. Despite this advantage, VSC falls short in providing structured interpretations across samples within the same class. Intuitively, samples from the same class are expected to share similar attributes while allowing for variations in those attributes. This expectation should manifest as consistent patterns of active dimensions in their latent representations, but VSC does not enforce such consistency. In this paper, we propose a novel approach to enhance the latent space interpretability by ensuring that the active dimensions in the latent space are consistent across samples within the same class. To achieve this, we introduce a new loss function that encourages samples from the same class to share similar active dimensions. This alignment creates a more structured and interpretable latent space, where each shared dimension corresponds to a high-level concept, or "factor." Unlike existing disentanglement-based methods that primarily focus on global factors shared across all classes, our method captures both global and class-specific factors, thereby enhancing the utility and interpretability of latent representations.

Title: Learning to Integrate Diffusion ODEs by Averaging the Derivatives

• Authors: Wenze Liu, Xiangyu Yue
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14502
• Pdf URL: https://arxiv.org/pdf/2505.14502
• Copy Paste: [[2505.14502]] Learning to Integrate Diffusion ODEs by Averaging the Derivatives(https://arxiv.org/abs/2505.14502)
• Keywords: diffusion
• Abstract: To accelerate diffusion model inference, numerical solvers perform poorly at extremely small steps, while distillation techniques often introduce complexity and instability. This work presents an intermediate strategy, balancing performance and cost, by learning ODE integration using loss functions derived from the derivative-integral relationship, inspired by Monte Carlo integration and Picard iteration. From a geometric perspective, the losses operate by gradually extending the tangent to the secant, thus are named as secant losses. The secant losses can rapidly convert (via fine-tuning or distillation) a pretrained diffusion model into its secant version. In our experiments, the secant version of EDM achieves a $10$-step FID of $2.14$ on CIFAR-10, while the secant version of SiT-XL/2 attains a $4$-step FID of $2.27$ and an $8$-step FID of $1.96$ on ImageNet-$256\times256$. Code will be available.

Title: Latent Flow Transformer

• Authors: Yen-Chen Wu, Feng-Ting Liao, Meng-Hsi Chen, Pei-Chen Ho, Farhang Nabiei, Da-shan Shiu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.14513
• Pdf URL: https://arxiv.org/pdf/2505.14513
• Copy Paste: [[2505.14513]] Latent Flow Transformer(https://arxiv.org/abs/2505.14513)
• Keywords: diffusion
• Abstract: Transformers, the standard implementation for large language models (LLMs), typically consist of tens to hundreds of discrete layers. While more layers can lead to better performance, this approach has been challenged as far from efficient, especially given the superiority of continuous layers demonstrated by diffusion and flow-based models for image generation. We propose the Latent Flow Transformer (LFT), which replaces a block of layers with a single learned transport operator trained via flow matching, offering significant compression while maintaining compatibility with the original architecture. Additionally, we address the limitations of existing flow-based methods in \textit{preserving coupling} by introducing the Flow Walking (FW) algorithm. On the Pythia-410M model, LFT trained with flow matching compresses 6 of 24 layers and outperforms directly skipping 2 layers (KL Divergence of LM logits at 0.407 vs. 0.529), demonstrating the feasibility of this design. When trained with FW, LFT further distills 12 layers into one while reducing the KL to 0.736 surpassing that from skipping 3 layers (0.932), significantly narrowing the gap between autoregressive and flow-based generation paradigms.

Title: SparC: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling

• Authors: Zhihao Li, Yufei Wang, Heliang Zheng, Yihao Luo, Bihan Wen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14521
• Pdf URL: https://arxiv.org/pdf/2505.14521
• Copy Paste: [[2505.14521]] SparC: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling(https://arxiv.org/abs/2505.14521)
• Keywords: diffusion, generative
• Abstract: High-fidelity 3D object synthesis remains significantly more challenging than 2D image generation due to the unstructured nature of mesh data and the cubic complexity of dense volumetric grids. Existing two-stage pipelines-compressing meshes with a VAE (using either 2D or 3D supervision), followed by latent diffusion sampling-often suffer from severe detail loss caused by inefficient representations and modality mismatches introduced in VAE. We introduce SparC, a unified framework that combines a sparse deformable marching cubes representation SparseCubes with a novel encoder SparConv-VAE. SparseCubes converts raw meshes into high-resolution ($1024^3$) surfaces with arbitrary topology by scattering signed distance and deformation fields onto a sparse cube, allowing differentiable optimization. SparConv-VAE is the first modality-consistent variational autoencoder built entirely upon sparse convolutional networks, enabling efficient and near-lossless 3D reconstruction suitable for high-resolution generative modeling through latent diffusion. SparC achieves state-of-the-art reconstruction fidelity on challenging inputs, including open surfaces, disconnected components, and intricate geometry. It preserves fine-grained shape details, reduces training and inference cost, and integrates naturally with latent diffusion models for scalable, high-resolution 3D generation.

Title: diffDemorph: Extending Reference-Free Demorphing to Unseen Faces

• Authors: Nitish Shukla, Arun Ross
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14527
• Pdf URL: https://arxiv.org/pdf/2505.14527
• Copy Paste: [[2505.14527]] diffDemorph: Extending Reference-Free Demorphing to Unseen Faces(https://arxiv.org/abs/2505.14527)
• Keywords: diffusion
• Abstract: A face morph is created by combining two (or more) face images corresponding to two (or more) identities to produce a composite that successfully matches the constituent identities. Reference-free (RF) demorphing reverses this process using only the morph image, without the need for additional reference images. Previous RF demorphing methods were overly constrained, as they rely on assumptions about the distributions of training and testing morphs such as the morphing technique used, face style, and images used to create the morph. In this paper, we introduce a novel diffusion-based approach that effectively disentangles component images from a composite morph image with high visual fidelity. Our method is the first to generalize across morph techniques and face styles, beating the current state of the art by $\geq 59.46\%$ under a common training protocol across all datasets tested. We train our method on morphs created using synthetically generated face images and test on real morphs, thereby enhancing the practicality of the technique. Experiments on six datasets and two face matchers establish the utility and efficacy of our method.

Title: Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions

• Authors: Utsav Dutta, Sina Khoshfetrat Pakazad, Henrik Ohlsson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.14543
• Pdf URL: https://arxiv.org/pdf/2505.14543
• Copy Paste: [[2505.14543]] Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions(https://arxiv.org/abs/2505.14543)
• Keywords: foundation model
• Abstract: Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering. While Transformer-based architectures have improved scalability, foundation models, commonplace in text, vision, and audio, remain under-explored for time series and are largely restricted to forecasting. We introduce $\textbf{CHARM}$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations. To address the unique difficulties of time series foundation learning, $\textbf{CHARM}$ incorporates architectural innovations that integrate channel-level textual descriptions while remaining invariant to channel order. The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability. Our $7$M-parameter model achieves state-of-the-art performance across diverse downstream tasks, setting a new benchmark for time series representation learning.

Title: Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI

• Authors: Marlène Careil, Yohann Benchetrit, Jean-Rémi King
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14556
• Pdf URL: https://arxiv.org/pdf/2505.14556
• Copy Paste: [[2505.14556]] Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI(https://arxiv.org/abs/2505.14556)
• Keywords: diffusion, generative
• Abstract: Brain-to-image decoding has been recently propelled by the progress in generative AI models and the availability of large ultra-high field functional Magnetic Resonance Imaging (fMRI). However, current approaches depend on complicated multi-stage pipelines and preprocessing steps that typically collapse the temporal dimension of brain recordings, thereby limiting time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural Activity Diffusion for Image Reconstruction), a new single-stage diffusion model designed for reconstructing images from dynamically evolving fMRI recordings. Our approach offers three main contributions. First, Dynadiff simplifies training as compared to existing approaches. Second, our model outperforms state-of-the-art models on time-resolved fMRI signals, especially on high-level semantic image reconstruction metrics, while remaining competitive on preprocessed fMRI data that collapse time. Third, this approach allows a precise characterization of the evolution of image representations in brain activity. Overall, this work lays the foundation for time-resolved brain-to-image decoding.

Title: Electrostatics from Laplacian Eigenbasis for Neural Network Interatomic Potentials

• Authors: Maksim Zhdanov, Vladislav Kurenkov
• Subjects: cs.LG, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2505.14606
• Pdf URL: https://arxiv.org/pdf/2505.14606
• Copy Paste: [[2505.14606]] Electrostatics from Laplacian Eigenbasis for Neural Network Interatomic Potentials(https://arxiv.org/abs/2505.14606)
• Keywords: self-supervised
• Abstract: Recent advances in neural network interatomic potentials have emerged as a promising research direction. However, popular deep learning models often lack auxiliary constraints grounded in physical laws, which could accelerate training and improve fidelity through physics-based regularization. In this work, we introduce $\Phi$-Module, a universal plugin module that enforces Poisson's equation within the message-passing framework to learn electrostatic interactions in a self-supervised manner. Specifically, each atom-wise representation is encouraged to satisfy a discretized Poisson's equation, making it possible to acquire a potential $\boldsymbol{\phi}$ and a corresponding charge density $\boldsymbol{\rho}$ linked to the learnable Laplacian eigenbasis coefficients of a given molecular graph. We then derive an electrostatic energy term, crucial for improved total energy predictions. This approach integrates seamlessly into any existing neural potential with insignificant computational overhead. Experiments on the OE62 and MD22 benchmarks confirm that models combined with $\Phi$-Module achieve robust improvements over baseline counterparts. For OE62 error reduction ranges from 4.5\% to 17.8\%, and for MD22, baseline equipped with $\Phi$-Module achieves best results on 5 out of 14 cases. Our results underscore how embedding a first-principles constraint in neural interatomic potentials can significantly improve performance while remaining hyperparameter-friendly, memory-efficient and lightweight in training. Code will be available at \href{this https URL}{dunnolab/phi-module}.

Title: General-Reasoner: Advancing LLM Reasoning Across All Domains

• Authors: Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, Wenhu Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.14652
• Pdf URL: https://arxiv.org/pdf/2505.14652
• Copy Paste: [[2505.14652]] General-Reasoner: Advancing LLM Reasoning Across All Domains(https://arxiv.org/abs/2505.14652)
• Keywords: generative
• Abstract: Reinforcement learning (RL) has recently demonstrated strong potential in enhancing the reasoning capabilities of large language models (LLMs). Particularly, the "Zero" reinforcement learning introduced by Deepseek-R1-Zero, enables direct RL training of base LLMs without relying on an intermediate supervised fine-tuning stage. Despite these advancements, current works for LLM reasoning mainly focus on mathematical and coding domains, largely due to data abundance and the ease of answer verification. This limits the applicability and generalization of such models to broader domains, where questions often have diverse answer representations, and data is more scarce. In this paper, we propose General-Reasoner, a novel training paradigm designed to enhance LLM reasoning capabilities across diverse domains. Our key contributions include: (1) constructing a large-scale, high-quality dataset of questions with verifiable answers curated by web crawling, covering a wide range of disciplines; and (2) developing a generative model-based answer verifier, which replaces traditional rule-based verification with the capability of chain-of-thought and context-awareness. We train a series of models and evaluate them on a wide range of datasets covering wide domains like physics, chemistry, finance, electronics etc. Our comprehensive evaluation across these 12 benchmarks (e.g. MMLU-Pro, GPQA, SuperGPQA, TheoremQA, BBEH and MATH AMC) demonstrates that General-Reasoner outperforms existing baseline methods, achieving robust and generalizable reasoning performance while maintaining superior effectiveness in mathematical reasoning tasks.

Title: EmoGist: Efficient In-Context Learning for Visual Emotion Understanding

• Authors: Ronald Seoh, Dan Goldwasser
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14660
• Pdf URL: https://arxiv.org/pdf/2505.14660
• Copy Paste: [[2505.14660]] EmoGist: Efficient In-Context Learning for Visual Emotion Understanding(https://arxiv.org/abs/2505.14660)
• Keywords: in-context
• Abstract: In this paper, we introduce EmoGist, a training-free, in-context learning method for performing visual emotion classification with LVLMs. The key intuition of our approach is that context-dependent definition of emotion labels could allow more accurate predictions of emotions, as the ways in which emotions manifest within images are highly context dependent and nuanced. EmoGist pre-generates multiple explanations of emotion labels, by analyzing the clusters of example images belonging to each category. At test time, we retrieve a version of explanation based on embedding similarity, and feed it to a fast VLM for classification. Through our experiments, we show that EmoGist allows up to 13 points improvement in micro F1 scores with the multi-label Memotion dataset, and up to 8 points in macro F1 in the multi-class FI dataset.

Title: Training-Free Watermarking for Autoregressive Image Generation

• Authors: Yu Tong, Zihao Pan, Shuai Yang, Kaiyang Zhou
• Subjects: cs.CV, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.14673
• Pdf URL: https://arxiv.org/pdf/2505.14673
• Copy Paste: [[2505.14673]] Training-Free Watermarking for Autoregressive Image Generation(https://arxiv.org/abs/2505.14673)
• Keywords: diffusion, generative
• Abstract: Invisible image watermarking can protect image ownership and prevent malicious misuse of visual generative models. However, existing generative watermarking methods are mainly designed for diffusion models while watermarking for autoregressive image generation models remains largely underexplored. We propose IndexMark, a training-free watermarking framework for autoregressive image generation models. IndexMark is inspired by the redundancy property of the codebook: replacing autoregressively generated indices with similar indices produces negligible visual differences. The core component in IndexMark is a simple yet effective match-then-replace method, which carefully selects watermark tokens from the codebook based on token similarity, and promotes the use of watermark tokens through token replacement, thereby embedding the watermark without affecting the image quality. Watermark verification is achieved by calculating the proportion of watermark tokens in generated images, with precision further improved by an Index Encoder. Furthermore, we introduce an auxiliary validation scheme to enhance robustness against cropping attacks. Experiments demonstrate that IndexMark achieves state-of-the-art performance in terms of image quality and verification accuracy, and exhibits robustness against various perturbations, including cropping, noises, Gaussian blur, random erasing, color jittering, and JPEG compression.

Title: Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers

• Authors: Sucheng Ren, Qihang Yu, Ju He, Alan Yuille, Liang-Chieh Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.14687
• Pdf URL: https://arxiv.org/pdf/2505.14687
• Copy Paste: [[2505.14687]] Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers(https://arxiv.org/abs/2505.14687)
• Keywords: diffusion, generative
• Abstract: Diffusion-based Transformers have demonstrated impressive generative capabilities, but their high computational costs hinder practical deployment, for example, generating an $8192\times 8192$ image can take over an hour on an A100 GPU. In this work, we propose GRAT (\textbf{GR}ouping first, \textbf{AT}tending smartly), a training-free attention acceleration strategy for fast image and video generation without compromising output quality. The key insight is to exploit the inherent sparsity in learned attention maps (which tend to be locally focused) in pretrained Diffusion Transformers and leverage better GPU parallelism. Specifically, GRAT first partitions contiguous tokens into non-overlapping groups, aligning both with GPU execution patterns and the local attention structures learned in pretrained generative Transformers. It then accelerates attention by having all query tokens within the same group share a common set of attendable key and value tokens. These key and value tokens are further restricted to structured regions, such as surrounding blocks or criss-cross regions, significantly reducing computational overhead (e.g., attaining a \textbf{35.8$\times$} speedup over full attention when generating $8192\times 8192$ images) while preserving essential attention patterns and long-range context. We validate GRAT on pretrained Flux and HunyuanVideo for image and video generation, respectively. In both cases, GRAT achieves substantially faster inference without any fine-tuning, while maintaining the performance of full attention. We hope GRAT will inspire future research on accelerating Diffusion Transformers for scalable visual generation.