2025-02-20

Title: SmartLLM: Smart Contract Auditing using Custom Generative AI

• Authors: Jun Kevin, Pujianto Yugopuspito
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13167
• Pdf URL: https://arxiv.org/pdf/2502.13167
• Copy Paste: [[2502.13167]] SmartLLM: Smart Contract Auditing using Custom Generative AI(https://arxiv.org/abs/2502.13167)
• Keywords: generative
• Abstract: Smart contracts are essential to decentralized finance (DeFi) and blockchain ecosystems but are increasingly vulnerable to exploits due to coding errors and complex attack vectors. Traditional static analysis tools and existing vulnerability detection methods often fail to address these challenges comprehensively, leading to high false-positive rates and an inability to detect dynamic vulnerabilities. This paper introduces SmartLLM, a novel approach leveraging fine-tuned LLaMA 3.1 models with Retrieval-Augmented Generation (RAG) to enhance the accuracy and efficiency of smart contract auditing. By integrating domain-specific knowledge from ERC standards and employing advanced techniques such as QLoRA for efficient fine-tuning, SmartLLM achieves superior performance compared to static analysis tools like Mythril and Slither, as well as zero-shot large language model (LLM) prompting methods such as GPT-3.5 and GPT-4. Experimental results demonstrate a perfect recall of 100% and an accuracy score of 70%, highlighting the model's robustness in identifying vulnerabilities, including reentrancy and access control issues. This research advances smart contract security by offering a scalable and effective auditing solution, supporting the secure adoption of decentralized applications.

Title: Web Phishing Net (WPN): A scalable machine learning approach for real-time phishing campaign detection

• Authors: Muhammad Fahad Zia, Sri Harish Kalidass
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13171
• Pdf URL: https://arxiv.org/pdf/2502.13171
• Copy Paste: [[2502.13171]] Web Phishing Net (WPN): A scalable machine learning approach for real-time phishing campaign detection(https://arxiv.org/abs/2502.13171)
• Keywords: generative
• Abstract: Phishing is the most prevalent type of cyber-attack today and is recognized as the leading source of data breaches with significant consequences for both individuals and corporations. Web-based phishing attacks are the most frequent with vectors such as social media posts and emails containing links to phishing URLs that once clicked on render host systems vulnerable to more sinister attacks. Research efforts to detect phishing URLs have involved the use of supervised learning techniques that use large amounts of data to train models and have high computational requirements. They also involve analysis of features derived from vectors including email contents thus affecting user privacy. Additionally, they suffer from a lack of resilience against evolution of threats especially with the advent of generative AI techniques to bypass these systems as with AI-generated phishing URLs. Unsupervised methods such as clustering techniques have also been used in phishing detection in the past, however, they are at times unscalable due to the use of pair-wise comparisons. They also lack high detection rates while detecting phishing campaigns. In this paper, we propose an unsupervised learning approach that is not only fast but scalable, as it does not involve pair-wise comparisons. It is able to detect entire campaigns at a time with a high detection rate while preserving user privacy; this includes the recent surge of campaigns with targeted phishing URLs generated by malicious entities using generative AI techniques.

Title: Generative Topology Optimization: Exploring Diverse Solutions in Structural Design

• Authors: Andreas Radler, Eric Volkmann, Johannes Brandstetter, Arturs Berzins
• Subjects: cs.LG, cond-mat.mtrl-sci, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13174
• Pdf URL: https://arxiv.org/pdf/2502.13174
• Copy Paste: [[2502.13174]] Generative Topology Optimization: Exploring Diverse Solutions in Structural Design(https://arxiv.org/abs/2502.13174)
• Keywords: generative
• Abstract: Topology optimization (TO) is a family of computational methods that derive near-optimal geometries from formal problem descriptions. Despite their success, established TO methods are limited to generating single solutions, restricting the exploration of alternative designs. To address this limitation, we introduce Generative Topology Optimization (GenTO) - a data-free method that trains a neural network to generate structurally compliant shapes and explores diverse solutions through an explicit diversity constraint. The network is trained with a solver-in-the-loop, optimizing the material distribution in each iteration. The trained model produces diverse shapes that closely adhere to the design requirements. We validate GenTO on 2D and 3D TO problems. Our results demonstrate that GenTO produces more diverse solutions than any prior method while maintaining near-optimality and being an order of magnitude faster due to inherent parallelism. These findings open new avenues for engineering and design, offering enhanced flexibility and innovation in structural optimization.

Title: A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models

• Authors: Longchao Da, Justin Turnau, Thirulogasankar Pranav Kutralingam, Alvaro Velasquez, Paulo Shakarian, Hua Wei
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.13187
• Pdf URL: https://arxiv.org/pdf/2502.13187
• Copy Paste: [[2502.13187]] A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models(https://arxiv.org/abs/2502.13187)
• Keywords: foundation model
• Abstract: Deep Reinforcement Learning (RL) has been explored and verified to be effective in solving decision-making tasks in various domains, such as robotics, transportation, recommender systems, etc. It learns from the interaction with environments and updates the policy using the collected experience. However, due to the limited real-world data and unbearable consequences of taking detrimental actions, the learning of RL policy is mainly restricted within the simulators. This practice guarantees safety in learning but introduces an inevitable sim-to-real gap in terms of deployment, thus causing degraded performance and risks in execution. There are attempts to solve the sim-to-real problems from different domains with various techniques, especially in the era with emerging techniques such as large foundations or language models that have cast light on the sim-to-real. This survey paper, to the best of our knowledge, is the first taxonomy that formally frames the sim-to-real techniques from key elements of the Markov Decision Process (State, Action, Transition, and Reward). Based on the framework, we cover comprehensive literature from the classic to the most advanced methods including the sim-to-real techniques empowered by foundation models, and we also discuss the specialties that are worth attention in different domains of sim-to-real problems. Then we summarize the formal evaluation process of sim-to-real performance with accessible code or benchmarks. The challenges and opportunities are also presented to encourage future exploration of this direction. We are actively maintaining a to include the most up-to-date sim-to-real research outcomes to help the researchers in their work.

Title: Learning To Explore With Predictive World Model Via Self-Supervised Learning

• Authors: Alana Santana, Paula P. Costa, Esther L. Colombini
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13200
• Pdf URL: https://arxiv.org/pdf/2502.13200
• Copy Paste: [[2502.13200]] Learning To Explore With Predictive World Model Via Self-Supervised Learning(https://arxiv.org/abs/2502.13200)
• Keywords: self-supervised
• Abstract: Autonomous artificial agents must be able to learn behaviors in complex environments without humans to design tasks and rewards. Designing these functions for each environment is not feasible, thus, motivating the development of intrinsic reward functions. In this paper, we propose using several cognitive elements that have been neglected for a long time to build an internal world model for an intrinsically motivated agent. Our agent performs satisfactory iterations with the environment, learning complex behaviors without needing previously designed reward functions. We used 18 Atari games to evaluate what cognitive skills emerge in games that require reactive and deliberative behaviors. Our results show superior performance compared to the state-of-the-art in many test cases with dense and sparse rewards.

Title: Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations

• Authors: Lee Cohen, Jack Hsieh, Connie Hong, Judy Hanwen Shen
• Subjects: cs.LG, cs.AI, cs.CY, cs.GT
• Abstract URL: https://arxiv.org/abs/2502.13221
• Pdf URL: https://arxiv.org/pdf/2502.13221
• Copy Paste: [[2502.13221]] Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations(https://arxiv.org/abs/2502.13221)
• Keywords: foundation model, generative
• Abstract: In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge about generative AI tools can harm both employers and candidates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a ``two-ticket'' scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipulated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an $n$-ticket scheme and prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential LLM access. Finally, we empirically validate our framework and the performance of our two-ticket scheme on real resumes using an open-source resume screening tool.

Title: MotionMatcher: Motion Customization of Text-to-Video Diffusion Models via Motion Feature Matching

• Authors: Yen-Siang Wu, Chi-Pin Huang, Fu-En Yang, Yu-Chiang Frank Wang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13234
• Pdf URL: https://arxiv.org/pdf/2502.13234
• Copy Paste: [[2502.13234]] MotionMatcher: Motion Customization of Text-to-Video Diffusion Models via Motion Feature Matching(https://arxiv.org/abs/2502.13234)
• Keywords: diffusion
• Abstract: Text-to-video (T2V) diffusion models have shown promising capabilities in synthesizing realistic videos from input text prompts. However, the input text description alone provides limited control over the precise objects movements and camera framing. In this work, we tackle the motion customization problem, where a reference video is provided as motion guidance. While most existing methods choose to fine-tune pre-trained diffusion models to reconstruct the frame differences of the reference video, we observe that such strategy suffer from content leakage from the reference video, and they cannot capture complex motion accurately. To address this issue, we propose MotionMatcher, a motion customization framework that fine-tunes the pre-trained T2V diffusion model at the feature level. Instead of using pixel-level objectives, MotionMatcher compares high-level, spatio-temporal motion features to fine-tune diffusion models, ensuring precise motion learning. For the sake of memory efficiency and accessibility, we utilize a pre-trained T2V diffusion model, which contains considerable prior knowledge about video motion, to compute these motion features. In our experiments, we demonstrate state-of-the-art motion customization performances, validating the design of our framework.

Title: A Survey of Anomaly Detection in Cyber-Physical Systems

• Authors: Danial Abshari, Meera Sridhar
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13256
• Pdf URL: https://arxiv.org/pdf/2502.13256
• Copy Paste: [[2502.13256]] A Survey of Anomaly Detection in Cyber-Physical Systems(https://arxiv.org/abs/2502.13256)
• Keywords: anomaly
• Abstract: In our increasingly interconnected world, Cyber-Physical Systems (CPS) play a crucial role in industries like healthcare, transportation, and manufacturing by combining physical processes with computing power. These systems, however, face many challenges, especially regarding security and system faults. Anomalies in CPS may indicate unexpected problems, from sensor malfunctions to cyber-attacks, and must be detected to prevent failures that can cause harm or disrupt services. This paper provides an overview of the different ways researchers have approached anomaly detection in CPS. We categorize and compare methods like machine learning, deep learning, mathematical models, invariant, and hybrid techniques. Our goal is to help readers understand the strengths and weaknesses of these methods and how they can be used to create safer, more reliable CPS. By identifying the gaps in current solutions, we aim to encourage future research that will make CPS more secure and adaptive in our increasingly automated world.

Title: Random Forest Autoencoders for Guided Representation Learning

• Authors: Adrien Aumon, Shuang Ni, Myriam Lizotte, Guy Wolf, Kevin R. Moon, Jake S. Rhodes
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13257
• Pdf URL: https://arxiv.org/pdf/2502.13257
• Copy Paste: [[2502.13257]] Random Forest Autoencoders for Guided Representation Learning(https://arxiv.org/abs/2502.13257)
• Keywords: diffusion
• Abstract: Decades of research have produced robust methods for unsupervised data visualization, yet supervised visualization$\unicode{x2013}$where expert labels guide representations$\unicode{x2013}$remains underexplored, as most supervised approaches prioritize classification over visualization. Recently, RF-PHATE, a diffusion-based manifold learning method leveraging random forests and information geometry, marked significant progress in supervised visualization. However, its lack of an explicit mapping function limits scalability and prevents application to unseen data, posing challenges for large datasets and label-scarce scenarios. To overcome these limitations, we introduce Random Forest Autoencoders (RF-AE), a neural network-based framework for out-of-sample kernel extension that combines the flexibility of autoencoders with the supervised learning strengths of random forests and the geometry captured by RF-PHATE. RF-AE enables efficient out-of-sample supervised visualization and outperforms existing methods, including RF-PHATE's standard kernel extension, in both accuracy and interpretability. Additionally, RF-AE is robust to the choice of hyper-parameters and generalizes to any kernel-based dimensionality reduction method.

Title: A Machine Learning Approach That Beats Large Rubik's Cubes

• Authors: Alexander Chervov, Kirill Khoruzhii, Nikita Bukhal, Jalal Naghiyev, Vladislav Zamkovoy, Ivan Koltsov, Lyudmila Cheldieva, Arsenii Sychev, Arsenii Lenin, Mark Obozov, Egor Urvanov, Alexey Romanov
• Subjects: cs.LG, cs.DM
• Abstract URL: https://arxiv.org/abs/2502.13266
• Pdf URL: https://arxiv.org/pdf/2502.13266
• Copy Paste: [[2502.13266]] A Machine Learning Approach That Beats Large Rubik's Cubes(https://arxiv.org/abs/2502.13266)
• Keywords: diffusion
• Abstract: The paper proposes a novel machine learning-based approach to the pathfinding problem on extremely large graphs. This method leverages diffusion distance estimation via a neural network and uses beam search for pathfinding. We demonstrate its efficiency by finding solutions for 4x4x4 and 5x5x5 Rubik's cubes with unprecedentedly short solution lengths, outperforming all available solvers and introducing the first machine learning solver beyond the 3x3x3 case. In particular, it surpasses every single case of the combined best results in the Kaggle Santa 2023 challenge, which involved over 1,000 teams. For the 3x3x3 Rubik's cube, our approach achieves an optimality rate exceeding 98%, matching the performance of task-specific solvers and significantly outperforming prior solutions such as DeepCubeA (60.3%) and EfficientCube (69.6%). Additionally, our solution is more than 26 times faster in solving 3x3x3 Rubik's cubes while requiring up to 18.5 times less model training time than the most efficient state-of-the-art competitor.

Title: Value Gradient Sampler: Sampling as Sequential Decision Making

• Authors: Sangwoong Yoon, Himchan Hwang, Hyeokju Jeong, Dong Kyu Shin, Che-Sang Park, Sehee Kwon, Frank Chongwoo Park
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13280
• Pdf URL: https://arxiv.org/pdf/2502.13280
• Copy Paste: [[2502.13280]] Value Gradient Sampler: Sampling as Sequential Decision Making(https://arxiv.org/abs/2502.13280)
• Keywords: anomaly
• Abstract: We propose the Value Gradient Sampler (VGS), a trainable sampler based on the interpretation of sampling as discrete-time sequential decision-making. VGS generates samples from a given unnormalized density (i.e., energy) by drifting and diffusing randomly initialized particles. In VGS, finding the optimal drift is equivalent to solving an optimal control problem where the cost is the upper bound of the KL divergence between the target density and the samples. We employ value-based dynamic programming to solve this optimal control problem, which gives the gradient of the value function as the optimal drift vector. The connection to sequential decision making allows VGS to leverage extensively studied techniques in reinforcement learning, making VGS a fast, adaptive, and accurate sampler that achieves competitive results in various sampling benchmarks. Furthermore, VGS can replace MCMC in contrastive divergence training of energy-based models. We demonstrate the effectiveness of VGS in training accurate energy-based models in industrial anomaly detection applications.

Title: Breaking the bonds of generative artificial intelligence by minimizing the maximum entropy

• Authors: Mattia Miotto, Lorenzo Monacelli
• Subjects: cs.LG, cond-mat.stat-mech, cs.IT
• Abstract URL: https://arxiv.org/abs/2502.13287
• Pdf URL: https://arxiv.org/pdf/2502.13287
• Copy Paste: [[2502.13287]] Breaking the bonds of generative artificial intelligence by minimizing the maximum entropy(https://arxiv.org/abs/2502.13287)
• Keywords: generative
• Abstract: The emergence of generative artificial intelligence (GenAI), comprising large language models, text-to-image generators, and AI algorithms for medical drug and material design, had a transformative impact on society. However, despite an initial exponential growth surpassing Moore's law, progress is now plateauing, suggesting we are approaching the limits of current technology. Indeed, these models are notoriously data-hungry, prone to overfitting, and challenging to direct during the generative process, hampering their effective professional employment. To cope with these limitations, we propose a paradigm shift in GenAI by introducing an ab initio method based on the minimal maximum entropy principle. Our approach does not fit the data. Instead, it compresses information in the training set by finding a latent representation parameterized by arbitrary nonlinear functions, such as neural networks. The result is a general physics-driven model, which is data-efficient, resistant to overfitting, and flexible, permitting to control and influence the generative process. Benchmarking shows that our method outperforms variational autoencoders (VAEs) with similar neural architectures, particularly on undersampled datasets. We demonstrate the methods effectiveness in generating images, even with limited training data, and its unprecedented capability to customize the generation process a posteriori without the need of any fine-tuning or retraining.

Title: VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detection

• Authors: Paul Boniol, Ashwin K. Krishna, Marine Bruel, Qinghua Liu, Mingyi Huang, Themis Palpanas, Ruey S. Tsay, Aaron Elmore, Michael J. Franklin, John Paparrizos
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13318
• Pdf URL: https://arxiv.org/pdf/2502.13318
• Copy Paste: [[2502.13318]] VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detection(https://arxiv.org/abs/2502.13318)
• Keywords: anomaly
• Abstract: Anomaly detection (AD) is a fundamental task for time-series analytics with important implications for the downstream performance of many applications. In contrast to other domains where AD mainly focuses on point-based anomalies (i.e., outliers in standalone observations), AD for time series is also concerned with range-based anomalies (i.e., outliers spanning multiple observations). Nevertheless, it is common to use traditional point-based information retrieval measures, such as Precision, Recall, and F-score, to assess the quality of methods by thresholding the anomaly score to mark each point as an anomaly or not. However, mapping discrete labels into continuous data introduces unavoidable shortcomings, complicating the evaluation of range-based anomalies. Notably, the choice of evaluation measure may significantly bias the experimental outcome. Despite over six decades of attention, there has never been a large-scale systematic quantitative and qualitative analysis of time-series AD evaluation measures. This paper extensively evaluates quality measures for time-series AD to assess their robustness under noise, misalignments, and different anomaly cardinality ratios. Our results indicate that measures producing quality values independently of a threshold (i.e., AUC-ROC and AUC-PR) are more suitable for time-series AD. Motivated by this observation, we first extend the AUC-based measures to account for range-based anomalies. Then, we introduce a new family of parameter-free and threshold-independent measures, Volume Under the Surface (VUS), to evaluate methods while varying parameters. We also introduce two optimized implementations for VUS that reduce significantly the execution time of the initial implementation. Our findings demonstrate that our four measures are significantly more robust in assessing the quality of time-series AD methods.

Title: Geometry-Aware Diffusion Models for Multiview Scene Inpainting

• Authors: Ahmad Salimi, Tristan Aumentado-Armstrong, Marcus A. Brubaker, Konstantinos G. Derpanis
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.13335
• Pdf URL: https://arxiv.org/pdf/2502.13335
• Copy Paste: [[2502.13335]] Geometry-Aware Diffusion Models for Multiview Scene Inpainting(https://arxiv.org/abs/2502.13335)
• Keywords: diffusion, generative
• Abstract: In this paper, we focus on 3D scene inpainting, where parts of an input image set, captured from different viewpoints, are masked out. The main challenge lies in generating plausible image completions that are geometrically consistent across views. Most recent work addresses this challenge by combining generative models with a 3D radiance field to fuse information across viewpoints. However, a major drawback of these methods is that they often produce blurry images due to the fusion of inconsistent cross-view images. To avoid blurry inpaintings, we eschew the use of an explicit or implicit radiance field altogether and instead fuse cross-view information in a learned space. In particular, we introduce a geometry-aware conditional generative model, capable of inpainting multi-view consistent images based on both geometric and appearance cues from reference images. A key advantage of our approach over existing methods is its unique ability to inpaint masked scenes with a limited number of views (i.e., few-view inpainting), whereas previous methods require relatively large image sets for their 3D model fitting step. Empirically, we evaluate and compare our scene-centric inpainting method on two datasets, SPIn-NeRF and NeRFiller, which contain images captured at narrow and wide baselines, respectively, and achieve state-of-the-art 3D inpainting performance on both. Additionally, we demonstrate the efficacy of our approach in the few-view setting compared to prior methods.

Title: How Expressive are Knowledge Graph Foundation Models?

• Authors: Xingyue Huang, Pablo Barceló, Michael M. Bronstein, İsmail İlkan Ceylan, Mikhail Galkin, Juan L Reutter, Miguel Romero Orth
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13339
• Pdf URL: https://arxiv.org/pdf/2502.13339
• Copy Paste: [[2502.13339]] How Expressive are Knowledge Graph Foundation Models?(https://arxiv.org/abs/2502.13339)
• Keywords: foundation model
• Abstract: Knowledge Graph Foundation Models (KGFMs) are at the frontier for deep learning on knowledge graphs (KGs), as they can generalize to completely novel knowledge graphs with different relational vocabularies. Despite their empirical success, our theoretical understanding of KGFMs remains very limited. In this paper, we conduct a rigorous study of the expressive power of KGFMs. Specifically, we show that the expressive power of KGFMs directly depends on the motifs that are used to learn the relation representations. We then observe that the most typical motifs used in the existing literature are binary, as the representations are learned based on how pairs of relations interact, which limits the model's expressiveness. As part of our study, we design more expressive KGFMs using richer motifs, which necessitate learning relation representations based on, e.g., how triples of relations interact with each other. Finally, we empirically validate our theoretical findings, showing that the use of richer motifs results in better performance on a wide range of datasets drawn from different domains.

Title: Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios

• Authors: Liangqi Lei, Keke Gai, Jing Yu, Liehuang Zhu, Qi Wu
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13345
• Pdf URL: https://arxiv.org/pdf/2502.13345
• Copy Paste: [[2502.13345]] Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios(https://arxiv.org/abs/2502.13345)
• Keywords: diffusion, generative
• Abstract: Latent diffusion models have exhibited considerable potential in generative tasks. Watermarking is considered to be an alternative to safeguard the copyright of generative models and prevent their misuse. However, in the context of model distribution scenarios, the accessibility of models to large scale of model users brings new challenges to the security, efficiency and robustness of existing watermark solutions. To address these issues, we propose a secure and efficient watermarking solution. A new security mechanism is designed to prevent watermark leakage and watermark escape, which considers watermark randomness and watermark-model association as two constraints for mandatory watermark injection. To reduce the time cost of training the security module, watermark injection and the security mechanism are decoupled, ensuring that fine-tuning VAE only accomplishes the security mechanism without the burden of learning watermark patterns. A watermark distribution-based verification strategy is proposed to enhance the robustness against diverse attacks in the model distribution scenarios. Experimental results prove that our watermarking consistently outperforms existing six baselines on effectiveness and robustness against ten image processing attacks and adversarial attacks, while enhancing security in the distribution scenarios.

Title: Quantum Recurrent Neural Networks with Encoder-Decoder for Time-Dependent Partial Differential Equations

• Authors: Yuan Chen, Abdul Khaliq, Khaled M. Furati
• Subjects: cs.LG, math.NA, quant-ph
• Abstract URL: https://arxiv.org/abs/2502.13370
• Pdf URL: https://arxiv.org/pdf/2502.13370
• Copy Paste: [[2502.13370]] Quantum Recurrent Neural Networks with Encoder-Decoder for Time-Dependent Partial Differential Equations(https://arxiv.org/abs/2502.13370)
• Keywords: diffusion
• Abstract: Nonlinear time-dependent partial differential equations are essential in modeling complex phenomena across diverse fields, yet they pose significant challenges due to their computational complexity, especially in higher dimensions. This study explores Quantum Recurrent Neural Networks within an encoder-decoder framework, integrating Variational Quantum Circuits into Gated Recurrent Units and Long Short-Term Memory networks. Using this architecture, the model efficiently compresses high-dimensional spatiotemporal data into a compact latent space, facilitating more efficient temporal evolution. We evaluate the algorithms on the Hamilton-Jacobi-Bellman equation, Burgers' equation, the Gray-Scott reaction-diffusion system, and the three dimensional Michaelis-Menten reaction-diffusion equation. The results demonstrate the superior performance of the quantum-based algorithms in capturing nonlinear dynamics, handling high-dimensional spaces, and providing stable solutions, highlighting their potential as an innovative tool in solving challenging and complex systems.

Title: Flow-based generative models as iterative algorithms in probability space

• Authors: Yao Xie, Xiuyuan Cheng
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.13394
• Pdf URL: https://arxiv.org/pdf/2502.13394
• Copy Paste: [[2502.13394]] Flow-based generative models as iterative algorithms in probability space(https://arxiv.org/abs/2502.13394)
• Keywords: generative, anomaly
• Abstract: Generative AI (GenAI) has revolutionized data-driven modeling by enabling the synthesis of high-dimensional data across various applications, including image generation, language modeling, biomedical signal processing, and anomaly detection. Flow-based generative models provide a powerful framework for capturing complex probability distributions, offering exact likelihood estimation, efficient sampling, and deterministic transformations between distributions. These models leverage invertible mappings governed by Ordinary Differential Equations (ODEs), enabling precise density estimation and likelihood evaluation. This tutorial presents an intuitive mathematical framework for flow-based generative models, formulating them as neural network-based representations of continuous probability densities. We explore key theoretical principles, including the Wasserstein metric, gradient flows, and density evolution governed by ODEs, to establish convergence guarantees and bridge empirical advancements with theoretical insights. By providing a rigorous yet accessible treatment, we aim to equip researchers and practitioners with the necessary tools to effectively apply flow-based generative models in signal processing and machine learning.

Title: Interleaved Gibbs Diffusion for Constrained Generation

• Authors: Gautham Govind Anil, Sachin Yadav, Dheeraj Nagaraj, Karthikeyan Shanmugam, Prateek Jain
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13450
• Pdf URL: https://arxiv.org/pdf/2502.13450
• Copy Paste: [[2502.13450]] Interleaved Gibbs Diffusion for Constrained Generation(https://arxiv.org/abs/2502.13450)
• Keywords: diffusion, generative
• Abstract: We introduce Interleaved Gibbs Diffusion (IGD), a novel generative modeling framework for mixed continuous-discrete data, focusing on constrained generation problems. Prior works on discrete and continuous-discrete diffusion models assume factorized denoising distribution for fast generation, which can hinder the modeling of strong dependencies between random variables encountered in constrained generation. IGD moves beyond this by interleaving continuous and discrete denoising algorithms via a discrete time Gibbs sampling type Markov chain. IGD provides flexibility in the choice of denoisers, allows conditional generation via state-space doubling and inference time scaling via the ReDeNoise method. Empirical evaluations on three challenging tasks-solving 3-SAT, generating molecule structures, and generating layouts-demonstrate state-of-the-art performance. Notably, IGD achieves a 7% improvement on 3-SAT out of the box and achieves state-of-the-art results in molecule generation without relying on equivariant diffusion or domain-specific architectures. We explore a wide range of modeling, and interleaving strategies along with hyperparameters in each of these problems.

Title: Estimating Commonsense Plausibility through Semantic Shifts

• Authors: Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13464
• Pdf URL: https://arxiv.org/pdf/2502.13464
• Copy Paste: [[2502.13464]] Estimating Commonsense Plausibility through Semantic Shifts(https://arxiv.org/abs/2502.13464)
• Keywords: generative
• Abstract: Commonsense plausibility estimation is critical for evaluating language models (LMs), yet existing generative approaches--reliant on likelihoods or verbalized judgments--struggle with fine-grained discrimination. In this paper, we propose ComPaSS, a novel discriminative framework that quantifies commonsense plausibility by measuring semantic shifts when augmenting sentences with commonsense-related information. Plausible augmentations induce minimal shifts in semantics, while implausible ones result in substantial deviations. Evaluations on two types of fine-grained commonsense plausibility estimation tasks across different backbones, including LLMs and vision-language models (VLMs), show that ComPaSS consistently outperforms baselines. It demonstrates the advantage of discriminative approaches over generative methods in fine-grained commonsense plausibility evaluation. Experiments also show that (1) VLMs yield superior performance to LMs, when integrated with ComPaSS, on vision-grounded commonsense tasks. (2) contrastive pre-training sharpens backbone models' ability to capture semantic nuances, thereby further enhancing ComPaSS.

Title: Towards Geo-Culturally Grounded LLM Generations

• Authors: Piyawat Lertvittayakumjorn, David Kinney, Vinodkumar Prabhakaran, Donald Martin, Sunipa Dev
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13497
• Pdf URL: https://arxiv.org/pdf/2502.13497
• Copy Paste: [[2502.13497]] Towards Geo-Culturally Grounded LLM Generations(https://arxiv.org/abs/2502.13497)
• Keywords: generative
• Abstract: Generative large language models (LLMs) have been demonstrated to have gaps in diverse, cultural knowledge across the globe. We investigate the effect of retrieval augmented generation and search-grounding techniques on the ability of LLMs to display familiarity with a diverse range of national cultures. Specifically, we compare the performance of standard LLMs, LLMs augmented with retrievals from a bespoke knowledge base (i.e., KB grounding), and LLMs augmented with retrievals from a web search (i.e., search grounding) on a series of cultural familiarity benchmarks. We find that search grounding significantly improves the LLM performance on multiple-choice benchmarks that test propositional knowledge (e.g., the norms, artifacts, and institutions of national cultures), while KB grounding's effectiveness is limited by inadequate knowledge base coverage and a suboptimal retriever. However, search grounding also increases the risk of stereotypical judgments by language models, while failing to improve evaluators' judgments of cultural familiarity in a human evaluation with adequate statistical power. These results highlight the distinction between propositional knowledge about a culture and open-ended cultural fluency when it comes to evaluating the cultural familiarity of generative LLMs.

Title: Unlocking Multimodal Integration in EHRs: A Prompt Learning Framework for Language and Time Series Fusion

• Authors: Shuai Niu, Jing Ma, Hongzhan Lin, Liang Bai, Zhihua Wang, Wei Bi, Yida Xu, Guo Li, Xian Yang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13509
• Pdf URL: https://arxiv.org/pdf/2502.13509
• Copy Paste: [[2502.13509]] Unlocking Multimodal Integration in EHRs: A Prompt Learning Framework for Language and Time Series Fusion(https://arxiv.org/abs/2502.13509)
• Keywords: self-supervised, anomaly
• Abstract: Large language models (LLMs) have shown remarkable performance in vision-language tasks, but their application in the medical field remains underexplored, particularly for integrating structured time series data with unstructured clinical notes. In clinical practice, dynamic time series data such as lab test results capture critical temporal patterns, while clinical notes provide rich semantic context. Merging these modalities is challenging due to the inherent differences between continuous signals and discrete text. To bridge this gap, we introduce ProMedTS, a novel self-supervised multimodal framework that employs prompt-guided learning to unify these heterogeneous data types. Our approach leverages lightweight anomaly detection to generate anomaly captions that serve as prompts, guiding the encoding of raw time series data into informative embeddings. These embeddings are aligned with textual representations in a shared latent space, preserving fine-grained temporal nuances alongside semantic insights. Furthermore, our framework incorporates tailored self-supervised objectives to enhance both intra- and inter-modal alignment. We evaluate ProMedTS on disease diagnosis tasks using real-world datasets, and the results demonstrate that our method consistently outperforms state-of-the-art approaches.

Title: AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learning

• Authors: Ruyue Liu, Rong Yin, Yong Liu, Xiaoshuai Hao, Haichao Shi, Can Ma, Weiping Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13525
• Pdf URL: https://arxiv.org/pdf/2502.13525
• Copy Paste: [[2502.13525]] AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learning(https://arxiv.org/abs/2502.13525)
• Keywords: diffusion, self-supervised
• Abstract: Graph Contrastive Learning (GCL) has emerged as the foremost approach for self-supervised learning on graph-structured data. GCL reduces reliance on labeled data by learning robust representations from various augmented views. However, existing GCL methods typically depend on consistent stochastic augmentations, which overlook their impact on the intrinsic structure of the spectral domain, thereby limiting the model's ability to generalize effectively. To address these limitations, we propose a novel paradigm called AS-GCL that incorporates asymmetric spectral augmentation for graph contrastive learning. A typical GCL framework consists of three key components: graph data augmentation, view encoding, and contrastive loss. Our method introduces significant enhancements to each of these components. Specifically, for data augmentation, we apply spectral-based augmentation to minimize spectral variations, strengthen structural invariance, and reduce noise. With respect to encoding, we employ parameter-sharing encoders with distinct diffusion operators to generate diverse, noise-resistant graph views. For contrastive loss, we introduce an upper-bound loss function that promotes generalization by maintaining a balanced distribution of intra- and inter-class distance. To our knowledge, we are the first to encode augmentation views of the spectral domain using asymmetric encoders. Extensive experiments on eight benchmark datasets across various node-level tasks demonstrate the advantages of the proposed method.

Title: Detecting Linguistic Bias in Government Documents Using Large language Models

• Authors: Milena de Swart, Floris den Hengst, Jieying Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13548
• Pdf URL: https://arxiv.org/pdf/2502.13548
• Copy Paste: [[2502.13548]] Detecting Linguistic Bias in Government Documents Using Large language Models(https://arxiv.org/abs/2502.13548)
• Keywords: generative
• Abstract: This paper addresses the critical need for detecting bias in government documents, an underexplored area with significant implications for governance. Existing methodologies often overlook the unique context and far-reaching impacts of governmental documents, potentially obscuring embedded biases that shape public policy and citizen-government interactions. To bridge this gap, we introduce the Dutch Government Data for Bias Detection (DGDB), a dataset sourced from the Dutch House of Representatives and annotated for bias by experts. We fine-tune several BERT-based models on this dataset and compare their performance with that of generative language models. Additionally, we conduct a comprehensive error analysis that includes explanations of the models' predictions. Our findings demonstrate that fine-tuned models achieve strong performance and significantly outperform generative language models, indicating the effectiveness of DGDB for bias detection. This work underscores the importance of labeled datasets for bias detection in various languages and contributes to more equitable governance practices.

Title: Are Large Language Models In-Context Graph Learners?

• Authors: Jintang Li, Ruofan Wu, Yuchang Zhu, Huizhe Zhang, Liang Chen, Zibin Zheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.13562
• Pdf URL: https://arxiv.org/pdf/2502.13562
• Copy Paste: [[2502.13562]] Are Large Language Models In-Context Graph Learners?(https://arxiv.org/abs/2502.13562)
• Keywords: in-context
• Abstract: Large language models (LLMs) have demonstrated remarkable in-context reasoning capabilities across a wide range of tasks, particularly with unstructured inputs such as language or images. However, LLMs struggle to handle structured data, such as graphs, due to their lack of understanding of non-Euclidean structures. As a result, without additional fine-tuning, their performance significantly lags behind that of graph neural networks (GNNs) in graph learning tasks. In this paper, we show that learning on graph data can be conceptualized as a retrieval-augmented generation (RAG) process, where specific instances (e.g., nodes or edges) act as queries, and the graph itself serves as the retrieved context. Building on this insight, we propose a series of RAG frameworks to enhance the in-context learning capabilities of LLMs for graph learning tasks. Comprehensive evaluations demonstrate that our proposed RAG frameworks significantly improve LLM performance on graph-based tasks, particularly in scenarios where a pretrained LLM must be used without modification or accessed via an API.

Title: Extracting Social Connections from Finnish Karelian Refugee Interviews Using LLMs

• Authors: Joonatan Laato, Jenna Kanerva, John Loehr, Virpi Lummaa, Filip Ginter
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13566
• Pdf URL: https://arxiv.org/pdf/2502.13566
• Copy Paste: [[2502.13566]] Extracting Social Connections from Finnish Karelian Refugee Interviews Using LLMs(https://arxiv.org/abs/2502.13566)
• Keywords: generative
• Abstract: We performed a zero-shot information extraction study on a historical collection of 89,339 brief Finnish-language interviews of refugee families relocated post-WWII from Finnish Eastern Karelia. Our research objective is two-fold. First, we aim to extract social organizations and hobbies from the free text of the interviews, separately for each family member. These can act as a proxy variable indicating the degree of social integration of refugees in their new environment. Second, we aim to evaluate several alternative ways to approach this task, comparing a number of generative models and a supervised learning approach, to gain a broader insight into the relative merits of these different approaches and their applicability in similar studies. We find that the best generative model (GPT-4) is roughly on par with human performance, at an F-score of 88.8%. Interestingly, the best open generative model (Llama-3-70B-Instruct) reaches almost the same performance, at 87.7% F-score, demonstrating that open models are becoming a viable alternative for some practical tasks even on non-English data. Additionally, we test a supervised learning alternative, where we fine-tune a Finnish BERT model (FinBERT) using GPT-4 generated training data. By this method, we achieved an F-score of 84.1% already with 6K interviews up to an F-score of 86.3% with 30k interviews. Such an approach would be particularly appealing in cases where the computational resources are limited, or there is a substantial mass of data to process.

Title: D.Va: Validate Your Demonstration First Before You Use It

• Authors: Qi Zhang, Zhiqing Xiao, Ruixuan Xiao, Lirong Gao, Junbo Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13646
• Pdf URL: https://arxiv.org/pdf/2502.13646
• Copy Paste: [[2502.13646]] D.Va: Validate Your Demonstration First Before You Use It(https://arxiv.org/abs/2502.13646)
• Keywords: in-context
• Abstract: In-context learning (ICL) has demonstrated significant potential in enhancing the capabilities of large language models (LLMs) during inference. It's well-established that ICL heavily relies on selecting effective demonstrations to generate outputs that better align with the expected results. As for demonstration selection, previous approaches have typically relied on intuitive metrics to evaluate the effectiveness of demonstrations, which often results in limited robustness and poor cross-model generalization capabilities. To tackle these challenges, we propose a novel method, \textbf{D}emonstration \textbf{VA}lidation (\textbf{this http URL}), which integrates a demonstration validation perspective into this field. By introducing the demonstration validation mechanism, our method effectively identifies demonstrations that are both effective and highly generalizable. \textbf{this http URL} surpasses all existing demonstration selection techniques across both natural language understanding (NLU) and natural language generation (NLG) tasks. Additionally, we demonstrate the robustness and generalizability of our approach across various language models with different retrieval models.

Title: Instruction Tuning on Public Government and Cultural Data for Low-Resource Language: a Case Study in Kazakh

• Authors: Nurkhan Laiyk, Daniil Orel, Rituraj Joshi, Maiya Goloburda, Yuxia Wang, Preslav Nakov, Fajri Koto
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13647
• Pdf URL: https://arxiv.org/pdf/2502.13647
• Copy Paste: [[2502.13647]] Instruction Tuning on Public Government and Cultural Data for Low-Resource Language: a Case Study in Kazakh(https://arxiv.org/abs/2502.13647)
• Keywords: generative
• Abstract: Instruction tuning in low-resource languages remains underexplored due to limited text data, particularly in government and cultural domains. To address this, we introduce and open-source a large-scale (10,600 samples) instruction-following (IFT) dataset, covering key institutional and cultural knowledge relevant to Kazakhstan. Our dataset enhances LLMs' understanding of procedural, legal, and structural governance topics. We employ LLM-assisted data generation, comparing open-weight and closed-weight models for dataset construction, and select GPT-4o as the backbone. Each entity of our dataset undergoes full manual verification to ensure high quality. We also show that fine-tuning Qwen, Falcon, and Gemma on our dataset leads to consistent performance improvements in both multiple-choice and generative tasks, demonstrating the potential of LLM-assisted instruction tuning for low-resource languages.

Title: SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation

• Authors: Song Duong, Florian Le Bronnec, Alexandre Allauzen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13674
• Pdf URL: https://arxiv.org/pdf/2502.13674
• Copy Paste: [[2502.13674]] SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation(https://arxiv.org/abs/2502.13674)
• Keywords: self-supervised
• Abstract: Large Language Models (LLMs), when used for conditional text generation, often produce hallucinations, i.e., information that is unfaithful or not grounded in the input context. This issue arises in typical conditional text generation tasks, such as text summarization and data-to-text generation, where the goal is to produce fluent text based on contextual input. When fine-tuned on specific domains, LLMs struggle to provide faithful answers to a given context, often adding information or generating errors. One underlying cause of this issue is that LLMs rely on statistical patterns learned from their training data. This reliance can interfere with the model's ability to stay faithful to a provided context, leading to the generation of ungrounded information. We build upon this observation and introduce a novel self-supervised method for generating a training set of unfaithful samples. We then refine the model using a training process that encourages the generation of grounded outputs over unfaithful ones, drawing on preference-based training. Our approach leads to significantly more grounded text generation, outperforming existing self-supervised techniques in faithfulness, as evaluated through automatic metrics, LLM-based assessments, and human evaluations.

Title: Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method

• Authors: Juyuan Zhang, Wei Zhu, Jiechao Gao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13725
• Pdf URL: https://arxiv.org/pdf/2502.13725
• Copy Paste: [[2502.13725]] Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method(https://arxiv.org/abs/2502.13725)
• Keywords: foundation model
• Abstract: Time series modeling holds significant importance in many real-world applications and has been extensively studied. While pre-trained foundation models have made impressive strides in the fields of natural language processing (NLP) and computer vision (CV), their development in time series domains has been constrained by data sparsity. A series of recent studies have demonstrated that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the current literature have yet striked a high-quality balance between (a) effectively aligning the time series and natural language modalities, and (b) keeping the inference efficiency. To address the above issues, we now propose the Time-LlaMA framework. Time-LlaMA first converts the time series input into token embeddings through a linear tokenization mechanism. Second, the time series token embeddings are aligned with the text prompts. Third, to further adapt the LLM backbone for time series modeling, we have developed a dynamic low-rank adaptation technique (D-LoRA). D-LoRA dynamically chooses the most suitable LoRA modules at each layer of the Transformer backbone for each time series input, enhancing the model's predictive capabilities. Our experimental results on an extensive collection of challenging real-world time series tasks confirm that our proposed method achieves the state-of-the-art (SOTA) performance.

Title: CARE: Confidence-Aware Regression Estimation of building density fine-tuning EO Foundation Models

• Authors: Nikolaos Dionelis, Jente Bosmans, Nicolas Longépé
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13734
• Pdf URL: https://arxiv.org/pdf/2502.13734
• Copy Paste: [[2502.13734]] CARE: Confidence-Aware Regression Estimation of building density fine-tuning EO Foundation Models(https://arxiv.org/abs/2502.13734)
• Keywords: foundation model
• Abstract: Performing accurate confidence quantification and assessment is important for deep neural networks to predict their failures, improve their performance and enhance their capabilities in real-world applications, for their practical deployment in real life. For pixel-wise regression tasks, confidence quantification and assessment has not been well addressed in the literature, in contrast to classification tasks like semantic segmentation. The softmax output layer is not used in deep neural networks that solve pixel-wise regression problems. In this paper, to address these problems, we develop, train and evaluate the proposed model Confidence-Aware Regression Estimation (CARE). Our model CARE computes and assigns confidence to regression output results. We focus on solving regression problems as downstream tasks of an AI Foundation Model for Earth Observation (EO). We evaluate the proposed model CARE and experimental results on data from the Copernicus Sentinel-2 satellite constellation for estimating the density of buildings show that the proposed method can be successfully applied to regression problems. We also show that our approach outperforms other methods.

Title: Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding

• Authors: Keqin Peng, Liang Ding, Yuanxin Ouyang, Meng Fang, Yancheng Yuan, Dacheng Tao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13738
• Pdf URL: https://arxiv.org/pdf/2502.13738
• Copy Paste: [[2502.13738]] Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding(https://arxiv.org/abs/2502.13738)
• Keywords: in-context
• Abstract: Large language models (LLMs) excel at a range of tasks through in-context learning (ICL), where only a few task examples guide their predictions. However, prior research highlights that LLMs often overlook input-label mapping information in ICL, relying more on their pre-trained knowledge. To address this issue, we introduce In-Context Contrastive Decoding (ICCD), a novel method that emphasizes input-label mapping by contrasting the output distributions between positive and negative in-context examples. Experiments on 7 natural language understanding (NLU) tasks show that our ICCD method brings consistent and significant improvement (up to +2.1 improvement on average) upon 6 different scales of LLMs without requiring additional training. Our approach is versatile, enhancing performance with various demonstration selection methods, demonstrating its broad applicability and effectiveness. The code and scripts will be publicly released.

Title: Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions

• Authors: Xinwei Shen, Nicolai Meinshausen, Tong Zhang
• Subjects: cs.LG, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.13747
• Pdf URL: https://arxiv.org/pdf/2502.13747
• Copy Paste: [[2502.13747]] Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions(https://arxiv.org/abs/2502.13747)
• Keywords: diffusion, generative
• Abstract: Learning complex distributions is a fundamental challenge in contemporary applications. Generative models, such as diffusion models, have demonstrated remarkable success in overcoming many limitations of traditional statistical methods. Shen and Meinshausen (2024) introduced engression, a generative approach based on scoring rules that maps noise (and covariates, if available) directly to data. While effective, engression struggles with highly complex distributions, such as those encountered in image data. In this work, we extend engression to improve its capability in learning complex distributions. We propose a framework that defines a general forward process transitioning from the target distribution to a known distribution (e.g., Gaussian) and then learns a reverse Markov process using multiple engression models. This reverse process reconstructs the target distribution step by step. Our approach supports general forward processes, allows for dimension reduction, and naturally discretizes the generative process. As a special case, when using a diffusion-based forward process, our framework offers a method to discretize the training and inference of diffusion models efficiently. Empirical evaluations on simulated and climate data validate our theoretical insights, demonstrating the effectiveness of our approach in capturing complex distributions.

Title: Refining embeddings with fill-tuning: data-efficient generalised performance improvements for materials foundation models

• Authors: Matthew P. Wilson, Edward O. Pyzer-Knapp, Nicolas Galichet, Luke Dicks
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.13886
• Pdf URL: https://arxiv.org/pdf/2502.13886
• Copy Paste: [[2502.13886]] Refining embeddings with fill-tuning: data-efficient generalised performance improvements for materials foundation models(https://arxiv.org/abs/2502.13886)
• Keywords: foundation model
• Abstract: Pretrained foundation models learn embeddings that can be used for a wide range of downstream tasks. These embeddings optimise general performance, and if insufficiently accurate at a specific task the model can be fine-tuned to improve performance. For all current methodologies this operation necessarily degrades performance on all out-of-distribution tasks. In this work we present 'fill-tuning', a novel methodology to generate datasets for continued pretraining of foundation models that are not suited to a particular downstream task, but instead aim to correct poor regions of the embedding. We present the application of roughness analysis to latent space topologies and illustrate how it can be used to propose data that will be most valuable to improving the embedding. We apply fill-tuning to a set of state-of-the-art materials foundation models trained on $O(10^9)$ data points and show model improvement of almost 1% in all downstream tasks with the addition of only 100 data points. This method provides a route to the general improvement of foundation models at the computational cost of fine-tuning.

Title: TESS 2: A Large-Scale Generalist Diffusion Language Model

• Authors: Jaesung Tae, Hamish Ivison, Sachin Kumar, Arman Cohan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.13917
• Pdf URL: https://arxiv.org/pdf/2502.13917
• Copy Paste: [[2502.13917]] TESS 2: A Large-Scale Generalist Diffusion Language Model(https://arxiv.org/abs/2502.13917)
• Keywords: diffusion
• Abstract: We introduce TESS 2, a general instruction-following diffusion language model that outperforms contemporary instruction-tuned diffusion models, as well as matches and sometimes exceeds strong autoregressive (AR) models. We train TESS 2 by first adapting a strong AR model via continued pretraining with the usual cross-entropy as diffusion loss, and then performing further instruction tuning. We find that adaptation training as well as the choice of the base model is crucial for training good instruction-following diffusion models. We further propose reward guidance, a novel and modular inference-time guidance procedure to align model outputs without needing to train the underlying model. Finally, we show that TESS 2 further improves with increased inference-time compute, highlighting the utility of diffusion LMs in having fine-grained controllability over the amount of compute used at inference time. Code and models are available at this https URL.

Title: Image compositing is all you need for data augmentation

• Authors: Ang Jia Ning Shermaine, Michalis Lazarou, Tania Stathaki
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.13936
• Pdf URL: https://arxiv.org/pdf/2502.13936
• Copy Paste: [[2502.13936]] Image compositing is all you need for data augmentation(https://arxiv.org/abs/2502.13936)
• Keywords: diffusion, generative
• Abstract: This paper investigates the impact of various data augmentation techniques on the performance of object detection models. Specifically, we explore classical augmentation methods, image compositing, and advanced generative models such as Stable Diffusion XL and ControlNet. The objective of this work is to enhance model robustness and improve detection accuracy, particularly when working with limited annotated data. Using YOLOv8, we fine-tune the model on a custom dataset consisting of commercial and military aircraft, applying different augmentation strategies. Our experiments show that image compositing offers the highest improvement in detection performance, as measured by precision, recall, and mean Average Precision (mAP@0.50). Other methods, including Stable Diffusion XL and ControlNet, also demonstrate significant gains, highlighting the potential of advanced data augmentation techniques for object detection tasks. The results underline the importance of dataset diversity and augmentation in achieving better generalization and performance in real-world applications. Future work will explore the integration of semi-supervised learning methods and further optimizations to enhance model performance across larger and more complex datasets.

Title: IP-Composer: Semantic Composition of Visual Concepts

• Authors: Sara Dorfman, Dana Cohen-Bar, Rinon Gal, Daniel Cohen-Or
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.13951
• Pdf URL: https://arxiv.org/pdf/2502.13951
• Copy Paste: [[2502.13951]] IP-Composer: Semantic Composition of Visual Concepts(https://arxiv.org/abs/2502.13951)
• Keywords: diffusion
• Abstract: Content creators often draw inspiration from multiple visual sources, combining distinct elements to craft new compositions. Modern computational approaches now aim to emulate this fundamental creative process. Although recent diffusion models excel at text-guided compositional synthesis, text as a medium often lacks precise control over visual details. Image-based composition approaches can capture more nuanced features, but existing methods are typically limited in the range of concepts they can capture, and require expensive training procedures or specialized data. We present IP-Composer, a novel training-free approach for compositional image generation that leverages multiple image references simultaneously, while using natural language to describe the concept to be extracted from each image. Our method builds on IP-Adapter, which synthesizes novel images conditioned on an input image's CLIP embedding. We extend this approach to multiple visual inputs by crafting composite embeddings, stitched from the projections of multiple input images onto concept-specific CLIP-subspaces identified through text. Through comprehensive evaluation, we show that our approach enables more precise control over a larger range of visual concept compositions.