2025-05-14

Title: Joint Detection of Fraud and Concept Drift inOnline Conversations with LLM-Assisted Judgment

• Authors: Ali Senol, Garima Agrawal, Huan Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.07852
• Pdf URL: https://arxiv.org/pdf/2505.07852
• Copy Paste: [[2505.07852]] Joint Detection of Fraud and Concept Drift inOnline Conversations with LLM-Assisted Judgment(https://arxiv.org/abs/2505.07852)
• Keywords: anomaly
• Abstract: Detecting fake interactions in digital communication platforms remains a challenging and insufficiently addressed problem. These interactions may appear as harmless spam or escalate into sophisticated scam attempts, making it difficult to flag malicious intent early. Traditional detection methods often rely on static anomaly detection techniques that fail to adapt to dynamic conversational shifts. One key limitation is the misinterpretation of benign topic transitions referred to as concept drift as fraudulent behavior, leading to either false alarms or missed threats. We propose a two stage detection framework that first identifies suspicious conversations using a tailored ensemble classification model. To improve the reliability of detection, we incorporate a concept drift analysis step using a One Class Drift Detector (OCDD) to isolate conversational shifts within flagged dialogues. When drift is detected, a large language model (LLM) assesses whether the shift indicates fraudulent manipulation or a legitimate topic change. In cases where no drift is found, the behavior is inferred to be spam like. We validate our framework using a dataset of social engineering chat scenarios and demonstrate its practical advantages in improving both accuracy and interpretability for real time fraud detection. To contextualize the trade offs, we compare our modular approach against a Dual LLM baseline that performs detection and judgment using different language models.

Title: TrumorGPT: Graph-Based Retrieval-Augmented Large Language Model for Fact-Checking

• Authors: Ching Nam Hang, Pei-Duo Yu, Chee Wei Tan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.07891
• Pdf URL: https://arxiv.org/pdf/2505.07891
• Copy Paste: [[2505.07891]] TrumorGPT: Graph-Based Retrieval-Augmented Large Language Model for Fact-Checking(https://arxiv.org/abs/2505.07891)
• Keywords: generative
• Abstract: In the age of social media, the rapid spread of misinformation and rumors has led to the emergence of infodemics, where false information poses a significant threat to society. To combat this issue, we introduce TrumorGPT , a novel generative artificial intelligence solution designed for fact-checking in the health domain. TrumorGPT aims to distinguish "trumors", which are health-related rumors that turn out to be true, providing a crucial tool in differentiating between mere speculation and verified facts. This framework leverages a large language model (LLM) with few-shot learning for semantic health knowledge graph construction and semantic reasoning. TrumorGPT incorporates graph-based retrieval-augmented generation (GraphRAG) to address the hallucination issue common in LLMs and the limitations of static training data. GraphRAG involves accessing and utilizing information from regularly updated semantic health knowledge graphs that consist of the latest medical news and health information, ensuring that fact-checking by TrumorGPT is based on the most recent data. Evaluating with extensive healthcare datasets, TrumorGPT demonstrates superior performance in fact-checking for public health claims. Its ability to effectively conduct fact-checking across various platforms marks a critical step forward in the fight against health-related misinformation, enhancing trust and accuracy in the digital information age.

Title: Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting

• Authors: Minh-Duc Nguyen, Hyung-Jeong Yang, Soo-Hyung Kim, Ji-Eun Shin, Seung-Won Kim
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.07901
• Pdf URL: https://arxiv.org/pdf/2505.07901
• Copy Paste: [[2505.07901]] Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting(https://arxiv.org/abs/2505.07901)
• Keywords: diffusion
• Abstract: The dyadic reaction generation task involves synthesizing responsive facial reactions that align closely with the behaviors of a conversational partner, enhancing the naturalness and effectiveness of human-like interaction simulations. This paper introduces a novel approach, the Latent Behavior Diffusion Model, comprising a context-aware autoencoder and a diffusion-based conditional generator that addresses the challenge of generating diverse and contextually relevant facial reactions from input speaker behaviors. The autoencoder compresses high-dimensional input features, capturing dynamic patterns in listener reactions while condensing complex input data into a concise latent representation, facilitating more expressive and contextually appropriate reaction synthesis. The diffusion-based conditional generator operates on the latent space generated by the autoencoder to predict realistic facial reactions in a non-autoregressive manner. This approach allows for generating diverse facial reactions that reflect subtle variations in conversational cues and emotional states. Experimental results demonstrate the effectiveness of our approach in achieving superior performance in dyadic reaction synthesis tasks compared to existing methods.

Title: Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review

• Authors: Chengmin Zhou, Ville Kyrki, Pasi Fränti, Laura Ruotsalainen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.07911
• Pdf URL: https://arxiv.org/pdf/2505.07911
• Copy Paste: [[2505.07911]] Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review(https://arxiv.org/abs/2505.07911)
• Keywords: generative
• Abstract: Bayesian inference has many advantages in decision making of agents (e.g. robotics/simulative agent) over a regular data-driven black-box neural network: Data-efficiency, generalization, interpretability, and safety where these advantages benefit directly/indirectly from the uncertainty quantification of Bayesian inference. However, there are few comprehensive reviews to summarize the progress of Bayesian inference on reinforcement learning (RL) for decision making to give researchers a systematic understanding. This paper focuses on combining Bayesian inference with RL that nowadays is an important approach in agent decision making. To be exact, this paper discusses the following five topics: 1) Bayesian methods that have potential for agent decision making. First basic Bayesian methods and models (Bayesian rule, Bayesian learning, and Bayesian conjugate models) are discussed followed by variational inference, Bayesian optimization, Bayesian deep learning, Bayesian active learning, Bayesian generative models, Bayesian meta-learning, and lifelong Bayesian learning. 2) Classical combinations of Bayesian methods with model-based RL (with approximation methods), model-free RL, and inverse RL. 3) Latest combinations of potential Bayesian methods with RL. 4) Analytical comparisons of methods that combine Bayesian methods with RL with respect to data-efficiency, generalization, interpretability, and safety. 5) In-depth discussions in six complex problem variants of RL, including unknown reward, partial-observability, multi-agent, multi-task, non-linear non-Gaussian, and hierarchical RL problems and the summary of how Bayesian methods work in the data collection, data processing and policy learning stages of RL to pave the way for better agent decision-making strategies.

Title: Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks

• Authors: Thomas R. Harvey, Fabian Ruehle, Cristofero S. Fraser-Taliente, James Halverson
• Subjects: cs.LG, cs.NE, cs.SC
• Abstract URL: https://arxiv.org/abs/2505.07956
• Pdf URL: https://arxiv.org/pdf/2505.07956
• Copy Paste: [[2505.07956]] Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks(https://arxiv.org/abs/2505.07956)
• Keywords: generative
• Abstract: We present a novel approach to symbolic regression using vision-capable large language models (LLMs) and the ideas behind Google DeepMind's Funsearch. The LLM is given a plot of a univariate function and tasked with proposing an ansatz for that function. The free parameters of the ansatz are fitted using standard numerical optimisers, and a collection of such ansätze make up the population of a genetic algorithm. Unlike other symbolic regression techniques, our method does not require the specification of a set of functions to be used in regression, but with appropriate prompt engineering, we can arbitrarily condition the generative step. By using Kolmogorov Arnold Networks (KANs), we demonstrate that ``univariate is all you need'' for symbolic regression, and extend this method to multivariate functions by learning the univariate function on each edge of a trained KAN. The combined expression is then simplified by further processing with a language model.

Title: Task-Adaptive Semantic Communications with Controllable Diffusion-based Data Regeneration

• Authors: Fupei Guo, Achintha Wijesinghe, Songyang Zhang, Zhi Ding
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.07980
• Pdf URL: https://arxiv.org/pdf/2505.07980
• Copy Paste: [[2505.07980]] Task-Adaptive Semantic Communications with Controllable Diffusion-based Data Regeneration(https://arxiv.org/abs/2505.07980)
• Keywords: diffusion
• Abstract: Semantic communications represent a new paradigm of next-generation networking that shifts bit-wise data delivery to conveying the semantic meanings for bandwidth efficiency. To effectively accommodate various potential downstream tasks at the receiver side, one should adaptively convey the most critical semantic information. This work presents a novel task-adaptive semantic communication framework based on diffusion models that is capable of dynamically adjusting the semantic message delivery according to various downstream tasks. Specifically, we initialize the transmission of a deep-compressed general semantic representation from the transmitter to enable diffusion-based coarse data reconstruction at the receiver. The receiver identifies the task-specific demands and generates textual prompts as feedback. Integrated with the attention mechanism, the transmitter updates the semantic transmission with more details to better align with the objectives of the intended receivers. Our test results demonstrate the efficacy of the proposed method in adaptively preserving critical task-relevant information for semantic communications while preserving high compression efficiency.

Title: Vision Foundation Model Embedding-Based Semantic Anomaly Detection

• Authors: Max Peter Ronecker, Matthew Foutter, Amine Elhafsi, Daniele Gammelli, Ihor Barakaiev, Marco Pavone, Daniel Watzenig
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.07998
• Pdf URL: https://arxiv.org/pdf/2505.07998
• Copy Paste: [[2505.07998]] Vision Foundation Model Embedding-Based Semantic Anomaly Detection(https://arxiv.org/abs/2505.07998)
• Keywords: foundation model, anomaly
• Abstract: Semantic anomalies are contextually invalid or unusual combinations of familiar visual elements that can cause undefined behavior and failures in system-level reasoning for autonomous systems. This work explores semantic anomaly detection by leveraging the semantic priors of state-of-the-art vision foundation models, operating directly on the image. We propose a framework that compares local vision embeddings from runtime images to a database of nominal scenarios in which the autonomous system is deemed safe and performant. In this work, we consider two variants of the proposed framework: one using raw grid-based embeddings, and another leveraging instance segmentation for object-centric representations. To further improve robustness, we introduce a simple filtering mechanism to suppress false positives. Our evaluations on CARLA-simulated anomalies show that the instance-based method with filtering achieves performance comparable to GPT-4o, while providing precise anomaly localization. These results highlight the potential utility of vision embeddings from foundation models for real-time anomaly detection in autonomous systems.

Title: Fréchet Power-Scenario Distance: A Metric for Evaluating Generative AI Models across Multiple Time-Scales in Smart Grids

• Authors: Yuting Cai, Shaohuai Liu, Chao Tian, Le Xie
• Subjects: cs.LG, cs.AI, cs.CV, eess.SP
• Abstract URL: https://arxiv.org/abs/2505.08082
• Pdf URL: https://arxiv.org/pdf/2505.08082
• Copy Paste: [[2505.08082]] Fréchet Power-Scenario Distance: A Metric for Evaluating Generative AI Models across Multiple Time-Scales in Smart Grids(https://arxiv.org/abs/2505.08082)
• Keywords: generative
• Abstract: Generative artificial intelligence (AI) models in smart grids have advanced significantly in recent years due to their ability to generate large amounts of synthetic data, which would otherwise be difficult to obtain in the real world due to confidentiality constraints. A key challenge in utilizing such synthetic data is how to assess the data quality produced from such generative models. Traditional Euclidean distance-based metrics only reflect pair-wise relations between two individual samples, and could fail in evaluating quality differences between groups of synthetic datasets. In this work, we propose a novel metric based on the Fréchet Distance (FD) estimated between two datasets in a learned feature space. The proposed method evaluates the quality of generation from a distributional perspective. Empirical results demonstrate the superiority of the proposed metric across timescales and models, enhancing the reliability of data-driven decision-making in smart grid operations.

Title: A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem

• Authors: Sunday Oyinlola Ogundoyin, Muhammad Ikram, Hassan Jameel Asghar, Benjamin Zi Hao Zhao, Dali Kaafar
• Subjects: cs.CR, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08148
• Pdf URL: https://arxiv.org/pdf/2505.08148
• Copy Paste: [[2505.08148]] A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem(https://arxiv.org/abs/2505.08148)
• Keywords: generative
• Abstract: Millions of users leverage generative pretrained transformer (GPT)-based language models developed by leading model providers for a wide range of tasks. To support enhanced user interaction and customization, many platforms-such as OpenAI-now enable developers to create and publish tailored model instances, known as custom GPTs, via dedicated repositories or application stores. These custom GPTs empower users to browse and interact with specialized applications designed to meet specific needs. However, as custom GPTs see growing adoption, concerns regarding their security vulnerabilities have intensified. Existing research on these vulnerabilities remains largely theoretical, often lacking empirical, large-scale, and statistically rigorous assessments of associated risks. In this study, we analyze 14,904 custom GPTs to assess their susceptibility to seven exploitable threats, such as roleplay-based attacks, system prompt leakage, phishing content generation, and malicious code synthesis, across various categories and popularity tiers within the OpenAI marketplace. We introduce a multi-metric ranking system to examine the relationship between a custom GPT's popularity and its associated security risks. Our findings reveal that over 95% of custom GPTs lack adequate security protections. The most prevalent vulnerabilities include roleplay-based vulnerabilities (96.51%), system prompt leakage (92.20%), and phishing (91.22%). Furthermore, we demonstrate that OpenAI's foundational models exhibit inherent security weaknesses, which are often inherited or amplified in custom GPTs. These results highlight the urgent need for enhanced security measures and stricter content moderation to ensure the safe deployment of GPT-based applications.

Title: DSADF: Thinking Fast and Slow for Decision Making

• Authors: Alex Zhihao Dou, Dongfei Cui, Jun Yan, Weida Wang, Benteng Chen, Haoming Wang, Zeke Xie, Shufei Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.08189
• Pdf URL: https://arxiv.org/pdf/2505.08189
• Copy Paste: [[2505.08189]] DSADF: Thinking Fast and Slow for Decision Making(https://arxiv.org/abs/2505.08189)
• Keywords: foundation model
• Abstract: Although Reinforcement Learning (RL) agents are effective in well-defined environments, they often struggle to generalize their learned policies to dynamic settings due to their reliance on trial-and-error interactions. Recent work has explored applying Large Language Models (LLMs) or Vision Language Models (VLMs) to boost the generalization of RL agents through policy optimization guidance or prior knowledge. However, these approaches often lack seamless coordination between the RL agent and the foundation model, leading to unreasonable decision-making in unfamiliar environments and efficiency bottlenecks. Making full use of the inferential capabilities of foundation models and the rapid response capabilities of RL agents and enhancing the interaction between the two to form a dual system is still a lingering scientific question. To address this problem, we draw inspiration from Kahneman's theory of fast thinking (System 1) and slow thinking (System 2), demonstrating that balancing intuition and deep reasoning can achieve nimble decision-making in a complex world. In this study, we propose a Dual-System Adaptive Decision Framework (DSADF), integrating two complementary modules: System 1, comprising an RL agent and a memory space for fast and intuitive decision making, and System 2, driven by a VLM for deep and analytical reasoning. DSADF facilitates efficient and adaptive decision-making by combining the strengths of both systems. The empirical study in the video game environment: Crafter and Housekeep demonstrates the effectiveness of our proposed method, showing significant improvements in decision abilities for both unseen and known tasks.

Title: Unsupervised Raindrop Removal from a Single Image using Conditional Diffusion Models

• Authors: Lhuqita Fazry, Valentino Vito
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08190
• Pdf URL: https://arxiv.org/pdf/2505.08190
• Copy Paste: [[2505.08190]] Unsupervised Raindrop Removal from a Single Image using Conditional Diffusion Models(https://arxiv.org/abs/2505.08190)
• Keywords: diffusion, generative
• Abstract: Raindrop removal is a challenging task in image processing. Removing raindrops while relying solely on a single image further increases the difficulty of the task. Common approaches include the detection of raindrop regions in the image, followed by performing a background restoration process conditioned on those regions. While various methods can be applied for the detection step, the most common architecture used for background restoration is the Generative Adversarial Network (GAN). Recent advances in the use of diffusion models have led to state-of-the-art image inpainting techniques. In this paper, we introduce a novel technique for raindrop removal from a single image using diffusion-based image inpainting.

Title: Visual Watermarking in the Era of Diffusion Models: Advances and Challenges

• Authors: Junxian Duan, Jiyang Guang, Wenkui Yang, Ran He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08197
• Pdf URL: https://arxiv.org/pdf/2505.08197
• Copy Paste: [[2505.08197]] Visual Watermarking in the Era of Diffusion Models: Advances and Challenges(https://arxiv.org/abs/2505.08197)
• Keywords: diffusion, generative
• Abstract: As generative artificial intelligence technologies like Stable Diffusion advance, visual content becomes more vulnerable to misuse, raising concerns about copyright infringement. Visual watermarks serve as effective protection mechanisms, asserting ownership and deterring unauthorized use. Traditional deepfake detection methods often rely on passive techniques that struggle with sophisticated manipulations. In contrast, diffusion models enhance detection accuracy by allowing for the effective learning of features, enabling the embedding of imperceptible and robust watermarks. We analyze the strengths and challenges of watermark techniques related to diffusion models, focusing on their robustness and application in watermark generation. By exploring the integration of advanced diffusion models and watermarking security, we aim to advance the discourse on preserving watermark robustness against evolving forgery threats. It emphasizes the critical importance of developing innovative solutions to protect digital content and ensure the preservation of ownership rights in the era of generative AI.

Title: Deep Probabilistic Modeling of User Behavior for Anomaly Detection via Mixture Density Networks

• Authors: Lu Dai, Wenxuan Zhu, Xuehui Quan, Renzi Meng, Sheng Cai, Yichen Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08220
• Pdf URL: https://arxiv.org/pdf/2505.08220
• Copy Paste: [[2505.08220]] Deep Probabilistic Modeling of User Behavior for Anomaly Detection via Mixture Density Networks(https://arxiv.org/abs/2505.08220)
• Keywords: anomaly
• Abstract: To improve the identification of potential anomaly patterns in complex user behavior, this paper proposes an anomaly detection method based on a deep mixture density network. The method constructs a Gaussian mixture model parameterized by a neural network, enabling conditional probability modeling of user behavior. It effectively captures the multimodal distribution characteristics commonly present in behavioral data. Unlike traditional classifiers that rely on fixed thresholds or a single decision boundary, this approach defines an anomaly scoring function based on probability density using negative log-likelihood. This significantly enhances the model's ability to detect rare and unstructured behaviors. Experiments are conducted on the real-world network user dataset UNSW-NB15. A series of performance comparisons and stability validation experiments are designed. These cover multiple evaluation aspects, including Accuracy, F1- score, AUC, and loss fluctuation. The results show that the proposed method outperforms several advanced neural network architectures in both performance and training stability. This study provides a more expressive and discriminative solution for user behavior modeling and anomaly detection. It strongly promotes the application of deep probabilistic modeling techniques in the fields of network security and intelligent risk control.

Title: Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix

• Authors: Unai Gurbindo, Axel Brando, Jaume Abella, Caroline König
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.08228
• Pdf URL: https://arxiv.org/pdf/2505.08228
• Copy Paste: [[2505.08228]] Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix(https://arxiv.org/abs/2505.08228)
• Keywords: diffusion
• Abstract: Enhancing the robustness of object detection systems under adverse weather conditions is crucial for the advancement of autonomous driving technology. This study presents a novel approach leveraging the diffusion model Instruct Pix2Pix to develop prompting methodologies that generate realistic datasets with weather-based augmentations aiming to mitigate the impact of adverse weather on the perception capabilities of state-of-the-art object detection models, including Faster R-CNN and YOLOv10. Experiments were conducted in two environments, in the CARLA simulator where an initial evaluation of the proposed data augmentation was provided, and then on the real-world image data sets BDD100K and ACDC demonstrating the effectiveness of the approach in real environments. The key contributions of this work are twofold: (1) identifying and quantifying the performance gap in object detection models under challenging weather conditions, and (2) demonstrating how tailored data augmentation strategies can significantly enhance the robustness of these models. This research establishes a solid foundation for improving the reliability of perception systems in demanding environmental scenarios, and provides a pathway for future advancements in autonomous driving.

Title: Removing Watermarks with Partial Regeneration using Semantic Information

• Authors: Krti Tallam, John Kevin Cava, Caleb Geniesse, N. Benjamin Erichson, Michael W. Mahoney
• Subjects: cs.CV, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.08234
• Pdf URL: https://arxiv.org/pdf/2505.08234
• Copy Paste: [[2505.08234]] Removing Watermarks with Partial Regeneration using Semantic Information(https://arxiv.org/abs/2505.08234)
• Keywords: diffusion, generative
• Abstract: As AI-generated imagery becomes ubiquitous, invisible watermarks have emerged as a primary line of defense for copyright and provenance. The newest watermarking schemes embed semantic signals - content-aware patterns that are designed to survive common image manipulations - yet their true robustness against adaptive adversaries remains under-explored. We expose a previously unreported vulnerability and introduce SemanticRegen, a three-stage, label-free attack that erases state-of-the-art semantic and invisible watermarks while leaving an image's apparent meaning intact. Our pipeline (i) uses a vision-language model to obtain fine-grained captions, (ii) extracts foreground masks with zero-shot segmentation, and (iii) inpaints only the background via an LLM-guided diffusion model, thereby preserving salient objects and style cues. Evaluated on 1,000 prompts across four watermarking systems - TreeRing, StegaStamp, StableSig, and DWT/DCT - SemanticRegen is the only method to defeat the semantic TreeRing watermark (p = 0.10 > 0.05) and reduces bit-accuracy below 0.75 for the remaining schemes, all while maintaining high perceptual quality (masked SSIM = 0.94 +/- 0.01). We further introduce masked SSIM (mSSIM) to quantify fidelity within foreground regions, showing that our attack achieves up to 12 percent higher mSSIM than prior diffusion-based attackers. These results highlight an urgent gap between current watermark defenses and the capabilities of adaptive, semantics-aware adversaries, underscoring the need for watermarking algorithms that are resilient to content-preserving regenerative attacks.

Title: EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation

• Authors: Hanle Zheng, Xujie Han, Zegang Peng, Shangbin Zhang, Guangxun Du, Zhuo Zou, Xilin Wang, Jibin Wu, Hao Guo, Lei Deng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08235
• Pdf URL: https://arxiv.org/pdf/2505.08235
• Copy Paste: [[2505.08235]] EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation(https://arxiv.org/abs/2505.08235)
• Keywords: diffusion
• Abstract: Video Frame Interpolation (VFI) is a fundamental yet challenging task in computer vision, particularly under conditions involving large motion, occlusion, and lighting variation. Recent advancements in event cameras have opened up new opportunities for addressing these challenges. While existing event-based VFI methods have succeeded in recovering large and complex motions by leveraging handcrafted intermediate representations such as optical flow, these designs often compromise high-fidelity image reconstruction under subtle motion scenarios due to their reliance on explicit motion modeling. Meanwhile, diffusion models provide a promising alternative for VFI by reconstructing frames through a denoising process, eliminating the need for explicit motion estimation or warping operations. In this work, we propose EventDiff, a unified and efficient event-based diffusion model framework for VFI. EventDiff features a novel Event-Frame Hybrid AutoEncoder (HAE) equipped with a lightweight Spatial-Temporal Cross Attention (STCA) module that effectively fuses dynamic event streams with static frames. Unlike previous event-based VFI methods, EventDiff performs interpolation directly in the latent space via a denoising diffusion process, making it more robust across diverse and challenging VFI scenarios. Through a two-stage training strategy that first pretrains the HAE and then jointly optimizes it with the diffusion model, our method achieves state-of-the-art performance across multiple synthetic and real-world event VFI datasets. The proposed method outperforms existing state-of-the-art event-based VFI methods by up to 1.98dB in PSNR on Vimeo90K-Triplet and shows superior performance in SNU-FILM tasks with multiple difficulty levels. Compared to the emerging diffusion-based VFI approach, our method achieves up to 5.72dB PSNR gain on Vimeo90K-Triplet and 4.24X faster inference.

Title: Identifying Memorization of Diffusion Models through p-Laplace Analysis

• Authors: Jonathan Brokman, Amit Giloni, Omer Hofman, Roman Vainshtein, Hisashi Kojima, Guy Gilboa
• Subjects: cs.CV, math.NA
• Abstract URL: https://arxiv.org/abs/2505.08246
• Pdf URL: https://arxiv.org/pdf/2505.08246
• Copy Paste: [[2505.08246]] Identifying Memorization of Diffusion Models through p-Laplace Analysis(https://arxiv.org/abs/2505.08246)
• Keywords: diffusion, generative
• Abstract: Diffusion models, today's leading image generative models, estimate the score function, i.e. the gradient of the log probability of (perturbed) data samples, without direct access to the underlying probability distribution. This work investigates whether the estimated score function can be leveraged to compute higher-order differentials, namely p-Laplace operators. We show here these operators can be employed to identify memorized training data. We propose a numerical p-Laplace approximation based on the learned score functions, showing its effectiveness in identifying key features of the probability landscape. We analyze the structured case of Gaussian mixture models, and demonstrate the results carry-over to image generative models, where memorization identification based on the p-Laplace operator is performed for the first time.

Title: Where the Devil Hides: Deepfake Detectors Can No Longer Be Trusted

• Authors: Shuaiwei Yuan, Junyu Dong, Yuezun Li
• Subjects: cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08255
• Pdf URL: https://arxiv.org/pdf/2505.08255
• Copy Paste: [[2505.08255]] Where the Devil Hides: Deepfake Detectors Can No Longer Be Trusted(https://arxiv.org/abs/2505.08255)
• Keywords: generative
• Abstract: With the advancement of AI generative techniques, Deepfake faces have become incredibly realistic and nearly indistinguishable to the human eye. To counter this, Deepfake detectors have been developed as reliable tools for assessing face authenticity. These detectors are typically developed on Deep Neural Networks (DNNs) and trained using third-party datasets. However, this protocol raises a new security risk that can seriously undermine the trustfulness of Deepfake detectors: Once the third-party data providers insert poisoned (corrupted) data maliciously, Deepfake detectors trained on these datasets will be injected ``backdoors'' that cause abnormal behavior when presented with samples containing specific triggers. This is a practical concern, as third-party providers may distribute or sell these triggers to malicious users, allowing them to manipulate detector performance and escape accountability. This paper investigates this risk in depth and describes a solution to stealthily infect Deepfake detectors. Specifically, we develop a trigger generator, that can synthesize passcode-controlled, semantic-suppression, adaptive, and invisible trigger patterns, ensuring both the stealthiness and effectiveness of these triggers. Then we discuss two poisoning scenarios, dirty-label poisoning and clean-label poisoning, to accomplish the injection of backdoors. Extensive experiments demonstrate the effectiveness, stealthiness, and practicality of our method compared to several baselines.

Title: Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion

• Authors: Anle Ke, Xu Zhang, Tong Chen, Ming Lu, Chao Zhou, Jiawen Gu, Zhan Ma
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.08281
• Pdf URL: https://arxiv.org/pdf/2505.08281
• Copy Paste: [[2505.08281]] Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion(https://arxiv.org/abs/2505.08281)
• Keywords: diffusion, generative
• Abstract: Existing multimodal large model-based image compression frameworks often rely on a fragmented integration of semantic retrieval, latent compression, and generative models, resulting in suboptimal performance in both reconstruction fidelity and coding efficiency. To address these challenges, we propose a residual-guided ultra lowrate image compression named ResULIC, which incorporates residual signals into both semantic retrieval and the diffusion-based generation process. Specifically, we introduce Semantic Residual Coding (SRC) to capture the semantic disparity between the original image and its compressed latent representation. A perceptual fidelity optimizer is further applied for superior reconstruction quality. Additionally, we present the Compression-aware Diffusion Model (CDM), which establishes an optimal alignment between bitrates and diffusion time steps, improving compression-reconstruction synergy. Extensive experiments demonstrate the effectiveness of ResULIC, achieving superior objective and subjective performance compared to state-of-the-art diffusion-based methods with - 80.7%, -66.3% BD-rate saving in terms of LPIPS and FID. Project page is available at https: //njuvision.this http URL.

Title: FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units

• Authors: Jian Wang, Baoyuan Wu, Li Liu, Qingshan Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08294
• Pdf URL: https://arxiv.org/pdf/2505.08294
• Copy Paste: [[2505.08294]] FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units(https://arxiv.org/abs/2505.08294)
• Keywords: generative
• Abstract: The rapid evolution of generative AI has increased the threat of realistic audio-visual deepfakes, demanding robust detection methods. Existing solutions primarily address unimodal (audio or visual) forgeries but struggle with multimodal manipulations due to inadequate handling of heterogeneous modality features and poor generalization across datasets. To this end, we propose a novel framework called FauForensics by introducing biologically invariant facial action units (FAUs), which is a quantitative descriptor of facial muscle activity linked to emotion physiology. It serves as forgery-resistant representations that reduce domain dependency while capturing subtle dynamics often disrupted in synthetic content. Besides, instead of comparing entire video clips as in prior works, our method computes fine-grained frame-wise audiovisual similarities via a dedicated fusion module augmented with learnable cross-modal queries. It dynamically aligns temporal-spatial lip-audio relationships while mitigating multi-modal feature heterogeneity issues. Experiments on FakeAVCeleb and LAV-DF show state-of-the-art (SOTA) performance and superior cross-dataset generalizability with up to an average of 4.83\% than existing methods.

Title: Structural-Temporal Coupling Anomaly Detection with Dynamic Graph Transformer

• Authors: Chang Zong, Yueting Zhuang, Jian Shao, Weiming Lu
• Subjects: cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2505.08330
• Pdf URL: https://arxiv.org/pdf/2505.08330
• Copy Paste: [[2505.08330]] Structural-Temporal Coupling Anomaly Detection with Dynamic Graph Transformer(https://arxiv.org/abs/2505.08330)
• Keywords: anomaly
• Abstract: Detecting anomalous edges in dynamic graphs is an important task in many applications over evolving triple-based data, such as social networks, transaction management, and epidemiology. A major challenge with this task is the absence of structural-temporal coupling information, which decreases the ability of the representation to distinguish anomalies from normal instances. Existing methods focus on handling independent structural and temporal features with embedding models, which ignore the deep interaction between these two types of information. In this paper, we propose a structural-temporal coupling anomaly detection architecture with a dynamic graph transformer model. Specifically, we introduce structural and temporal features from two integration levels to provide anomaly-aware graph evolutionary patterns. Then, a dynamic graph transformer enhanced by two-dimensional positional encoding is implemented to capture both discrimination and contextual consistency signals. Extensive experiments on six datasets demonstrate that our method outperforms current state-of-the-art models. Finally, a case study illustrates the strength of our method when applied to a real-world task.

Title: ConDiSim: Conditional Diffusion Models for Simulation Based Inference

• Authors: Mayank Nautiyal, Andreas Hellander, Prashant Singh
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.08403
• Pdf URL: https://arxiv.org/pdf/2505.08403
• Copy Paste: [[2505.08403]] ConDiSim: Conditional Diffusion Models for Simulation Based Inference(https://arxiv.org/abs/2505.08403)
• Keywords: diffusion
• Abstract: We present a conditional diffusion model - ConDiSim, for simulation-based inference of complex systems with intractable likelihoods. ConDiSim leverages denoising diffusion probabilistic models to approximate posterior distributions, consisting of a forward process that adds Gaussian noise to parameters, and a reverse process learning to denoise, conditioned on observed data. This approach effectively captures complex dependencies and multi-modalities within posteriors. ConDiSim is evaluated across ten benchmark problems and two real-world test problems, where it demonstrates effective posterior approximation accuracy while maintaining computational efficiency and stability in model training. ConDiSim offers a robust and extensible framework for simulation-based inference, particularly suitable for parameter inference workflows requiring fast inference methods.

Title: Visual Image Reconstruction from Brain Activity via Latent Representation

• Authors: Yukiyasu Kamitani, Misato Tanaka, Ken Shirakawa
• Subjects: cs.CV, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2505.08429
• Pdf URL: https://arxiv.org/pdf/2505.08429
• Copy Paste: [[2505.08429]] Visual Image Reconstruction from Brain Activity via Latent Representation(https://arxiv.org/abs/2505.08429)
• Keywords: generative
• Abstract: Visual image reconstruction, the decoding of perceptual content from brain activity into images, has advanced significantly with the integration of deep neural networks (DNNs) and generative models. This review traces the field's evolution from early classification approaches to sophisticated reconstructions that capture detailed, subjective visual experiences, emphasizing the roles of hierarchical latent representations, compositional strategies, and modular architectures. Despite notable progress, challenges remain, such as achieving true zero-shot generalization for unseen images and accurately modeling the complex, subjective aspects of perception. We discuss the need for diverse datasets, refined evaluation metrics aligned with human perceptual judgments, and compositional representations that strengthen model robustness and generalizability. Ethical issues, including privacy, consent, and potential misuse, are underscored as critical considerations for responsible development. Visual image reconstruction offers promising insights into neural coding and enables new psychological measurements of visual experiences, with applications spanning clinical diagnostics and brain-machine interfaces.

Title: TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection

• Authors: Wenkui Yang, Zhida Zhang, Xiaoqiang Zhou, Junxian Duan, Jie Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08437
• Pdf URL: https://arxiv.org/pdf/2505.08437
• Copy Paste: [[2505.08437]] TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection(https://arxiv.org/abs/2505.08437)
• Keywords: diffusion, generative
• Abstract: The emergence and popularity of facial deepfake methods spur the vigorous development of deepfake datasets and facial forgery detection, which to some extent alleviates the security concerns about facial-related artificial intelligence technologies. However, when it comes to human body forgery, there has been a persistent lack of datasets and detection methods, due to the later inception and complexity of human body generation methods. To mitigate this issue, we introduce TikTok-DeepFake (TT-DF), a novel large-scale diffusion-based dataset containing 6,120 forged videos with 1,378,857 synthetic frames, specifically tailored for body forgery detection. TT-DF offers a wide variety of forgery methods, involving multiple advanced human image animation models utilized for manipulation, two generative configurations based on the disentanglement of identity and pose information, as well as different compressed versions. The aim is to simulate any potential unseen forged data in the wild as comprehensively as possible, and we also furnish a benchmark on TT-DF. Additionally, we propose an adapted body forgery detection model, Temporal Optical Flow Network (TOF-Net), which exploits the spatiotemporal inconsistencies and optical flow distribution differences between natural data and forged data. Our experiments demonstrate that TOF-Net achieves favorable performance on TT-DF, outperforming current state-of-the-art extendable facial forgery detection models. For our TT-DF dataset, please refer to this https URL.

Title: IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation

• Authors: Kazuki Hayashi, Hidetaka Kamigaito, Shinya Kouda, Taro Watanabe
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.08450
• Pdf URL: https://arxiv.org/pdf/2505.08450
• Copy Paste: [[2505.08450]] IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation(https://arxiv.org/abs/2505.08450)
• Keywords: in-context
• Abstract: Retrieval-Augmented Generation (RAG) has emerged as a way to complement the in-context knowledge of Large Language Models (LLMs) by integrating external documents. However, real-world applications demand not only accuracy but also interpretability. While dense retrieval methods provide high accuracy, they lack interpretability; conversely, sparse retrieval methods offer transparency but often fail to capture the full intent of queries due to their reliance on keyword matching. To address these issues, we introduce IterKey, an LLM-driven iterative keyword generation framework that enhances RAG via sparse retrieval. IterKey consists of three LLM-driven stages: generating keywords for retrieval, generating answers based on retrieved documents, and validating the answers. If validation fails, the process iteratively repeats with refined keywords. Across four QA tasks, experimental results show that IterKey achieves 5% to 20% accuracy improvements over BM25-based RAG and simple baselines. Its performance is comparable to dense retrieval-based RAG and prior iterative query refinement methods using dense models. In summary, IterKey is a novel BM25-based approach leveraging LLMs to iteratively refine RAG, effectively balancing accuracy with interpretability.

Title: Isolation Forest in Novelty Detection Scenario

• Authors: Adam Ulrich, Jan Krňávek, Roman Šenkeřík, Zuzana Komínková Oplatková, Radek Vala
• Subjects: cs.LG, cs.DM
• Abstract URL: https://arxiv.org/abs/2505.08489
• Pdf URL: https://arxiv.org/pdf/2505.08489
• Copy Paste: [[2505.08489]] Isolation Forest in Novelty Detection Scenario(https://arxiv.org/abs/2505.08489)
• Keywords: anomaly
• Abstract: Data mining offers a diverse toolbox for extracting meaningful structures from complex datasets, with anomaly detection emerging as a critical subfield particularly in the context of streaming or real-time data. Within anomaly detection, novelty detection focuses on identifying previously unseen patterns after training solely on regular data. While classic algorithms such as One-Class SVM or Local Outlier Factor (LOF) have been widely applied, they often lack interpretability and scalability. In this work, we explore the Half-Space Tree (HST) algorithm, originally proposed for streaming anomaly detection, and propose a novel theoretical modification to adapt it specifically for novelty detection tasks. Our approach is grounded in the idea that anomalies i.e., novelties tend to appear in the higher leaves of the tree, which are less frequently visited by regular instances. We analytically demonstrate the effectiveness of this approach using probabilistic analysis, expected depth (EXD) calculations, and combinatorial reasoning. A comparative analysis of expected depths between our modified HST and the original Isolation Forest highlights that novelty points are significantly more isolated in our approach. This supports the hypothesis that HSTs, with appropriate structural adaptation, can serve as interpretable and efficient novelty detectors. The paper contributes a theoretical foundation and supporting analysis for this adaptation, setting the stage for further application and experimentation.

Title: A Deep Learning-Driven Framework for Inhalation Injury Grading Using Bronchoscopy Images

• Authors: Yifan Li, Alan W Pang, Jo Woon Chong
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08517
• Pdf URL: https://arxiv.org/pdf/2505.08517
• Copy Paste: [[2505.08517]] A Deep Learning-Driven Framework for Inhalation Injury Grading Using Bronchoscopy Images(https://arxiv.org/abs/2505.08517)
• Keywords: generative
• Abstract: Inhalation injuries face a challenge in clinical diagnosis and grading due to the limitations of traditional methods, such as Abbreviated Injury Score (AIS), which rely on subjective assessments and show weak correlations with clinical outcomes. This study introduces a novel deep learning-based framework for grading inhalation injuries using bronchoscopy images with the duration of mechanical ventilation as an objective metric. To address the scarcity of medical imaging data, we propose enhanced StarGAN, a generative model that integrates Patch Loss and SSIM Loss to improve synthetic images' quality and clinical relevance. The augmented dataset generated by enhanced StarGAN significantly improved classification performance when evaluated using the Swin Transformer, achieving an accuracy of 77.78%, an 11.11% improvement over the original dataset. Image quality was assessed using the Fréchet Inception Distance (FID), where Enhanced StarGAN achieved the lowest FID of 30.06, outperforming baseline models. Burn surgeons confirmed the realism and clinical relevance of the generated images, particularly the preservation of bronchial structures and color distribution. These results highlight the potential of enhanced StarGAN in addressing data limitations and improving classification accuracy for inhalation injury grading.

Title: Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysis

• Authors: Pratibha Kumari, Daniel Reisenbüchler, Afshin Bozorgpour, Nadine S. Schaadt, Friedrich Feuerhake, Dorit Merhof
• Subjects: cs.CV, cs.ET
• Abstract URL: https://arxiv.org/abs/2505.08524
• Pdf URL: https://arxiv.org/pdf/2505.08524
• Copy Paste: [[2505.08524]] Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysis(https://arxiv.org/abs/2505.08524)
• Keywords: generative
• Abstract: Whole slide image (WSI) classification has emerged as a powerful tool in computational pathology, but remains constrained by domain shifts, e.g., due to different organs, diseases, or institution-specific variations. To address this challenge, we propose an Attention-based Generative Latent Replay Continual Learning framework (AGLR-CL), in a multiple instance learning (MIL) setup for domain incremental WSI classification. Our method employs Gaussian Mixture Models (GMMs) to synthesize WSI representations and patch count distributions, preserving knowledge of past domains without explicitly storing original data. A novel attention-based filtering step focuses on the most salient patch embeddings, ensuring high-quality synthetic samples. This privacy-aware strategy obviates the need for replay buffers and outperforms other buffer-free counterparts while matching the performance of buffer-based solutions. We validate AGLR-CL on clinically relevant biomarker detection and molecular status prediction across multiple public datasets with diverse centers, organs, and patient cohorts. Experimental results confirm its ability to retain prior knowledge and adapt to new domains, offering an effective, privacy-preserving avenue for domain incremental continual learning in WSI classification.

Title: ExEBench: Benchmarking Foundation Models on Extreme Earth Events

• Authors: Shan Zhao, Zhitong Xiong, Jie Zhao, Xiao Xiang Zhu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.08529
• Pdf URL: https://arxiv.org/pdf/2505.08529
• Copy Paste: [[2505.08529]] ExEBench: Benchmarking Foundation Models on Extreme Earth Events(https://arxiv.org/abs/2505.08529)
• Keywords: foundation model
• Abstract: Our planet is facing increasingly frequent extreme events, which pose major risks to human lives and ecosystems. Recent advances in machine learning (ML), especially with foundation models (FMs) trained on extensive datasets, excel in extracting features and show promise in disaster management. Nevertheless, these models often inherit biases from training data, challenging their performance over extreme values. To explore the reliability of FM in the context of extreme events, we introduce \textbf{ExE}Bench (\textbf{Ex}treme \textbf{E}arth Benchmark), a collection of seven extreme event categories across floods, wildfires, storms, tropical cyclones, extreme precipitation, heatwaves, and cold waves. The dataset features global coverage, varying data volumes, and diverse data sources with different spatial, temporal, and spectral characteristics. To broaden the real-world impact of FMs, we include multiple challenging ML tasks that are closely aligned with operational needs in extreme events detection, monitoring, and forecasting. ExEBench aims to (1) assess FM generalizability across diverse, high-impact tasks and domains, (2) promote the development of novel ML methods that benefit disaster management, and (3) offer a platform for analyzing the interactions and cascading effects of extreme events to advance our understanding of Earth system, especially under the climate change expected in the decades to come. The dataset and code are public this https URL.

Title: DFA-CON: A Contrastive Learning Approach for Detecting Copyright Infringement in DeepFake Art

• Authors: Haroon Wahab, Hassan Ugail, Irfan Mehmood
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08552
• Pdf URL: https://arxiv.org/pdf/2505.08552
• Copy Paste: [[2505.08552]] DFA-CON: A Contrastive Learning Approach for Detecting Copyright Infringement in DeepFake Art(https://arxiv.org/abs/2505.08552)
• Keywords: foundation model, generative
• Abstract: Recent proliferation of generative AI tools for visual content creation-particularly in the context of visual artworks-has raised serious concerns about copyright infringement and forgery. The large-scale datasets used to train these models often contain a mixture of copyrighted and non-copyrighted artworks. Given the tendency of generative models to memorize training patterns, they are susceptible to varying degrees of copyright violation. Building on the recently proposed DeepfakeArt Challenge benchmark, this work introduces DFA-CON, a contrastive learning framework designed to detect copyright-infringing or forged AI-generated art. DFA-CON learns a discriminative representation space, posing affinity among original artworks and their forged counterparts within a contrastive learning framework. The model is trained across multiple attack types, including inpainting, style transfer, adversarial perturbation, and cutmix. Evaluation results demonstrate robust detection performance across most attack types, outperforming recent pretrained foundation models. Code and model checkpoints will be released publicly upon acceptance.

Title: Reinforcement Learning meets Masked Video Modeling : Trajectory-Guided Adaptive Token Selection

• Authors: Ayush K. Rai, Kyle Min, Tarun Krishna, Feiyan Hu, Alan F. Smeaton, Noel E. O'Connor
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08561
• Pdf URL: https://arxiv.org/pdf/2505.08561
• Copy Paste: [[2505.08561]] Reinforcement Learning meets Masked Video Modeling : Trajectory-Guided Adaptive Token Selection(https://arxiv.org/abs/2505.08561)
• Keywords: foundation model
• Abstract: Masked video modeling~(MVM) has emerged as a highly effective pre-training strategy for visual foundation models, whereby the model reconstructs masked spatiotemporal tokens using information from visible tokens. However, a key challenge in such approaches lies in selecting an appropriate masking strategy. Previous studies have explored predefined masking techniques, including random and tube-based masking, as well as approaches that leverage key motion priors, optical flow and semantic cues from externally pre-trained models. In this work, we introduce a novel and generalizable Trajectory-Aware Adaptive Token Sampler (TATS), which models the motion dynamics of tokens and can be seamlessly integrated into the masked autoencoder (MAE) framework to select motion-centric tokens in videos. Additionally, we propose a unified training strategy that enables joint optimization of both MAE and TATS from scratch using Proximal Policy Optimization (PPO). We show that our model allows for aggressive masking without compromising performance on the downstream task of action recognition while also ensuring that the pre-training remains memory efficient. Extensive experiments of the proposed approach across four benchmarks, including Something-Something v2, Kinetics-400, UCF101, and HMDB51, demonstrate the effectiveness, transferability, generalization, and efficiency of our work compared to other state-of-the-art methods.

Title: Enhancing Thyroid Cytology Diagnosis with RAG-Optimized LLMs and Pa-thology Foundation Models

• Authors: Hussien Al-Asi, Jordan P Reynolds, Shweta Agarwal, Bryan J Dangott, Aziza Nassar, Zeynettin Akkus
• Subjects: cs.CL, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2505.08590
• Pdf URL: https://arxiv.org/pdf/2505.08590
• Copy Paste: [[2505.08590]] Enhancing Thyroid Cytology Diagnosis with RAG-Optimized LLMs and Pa-thology Foundation Models(https://arxiv.org/abs/2505.08590)
• Keywords: foundation model
• Abstract: Advancements in artificial intelligence (AI) are transforming pathology by integrat-ing large language models (LLMs) with retrieval-augmented generation (RAG) and domain-specific foundation models. This study explores the application of RAG-enhanced LLMs coupled with pathology foundation models for thyroid cytology diagnosis, addressing challenges in cytological interpretation, standardization, and diagnostic accuracy. By leveraging a curated knowledge base, RAG facilitates dy-namic retrieval of relevant case studies, diagnostic criteria, and expert interpreta-tion, improving the contextual understanding of LLMs. Meanwhile, pathology foun-dation models, trained on high-resolution pathology images, refine feature extrac-tion and classification capabilities. The fusion of these AI-driven approaches en-hances diagnostic consistency, reduces variability, and supports pathologists in dis-tinguishing benign from malignant thyroid lesions. Our results demonstrate that integrating RAG with pathology-specific LLMs significantly improves diagnostic efficiency and interpretability, paving the way for AI-assisted thyroid cytopathology, with foundation model UNI achieving AUC 0.73-0.93 for correct prediction of surgi-cal pathology diagnosis from thyroid cytology samples.

Title: Boosting Zero-shot Stereo Matching using Large-scale Mixed Images Sources in the Real World

• Authors: Yuran Wang, Yingping Liang, Ying Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08607
• Pdf URL: https://arxiv.org/pdf/2505.08607
• Copy Paste: [[2505.08607]] Boosting Zero-shot Stereo Matching using Large-scale Mixed Images Sources in the Real World(https://arxiv.org/abs/2505.08607)
• Keywords: diffusion, foundation model
• Abstract: Stereo matching methods rely on dense pixel-wise ground truth labels, which are laborious to obtain, especially for real-world datasets. The scarcity of labeled data and domain gaps between synthetic and real-world images also pose notable challenges. In this paper, we propose a novel framework, \textbf{BooSTer}, that leverages both vision foundation models and large-scale mixed image sources, including synthetic, real, and single-view images. First, to fully unleash the potential of large-scale single-view images, we design a data generation strategy combining monocular depth estimation and diffusion models to generate dense stereo matching data from single-view images. Second, to tackle sparse labels in real-world datasets, we transfer knowledge from monocular depth estimation models, using pseudo-mono depth labels and a dynamic scale- and shift-invariant loss for additional supervision. Furthermore, we incorporate vision foundation model as an encoder to extract robust and transferable features, boosting accuracy and generalization. Extensive experiments on benchmark datasets demonstrate the effectiveness of our approach, achieving significant improvements in accuracy over existing methods, particularly in scenarios with limited labeled data and domain shifts.

Title: Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing

• Authors: Chen Wu, Yin Song
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08651
• Pdf URL: https://arxiv.org/pdf/2505.08651
• Copy Paste: [[2505.08651]] Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing(https://arxiv.org/abs/2505.08651)
• Keywords: in-context
• Abstract: We present MegaBeam-Mistral-7B, a language model that supports 512K-token context length. Our work addresses practical limitations in long-context training, supporting real-world tasks such as compliance monitoring and verification. Evaluated on three long-context benchmarks, our 7B-parameter model demonstrates superior in-context learning performance on HELMET and robust retrieval and tracing capability on RULER. It is currently the only open model to achieve competitive long-range reasoning on BABILong at 512K context length without RAG or targeted fine-tuning. Released as fully open source under the Apache 2.0 license, the model has been downloaded over 100,000 times on Hugging Face. Model available at: this https URL

Title: Controllable Image Colorization with Instance-aware Texts and Masks

• Authors: Yanru An, Ling Gui, Qiang Hu, Chunlei Cai, Tianxiao Ye, Xiaoyun Zhang, Yanfeng Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.08705
• Pdf URL: https://arxiv.org/pdf/2505.08705
• Copy Paste: [[2505.08705]] Controllable Image Colorization with Instance-aware Texts and Masks(https://arxiv.org/abs/2505.08705)
• Keywords: diffusion
• Abstract: Recently, the application of deep learning in image colorization has received widespread attention. The maturation of diffusion models has further advanced the development of image colorization models. However, current mainstream image colorization models still face issues such as color bleeding and color binding errors, and cannot colorize images at the instance level. In this paper, we propose a diffusion-based colorization method MT-Color to achieve precise instance-aware colorization with use-provided guidance. To tackle color bleeding issue, we design a pixel-level mask attention mechanism that integrates latent features and conditional gray image features through cross-attention. We use segmentation masks to construct cross-attention masks, preventing pixel information from exchanging between different instances. We also introduce an instance mask and text guidance module that extracts instance masks and text representations of each instance, which are then fused with latent features through self-attention, utilizing instance masks to form self-attention masks to prevent instance texts from guiding the colorization of other areas, thus mitigating color binding errors. Furthermore, we apply a multi-instance sampling strategy, which involves sampling each instance region separately and then fusing the results. Additionally, we have created a specialized dataset for instance-level colorization tasks, GPT-color, by leveraging large visual language models on existing image datasets. Qualitative and quantitative experiments show that our model and dataset outperform previous methods and datasets.

Title: TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series

• Authors: Xiaolei Qin, Di Wang, Jing Zhang, Fengxiang Wang, Xin Su, Bo Du, Liangpei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.08723
• Pdf URL: https://arxiv.org/pdf/2505.08723
• Copy Paste: [[2505.08723]] TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series(https://arxiv.org/abs/2505.08723)
• Keywords: foundation model
• Abstract: Satellite image time series (SITS) provide continuous observations of the Earth's surface, making them essential for applications such as environmental management and disaster assessment. However, existing spatiotemporal foundation models rely on plain vision transformers, which encode entire temporal sequences without explicitly capturing multiscale spatiotemporal relationships between land objects. This limitation hinders their effectiveness in downstream tasks. To overcome this challenge, we propose TiMo, a novel hierarchical vision transformer foundation model tailored for SITS analysis. At its core, we introduce a spatiotemporal gyroscope attention mechanism that dynamically captures evolving multiscale patterns across both time and space. For pre-training, we curate MillionST, a large-scale dataset of one million images from 100,000 geographic locations, each captured across 10 temporal phases over five years, encompassing diverse geospatial changes and seasonal variations. Leveraging this dataset, we adapt masked image modeling to pre-train TiMo, enabling it to effectively learn and encode generalizable spatiotemporal this http URL experiments across multiple spatiotemporal tasks-including deforestation monitoring, land cover segmentation, crop type classification, and flood detection-demonstrate TiMo's superiority over state-of-the-art methods. Code, model, and dataset will be released at this https URL.

Title: NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Context

• Authors: Ben Yao, Qiuchi Li, Yazhou Zhang, Siyu Yang, Bohan Zhang, Prayag Tiwari, Jing Qin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.08734
• Pdf URL: https://arxiv.org/pdf/2505.08734
• Copy Paste: [[2505.08734]] NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Context(https://arxiv.org/abs/2505.08734)
• Keywords: in-context
• Abstract: This work introduces the first benchmark for nursing value alignment, consisting of five core value dimensions distilled from international nursing codes: Altruism, Human Dignity, Integrity, Justice, and Professionalism. The benchmark comprises 1,100 real-world nursing behavior instances collected through a five-month longitudinal field study across three hospitals of varying tiers. These instances are annotated by five clinical nurses and then augmented with LLM-generated counterfactuals with reversed ethic polarity. Each original case is paired with a value-aligned and a value-violating version, resulting in 2,200 labeled instances that constitute the Easy-Level dataset. To increase adversarial complexity, each instance is further transformed into a dialogue-based format that embeds contextual cues and subtle misleading signals, yielding a Hard-Level dataset. We evaluate 23 state-of-the-art (SoTA) LLMs on their alignment with nursing values. Our findings reveal three key insights: (1) DeepSeek-V3 achieves the highest performance on the Easy-Level dataset (94.55), where Claude 3.5 Sonnet outperforms other models on the Hard-Level dataset (89.43), significantly surpassing the medical LLMs; (2) Justice is consistently the most difficult nursing value dimension to evaluate; and (3) in-context learning significantly improves alignment. This work aims to provide a foundation for value-sensitive LLMs development in clinical settings. The dataset and the code are available at this https URL.

Title: Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data

• Authors: James Giroux, Cristiano Fanelli
• Subjects: cs.LG, hep-ex, nucl-ex, physics.ins-det
• Abstract URL: https://arxiv.org/abs/2505.08736
• Pdf URL: https://arxiv.org/pdf/2505.08736
• Copy Paste: [[2505.08736]] Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data(https://arxiv.org/abs/2505.08736)
• Keywords: foundation model
• Abstract: We present a (proto) Foundation Model for Nuclear Physics, capable of operating on low-level detector inputs from Imaging Cherenkov Detectors at the future Electron Ion Collider. To address limitations in existing next-token prediction approaches-namely resolution loss from VQ-VAE tokenization and lack of conditional generation-we propose three key innovations: (i) separate vocabularies for discrete spatial features and continuous variates, combined via Causal Multi-Head Cross-Attention (CMHCA), (ii) continuous kinematic conditioning through prepended context embeddings, and (iii) scalable and simple, high-resolution continuous variate tokenization without joint vocabulary inflation. Our model enables fast, high-fidelity generation of pixel and time sequences for Cherenkov photons, validated through closure tests in the High Performance DIRC. We also show our model generalizes to reconstruction tasks such as pion and kaon identification, in which we show its ability to leverage fine-tuning.

Title: Aya Vision: Advancing the Frontier of Multilingual Multimodality

• Authors: Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian, Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller-Morales, Jeremy Pekmez, Jason Ozuzu, Pierre Richemond, Acyr Locatelli, Nick Frosst, Phil Blunsom, Aidan Gomez, Ivan Zhang, Marzieh Fadaee, Manoj Govindassamy, Sudip Roy, Matthias Gallé, Beyza Ermis, Ahmet Üstün, Sara Hooker
• Subjects: cs.CL, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.08751
• Pdf URL: https://arxiv.org/pdf/2505.08751
• Copy Paste: [[2505.08751]] Aya Vision: Advancing the Frontier of Multilingual Multimodality(https://arxiv.org/abs/2505.08751)
• Keywords: generative
• Abstract: Building multimodal language models is fundamentally challenging: it requires aligning vision and language modalities, curating high-quality instruction data, and avoiding the degradation of existing text-only capabilities once vision is introduced. These difficulties are further magnified in the multilingual setting, where the need for multimodal data in different languages exacerbates existing data scarcity, machine translation often distorts meaning, and catastrophic forgetting is more pronounced. To address the aforementioned challenges, we introduce novel techniques spanning both data and modeling. First, we develop a synthetic annotation framework that curates high-quality, diverse multilingual multimodal instruction data, enabling Aya Vision models to produce natural, human-preferred responses to multimodal inputs across many languages. Complementing this, we propose a cross-modal model merging technique that mitigates catastrophic forgetting, effectively preserving text-only capabilities while simultaneously enhancing multimodal generative performance. Aya-Vision-8B achieves best-in-class performance compared to strong multimodal models such as Qwen-2.5-VL-7B, Pixtral-12B, and even much larger Llama-3.2-90B-Vision. We further scale this approach with Aya-Vision-32B, which outperforms models more than twice its size, such as Molmo-72B and LLaMA-3.2-90B-Vision. Our work advances multilingual progress on the multi-modal frontier, and provides insights into techniques that effectively bend the need for compute while delivering extremely high performance.