2025-02-24

Title: KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models

• Authors: Dong Chen, Zhengqing Hu, Peiguang Fan, Yueting Zhuang, Yafei Li, Qidong Liu, Xiaoheng Jiang, Mingliang Xu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14880
• Pdf URL: https://arxiv.org/pdf/2502.14880
• Copy Paste: [[2502.14880]] KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models(https://arxiv.org/abs/2502.14880)
• Keywords: anomaly
• Abstract: Vision anomaly detection, particularly in unsupervised settings, often struggles to distinguish between normal samples and anomalies due to the wide variability in anomalies. Recently, an increasing number of studies have focused on generating anomalies to help detectors learn more effective boundaries between normal samples and anomalies. However, as the generated anomalies are often derived from random factors, they frequently lack realism. Additionally, randomly generated anomalies typically offer limited support in constructing effective boundaries, as most differ substantially from normal samples and lie far from the boundary. To address these challenges, we propose Key Knowledge Augmentation (KKA), a method that extracts anomaly-related knowledge from large language models (LLMs). More specifically, KKA leverages the extensive prior knowledge of LLMs to generate meaningful anomalies based on normal samples. Then, KKA classifies the generated anomalies as easy anomalies and hard anomalies according to their similarity to normal samples. Easy anomalies exhibit significant differences from normal samples, whereas hard anomalies closely resemble normal samples. KKA iteratively updates the generated anomalies, and gradually increasing the proportion of hard anomalies to enable the detector to learn a more effective boundary. Experimental results show that the proposed method significantly improves the performance of various vision anomaly detectors while maintaining low generation costs. The code for CMG can be found at this https URL.

Title: Vision-Enhanced Time Series Forecasting via Latent Diffusion Models

• Authors: Weilin Ruan, Siru Zhong, Haomin Wen, Yuxuan Liang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14887
• Pdf URL: https://arxiv.org/pdf/2502.14887
• Copy Paste: [[2502.14887]] Vision-Enhanced Time Series Forecasting via Latent Diffusion Models(https://arxiv.org/abs/2502.14887)
• Keywords: diffusion
• Abstract: Diffusion models have recently emerged as powerful frameworks for generating high-quality images. While recent studies have explored their application to time series forecasting, these approaches face significant challenges in cross-modal modeling and transforming visual information effectively to capture temporal patterns. In this paper, we propose LDM4TS, a novel framework that leverages the powerful image reconstruction capabilities of latent diffusion models for vision-enhanced time series forecasting. Instead of introducing external visual data, we are the first to use complementary transformation techniques to convert time series into multi-view visual representations, allowing the model to exploit the rich feature extraction capabilities of the pre-trained vision encoder. Subsequently, these representations are reconstructed using a latent diffusion model with a cross-modal conditioning mechanism as well as a fusion module. Experimental results demonstrate that LDM4TS outperforms various specialized forecasting models for time series forecasting tasks.

Title: CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection

• Authors: Zhe Huang, Shuo Wang, Yongcai Wang, Lei Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14891
• Pdf URL: https://arxiv.org/pdf/2502.14891
• Copy Paste: [[2502.14891]] CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection(https://arxiv.org/abs/2502.14891)
• Keywords: diffusion
• Abstract: Collaborative 3D object detection holds significant importance in the field of autonomous driving, as it greatly enhances the perception capabilities of each individual agent by facilitating information exchange among multiple agents. However, in practice, due to pose estimation errors and time delays, the fusion of information across agents often results in feature representations with spatial and temporal noise, leading to detection errors. Diffusion models naturally have the ability to denoise noisy samples to the ideal data, which motivates us to explore the use of diffusion models to address the noise problem between multi-agent systems. In this work, we propose CoDiff, a novel robust collaborative perception framework that leverages the potential of diffusion models to generate more comprehensive and clearer feature representations. To the best of our knowledge, this is the first work to apply diffusion models to multi-agent collaborative perception. Specifically, we project high-dimensional feature map into the latent space of a powerful pre-trained autoencoder. Within this space, individual agent information serves as a condition to guide the diffusion model's sampling. This process denoises coarse feature maps and progressively refines the fused features. Experimental study on both simulated and real-world datasets demonstrates that the proposed framework CoDiff consistently outperforms existing relevant methods in terms of the collaborative object detection performance, and exhibits highly desired robustness when the pose and delay information of agents is with high-level noise.

Title: A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models

• Authors: Changhoon Kim, Yanjun Qi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14896
• Pdf URL: https://arxiv.org/pdf/2502.14896
• Copy Paste: [[2502.14896]] A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.14896)
• Keywords: diffusion
• Abstract: Text-to-Image (T2I) models have made remarkable progress in generating high-quality, diverse visual content from natural language prompts. However, their ability to reproduce copyrighted styles, sensitive imagery, and harmful content raises significant ethical and legal concerns. Concept erasure offers a proactive alternative to external filtering by modifying T2I models to prevent the generation of undesired content. In this survey, we provide a structured overview of concept erasure, categorizing existing methods based on their optimization strategies and the architectural components they modify. We categorize concept erasure methods into fine-tuning for parameter updates, closed-form solutions for efficient edits, and inference-time interventions for content restriction without weight modification. Additionally, we explore adversarial attacks that bypass erasure techniques and discuss emerging defenses. To support further research, we consolidate key datasets, evaluation metrics, and benchmarks for assessing erasure effectiveness and model robustness. This survey serves as a comprehensive resource, offering insights into the evolving landscape of concept erasure, its challenges, and future directions.

Title: Retrieval-augmented systems can be dangerous medical communicators

• Authors: Lionel Wong, Ayman Ali, Raymond Xiong, Shannon Zeijang Shen, Yoon Kim, Monica Agrawal
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.14898
• Pdf URL: https://arxiv.org/pdf/2502.14898
• Copy Paste: [[2502.14898]] Retrieval-augmented systems can be dangerous medical communicators(https://arxiv.org/abs/2502.14898)
• Keywords: generative
• Abstract: Patients have long sought health information online, and increasingly, they are turning to generative AI to answer their health-related queries. Given the high stakes of the medical domain, techniques like retrieval-augmented generation and citation grounding have been widely promoted as methods to reduce hallucinations and improve the accuracy of AI-generated responses and have been widely adopted into search engines. This paper argues that even when these methods produce literally accurate content drawn from source documents sans hallucinations, they can still be highly misleading. Patients may derive significantly different interpretations from AI-generated outputs than they would from reading the original source material, let alone consulting a knowledgeable clinician. Through a large-scale query analysis on topics including disputed diagnoses and procedure safety, we support our argument with quantitative and qualitative evidence of the suboptimal answers resulting from current systems. In particular, we highlight how these models tend to decontextualize facts, omit critical relevant sources, and reinforce patient misconceptions or biases. We propose a series of recommendations -- such as the incorporation of communication pragmatics and enhanced comprehension of source documents -- that could help mitigate these issues and extend beyond the medical domain.

Title: FacaDiffy: Inpainting Unseen Facade Parts Using Diffusion Models

• Authors: Thomas Froech, Olaf Wysocki, Yan Xia, Junyu Xie, Benedikt Schwab, Daniel Cremers, Thomas H. Kolbe
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14940
• Pdf URL: https://arxiv.org/pdf/2502.14940
• Copy Paste: [[2502.14940]] FacaDiffy: Inpainting Unseen Facade Parts Using Diffusion Models(https://arxiv.org/abs/2502.14940)
• Keywords: diffusion
• Abstract: High-detail semantic 3D building models are frequently utilized in robotics, geoinformatics, and computer vision. One key aspect of creating such models is employing 2D conflict maps that detect openings' locations in building facades. Yet, in reality, these maps are often incomplete due to obstacles encountered during laser scanning. To address this challenge, we introduce FacaDiffy, a novel method for inpainting unseen facade parts by completing conflict maps with a personalized Stable Diffusion model. Specifically, we first propose a deterministic ray analysis approach to derive 2D conflict maps from existing 3D building models and corresponding laser scanning point clouds. Furthermore, we facilitate the inpainting of unseen facade objects into these 2D conflict maps by leveraging the potential of personalizing a Stable Diffusion model. To complement the scarcity of real-world training data, we also develop a scalable pipeline to produce synthetic conflict maps using random city model generators and annotated facade images. Extensive experiments demonstrate that FacaDiffy achieves state-of-the-art performance in conflict map completion compared to various inpainting baselines and increases the detection rate by $22\%$ when applying the completed conflict maps for high-definition 3D semantic building reconstruction. The code is be publicly available in the corresponding GitHub repository: this https URL

Title: CyberSentinel: An Emergent Threat Detection System for AI Security

• Authors: Krti Tallam
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.14966
• Pdf URL: https://arxiv.org/pdf/2502.14966
• Copy Paste: [[2502.14966]] CyberSentinel: An Emergent Threat Detection System for AI Security(https://arxiv.org/abs/2502.14966)
• Keywords: anomaly
• Abstract: The rapid advancement of artificial intelligence (AI) has significantly expanded the attack surface for AI-driven cybersecurity threats, necessitating adaptive defense strategies. This paper introduces CyberSentinel, a unified, single-agent system for emergent threat detection, designed to identify and mitigate novel security risks in real time. CyberSentinel integrates: (1) Brute-force attack detection through SSH log analysis, (2) Phishing threat assessment using domain blacklists and heuristic URL scoring, and (3) Emergent threat detection via machine learning-based anomaly detection. By continuously adapting to evolving adversarial tactics, CyberSentinel strengthens proactive cybersecurity defense, addressing critical vulnerabilities in AI security.

Title: LAVID: An Agentic LVLM Framework for Diffusion-Generated Video Detection

• Authors: Qingyuan Liu, Yun-Yun Tsai, Ruijian Zha, Victoria Li, Pengyuan Shi, Chengzhi Mao, Junfeng Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.14994
• Pdf URL: https://arxiv.org/pdf/2502.14994
• Copy Paste: [[2502.14994]] LAVID: An Agentic LVLM Framework for Diffusion-Generated Video Detection(https://arxiv.org/abs/2502.14994)
• Keywords: diffusion, generative
• Abstract: The impressive achievements of generative models in creating high-quality videos have raised concerns about digital integrity and privacy vulnerabilities. Recent works of AI-generated content detection have been widely studied in the image field (e.g., deepfake), yet the video field has been unexplored. Large Vision Language Model (LVLM) has become an emerging tool for AI-generated content detection for its strong reasoning and multimodal capabilities. It breaks the limitations of traditional deep learning based methods faced with like lack of transparency and inability to recognize new artifacts. Motivated by this, we propose LAVID, a novel LVLMs-based ai-generated video detection with explicit knowledge enhancement. Our insight list as follows: (1) The leading LVLMs can call external tools to extract useful information to facilitate its own video detection task; (2) Structuring the prompt can affect LVLM's reasoning ability to interpret information in video content. Our proposed pipeline automatically selects a set of explicit knowledge tools for detection, and then adaptively adjusts the structure prompt by self-rewriting. Different from prior SOTA that trains additional detectors, our method is fully training-free and only requires inference of the LVLM for detection. To facilitate our research, we also create a new benchmark \vidfor with high-quality videos generated from multiple sources of video generation tools. Evaluation results show that LAVID improves F1 scores by 6.2 to 30.2% over the top baselines on our datasets across four SOTA LVLMs.

Title: Generative Modeling of Individual Behavior at Scale

• Authors: Nabil Omi, Lucas Caccia, Anurag Sarkar, Jordan T. Ash, Siddhartha Sen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.14998
• Pdf URL: https://arxiv.org/pdf/2502.14998
• Copy Paste: [[2502.14998]] Generative Modeling of Individual Behavior at Scale(https://arxiv.org/abs/2502.14998)
• Keywords: generative
• Abstract: There has been a growing interest in using AI to model human behavior, particularly in domains where humans interact with this technology. While most existing work models human behavior at an aggregate level, our goal is to model behavior at the individual level. Recent approaches to behavioral stylometry -- or the task of identifying a person from their actions alone -- have shown promise in domains like chess, but these approaches are either not scalable (e.g., fine-tune a separate model for each person) or not generative, in that they cannot generate actions. We address these limitations by framing behavioral stylometry as a multi-task learning problem -- where each task represents a distinct person -- and use parameter-efficient fine-tuning (PEFT) methods to learn an explicit style vector for each person. Style vectors are generative: they selectively activate shared "skill" parameters to generate actions in the style of each person. They also induce a latent space that we can interpret and manipulate algorithmically. In particular, we develop a general technique for style steering that allows us to steer a player's style vector towards a desired property. We apply our approach to two very different games, at unprecedented scales: chess (47,864 players) and Rocket League (2,000 players). We also show generality beyond gaming by applying our method to image generation, where we learn style vectors for 10,177 celebrities and use these vectors to steer their images.

Title: Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMs

• Authors: Raymond Wilson, Chase Carter, Cole Graham
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15009
• Pdf URL: https://arxiv.org/pdf/2502.15009
• Copy Paste: [[2502.15009]] Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMs(https://arxiv.org/abs/2502.15009)
• Keywords: in-context
• Abstract: Conversational query rewriting is crucial for effective conversational search, yet traditional supervised methods require substantial labeled data, which is scarce in low-resource settings. This paper introduces Prompt-Guided In-Context Learning, a novel approach that leverages the in-context learning capabilities of Large Language Models (LLMs) for few-shot conversational query rewriting. Our method employs carefully designed prompts, incorporating task descriptions, input/output format specifications, and a small set of illustrative examples, to guide pre-trained LLMs to generate context-independent queries without explicit fine-tuning. Extensive experiments on benchmark datasets, TREC and Taskmaster-1, demonstrate that our approach significantly outperforms strong baselines, including supervised models and contrastive co-training methods, across various evaluation metrics such as BLEU, ROUGE-L, Success Rate, and MRR. Ablation studies confirm the importance of in-context examples, and human evaluations further validate the superior fluency, relevance, and context utilization of our generated rewrites. The results highlight the potential of prompt-guided in-context learning as an efficient and effective paradigm for low-resource conversational query rewriting, reducing the reliance on extensive labeled data and complex training procedures.

Title: Towards Physics-Guided Foundation Models

• Authors: Majid Farhadloo, Arun Sharma, Mingzhou Yang, Bharat Jayaprakash, William Northrop, Shashi Shekhar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15013
• Pdf URL: https://arxiv.org/pdf/2502.15013
• Copy Paste: [[2502.15013]] Towards Physics-Guided Foundation Models(https://arxiv.org/abs/2502.15013)
• Keywords: foundation model
• Abstract: Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.

Title: Hardware-Friendly Static Quantization Method for Video Diffusion Transformers

• Authors: Sanghyun Yi, Qingfeng Liu, Mostafa El-Khamy
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15077
• Pdf URL: https://arxiv.org/pdf/2502.15077
• Copy Paste: [[2502.15077]] Hardware-Friendly Static Quantization Method for Video Diffusion Transformers(https://arxiv.org/abs/2502.15077)
• Keywords: diffusion, generative
• Abstract: Diffusion Transformers for video generation have gained significant research interest since the impressive performance of SORA. Efficient deployment of such generative-AI models on GPUs has been demonstrated with dynamic quantization. However, resource-constrained devices cannot support dynamic quantization, and need static quantization of the models for their efficient deployment on AI processors. In this paper, we propose a novel method for the post-training quantization of OpenSora\cite{opensora}, a Video Diffusion Transformer, without relying on dynamic quantization techniques. Our approach employs static quantization, achieving video quality comparable to FP16 and dynamically quantized ViDiT-Q methods, as measured by CLIP, and VQA metrics. In particular, we utilize per-step calibration data to adequately provide a post-training statically quantized model for each time step, incorporating channel-wise quantization for weights and tensor-wise quantization for activations. By further applying the smooth-quantization technique, we can obtain high-quality video outputs with the statically quantized models. Extensive experimental results demonstrate that static quantization can be a viable alternative to dynamic quantization for video diffusion transformers, offering a more efficient approach without sacrificing performance.

Title: Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps

• Authors: Yen-Che Hsiao, Abhishek Dutta
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15120
• Pdf URL: https://arxiv.org/pdf/2502.15120
• Copy Paste: [[2502.15120]] Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps(https://arxiv.org/abs/2502.15120)
• Keywords: in-context
• Abstract: This study investigates the in-context learning capabilities of various decoder-only transformer-based language models with different model sizes and training data, including GPT2, SmolLM2, OpenELM, TinyLlama, Stable LM, and Gemma 2. We identify a critical parameter threshold (~1.6 billion), beyond which reasoning performance improves significantly in tasks such as commonsense reasoning in multiple-choice question answering and deductive reasoning. Specifically, models above this threshold achieve better success rates in chain-of-thought (CoT) prompting for deductive reasoning tasks, especially those requiring longer reasoning chains, such as proof by contradiction and disjunction elimination. To address limitations in sub-threshold models, we demonstrate that fine-tuning with task-specific exemplars substantially enhances reasoning performance, enabling accurate CoT generation even without additional exemplars in the prompt for tasks with shorter reasoning chains. Finally, our analysis of attention maps reveals that models capable of generating correct CoTs exhibit higher token-level attention scores on subsequent correct tokens and the correct parts of speech, providing interpretability insights into reasoning processes. These findings collectively advance understanding of reasoning capabilities in decoder-only transformer-based models. The code is available at: this https URL.

Title: TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba

• Authors: Xiuwei Chen, Sihao Lin, Xiao Dong, Zisheng Chen, Meng Cao, Jianhua Han, Hang Xu, Xiaodan Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15130
• Pdf URL: https://arxiv.org/pdf/2502.15130
• Copy Paste: [[2502.15130]] TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba(https://arxiv.org/abs/2502.15130)
• Keywords: foundation model
• Abstract: Transformers have been favored in both uni-modal and multi-modal foundation models for their flexible scalability in attention modules. Consequently, a number of pre-trained Transformer models, e.g., LLaVA, CLIP, and DEIT, are publicly available. Recent research has introduced subquadratic architectures like Mamba, which enables global awareness with linear complexity. Nevertheless, training specialized subquadratic architectures from scratch for certain tasks is both resource-intensive and time-consuming. As a motivator, we explore cross-architecture training to transfer the ready knowledge in existing Transformer models to alternative architecture Mamba, termed TransMamba. Our approach employs a two-stage strategy to expedite training new Mamba models, ensuring effectiveness in across uni-modal and cross-modal tasks. Concerning architecture disparities, we project the intermediate features into an aligned latent space before transferring knowledge. On top of that, a Weight Subcloning and Adaptive Bidirectional distillation method (WSAB) is introduced for knowledge transfer without limitations on varying layer counts. For cross-modal learning, we propose a cross-Mamba module that integrates language awareness into Mamba's visual features, enhancing the cross-modal interaction capabilities of Mamba architecture. Despite using less than 75% of the training data typically required for training from scratch, TransMamba boasts substantially stronger performance across various network architectures and downstream tasks, including image classification, visual question answering, and text-video retrieval. The code will be publicly available.

Title: CoT-ICL Lab: A Petri Dish for Studying Chain-of-Thought Learning from In-Context Demonstrations

• Authors: Vignesh Kothapalli, Hamed Firooz, Maziar Sanjabi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.15132
• Pdf URL: https://arxiv.org/pdf/2502.15132
• Copy Paste: [[2502.15132]] CoT-ICL Lab: A Petri Dish for Studying Chain-of-Thought Learning from In-Context Demonstrations(https://arxiv.org/abs/2502.15132)
• Keywords: in-context
• Abstract: We introduce CoT-ICL Lab, a framework and methodology to generate synthetic tokenized datasets and systematically study chain-of-thought (CoT) in-context learning (ICL) in language models. CoT-ICL Lab allows fine grained control over the complexity of in-context examples by decoupling (1) the causal structure involved in chain token generation from (2) the underlying token processing functions. We train decoder-only transformers (up to 700M parameters) on these datasets and show that CoT accelerates the accuracy transition to higher values across model sizes. In particular, we find that model depth is crucial for leveraging CoT with limited in-context examples, while more examples help shallow models match deeper model performance. Additionally, limiting the diversity of token processing functions throughout training improves causal structure learning via ICL. We also interpret these transitions by analyzing transformer embeddings and attention maps. Overall, CoT-ICL Lab serves as a simple yet powerful testbed for theoretical and empirical insights into ICL and CoT in language models.

Title: Methods and Trends in Detecting Generated Images: A Comprehensive Review

• Authors: Arpan Mahara, Naphtali Rishe
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15176
• Pdf URL: https://arxiv.org/pdf/2502.15176
• Copy Paste: [[2502.15176]] Methods and Trends in Detecting Generated Images: A Comprehensive Review(https://arxiv.org/abs/2502.15176)
• Keywords: diffusion, generative
• Abstract: The proliferation of generative models, such as Generative Adversarial Networks (GANs), Diffusion Models, and Variational Autoencoders (VAEs), has enabled the synthesis of high-quality multimedia data. However, these advancements have also raised significant concerns regarding adversarial attacks, unethical usage, and societal harm. Recognizing these challenges, researchers have increasingly focused on developing methodologies to detect synthesized data effectively, aiming to mitigate potential risks. Prior reviews have primarily focused on deepfake detection and often lack coverage of recent advancements in synthetic image detection, particularly methods leveraging multimodal frameworks for improved forensic analysis. To address this gap, the present survey provides a comprehensive review of state-of-the-art methods for detecting and classifying synthetic images generated by advanced generative AI models. This review systematically examines core detection methodologies, identifies commonalities among approaches, and categorizes them into meaningful taxonomies. Furthermore, given the crucial role of large-scale datasets in this field, we present an overview of publicly available datasets that facilitate further research and benchmarking in synthetic data detection.

Title: Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing

• Authors: Zhilin Wang, Yafu Li, Jianhao Yan, Yu Cheng, Yue Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15208
• Pdf URL: https://arxiv.org/pdf/2502.15208
• Copy Paste: [[2502.15208]] Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing(https://arxiv.org/abs/2502.15208)
• Keywords: generative
• Abstract: Dynamical systems theory provides a framework for analyzing iterative processes and evolution over time. Within such systems, repetitive transformations can lead to stable configurations, known as attractors, including fixed points and limit cycles. Applying this perspective to large language models (LLMs), which iteratively map input text to output text, provides a principled approach to characterizing long-term behaviors. Successive paraphrasing serves as a compelling testbed for exploring such dynamics, as paraphrases re-express the same underlying meaning with linguistic variation. Although LLMs are expected to explore a diverse set of paraphrases in the text space, our study reveals that successive paraphrasing converges to stable periodic states, such as 2-period attractor cycles, limiting linguistic diversity. This phenomenon is attributed to the self-reinforcing nature of LLMs, as they iteratively favour and amplify certain textual forms over others. This pattern persists with increasing generation randomness or alternating prompts and LLMs. These findings underscore inherent constraints in LLM generative capability, while offering a novel dynamical systems perspective for studying their expressive potential.

Title: Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction

• Authors: Jingheng Ye, Shang Qin, Yinghui Li, Hai-Tao Zheng, Shen Wang, Qingsong Wen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15261
• Pdf URL: https://arxiv.org/pdf/2502.15261
• Copy Paste: [[2502.15261]] Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction(https://arxiv.org/abs/2502.15261)
• Keywords: generative
• Abstract: Grammatical Error Correction (GEC) faces a critical challenge concerning explainability, notably when GEC systems are designed for language learners. Existing research predominantly focuses on explaining grammatical errors extracted in advance, thus neglecting the relationship between explanations and corrections. To address this gap, we introduce EXGEC, a unified explainable GEC framework that integrates explanation and correction tasks in a generative manner, advocating that these tasks mutually reinforce each other. Experiments have been conducted on EXPECT, a recent human-labeled dataset for explainable GEC, comprising around 20k samples. Moreover, we detect significant noise within EXPECT, potentially compromising model training and evaluation. Therefore, we introduce an alternative dataset named EXPECT-denoised, ensuring a more objective framework for training and evaluation. Results on various NLP models (BART, T5, and Llama3) show that EXGEC models surpass single-task baselines in both tasks, demonstrating the effectiveness of our approach.

Title: CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Models

• Authors: Shunchang Liu, Zhuan Shi, Lingjuan Lyu, Yaochu Jin, Boi Faltings
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15278
• Pdf URL: https://arxiv.org/pdf/2502.15278
• Copy Paste: [[2502.15278]] CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.15278)
• Keywords: diffusion
• Abstract: Assessing whether AI-generated images are substantially similar to copyrighted works is a crucial step in resolving copyright disputes. In this paper, we propose CopyJudge, an automated copyright infringement identification framework that leverages large vision-language models (LVLMs) to simulate practical court processes for determining substantial similarity between copyrighted images and those generated by text-to-image diffusion models. Specifically, we employ an abstraction-filtration-comparison test framework with multi-LVLM debate to assess the likelihood of infringement and provide detailed judgment rationales. Based on the judgments, we further introduce a general LVLM-based mitigation strategy that automatically optimizes infringing prompts by avoiding sensitive expressions while preserving the non-infringing content. Besides, our approach can be enhanced by exploring non-infringing noise vectors within the diffusion latent space via reinforcement learning, even without modifying the original prompts. Experimental results show that our identification method achieves comparable state-of-the-art performance, while offering superior generalization and interpretability across various forms of infringement, and that our mitigation method could more effectively mitigate memorization and IP infringement without losing non-infringing expressions.

Title: Efficiently Solving Discounted MDPs with Predictions on Transition Matrices

• Authors: Lixing Lyu, Jiashuo Jiang, Wang Chi Cheung
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.15345
• Pdf URL: https://arxiv.org/pdf/2502.15345
• Copy Paste: [[2502.15345]] Efficiently Solving Discounted MDPs with Predictions on Transition Matrices(https://arxiv.org/abs/2502.15345)
• Keywords: generative
• Abstract: We study infinite-horizon Discounted Markov Decision Processes (DMDPs) under a generative model. Motivated by the Algorithm with Advice framework Mitzenmacher and Vassilvitskii 2022, we propose a novel framework to investigate how a prediction on the transition matrix can enhance the sample efficiency in solving DMDPs and improve sample complexity bounds. We focus on the DMDPs with $N$ state-action pairs and discounted factor $\gamma$. Firstly, we provide an impossibility result that, without prior knowledge of the prediction accuracy, no sampling policy can compute an $\epsilon$-optimal policy with a sample complexity bound better than $\tilde{O}((1-\gamma)^{-3} N\epsilon^{-2})$, which matches the state-of-the-art minimax sample complexity bound with no prediction. In complement, we propose an algorithm based on minimax optimization techniques that leverages the prediction on the transition matrix. Our algorithm achieves a sample complexity bound depending on the prediction error, and the bound is uniformly better than $\tilde{O}((1-\gamma)^{-4} N \epsilon^{-2})$, the previous best result derived from convex optimization methods. These theoretical findings are further supported by our numerical experiments.

Title: Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning

• Authors: Xuetao Ma, Wenbin Jiang, Hua Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.15401
• Pdf URL: https://arxiv.org/pdf/2502.15401
• Copy Paste: [[2502.15401]] Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning(https://arxiv.org/abs/2502.15401)
• Keywords: in-context
• Abstract: In-context learning (ICL) can significantly enhance the complex reasoning capabilities of large language models (LLMs), with the key lying in the selection and ordering of demonstration examples. Previous methods typically relied on simple features to measure the relevance between examples. We argue that these features are not sufficient to reflect the intrinsic connections between examples. In this study, we propose a curriculum ICL strategy guided by problem-solving logic. We select demonstration examples by analyzing the problem-solving logic and order them based on curriculum learning. Specifically, we constructed a problem-solving logic instruction set based on the BREAK dataset and fine-tuned a language model to analyze the problem-solving logic of examples. Subsequently, we selected appropriate demonstration examples based on problem-solving logic and assessed their difficulty according to the number of problem-solving steps. In accordance with the principles of curriculum learning, we ordered the examples from easy to hard to serve as contextual prompts. Experimental results on multiple benchmarks indicate that our method outperforms previous ICL approaches in terms of performance and efficiency, effectively enhancing the complex reasoning capabilities of LLMs. Our project will be publicly available subsequently.

Title: Evaluating Multimodal Generative AI with Korean Educational Standards

• Authors: Sanghee Park, Geewook Kim
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15422
• Pdf URL: https://arxiv.org/pdf/2502.15422
• Copy Paste: [[2502.15422]] Evaluating Multimodal Generative AI with Korean Educational Standards(https://arxiv.org/abs/2502.15422)
• Keywords: generative
• Abstract: This paper presents the Korean National Educational Test Benchmark (KoNET), a new benchmark designed to evaluate Multimodal Generative AI Systems using Korean national educational tests. KoNET comprises four exams: the Korean Elementary General Educational Development Test (KoEGED), Middle (KoMGED), High (KoHGED), and College Scholastic Ability Test (KoCSAT). These exams are renowned for their rigorous standards and diverse questions, facilitating a comprehensive analysis of AI performance across different educational levels. By focusing on Korean, KoNET provides insights into model performance in less-explored languages. We assess a range of models - open-source, open-access, and closed APIs - by examining difficulties, subject diversity, and human error rates. The code and dataset builder will be made fully open-sourced at this https URL.

Title: Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning

• Authors: Raghav Singhal, Kaustubh Ponkshe, Rohit Vartak, Lav R. Varshney, Praneeth Vepakomma
• Subjects: cs.LG, cs.AI, cs.CL, cs.DC
• Abstract URL: https://arxiv.org/abs/2502.15436
• Pdf URL: https://arxiv.org/pdf/2502.15436
• Copy Paste: [[2502.15436]] Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning(https://arxiv.org/abs/2502.15436)
• Keywords: foundation model
• Abstract: Low-Rank Adaptation (LoRA) has become ubiquitous for efficiently fine-tuning foundation models. However, federated fine-tuning using LoRA is challenging due to suboptimal updates arising from traditional federated averaging of individual adapters. Existing solutions either incur prohibitively high communication cost that scales linearly with the number of clients or suffer from performance degradation due to limited expressivity. We introduce Federated Silver Bullet (Fed-SB), a novel approach for federated fine-tuning of LLMs using LoRA-SB, a recently proposed low-rank adaptation method. LoRA-SB optimally aligns the optimization trajectory with the ideal low-rank full fine-tuning projection by learning a small square matrix (R) between adapters B and A, keeping other components fixed. Direct averaging of R guarantees exact updates, substantially reducing communication cost, which remains independent of the number of clients, and enables scalability. Fed-SB achieves state-of-the-art performance across commonsense reasoning, arithmetic reasoning, and language inference tasks while reducing communication costs by up to 230x. In private settings, Fed-SB further improves performance by (1) reducing trainable parameters, thereby lowering the noise required for differential privacy and (2) avoiding noise amplification introduced by other methods. Overall, Fed-SB establishes a new Pareto frontier in the tradeoff between communication and performance, offering an efficient and scalable solution for both private and non-private federated fine-tuning. Our code is publicly available at this https URL.

Title: Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation

• Authors: Wenxuan Wang, Kai Wu, Yujian Betterest Li, Dan Wang, Xiaoyu Zhang, Jing Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.15466
• Pdf URL: https://arxiv.org/pdf/2502.15466
• Copy Paste: [[2502.15466]] Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation(https://arxiv.org/abs/2502.15466)
• Keywords: foundation model
• Abstract: Foundation models for time series analysis (TSA) have attracted significant attention. However, challenges such as data scarcity and data imbalance continue to hinder their development. To address this, we consider modeling complex systems through symbolic expressions that serve as semantic descriptors of time series. Building on this concept, we introduce a series-symbol (S2) dual-modulity data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic representations. Leveraging the S2 dataset, we develop SymTime, a pre-trained foundation model for TSA. SymTime demonstrates competitive performance across five major TSA tasks when fine-tuned with downstream task, rivaling foundation models pre-trained on real-world datasets. This approach underscores the potential of dual-modality data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance.

Title: On the Robustness of Transformers against Context Hijacking for Linear Classification

• Authors: Tianle Li, Chenyang Zhang, Xingwu Chen, Yuan Cao, Difan Zou
• Subjects: cs.CL, cs.AI, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.15609
• Pdf URL: https://arxiv.org/pdf/2502.15609
• Copy Paste: [[2502.15609]] On the Robustness of Transformers against Context Hijacking for Linear Classification(https://arxiv.org/abs/2502.15609)
• Keywords: in-context
• Abstract: Transformer-based Large Language Models (LLMs) have demonstrated powerful in-context learning capabilities. However, their predictions can be disrupted by factually correct context, a phenomenon known as context hijacking, revealing a significant robustness issue. To understand this phenomenon theoretically, we explore an in-context linear classification problem based on recent advances in linear transformers. In our setup, context tokens are designed as factually correct query-answer pairs, where the queries are similar to the final query but have opposite labels. Then, we develop a general theoretical analysis on the robustness of the linear transformers, which is formulated as a function of the model depth, training context lengths, and number of hijacking context tokens. A key finding is that a well-trained deeper transformer can achieve higher robustness, which aligns with empirical observations. We show that this improvement arises because deeper layers enable more fine-grained optimization steps, effectively mitigating interference from context hijacking. This is also well supported by our numerical experiments. Our findings provide theoretical insights into the benefits of deeper architectures and contribute to enhancing the understanding of transformer architectures.

Title: Mantis: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification

• Authors: Vasilii Feofanov, Songkang Wen, Marius Alonso, Romain Ilbert, Hongbo Guo, Malik Tiomoko, Lujia Pan, Jianfeng Zhang, Ievgen Redko
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.15637
• Pdf URL: https://arxiv.org/pdf/2502.15637
• Copy Paste: [[2502.15637]] Mantis: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification(https://arxiv.org/abs/2502.15637)
• Keywords: foundation model
• Abstract: In recent years, there has been increasing interest in developing foundation models for time series data that can generalize across diverse downstream tasks. While numerous forecasting-oriented foundation models have been introduced, there is a notable scarcity of models tailored for time series classification. To address this gap, we present Mantis, a new open-source foundation model for time series classification based on the Vision Transformer (ViT) architecture that has been pre-trained using a contrastive learning approach. Our experimental results show that Mantis outperforms existing foundation models both when the backbone is frozen and when fine-tuned, while achieving the lowest calibration error. In addition, we propose several adapters to handle the multivariate setting, reducing memory requirements and modeling channel interdependence.

Title: AutoTandemML: Active Learning Enhanced Tandem Neural Networks for Inverse Design Problems

• Authors: Luka Grbcic, Juliane Müller, Wibe Albert de Jong
• Subjects: cs.LG, cs.AI, cs.CE, cs.NE
• Abstract URL: https://arxiv.org/abs/2502.15643
• Pdf URL: https://arxiv.org/pdf/2502.15643
• Copy Paste: [[2502.15643]] AutoTandemML: Active Learning Enhanced Tandem Neural Networks for Inverse Design Problems(https://arxiv.org/abs/2502.15643)
• Keywords: diffusion
• Abstract: Inverse design in science and engineering involves determining optimal design parameters that achieve desired performance outcomes, a process often hindered by the complexity and high dimensionality of design spaces, leading to significant computational costs. To tackle this challenge, we propose a novel hybrid approach that combines active learning with Tandem Neural Networks to enhance the efficiency and effectiveness of solving inverse design problems. Active learning allows to selectively sample the most informative data points, reducing the required dataset size without compromising accuracy. We investigate this approach using three benchmark problems: airfoil inverse design, photonic surface inverse design, and scalar boundary condition reconstruction in diffusion partial differential equations. We demonstrate that integrating active learning with Tandem Neural Networks outperforms standard approaches across the benchmark suite, achieving better accuracy with fewer training samples.

Title: Machine-generated text detection prevents language model collapse

• Authors: George Drayson, Vasileios Lampos
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.15654
• Pdf URL: https://arxiv.org/pdf/2502.15654
• Copy Paste: [[2502.15654]] Machine-generated text detection prevents language model collapse(https://arxiv.org/abs/2502.15654)
• Keywords: generative
• Abstract: As Large Language Models (LLMs) become increasingly prevalent, their generated outputs are proliferating across the web, risking a future where machine-generated content dilutes human-authored text. Since web data is the primary resource for LLM pretraining, future models will be trained on an unknown portion of synthetic data. This will lead to model collapse, a degenerative process which causes models to reinforce their own errors and experience a drop in model performance. In this study, we investigate the impact of decoding strategy on model collapse, where we analyse the characteristics of the generated data during recursive training, its similarity to human references and the resulting model performance. Using the decoding strategies that lead to the most significant model degradation, we tackle the question: how to avoid model collapse when the origin (human or synthetic) of the training data is unknown. We design a novel methodology based on resampling the data distribution using importance weights from our machine-generated text detector. Our method is validated on two LLM variants (GPT-2 and SmolLM2) on the open-ended text generation task, demonstrating that we can successfully prevent model collapse and when there is enough human-authored data in the training dataset, our method improves model performance.

Title: VaViM and VaVAM: Autonomous Driving through Video Generative Modeling

• Authors: Florent Bartoccioni, Elias Ramzi, Victor Besnier, Shashanka Venkataramanan, Tuan-Hung Vu, Yihong Xu, Loick Chambon, Spyros Gidaris, Serkan Odabas, David Hurych, Renaud Marlet, Alexandre Boulch, Mickael Chen, Éloi Zablocki, Andrei Bursuc, Eduardo Valle, Matthieu Cord
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.15672
• Pdf URL: https://arxiv.org/pdf/2502.15672
• Copy Paste: [[2502.15672]] VaViM and VaVAM: Autonomous Driving through Video Generative Modeling(https://arxiv.org/abs/2502.15672)
• Keywords: generative
• Abstract: We explore the potential of large-scale generative video models for autonomous driving, introducing an open-source auto-regressive video model (VaViM) and its companion video-action model (VaVAM) to investigate how video pre-training transfers to real-world driving. VaViM is a simple auto-regressive video model that predicts frames using spatio-temporal token sequences. We show that it captures the semantics and dynamics of driving scenes. VaVAM, the video-action model, leverages the learned representations of VaViM to generate driving trajectories through imitation learning. Together, the models form a complete perception-to-action pipeline. We evaluate our models in open- and closed-loop driving scenarios, revealing that video-based pre-training holds promise for autonomous driving. Key insights include the semantic richness of the learned representations, the benefits of scaling for video synthesis, and the complex relationship between model size, data, and safety metrics in closed-loop evaluations. We release code and model weights at this https URL

Title: One-step Diffusion Models with $f$-Divergence Distribution Matching

• Authors: Yilun Xu, Weili Nie, Arash Vahdat
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15681
• Pdf URL: https://arxiv.org/pdf/2502.15681
• Copy Paste: [[2502.15681]] One-step Diffusion Models with $f$-Divergence Distribution Matching(https://arxiv.org/abs/2502.15681)
• Keywords: diffusion
• Abstract: Sampling from diffusion models involves a slow iterative process that hinders their practical deployment, especially for interactive applications. To accelerate generation speed, recent approaches distill a multi-step diffusion model into a single-step student generator via variational score distillation, which matches the distribution of samples generated by the student to the teacher's distribution. However, these approaches use the reverse Kullback-Leibler (KL) divergence for distribution matching which is known to be mode seeking. In this paper, we generalize the distribution matching approach using a novel $f$-divergence minimization framework, termed $f$-distill, that covers different divergences with different trade-offs in terms of mode coverage and training variance. We derive the gradient of the $f$-divergence between the teacher and student distributions and show that it is expressed as the product of their score differences and a weighting function determined by their density ratio. This weighting function naturally emphasizes samples with higher density in the teacher distribution, when using a less mode-seeking divergence. We observe that the popular variational score distillation approach using the reverse-KL divergence is a special case within our framework. Empirically, we demonstrate that alternative $f$-divergences, such as forward-KL and Jensen-Shannon divergences, outperform the current best variational score distillation methods across image generation tasks. In particular, when using Jensen-Shannon divergence, $f$-distill achieves current state-of-the-art one-step generation performance on ImageNet64 and zero-shot text-to-image generation on MS-COCO. Project page: this https URL

Title: ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval

• Authors: Guanqi Zhan, Yuanpei Liu, Kai Han, Weidi Xie, Andrew Zisserman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.15682
• Pdf URL: https://arxiv.org/pdf/2502.15682
• Copy Paste: [[2502.15682]] ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval(https://arxiv.org/abs/2502.15682)
• Keywords: foundation model
• Abstract: The objective in this paper is to improve the performance of text-to-image retrieval. To this end, we introduce a new framework that can boost the performance of large-scale pre-trained vision-language models, so that they can be used for text-to-image re-ranking. The approach, Enhanced Language-Image Pre-training (ELIP), uses the text query to predict a set of visual prompts to condition the ViT image encoding. ELIP can easily be applied to the commonly used CLIP/SigLIP and the state-of-the-art BLIP-2 architectures. To train the architecture with limited computing resources, we develop a 'student friendly' best practice involving global hard sample mining, and selection and curation of a large-scale dataset. On the evaluation side, we set up two new out-of-distribution benchmarks, Occluded COCO and ImageNet-R, to assess the zero-shot generalisation of the models to different domains. Benefiting from the novel architecture and data curation, experiments show our enhanced network significantly boosts CLIP/SigLIP performance and outperforms the state-of-the-art BLIP-2 model on text-to-image retrieval.