2025-02-17

Title: Jailbreaking to Jailbreak

• Authors: Jeremy Kritz, Vaughn Robinson, Robert Vacareanu, Bijan Varjavand, Michael Choi, Bobby Gogov, Scale Red Team, Summer Yue, Willow E. Primack, Zifan Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09638
• Pdf URL: https://arxiv.org/pdf/2502.09638
• Copy Paste: [[2502.09638]] Jailbreaking to Jailbreak(https://arxiv.org/abs/2502.09638)
• Keywords: in-context
• Abstract: Refusal training on Large Language Models (LLMs) prevents harmful outputs, yet this defense remains vulnerable to both automated and human-crafted jailbreaks. We present a novel LLM-as-red-teamer approach in which a human jailbreaks a refusal-trained LLM to make it willing to jailbreak itself or other LLMs. We refer to the jailbroken LLMs as $J_2$ attackers, which can systematically evaluate target models using various red teaming strategies and improve its performance via in-context learning from the previous failures. Our experiments demonstrate that Sonnet 3.5 and Gemini 1.5 pro outperform other LLMs as $J_2$, achieving 93.0% and 91.0% attack success rates (ASRs) respectively against GPT-4o (and similar results across other capable LLMs) on Harmbench. Our work not only introduces a scalable approach to strategic red teaming, drawing inspiration from human red teamers, but also highlights jailbreaking-to-jailbreak as an overlooked failure mode of the safeguard. Specifically, an LLM can bypass its own safeguards by employing a jailbroken version of itself that is willing to assist in further jailbreaking. To prevent any direct misuse with $J_2$, while advancing research in AI safety, we publicly share our methodology while keeping specific prompting details private.

Title: Krutrim LLM: Multilingual Foundational Model for over a Billion People

• Authors: Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv, Deepak Kumar, Raghav Awasthi, Arveti Manjunath, Shubham Agarwal, Kumar Ashish, Gautam Bhargava, Chandra Khatri
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09642
• Pdf URL: https://arxiv.org/pdf/2502.09642
• Copy Paste: [[2502.09642]] Krutrim LLM: Multilingual Foundational Model for over a Billion People(https://arxiv.org/abs/2502.09642)
• Keywords: foundation model
• Abstract: India is a diverse society with unique challenges in developing AI systems, including linguistic diversity, oral traditions, data accessibility, and scalability. Existing foundation models are primarily trained on English, limiting their effectiveness for India's population. Indic languages comprise only 1 percent of Common Crawl corpora despite India representing 18 percent of the global population, leading to linguistic biases. Thousands of regional languages, dialects, and code mixing create additional representation challenges due to sparse training data. We introduce Krutrim LLM, a 2 trillion token multilingual model designed for India's linguistic landscape. It incorporates the largest known Indic dataset, mitigating data scarcity and ensuring balanced performance across dialects. Krutrim outperforms or matches state-of-the-art models on Indic benchmarks while maintaining competitive English performance. Despite being significantly smaller in training flops, Krutrim LLM matches or exceeds models like LLAMA-2 on 10 out of 16 tasks, with an average score of 0.57 versus 0.55. This evidences Krutrim's flexible multilingual fluency across diverse linguistic contexts. Krutrim is integrated with real-time search to improve factual accuracy in conversational AI applications. This enhances accessibility for over 1 billion users worldwide. Through intentional design choices addressing data imbalances, Krutrim LLM signifies meaningful progress in building ethical, globally representative AI models.

Title: From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models

• Authors: Mayank Vatsa, Aparna Bharati, Surbhi Mittal, Richa Singh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09645
• Pdf URL: https://arxiv.org/pdf/2502.09645
• Copy Paste: [[2502.09645]] From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models(https://arxiv.org/abs/2502.09645)
• Keywords: foundation model
• Abstract: Negation, a linguistic construct conveying absence, denial, or contradiction, poses significant challenges for multilingual multimodal foundation models. These models excel in tasks like machine translation, text-guided generation, image captioning, audio interactions, and video processing but often struggle to accurately interpret negation across diverse languages and cultural contexts. In this perspective paper, we propose a comprehensive taxonomy of negation constructs, illustrating how structural, semantic, and cultural factors influence multimodal foundation models. We present open research questions and highlight key challenges, emphasizing the importance of addressing these issues to achieve robust negation handling. Finally, we advocate for specialized benchmarks, language-specific tokenization, fine-grained attention mechanisms, and advanced multimodal architectures. These strategies can foster more adaptable and semantically precise multimodal foundation models, better equipped to navigate and accurately interpret the complexities of negation in multilingual, multimodal environments.

Title: GraphCompNet: A Position-Aware Model for Predicting and Compensating Shape Deviations in 3D Printing

• Authors: Lei (Rachel)Chen, Juheon Lee, Juan Carlos Catana, Tsegai Yhdego, Nathan Moroney, Mohammad Amin Nabian, Hui Wang, Jun Zeng
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09652
• Pdf URL: https://arxiv.org/pdf/2502.09652
• Copy Paste: [[2502.09652]] GraphCompNet: A Position-Aware Model for Predicting and Compensating Shape Deviations in 3D Printing(https://arxiv.org/abs/2502.09652)
• Keywords: generative
• Abstract: This paper introduces a data-driven algorithm for modeling and compensating shape deviations in additive manufacturing (AM), addressing challenges in geometric accuracy and batch production. While traditional methods, such as analytical models and metrology, laid the groundwork for geometric precision, they are often impractical for large-scale production. Recent advancements in machine learning (ML) have improved compensation precision, but issues remain in generalizing across complex geometries and adapting to position-dependent variations. We present a novel approach for powder bed fusion (PBF) processes, using GraphCompNet, which is a computational framework combining graph-based neural networks with a generative adversarial network (GAN)-inspired training process. By leveraging point cloud data and dynamic graph convolutional neural networks (DGCNNs), GraphCompNet models complex shapes and incorporates position-specific thermal and mechanical factors. A two-stage adversarial training procedure iteratively refines compensated designs via a compensator-predictor architecture, offering real-time feedback and optimization. Experimental validation across diverse shapes and positions shows the framework significantly improves compensation accuracy (35 to 65 percent) across the entire print space, adapting to position-dependent variations. This work advances the development of Digital Twin technology for AM, enabling scalable, real-time monitoring and compensation, and addressing critical gaps in AM process control. The proposed method supports high-precision, automated industrial-scale design and manufacturing systems.

Title: Bidirectional Diffusion Bridge Models

• Authors: Duc Kieu, Kien Do, Toan Nguyen, Dang Nguyen, Thin Nguyen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09655
• Pdf URL: https://arxiv.org/pdf/2502.09655
• Copy Paste: [[2502.09655]] Bidirectional Diffusion Bridge Models(https://arxiv.org/abs/2502.09655)
• Keywords: diffusion
• Abstract: Diffusion bridges have shown potential in paired image-to-image (I2I) translation tasks. However, existing methods are limited by their unidirectional nature, requiring separate models for forward and reverse translations. This not only doubles the computational cost but also restricts their practicality. In this work, we introduce the Bidirectional Diffusion Bridge Model (BDBM), a scalable approach that facilitates bidirectional translation between two coupled distributions using a single network. BDBM leverages the Chapman-Kolmogorov Equation for bridges, enabling it to model data distribution shifts across timesteps in both forward and backward directions by exploiting the interchangeability of the initial and target timesteps within this framework. Notably, when the marginal distribution given endpoints is Gaussian, BDBM's transition kernels in both directions possess analytical forms, allowing for efficient learning with a single network. We demonstrate the connection between BDBM and existing bridge methods, such as Doob's h-transform and variational approaches, and highlight its advantages. Extensive experiments on high-resolution I2I translation tasks demonstrate that BDBM not only enables bidirectional translation with minimal additional cost but also outperforms state-of-the-art bridge models. Our source code is available at [this https URL||this https URL].

Title: Neuro-Conceptual Artificial Intelligence: Integrating OPM with Deep Learning to Enhance Question Answering Quality

• Authors: Xin Kang, Veronika Shteingardt, Yuhan Wang, Dov Dori
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09658
• Pdf URL: https://arxiv.org/pdf/2502.09658
• Copy Paste: [[2502.09658]] Neuro-Conceptual Artificial Intelligence: Integrating OPM with Deep Learning to Enhance Question Answering Quality(https://arxiv.org/abs/2502.09658)
• Keywords: in-context
• Abstract: Knowledge representation and reasoning are critical challenges in Artificial Intelligence (AI), particularly in integrating neural and symbolic approaches to achieve explainable and transparent AI systems. Traditional knowledge representation methods often fall short of capturing complex processes and state changes. We introduce Neuro-Conceptual Artificial Intelligence (NCAI), a specialization of the neuro-symbolic AI approach that integrates conceptual modeling using Object-Process Methodology (OPM) ISO 19450:2024 with deep learning to enhance question-answering (QA) quality. By converting natural language text into OPM models using in-context learning, NCAI leverages the expressive power of OPM to represent complex OPM elements-processes, objects, and states-beyond what traditional triplet-based knowledge graphs can easily capture. This rich structured knowledge representation improves reasoning transparency and answer accuracy in an OPM-QA system. We further propose transparency evaluation metrics to quantitatively measure how faithfully the predicted reasoning aligns with OPM-based conceptual logic. Our experiments demonstrate that NCAI outperforms traditional methods, highlighting its potential for advancing neuro-symbolic AI by providing rich knowledge representations, measurable transparency, and improved reasoning.

Title: Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models

• Authors: Hasin Rehana, Jie Zheng, Leo Yeh, Benu Bansal, Nur Bengisu Çam, Christianah Jemiyo, Brett McGregor, Arzucan Özgür, Yongqun He, Junguk Hur
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.09659
• Pdf URL: https://arxiv.org/pdf/2502.09659
• Copy Paste: [[2502.09659]] Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models(https://arxiv.org/abs/2502.09659)
• Keywords: generative
• Abstract: Motivation: An adjuvant is a chemical incorporated into vaccines that enhances their efficacy by improving the immune response. Identifying adjuvant names from cancer vaccine studies is essential for furthering research and enhancing immunotherapies. However, the manual curation from the constantly expanding biomedical literature poses significant challenges. This study explores the automated recognition of vaccine adjuvant names using Large Language Models (LLMs), specifically Generative Pretrained Transformers (GPT) and Large Language Model Meta AI (Llama). Methods: We utilized two datasets: 97 clinical trial records from AdjuvareDB and 290 abstracts annotated with the Vaccine Adjuvant Compendium (VAC). GPT-4o and Llama 3.2 were employed in zero-shot and few-shot learning paradigms with up to four examples per prompt. Prompts explicitly targeted adjuvant names, testing the impact of contextual information such as substances or interventions. Outputs underwent automated and manual validation for accuracy and consistency. Results: GPT-4o attained 100% Precision across all situations while exhibiting notable improve in Recall and F1-scores, particularly with incorporating interventions. On the VAC dataset, GPT-4o achieved a maximum F1-score of 77.32% with interventions, surpassing Llama-3.2-3B by approximately 2%. On the AdjuvareDB dataset, GPT-4o reached an F1-score of 81.67% for three-shot prompting with interventions, surpassing Llama-3.2-3 B's maximum F1-score of 65.62%. Conclusion: Our findings demonstrate that LLMs excel at identifying adjuvant names, including rare variations of naming representation. This study emphasizes the capability of LLMs to enhance cancer vaccine development by efficiently extracting insights. Future work aims to broaden the framework to encompass various biomedical literature and enhance model generalizability across various vaccines and adjuvants.

Title: DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations

• Authors: Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio
• Subjects: cs.CV, cs.AI, cs.LG, q-bio.CB
• Abstract URL: https://arxiv.org/abs/2502.09663
• Pdf URL: https://arxiv.org/pdf/2502.09663
• Copy Paste: [[2502.09663]] DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations(https://arxiv.org/abs/2502.09663)
• Keywords: diffusion
• Abstract: In recent years, deep learning models have been extensively applied to biological data across various modalities. Discriminative deep learning models have excelled at classifying images into categories (e.g., healthy versus diseased, treated versus untreated). However, these models are often perceived as black boxes due to their complexity and lack of interpretability, limiting their application in real-world biological contexts. In biological research, explainability is essential: understanding classifier decisions and identifying subtle differences between conditions are critical for elucidating the effects of treatments, disease progression, and biological processes. To address this challenge, we propose DiffEx, a method for generating visually interpretable attributes to explain classifiers and identify microscopic cellular variations between different conditions. We demonstrate the effectiveness of DiffEx in explaining classifiers trained on natural and biological images. Furthermore, we use DiffEx to uncover phenotypic differences within microscopy datasets. By offering insights into cellular variations through classifier explanations, DiffEx has the potential to advance the understanding of diseases and aid drug discovery by identifying novel biomarkers.

Title: Image Super-Resolution with Guarantees via Conformal Generative Models

• Authors: Eduardo Adame, Daniel Csillag, Guilherme Tegoni Goedert
• Subjects: cs.CV, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09664
• Pdf URL: https://arxiv.org/pdf/2502.09664
• Copy Paste: [[2502.09664]] Image Super-Resolution with Guarantees via Conformal Generative Models(https://arxiv.org/abs/2502.09664)
• Keywords: foundation model, generative
• Abstract: The increasing use of generative ML foundation models for image super-resolution calls for robust and interpretable uncertainty quantification methods. We address this need by presenting a novel approach based on conformal prediction techniques to create a "confidence mask" capable of reliably and intuitively communicating where the generated image can be trusted. Our method is adaptable to any black-box generative model, including those locked behind an opaque API, requires only easily attainable data for calibration, and is highly customizable via the choice of a local image similarity metric. We prove strong theoretical guarantees for our method that span fidelity error control (according to our local image similarity metric), reconstruction quality, and robustness in the face of data leakage. Finally, we empirically evaluate these results and establish our method's solid performance.

Title: Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models

• Authors: Anis Bourou, Biel Castaño Segade, Thomas Boye, Valérie Mezger, Auguste Genovesio
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09665
• Pdf URL: https://arxiv.org/pdf/2502.09665
• Copy Paste: [[2502.09665]] Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models(https://arxiv.org/abs/2502.09665)
• Keywords: diffusion, generative
• Abstract: Identifying subtle phenotypic variations in cellular images is critical for advancing biological research and accelerating drug discovery. These variations are often masked by the inherent cellular heterogeneity, making it challenging to distinguish differences between experimental conditions. Recent advancements in deep generative models have demonstrated significant potential for revealing these nuanced phenotypes through image translation, opening new frontiers in cellular and molecular biology as well as the identification of novel biomarkers. Among these generative models, diffusion models stand out for their ability to produce high-quality, realistic images. However, training diffusion models typically requires large datasets and substantial computational resources, both of which can be limited in biological research. In this work, we propose a novel approach that leverages pre-trained latent diffusion models to uncover subtle phenotypic changes. We validate our approach qualitatively and quantitatively on several small datasets of microscopy images. Our findings reveal that our approach enables effective detection of phenotypic variations, capturing both visually apparent and imperceptible differences. Ultimately, our results highlight the promising potential of this approach for phenotype detection, especially in contexts constrained by limited data and computational capacity.

Title: The Science of Evaluating Foundation Models

• Authors: Jiayi Yuan, Jiamu Zhang, Andrew Wen, Xia Hu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09670
• Pdf URL: https://arxiv.org/pdf/2502.09670
• Copy Paste: [[2502.09670]] The Science of Evaluating Foundation Models(https://arxiv.org/abs/2502.09670)
• Keywords: foundation model
• Abstract: The emergent phenomena of large foundation models have revolutionized natural language processing. However, evaluating these models presents significant challenges due to their size, capabilities, and deployment across diverse applications. Existing literature often focuses on individual aspects, such as benchmark performance or specific tasks, but fails to provide a cohesive process that integrates the nuances of diverse use cases with broader ethical and operational considerations. This work focuses on three key aspects: (1) Formalizing the Evaluation Process by providing a structured framework tailored to specific use-case contexts, (2) Offering Actionable Tools and Frameworks such as checklists and templates to ensure thorough, reproducible, and practical evaluations, and (3) Surveying Recent Work with a targeted review of advancements in LLM evaluation, emphasizing real-world applications.

Title: Object-Centric Latent Action Learning

• Authors: Albina Klepach, Alexander Nikulin, Ilya Zisman, Denis Tarasov, Alexander Derevyagin, Andrei Polubarov, Nikita Lyubaykin, Vladislav Kurenkov
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09680
• Pdf URL: https://arxiv.org/pdf/2502.09680
• Copy Paste: [[2502.09680]] Object-Centric Latent Action Learning(https://arxiv.org/abs/2502.09680)
• Keywords: self-supervised
• Abstract: Leveraging vast amounts of internet video data for Embodied AI is currently bottle-necked by the lack of action annotations and the presence of action-correlated distractors. We propose a novel object-centric latent action learning approach, based on VideoSaur and LAPO, that employs self-supervised decomposition of scenes into object representations and annotates video data with proxy-action labels. This method effectively disentangles causal agent-object interactions from irrelevant background noise and reduces the performance degradation of latent action learning approaches caused by distractors. Our preliminary experiments with the Distracting Control Suite show that latent action pretraining based on object decompositions improve the quality of inferred latent actions by x2.7 and efficiency of downstream fine-tuning with a small set of labeled actions, increasing return by x2.6 on average.

Title: Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling

• Authors: Benjamin D. Killeen, Bohua Wan, Aditya V. Kulkarni, Nathan Drenkow, Michael Oberst, Paul H. Yi, Mathias Unberath
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09688
• Pdf URL: https://arxiv.org/pdf/2502.09688
• Copy Paste: [[2502.09688]] Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling(https://arxiv.org/abs/2502.09688)
• Keywords: generative
• Abstract: Artificial intelligence (AI) is poised to transform healthcare by enabling personalized and efficient care through data-driven insights. Although radiology is at the forefront of AI adoption, in practice, the potential of AI models is often overshadowed by severe failures to generalize: AI models can have performance degradation of up to 20% when transitioning from controlled test environments to clinical use by radiologists. This mismatch raises concerns that radiologists will be misled by incorrect AI predictions in practice and/or grow to distrust AI, rendering these promising technologies practically ineffectual. Exhaustive clinical trials of AI models on abundant and diverse data is thus critical to anticipate AI model degradation when encountering varied data samples. Achieving these goals, however, is challenging due to the high costs of collecting diverse data samples and corresponding annotations. To overcome these limitations, we introduce a novel conditional generative AI model designed for virtual clinical trials (VCTs) of radiology AI, capable of realistically synthesizing full-body CT images of patients with specified attributes. By learning the joint distribution of images and anatomical structures, our model enables precise replication of real-world patient populations with unprecedented detail at this scale. We demonstrate meaningful evaluation of radiology AI models through VCTs powered by our synthetic CT study populations, revealing model degradation and facilitating algorithmic auditing for bias-inducing data attributes. Our generative AI approach to VCTs is a promising avenue towards a scalable solution to assess model robustness, mitigate biases, and safeguard patient care by enabling simpler testing and evaluation of AI models in any desired range of diverse patient populations.

Title: Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes

• Authors: Taylan G. Topcu, Mohammed Husain, Max Ofsa, Paul Wach
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09690
• Pdf URL: https://arxiv.org/pdf/2502.09690
• Copy Paste: [[2502.09690]] Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes(https://arxiv.org/abs/2502.09690)
• Keywords: generative
• Abstract: Multi-purpose Large Language Models (LLMs), a subset of generative Artificial Intelligence (AI), have recently made significant progress. While expectations for LLMs to assist systems engineering (SE) tasks are paramount; the interdisciplinary and complex nature of systems, along with the need to synthesize deep-domain knowledge and operational context, raise questions regarding the efficacy of LLMs to generate SE artifacts, particularly given that they are trained using data that is broadly available on the internet. To that end, we present results from an empirical exploration, where a human expert-generated SE artifact was taken as a benchmark, parsed, and fed into various LLMs through prompt engineering to generate segments of typical SE artifacts. This procedure was applied without any fine-tuning or calibration to document baseline LLM performance. We then adopted a two-fold mixed-methods approach to compare AI generated artifacts against the benchmark. First, we quantitatively compare the artifacts using natural language processing algorithms and find that when prompted carefully, the state-of-the-art algorithms cannot differentiate AI-generated artifacts from the human-expert benchmark. Second, we conduct a qualitative deep dive to investigate how they differ in terms of quality. We document that while the two-material appear very similar, AI generated artifacts exhibit serious failure modes that could be difficult to detect. We characterize these as: premature requirements definition, unsubstantiated numerical estimates, and propensity to overspecify. We contend that this study tells a cautionary tale about why the SE community must be more cautious adopting AI suggested feedback, at least when generated by multi-purpose LLMs.

Title: A CNN Approach to Automated Detection and Classification of Brain Tumors

• Authors: Md. Zahid Hasan, Abdullah Tamim, D.M. Asadujjaman, Md. Mahfujur Rahman, Md. Abu Ahnaf Mollick, Nosin Anjum Dristi, Abdullah-Al-Noman
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09731
• Pdf URL: https://arxiv.org/pdf/2502.09731
• Copy Paste: [[2502.09731]] A CNN Approach to Automated Detection and Classification of Brain Tumors(https://arxiv.org/abs/2502.09731)
• Keywords: diffusion
• Abstract: Brain tumors require an assessment to ensure timely diagnosis and effective patient treatment. Morphological factors such as size, location, texture, and variable appearance com- plicate tumor inspection. Medical imaging presents challenges, including noise and incomplete images. This research article presents a methodology for processing Magnetic Resonance Imag- ing (MRI) data, encompassing techniques for image classification and denoising. The effective use of MRI images allows medical professionals to detect brain disorders, including tumors. This research aims to categorize healthy brain tissue and brain tumors by analyzing the provided MRI data. Unlike alternative methods like Computed Tomography (CT), MRI technology offers a more detailed representation of internal anatomical components, mak- ing it a suitable option for studying data related to brain tumors. The MRI picture is first subjected to a denoising technique utilizing an Anisotropic diffusion filter. The dataset utilized for the models creation is a publicly accessible and validated Brain Tumour Classification (MRI) database, comprising 3,264 brain MRI scans. SMOTE was employed for data augmentation and dataset balancing. Convolutional Neural Networks(CNN) such as ResNet152V2, VGG, ViT, and EfficientNet were employed for the classification procedure. EfficientNet attained an accuracy of 98%, the highest recorded.

Title: Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting

• Authors: Mahad Ali, Curtis Lisle, Patrick W. Moore, Tammer Barkouki, Brian J. Kirkwood, Laura J. Brattain
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09744
• Pdf URL: https://arxiv.org/pdf/2502.09744
• Copy Paste: [[2502.09744]] Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting(https://arxiv.org/abs/2502.09744)
• Keywords: foundation model
• Abstract: Federated Learning (FL) provides a decentralized machine learning approach, where multiple devices or servers collaboratively train a model without sharing their raw data, thus enabling data privacy. This approach has gained significant interest in academia and industry due to its privacy-preserving properties, which are particularly valuable in the medical domain where data availability is often protected under strict regulations. A relatively unexplored area is the use of FL to fine-tune Foundation Models (FMs) for time series forecasting, potentially enhancing model efficacy by overcoming data limitation while maintaining privacy. In this paper, we fine-tuned time series FMs with Electrocardiogram (ECG) and Impedance Cardiography (ICG) data using different FL techniques. We then examined various scenarios and discussed the challenges FL faces under different data heterogeneity configurations. Our empirical results demonstrated that while FL can be effective for fine-tuning FMs on time series forecasting tasks, its benefits depend on the data distribution across clients. We highlighted the trade-offs in applying FL to FM fine-tuning.

Title: The Widespread Adoption of Large Language Model-Assisted Writing Across Society

• Authors: Weixin Liang, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, James Zou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09747
• Pdf URL: https://arxiv.org/pdf/2502.09747
• Copy Paste: [[2502.09747]] The Widespread Adoption of Large Language Model-Assisted Writing Across Society(https://arxiv.org/abs/2502.09747)
• Keywords: generative
• Abstract: The recent advances in large language models (LLMs) attracted significant public and policymaker interest in its adoption patterns. In this paper, we systematically analyze LLM-assisted writing across four domains-consumer complaints, corporate communications, job postings, and international organization press releases-from January 2022 to September 2024. Our dataset includes 687,241 consumer complaints, 537,413 corporate press releases, 304.3 million job postings, and 15,919 United Nations (UN) press releases. Using a robust population-level statistical framework, we find that LLM usage surged following the release of ChatGPT in November 2022. By late 2024, roughly 18% of financial consumer complaint text appears to be LLM-assisted, with adoption patterns spread broadly across regions and slightly higher in urban areas. For corporate press releases, up to 24% of the text is attributable to LLMs. In job postings, LLM-assisted writing accounts for just below 10% in small firms, and is even more common among younger firms. UN press releases also reflect this trend, with nearly 14% of content being generated or modified by LLMs. Although adoption climbed rapidly post-ChatGPT, growth appears to have stabilized by 2024, reflecting either saturation in LLM adoption or increasing subtlety of more advanced models. Our study shows the emergence of a new reality in which firms, consumers and even international organizations substantially rely on generative AI for communications.

Title: Non-Markovian Discrete Diffusion with Causal Language Models

• Authors: Yangtian Zhang, Sizhuang He, Daniel Levine, Lawrence Zhao, David Zhang, Syed A Rizvi, Emanuele Zappala, Rex Ying, David van Dijk
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09767
• Pdf URL: https://arxiv.org/pdf/2502.09767
• Copy Paste: [[2502.09767]] Non-Markovian Discrete Diffusion with Causal Language Models(https://arxiv.org/abs/2502.09767)
• Keywords: diffusion
• Abstract: Discrete diffusion models have emerged as a flexible and controllable paradigm for structured sequence modeling, yet they still lag behind causal language models in expressiveness. To bridge the gap between two paradigms, we introduce CaDDi, a causal discrete diffusion model that unifies sequential and temporal modeling within a non-Markovian diffusion framework. Unlike conventional diffusion models that operate step by step with no access to prior states, CaDDi integrates the temporal trajectory, enabling more expressive and controllable generation. Our approach also treats causal language models as a special case, allowing seamless adoption of pretrained large language models (LLMs) for discrete diffusion without the need for architectural modifications. Empirically, we demonstrate that CaDDi outperforms state-of-the-art discrete diffusion models on both natural language and biological sequence tasks, narrowing the gap between diffusion-based methods and large-scale autoregressive transformers.

Title: Noise Controlled CT Super-Resolution with Conditional Diffusion Model

• Authors: Yuang Wang, Siyeop Yoon, Rui Hu, Baihui Yu, Duhgoon Lee, Rajiv Gupta, Li Zhang, Zhiqiang Chen, Dufan Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09793
• Pdf URL: https://arxiv.org/pdf/2502.09793
• Copy Paste: [[2502.09793]] Noise Controlled CT Super-Resolution with Conditional Diffusion Model(https://arxiv.org/abs/2502.09793)
• Keywords: diffusion
• Abstract: Improving the spatial resolution of CT images is a meaningful yet challenging task, often accompanied by the issue of noise amplification. This article introduces an innovative framework for noise-controlled CT super-resolution utilizing the conditional diffusion model. The model is trained on hybrid datasets, combining noise-matched simulation data with segmented details from real data. Experimental results with real CT images validate the effectiveness of our proposed framework, showing its potential for practical applications in CT imaging.

Title: A Solver-Aided Hierarchical Language for LLM-Driven CAD Design

• Authors: Benjamin T. Jones, Felix Hähnlein, Zihan Zhang, Maaz Ahmad, Vladimir Kim, Adriana Schulz
• Subjects: cs.CV, cs.AI, cs.GR, cs.LG, cs.PL
• Abstract URL: https://arxiv.org/abs/2502.09819
• Pdf URL: https://arxiv.org/pdf/2502.09819
• Copy Paste: [[2502.09819]] A Solver-Aided Hierarchical Language for LLM-Driven CAD Design(https://arxiv.org/abs/2502.09819)
• Keywords: generative
• Abstract: Large language models (LLMs) have been enormously successful in solving a wide variety of structured and unstructured generative tasks, but they struggle to generate procedural geometry in Computer Aided Design (CAD). These difficulties arise from an inability to do spatial reasoning and the necessity to guide a model through complex, long range planning to generate complex geometry. We enable generative CAD Design with LLMs through the introduction of a solver-aided, hierarchical domain specific language (DSL) called AIDL, which offloads the spatial reasoning requirements to a geometric constraint solver. Additionally, we show that in the few-shot regime, AIDL outperforms even a language with in-training data (OpenSCAD), both in terms of generating visual results closer to the prompt and creating objects that are easier to post-process and reason about.

Title: Solving Empirical Bayes via Transformers

• Authors: Anzo Teh, Mark Jabbour, Yury Polyanskiy
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09844
• Pdf URL: https://arxiv.org/pdf/2502.09844
• Copy Paste: [[2502.09844]] Solving Empirical Bayes via Transformers(https://arxiv.org/abs/2502.09844)
• Keywords: in-context
• Abstract: This work applies modern AI tools (transformers) to solving one of the oldest statistical problems: Poisson means under empirical Bayes (Poisson-EB) setting. In Poisson-EB a high-dimensional mean vector $\theta$ (with iid coordinates sampled from an unknown prior $\pi$) is estimated on the basis of $X=\mathrm{Poisson}(\theta)$. A transformer model is pre-trained on a set of synthetically generated pairs $(X,\theta)$ and learns to do in-context learning (ICL) by adapting to unknown $\pi$. Theoretically, we show that a sufficiently wide transformer can achieve vanishing regret with respect to an oracle estimator who knows $\pi$ as dimension grows to infinity. Practically, we discover that already very small models (100k parameters) are able to outperform the best classical algorithm (non-parametric maximum likelihood, or NPMLE) both in runtime and validation loss, which we compute on out-of-distribution synthetic data as well as real-world datasets (NHL hockey, MLB baseball, BookCorpusOpen). Finally, by using linear probes, we confirm that the transformer's EB estimator appears to internally work differently from either NPMLE or Robbins' estimators.

Title: Solvable Dynamics of Self-Supervised Word Embeddings and the Emergence of Analogical Reasoning

• Authors: Dhruva Karkada, James B. Simon, Yasaman Bahri, Michael R. DeWeese
• Subjects: cs.LG, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09863
• Pdf URL: https://arxiv.org/pdf/2502.09863
• Copy Paste: [[2502.09863]] Solvable Dynamics of Self-Supervised Word Embeddings and the Emergence of Analogical Reasoning(https://arxiv.org/abs/2502.09863)
• Keywords: self-supervised
• Abstract: The remarkable success of large language models relies on their ability to implicitly learn structured latent representations from the pretraining corpus. As a simpler surrogate for representation learning in language modeling, we study a class of solvable contrastive self-supervised algorithms which we term quadratic word embedding models. These models resemble the word2vec algorithm and perform similarly on downstream tasks. Our main contributions are analytical solutions for both the training dynamics (under certain hyperparameter choices) and the final word embeddings, given in terms of only the corpus statistics. Our solutions reveal that these models learn orthogonal linear subspaces one at a time, each one incrementing the effective rank of the embeddings until model capacity is saturated. Training on WikiText, we find that the top subspaces represent interpretable concepts. Finally, we use our dynamical theory to predict how and when models acquire the ability to complete analogies.

Title: Compression-Aware One-Step Diffusion Model for JPEG Artifact Removal

• Authors: Jinpei Guo, Zheng Chen, Wenbo Li, Yong Guo, Yulun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09873
• Pdf URL: https://arxiv.org/pdf/2502.09873
• Copy Paste: [[2502.09873]] Compression-Aware One-Step Diffusion Model for JPEG Artifact Removal(https://arxiv.org/abs/2502.09873)
• Keywords: diffusion
• Abstract: Diffusion models have demonstrated remarkable success in image restoration tasks. However, their multi-step denoising process introduces significant computational overhead, limiting their practical deployment. Furthermore, existing methods struggle to effectively remove severe JPEG artifact, especially in highly compressed images. To address these challenges, we propose CODiff, a compression-aware one-step diffusion model for JPEG artifact removal. The core of CODiff is the compression-aware visual embedder (CaVE), which extracts and leverages JPEG compression priors to guide the diffusion model. We propose a dual learning strategy that combines explicit and implicit learning. Specifically, explicit learning enforces a quality prediction objective to differentiate low-quality images with different compression levels. Implicit learning employs a reconstruction objective that enhances the model's generalization. This dual learning allows for a deeper and more comprehensive understanding of JPEG compression. Experimental results demonstrate that CODiff surpasses recent leading methods in both quantitative and visual quality metrics. The code and models will be released at this https URL.

Title: Symmetry-Preserving Diffusion Models via Target Symmetrization

• Authors: Vinh Tong, Yun Ye, Trung-Dung Hoang, Anji Liu, Guy Van den Broeck, Mathias Niepert
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09890
• Pdf URL: https://arxiv.org/pdf/2502.09890
• Copy Paste: [[2502.09890]] Symmetry-Preserving Diffusion Models via Target Symmetrization(https://arxiv.org/abs/2502.09890)
• Keywords: diffusion, generative
• Abstract: Diffusion models are powerful tools for capturing complex distributions, but modeling data with inherent symmetries, such as molecular structures, remains challenging. Equivariant denoisers are commonly used to address this, but they introduce architectural complexity and optimization challenges, including noisy gradients and convergence issues. We propose a novel approach that enforces equivariance through a symmetrized loss function, which applies a time-dependent weighted averaging operation over group actions to the model's prediction target. This ensures equivariance without explicit architectural constraints and reduces gradient variance, leading to more stable and efficient optimization. Our method uses Monte Carlo sampling to estimate the average, incurring minimal computational overhead. We provide theoretical guarantees of equivariance for the minimizer of our loss function and demonstrate its effectiveness on synthetic datasets and the molecular conformation generation task using the GEOM-QM9 dataset. Experiments show improved sample quality compared to existing methods, highlighting the potential of our approach to enhance the scalability and practicality of equivariant diffusion models in generative tasks.

Title: Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding

• Authors: Thanh-Dat Truong, Hoang-Quan Nguyen, Xuan-Bac Nguyen, Ashley Dowling, Xin Li, Khoa Luu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09906
• Pdf URL: https://arxiv.org/pdf/2502.09906
• Copy Paste: [[2502.09906]] Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding(https://arxiv.org/abs/2502.09906)
• Keywords: self-supervised, foundation model, generative
• Abstract: Multimodal conversational generative AI has shown impressive capabilities in various vision and language understanding through learning massive text-image data. However, current conversational models still lack knowledge about visual insects since they are often trained on the general knowledge of vision-language data. Meanwhile, understanding insects is a fundamental problem in precision agriculture, helping to promote sustainable development in agriculture. Therefore, this paper proposes a novel multimodal conversational model, Insect-LLaVA, to promote visual understanding in insect-domain knowledge. In particular, we first introduce a new large-scale Multimodal Insect Dataset with Visual Insect Instruction Data that enables the capability of learning the multimodal foundation models. Our proposed dataset enables conversational models to comprehend the visual and semantic features of the insects. Second, we propose a new Insect-LLaVA model, a new general Large Language and Vision Assistant in Visual Insect Understanding. Then, to enhance the capability of learning insect features, we develop an Insect Foundation Model by introducing a new micro-feature self-supervised learning with a Patch-wise Relevant Attention mechanism to capture the subtle differences among insect images. We also present Description Consistency loss to improve micro-feature learning via text descriptions. The experimental results evaluated on our new Visual Insect Question Answering benchmarks illustrate the effective performance of our proposed approach in visual insect understanding and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks.

Title: Robust Anomaly Detection via Tensor Chidori Pseudoskeleton Decomposition

• Authors: Bowen Su
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09926
• Pdf URL: https://arxiv.org/pdf/2502.09926
• Copy Paste: [[2502.09926]] Robust Anomaly Detection via Tensor Chidori Pseudoskeleton Decomposition(https://arxiv.org/abs/2502.09926)
• Keywords: anomaly
• Abstract: Anomaly detection plays a critical role in modern data-driven applications, from identifying fraudulent transactions and safeguarding network infrastructure to monitoring sensor systems for irregular patterns. Traditional approaches, such as distance, density, or cluster-based methods, face significant challenges when applied to high dimensional tensor data, where complex interdependencies across dimensions amplify noise and computational complexity. To address these limitations, this paper leverages Tensor Chidori pseudoskeleton decomposition within a tensor-robust principal component analysis framework to extract low Tucker rank structure while isolating sparse anomalies, ensuring robustness to anomaly detection. We establish theoretical results regarding convergence, and estimation error, demonstrating the stability and accuracy of the proposed approach. Numerical experiments on real-world spatiotemporal data from New York City taxi trip records validate the superiority of the proposed method in detecting anomalous urban events compared to existing benchmark methods. The results underscore the potential of Tensor Chidori pseudoskeleton decomposition to enhance anomaly detection for large-scale, high-dimensional data.

Title: Precise Parameter Localization for Textual Generation in Diffusion Models

• Authors: Łukasz Staniszewski, Bartosz Cywiński, Franziska Boenisch, Kamil Deja, Adam Dziedzic
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09935
• Pdf URL: https://arxiv.org/pdf/2502.09935
• Copy Paste: [[2502.09935]] Precise Parameter Localization for Textual Generation in Diffusion Models(https://arxiv.org/abs/2502.09935)
• Keywords: diffusion
• Abstract: Novel diffusion models can synthesize photo-realistic images with integrated high-quality text. Surprisingly, we demonstrate through attention activation patching that only less than 1% of diffusion models' parameters, all contained in attention layers, influence the generation of textual content within the images. Building on this observation, we improve textual generation efficiency and performance by targeting cross and joint attention layers of diffusion models. We introduce several applications that benefit from localizing the layers responsible for textual content generation. We first show that a LoRA-based fine-tuning solely of the localized layers enhances, even more, the general text-generation capabilities of large diffusion models while preserving the quality and diversity of the diffusion models' generations. Then, we demonstrate how we can use the localized layers to edit textual content in generated images. Finally, we extend this idea to the practical use case of preventing the generation of toxic text in a cost-free manner. In contrast to prior work, our localization approach is broadly applicable across various diffusion model architectures, including U-Net (e.g., LDM and SDXL) and transformer-based (e.g., DeepFloyd IF and Stable Diffusion 3), utilizing diverse text encoders (e.g., from CLIP to the large language models like T5). Project page available at this https URL.

Title: Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization

• Authors: Weiran Xu, Kengo Hirami, Koji Eguchi
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09944
• Pdf URL: https://arxiv.org/pdf/2502.09944
• Copy Paste: [[2502.09944]] Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization(https://arxiv.org/abs/2502.09944)
• Keywords: self-supervised
• Abstract: In our study, we propose a self-supervised neural topic model (NTM) that combines the power of NTMs and regularized self-supervised learning methods to improve performance. NTMs use neural networks to learn latent topics hidden behind the words in documents, enabling greater flexibility and the ability to estimate more coherent topics compared to traditional topic models. On the other hand, some self-supervised learning methods use a joint embedding architecture with two identical networks that produce similar representations for two augmented versions of the same input. Regularizations are applied to these representations to prevent collapse, which would otherwise result in the networks outputting constant or redundant representations for all inputs. Our model enhances topic quality by explicitly regularizing latent topic representations of anchor and positive samples. We also introduced an adversarial data augmentation method to replace the heuristic sampling method. We further developed several variation models including those on the basis of an NTM that incorporates contrastive learning with both positive and negative samples. Experimental results on three datasets showed that our models outperformed baselines and state-of-the-art models both quantitatively and qualitatively.

Title: KGGen: Extracting Knowledge Graphs from Plain Text with Language Models

• Authors: Belinda Mo, Kyssen Yu, Joshua Kazdan, Proud Mpala, Lisa Yu, Chris Cundy, Charilaos Kanatsoulis, Sanmi Koyejo
• Subjects: cs.CL, cs.AI, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09956
• Pdf URL: https://arxiv.org/pdf/2502.09956
• Copy Paste: [[2502.09956]] KGGen: Extracting Knowledge Graphs from Plain Text with Language Models(https://arxiv.org/abs/2502.09956)
• Keywords: foundation model
• Abstract: Recent interest in building foundation models for KGs has highlighted a fundamental challenge: knowledge-graph data is relatively scarce. The best-known KGs are primarily human-labeled, created by pattern-matching, or extracted using early NLP techniques. While human-generated KGs are in short supply, automatically extracted KGs are of questionable quality. We present a solution to this data scarcity problem in the form of a text-to-KG generator (KGGen), a package that uses language models to create high-quality graphs from plaintext. Unlike other KG extractors, KGGen clusters related entities to reduce sparsity in extracted KGs. KGGen is available as a Python library (\texttt{pip install kg-gen}), making it accessible to everyone. Along with KGGen, we release the first benchmark, Measure of of Information in Nodes and Edges (MINE), that tests an extractor's ability to produce a useful KG from plain text. We benchmark our new tool against existing extractors and demonstrate far superior performance.

Title: Generating on Generated: An Approach Towards Self-Evolving Diffusion Models

• Authors: Xulu Zhang, Xiaoyong Wei, Jinlin Wu, Jiaxin Wu, Zhaoxiang Zhang, Zhen Lei, Qing Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09963
• Pdf URL: https://arxiv.org/pdf/2502.09963
• Copy Paste: [[2502.09963]] Generating on Generated: An Approach Towards Self-Evolving Diffusion Models(https://arxiv.org/abs/2502.09963)
• Keywords: diffusion, generative
• Abstract: Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data. We identify two key factors contributing to this collapse: the lack of perceptual alignment and the accumulation of generative hallucinations. To mitigate these issues, we propose three strategies: (1) a prompt construction and filtering pipeline designed to facilitate the generation of perceptual aligned data, (2) a preference sampling method to identify human-preferred samples and filter out generative hallucinations, and (3) a distribution-based weighting scheme to penalize selected samples with hallucinatory errors. Our extensive experiments validate the effectiveness of these approaches.

Title: Large Language Diffusion Models

• Authors: Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, Chongxuan Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09992
• Pdf URL: https://arxiv.org/pdf/2502.09992
• Copy Paste: [[2502.09992]] Large Language Diffusion Models(https://arxiv.org/abs/2502.09992)
• Keywords: diffusion, generative, in-context
• Abstract: Autoregressive models (ARMs) are widely regarded as the cornerstone of large language models (LLMs). We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm. LLaDA models distributions through a forward data masking process and a reverse process, parameterized by a vanilla Transformer to predict masked tokens. By optimizing a likelihood bound, it provides a principled generative approach for probabilistic inference. Across extensive benchmarks, LLaDA demonstrates strong scalability, outperforming our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings establish diffusion models as a viable and promising alternative to ARMs, challenging the assumption that key LLM capabilities discussed above are inherently tied to ARMs.

Title: RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control

• Authors: Teng Li, Guangcong Zheng, Rui Jiang, Shuigenzhan, Tao Wu, Yehao Lu, Yining Lin, Xi Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10059
• Pdf URL: https://arxiv.org/pdf/2502.10059
• Copy Paste: [[2502.10059]] RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control(https://arxiv.org/abs/2502.10059)
• Keywords: diffusion, generative
• Abstract: Recent advancements in camera-trajectory-guided image-to-video generation offer higher precision and better support for complex camera control compared to text-based approaches. However, they also introduce significant usability challenges, as users often struggle to provide precise camera parameters when working with arbitrary real-world images without knowledge of their depth nor scene scale. To address these real-world application issues, we propose RealCam-I2V, a novel diffusion-based video generation framework that integrates monocular metric depth estimation to establish 3D scene reconstruction in a preprocessing step. During training, the reconstructed 3D scene enables scaling camera parameters from relative to absolute values, ensuring compatibility and scale consistency across diverse real-world images. In inference, RealCam-I2V offers an intuitive interface where users can precisely draw camera trajectories by dragging within the 3D scene. To further enhance precise camera control and scene consistency, we propose scene-constrained noise shaping, which shapes high-level noise and also allows the framework to maintain dynamic, coherent video generation in lower noise stages. RealCam-I2V achieves significant improvements in controllability and video quality on the RealEstate10K and out-of-domain images. We further enables applications like camera-controlled looping video generation and generative frame interpolation. We will release our absolute-scale annotation, codes, and all checkpoints. Please see dynamic results in this https URL.

Title: A novel approach to data generation in generative model

• Authors: JaeHong Kim (1), Jaewon Shim (2) ((1) Healthcare, Legal and Policy Center, Graduate school of Law, Korea University, Seoul 02841, Korea, Human-Inspired AI Research, Korea University, Seoul 02841, Korea , (2) Center for 0D Nanofluidics, Institute of Applied Physics, Department of Physics and Astronomy, Seoul National University, Seoul 08826, Korea)
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10092
• Pdf URL: https://arxiv.org/pdf/2502.10092
• Copy Paste: [[2502.10092]] A novel approach to data generation in generative model(https://arxiv.org/abs/2502.10092)
• Keywords: generative
• Abstract: Variational Autoencoders (VAEs) and other generative models are widely employed in artificial intelligence to synthesize new data. However, current approaches rely on Euclidean geometric assumptions and statistical approximations that fail to capture the structured and emergent nature of data generation. This paper introduces the Convergent Fusion Paradigm (CFP) theory, a novel geometric framework that redefines data generation by integrating dimensional expansion accompanied by qualitative transformation. By modifying the latent space geometry to interact with emergent high-dimensional structures, CFP theory addresses key challenges such as identifiability issues and unintended artifacts like hallucinations in Large Language Models (LLMs). CFP theory is based on two key conceptual hypotheses that redefine how generative models structure relationships between data and algorithms. Through the lens of CFP theory, we critically examine existing metric-learning approaches. CFP theory advances this perspective by introducing time-reversed metric embeddings and structural convergence mechanisms, leading to a novel geometric approach that better accounts for data generation as a structured epistemic process. Beyond its computational implications, CFP theory provides philosophical insights into the ontological underpinnings of data generation. By offering a systematic framework for high-dimensional learning dynamics, CFP theory contributes to establishing a theoretical foundation for understanding the data-relationship structures in AI. Finally, future research in CFP theory will be led to its implications for fully realizing qualitative transformations, introducing the potential of Hilbert space in generative modeling.

Title: From Markov to Laplace: How Mamba In-Context Learns Markov Chains

• Authors: Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva
• Subjects: cs.LG, cs.AI, cs.IT
• Abstract URL: https://arxiv.org/abs/2502.10178
• Pdf URL: https://arxiv.org/pdf/2502.10178
• Copy Paste: [[2502.10178]] From Markov to Laplace: How Mamba In-Context Learns Markov Chains(https://arxiv.org/abs/2502.10178)
• Keywords: in-context
• Abstract: While transformer-based language models have driven the AI revolution thus far, their computational complexity has spurred growing interest in viable alternatives, such as structured state space sequence models (SSMs) and Selective SSMs. Among these, Mamba (S6) and its variant Mamba-2 have shown remarkable inference speed ups over transformers while achieving comparable or superior performance on complex language modeling tasks. However, despite these architectural innovations and empirical successes, the fundamental learning capabilities of Mamba remain poorly understood. In this paper, we address this gap by studying in-context learning (ICL) on Markov chains and uncovering a surprising phenomenon: unlike transformers, even a single-layer Mamba efficiently learns the in-context Laplacian smoothing estimator, which is both Bayes and minimax optimal, for all Markovian orders. To explain this, we theoretically characterize the representation capacity of Mamba and reveal the fundamental role of convolution in enabling it to represent the optimal Laplacian smoothing. These theoretical insights align strongly with empirical results and, to the best of our knowledge, represent the first formal connection between Mamba and optimal statistical estimators. Finally, we outline promising research directions inspired by these findings.

Title: Control-flow anomaly detection by process mining-based feature extraction and dimensionality reduction

• Authors: Francesco Vitale, Marco Pegoraro, Wil M. P. van der Aalst, Nicola Mazzocca
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10211
• Pdf URL: https://arxiv.org/pdf/2502.10211
• Copy Paste: [[2502.10211]] Control-flow anomaly detection by process mining-based feature extraction and dimensionality reduction(https://arxiv.org/abs/2502.10211)
• Keywords: anomaly
• Abstract: The business processes of organizations may deviate from normal control flow due to disruptive anomalies, including unknown, skipped, and wrongly-ordered activities. To identify these control-flow anomalies, process mining can check control-flow correctness against a reference process model through conformance checking, an explainable set of algorithms that allows linking any deviations with model elements. However, the effectiveness of conformance checking-based techniques is negatively affected by noisy event data and low-quality process models. To address these shortcomings and support the development of competitive and explainable conformance checking-based techniques for control-flow anomaly detection, we propose a novel process mining-based feature extraction approach with alignment-based conformance checking. This variant aligns the deviating control flow with a reference process model; the resulting alignment can be inspected to extract additional statistics such as the number of times a given activity caused mismatches. We integrate this approach into a flexible and explainable framework for developing techniques for control-flow anomaly detection. The framework combines process mining-based feature extraction and dimensionality reduction to handle high-dimensional feature sets, achieve detection effectiveness, and support explainability. The results show that the framework techniques implementing our approach outperform the baseline conformance checking-based techniques while maintaining the explainable nature of conformance checking. We also provide an explanation of why existing conformance checking-based techniques may be ineffective.

Title: Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control

• Authors: Thomas Jiralerspong, Berton Earnshaw, Jason Hartford, Yoshua Bengio, Luca Scimeca
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10236
• Pdf URL: https://arxiv.org/pdf/2502.10236
• Copy Paste: [[2502.10236]] Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control(https://arxiv.org/abs/2502.10236)
• Keywords: diffusion, generative
• Abstract: Diffusion Probabilistic Models (DPMs) are powerful generative models that have achieved unparalleled success in a number of generative tasks. In this work, we aim to build inductive biases into the training and sampling of diffusion models to better accommodate the target distribution of the data to model. For topologically structured data, we devise a frequency-based noising operator to purposefully manipulate, and set, these inductive biases. We first show that appropriate manipulations of the noising forward process can lead DPMs to focus on particular aspects of the distribution to learn. We show that different datasets necessitate different inductive biases, and that appropriate frequency-based noise control induces increased generative performance compared to standard diffusion. Finally, we demonstrate the possibility of ignoring information at particular frequencies while learning. We show this in an image corruption and recovery task, where we train a DPM to recover the original target distribution after severe noise corruption.

Title: Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model

• Authors: Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, Yu Zhou, Deshan Sun, Deyu Zhou, Jian Zhou, Kaijun Tan, Kang An, Mei Chen, Wei Ji, Qiling Wu, Wen Sun, Xin Han, Yanan Wei, Zheng Ge, Aojie Li, Bin Wang, Bizhu Huang, Bo Wang, Brian Li, Changxing Miao, Chen Xu, Chenfei Wu, Chenguang Yu, Dapeng Shi, Dingyuan Hu, Enle Liu, Gang Yu, Ge Yang, Guanzhe Huang, Gulin Yan, Haiyang Feng, Hao Nie, Haonan Jia, Hanpeng Hu, Hanqi Chen, Haolong Yan, Heng Wang, Hongcheng Guo, Huilin Xiong, Huixin Xiong, Jiahao Gong, Jianchang Wu, Jiaoren Wu, Jie Wu, Jie Yang, Jiashuai Liu, Jiashuo Li, Jingyang Zhang, Junjing Guo, Junzhe Lin, Kaixiang Li, Lei Liu, Lei Xia, Liang Zhao, Liguo Tan, Liwen Huang, Liying Shi, Ming Li, Mingliang Li, Muhua Cheng, Na Wang, Qiaohui Chen, Qinglin He, Qiuyan Liang, Quan Sun, Ran Sun, Rui Wang, Shaoliang Pang, Shiliang Yang, Sitong Liu, Siqi Liu, Shuli Gao, Tiancheng Cao, Tianyu Wang, Weipeng Ming, Wenqing He, Xu Zhao, Xuelin Zhang, Xianfang Zeng, Xiaojia Liu, Xuan Yang, Yaqi Dai, Yanbo Yu, Yang Li, Yineng Deng, Yingming Wang, Yilei Wang, Yuanwei Lu, Yu Chen, Yu Luo, Yuchu Luo
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10248
• Pdf URL: https://arxiv.org/pdf/2502.10248
• Copy Paste: [[2502.10248]] Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model(https://arxiv.org/abs/2502.10248)
• Keywords: diffusion, foundation model
• Abstract: We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model with 30B parameters and the ability to generate videos up to 204 frames in length. A deep compression Variational Autoencoder, Video-VAE, is designed for video generation tasks, achieving 16x16 spatial and 8x temporal compression ratios, while maintaining exceptional video reconstruction quality. User prompts are encoded using two bilingual text encoders to handle both English and Chinese. A DiT with 3D full attention is trained using Flow Matching and is employed to denoise input noise into latent frames. A video-based DPO approach, Video-DPO, is applied to reduce artifacts and improve the visual quality of the generated videos. We also detail our training strategies and share key observations and insights. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, Step-Video-T2V-Eval, demonstrating its state-of-the-art text-to-video quality when compared with both open-source and commercial engines. Additionally, we discuss the limitations of current diffusion-based model paradigm and outline future directions for video foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval available at this https URL. The online version can be accessed from this https URL as well. Our goal is to accelerate the innovation of video foundation models and empower video content creators.

Title: Probabilistic Super-Resolution for High-Fidelity Physical System Simulations with Uncertainty Quantification

• Authors: Pengyu Zhang, Connor Duffin, Alex Glyn-Davies, Arnaud Vadeboncoeur, Mark Girolami
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10280
• Pdf URL: https://arxiv.org/pdf/2502.10280
• Copy Paste: [[2502.10280]] Probabilistic Super-Resolution for High-Fidelity Physical System Simulations with Uncertainty Quantification(https://arxiv.org/abs/2502.10280)
• Keywords: generative
• Abstract: Super-resolution (SR) is a promising tool for generating high-fidelity simulations of physical systems from low-resolution data, enabling fast and accurate predictions in engineering applications. However, existing deep-learning based SR methods, require large labeled datasets and lack reliable uncertainty quantification (UQ), limiting their applicability in real-world scenarios. To overcome these challenges, we propose a probabilistic SR framework that leverages the Statistical Finite Element Method and energy-based generative modeling. Our method enables efficient high-resolution predictions with inherent UQ, while eliminating the need for extensive labeled datasets. The method is validated on a 2D Poisson example and compared with bicubic interpolation upscaling. Results demonstrate a computational speed-up over high-resolution numerical solvers while providing reliable uncertainty estimates.

Title: Anomaly Detection with LWE Encrypted Control

• Authors: Rijad Alisic, Junsoo Kim, Henrik Sandberg
• Subjects: cs.CR, eess.SY
• Abstract URL: https://arxiv.org/abs/2502.10283
• Pdf URL: https://arxiv.org/pdf/2502.10283
• Copy Paste: [[2502.10283]] Anomaly Detection with LWE Encrypted Control(https://arxiv.org/abs/2502.10283)
• Keywords: anomaly
• Abstract: Detecting attacks using encrypted signals is challenging since encryption hides its information content. We present a novel mechanism for anomaly detection over Learning with Errors (LWE) encrypted signals without using decryption, secure channels, nor complex communication schemes. Instead, the detector exploits the homomorphic property of LWE encryption to perform hypothesis tests on transformations of the encrypted samples. The specific transformations are determined by solutions to a hard lattice-based minimization problem. While the test's sensitivity deteriorates with suboptimal solutions, similar to the exponential deterioration of the (related) test that breaks the cryptosystem, we show that the deterioration is polynomial for our test. This rate gap can be exploited to pick parameters that lead to somewhat weaker encryption but large gains in detection capability. Finally, we conclude the paper by presenting a numerical example that simulates anomaly detection, demonstrating the effectiveness of our method in identifying attacks.

Title: SPIRIT: Short-term Prediction of solar IRradIance for zero-shot Transfer learning using Foundation Models

• Authors: Aditya Mishra, Ravindra T, Srinivasan Iyengar, Shivkumar Kalyanaraman, Ponnurangam Kumaraguru
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10307
• Pdf URL: https://arxiv.org/pdf/2502.10307
• Copy Paste: [[2502.10307]] SPIRIT: Short-term Prediction of solar IRradIance for zero-shot Transfer learning using Foundation Models(https://arxiv.org/abs/2502.10307)
• Keywords: foundation model
• Abstract: Traditional solar forecasting models are based on several years of site-specific historical irradiance data, often spanning five or more years, which are unavailable for newer photovoltaic farms. As renewable energy is highly intermittent, building accurate solar irradiance forecasting systems is essential for efficient grid management and enabling the ongoing proliferation of solar energy, which is crucial to achieve the United Nations' net zero goals. In this work, we propose SPIRIT, a novel approach leveraging foundation models for solar irradiance forecasting, making it applicable to newer solar installations. Our approach outperforms state-of-the-art models in zero-shot transfer learning by about 70%, enabling effective performance at new locations without relying on any historical data. Further improvements in performance are achieved through fine-tuning, as more location-specific data becomes available. These findings are supported by statistical significance, further validating our approach. SPIRIT represents a pivotal step towards rapid, scalable, and adaptable solar forecasting solutions, advancing the integration of renewable energy into global power systems.

Title: ExplainReduce: Summarising local explanations via proxies

• Authors: Lauri Seppäläinen, Mudong Guo, Kai Puolamäki
• Subjects: cs.LG, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.10311
• Pdf URL: https://arxiv.org/pdf/2502.10311
• Copy Paste: [[2502.10311]] ExplainReduce: Summarising local explanations via proxies(https://arxiv.org/abs/2502.10311)
• Keywords: generative
• Abstract: Most commonly used non-linear machine learning methods are closed-box models, uninterpretable to humans. The field of explainable artificial intelligence (XAI) aims to develop tools to examine the inner workings of these closed boxes. An often-used model-agnostic approach to XAI involves using simple models as local approximations to produce so-called local explanations; examples of this approach include LIME, SHAP, and SLISEMAP. This paper shows how a large set of local explanations can be reduced to a small "proxy set" of simple models, which can act as a generative global explanation. This reduction procedure, ExplainReduce, can be formulated as an optimisation problem and approximated efficiently using greedy heuristics.

Title: DiOpt: Self-supervised Diffusion for Constrained Optimization

• Authors: Shutong Ding, Yimiao Zhou, Ke Hu, Xi Yao, Junchi Yan, Xiaoying Tang, Ye Shi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10330
• Pdf URL: https://arxiv.org/pdf/2502.10330
• Copy Paste: [[2502.10330]] DiOpt: Self-supervised Diffusion for Constrained Optimization(https://arxiv.org/abs/2502.10330)
• Keywords: diffusion, self-supervised
• Abstract: Recent advances in diffusion models show promising potential for learning-based optimization by leveraging their multimodal sampling capability to escape local optima. However, existing diffusion-based optimization approaches, often reliant on supervised training, lacks a mechanism to ensure strict constraint satisfaction which is often required in real-world applications. One resulting observation is the distributional misalignment, i.e. the generated solution distribution often exhibits small overlap with the feasible domain. In this paper, we propose DiOpt, a novel diffusion paradigm that systematically learns near-optimal feasible solution distributions through iterative self-training. Our framework introduces several key innovations: a target distribution specifically designed to maximize overlap with the constrained solution manifold; a bootstrapped self-training mechanism that adaptively weights candidate solutions based on the severity of constraint violations and optimality gaps; and a dynamic memory buffer that accelerates convergence by retaining high-quality solutions over training iterations. To our knowledge, DiOpt represents the first successful integration of self-supervised diffusion with hard constraint satisfaction. Evaluations on diverse tasks, including power grid control, motion retargeting, wireless allocation demonstrate its superiority in terms of both optimality and constraint satisfaction.

Title: InfoPos: A ML-Assisted Solution Design Support Framework for Industrial Cyber-Physical Systems

• Authors: Uraz Odyurt, Richard Loendersloot, Tiedo Tinga
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10331
• Pdf URL: https://arxiv.org/pdf/2502.10331
• Copy Paste: [[2502.10331]] InfoPos: A ML-Assisted Solution Design Support Framework for Industrial Cyber-Physical Systems(https://arxiv.org/abs/2502.10331)
• Keywords: anomaly
• Abstract: The variety of building blocks and algorithms incorporated in data-centric and ML-assisted solutions is high, contributing to two challenges: selection of most effective set and order of building blocks, as well as achieving such a selection with minimum cost. Considering that ML-assisted solution design is influenced by the extent of available data, as well as available knowledge of the target system, it is advantageous to be able to select matching building blocks. We introduce the first iteration of our InfoPos framework, allowing the placement of use-cases considering the available positions (levels), i.e., from poor to rich, of knowledge and data dimensions. With that input, designers and developers can reveal the most effective corresponding choice(s), streamlining the solution design process. The results from our demonstrator, an anomaly identification use-case for industrial Cyber-Physical Systems, reflects achieved effects upon the use of different building blocks throughout knowledge and data positions. The achieved ML model performance is considered as the indicator. Our data processing code and the composed data sets are publicly available.

Title: Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network

• Authors: Arun Kunwar, Dibakar Raj Pant, Jukka Heikkonen, Rajeev Kanth
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10334
• Pdf URL: https://arxiv.org/pdf/2502.10334
• Copy Paste: [[2502.10334]] Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network(https://arxiv.org/abs/2502.10334)
• Keywords: generative
• Abstract: The Convolutional Neural Network (CNN) has shown impressive performance in image classification because of its strong learning capabilities. However, it demands a substantial and balanced dataset for effective training. Otherwise, networks frequently exhibit over fitting and struggle to generalize to new examples. Publicly available dataset of fundus images of ocular disease is insufficient to train any classification model to achieve satisfactory accuracy. So, we propose Generative Adversarial Network(GAN) based data generation technique to synthesize dataset for training CNN based classification model and later use original disease containing ocular images to test the model. During testing the model classification accuracy with the original ocular image, the model achieves an accuracy rate of 78.6% for myopia, 88.6% for glaucoma, and 84.6% for cataract, with an overall classification accuracy of 84.6%.

Title: Dimension-free Score Matching and Time Bootstrapping for Diffusion Models

• Authors: Syamantak Kumar, Dheeraj Nagaraj, Purnamrita Sarkar
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10354
• Pdf URL: https://arxiv.org/pdf/2502.10354
• Copy Paste: [[2502.10354]] Dimension-free Score Matching and Time Bootstrapping for Diffusion Models(https://arxiv.org/abs/2502.10354)
• Keywords: diffusion, generative
• Abstract: Diffusion models generate samples by estimating the score function of the target distribution at various noise levels. The model is trained using samples drawn from the target distribution, progressively adding noise. In this work, we establish the first (nearly) dimension-free sample complexity bounds for learning these score functions, achieving a double exponential improvement in dimension over prior results. A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels, a critical feature of diffusion models in practice which enables generalization across timesteps. Our analysis introduces a novel martingale-based error decomposition and sharp variance bounds, enabling efficient learning from dependent data generated by Markov processes, which may be of independent interest. Building on these insights, we propose Bootstrapped Score Matching (BSM), a variance reduction technique that utilizes previously learned scores to improve accuracy at higher noise levels. These results provide crucial insights into the efficiency and effectiveness of diffusion models for generative modeling.

Title: AffinityFlow: Guided Flows for Antibody Affinity Maturation

• Authors: Can Chen, Karla-Luise Herpoldt, Chenchao Zhao, Zichen Wang, Marcus Collins, Shang Shang, Ron Benson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10365
• Pdf URL: https://arxiv.org/pdf/2502.10365
• Copy Paste: [[2502.10365]] AffinityFlow: Guided Flows for Antibody Affinity Maturation(https://arxiv.org/abs/2502.10365)
• Keywords: generative
• Abstract: Antibodies are widely used as therapeutics, but their development requires costly affinity maturation, involving iterative mutations to enhance binding this http URL paper explores a sequence-only scenario for affinity maturation, using solely antibody and antigen sequences. Recently AlphaFlow wraps AlphaFold within flow matching to generate diverse protein structures, enabling a sequence-conditioned generative model of structure. Building on this, we propose an alternating optimization framework that (1) fixes the sequence to guide structure generation toward high binding affinity using a structure-based affinity predictor, then (2) applies inverse folding to create sequence mutations, refined by a sequence-based affinity predictor for post selection. To address this, we develop a co-teaching module that incorporates valuable information from noisy biophysical energies into predictor refinement. The sequence-based predictor selects consensus samples to teach the structure-based predictor, and vice versa. Our method, AffinityFlow, achieves state-of-the-art performance in affinity maturation experiments. We plan to open-source our code after acceptance.

Title: ReStyle3D: Scene-Level Appearance Transfer with Semantic Correspondences

• Authors: Liyuan Zhu, Shengqu Cai, Shengyu Huang, Gordon Wetzstein, Naji Khosravan, Iro Armeni
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.10377
• Pdf URL: https://arxiv.org/pdf/2502.10377
• Copy Paste: [[2502.10377]] ReStyle3D: Scene-Level Appearance Transfer with Semantic Correspondences(https://arxiv.org/abs/2502.10377)
• Keywords: diffusion
• Abstract: We introduce ReStyle3D, a novel framework for scene-level appearance transfer from a single style image to a real-world scene represented by multiple views. The method combines explicit semantic correspondences with multi-view consistency to achieve precise and coherent stylization. Unlike conventional stylization methods that apply a reference style globally, ReStyle3D uses open-vocabulary segmentation to establish dense, instance-level correspondences between the style and real-world images. This ensures that each object is stylized with semantically matched textures. It first transfers the style to a single view using a training-free semantic-attention mechanism in a diffusion model. It then lifts the stylization to additional views via a learned warp-and-refine network guided by monocular depth and pixel-wise correspondences. Experiments show that ReStyle3D consistently outperforms prior methods in structure preservation, perceptual style similarity, and multi-view coherence. User studies further validate its ability to produce photo-realistic, semantically faithful results. Our code, pretrained models, and dataset will be publicly released, to support new applications in interior design, virtual staging, and 3D-consistent stylization.

Title: Region-Adaptive Sampling for Diffusion Transformers

• Authors: Ziming Liu, Yifan Yang, Chengruidong Zhang, Yiqi Zhang, Lili Qiu, Yang You, Yuqing Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10389
• Pdf URL: https://arxiv.org/pdf/2502.10389
• Copy Paste: [[2502.10389]] Region-Adaptive Sampling for Diffusion Transformers(https://arxiv.org/abs/2502.10389)
• Keywords: diffusion, generative
• Abstract: Diffusion models (DMs) have become the leading choice for generative tasks across diverse domains. However, their reliance on multiple sequential forward passes significantly limits real-time performance. Previous acceleration methods have primarily focused on reducing the number of sampling steps or reusing intermediate results, failing to leverage variations across spatial regions within the image due to the constraints of convolutional U-Net structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in handling variable number of tokens, we introduce RAS, a novel, training-free sampling strategy that dynamically assigns different sampling ratios to regions within an image based on the focus of the DiT model. Our key observation is that during each sampling step, the model concentrates on semantically meaningful regions, and these areas of focus exhibit strong continuity across consecutive steps. Leveraging this insight, RAS updates only the regions currently in focus, while other regions are updated using cached noise from the previous step. The model's focus is determined based on the output from the preceding step, capitalizing on the temporal consistency we observed. We evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up to 2.36x and 2.51x, respectively, with minimal degradation in generation quality. Additionally, a user study reveals that RAS delivers comparable qualities under human evaluation while achieving a 1.6x speedup. Our approach makes a significant step towards more efficient diffusion transformers, enhancing their potential for real-time applications.

Title: (How) Can Transformers Predict Pseudo-Random Numbers?

• Authors: Tao Tao, Darshil Doshi, Dayal Singh Kalra, Tianyu He, Maissam Barkeshli
• Subjects: cs.LG, cond-mat.dis-nn, cs.CR, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10390
• Pdf URL: https://arxiv.org/pdf/2502.10390
• Copy Paste: [[2502.10390]] (How) Can Transformers Predict Pseudo-Random Numbers?(https://arxiv.org/abs/2502.10390)
• Keywords: in-context
• Abstract: Transformers excel at discovering patterns in sequential data, yet their fundamental limitations and learning mechanisms remain crucial topics of investigation. In this paper, we study the ability of Transformers to learn pseudo-random number sequences from linear congruential generators (LCGs), defined by the recurrence relation $x_{t+1} = a x_t + c \;\mathrm{mod}\; m$. Our analysis reveals that with sufficient architectural capacity and training data variety, Transformers can perform in-context prediction of LCG sequences with unseen moduli ($m$) and parameters ($a,c$). Through analysis of embedding layers and attention patterns, we uncover how Transformers develop algorithmic structures to learn these sequences in two scenarios of increasing complexity. First, we analyze how Transformers learn LCG sequences with unseen ($a, c$) but fixed modulus, and we demonstrate successful learning up to $m = 2^{32}$. Our analysis reveals that models learn to factorize the modulus and utilize digit-wise number representations to make sequential predictions. In the second, more challenging scenario of unseen moduli, we show that Transformers can generalize to unseen moduli up to $m_{\text{test}} = 2^{16}$. In this case, the model employs a two-step strategy: first estimating the unknown modulus from the context, then utilizing prime factorizations to generate predictions. For this task, we observe a sharp transition in the accuracy at a critical depth $=3$. We also find that the number of in-context sequence elements needed to reach high accuracy scales sublinearly with the modulus.