2025-02-17

Title: Jailbreaking to Jailbreak

• Authors: Jeremy Kritz, Vaughn Robinson, Robert Vacareanu, Bijan Varjavand, Michael Choi, Bobby Gogov, Scale Red Team, Summer Yue, Willow E. Primack, Zifan Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09638
• Pdf URL: https://arxiv.org/pdf/2502.09638
• Copy Paste: [[2502.09638]] Jailbreaking to Jailbreak(https://arxiv.org/abs/2502.09638)
• Keywords: defense, attack, large language model
• Abstract: Refusal training on Large Language Models (LLMs) prevents harmful outputs, yet this defense remains vulnerable to both automated and human-crafted jailbreaks. We present a novel LLM-as-red-teamer approach in which a human jailbreaks a refusal-trained LLM to make it willing to jailbreak itself or other LLMs. We refer to the jailbroken LLMs as $J_2$ attackers, which can systematically evaluate target models using various red teaming strategies and improve its performance via in-context learning from the previous failures. Our experiments demonstrate that Sonnet 3.5 and Gemini 1.5 pro outperform other LLMs as $J_2$, achieving 93.0% and 91.0% attack success rates (ASRs) respectively against GPT-4o (and similar results across other capable LLMs) on Harmbench. Our work not only introduces a scalable approach to strategic red teaming, drawing inspiration from human red teamers, but also highlights jailbreaking-to-jailbreak as an overlooked failure mode of the safeguard. Specifically, an LLM can bypass its own safeguards by employing a jailbroken version of itself that is willing to assist in further jailbreaking. To prevent any direct misuse with $J_2$, while advancing research in AI safety, we publicly share our methodology while keeping specific prompting details private.

Title: Online Social Support Detection in Spanish Social Media Texts

• Authors: Moein Shahiki Tash, Luis Ramos, Zahra Ahani, Raul Monroy, Olga kolesnikova, Hiram Calvo, Grigori Sidorov
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09640
• Pdf URL: https://arxiv.org/pdf/2502.09640
• Copy Paste: [[2502.09640]] Online Social Support Detection in Spanish Social Media Texts(https://arxiv.org/abs/2502.09640)
• Keywords: transformer
• Abstract: The advent of social media has transformed communication, enabling individuals to share their experiences, seek support, and participate in diverse discussions. While extensive research has focused on identifying harmful content like hate speech, the recognition and promotion of positive and supportive interactions remain largely unexplored. This study proposes an innovative approach to detecting online social support in Spanish-language social media texts. We introduce the first annotated dataset specifically created for this task, comprising 3,189 YouTube comments classified as supportive or non-supportive. To address data imbalance, we employed GPT-4o to generate paraphrased comments and create a balanced dataset. We then evaluated social support classification using traditional machine learning models, deep learning architectures, and transformer-based models, including GPT-4o, but only on the unbalanced dataset. Subsequently, we utilized a transformer model to compare the performance between the balanced and unbalanced datasets. Our findings indicate that the balanced dataset yielded improved results for Task 2 (Individual and Group) and Task 3 (Nation, Other, LGBTQ, Black Community, Women, Religion), whereas GPT-4o performed best for Task 1 (Social Support and Non-Support). This study highlights the significance of fostering a supportive online environment and lays the groundwork for future research in automated social support detection.

Title: From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models

• Authors: Mayank Vatsa, Aparna Bharati, Surbhi Mittal, Richa Singh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09645
• Pdf URL: https://arxiv.org/pdf/2502.09645
• Copy Paste: [[2502.09645]] From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models(https://arxiv.org/abs/2502.09645)
• Keywords: robust
• Abstract: Negation, a linguistic construct conveying absence, denial, or contradiction, poses significant challenges for multilingual multimodal foundation models. These models excel in tasks like machine translation, text-guided generation, image captioning, audio interactions, and video processing but often struggle to accurately interpret negation across diverse languages and cultural contexts. In this perspective paper, we propose a comprehensive taxonomy of negation constructs, illustrating how structural, semantic, and cultural factors influence multimodal foundation models. We present open research questions and highlight key challenges, emphasizing the importance of addressing these issues to achieve robust negation handling. Finally, we advocate for specialized benchmarks, language-specific tokenization, fine-grained attention mechanisms, and advanced multimodal architectures. These strategies can foster more adaptable and semantically precise multimodal foundation models, better equipped to navigate and accurately interpret the complexities of negation in multilingual, multimodal environments.

Title: UKTA: Unified Korean Text Analyzer

• Authors: Seokho Ahn, Junhyung Park, Ganghee Go, Chulhui Kim, Jiho Jung, Myung Sun Shin, Do-Guk Kim, Young-Duk Seo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09648
• Pdf URL: https://arxiv.org/pdf/2502.09648
• Copy Paste: [[2502.09648]] UKTA: Unified Korean Text Analyzer(https://arxiv.org/abs/2502.09648)
• Keywords: explainability
• Abstract: Evaluating writing quality is complex and time-consuming often delaying feedback to learners. While automated writing evaluation tools are effective for English, Korean automated writing evaluation tools face challenges due to their inability to address multi-view analysis, error propagation, and evaluation explainability. To overcome these challenges, we introduce UKTA (Unified Korean Text Analyzer), a comprehensive Korea text analysis and writing evaluation system. UKTA provides accurate low-level morpheme analysis, key lexical features for mid-level explainability, and transparent high-level rubric-based writing scores. Our approach enhances accuracy and quadratic weighted kappa over existing baseline, positioning UKTA as a leading multi-perspective tool for Korean text analysis and writing evaluation.

Title: Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples

• Authors: Chengqian Gao, Haonan Li, Liu Liu, Zeke Xie, Peilin Zhao, Zhiqiang Xu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09650
• Pdf URL: https://arxiv.org/pdf/2502.09650
• Copy Paste: [[2502.09650]] Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples(https://arxiv.org/abs/2502.09650)
• Keywords: large language model
• Abstract: The alignment of large language models (LLMs) often assumes that using more clean data yields better outcomes, overlooking the match between model capacity and example difficulty. Challenging this, we propose a new principle: Preference data vary in difficulty, and overly difficult examples hinder alignment, by exceeding the model's capacity. Through systematic experimentation, we validate this principle with three key findings: (1) preference examples vary in difficulty, as evidenced by consistent learning orders across alignment runs; (2) overly difficult examples significantly degrade performance across four LLMs and two datasets; and (3) the capacity of a model dictates its threshold for handling difficult examples, underscoring a critical relationship between data selection and model capacity. Building on this principle, we introduce Selective DPO, which filters out overly difficult examples. This simple adjustment improves alignment performance by 9-16% in win rates on the AlpacaEval 2 benchmark compared to the DPO baseline, suppressing a series of DPO variants with different algorithmic adjustments. Together, these results illuminate the importance of aligning data difficulty with model capacity, offering a transformative perspective for improving alignment strategies in LLMs. Code is available at this https URL.

Title: AI-VERDE: A Gateway for Egalitarian Access to Large Language Model-Based Resources For Educational Institutions

• Authors: Paul Mithun, Enrique Noriega-Atala, Nirav Merchant, Edwin Skidmore
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.09651
• Pdf URL: https://arxiv.org/pdf/2502.09651
• Copy Paste: [[2502.09651]] AI-VERDE: A Gateway for Egalitarian Access to Large Language Model-Based Resources For Educational Institutions(https://arxiv.org/abs/2502.09651)
• Keywords: privacy, robust, large language model
• Abstract: We present AI-VERDE, a unified LLM-as-a-platform service designed to facilitate seamless integration of commercial, cloud-hosted, and on-premise open LLMs in academic settings. AI-VERDE streamlines access management for instructional and research groups by providing features such as robust access control, privacy-preserving mechanisms, native Retrieval-Augmented Generation (RAG) support, budget management for third-party LLM services, and both a conversational web interface and API access. In a pilot deployment at a large public university, AI-VERDE demonstrated significant engagement across diverse educational and research groups, enabling activities that would typically require substantial budgets for commercial LLM services with limited user and team management capabilities. To the best of our knowledge, AI-Verde is the first platform to address both academic and research needs for LLMs within an higher education institutional framework.

Title: GraphCompNet: A Position-Aware Model for Predicting and Compensating Shape Deviations in 3D Printing

• Authors: Lei (Rachel)Chen, Juheon Lee, Juan Carlos Catana, Tsegai Yhdego, Nathan Moroney, Mohammad Amin Nabian, Hui Wang, Jun Zeng
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09652
• Pdf URL: https://arxiv.org/pdf/2502.09652
• Copy Paste: [[2502.09652]] GraphCompNet: A Position-Aware Model for Predicting and Compensating Shape Deviations in 3D Printing(https://arxiv.org/abs/2502.09652)
• Keywords: generative
• Abstract: This paper introduces a data-driven algorithm for modeling and compensating shape deviations in additive manufacturing (AM), addressing challenges in geometric accuracy and batch production. While traditional methods, such as analytical models and metrology, laid the groundwork for geometric precision, they are often impractical for large-scale production. Recent advancements in machine learning (ML) have improved compensation precision, but issues remain in generalizing across complex geometries and adapting to position-dependent variations. We present a novel approach for powder bed fusion (PBF) processes, using GraphCompNet, which is a computational framework combining graph-based neural networks with a generative adversarial network (GAN)-inspired training process. By leveraging point cloud data and dynamic graph convolutional neural networks (DGCNNs), GraphCompNet models complex shapes and incorporates position-specific thermal and mechanical factors. A two-stage adversarial training procedure iteratively refines compensated designs via a compensator-predictor architecture, offering real-time feedback and optimization. Experimental validation across diverse shapes and positions shows the framework significantly improves compensation accuracy (35 to 65 percent) across the entire print space, adapting to position-dependent variations. This work advances the development of Digital Twin technology for AM, enabling scalable, real-time monitoring and compensation, and addressing critical gaps in AM process control. The proposed method supports high-precision, automated industrial-scale design and manufacturing systems.

Title: Bidirectional Diffusion Bridge Models

• Authors: Duc Kieu, Kien Do, Toan Nguyen, Dang Nguyen, Thin Nguyen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09655
• Pdf URL: https://arxiv.org/pdf/2502.09655
• Copy Paste: [[2502.09655]] Bidirectional Diffusion Bridge Models(https://arxiv.org/abs/2502.09655)
• Keywords: diffusion
• Abstract: Diffusion bridges have shown potential in paired image-to-image (I2I) translation tasks. However, existing methods are limited by their unidirectional nature, requiring separate models for forward and reverse translations. This not only doubles the computational cost but also restricts their practicality. In this work, we introduce the Bidirectional Diffusion Bridge Model (BDBM), a scalable approach that facilitates bidirectional translation between two coupled distributions using a single network. BDBM leverages the Chapman-Kolmogorov Equation for bridges, enabling it to model data distribution shifts across timesteps in both forward and backward directions by exploiting the interchangeability of the initial and target timesteps within this framework. Notably, when the marginal distribution given endpoints is Gaussian, BDBM's transition kernels in both directions possess analytical forms, allowing for efficient learning with a single network. We demonstrate the connection between BDBM and existing bridge methods, such as Doob's h-transform and variational approaches, and highlight its advantages. Extensive experiments on high-resolution I2I translation tasks demonstrate that BDBM not only enables bidirectional translation with minimal additional cost but also outperforms state-of-the-art bridge models. Our source code is available at [this https URL||this https URL].

Title: Integrating Spatiotemporal Vision Transformer into Digital Twins for High-Resolution Heat Stress Forecasting in Campus Environments

• Authors: Wenjing Gong, Xinyue Ye, Keshu Wu, Suphanut Jamonnak, Wenyu Zhang, Yifan Yang, Xiao Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09657
• Pdf URL: https://arxiv.org/pdf/2502.09657
• Copy Paste: [[2502.09657]] Integrating Spatiotemporal Vision Transformer into Digital Twins for High-Resolution Heat Stress Forecasting in Campus Environments(https://arxiv.org/abs/2502.09657)
• Keywords: transformer
• Abstract: Extreme heat events exacerbated by climate change pose significant challenges to urban resilience and planning. This study introduces a climate-responsive digital twin framework integrating the Spatiotemporal Vision Transformer (ST-ViT) model to enhance heat stress forecasting and decision-making. Using a Texas campus as a testbed, we synthesized high-resolution physical model simulations with spatial and meteorological data to develop fine-scale human thermal predictions. The ST-ViT-powered digital twin enables efficient, data-driven insights for planners, policymakers, and campus stakeholders, supporting targeted heat mitigation strategies and advancing climate-adaptive urban design.

Title: Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models

• Authors: Hasin Rehana, Jie Zheng, Leo Yeh, Benu Bansal, Nur Bengisu Çam, Christianah Jemiyo, Brett McGregor, Arzucan Özgür, Yongqun He, Junguk Hur
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.09659
• Pdf URL: https://arxiv.org/pdf/2502.09659
• Copy Paste: [[2502.09659]] Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models(https://arxiv.org/abs/2502.09659)
• Keywords: transformer, generative, large language model
• Abstract: Motivation: An adjuvant is a chemical incorporated into vaccines that enhances their efficacy by improving the immune response. Identifying adjuvant names from cancer vaccine studies is essential for furthering research and enhancing immunotherapies. However, the manual curation from the constantly expanding biomedical literature poses significant challenges. This study explores the automated recognition of vaccine adjuvant names using Large Language Models (LLMs), specifically Generative Pretrained Transformers (GPT) and Large Language Model Meta AI (Llama). Methods: We utilized two datasets: 97 clinical trial records from AdjuvareDB and 290 abstracts annotated with the Vaccine Adjuvant Compendium (VAC). GPT-4o and Llama 3.2 were employed in zero-shot and few-shot learning paradigms with up to four examples per prompt. Prompts explicitly targeted adjuvant names, testing the impact of contextual information such as substances or interventions. Outputs underwent automated and manual validation for accuracy and consistency. Results: GPT-4o attained 100% Precision across all situations while exhibiting notable improve in Recall and F1-scores, particularly with incorporating interventions. On the VAC dataset, GPT-4o achieved a maximum F1-score of 77.32% with interventions, surpassing Llama-3.2-3B by approximately 2%. On the AdjuvareDB dataset, GPT-4o reached an F1-score of 81.67% for three-shot prompting with interventions, surpassing Llama-3.2-3 B's maximum F1-score of 65.62%. Conclusion: Our findings demonstrate that LLMs excel at identifying adjuvant names, including rare variations of naming representation. This study emphasizes the capability of LLMs to enhance cancer vaccine development by efficiently extracting insights. Future work aims to broaden the framework to encompass various biomedical literature and enhance model generalizability across various vaccines and adjuvants.

Title: Towards Fine-grained Interactive Segmentation in Images and Videos

• Authors: Yuan Yao, Qiushi Yang, Miaomiao Cui, Liefeng Bo
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.09660
• Pdf URL: https://arxiv.org/pdf/2502.09660
• Copy Paste: [[2502.09660]] Towards Fine-grained Interactive Segmentation in Images and Videos(https://arxiv.org/abs/2502.09660)
• Keywords: robust, segmentation
• Abstract: The recent Segment Anything Models (SAMs) have emerged as foundational visual models for general interactive segmentation. Despite demonstrating robust generalization abilities, they still suffer performance degradations in scenarios demanding accurate masks. Existing methods for high-precision interactive segmentation face a trade-off between the ability to perceive intricate local details and maintaining stable prompting capability, which hinders the applicability and effectiveness of foundational segmentation models. To this end, we present an SAM2Refiner framework built upon the SAM2 backbone. This architecture allows SAM2 to generate fine-grained segmentation masks for both images and videos while preserving its inherent strengths. Specifically, we design a localization augment module, which incorporates local contextual cues to enhance global features via a cross-attention mechanism, thereby exploiting potential detailed patterns and maintaining semantic information. Moreover, to strengthen the prompting ability toward the enhanced object embedding, we introduce a prompt retargeting module to renew the embedding with spatially aligned prompt features. In addition, to obtain accurate high resolution segmentation masks, a mask refinement module is devised by employing a multi-scale cascaded structure to fuse mask features with hierarchical representations from the encoder. Extensive experiments demonstrate the effectiveness of our approach, revealing that the proposed method can produce highly precise masks for both images and videos, surpassing state-of-the-art methods.

Title: DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations

• Authors: Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio
• Subjects: cs.CV, cs.AI, cs.LG, q-bio.CB
• Abstract URL: https://arxiv.org/abs/2502.09663
• Pdf URL: https://arxiv.org/pdf/2502.09663
• Copy Paste: [[2502.09663]] DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations(https://arxiv.org/abs/2502.09663)
• Keywords: interpretability, explainability, diffusion
• Abstract: In recent years, deep learning models have been extensively applied to biological data across various modalities. Discriminative deep learning models have excelled at classifying images into categories (e.g., healthy versus diseased, treated versus untreated). However, these models are often perceived as black boxes due to their complexity and lack of interpretability, limiting their application in real-world biological contexts. In biological research, explainability is essential: understanding classifier decisions and identifying subtle differences between conditions are critical for elucidating the effects of treatments, disease progression, and biological processes. To address this challenge, we propose DiffEx, a method for generating visually interpretable attributes to explain classifiers and identify microscopic cellular variations between different conditions. We demonstrate the effectiveness of DiffEx in explaining classifiers trained on natural and biological images. Furthermore, we use DiffEx to uncover phenotypic differences within microscopy datasets. By offering insights into cellular variations through classifier explanations, DiffEx has the potential to advance the understanding of diseases and aid drug discovery by identifying novel biomarkers.

Title: Image Super-Resolution with Guarantees via Conformal Generative Models

• Authors: Eduardo Adame, Daniel Csillag, Guilherme Tegoni Goedert
• Subjects: cs.CV, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09664
• Pdf URL: https://arxiv.org/pdf/2502.09664
• Copy Paste: [[2502.09664]] Image Super-Resolution with Guarantees via Conformal Generative Models(https://arxiv.org/abs/2502.09664)
• Keywords: robust, generative
• Abstract: The increasing use of generative ML foundation models for image super-resolution calls for robust and interpretable uncertainty quantification methods. We address this need by presenting a novel approach based on conformal prediction techniques to create a "confidence mask" capable of reliably and intuitively communicating where the generated image can be trusted. Our method is adaptable to any black-box generative model, including those locked behind an opaque API, requires only easily attainable data for calibration, and is highly customizable via the choice of a local image similarity metric. We prove strong theoretical guarantees for our method that span fidelity error control (according to our local image similarity metric), reconstruction quality, and robustness in the face of data leakage. Finally, we empirically evaluate these results and establish our method's solid performance.

Title: Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models

• Authors: Anis Bourou, Biel Castaño Segade, Thomas Boye, Valérie Mezger, Auguste Genovesio
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09665
• Pdf URL: https://arxiv.org/pdf/2502.09665
• Copy Paste: [[2502.09665]] Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models(https://arxiv.org/abs/2502.09665)
• Keywords: diffusion, generative
• Abstract: Identifying subtle phenotypic variations in cellular images is critical for advancing biological research and accelerating drug discovery. These variations are often masked by the inherent cellular heterogeneity, making it challenging to distinguish differences between experimental conditions. Recent advancements in deep generative models have demonstrated significant potential for revealing these nuanced phenotypes through image translation, opening new frontiers in cellular and molecular biology as well as the identification of novel biomarkers. Among these generative models, diffusion models stand out for their ability to produce high-quality, realistic images. However, training diffusion models typically requires large datasets and substantial computational resources, both of which can be limited in biological research. In this work, we propose a novel approach that leverages pre-trained latent diffusion models to uncover subtle phenotypic changes. We validate our approach qualitatively and quantitatively on several small datasets of microscopy images. Our findings reveal that our approach enables effective detection of phenotypic variations, capturing both visually apparent and imperceptible differences. Ultimately, our results highlight the promising potential of this approach for phenotype detection, especially in contexts constrained by limited data and computational capacity.

Title: k-LLMmeans: Summaries as Centroids for Interpretable and Scalable LLM-Based Text Clustering

• Authors: Jairo Diaz-Rodriguez
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09667
• Pdf URL: https://arxiv.org/pdf/2502.09667
• Copy Paste: [[2502.09667]] k-LLMmeans: Summaries as Centroids for Interpretable and Scalable LLM-Based Text Clustering(https://arxiv.org/abs/2502.09667)
• Keywords: interpretability
• Abstract: We introduce k-LLMmeans, a novel modification of the k-means clustering algorithm that utilizes LLMs to generate textual summaries as cluster centroids, thereby capturing contextual and semantic nuances often lost when relying on purely numerical means of document embeddings. This modification preserves the properties of k-means while offering greater interpretability: the cluster centroid is represented by an LLM-generated summary, whose embedding guides cluster assignments. We also propose a mini-batch variant, enabling efficient online clustering for streaming text data and providing real-time interpretability of evolving cluster centroids. Through extensive simulations, we show that our methods outperform vanilla k-means on multiple metrics while incurring only modest LLM usage that does not scale with dataset size. Finally, We present a case study showcasing the interpretability of evolving cluster centroids in sequential text streams. As part of our evaluation, we compile a new dataset from StackExchange, offering a benchmark for text-stream clustering.

Title: Meta-INR: Efficient Encoding of Volumetric Data via Meta-Learning Implicit Neural Representation

• Authors: Maizhe Yang, Kaiyuan Tang, Chaoli Wang
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.09669
• Pdf URL: https://arxiv.org/pdf/2502.09669
• Copy Paste: [[2502.09669]] Meta-INR: Efficient Encoding of Volumetric Data via Meta-Learning Implicit Neural Representation(https://arxiv.org/abs/2502.09669)
• Keywords: interpretability
• Abstract: Implicit neural representation (INR) has emerged as a promising solution for encoding volumetric data, offering continuous representations and seamless compatibility with the volume rendering pipeline. However, optimizing an INR network from randomly initialized parameters for each new volume is computationally inefficient, especially for large-scale time-varying or ensemble volumetric datasets where volumes share similar structural patterns but require independent training. To close this gap, we propose Meta-INR, a pretraining strategy adapted from meta-learning algorithms to learn initial INR parameters from partial observation of a volumetric dataset. Compared to training an INR from scratch, the learned initial parameters provide a strong prior that enhances INR generalizability, allowing significantly faster convergence with just a few gradient updates when adapting to a new volume and better interpretability when analyzing the parameters of the adapted INRs. We demonstrate that Meta-INR can effectively extract high-quality generalizable features that help encode unseen similar volume data across diverse datasets. Furthermore, we highlight its utility in tasks such as simulation parameter analysis and representative timestep selection. The code is available at this https URL.

Title: Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning

• Authors: Ang Li, Yichuan Mo, Mingjie Li, Yifei Wang, Yisen Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09673
• Pdf URL: https://arxiv.org/pdf/2502.09673
• Copy Paste: [[2502.09673]] Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning(https://arxiv.org/abs/2502.09673)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable success across various NLP benchmarks. However, excelling in complex tasks that require nuanced reasoning and precise decision-making demands more than raw language proficiency--LLMs must reason, i.e., think logically, draw from past experiences, and synthesize information to reach conclusions and take action. To enhance reasoning abilities, approaches such as prompting and fine-tuning have been widely explored. While these methods have led to clear improvements in reasoning, their impact on LLM safety remains less understood. In this work, we investigate the interplay between reasoning and safety in LLMs. We highlight the latent safety risks that arise as reasoning capabilities improve, shedding light on previously overlooked vulnerabilities. At the same time, we explore how reasoning itself can be leveraged to enhance safety, uncovering potential mitigation strategies. By examining both the risks and opportunities in reasoning-driven LLM safety, our study provides valuable insights for developing models that are not only more capable but also more trustworthy in real-world deployments.

Title: The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Safety Analysis

• Authors: Wenbo Pan, Zhichao Liu, Qiguang Chen, Xiangyang Zhou, Haining Yu, Xiaohua Jia
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09674
• Pdf URL: https://arxiv.org/pdf/2502.09674
• Copy Paste: [[2502.09674]] The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Safety Analysis(https://arxiv.org/abs/2502.09674)
• Keywords: large language model
• Abstract: Large Language Models' safety-aligned behaviors, such as refusing harmful queries, can be represented by linear directions in activation space. Previous research modeled safety behavior with a single direction, limiting mechanistic understanding to an isolated safety feature. In this work, we discover that safety-aligned behavior is jointly controlled by multi-dimensional directions. Namely, we study the vector space of representation shifts during safety fine-tuning on Llama 3 8B for refusing jailbreaks. By studying orthogonal directions in the space, we first find that a dominant direction governs the model's refusal behavior, while multiple smaller directions represent distinct and interpretable features like hypothetical narrative and role-playing. We then measure how different directions promote or suppress the dominant direction, showing the important role of secondary directions in shaping the model's refusal representation. Finally, we demonstrate that removing certain trigger tokens in harmful queries can mitigate these directions to bypass the learned safety capability, providing new insights on understanding safety alignment vulnerability from a multi-dimensional perspective. Code and artifacts are available at this https URL.

Title: Channel Dependence, Limited Lookback Windows, and the Simplicity of Datasets: How Biased is Time Series Forecasting?

• Authors: Ibram Abdelmalak, Kiran Madhusudhanan, Jungmin Choi, Maximilian Stubbemann, Lars Schmidt-Thieme
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09683
• Pdf URL: https://arxiv.org/pdf/2502.09683
• Copy Paste: [[2502.09683]] Channel Dependence, Limited Lookback Windows, and the Simplicity of Datasets: How Biased is Time Series Forecasting?(https://arxiv.org/abs/2502.09683)
• Keywords: transformer
• Abstract: Time-series forecasting research has converged to a small set of datasets and a standardized collection of evaluation scenarios. Such a standardization is to a specific extent needed for comparable research. However, the underlying assumption is, that the considered setting is a representative for the problem as a whole. In this paper, we challenge this assumption and show that the current scenario gives a strongly biased perspective on the state of time-series forecasting research. To be more detailed, we show that the current evaluation scenario is heavily biased by the simplicity of the current datasets. We furthermore emphasize, that when the lookback-window is properly tuned, current models usually do not need any information flow across channels. However, when using more complex benchmark data, the situation changes: Here, modeling channel-interactions in a sophisticated manner indeed enhances performances. Furthermore, in this complex evaluation scenario, Crossformer, a method regularly neglected as an important baseline, is the SOTA method for time series forecasting. Based on this, we present the Fast Channel-dependent Transformer (FaCT), a simplified version of Crossformer which closes the runtime gap between Crossformer and TimeMixer, leading to an efficient model for complex forecasting datasets.

Title: Leveraging Machine Learning and Deep Learning Techniques for Improved Pathological Staging of Prostate Cancer

• Authors: Raziehsadat Ghalamkarian, Marziehsadat Ghalamkarian, MortezaAli Ahmadi, Sayed Mohammad Ahmadi, Abolfazl Diyanat
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09686
• Pdf URL: https://arxiv.org/pdf/2502.09686
• Copy Paste: [[2502.09686]] Leveraging Machine Learning and Deep Learning Techniques for Improved Pathological Staging of Prostate Cancer(https://arxiv.org/abs/2502.09686)
• Keywords: extraction
• Abstract: Prostate cancer (Pca) continues to be a leading cause of cancer-related mortality in men, and the limitations in precision of traditional diagnostic methods such as the Digital Rectal Exam (DRE), Prostate-Specific Antigen (PSA) testing, and biopsies underscore the critical importance of accurate staging detection in enhancing treatment outcomes and improving patient prognosis. This study leverages machine learning and deep learning approaches, along with feature selection and extraction methods, to enhance PCa pathological staging predictions using RNA sequencing data from The Cancer Genome Atlas (TCGA). Gene expression profiles from 486 tumors were analyzed using advanced algorithms, including Random Forest (RF), Logistic Regression (LR), Extreme Gradient Boosting (XGB), and Support Vector Machine (SVM). The performance of the study is measured with respect to the F1-score, as well as precision and recall, all of which are calculated as weighted averages. The results reveal that the highest test F1-score, approximately 83%, was achieved by the Random Forest algorithm, followed by Logistic Regression at 80%, while both Extreme Gradient Boosting (XGB) and Support Vector Machine (SVM) scored around 79%. Furthermore, deep learning models with data augmentation achieved an accuracy of 71. 23%, while PCA-based dimensionality reduction reached an accuracy of 69.86%. This research highlights the potential of AI-driven approaches in clinical oncology, paving the way for more reliable diagnostic tools that can ultimately improve patient outcomes.

Title: Mind What You Ask For: Emotional and Rational Faces of Persuasion by Large Language Models

• Authors: Wiktoria Mieleszczenko-Kowszewicz, Beata Bajcar, Jolanta Babiak, Berenika Dyczek, Jakub Świstak, Przemysław Biecek
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.09687
• Pdf URL: https://arxiv.org/pdf/2502.09687
• Copy Paste: [[2502.09687]] Mind What You Ask For: Emotional and Rational Faces of Persuasion by Large Language Models(https://arxiv.org/abs/2502.09687)
• Keywords: large language model
• Abstract: Be careful what you ask for, you just might get it. This saying fits with the way large language models (LLMs) are trained, which, instead of being rewarded for correctness, are increasingly rewarded for pleasing the recipient. So, they are increasingly effective at persuading us that their answers are valuable. But what tricks do they use in this persuasion? In this study, we examine what are the psycholinguistic features of the responses used by twelve different language models. By grouping response content according to rational or emotional prompts and exploring social influence principles employed by LLMs, we ask whether and how we can mitigate the risks of LLM-driven mass misinformation. We position this study within the broader discourse on human-centred AI, emphasizing the need for interdisciplinary approaches to mitigate cognitive and societal risks posed by persuasive AI responses.

Title: Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling

• Authors: Benjamin D. Killeen, Bohua Wan, Aditya V. Kulkarni, Nathan Drenkow, Michael Oberst, Paul H. Yi, Mathias Unberath
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09688
• Pdf URL: https://arxiv.org/pdf/2502.09688
• Copy Paste: [[2502.09688]] Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling(https://arxiv.org/abs/2502.09688)
• Keywords: robust, generative
• Abstract: Artificial intelligence (AI) is poised to transform healthcare by enabling personalized and efficient care through data-driven insights. Although radiology is at the forefront of AI adoption, in practice, the potential of AI models is often overshadowed by severe failures to generalize: AI models can have performance degradation of up to 20% when transitioning from controlled test environments to clinical use by radiologists. This mismatch raises concerns that radiologists will be misled by incorrect AI predictions in practice and/or grow to distrust AI, rendering these promising technologies practically ineffectual. Exhaustive clinical trials of AI models on abundant and diverse data is thus critical to anticipate AI model degradation when encountering varied data samples. Achieving these goals, however, is challenging due to the high costs of collecting diverse data samples and corresponding annotations. To overcome these limitations, we introduce a novel conditional generative AI model designed for virtual clinical trials (VCTs) of radiology AI, capable of realistically synthesizing full-body CT images of patients with specified attributes. By learning the joint distribution of images and anatomical structures, our model enables precise replication of real-world patient populations with unprecedented detail at this scale. We demonstrate meaningful evaluation of radiology AI models through VCTs powered by our synthetic CT study populations, revealing model degradation and facilitating algorithmic auditing for bias-inducing data attributes. Our generative AI approach to VCTs is a promising avenue towards a scalable solution to assess model robustness, mitigate biases, and safeguard patient care by enabling simpler testing and evaluation of AI models in any desired range of diverse patient populations.

Title: Large Language Models and Provenance Metadata for Determining the Relevance of Images and Videos in News Stories

• Authors: Tomas Peterka, Matyas Bohacek
• Subjects: cs.CL, cs.CV, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.09689
• Pdf URL: https://arxiv.org/pdf/2502.09689
• Copy Paste: [[2502.09689]] Large Language Models and Provenance Metadata for Determining the Relevance of Images and Videos in News Stories(https://arxiv.org/abs/2502.09689)
• Keywords: large language model
• Abstract: The most effective misinformation campaigns are multimodal, often combining text with images and videos taken out of context -- or fabricating them entirely -- to support a given narrative. Contemporary methods for detecting misinformation, whether in deepfakes or text articles, often miss the interplay between multiple modalities. Built around a large language model, the system proposed in this paper addresses these challenges. It analyzes both the article's text and the provenance metadata of included images and videos to determine whether they are relevant. We open-source the system prototype and interactive web interface.

Title: Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes

• Authors: Taylan G. Topcu, Mohammed Husain, Max Ofsa, Paul Wach
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09690
• Pdf URL: https://arxiv.org/pdf/2502.09690
• Copy Paste: [[2502.09690]] Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes(https://arxiv.org/abs/2502.09690)
• Keywords: generative, large language model
• Abstract: Multi-purpose Large Language Models (LLMs), a subset of generative Artificial Intelligence (AI), have recently made significant progress. While expectations for LLMs to assist systems engineering (SE) tasks are paramount; the interdisciplinary and complex nature of systems, along with the need to synthesize deep-domain knowledge and operational context, raise questions regarding the efficacy of LLMs to generate SE artifacts, particularly given that they are trained using data that is broadly available on the internet. To that end, we present results from an empirical exploration, where a human expert-generated SE artifact was taken as a benchmark, parsed, and fed into various LLMs through prompt engineering to generate segments of typical SE artifacts. This procedure was applied without any fine-tuning or calibration to document baseline LLM performance. We then adopted a two-fold mixed-methods approach to compare AI generated artifacts against the benchmark. First, we quantitatively compare the artifacts using natural language processing algorithms and find that when prompted carefully, the state-of-the-art algorithms cannot differentiate AI-generated artifacts from the human-expert benchmark. Second, we conduct a qualitative deep dive to investigate how they differ in terms of quality. We document that while the two-material appear very similar, AI generated artifacts exhibit serious failure modes that could be difficult to detect. We characterize these as: premature requirements definition, unsubstantiated numerical estimates, and propensity to overspecify. We contend that this study tells a cautionary tale about why the SE community must be more cautious adopting AI suggested feedback, at least when generated by multi-purpose LLMs.

Title: NeuralCFD: Deep Learning on High-Fidelity Automotive Aerodynamics Simulations

• Authors: Maurits Bleeker, Matthias Dorfer, Tobias Kronlachner, Reinhard Sonnleitner, Benedikt Alkin, Johannes Brandstetter
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09692
• Pdf URL: https://arxiv.org/pdf/2502.09692
• Copy Paste: [[2502.09692]] NeuralCFD: Deep Learning on High-Fidelity Automotive Aerodynamics Simulations(https://arxiv.org/abs/2502.09692)
• Keywords: transformer
• Abstract: Recent advancements in neural operator learning are paving the way for transformative innovations in fields such as automotive aerodynamics. However, key challenges must be overcome before neural network-based simulation surrogates can be implemented at an industry scale. First, surrogates must become scalable to large surface and volume meshes, especially when using raw geometry inputs only, i.e., without relying on the simulation mesh. Second, surrogates must be trainable with a limited number of high-fidelity numerical simulation samples while still reaching the required performance levels. To this end, we introduce Geometry-preserving Universal Physics Transformer (GP-UPT), which separates geometry encoding and physics predictions, ensuring flexibility with respect to geometry representations and surface sampling strategies. GP-UPT enables independent scaling of the respective parts of the model according to practical requirements, offering scalable solutions to open challenges. GP-UPT circumvents the creation of high-quality simulation meshes, enables accurate 3D velocity field predictions at 20 million mesh cells, and excels in transfer learning from low-fidelity to high-fidelity simulation datasets, requiring less than half of the high-fidelity data to match the performance of models trained from scratch.

Title: NestQuant: Nested Lattice Quantization for Matrix Products and LLMs

• Authors: Semyon Savkin, Eitan Porat, Or Ordentlich, Yury Polyanskiy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09720
• Pdf URL: https://arxiv.org/pdf/2502.09720
• Copy Paste: [[2502.09720]] NestQuant: Nested Lattice Quantization for Matrix Products and LLMs(https://arxiv.org/abs/2502.09720)
• Keywords: large language model
• Abstract: Post-training quantization (PTQ) has emerged as a critical technique for efficient deployment of large language models (LLMs). This work proposes NestQuant, a novel PTQ scheme for weights and activations that is based on self-similar nested lattices. Recent work have mathematically shown such quantizers to be information-theoretically optimal for low-precision matrix multiplication. We implement a practical low-complexity version of NestQuant based on Gosset lattice, making it a drop-in quantizer for any matrix multiplication step (e.g., in self-attention, MLP etc). For example, NestQuant quantizes weights, KV-cache, and activations of Llama-3-8B to 4 bits, achieving perplexity of 6.6 on wikitext2. This represents more than 55% reduction in perplexity gap with respect to unquantized model (perplexity of 6.14) compared to state-of-the-art Meta's SpinQuant (perplexity 7.3). Comparisons on various LLM evaluation benchmarks also show a reduction in performance degradation induced by quantization.

Title: Making Them a Malicious Database: Exploiting Query Code to Jailbreak Aligned Large Language Models

• Authors: Qingsong Zou, Jingyu Xiao, Qing Li, Zhi Yan, Yuhang Wang, Li Xu, Wenxuan Wang, Kuofeng Gao, Ruoyu Li, Yong Jiang
• Subjects: cs.CR, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09723
• Pdf URL: https://arxiv.org/pdf/2502.09723
• Copy Paste: [[2502.09723]] Making Them a Malicious Database: Exploiting Query Code to Jailbreak Aligned Large Language Models(https://arxiv.org/abs/2502.09723)
• Keywords: security, defense, attack, large language model
• Abstract: Recent advances in large language models (LLMs) have demonstrated remarkable potential in the field of natural language processing. Unfortunately, LLMs face significant security and ethical risks. Although techniques such as safety alignment are developed for defense, prior researches reveal the possibility of bypassing such defenses through well-designed jailbreak attacks. In this paper, we propose QueryAttack, a novel framework to systematically examine the generalizability of safety alignment. By treating LLMs as knowledge databases, we translate malicious queries in natural language into code-style structured query to bypass the safety alignment mechanisms of LLMs. We conduct extensive experiments on mainstream LLMs, ant the results show that QueryAttack achieves high attack success rates (ASRs) across LLMs with different developers and capabilities. We also evaluate QueryAttack's performance against common defenses, confirming that it is difficult to mitigate with general defensive techniques. To defend against QueryAttack, we tailor a defense method which can reduce ASR by up to 64\% on GPT-4-1106. The code of QueryAttack can be found on this https URL.

Title: Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning

• Authors: Cheol Woo Kim, Jai Moondra, Shresth Verma, Madeleine Pollack, Lingkai Kong, Milind Tambe, Swati Gupta
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09724
• Pdf URL: https://arxiv.org/pdf/2502.09724
• Copy Paste: [[2502.09724]] Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning(https://arxiv.org/abs/2502.09724)
• Keywords: fair
• Abstract: In many real-world applications of reinforcement learning (RL), deployed policies have varied impacts on different stakeholders, creating challenges in reaching consensus on how to effectively aggregate their preferences. Generalized $p$-means form a widely used class of social welfare functions for this purpose, with broad applications in fair resource allocation, AI alignment, and decision-making. This class includes well-known welfare functions such as Egalitarian, Nash, and Utilitarian welfare. However, selecting the appropriate social welfare function is challenging for decision-makers, as the structure and outcomes of optimal policies can be highly sensitive to the choice of $p$. To address this challenge, we study the concept of an $\alpha$-approximate portfolio in RL, a set of policies that are approximately optimal across the family of generalized $p$-means for all $p \in [-\infty, 1]$. We propose algorithms to compute such portfolios and provide theoretical guarantees on the trade-offs among approximation factor, portfolio size, and computational efficiency. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of our approach in summarizing the policy space induced by varying $p$ values, empowering decision-makers to navigate this landscape more effectively.

Title: Analysis of Robust and Secure DNS Protocols for IoT Devices

• Authors: Abdullah Aydeger, Sanzida Hoque, Engin Zeydan, Kapal Dev
• Subjects: cs.CR, cs.NI, cs.PF
• Abstract URL: https://arxiv.org/abs/2502.09726
• Pdf URL: https://arxiv.org/pdf/2502.09726
• Copy Paste: [[2502.09726]] Analysis of Robust and Secure DNS Protocols for IoT Devices(https://arxiv.org/abs/2502.09726)
• Keywords: secure, security, robust
• Abstract: The DNS (Domain Name System) protocol has been in use since the early days of the Internet. Although DNS as a de facto networking protocol had no security considerations in its early years, there have been many security enhancements, such as DNSSec (Domain Name System Security Extensions), DoT (DNS over Transport Layer Security), DoH (DNS over HTTPS) and DoQ (DNS over QUIC). With all these security improvements, it is not yet clear what resource-constrained Internet-of-Things (IoT) devices should be used for robustness. In this paper, we investigate different DNS security approaches using an edge DNS resolver implemented as a Virtual Network Function (VNF) to replicate the impact of the protocol from an IoT perspective and compare their performances under different conditions. We present our results for cache-based and non-cached responses and evaluate the corresponding security benefits. Our results and framework can greatly help consumers, manufacturers, and the research community decide and implement their DNS protocols depending on the given dynamic network conditions and enable robust Internet access via DNS for different devices.

Title: A CNN Approach to Automated Detection and Classification of Brain Tumors

• Authors: Md. Zahid Hasan, Abdullah Tamim, D.M. Asadujjaman, Md. Mahfujur Rahman, Md. Abu Ahnaf Mollick, Nosin Anjum Dristi, Abdullah-Al-Noman
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09731
• Pdf URL: https://arxiv.org/pdf/2502.09731
• Copy Paste: [[2502.09731]] A CNN Approach to Automated Detection and Classification of Brain Tumors(https://arxiv.org/abs/2502.09731)
• Keywords: diffusion
• Abstract: Brain tumors require an assessment to ensure timely diagnosis and effective patient treatment. Morphological factors such as size, location, texture, and variable appearance com- plicate tumor inspection. Medical imaging presents challenges, including noise and incomplete images. This research article presents a methodology for processing Magnetic Resonance Imag- ing (MRI) data, encompassing techniques for image classification and denoising. The effective use of MRI images allows medical professionals to detect brain disorders, including tumors. This research aims to categorize healthy brain tissue and brain tumors by analyzing the provided MRI data. Unlike alternative methods like Computed Tomography (CT), MRI technology offers a more detailed representation of internal anatomical components, mak- ing it a suitable option for studying data related to brain tumors. The MRI picture is first subjected to a denoising technique utilizing an Anisotropic diffusion filter. The dataset utilized for the models creation is a publicly accessible and validated Brain Tumour Classification (MRI) database, comprising 3,264 brain MRI scans. SMOTE was employed for data augmentation and dataset balancing. Convolutional Neural Networks(CNN) such as ResNet152V2, VGG, ViT, and EfficientNet were employed for the classification procedure. EfficientNet attained an accuracy of 98%, the highest recorded.

Title: FoNE: Precise Single-Token Number Embeddings via Fourier Features

• Authors: Tianyi Zhou, Deqing Fu, Mahdi Soltanolkotabi, Robin Jia, Vatsal Sharan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09741
• Pdf URL: https://arxiv.org/pdf/2502.09741
• Copy Paste: [[2502.09741]] FoNE: Precise Single-Token Number Embeddings via Fourier Features(https://arxiv.org/abs/2502.09741)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) typically represent numbers using multiple tokens, which requires the model to aggregate these tokens to interpret numerical values. This fragmentation makes both training and inference less efficient and adversely affects the model's performance on number-related tasks. Inspired by the observation that pre-trained LLMs internally learn Fourier-like features for number tokens, we propose Fourier Number Embedding (FoNE), a novel method that directly maps numbers into the embedding space with their Fourier features. FoNE encodes each number as a single token with only two embedding dimensions per digit, effectively capturing numerical values without fragmentation. This compact representation accelerates both training and inference. Compared to traditional subword and digit-wise embeddings, FoNE not only reduces computational overhead but also achieves higher accuracy across various numerical tasks including addition, subtraction and multiplication. On 6-digit decimal addition, FoNE requires 64$\times$ less data to achieve 99% accuracy than subword and digit-wise embeddings while using 3$\times$ and 6$\times$ fewer tokens per number, respectively. Furthermore, FoNE is the only method that yields 100% accuracy on over 100,000 test examples for addition, subtraction, and multiplication. The codes and visualization are available at this https URL.

Title: Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting

• Authors: Mahad Ali, Curtis Lisle, Patrick W. Moore, Tammer Barkouki, Brian J. Kirkwood, Laura J. Brattain
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09744
• Pdf URL: https://arxiv.org/pdf/2502.09744
• Copy Paste: [[2502.09744]] Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting(https://arxiv.org/abs/2502.09744)
• Keywords: privacy, protect, federate
• Abstract: Federated Learning (FL) provides a decentralized machine learning approach, where multiple devices or servers collaboratively train a model without sharing their raw data, thus enabling data privacy. This approach has gained significant interest in academia and industry due to its privacy-preserving properties, which are particularly valuable in the medical domain where data availability is often protected under strict regulations. A relatively unexplored area is the use of FL to fine-tune Foundation Models (FMs) for time series forecasting, potentially enhancing model efficacy by overcoming data limitation while maintaining privacy. In this paper, we fine-tuned time series FMs with Electrocardiogram (ECG) and Impedance Cardiography (ICG) data using different FL techniques. We then examined various scenarios and discussed the challenges FL faces under different data heterogeneity configurations. Our empirical results demonstrated that while FL can be effective for fine-tuning FMs on time series forecasting tasks, its benefits depend on the data distribution across clients. We highlighted the trade-offs in applying FL to FM fine-tuning.

Title: The Widespread Adoption of Large Language Model-Assisted Writing Across Society

• Authors: Weixin Liang, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, James Zou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09747
• Pdf URL: https://arxiv.org/pdf/2502.09747
• Copy Paste: [[2502.09747]] The Widespread Adoption of Large Language Model-Assisted Writing Across Society(https://arxiv.org/abs/2502.09747)
• Keywords: robust, generative, large language model
• Abstract: The recent advances in large language models (LLMs) attracted significant public and policymaker interest in its adoption patterns. In this paper, we systematically analyze LLM-assisted writing across four domains-consumer complaints, corporate communications, job postings, and international organization press releases-from January 2022 to September 2024. Our dataset includes 687,241 consumer complaints, 537,413 corporate press releases, 304.3 million job postings, and 15,919 United Nations (UN) press releases. Using a robust population-level statistical framework, we find that LLM usage surged following the release of ChatGPT in November 2022. By late 2024, roughly 18% of financial consumer complaint text appears to be LLM-assisted, with adoption patterns spread broadly across regions and slightly higher in urban areas. For corporate press releases, up to 24% of the text is attributable to LLMs. In job postings, LLM-assisted writing accounts for just below 10% in small firms, and is even more common among younger firms. UN press releases also reflect this trend, with nearly 14% of content being generated or modified by LLMs. Although adoption climbed rapidly post-ChatGPT, growth appears to have stabilized by 2024, reflecting either saturation in LLM adoption or increasing subtlety of more advanced models. Our study shows the emergence of a new reality in which firms, consumers and even international organizations substantially rely on generative AI for communications.

Title: Enhancing Jailbreak Attacks via Compliance-Refusal-Based Initialization

• Authors: Amit Levi, Rom Himelstein, Yaniv Nemcovsky, Avi Mendelson, Chaim Baskin
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09755
• Pdf URL: https://arxiv.org/pdf/2502.09755
• Copy Paste: [[2502.09755]] Enhancing Jailbreak Attacks via Compliance-Refusal-Based Initialization(https://arxiv.org/abs/2502.09755)
• Keywords: attack, large language model
• Abstract: Jailbreak attacks aim to exploit large language models (LLMs) and pose a significant threat to their proper conduct; they seek to bypass models' safeguards and often provoke transgressive behaviors. However, existing automatic jailbreak attacks require extensive computational resources and are prone to converge on suboptimal solutions. In this work, we propose \textbf{C}ompliance \textbf{R}efusal \textbf{I}nitialization (CRI), a novel, attack-agnostic framework that efficiently initializes the optimization in the proximity of the compliance subspace of harmful prompts. By narrowing the initial gap to the adversarial objective, CRI substantially improves adversarial success rates (ASR) and drastically reduces computational overhead -- often requiring just a single optimization step. We evaluate CRI on the widely-used AdvBench dataset over the standard jailbreak attacks of GCG and AutoDAN. Results show that CRI boosts ASR and decreases the median steps to success by up to \textbf{\(\times 60\)}. The project page, along with the reference implementation, is publicly available at \texttt{this https URL}.

Title: SoK: Come Together -- Unifying Security, Information Theory, and Cognition for a Mixed Reality Deception Attack Ontology & Analysis Framework

• Authors: Ali Teymourian, Andrew M. Webb, Taha Gharaibeh, Arushi Ghildiyal, Ibrahim Baggili
• Subjects: cs.CR, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.09763
• Pdf URL: https://arxiv.org/pdf/2502.09763
• Copy Paste: [[2502.09763]] SoK: Come Together -- Unifying Security, Information Theory, and Cognition for a Mixed Reality Deception Attack Ontology & Analysis Framework(https://arxiv.org/abs/2502.09763)
• Keywords: security, attack
• Abstract: We present a primary attack ontology and analysis framework for deception attacks in Mixed Reality (MR). This is achieved through multidisciplinary Systematization of Knowledge (SoK), integrating concepts from MR security, information theory, and cognition. While MR grows in popularity, it presents many cybersecurity challenges, particularly concerning deception attacks and their effects on humans. In this paper, we use the Borden-Kopp model of deception to develop a comprehensive ontology of MR deception attacks. Further, we derive two models to assess impact of MR deception attacks on information communication and decision-making. The first, an information-theoretic model, mathematically formalizes the effects of attacks on information communication. The second, a decision-making model, details the effects of attacks on interlaced cognitive processes. Using our ontology and models, we establish the MR Deception Analysis Framework (DAF) to assess the effects of MR deception attacks on information channels, perception, and attention. Our SoK uncovers five key findings for research and practice and identifies five research gaps to guide future work.

Title: Differential Adjusted Parity for Learning Fair Representations

• Authors: Bucher Sahyouni, Matthew Vowels, Liqun Chen, Simon Hadfield
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09765
• Pdf URL: https://arxiv.org/pdf/2502.09765
• Copy Paste: [[2502.09765]] Differential Adjusted Parity for Learning Fair Representations(https://arxiv.org/abs/2502.09765)
• Keywords: fair
• Abstract: The development of fair and unbiased machine learning models remains an ongoing objective for researchers in the field of artificial intelligence. We introduce the Differential Adjusted Parity (DAP) loss to produce unbiased informative representations. It utilises a differentiable variant of the adjusted parity metric to create a unified objective function. By combining downstream task classification accuracy and its inconsistency across sensitive feature domains, it provides a single tool to increase performance and mitigate bias. A key element in this approach is the use of soft balanced accuracies. In contrast to previous non-adversarial approaches, DAP does not suffer a degeneracy where the metric is satisfied by performing equally poorly across all sensitive domains. It outperforms several adversarial models on downstream task accuracy and fairness in our analysis. Specifically, it improves the demographic parity, equalized odds and sensitive feature accuracy by as much as 22.5\%, 44.1\% and 40.1\%, respectively, when compared to the best performing adversarial approaches on these metrics. Overall, the DAP loss and its associated metric can play a significant role in creating more fair machine learning models.

Title: Non-Markovian Discrete Diffusion with Causal Language Models

• Authors: Yangtian Zhang, Sizhuang He, Daniel Levine, Lawrence Zhao, David Zhang, Syed A Rizvi, Emanuele Zappala, Rex Ying, David van Dijk
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09767
• Pdf URL: https://arxiv.org/pdf/2502.09767
• Copy Paste: [[2502.09767]] Non-Markovian Discrete Diffusion with Causal Language Models(https://arxiv.org/abs/2502.09767)
• Keywords: diffusion, transformer, large language model
• Abstract: Discrete diffusion models have emerged as a flexible and controllable paradigm for structured sequence modeling, yet they still lag behind causal language models in expressiveness. To bridge the gap between two paradigms, we introduce CaDDi, a causal discrete diffusion model that unifies sequential and temporal modeling within a non-Markovian diffusion framework. Unlike conventional diffusion models that operate step by step with no access to prior states, CaDDi integrates the temporal trajectory, enabling more expressive and controllable generation. Our approach also treats causal language models as a special case, allowing seamless adoption of pretrained large language models (LLMs) for discrete diffusion without the need for architectural modifications. Empirically, we demonstrate that CaDDi outperforms state-of-the-art discrete diffusion models on both natural language and biological sequence tasks, narrowing the gap between diffusion-based methods and large-scale autoregressive transformers.

Title: Improving Acoustic Side-Channel Attacks on Keyboards Using Transformers and Large Language Models

• Authors: Jin Hyun Park, Seyyed Ali Ayati, Yichen Cai
• Subjects: cs.LG, cs.AI, cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.09782
• Pdf URL: https://arxiv.org/pdf/2502.09782
• Copy Paste: [[2502.09782]] Improving Acoustic Side-Channel Attacks on Keyboards Using Transformers and Large Language Models(https://arxiv.org/abs/2502.09782)
• Keywords: attack, transformer, large language model
• Abstract: The increasing prevalence of microphones in everyday devices and the growing reliance on online services have amplified the risk of acoustic side-channel attacks (ASCAs) targeting keyboards. This study explores deep learning techniques, specifically vision transformers (VTs) and large language models (LLMs), to enhance the effectiveness and applicability of such attacks. We present substantial improvements over prior research, with the CoAtNet model achieving state-of-the-art performance. Our CoAtNet shows a 5.0% improvement for keystrokes recorded via smartphone (Phone) and 5.9% for those recorded via Zoom compared to previous benchmarks. We also evaluate transformer architectures and language models, with the best VT model matching CoAtNet's performance. A key advancement is the introduction of a noise mitigation method for real-world scenarios. By using LLMs for contextual understanding, we detect and correct erroneous keystrokes in noisy environments, enhancing ASCA performance. Additionally, fine-tuned lightweight language models with Low-Rank Adaptation (LoRA) deliver comparable performance to heavyweight models with 67X more parameters. This integration of VTs and LLMs improves the practical applicability of ASCA mitigation, marking the first use of these technologies to address ASCAs and error correction in real-world scenarios.

Title: MANTIS: Detection of Zero-Day Malicious Domains Leveraging Low Reputed Hosting Infrastructure

• Authors: Fatih Deniz, Mohamed Nabeel, Ting Yu, Issa Khalil
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09788
• Pdf URL: https://arxiv.org/pdf/2502.09788
• Copy Paste: [[2502.09788]] MANTIS: Detection of Zero-Day Malicious Domains Leveraging Low Reputed Hosting Infrastructure(https://arxiv.org/abs/2502.09788)
• Keywords: attack
• Abstract: Internet miscreants increasingly utilize short-lived disposable domains to launch various attacks. Existing detection mechanisms are either too late to catch such malicious domains due to limited information and their short life spans or unable to catch them due to evasive techniques such as cloaking and captcha. In this work, we investigate the possibility of detecting malicious domains early in their life cycle using a content-agnostic approach. We observe that attackers often reuse or rotate hosting infrastructures to host multiple malicious domains due to increased utilization of automation and economies of scale. Thus, it gives defenders the opportunity to monitor such infrastructure to identify newly hosted malicious domains. However, such infrastructures are often shared hosting environments where benign domains are also hosted, which could result in a prohibitive number of false positives. Therefore, one needs innovative mechanisms to better distinguish malicious domains from benign ones even when they share hosting infrastructures. In this work, we build MANTIS, a highly accurate practical system that not only generates daily blocklists of malicious domains but also is able to predict malicious domains on-demand. We design a network graph based on the hosting infrastructure that is accurate and generalizable over time. Consistently, our models achieve a precision of 99.7%, a recall of 86.9% with a very low false positive rate (FPR) of 0.1% and on average detects 19K new malicious domains per day, which is over 5 times the new malicious domains flagged daily in VirusTotal. Further, MANTIS predicts malicious domains days to weeks before they appear in popular blocklists.

Title: Noise Controlled CT Super-Resolution with Conditional Diffusion Model

• Authors: Yuang Wang, Siyeop Yoon, Rui Hu, Baihui Yu, Duhgoon Lee, Rajiv Gupta, Li Zhang, Zhiqiang Chen, Dufan Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09793
• Pdf URL: https://arxiv.org/pdf/2502.09793
• Copy Paste: [[2502.09793]] Noise Controlled CT Super-Resolution with Conditional Diffusion Model(https://arxiv.org/abs/2502.09793)
• Keywords: diffusion
• Abstract: Improving the spatial resolution of CT images is a meaningful yet challenging task, often accompanied by the issue of noise amplification. This article introduces an innovative framework for noise-controlled CT super-resolution utilizing the conditional diffusion model. The model is trained on hybrid datasets, combining noise-matched simulation data with segmented details from real data. Experimental results with real CT images validate the effectiveness of our proposed framework, showing its potential for practical applications in CT imaging.

Title: Vision-based Geo-Localization of Future Mars Rotorcraft in Challenging Illumination Conditions

• Authors: Dario Pisanti, Robert Hewitt, Roland Brockers, Georgios Georgakis
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.09795
• Pdf URL: https://arxiv.org/pdf/2502.09795
• Copy Paste: [[2502.09795]] Vision-based Geo-Localization of Future Mars Rotorcraft in Challenging Illumination Conditions(https://arxiv.org/abs/2502.09795)
• Keywords: robust
• Abstract: Planetary exploration using aerial assets has the potential for unprecedented scientific discoveries on Mars. While NASA's Mars helicopter Ingenuity proved flight in Martian atmosphere is possible, future Mars rotocrafts will require advanced navigation capabilities for long-range flights. One such critical capability is Map-based Localization (MbL) which registers an onboard image to a reference map during flight in order to mitigate cumulative drift from visual odometry. However, significant illumination differences between rotocraft observations and a reference map prove challenging for traditional MbL systems, restricting the operational window of the vehicle. In this work, we investigate a new MbL system and propose Geo-LoFTR, a geometry-aided deep learning model for image registration that is more robust under large illumination differences than prior models. The system is supported by a custom simulation framework that uses real orbital maps to produce large amounts of realistic images of the Martian terrain. Comprehensive evaluations show that our proposed system outperforms prior MbL efforts in terms of localization accuracy under significant lighting and scale variations. Furthermore, we demonstrate the validity of our approach across a simulated Martian day.

Title: VIRGOS: Secure Graph Convolutional Network on Vertically Split Data from Sparse Matrix Decomposition

• Authors: Yu Zheng, Qizhi Zhang, Lichun Li, Kai Zhou, Shan Yin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09808
• Pdf URL: https://arxiv.org/pdf/2502.09808
• Copy Paste: [[2502.09808]] VIRGOS: Secure Graph Convolutional Network on Vertically Split Data from Sparse Matrix Decomposition(https://arxiv.org/abs/2502.09808)
• Keywords: secure, privacy
• Abstract: Securely computing graph convolutional networks (GCNs) is critical for applying their analytical capabilities to privacy-sensitive data like social/credit networks. Multiplying a sparse yet large adjacency matrix of a graph in GCN--a core operation in training/inference--poses a performance bottleneck in secure GCNs. Consider a GCN with $|V|$ nodes and $|E|$ edges; it incurs a large $O(|V|^2)$ communication overhead. Modeling bipartite graphs and leveraging the monotonicity of non-zero entry locations, we propose a co-design harmonizing secure multi-party computation (MPC) with matrix sparsity. Our sparse matrix decomposition transforms an arbitrary sparse matrix into a product of structured matrices. Specialized MPC protocols for oblivious permutation and selection multiplication are then tailored, enabling our secure sparse matrix multiplication ($(SM)^2$) protocol, optimized for secure multiplication of these structured matrices. Together, these techniques take $O(|E|)$ communication in constant rounds. Supported by $(SM)^2$, we present Virgos, a secure 2-party framework that is communication-efficient and memory-friendly on standard vertically-partitioned graph datasets. Performance of Virgos has been empirically validated across diverse network conditions.

Title: AgentGuard: Repurposing Agentic Orchestrator for Safety Evaluation of Tool Orchestration

• Authors: Jizhou Chen, Samuel Lee Cong
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09809
• Pdf URL: https://arxiv.org/pdf/2502.09809
• Copy Paste: [[2502.09809]] AgentGuard: Repurposing Agentic Orchestrator for Safety Evaluation of Tool Orchestration(https://arxiv.org/abs/2502.09809)
• Keywords: security, large language model
• Abstract: The integration of tool use into large language models (LLMs) enables agentic systems with real-world impact. In the meantime, unlike standalone LLMs, compromised agents can execute malicious workflows with more consequential impact, signified by their tool-use capability. We propose AgentGuard, a framework to autonomously discover and validate unsafe tool-use workflows, followed by generating safety constraints to confine the behaviors of agents, achieving the baseline of safety guarantee at deployment. AgentGuard leverages the LLM orchestrator's innate capabilities - knowledge of tool functionalities, scalable and realistic workflow generation, and tool execution privileges - to act as its own safety evaluator. The framework operates through four phases: identifying unsafe workflows, validating them in real-world execution, generating safety constraints, and validating constraint efficacy. The output, an evaluation report with unsafe workflows, test cases, and validated constraints, enables multiple security applications. We empirically demonstrate AgentGuard's feasibility with experiments. With this exploratory work, we hope to inspire the establishment of standardized testing and hardening procedures for LLM agents to enhance their trustworthiness in real-world applications.

Title: Face Deepfakes - A Comprehensive Review

• Authors: Tharindu Fernando, Darshana Priyasad, Sridha Sridharan, Arun Ross, Clinton Fookes
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09812
• Pdf URL: https://arxiv.org/pdf/2502.09812
• Copy Paste: [[2502.09812]] Face Deepfakes - A Comprehensive Review(https://arxiv.org/abs/2502.09812)
• Keywords: biometric
• Abstract: In recent years, remarkable advancements in deep- fake generation technology have led to unprecedented leaps in its realism and capabilities. Despite these advances, we observe a notable lack of structured and deep analysis deepfake technology. The principal aim of this survey is to contribute a thorough theoretical analysis of state-of-the-art face deepfake generation and detection methods. Furthermore, we provide a coherent and systematic evaluation of the implications of deepfakes on face biometric recognition approaches. In addition, we outline key applications of face deepfake technology, elucidating both positive and negative applications of the technology, provide a detailed discussion regarding the gaps in existing research, and propose key research directions for further investigation.

Title: INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages

• Authors: Hao Yu, Jesujoba O. Alabi, Andiswa Bukula, Jian Yun Zhuang, En-Shiun Annie Lee, Tadesse Kebede Guge, Israel Abebe Azime, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson K. Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Shamsuddeen Hassan Muhammad, Salomey Osei, Sokhar Samb, Juliet W. Murage, Dietrich Klakow, David Ifeoluwa Adelani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09814
• Pdf URL: https://arxiv.org/pdf/2502.09814
• Copy Paste: [[2502.09814]] INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages(https://arxiv.org/abs/2502.09814)
• Keywords: transformer, large language model
• Abstract: Slot-filling and intent detection are well-established tasks in Conversational AI. However, current large-scale benchmarks for these tasks often exclude evaluations of low-resource languages and rely on translations from English benchmarks, thereby predominantly reflecting Western-centric concepts. In this paper, we introduce Injongo -- a multicultural, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains, including banking, travel, home, and dining. Through extensive experiments, we benchmark the fine-tuning multilingual transformer models and the prompting large language models (LLMs), and show the advantage of leveraging African-cultural utterances over Western-centric utterances for improving cross-lingual transfer from the English language. Experimental results reveal that current LLMs struggle with the slot-filling task, with GPT-4o achieving an average performance of 26 F1-score. In contrast, intent detection performance is notably better, with an average accuracy of 70.6%, though it still falls behind the fine-tuning baselines. Compared to the English language, GPT-4o and fine-tuning baselines perform similarly on intent detection, achieving an accuracy of approximately 81%. Our findings suggest that the performance of LLMs is still behind for many low-resource African languages, and more work is needed to further improve their downstream performance.

Title: Statistical Coherence Alignment for Large Language Model Representation Learning Through Tensor Field Convergence

• Authors: Jonathan Gale, Godfrey Aldington, Harriet Thistlewood, Thomas Tattershall, Basil Wentworth, Vincent Enoasmo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09815
• Pdf URL: https://arxiv.org/pdf/2502.09815
• Copy Paste: [[2502.09815]] Statistical Coherence Alignment for Large Language Model Representation Learning Through Tensor Field Convergence(https://arxiv.org/abs/2502.09815)
• Keywords: large language model
• Abstract: Representation learning plays a central role in structuring internal embeddings to capture the statistical properties of language, influencing the coherence and contextual consistency of generated text. Statistical Coherence Alignment is introduced as a method to enforce structured token representations through tensor field convergence, guiding embeddings to reflect statistical dependencies inherent in linguistic data. A mathematical framework is established to quantify coherence alignment, integrating a loss function that optimizes representational consistency across training iterations. Empirical evaluations demonstrate that applying coherence constraints improves perplexity, enhances classification accuracy, and refines rare word embeddings, contributing to a more stable representation space. Comparative analyses with baseline models reveal that the proposed method fosters a more interpretable internal structure, ensuring that embeddings retain contextual dependencies while mitigating representation collapse. The impact on coherence score distributions suggests that the alignment mechanism strengthens semantic integrity across diverse linguistic constructs, leading to a more balanced organization of learned embeddings. Computational assessments indicate that while the method introduces additional memory and training costs, the structured optimization process justifies the trade-offs in applications requiring heightened contextual fidelity. Experimental results validate the effectiveness of coherence alignment in optimizing token representations, providing insights into how statistical dependencies can be leveraged to improve language model training.

Title: On the robustness of multimodal language model towards distractions

• Authors: Ming Liu, Hao Chen, Jindong Wang, Wensheng Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09818
• Pdf URL: https://arxiv.org/pdf/2502.09818
• Copy Paste: [[2502.09818]] On the robustness of multimodal language model towards distractions(https://arxiv.org/abs/2502.09818)
• Keywords: robust
• Abstract: Although vision-language models (VLMs) have achieved significant success in various applications such as visual question answering, their resilience to prompt variations remains an under-explored area. Understanding how distractions affect VLMs is crucial for improving their real-world applicability, as inputs could have noisy and irrelevant information in many practical scenarios. This paper aims to assess the robustness of VLMs against both visual and textual distractions in the context of science question answering. Built on the ScienceQA dataset, we developed a new benchmark that introduces distractions in both the visual and textual contexts to evaluate the reasoning capacity of VLMs amid these distractions. Our findings reveal that most-of-the-art VLMs, including GPT-4, are vulnerable to various types of distractions, experiencing noticeable degradation in reasoning capabilities when confronted with distractions. Notably, models such as InternVL2 demonstrate a higher degree of robustness to these distractions. We also found that models exhibit greater sensitivity to textual distractions than visual ones. Additionally, we explored various mitigation strategies, such as prompt engineering, to counteract the impact of distractions. While these strategies improved solution accuracy, our analysis shows that there remain significant opportunities for improvement.

Title: A Solver-Aided Hierarchical Language for LLM-Driven CAD Design

• Authors: Benjamin T. Jones, Felix Hähnlein, Zihan Zhang, Maaz Ahmad, Vladimir Kim, Adriana Schulz
• Subjects: cs.CV, cs.AI, cs.GR, cs.LG, cs.PL
• Abstract URL: https://arxiv.org/abs/2502.09819
• Pdf URL: https://arxiv.org/pdf/2502.09819
• Copy Paste: [[2502.09819]] A Solver-Aided Hierarchical Language for LLM-Driven CAD Design(https://arxiv.org/abs/2502.09819)
• Keywords: generative, large language model
• Abstract: Large language models (LLMs) have been enormously successful in solving a wide variety of structured and unstructured generative tasks, but they struggle to generate procedural geometry in Computer Aided Design (CAD). These difficulties arise from an inability to do spatial reasoning and the necessity to guide a model through complex, long range planning to generate complex geometry. We enable generative CAD Design with LLMs through the introduction of a solver-aided, hierarchical domain specific language (DSL) called AIDL, which offloads the spatial reasoning requirements to a geometric constraint solver. Additionally, we show that in the few-shot regime, AIDL outperforms even a language with in-training data (OpenSCAD), both in terms of generating visual results closer to the prompt and creating objects that are easier to post-process and reason about.

Title: Learning Fair Policies for Infectious Diseases Mitigation using Path Integral Control

• Authors: Zhuangzhuang Jia, Hyuk Park, Gökçe Dayanıklı, Grani A. Hanasusanto
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2502.09831
• Pdf URL: https://arxiv.org/pdf/2502.09831
• Copy Paste: [[2502.09831]] Learning Fair Policies for Infectious Diseases Mitigation using Path Integral Control(https://arxiv.org/abs/2502.09831)
• Keywords: fair
• Abstract: Infectious diseases pose major public health challenges to society, highlighting the importance of designing effective policies to reduce economic loss and mortality. In this paper, we propose a framework for sequential decision-making under uncertainty to design fairness-aware disease mitigation policies that incorporate various measures of unfairness. Specifically, our approach learns equitable vaccination and lockdown strategies based on a stochastic multi-group SIR model. To address the challenges of solving the resulting sequential decision-making problem, we adopt the path integral control algorithm as an efficient solution scheme. Through a case study, we demonstrate that our approach effectively improves fairness compared to conventional methods and provides valuable insights for policymakers.

Title: Decentralized Entropy-Based Ransomware Detection Using Autonomous Feature Resonance

• Authors: Barnaby Quince, Levi Gareth, Sophie Larkspur, Thaddeus Wobblethorn, Thomas Quibble
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09833
• Pdf URL: https://arxiv.org/pdf/2502.09833
• Copy Paste: [[2502.09833]] Decentralized Entropy-Based Ransomware Detection Using Autonomous Feature Resonance(https://arxiv.org/abs/2502.09833)
• Keywords: security, robust
• Abstract: The increasing sophistication of cyber threats has necessitated the development of advanced detection mechanisms capable of identifying malicious activities with high precision and efficiency. A novel approach, termed Autonomous Feature Resonance, is introduced to address the limitations of traditional ransomware detection methods through the analysis of entropy-based feature interactions within system processes. The proposed method achieves an overall detection accuracy of 97.3\%, with false positive and false negative rates of 1.8\% and 2.1\%, respectively, outperforming existing techniques such as signature-based detection and behavioral analysis. Its decentralized architecture enables local processing of data, reducing latency and improving scalability, while a self-learning mechanism ensures continuous adaptation to emerging threats. Experimental results demonstrate consistent performance across diverse ransomware families, including LockBit 3.0, BlackCat, and Royal, with low detection latency and efficient resource utilization. The method's reliance on entropy as a distinguishing feature provides robustness against obfuscation techniques, making it suitable for real-time deployment in high-throughput environments. These findings highlight the potential of entropy-based approaches to enhance cybersecurity frameworks, offering a scalable and adaptive solution for modern ransomware detection challenges.

Title: SoK: State of the time: On Trustworthiness of Digital Clocks

• Authors: Adeel Nasrullah, Fatima M. Anwar
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09837
• Pdf URL: https://arxiv.org/pdf/2502.09837
• Copy Paste: [[2502.09837]] SoK: State of the time: On Trustworthiness of Digital Clocks(https://arxiv.org/abs/2502.09837)
• Keywords: security, attack
• Abstract: Despite the critical role of timing infrastructure in enabling essential services, from public key infrastructure and smart grids to autonomous navigation and high-frequency trading, modern timing stacks remain highly vulnerable to malicious attacks. These threats emerge due to several reasons, including inadequate security mechanisms, the timing architectures unique vulnerability to delays, and implementation issues. In this paper, we aim to obtain a holistic understanding of the issues that make the timing stacks vulnerable to adversarial manipulations, what the challenges are in securing them, and what solutions can be borrowed from the research community to address them. To this end, we perform a systematic analysis of the security vulnerabilities of the timing stack. In doing so, we discover new attack surfaces, i.e., physical timing components and on-device timekeeping, which are often overlooked by existing research that predominantly studies the security of time synchronization protocols. We also show that the emerging trusted timing architectures are flawed and risk compromising wider system security, and propose an alternative design using hardware-software co-design.

Title: HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation

• Authors: Tianwei Lin, Wenqiao Zhang, Sijing Li, Yuqian Yuan, Binhe Yu, Haoyuan Li, Wanggui He, Hao Jiang, Mengze Li, Xiaohui Song, Siliang Tang, Jun Xiao, Hui Lin, Yueting Zhuang, Beng Chin Ooi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09838
• Pdf URL: https://arxiv.org/pdf/2502.09838
• Copy Paste: [[2502.09838]] HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation(https://arxiv.org/abs/2502.09838)
• Keywords: large language model
• Abstract: We present HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm. Our bootstrapping philosophy is to progressively adapt heterogeneous comprehension and generation knowledge to pre-trained large language models (LLMs). This is achieved through a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is complemented by a tailored hierarchical visual perception approach and a three-stage learning strategy. To effectively learn the HealthGPT, we devise a comprehensive medical domain-specific comprehension and generation dataset called VL-Health. Experimental results demonstrate exceptional performance and scalability of HealthGPT in medical visual unified tasks. Our project can be accessed at this https URL.

Title: Solving Empirical Bayes via Transformers

• Authors: Anzo Teh, Mark Jabbour, Yury Polyanskiy
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09844
• Pdf URL: https://arxiv.org/pdf/2502.09844
• Copy Paste: [[2502.09844]] Solving Empirical Bayes via Transformers(https://arxiv.org/abs/2502.09844)
• Keywords: transformer
• Abstract: This work applies modern AI tools (transformers) to solving one of the oldest statistical problems: Poisson means under empirical Bayes (Poisson-EB) setting. In Poisson-EB a high-dimensional mean vector $\theta$ (with iid coordinates sampled from an unknown prior $\pi$) is estimated on the basis of $X=\mathrm{Poisson}(\theta)$. A transformer model is pre-trained on a set of synthetically generated pairs $(X,\theta)$ and learns to do in-context learning (ICL) by adapting to unknown $\pi$. Theoretically, we show that a sufficiently wide transformer can achieve vanishing regret with respect to an oracle estimator who knows $\pi$ as dimension grows to infinity. Practically, we discover that already very small models (100k parameters) are able to outperform the best classical algorithm (non-parametric maximum likelihood, or NPMLE) both in runtime and validation loss, which we compute on out-of-distribution synthetic data as well as real-world datasets (NHL hockey, MLB baseball, BookCorpusOpen). Finally, by using linear probes, we confirm that the transformer's EB estimator appears to internally work differently from either NPMLE or Robbins' estimators.

Title: Elastic Representation: Mitigating Spurious Correlations for Group Robustness

• Authors: Tao Wen, Zihan Wang, Quan Zhang, Qi Lei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09850
• Pdf URL: https://arxiv.org/pdf/2502.09850
• Copy Paste: [[2502.09850]] Elastic Representation: Mitigating Spurious Correlations for Group Robustness(https://arxiv.org/abs/2502.09850)
• Keywords: robust
• Abstract: Deep learning models can suffer from severe performance degradation when relying on spurious correlations between input features and labels, making the models perform well on training data but have poor prediction accuracy for minority groups. This problem arises especially when training data are limited or imbalanced. While most prior work focuses on learning invariant features (with consistent correlations to y), it overlooks the potential harm of spurious correlations between features. We hereby propose Elastic Representation (ElRep) to learn features by imposing Nuclear- and Frobenius-norm penalties on the representation from the last layer of a neural network. Similar to the elastic net, ElRep enjoys the benefits of learning important features without losing feature diversity. The proposed method is simple yet effective. It can be integrated into many deep learning approaches to mitigate spurious correlations and improve group robustness. Moreover, we theoretically show that ElRep has minimum negative impacts on in-distribution predictions. This is a remarkable advantage over approaches that prioritize minority groups at the cost of overall performance.

Title: Efficient Multitask Learning in Small Language Models Through Upside-Down Reinforcement Learning

• Authors: Yu-Chen Lin, Sanat Sharma, Hari Manikandan, Jayant Kumar, Tracy Holloway King, Jing Zheng
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09854
• Pdf URL: https://arxiv.org/pdf/2502.09854
• Copy Paste: [[2502.09854]] Efficient Multitask Learning in Small Language Models Through Upside-Down Reinforcement Learning(https://arxiv.org/abs/2502.09854)
• Keywords: large language model
• Abstract: In this work, we demonstrate that small language models (SLMs), specifically a 100M parameter GPT-2 model, can achieve competitive performance in multitask prompt generation tasks while requiring only a fraction of the computational resources needed by large language models (LLMs). Through a novel combination of upside-down reinforcement learning and synthetic data distillation from a powerful LLM, Llama-3, we train an SLM that achieves relevance scores within 5% of state-of-the-art models, including Llama-3, Qwen2, and Mistral, despite being up to 80 times smaller, making it highly suitable for resource-constrained and real-time applications. This study highlights the potential of SLMs as efficient multitask learners in multimodal settings, providing a promising alternative to LLMs for scalable, low-latency deployments.

Title: Automated Hypothesis Validation with Agentic Sequential Falsifications

• Authors: Kexin Huang, Ying Jin, Ryan Li, Michael Y. Li, Emmanuel Candès, Jure Leskovec
• Subjects: cs.LG, cs.AI, cs.CL, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.09858
• Pdf URL: https://arxiv.org/pdf/2502.09858
• Copy Paste: [[2502.09858]] Automated Hypothesis Validation with Agentic Sequential Falsifications(https://arxiv.org/abs/2502.09858)
• Keywords: robust, large language model
• Abstract: Hypotheses are central to information acquisition, decision-making, and discovery. However, many real-world hypotheses are abstract, high-level statements that are difficult to validate directly. This challenge is further intensified by the rise of hypothesis generation from Large Language Models (LLMs), which are prone to hallucination and produce hypotheses in volumes that make manual validation impractical. Here we propose Popper, an agentic framework for rigorous automated validation of free-form hypotheses. Guided by Karl Popper's principle of falsification, Popper validates a hypothesis using LLM agents that design and execute falsification experiments targeting its measurable implications. A novel sequential testing framework ensures strict Type-I error control while actively gathering evidence from diverse observations, whether drawn from existing data or newly conducted procedures. We demonstrate Popper on six domains including biology, economics, and sociology. Popper delivers robust error control, high power, and scalability. Furthermore, compared to human scientists, Popper achieved comparable performance in validating complex biological hypotheses while reducing time by 10 folds, providing a scalable, rigorous solution for hypothesis validation.

Title: Solvable Dynamics of Self-Supervised Word Embeddings and the Emergence of Analogical Reasoning

• Authors: Dhruva Karkada, James B. Simon, Yasaman Bahri, Michael R. DeWeese
• Subjects: cs.LG, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09863
• Pdf URL: https://arxiv.org/pdf/2502.09863
• Copy Paste: [[2502.09863]] Solvable Dynamics of Self-Supervised Word Embeddings and the Emergence of Analogical Reasoning(https://arxiv.org/abs/2502.09863)
• Keywords: large language model
• Abstract: The remarkable success of large language models relies on their ability to implicitly learn structured latent representations from the pretraining corpus. As a simpler surrogate for representation learning in language modeling, we study a class of solvable contrastive self-supervised algorithms which we term quadratic word embedding models. These models resemble the word2vec algorithm and perform similarly on downstream tasks. Our main contributions are analytical solutions for both the training dynamics (under certain hyperparameter choices) and the final word embeddings, given in terms of only the corpus statistics. Our solutions reveal that these models learn orthogonal linear subspaces one at a time, each one incrementing the effective rank of the embeddings until model capacity is saturated. Training on WikiText, we find that the top subspaces represent interpretable concepts. Finally, we use our dynamical theory to predict how and when models acquire the ability to complete analogies.

Title: U Can Touch This! Microarchitectural Timing Attacks via Machine Clears

• Authors: Billy Bob Brumley
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09864
• Pdf URL: https://arxiv.org/pdf/2502.09864
• Copy Paste: [[2502.09864]] U Can Touch This! Microarchitectural Timing Attacks via Machine Clears(https://arxiv.org/abs/2502.09864)
• Keywords: attack
• Abstract: Microarchitectural timing attacks exploit subtle timing variations caused by hardware behaviors to leak sensitive information. In this paper, we introduce MCHammer, a novel side-channel technique that leverages machine clears induced by self-modifying code detection mechanisms. Unlike most traditional techniques, MCHammer does not require memory access or waiting periods, making it highly efficient. We compare MCHammer to the classical Flush+Reload technique, improving in terms of trace granularity, providing a powerful side-channel attack vector. Using MCHammer, we successfully recover keys from a deployed implementation of a cryptographic tool. Our findings highlight the practical implications of MCHammer and its potential impact on real-world systems.

Title: Learning to Calibrate for Reliable Visual Fire Detection

• Authors: Ziqi Zhang, Xiuzhuang Zhou, Xiangyang Gong
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09872
• Pdf URL: https://arxiv.org/pdf/2502.09872
• Copy Paste: [[2502.09872]] Learning to Calibrate for Reliable Visual Fire Detection(https://arxiv.org/abs/2502.09872)
• Keywords: protect
• Abstract: Fire is characterized by its sudden onset and destructive power, making early fire detection crucial for ensuring human safety and protecting property. With the advancement of deep learning, the application of computer vision in fire detection has significantly improved. However, deep learning models often exhibit a tendency toward overconfidence, and most existing works focus primarily on enhancing classification performance, with limited attention given to uncertainty modeling. To address this issue, we propose transforming the Expected Calibration Error (ECE), a metric for measuring uncertainty, into a differentiable ECE loss function. This loss is then combined with the cross-entropy loss to guide the training process of multi-class fire detection models. Additionally, to achieve a good balance between classification accuracy and reliable decision, we introduce a curriculum learning-based approach that dynamically adjusts the weight of the ECE loss during training. Extensive experiments are conducted on two widely used multi-class fire detection datasets, DFAN and EdgeFireSmoke, validating the effectiveness of our uncertainty modeling method.

Title: Compression-Aware One-Step Diffusion Model for JPEG Artifact Removal

• Authors: Jinpei Guo, Zheng Chen, Wenbo Li, Yong Guo, Yulun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09873
• Pdf URL: https://arxiv.org/pdf/2502.09873
• Copy Paste: [[2502.09873]] Compression-Aware One-Step Diffusion Model for JPEG Artifact Removal(https://arxiv.org/abs/2502.09873)
• Keywords: diffusion
• Abstract: Diffusion models have demonstrated remarkable success in image restoration tasks. However, their multi-step denoising process introduces significant computational overhead, limiting their practical deployment. Furthermore, existing methods struggle to effectively remove severe JPEG artifact, especially in highly compressed images. To address these challenges, we propose CODiff, a compression-aware one-step diffusion model for JPEG artifact removal. The core of CODiff is the compression-aware visual embedder (CaVE), which extracts and leverages JPEG compression priors to guide the diffusion model. We propose a dual learning strategy that combines explicit and implicit learning. Specifically, explicit learning enforces a quality prediction objective to differentiate low-quality images with different compression levels. Implicit learning employs a reconstruction objective that enhances the model's generalization. This dual learning allows for a deeper and more comprehensive understanding of JPEG compression. Experimental results demonstrate that CODiff surpasses recent leading methods in both quantitative and visual quality metrics. The code and models will be released at this https URL.

Title: FrGNet: A fourier-guided weakly-supervised framework for nuclear instance segmentation

• Authors: Peng Ling
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09874
• Pdf URL: https://arxiv.org/pdf/2502.09874
• Copy Paste: [[2502.09874]] FrGNet: A fourier-guided weakly-supervised framework for nuclear instance segmentation(https://arxiv.org/abs/2502.09874)
• Keywords: segmentation
• Abstract: Nuclear instance segmentation has played a critical role in pathology image analysis. The main challenges arise from the difficulty in accurately segmenting instances and the high cost of precise mask-level annotations for fully-supervised this http URL this work, we propose a fourier guidance framework for solving the weakly-supervised nuclear instance segmentation problem. In this framework, we construct a fourier guidance module to fuse the priori information into the training process of the model, which facilitates the model to capture the relevant features of the this http URL, in order to further improve the model's ability to represent the features of nuclear, we propose the guide-based instance level contrastive module. This module makes full use of the framework's own properties and guide information to effectively enhance the representation features of nuclear. We show on two public datasets that our model can outperform current SOTA methods under fully-supervised design, and in weakly-supervised experiments, with only a small amount of labeling our model still maintains close to the performance under full this http URL addition, we also perform generalization experiments on a private dataset, and without any labeling, our model is able to segment nuclear images that have not been seen during training quite effectively. As open science, all codes and pre-trained models are available at this https URL.

Title: Comprehensive Review of Neural Differential Equations for Time Series Analysis

• Authors: YongKyung Oh, Seungsu Kam, Jonghun Lee, Dong-Young Lim, Sungil Kim, Alex Bui
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09885
• Pdf URL: https://arxiv.org/pdf/2502.09885
• Copy Paste: [[2502.09885]] Comprehensive Review of Neural Differential Equations for Time Series Analysis(https://arxiv.org/abs/2502.09885)
• Keywords: transformer
• Abstract: Time series modeling and analysis has become critical in various domains. Conventional methods such as RNNs and Transformers, while effective for discrete-time and regularly sampled data, face significant challenges in capturing the continuous dynamics and irregular sampling patterns inherent in real-world scenarios. Neural Differential Equations (NDEs) represent a paradigm shift by combining the flexibility of neural networks with the mathematical rigor of differential equations. This paper presents a comprehensive review of NDE-based methods for time series analysis, including neural ordinary differential equations, neural controlled differential equations, and neural stochastic differential equations. We provide a detailed discussion of their mathematical formulations, numerical methods, and applications, highlighting their ability to model continuous-time dynamics. Furthermore, we address key challenges and future research directions. This survey serves as a foundation for researchers and practitioners seeking to leverage NDEs for advanced time series analysis.

Title: Symmetry-Preserving Diffusion Models via Target Symmetrization

• Authors: Vinh Tong, Yun Ye, Trung-Dung Hoang, Anji Liu, Guy Van den Broeck, Mathias Niepert
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09890
• Pdf URL: https://arxiv.org/pdf/2502.09890
• Copy Paste: [[2502.09890]] Symmetry-Preserving Diffusion Models via Target Symmetrization(https://arxiv.org/abs/2502.09890)
• Keywords: diffusion, generative
• Abstract: Diffusion models are powerful tools for capturing complex distributions, but modeling data with inherent symmetries, such as molecular structures, remains challenging. Equivariant denoisers are commonly used to address this, but they introduce architectural complexity and optimization challenges, including noisy gradients and convergence issues. We propose a novel approach that enforces equivariance through a symmetrized loss function, which applies a time-dependent weighted averaging operation over group actions to the model's prediction target. This ensures equivariance without explicit architectural constraints and reduces gradient variance, leading to more stable and efficient optimization. Our method uses Monte Carlo sampling to estimate the average, incurring minimal computational overhead. We provide theoretical guarantees of equivariance for the minimizer of our loss function and demonstrate its effectiveness on synthetic datasets and the molecular conformation generation task using the GEOM-QM9 dataset. Experiments show improved sample quality compared to existing methods, highlighting the potential of our approach to enhance the scalability and practicality of equivariant diffusion models in generative tasks.

Title: ChatIoT: Large Language Model-based Security Assistant for Internet of Things with Retrieval-Augmented Generation

• Authors: Ye Dong, Yan Lin Aung, Sudipta Chattopadhyay, Jianying Zhou
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09896
• Pdf URL: https://arxiv.org/pdf/2502.09896
• Copy Paste: [[2502.09896]] ChatIoT: Large Language Model-based Security Assistant for Internet of Things with Retrieval-Augmented Generation(https://arxiv.org/abs/2502.09896)
• Keywords: security, attack, large language model
• Abstract: Internet of Things (IoT) has gained widespread popularity, revolutionizing industries and daily life. However, it has also emerged as a prime target for attacks. Numerous efforts have been made to improve IoT security, and substantial IoT security and threat information, such as datasets and reports, have been developed. However, existing research often falls short in leveraging these insights to assist or guide users in harnessing IoT security practices in a clear and actionable way. In this paper, we propose ChatIoT, a large language model (LLM)-based IoT security assistant designed to disseminate IoT security and threat intelligence. By leveraging the versatile property of retrieval-augmented generation (RAG), ChatIoT successfully integrates the advanced language understanding and reasoning capabilities of LLM with fast-evolving IoT security information. Moreover, we develop an end-to-end data processing toolkit to handle heterogeneous datasets. This toolkit converts datasets of various formats into retrievable documents and optimizes chunking strategies for efficient retrieval. Additionally, we define a set of common use case specifications to guide the LLM in generating answers aligned with users' specific needs and expertise levels. Finally, we implement a prototype of ChatIoT and conduct extensive experiments with different LLMs, such as LLaMA3, LLaMA3.1, and GPT-4o. Experimental evaluations demonstrate that ChatIoT can generate more reliable, relevant, and technical in-depth answers for most use cases. When evaluating the answers with LLaMA3:70B, ChatIoT improves the above metrics by over 10% on average, particularly in relevance and technicality, compared to using LLMs alone.

Title: Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding

• Authors: Thanh-Dat Truong, Hoang-Quan Nguyen, Xuan-Bac Nguyen, Ashley Dowling, Xin Li, Khoa Luu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09906
• Pdf URL: https://arxiv.org/pdf/2502.09906
• Copy Paste: [[2502.09906]] Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding(https://arxiv.org/abs/2502.09906)
• Keywords: generative
• Abstract: Multimodal conversational generative AI has shown impressive capabilities in various vision and language understanding through learning massive text-image data. However, current conversational models still lack knowledge about visual insects since they are often trained on the general knowledge of vision-language data. Meanwhile, understanding insects is a fundamental problem in precision agriculture, helping to promote sustainable development in agriculture. Therefore, this paper proposes a novel multimodal conversational model, Insect-LLaVA, to promote visual understanding in insect-domain knowledge. In particular, we first introduce a new large-scale Multimodal Insect Dataset with Visual Insect Instruction Data that enables the capability of learning the multimodal foundation models. Our proposed dataset enables conversational models to comprehend the visual and semantic features of the insects. Second, we propose a new Insect-LLaVA model, a new general Large Language and Vision Assistant in Visual Insect Understanding. Then, to enhance the capability of learning insect features, we develop an Insect Foundation Model by introducing a new micro-feature self-supervised learning with a Patch-wise Relevant Attention mechanism to capture the subtle differences among insect images. We also present Description Consistency loss to improve micro-feature learning via text descriptions. The experimental results evaluated on our new Visual Insect Question Answering benchmarks illustrate the effective performance of our proposed approach in visual insect understanding and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks.

Title: AttenGluco: Multimodal Transformer-Based Blood Glucose Forecasting on AI-READI Dataset

• Authors: Ebrahim Farahmand, Reza Rahimi Azghan, Nooshin Taheri Chatrudi, Eric Kim, Gautham Krishna Gudur, Edison Thomaz, Giulia Pedrielli, Pavan Turaga, Hassan Ghasemzadeh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09919
• Pdf URL: https://arxiv.org/pdf/2502.09919
• Copy Paste: [[2502.09919]] AttenGluco: Multimodal Transformer-Based Blood Glucose Forecasting on AI-READI Dataset(https://arxiv.org/abs/2502.09919)
• Keywords: transformer
• Abstract: Diabetes is a chronic metabolic disorder characterized by persistently high blood glucose levels (BGLs), leading to severe complications such as cardiovascular disease, neuropathy, and retinopathy. Predicting BGLs enables patients to maintain glucose levels within a safe range and allows caregivers to take proactive measures through lifestyle modifications. Continuous Glucose Monitoring (CGM) systems provide real-time tracking, offering a valuable tool for monitoring BGLs. However, accurately forecasting BGLs remains challenging due to fluctuations due to physical activity, diet, and other factors. Recent deep learning models show promise in improving BGL prediction. Nonetheless, forecasting BGLs accurately from multimodal, irregularly sampled data over long prediction horizons remains a challenging research problem. In this paper, we propose AttenGluco, a multimodal Transformer-based framework for long-term blood glucose prediction. AttenGluco employs cross-attention to effectively integrate CGM and activity data, addressing challenges in fusing data with different sampling rates. Moreover, it employs multi-scale attention to capture long-term dependencies in temporal data, enhancing forecasting accuracy. To evaluate the performance of AttenGluco, we conduct forecasting experiments on the recently released AIREADI dataset, analyzing its predictive accuracy across different subject cohorts including healthy individuals, people with prediabetes, and those with type 2 diabetes. Furthermore, we investigate its performance improvements and forgetting behavior as new cohorts are introduced. Our evaluations show that AttenGluco improves all error metrics, such as root mean square error (RMSE), mean absolute error (MAE), and correlation, compared to the multimodal LSTM model. AttenGluco outperforms this baseline model by about 10% and 15% in terms of RMSE and MAE, respectively.

Title: Self-Consistent Model-based Adaptation for Visual Reinforcement Learning

• Authors: Xinning Zhou, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09923
• Pdf URL: https://arxiv.org/pdf/2502.09923
• Copy Paste: [[2502.09923]] Self-Consistent Model-based Adaptation for Visual Reinforcement Learning(https://arxiv.org/abs/2502.09923)
• Keywords: robust
• Abstract: Visual reinforcement learning agents typically face serious performance declines in real-world applications caused by visual distractions. Existing methods rely on fine-tuning the policy's representations with hand-crafted augmentations. In this work, we propose Self-Consistent Model-based Adaptation (SCMA), a novel method that fosters robust adaptation without modifying the policy. By transferring cluttered observations to clean ones with a denoising model, SCMA can mitigate distractions for various policies as a plug-and-play enhancement. To optimize the denoising model in an unsupervised manner, we derive an unsupervised distribution matching objective with a theoretical analysis of its optimality. We further present a practical algorithm to optimize the objective by estimating the distribution of clean observations with a pre-trained world model. Extensive experiments on multiple visual generalization benchmarks and real robot data demonstrate that SCMA effectively boosts performance across various distractions and exhibits better sample efficiency.

Title: Robust Anomaly Detection via Tensor Chidori Pseudoskeleton Decomposition

• Authors: Bowen Su
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09926
• Pdf URL: https://arxiv.org/pdf/2502.09926
• Copy Paste: [[2502.09926]] Robust Anomaly Detection via Tensor Chidori Pseudoskeleton Decomposition(https://arxiv.org/abs/2502.09926)
• Keywords: robust
• Abstract: Anomaly detection plays a critical role in modern data-driven applications, from identifying fraudulent transactions and safeguarding network infrastructure to monitoring sensor systems for irregular patterns. Traditional approaches, such as distance, density, or cluster-based methods, face significant challenges when applied to high dimensional tensor data, where complex interdependencies across dimensions amplify noise and computational complexity. To address these limitations, this paper leverages Tensor Chidori pseudoskeleton decomposition within a tensor-robust principal component analysis framework to extract low Tucker rank structure while isolating sparse anomalies, ensuring robustness to anomaly detection. We establish theoretical results regarding convergence, and estimation error, demonstrating the stability and accuracy of the proposed approach. Numerical experiments on real-world spatiotemporal data from New York City taxi trip records validate the superiority of the proposed method in detecting anomalous urban events compared to existing benchmark methods. The results underscore the potential of Tensor Chidori pseudoskeleton decomposition to enhance anomaly detection for large-scale, high-dimensional data.

Title: Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence

• Authors: Granite Vision Team: Leonid Karlinsky, Assaf Arbelle, Abraham Daniels, Ahmed Nassar, Amit Alfassi, Bo Wu, Eli Schwartz, Dhiraj Joshi, Jovana Kondic, Nimrod Shabtay, Pengyuan Li, Roei Herzig, Shafiq Abedin, Shaked Perek, Sivan Harary, Udi Barzelay, Adi Raz Goldfarb, Aude Oliva, Ben Wieles, Bishwaranjan Bhattacharjee, Brandon Huang, Christoph Auer, Dan Gutfreund, David Beymer, David Wood, Hilde Kuehne, Jacob Hansen, Joseph Shtok, Ken Wong, Luis Angel Bathen, Mayank Mishra, Maksym Lysak, Michele Dolfi, Mikhail Yurochkin, Nikolaos Livathinos, Nimrod Harel, Ophir Azulai, Oshri Naparstek, Rafael Teixeira de Lima, Rameswar Panda, Sivan Doveh, Shubham Gupta, Subhro Das, Syed Zawad, Yusik Kim, Zexue He, Alexander Brooks, Gabe Goodhart, Anita Govindjee, Derek Leist, Ibrahim Ibrahim, Aya Soffer, David Cox, Kate Soule, Luis Lastras, Nirmit Desai, Shila Ofek-koifman, Sriram Raghavan, Tanveer Syeda-Mahmood, Peter Staar, Tal Drory, Rogerio Feris
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09927
• Pdf URL: https://arxiv.org/pdf/2502.09927
• Copy Paste: [[2502.09927]] Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence(https://arxiv.org/abs/2502.09927)
• Keywords: extraction, large language model
• Abstract: We introduce Granite Vision, a lightweight large language model with vision capabilities, specifically designed to excel in enterprise use cases, particularly in visual document understanding. Our model is trained on a comprehensive instruction-following dataset, including document-related tasks, such as content extraction from tables, charts, diagrams, sketches, and infographics, as well as general image tasks. The architecture of Granite Vision is centered around visual modality alignment with a decoder-only, 2 billion parameter Granite large language model. Additionally, we introduce a dedicated safety classification approach in test-time that leverages a sparse set of attention vectors to identify potential harmful inputs. Despite its lightweight architecture, Granite Vision achieves strong results in standard benchmarks related to visual document understanding, as well as on the LiveXiv benchmark, which is designed to avoid test set contamination by using a constantly updated corpus of recently published Arxiv papers. We are releasing the model under the Apache-2 license, allowing for both research and commercial use, while offering complete visibility into the training data and other relevant details. See this https URL for model weights.

Title: TransGUNet: Transformer Meets Graph-based Skip Connection for Medical Image Segmentation

• Authors: Ju-Hyeon Nam, Nur Suriza Syazwany, Sang-Chul Lee
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09931
• Pdf URL: https://arxiv.org/pdf/2502.09931
• Copy Paste: [[2502.09931]] TransGUNet: Transformer Meets Graph-based Skip Connection for Medical Image Segmentation(https://arxiv.org/abs/2502.09931)
• Keywords: robust, transformer, segmentation
• Abstract: Skip connection engineering is primarily employed to address the semantic gap between the encoder and decoder, while also integrating global dependencies to understand the relationships among complex anatomical structures in medical image segmentation. Although several models have proposed transformer-based approaches to incorporate global dependencies within skip connections, they often face limitations in capturing detailed local features with high computational complexity. In contrast, graph neural networks (GNNs) exploit graph structures to effectively capture local and global features. Leveraging these properties, we introduce an attentional cross-scale graph neural network (ACS-GNN), which enhances the skip connection framework by converting cross-scale feature maps into a graph structure and capturing complex anatomical structures through node attention. Additionally, we observed that deep learning models often produce uninformative feature maps, which degrades the quality of spatial attention maps. To address this problem, we integrated entropy-driven feature selection (EFS) with spatial attention, calculating an entropy score for each channel and filtering out high-entropy feature maps. Our innovative framework, TransGUNet, comprises ACS-GNN and EFS-based spatial attentio} to effectively enhance domain generalizability across various modalities by leveraging GNNs alongside a reliable spatial attention map, ensuring more robust features within the skip connection. Through comprehensive experiments and analysis, TransGUNet achieved superior segmentation performance on six seen and eight unseen datasets, demonstrating significantly higher efficiency compared to previous methods.

Title: Fused Partial Gromov-Wasserstein for Structured Objects

• Authors: Yikun Bai, Huy Tran, Hengrong Du, Xinran Liu, Soheil Kolouri
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09934
• Pdf URL: https://arxiv.org/pdf/2502.09934
• Copy Paste: [[2502.09934]] Fused Partial Gromov-Wasserstein for Structured Objects(https://arxiv.org/abs/2502.09934)
• Keywords: robust
• Abstract: Structured data, such as graphs, are vital in machine learning due to their capacity to capture complex relationships and interactions. In recent years, the Fused Gromov-Wasserstein (FGW) distance has attracted growing interest because it enables the comparison of structured data by jointly accounting for feature similarity and geometric structure. However, as a variant of optimal transport (OT), classical FGW assumes an equal mass constraint on the compared data. In this work, we relax this mass constraint and propose the Fused Partial Gromov-Wasserstein (FPGW) framework, which extends FGW to accommodate unbalanced data. Theoretically, we establish the relationship between FPGW and FGW and prove the metric properties of FPGW. Numerically, we introduce Frank-Wolfe solvers for the proposed FPGW framework and provide a convergence analysis. Finally, we evaluate the FPGW distance through graph classification and clustering experiments, demonstrating its robust performance, especially when data is corrupted by outlier noise.

Title: Precise Parameter Localization for Textual Generation in Diffusion Models

• Authors: Łukasz Staniszewski, Bartosz Cywiński, Franziska Boenisch, Kamil Deja, Adam Dziedzic
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09935
• Pdf URL: https://arxiv.org/pdf/2502.09935
• Copy Paste: [[2502.09935]] Precise Parameter Localization for Textual Generation in Diffusion Models(https://arxiv.org/abs/2502.09935)
• Keywords: diffusion, transformer, large language model
• Abstract: Novel diffusion models can synthesize photo-realistic images with integrated high-quality text. Surprisingly, we demonstrate through attention activation patching that only less than 1% of diffusion models' parameters, all contained in attention layers, influence the generation of textual content within the images. Building on this observation, we improve textual generation efficiency and performance by targeting cross and joint attention layers of diffusion models. We introduce several applications that benefit from localizing the layers responsible for textual content generation. We first show that a LoRA-based fine-tuning solely of the localized layers enhances, even more, the general text-generation capabilities of large diffusion models while preserving the quality and diversity of the diffusion models' generations. Then, we demonstrate how we can use the localized layers to edit textual content in generated images. Finally, we extend this idea to the practical use case of preventing the generation of toxic text in a cost-free manner. In contrast to prior work, our localization approach is broadly applicable across various diffusion model architectures, including U-Net (e.g., LDM and SDXL) and transformer-based (e.g., DeepFloyd IF and Stable Diffusion 3), utilizing diverse text encoders (e.g., from CLIP to the large language models like T5). Project page available at this https URL.

Title: A Preliminary Exploration with GPT-4o Voice Mode

• Authors: Yu-Xiang Lin, Chih-Kai Yang, Wei-Chih Chen, Chen-An Li, Chien-yu Huang, Xuanjun Chen, Hung-yi Lee
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.09940
• Pdf URL: https://arxiv.org/pdf/2502.09940
• Copy Paste: [[2502.09940]] A Preliminary Exploration with GPT-4o Voice Mode(https://arxiv.org/abs/2502.09940)
• Keywords: robust, large language model
• Abstract: With the rise of multimodal large language models, GPT-4o stands out as a pioneering model, driving us to evaluate its capabilities. This report assesses GPT-4o across various tasks to analyze its audio processing and reasoning abilities. We find that GPT-4o exhibits strong knowledge in audio, speech, and music understanding, performing well in tasks like intent classification, spoken command classification, semantic and grammatical reasoning., multilingual speech recognition, and singing analysis. It also shows greater robustness against hallucinations than other large audio-language models (LALMs). However, it struggles with tasks such as audio duration prediction and instrument classification. Additionally, GPT-4o's safety mechanisms cause it to decline tasks like speaker identification, age classification, MOS prediction, and audio deepfake detection. Notably, the model exhibits a significantly different refusal rate when responding to speaker verification tasks on different datasets. This is likely due to variations in the accompanying instructions or the quality of the input audio, suggesting the sensitivity of its built-in safeguards. Finally, we acknowledge that model performance varies with evaluation protocols. This report only serves as a preliminary exploration of the current state of LALMs.

Title: A Lightweight and Effective Image Tampering Localization Network with Vision Mamba

• Authors: Kun Guo, Gang Cao, Zijie Lou, Xianglin Huang, Jiaoyun Liu
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.09941
• Pdf URL: https://arxiv.org/pdf/2502.09941
• Copy Paste: [[2502.09941]] A Lightweight and Effective Image Tampering Localization Network with Vision Mamba(https://arxiv.org/abs/2502.09941)
• Keywords: robust, transformer
• Abstract: Current image tampering localization methods primarily rely on Convolutional Neural Networks (CNNs) and Transformers. While CNNs suffer from limited local receptive fields, Transformers offer global context modeling at the expense of quadratic computational complexity. Recently, the state space model Mamba has emerged as a competitive alternative, enabling linear-complexity global dependency modeling. Inspired by it, we propose a lightweight and effective FORensic network based on vision MAmba (ForMa) for blind image tampering localization. Firstly, ForMa captures multi-scale global features that achieves efficient global dependency modeling through linear complexity. Then the pixel-wise localization map is generated by a lightweight decoder, which employs a parameter-free pixel shuffle layer for upsampling. Additionally, a noise-assisted decoding strategy is proposed to integrate complementary manipulation traces from tampered images, boosting decoder sensitivity to forgery cues. Experimental results on 10 standard datasets demonstrate that ForMa achieves state-of-the-art generalization ability and robustness, while maintaining the lowest computational complexity. Code is available at this https URL.

Title: Using MRNet to Predict Lunar Rock Categories Detected by Chang'e 5 Probe

• Authors: Jin Cui, Yifei Zou, Siyuan Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09952
• Pdf URL: https://arxiv.org/pdf/2502.09952
• Copy Paste: [[2502.09952]] Using MRNet to Predict Lunar Rock Categories Detected by Chang'e 5 Probe(https://arxiv.org/abs/2502.09952)
• Keywords: extraction
• Abstract: China's Chang'e 5 mission has been a remarkable success, with the chang'e 5 lander traveling on the Oceanus Procellarum to collect images of the lunar surface. Over the past half century, people have brought back some lunar rock samples, but its quantity does not meet the need for research. Under current circumstances, people still mainly rely on the analysis of rocks on the lunar surface through the detection of lunar rover. The Oceanus Procellarum, chosen by Chang'e 5 mission, contains various kind of rock species. Therefore, we first applied to the National Astronomical Observatories of the China under the Chinese Academy of Sciences for the Navigation and Terrain Camera (NaTeCam) of the lunar surface image, and established a lunar surface rock image data set CE5ROCK. The data set contains 100 images, which randomly divided into training, validation and test set. Experimental results show that the identification accuracy testing on convolutional neural network (CNN) models like AlexNet or MobileNet is about to 40.0%. In order to make full use of the global information in Moon images, this paper proposes the MRNet (MoonRockNet) network architecture. The encoding structure of the network uses VGG16 for feature extraction, and the decoding part adds dilated convolution and commonly used U-Net structure on the original VGG16 decoding structure, which is more conducive to identify more refined but more sparsely distributed types of lunar rocks. We have conducted extensive experiments on the established CE5ROCK data set, and the experimental results show that MRNet can achieve more accurate rock type identification, and outperform other existing mainstream algorithms in the identification performance.

Title: Generating on Generated: An Approach Towards Self-Evolving Diffusion Models

• Authors: Xulu Zhang, Xiaoyong Wei, Jinlin Wu, Jiaxin Wu, Zhaoxiang Zhang, Zhen Lei, Qing Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09963
• Pdf URL: https://arxiv.org/pdf/2502.09963
• Copy Paste: [[2502.09963]] Generating on Generated: An Approach Towards Self-Evolving Diffusion Models(https://arxiv.org/abs/2502.09963)
• Keywords: diffusion, generative
• Abstract: Recursive Self-Improvement (RSI) enables intelligence systems to autonomously refine their capabilities. This paper explores the application of RSI in text-to-image diffusion models, addressing the challenge of training collapse caused by synthetic data. We identify two key factors contributing to this collapse: the lack of perceptual alignment and the accumulation of generative hallucinations. To mitigate these issues, we propose three strategies: (1) a prompt construction and filtering pipeline designed to facilitate the generation of perceptual aligned data, (2) a preference sampling method to identify human-preferred samples and filter out generative hallucinations, and (3) a distribution-based weighting scheme to penalize selected samples with hallucinatory errors. Our extensive experiments validate the effectiveness of these approaches.

Title: Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression

• Authors: Siqi Wu, Yinda Chen, Dong Liu, Zhihai He
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09971
• Pdf URL: https://arxiv.org/pdf/2502.09971
• Copy Paste: [[2502.09971]] Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression(https://arxiv.org/abs/2502.09971)
• Keywords: robust
• Abstract: In this paper, we study how to synthesize a dynamic reference from an external dictionary to perform conditional coding of the input image in the latent domain and how to learn the conditional latent synthesis and coding modules in an end-to-end manner. Our approach begins by constructing a universal image feature dictionary using a multi-stage approach involving modified spatial pyramid pooling, dimension reduction, and multi-scale feature clustering. For each input image, we learn to synthesize a conditioning latent by selecting and synthesizing relevant features from the dictionary, which significantly enhances the model's capability in capturing and exploring image source correlation. This conditional latent synthesis involves a correlation-based feature matching and alignment strategy, comprising a Conditional Latent Matching (CLM) module and a Conditional Latent Synthesis (CLS) module. The synthesized latent is then used to guide the encoding process, allowing for more efficient compression by exploiting the correlation between the input image and the reference dictionary. According to our theoretical analysis, the proposed conditional latent coding (CLC) method is robust to perturbations in the external dictionary samples and the selected conditioning latent, with an error bound that scales logarithmically with the dictionary size, ensuring stability even with large and diverse dictionaries. Experimental results on benchmark datasets show that our new method improves the coding performance by a large margin (up to 1.2 dB) with a very small overhead of approximately 0.5\% bits per pixel. Our code is publicly available at this https URL.

Title: LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs - No Silver Bullet for LC or RAG Routing

• Authors: Kuan Li, Liwen Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Shuai Wang, Minhao Cheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09977
• Pdf URL: https://arxiv.org/pdf/2502.09977
• Copy Paste: [[2502.09977]] LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs - No Silver Bullet for LC or RAG Routing(https://arxiv.org/abs/2502.09977)
• Keywords: large language model
• Abstract: Effectively incorporating external knowledge into Large Language Models (LLMs) is crucial for enhancing their capabilities and addressing real-world needs. Retrieval-Augmented Generation (RAG) offers an effective method for achieving this by retrieving the most relevant fragments into LLMs. However, the advancements in context window size for LLMs offer an alternative approach, raising the question of whether RAG remains necessary for effectively handling external knowledge. Several existing studies provide inconclusive comparisons between RAG and long-context (LC) LLMs, largely due to limitations in the benchmark designs. In this paper, we present LaRA, a novel benchmark specifically designed to rigorously compare RAG and LC LLMs. LaRA encompasses 2,326 test cases across four practical QA task categories and three types of naturally occurring long texts. Through systematic evaluation of seven open-source and four proprietary LLMs, we find that the optimal choice between RAG and LC depends on a complex interplay of factors, including the model's parameter size, long-text capabilities, context length, task type, and the characteristics of the retrieved chunks. Our findings provide actionable guidelines for practitioners to effectively leverage both RAG and LC approaches in developing and deploying LLM applications. Our code and dataset is provided at: \href{this https URL}{\textbf{this https URL}}.

Title: V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multi-Modal Large Language Models

• Authors: Hsu-kuang Chiu, Ryo Hachiuma, Chien-Yi Wang, Stephen F. Smith, Yu-Chiang Frank Wang, Min-Hung Chen
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.09980
• Pdf URL: https://arxiv.org/pdf/2502.09980
• Copy Paste: [[2502.09980]] V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multi-Modal Large Language Models(https://arxiv.org/abs/2502.09980)
• Keywords: large language model
• Abstract: Current autonomous driving vehicles rely mainly on their individual sensors to understand surrounding scenes and plan for future trajectories, which can be unreliable when the sensors are malfunctioning or occluded. To address this problem, cooperative perception methods via vehicle-to-vehicle (V2V) communication have been proposed, but they have tended to focus on detection and tracking. How those approaches contribute to overall cooperative planning performance is still under-explored. Inspired by recent progress using Large Language Models (LLMs) to build autonomous driving systems, we propose a novel problem setting that integrates an LLM into cooperative autonomous driving, with the proposed Vehicle-to-Vehicle Question-Answering (V2V-QA) dataset and benchmark. We also propose our baseline method Vehicle-to-Vehicle Large Language Model (V2V-LLM), which uses an LLM to fuse perception information from multiple connected autonomous vehicles (CAVs) and answer driving-related questions: grounding, notable object identification, and planning. Experimental results show that our proposed V2V-LLM can be a promising unified model architecture for performing various tasks in cooperative autonomous driving, and outperforms other baseline methods that use different fusion approaches. Our work also creates a new research direction that can improve the safety of future autonomous driving systems. Our project website: this https URL .

Title: Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Data

• Authors: Harsh Poonia, Felix Divo, Kristian Kersting, Devendra Singh Dhami
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09981
• Pdf URL: https://arxiv.org/pdf/2502.09981
• Copy Paste: [[2502.09981]] Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Data(https://arxiv.org/abs/2502.09981)
• Keywords: robust
• Abstract: Causality in time series can be difficult to determine, especially in the presence of non-linear dependencies. The concept of Granger causality helps analyze potential relationships between variables, thereby offering a method to determine whether one time series can predict-Granger cause-future values of another. Although successful, Granger causal methods still struggle with capturing long-range relations between variables. To this end, we leverage the recently successful Extended Long Short-Term Memory (xLSTM) architecture and propose Granger causal xLSTMs (GC-xLSTM). It first enforces sparsity between the time series components by using a novel dynamic lass penalty on the initial projection. Specifically, we adaptively improve the model and identify sparsity candidates. Our joint optimization procedure then ensures that the Granger causal relations are recovered in a robust fashion. Our experimental evaluations on three datasets demonstrate the overall efficacy of our proposed GC-xLSTM model.

Title: X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability

• Authors: Xiaoya Lu, Dongrui Liu, Yi Yu, Luxin Xu, Jing Shao
• Subjects: cs.CR, cs.AI, cs.CL, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09990
• Pdf URL: https://arxiv.org/pdf/2502.09990
• Copy Paste: [[2502.09990]] X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability(https://arxiv.org/abs/2502.09990)
• Keywords: defense, robust, interpretability
• Abstract: Despite the rapid development of safety alignment techniques for LLMs, defending against multi-turn jailbreaks is still a challenging task. In this paper, we conduct a comprehensive comparison, revealing that some existing defense methods can improve the robustness of LLMs against multi-turn jailbreaks but compromise usability, i.e., reducing general capabilities or causing the over-refusal problem. From the perspective of mechanism interpretability of LLMs, we discover that these methods fail to establish a boundary that exactly distinguishes safe and harmful feature representations. Therefore, boundary-safe representations close to harmful representations are inevitably disrupted, leading to a decline in usability. To address this issue, we propose X-Boundary to push harmful representations away from boundary-safe representations and obtain an exact distinction boundary. In this way, harmful representations can be precisely erased without disrupting safe ones. Experimental results show that X-Boundary achieves state-of-the-art defense performance against multi-turn jailbreaks, while reducing the over-refusal rate by about 20% and maintaining nearly complete general capability. Furthermore, we theoretically prove and empirically verify that X-Boundary can accelerate the convergence process during training. Please see our code at: this https URL.

Title: Large Language Diffusion Models

• Authors: Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, Chongxuan Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09992
• Pdf URL: https://arxiv.org/pdf/2502.09992
• Copy Paste: [[2502.09992]] Large Language Diffusion Models(https://arxiv.org/abs/2502.09992)
• Keywords: diffusion, transformer, generative, large language model
• Abstract: Autoregressive models (ARMs) are widely regarded as the cornerstone of large language models (LLMs). We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm. LLaDA models distributions through a forward data masking process and a reverse process, parameterized by a vanilla Transformer to predict masked tokens. By optimizing a likelihood bound, it provides a principled generative approach for probabilistic inference. Across extensive benchmarks, LLaDA demonstrates strong scalability, outperforming our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings establish diffusion models as a viable and promising alternative to ARMs, challenging the assumption that key LLM capabilities discussed above are inherently tied to ARMs.

Title: Navigating Label Ambiguity for Facial Expression Recognition in the Wild

• Authors: JunGyu Lee, Yeji Choi, Haksub Kim, Ig-Jae Kim, Gi Pyo Nam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09993
• Pdf URL: https://arxiv.org/pdf/2502.09993
• Copy Paste: [[2502.09993]] Navigating Label Ambiguity for Facial Expression Recognition in the Wild(https://arxiv.org/abs/2502.09993)
• Keywords: robust
• Abstract: Facial expression recognition (FER) remains a challenging task due to label ambiguity caused by the subjective nature of facial expressions and noisy samples. Additionally, class imbalance, which is common in real-world datasets, further complicates FER. Although many studies have shown impressive improvements, they typically address only one of these issues, leading to suboptimal results. To tackle both challenges simultaneously, we propose a novel framework called Navigating Label Ambiguity (NLA), which is robust under real-world conditions. The motivation behind NLA is that dynamically estimating and emphasizing ambiguous samples at each iteration helps mitigate noise and class imbalance by reducing the model's bias toward majority classes. To achieve this, NLA consists of two main components: Noise-aware Adaptive Weighting (NAW) and consistency regularization. Specifically, NAW adaptively assigns higher importance to ambiguous samples and lower importance to noisy ones, based on the correlation between the intermediate prediction scores for the ground truth and the nearest negative. Moreover, we incorporate a regularization term to ensure consistent latent distributions. Consequently, NLA enables the model to progressively focus on more challenging ambiguous samples, which primarily belong to the minority class, in the later stages of training. Extensive experiments demonstrate that NLA outperforms existing methods in both overall and mean accuracy, confirming its robustness against noise and class imbalance. To the best of our knowledge, this is the first framework to address both problems simultaneously.

Title: EmbBERT-Q: Breaking Memory Barriers in Embedded NLP

• Authors: Riccardo Bravin, Massimo Pavan, Hazem Hesham Yousef Shalby, Fabrizio Pittorino, Manuel Roveri
• Subjects: cs.CL, cs.AR, cs.DC, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10001
• Pdf URL: https://arxiv.org/pdf/2502.10001
• Copy Paste: [[2502.10001]] EmbBERT-Q: Breaking Memory Barriers in Embedded NLP(https://arxiv.org/abs/2502.10001)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have revolutionized natural language processing, setting new standards across a wide range of applications. However, their relevant memory and computational demands make them impractical for deployment on technologically-constrained tiny devices such as wearable devices and Internet-of-Things units. To address this limitation, we introduce EmbBERT-Q, a novel tiny language model specifically designed for tiny devices with stringent memory constraints. EmbBERT-Q achieves state-of-the-art (SotA) accuracy in Natural Language Processing tasks in this scenario, with a total memory footprint (weights and activations) of just 781 kB, representing a 25x reduction in size with respect to SotA models. By combining architectural innovations with hardware-compatible 8-bit quantization, EmbBERT-Q consistently outperforms several baseline models scaled down to a 2 MB memory budget (i.e., the maximum memory typically available in tiny devices), including heavily compressed versions of BERT and MAMBA. Extensive experimental evaluations on both a selected benchmark dataset, TinyNLP, specifically curated to evaluate Tiny Language Models in NLP tasks and real-world scenarios, and the GLUE benchmark, demonstrate EmbBERT-Q ability to deliver competitive accuracy with respect to existing approaches, achieving an unmatched balance between memory and performance. To ensure the complete and immediate reproducibility of all our results, we release all code, scripts, and model checkpoints at this https URL.

Title: Probabilistic Lexical Manifold Construction in Large Language Models via Hierarchical Vector Field Interpolation

• Authors: Clive Pendleton, Ewan Harrington, Giles Fairbrother, Jasper Arkwright, Nigel Fenwick, Richard Katrix
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10013
• Pdf URL: https://arxiv.org/pdf/2502.10013
• Copy Paste: [[2502.10013]] Probabilistic Lexical Manifold Construction in Large Language Models via Hierarchical Vector Field Interpolation(https://arxiv.org/abs/2502.10013)
• Keywords: transformer, large language model
• Abstract: Hierarchical vector field interpolation introduces a structured probabilistic framework for lexical representation, ensuring that word embeddings transition smoothly across a continuous manifold rather than being constrained to discrete token mappings. The proposed methodology constructs a probabilistic function space where word representations adhere to topological consistency, mitigating representational discontinuities commonly observed in transformer-based embeddings. Empirical evaluations reveal that probabilistic constraints enhance lexical coherence by refining contextual relationships, leading to improvements in semantic stability across multiple linguistic distributions. The application of divergence minimization techniques ensures that interpolated embeddings maintain probabilistic consistency while preserving computational feasibility for large-scale implementations. Experimental findings demonstrate that interpolated lexical manifolds improve representation density alignment, reducing anisotropic distortions in contextual embedding distributions. Comparative analyses with standard transformer-based models highlight that structured interpolation yields more stable representations, particularly in tasks requiring fine-grained semantic differentiation. The statistical evaluation of embedding divergence confirms that probabilistic lexical manifolds reduce representational inconsistencies while maintaining coherence across varying scales of contextual abstraction. An assessment of computational efficiency reveals that while interpolation introduces minor processing overhead, the structured representation learning approach remains scalable for practical deployment.

Title: ManiTrend: Bridging Future Generation and Action Prediction with 3D Flow for Robotic Manipulation

• Authors: Yuxin He, Qiang Nie
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.10028
• Pdf URL: https://arxiv.org/pdf/2502.10028
• Copy Paste: [[2502.10028]] ManiTrend: Bridging Future Generation and Action Prediction with 3D Flow for Robotic Manipulation(https://arxiv.org/abs/2502.10028)
• Keywords: transformer
• Abstract: Language-conditioned manipulation is a vital but challenging robotic task due to the high-level abstraction of language. To address this, researchers have sought improved goal representations derived from natural language. In this paper, we highlight 3D flow - representing the motion trend of 3D particles within a scene - as an effective bridge between language-based future image generation and fine-grained action prediction. To this end, we develop ManiTrend, a unified framework that models the dynamics of 3D particles, vision observations and manipulation actions with a causal transformer. Within this framework, features for 3D flow prediction serve as additional conditions for future image generation and action prediction, alleviating the complexity of pixel-wise spatiotemporal modeling and providing seamless action guidance. Furthermore, 3D flow can substitute missing or heterogeneous action labels during large-scale pretraining on cross-embodiment demonstrations. Experiments on two comprehensive benchmarks demonstrate that our method achieves state-of-the-art performance with high efficiency. Our code and model checkpoints will be available upon acceptance.

Title: ORI: O Routing Intelligence

• Authors: Ahmad Shadid, Rahul Kumar, Mohit Mayank
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10051
• Pdf URL: https://arxiv.org/pdf/2502.10051
• Copy Paste: [[2502.10051]] ORI: O Routing Intelligence(https://arxiv.org/abs/2502.10051)
• Keywords: large language model
• Abstract: Single large language models (LLMs) often fall short when faced with the ever-growing range of tasks, making a single-model approach insufficient. We address this challenge by proposing ORI (O Routing Intelligence), a dynamic framework that leverages a set of LLMs. By intelligently routing incoming queries to the most suitable model, ORI not only improves task-specific accuracy, but also maintains efficiency. Comprehensive evaluations across diverse benchmarks demonstrate consistent accuracy gains while controlling computational overhead. By intelligently routing queries, ORI outperforms the strongest individual models by up to 2.7 points on MMLU and 1.8 points on MuSR, ties the top performance on ARC, and on BBH. These results underscore the benefits of a multi-model strategy and demonstrate how ORI's adaptive architecture can more effectively handle diverse tasks, offering a scalable, high-performance solution for a system of multiple large language models.

Title: MTLM: an Innovative Language Model Training Paradigm for ASR

• Authors: Qingliang Meng, Pengju Ren, Tian Li, Changsong Dai
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.10058
• Pdf URL: https://arxiv.org/pdf/2502.10058
• Copy Paste: [[2502.10058]] MTLM: an Innovative Language Model Training Paradigm for ASR(https://arxiv.org/abs/2502.10058)
• Keywords: transformer
• Abstract: Pre-training Transformer-based language models (LMs) on a large amount of text has proven crucial for improving automatic speech recognition (ASR) performance. Generally, traditional LMs are unidirectional and unable to access the context on the right. This paper proposes a method for training LMs that enable traditional unidirectional LMs to fully utilize left and right contexts. Compared with the unidirectional LMs, our LM facilitates ASR to transcribe hypotheses more consistently and in a more semantically unambiguous way, as it incorporates richer contextual representations. Finally, our experimental results on the LibriSpeech corpus demonstrate that our model outperforms traditional unidirectional LMs, whether n-best rescoring or shallow fusion is used as the decoding algorithm.

Title: RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control

• Authors: Teng Li, Guangcong Zheng, Rui Jiang, Shuigenzhan, Tao Wu, Yehao Lu, Yining Lin, Xi Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10059
• Pdf URL: https://arxiv.org/pdf/2502.10059
• Copy Paste: [[2502.10059]] RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control(https://arxiv.org/abs/2502.10059)
• Keywords: diffusion, generative
• Abstract: Recent advancements in camera-trajectory-guided image-to-video generation offer higher precision and better support for complex camera control compared to text-based approaches. However, they also introduce significant usability challenges, as users often struggle to provide precise camera parameters when working with arbitrary real-world images without knowledge of their depth nor scene scale. To address these real-world application issues, we propose RealCam-I2V, a novel diffusion-based video generation framework that integrates monocular metric depth estimation to establish 3D scene reconstruction in a preprocessing step. During training, the reconstructed 3D scene enables scaling camera parameters from relative to absolute values, ensuring compatibility and scale consistency across diverse real-world images. In inference, RealCam-I2V offers an intuitive interface where users can precisely draw camera trajectories by dragging within the 3D scene. To further enhance precise camera control and scene consistency, we propose scene-constrained noise shaping, which shapes high-level noise and also allows the framework to maintain dynamic, coherent video generation in lower noise stages. RealCam-I2V achieves significant improvements in controllability and video quality on the RealEstate10K and out-of-domain images. We further enables applications like camera-controlled looping video generation and generative frame interpolation. We will release our absolute-scale annotation, codes, and all checkpoints. Please see dynamic results in this https URL.

Title: DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery

• Authors: Utkarsh Mall, Cheng Perng Phoo, Mia Chiquier, Bharath Hariharan, Kavita Bala, Carl Vondrick
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10060
• Pdf URL: https://arxiv.org/pdf/2502.10060
• Copy Paste: [[2502.10060]] DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery(https://arxiv.org/abs/2502.10060)
• Keywords: large language model
• Abstract: Visual data is used in numerous different scientific workflows ranging from remote sensing to ecology. As the amount of observation data increases, the challenge is not just to make accurate predictions but also to understand the underlying mechanisms for those predictions. Good interpretation is important in scientific workflows, as it allows for better decision-making by providing insights into the data. This paper introduces an automatic way of obtaining such interpretable-by-design models, by learning programs that interleave neural networks. We propose DiSciPLE (Discovering Scientific Programs using LLMs and Evolution) an evolutionary algorithm that leverages common sense and prior knowledge of large language models (LLMs) to create Python programs explaining visual data. Additionally, we propose two improvements: a program critic and a program simplifier to improve our method further to synthesize good programs. On three different real-world problems, DiSciPLE learns state-of-the-art programs on novel tasks with no prior literature. For example, we can learn programs with 35% lower error than the closest non-interpretable baseline for population density estimation.

Title: A novel approach to data generation in generative model

• Authors: JaeHong Kim (1), Jaewon Shim (2) ((1) Healthcare, Legal and Policy Center, Graduate school of Law, Korea University, Seoul 02841, Korea, Human-Inspired AI Research, Korea University, Seoul 02841, Korea , (2) Center for 0D Nanofluidics, Institute of Applied Physics, Department of Physics and Astronomy, Seoul National University, Seoul 08826, Korea)
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10092
• Pdf URL: https://arxiv.org/pdf/2502.10092
• Copy Paste: [[2502.10092]] A novel approach to data generation in generative model(https://arxiv.org/abs/2502.10092)
• Keywords: generative, large language model
• Abstract: Variational Autoencoders (VAEs) and other generative models are widely employed in artificial intelligence to synthesize new data. However, current approaches rely on Euclidean geometric assumptions and statistical approximations that fail to capture the structured and emergent nature of data generation. This paper introduces the Convergent Fusion Paradigm (CFP) theory, a novel geometric framework that redefines data generation by integrating dimensional expansion accompanied by qualitative transformation. By modifying the latent space geometry to interact with emergent high-dimensional structures, CFP theory addresses key challenges such as identifiability issues and unintended artifacts like hallucinations in Large Language Models (LLMs). CFP theory is based on two key conceptual hypotheses that redefine how generative models structure relationships between data and algorithms. Through the lens of CFP theory, we critically examine existing metric-learning approaches. CFP theory advances this perspective by introducing time-reversed metric embeddings and structural convergence mechanisms, leading to a novel geometric approach that better accounts for data generation as a structured epistemic process. Beyond its computational implications, CFP theory provides philosophical insights into the ontological underpinnings of data generation. By offering a systematic framework for high-dimensional learning dynamics, CFP theory contributes to establishing a theoretical foundation for understanding the data-relationship structures in AI. Finally, future research in CFP theory will be led to its implications for fully realizing qualitative transformations, introducing the potential of Hilbert space in generative modeling.

Title: Representation Learning on Out of Distribution in Tabular Data

• Authors: Achmad Ginanjar, Xue Li, Priyanka Singh, Wen Hua
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10095
• Pdf URL: https://arxiv.org/pdf/2502.10095
• Copy Paste: [[2502.10095]] Representation Learning on Out of Distribution in Tabular Data(https://arxiv.org/abs/2502.10095)
• Keywords: transformer
• Abstract: The open-world assumption in model development suggests that a model might lack sufficient information to adequately handle data that is entirely distinct or out of distribution (OOD). While deep learning methods have shown promising results in handling OOD data through generalization techniques, they often require specialized hardware that may not be accessible to all users. We present TCL, a lightweight yet effective solution that operates efficiently on standard CPU hardware. Our approach adapts contrastive learning principles specifically for tabular data structures, incorporating full matrix augmentation and simplified loss calculation. Through comprehensive experiments across 10 diverse datasets, we demonstrate that TCL outperforms existing models, including FT-Transformer and ResNet, particularly in classification tasks, while maintaining competitive performance in regression problems. TCL achieves these results with significantly reduced computational requirements, making it accessible to users with limited hardware capabilities. This study also provides practical guidance for detecting and evaluating OOD data through straightforward experiments and visualizations. Our findings show that TCL offers a promising balance between performance and efficiency in handling OOD prediction tasks, which is particularly beneficial for general machine learning practitioners working with computational constraints.

Title: NeuroXVocal: Detection and Explanation of Alzheimer's Disease through Non-invasive Analysis of Picture-prompted Speech

• Authors: Nikolaos Ntampakis, Konstantinos Diamantaras, Ioanna Chouvarda, Magda Tsolaki, Vasileios Argyriou, Panagiotis Sarigianndis
• Subjects: cs.LG, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2502.10108
• Pdf URL: https://arxiv.org/pdf/2502.10108
• Copy Paste: [[2502.10108]] NeuroXVocal: Detection and Explanation of Alzheimer's Disease through Non-invasive Analysis of Picture-prompted Speech(https://arxiv.org/abs/2502.10108)
• Keywords: robust, explainability, transformer, large language model
• Abstract: The early diagnosis of Alzheimer's Disease (AD) through non invasive methods remains a significant healthcare challenge. We present NeuroXVocal, a novel dual-component system that not only classifies but also explains potential AD cases through speech analysis. The classification component (Neuro) processes three distinct data streams: acoustic features capturing speech patterns and voice characteristics, textual features extracted from speech transcriptions, and precomputed embeddings representing linguistic patterns. These streams are fused through a custom transformer-based architecture that enables robust cross-modal interactions. The explainability component (XVocal) implements a Retrieval-Augmented Generation (RAG) approach, leveraging Large Language Models combined with a domain-specific knowledge base of AD research literature. This architecture enables XVocal to retrieve relevant clinical studies and research findings to generate evidence-based context-sensitive explanations of the acoustic and linguistic markers identified in patient speech. Using the IS2021 ADReSSo Challenge benchmark dataset, our system achieved state-of-the-art performance with 95.77% accuracy in AD classification, significantly outperforming previous approaches. The explainability component was qualitatively evaluated using a structured questionnaire completed by medical professionals, validating its clinical relevance. NeuroXVocal's unique combination of high-accuracy classification and interpretable, literature-grounded explanations demonstrates its potential as a practical tool for supporting clinical AD diagnosis.

Title: ScamFerret: Detecting Scam Websites Autonomously with Large Language Models

• Authors: Hiroki Nakano, Takashi Koide, Daiki Chiba
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.10110
• Pdf URL: https://arxiv.org/pdf/2502.10110
• Copy Paste: [[2502.10110]] ScamFerret: Detecting Scam Websites Autonomously with Large Language Models(https://arxiv.org/abs/2502.10110)
• Keywords: security, large language model
• Abstract: With the rise of sophisticated scam websites that exploit human psychological vulnerabilities, distinguishing between legitimate and scam websites has become increasingly challenging. This paper presents ScamFerret, an innovative agent system employing a large language model (LLM) to autonomously collect and analyze data from a given URL to determine whether it is a scam. Unlike traditional machine learning models that require large datasets and feature engineering, ScamFerret leverages LLMs' natural language understanding to accurately identify scam websites of various types and languages without requiring additional training or fine-tuning. Our evaluation demonstrated that ScamFerret achieves 0.972 accuracy in classifying four scam types in English and 0.993 accuracy in classifying online shopping websites across three different languages, particularly when using GPT-4. Furthermore, we confirmed that ScamFerret collects and analyzes external information such as web content, DNS records, and user reviews as necessary, providing a basis for identifying scam websites from multiple perspectives. These results suggest that LLMs have significant potential in enhancing cybersecurity measures against sophisticated scam websites.

Title: COMBINEX: A Unified Counterfactual Explainer for Graph Neural Networks via Node Feature and Structural Perturbations

• Authors: Flavio Giorgi, Fabrizio Silvestri, Gabriele Tolomei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10111
• Pdf URL: https://arxiv.org/pdf/2502.10111
• Copy Paste: [[2502.10111]] COMBINEX: A Unified Counterfactual Explainer for Graph Neural Networks via Node Feature and Structural Perturbations(https://arxiv.org/abs/2502.10111)
• Keywords: robust
• Abstract: Counterfactual explanations have emerged as a powerful tool to unveil the opaque decision-making processes of graph neural networks (GNNs). However, existing techniques primarily focus on edge modifications, often overlooking the crucial role of node feature perturbations in shaping model predictions. To address this limitation, we propose COMBINEX, a novel GNN explainer that generates counterfactual explanations for both node and graph classification tasks. Unlike prior methods, which treat structural and feature-based changes independently, COMBINEX optimally balances modifications to edges and node features by jointly optimizing these perturbations. This unified approach ensures minimal yet effective changes required to flip a model's prediction, resulting in realistic and interpretable counterfactuals. Additionally, COMBINEX seamlessly handles both continuous and discrete node features, enhancing its versatility across diverse datasets and GNN architectures. Extensive experiments on real-world datasets and various GNN architectures demonstrate the effectiveness and robustness of our approach over existing baselines.

Title: Image Embedding Sampling Method for Diverse Captioning

• Authors: Sania Waheed, Na Min An
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10118
• Pdf URL: https://arxiv.org/pdf/2502.10118
• Copy Paste: [[2502.10118]] Image Embedding Sampling Method for Diverse Captioning(https://arxiv.org/abs/2502.10118)
• Keywords: segmentation
• Abstract: Image Captioning for state-of-the-art VLMs has significantly improved over time; however, this comes at the cost of increased computational complexity, making them less accessible for resource-constrained applications such as mobile devices and assistive technologies. Alternatively, smaller VLMs prioritize high-level scene descriptions, overlooking finer details that contribute to a richer understanding of an image. In this paper, we introduce a training-free framework that enhances caption diversity and informativeness by explicitly attending to distinct image regions using a comparably small VLM, BLIP, as the backbone. Our approach leverages structured segmentation to produce hierarchical representations that capture both global and localized semantics. Without requiring additional model training, we demonstrate that our method allows smaller VLMs to achieve performance comparable to larger models in terms of image-caption alignment, semantic integrity, and diversity. We evaluate our framework on MSCOCO, Flickr30k, and Nocaps test datasets, achieving a Div-2 score of 0.735, 0.750, and 0.748 for each dataset respectively, while maintaining strong image-caption relevancy and semantic integrity with the human-annotated captions.

Title: Compress image to patches for Vision Transformer

• Authors: Xinfeng Zhao, Yaoru Sun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10120
• Pdf URL: https://arxiv.org/pdf/2502.10120
• Copy Paste: [[2502.10120]] Compress image to patches for Vision Transformer(https://arxiv.org/abs/2502.10120)
• Keywords: transformer
• Abstract: The Vision Transformer (ViT) has made significant strides in the field of computer vision. However, as the depth of the model and the resolution of the input images increase, the computational cost associated with training and running ViT models has surged this http URL paper proposes a hybrid model based on CNN and Vision Transformer, named CI2P-ViT. The model incorporates a module called CI2P, which utilizes the CompressAI encoder to compress images and subsequently generates a sequence of patches through a series of convolutions. CI2P can replace the Patch Embedding component in the ViT model, enabling seamless integration into existing ViT this http URL to ViT-B/16, CI2P-ViT has the number of patches input to the self-attention layer reduced to a quarter of the this http URL design not only significantly reduces the computational cost of the ViT model but also effectively enhances the model's accuracy by introducing the inductive bias properties of this http URL ViT model's precision is markedly this http URL trained from the ground up on the Animals-10 dataset, CI2P-ViT achieved an accuracy rate of 92.37%, representing a 3.3% improvement over the ViT-B/16 baseline. Additionally, the model's computational operations, measured in floating-point operations per second (FLOPs), were diminished by 63.35%, and it exhibited a 2-fold increase in training velocity on identical hardware configurations.

Title: Modern Hopfield Networks with Continuous-Time Memories

• Authors: Saul Santos, António Farinhas, Daniel C. McNamee, André F.T. Martins
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10122
• Pdf URL: https://arxiv.org/pdf/2502.10122
• Copy Paste: [[2502.10122]] Modern Hopfield Networks with Continuous-Time Memories(https://arxiv.org/abs/2502.10122)
• Keywords: transformer
• Abstract: Recent research has established a connection between modern Hopfield networks (HNs) and transformer attention heads, with guarantees of exponential storage capacity. However, these models still face challenges scaling storage efficiently. Inspired by psychological theories of continuous neural resource allocation in working memory, we propose an approach that compresses large discrete Hopfield memories into smaller, continuous-time memories. Leveraging continuous attention, our new energy function modifies the update rule of HNs, replacing the traditional softmax-based probability mass function with a probability density, over the continuous memory. This formulation aligns with modern perspectives on human executive function, offering a principled link between attractor dynamics in working memory and resource-efficient memory allocation. Our framework maintains competitive performance with HNs while leveraging a compressed memory, reducing computational costs across synthetic and video datasets.

Title: Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages

• Authors: Daniil Gurgurov, Ivan Vykopal, Josef van Genabith, Simon Ostermann
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10140
• Pdf URL: https://arxiv.org/pdf/2502.10140
• Copy Paste: [[2502.10140]] Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages(https://arxiv.org/abs/2502.10140)
• Keywords: large language model
• Abstract: Low-resource languages (LRLs) face significant challenges in natural language processing (NLP) due to limited data. While current state-of-the-art large language models (LLMs) still struggle with LRLs, smaller multilingual models (mLMs) such as mBERT and XLM-R offer greater promise due to a better fit of their capacity to low training data sizes. This study systematically investigates parameter-efficient adapter-based methods for adapting mLMs to LRLs, evaluating three architectures: Sequential Bottleneck, Invertible Bottleneck, and Low-Rank Adaptation. Using unstructured text from GlotCC and structured knowledge from ConceptNet, we show that small adaptation datasets (e.g., up to 1 GB of free-text or a few MB of knowledge graph data) yield gains in intrinsic (masked language modeling) and extrinsic tasks (topic classification, sentiment analysis, and named entity recognition). We find that Sequential Bottleneck adapters excel in language modeling, while Invertible Bottleneck adapters slightly outperform other methods on downstream tasks due to better embedding alignment and larger parameter counts. Adapter-based methods match or outperform full fine-tuning while using far fewer parameters, and smaller mLMs prove more effective for LRLs than massive LLMs like LLaMA-3, GPT-4, and DeepSeek-R1-based distilled models. While adaptation improves performance, pre-training data size remains the dominant factor, especially for languages with extensive pre-training coverage.

Title: Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling

• Authors: Xinyu Li, Marwa Mahmoud
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.10145
• Pdf URL: https://arxiv.org/pdf/2502.10145
• Copy Paste: [[2502.10145]] Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling(https://arxiv.org/abs/2502.10145)
• Keywords: protect, biometric, interpretability, explainability
• Abstract: In the contemporary era of intelligent connectivity, Affective Computing (AC), which enables systems to recognize, interpret, and respond to human behavior states, has become an integrated part of many AI systems. As one of the most critical components of responsible AI and trustworthiness in all human-centered systems, explainability has been a major concern in AC. Particularly, the recently released EU General Data Protection Regulation requires any high-risk AI systems to be sufficiently interpretable, including biometric-based systems and emotion recognition systems widely used in the affective computing field. Existing explainable methods often compromise between interpretability and performance. Most of them focus only on highlighting key network parameters without offering meaningful, domain-specific explanations to the stakeholders. Additionally, they also face challenges in effectively co-learning and explaining insights from multimodal data sources. To address these limitations, we propose a novel and generalizable framework, namely the Attention-Guided Concept Model (AGCM), which provides learnable conceptual explanations by identifying what concepts that lead to the predictions and where they are observed. AGCM is extendable to any spatial and temporal signals through multimodal concept alignment and co-learning, empowering stakeholders with deeper insights into the model's decision-making process. We validate the efficiency of AGCM on well-established Facial Expression Recognition benchmark datasets while also demonstrating its generalizability on more complex real-world human behavior understanding applications.

Title: From Markov to Laplace: How Mamba In-Context Learns Markov Chains

• Authors: Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva
• Subjects: cs.LG, cs.AI, cs.IT
• Abstract URL: https://arxiv.org/abs/2502.10178
• Pdf URL: https://arxiv.org/pdf/2502.10178
• Copy Paste: [[2502.10178]] From Markov to Laplace: How Mamba In-Context Learns Markov Chains(https://arxiv.org/abs/2502.10178)
• Keywords: transformer
• Abstract: While transformer-based language models have driven the AI revolution thus far, their computational complexity has spurred growing interest in viable alternatives, such as structured state space sequence models (SSMs) and Selective SSMs. Among these, Mamba (S6) and its variant Mamba-2 have shown remarkable inference speed ups over transformers while achieving comparable or superior performance on complex language modeling tasks. However, despite these architectural innovations and empirical successes, the fundamental learning capabilities of Mamba remain poorly understood. In this paper, we address this gap by studying in-context learning (ICL) on Markov chains and uncovering a surprising phenomenon: unlike transformers, even a single-layer Mamba efficiently learns the in-context Laplacian smoothing estimator, which is both Bayes and minimax optimal, for all Markovian orders. To explain this, we theoretically characterize the representation capacity of Mamba and reveal the fundamental role of convolution in enabling it to represent the optimal Laplacian smoothing. These theoretical insights align strongly with empirical results and, to the best of our knowledge, represent the first formal connection between Mamba and optimal statistical estimators. Finally, we outline promising research directions inspired by these findings.

Title: Realistic Evaluation of Deep Partial-Label Learning Algorithms

• Authors: Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10184
• Pdf URL: https://arxiv.org/pdf/2502.10184
• Copy Paste: [[2502.10184]] Realistic Evaluation of Deep Partial-Label Learning Algorithms(https://arxiv.org/abs/2502.10184)
• Keywords: fair
• Abstract: Partial-label learning (PLL) is a weakly supervised learning problem in which each example is associated with multiple candidate labels and only one is the true label. In recent years, many deep PLL algorithms have been developed to improve model performance. However, we find that some early developed algorithms are often underestimated and can outperform many later algorithms with complicated designs. In this paper, we delve into the empirical perspective of PLL and identify several critical but previously overlooked issues. First, model selection for PLL is non-trivial, but has never been systematically studied. Second, the experimental settings are highly inconsistent, making it difficult to evaluate the effectiveness of the algorithms. Third, there is a lack of real-world image datasets that can be compatible with modern network architectures. Based on these findings, we propose PLENCH, the first Partial-Label learning bENCHmark to systematically compare state-of-the-art deep PLL algorithms. We investigate the model selection problem for PLL for the first time, and propose novel model selection criteria with theoretical guarantees. We also create Partial-Label CIFAR-10 (PLCIFAR10), an image dataset of human-annotated partial labels collected from Amazon Mechanical Turk, to provide a testbed for evaluating the performance of PLL algorithms in more realistic scenarios. Researchers can quickly and conveniently perform a comprehensive and fair evaluation and verify the effectiveness of newly developed algorithms based on PLENCH. We hope that PLENCH will facilitate standardized, fair, and practical evaluation of PLL algorithms in the future.

Title: Translating Common Security Assertions Across Processor Designs: A RISC-V Case Study

• Authors: Sharjeel Imtiaz, Uljana Reinsalu, Tara Ghasempouri
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2502.10194
• Pdf URL: https://arxiv.org/pdf/2502.10194
• Copy Paste: [[2502.10194]] Translating Common Security Assertions Across Processor Designs: A RISC-V Case Study(https://arxiv.org/abs/2502.10194)
• Keywords: security, attack, robust, large language model
• Abstract: RISC-V is gaining popularity for its adaptability and cost-effectiveness in processor design. With the increasing adoption of RISC-V, the importance of implementing robust security verification has grown significantly. In the state of the art, various approaches have been developed to strengthen the security verification process. Among these methods, assertion-based security verification has proven to be a promising approach for ensuring that security features are effectively met. To this end, some approaches manually define security assertions for processor designs; however, these manual methods require significant time, cost, and human expertise. Consequently, recent approaches focus on translating pre-defined security assertions from one design to another. Nonetheless, these methods are not primarily centered on processor security, particularly RISC-V. Furthermore, many of these approaches have not been validated against real-world attacks, such as hardware Trojans. In this work, we introduce a methodology for translating security assertions across processors with different architectures, using RISC-V as a case study. Our approach reduces time and cost compared to developing security assertions manually from the outset. Our methodology was applied to five critical security modules with assertion translation achieving nearly 100% success across all modules. These results validate the efficacy of our approach and highlight its potential for enhancing security verification in modern processor designs. The effectiveness of the translated assertions was rigorously tested against hardware Trojans defined by large language models (LLMs), demonstrating their reliability in detecting security breaches.

Title: Prediction hubs are context-informed frequent tokens in LLMs

• Authors: Beatrix M. G. Nielsen, Iuri Macocco, Marco Baroni
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10201
• Pdf URL: https://arxiv.org/pdf/2502.10201
• Copy Paste: [[2502.10201]] Prediction hubs are context-informed frequent tokens in LLMs(https://arxiv.org/abs/2502.10201)
• Keywords: large language model
• Abstract: Hubness, the tendency for few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to high-dimensional data, often negatively impacting distance-based analysis. As autoregressive large language models (LLMs) operate on high-dimensional representations, we ask whether they are also affected by hubness. We first show, theoretically, that the only representation comparison operation performed by LLMs, namely that between context and unembedding vectors to determine continuation probabilities, is not characterized by the concentration of distances phenomenon that typically causes the appeareance of nuisance hubness. We then empirically show that this comparison still leads to a high degree of hubness, but the hubs in this case do not constitute a disturbance. They are rather the result of context-modulated frequent tokens often appearing in the pool of likely candidates for next token prediction. On the other hand, when other distance computations involving LLM representations are performed, we do not have the same theoretical guarantees, and, indeed, we see nuisance hubs appear. In summary, our work highlights, on the one hand, how hubness, while omnipresent in high-dimensional spaces, is not always a negative property that needs to be mitigated, and, on the other hand, it shows that various widely-used LLMs have developed a guessing strategy that consists in constantly assigning a high probability to frequent tokens.

Title: Can Post-Training Quantization Benefit from an Additional QLoRA Integration?

• Authors: Xiliang Zhu, Elena Khasanova, Cheng Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10202
• Pdf URL: https://arxiv.org/pdf/2502.10202
• Copy Paste: [[2502.10202]] Can Post-Training Quantization Benefit from an Additional QLoRA Integration?(https://arxiv.org/abs/2502.10202)
• Keywords: large language model
• Abstract: Large language models (LLMs) have transformed natural language processing but pose significant challenges for real-world deployment. These models necessitate considerable computing resources, which can be costly and frequently unavailable. Model compression techniques such as quantization are often leveraged to alleviate resource demand, but they may have a negative impact on the generation quality. In this study, we explore the integration of 4-bit Post-training Quantization (PTQ) with QLoRA to address these issues. We demonstrate through extensive experiments that this integration outperforms standard PTQ, and in some cases even 16-bit full-parameter fine-tuning on LLMs, validated across proprietary and public datasets with different quantization algorithms. The results demonstrate the efficacy of PTQ-QLoRA integration, offering a viable solution for deploying powerful LLMs in resource-constrained environments without compromising on performance.

Title: Control-flow anomaly detection by process mining-based feature extraction and dimensionality reduction

• Authors: Francesco Vitale, Marco Pegoraro, Wil M. P. van der Aalst, Nicola Mazzocca
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10211
• Pdf URL: https://arxiv.org/pdf/2502.10211
• Copy Paste: [[2502.10211]] Control-flow anomaly detection by process mining-based feature extraction and dimensionality reduction(https://arxiv.org/abs/2502.10211)
• Keywords: extraction, explainability
• Abstract: The business processes of organizations may deviate from normal control flow due to disruptive anomalies, including unknown, skipped, and wrongly-ordered activities. To identify these control-flow anomalies, process mining can check control-flow correctness against a reference process model through conformance checking, an explainable set of algorithms that allows linking any deviations with model elements. However, the effectiveness of conformance checking-based techniques is negatively affected by noisy event data and low-quality process models. To address these shortcomings and support the development of competitive and explainable conformance checking-based techniques for control-flow anomaly detection, we propose a novel process mining-based feature extraction approach with alignment-based conformance checking. This variant aligns the deviating control flow with a reference process model; the resulting alignment can be inspected to extract additional statistics such as the number of times a given activity caused mismatches. We integrate this approach into a flexible and explainable framework for developing techniques for control-flow anomaly detection. The framework combines process mining-based feature extraction and dimensionality reduction to handle high-dimensional feature sets, achieve detection effectiveness, and support explainability. The results show that the framework techniques implementing our approach outperform the baseline conformance checking-based techniques while maintaining the explainable nature of conformance checking. We also provide an explanation of why existing conformance checking-based techniques may be ineffective.

Title: Mapping bathymetry of inland water bodies on the North Slope of Alaska with Landsat using Random Forest

• Authors: Mark L. Carroll (1), Margaret R. Wooten (2 and 3), Claire E. Simpson (4), Caleb S. Spradlin (1 and 5), Melanie J. Frost (1 and 5), Mariana Blanco-Rojas (1), Zachary W. Williams (1 and 5), Jordan A. Caraballo-Vega (1), Christopher S. R. Neigh (2) ((1) NASA Data Science Group, Goddard Space Flight Center, 8800 Greenbelt Rd. mail code 606.3 Greenbelt, MD 20771, USA, (2) NASA Biospheric Sciences Laboratory, Goddard Space Flight Center, 8800 Greenbelt Rd. mail code 618 Greenbelt, MD 20771, USA, (3) Science Systems and Applications Incorporated, 10210 Greenbelt Rd Suite 600 Lanham, MD 20706, USA, (4) Department of Geography, University of Colorado Boulder, Boulder, Colorado, 80309, USA, (5) ASRC Federal Goddard Space Flight Center, 8800 Greenbelt Rd. mail code 606.3 Greenbelt, MD 20771, USA)
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10214
• Pdf URL: https://arxiv.org/pdf/2502.10214
• Copy Paste: [[2502.10214]] Mapping bathymetry of inland water bodies on the North Slope of Alaska with Landsat using Random Forest(https://arxiv.org/abs/2502.10214)
• Keywords: robust
• Abstract: The North Slope of Alaska is dominated by small waterbodies that provide critical ecosystem services for local population and wildlife. Detailed information on the depth of the waterbodies is scarce due to the challenges with collecting such information. In this work we have trained a machine learning (Random Forest Regressor) model to predict depth from multispectral Landsat data in waterbodies across the North Slope of Alaska. The greatest challenge is the scarcity of in situ data, which is expensive and difficult to obtain, to train the model. We overcame this challenge by using modeled depth predictions from a prior study as synthetic training data to provide a more diverse training data pool for the Random Forest. The final Random Forest model was more robust than models trained directly on the in situ data and when applied to 208 Landsat 8 scenes from 2016 to 2018 yielded a map with an overall $r^{2}$ value of 0.76 on validation. The final map has been made available through the Oak Ridge National Laboratory Distribute Active Archive Center (ORNL-DAAC). This map represents a first of its kind regional assessment of waterbody depth with per pixel estimates of depth for the entire North Slope of Alaska.

Title: Forget the Data and Fine-Tuning! Just Fold the Network to Compress

• Authors: Dong Wang, Haris Šikić, Lothar Thiele, Olga Saukh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10216
• Pdf URL: https://arxiv.org/pdf/2502.10216
• Copy Paste: [[2502.10216]] Forget the Data and Fine-Tuning! Just Fold the Network to Compress(https://arxiv.org/abs/2502.10216)
• Keywords: data-free
• Abstract: We introduce model folding, a novel data-free model compression technique that merges structurally similar neurons across layers, significantly reducing the model size without the need for fine-tuning or access to training data. Unlike existing methods, model folding preserves data statistics during compression by leveraging k-means clustering, and using novel data-free techniques to prevent variance collapse or explosion. Our theoretical framework and experiments across standard benchmarks, including ResNet18 and LLaMA-7B, demonstrate that model folding achieves comparable performance to data-driven compression techniques and outperforms recently proposed data-free methods, especially at high sparsity levels. This approach is particularly effective for compressing large-scale models, making it suitable for deployment in resource-constrained environments.

Title: Comparison of Deep Recurrent Neural Networks and Bayesian Neural Networks for Detecting Electric Motor Damage Through Sound Signal Analysis

• Authors: Waldemar Bauer, Jerzy Baranowski
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10224
• Pdf URL: https://arxiv.org/pdf/2502.10224
• Copy Paste: [[2502.10224]] Comparison of Deep Recurrent Neural Networks and Bayesian Neural Networks for Detecting Electric Motor Damage Through Sound Signal Analysis(https://arxiv.org/abs/2502.10224)
• Keywords: robust
• Abstract: Fault detection in electric motors is a critical challenge in various industries, where failures can result in significant operational disruptions. This study investigates the use of Recurrent Neural Networks (RNNs) and Bayesian Neural Networks (BNNs) for diagnosing motor damage using acoustic signal analysis. A novel approach is proposed, leveraging frequency domain representation of sound signals for enhanced diagnostic accuracy. The architectures of both RNNs and BNNs are designed and evaluated on real-world acoustic data collected from household appliances using smartphones. Experimental results demonstrate that BNNs provide superior fault detection performance, particularly for imbalanced datasets, offering more robust and interpretable predictions compared to traditional methods. The findings suggest that BNNs, with their ability to incorporate uncertainty, are well-suited for industrial diagnostic applications. Further analysis and benchmarks are suggested to explore resource efficiency and classification capabilities of these architectures.

Title: Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control

• Authors: Thomas Jiralerspong, Berton Earnshaw, Jason Hartford, Yoshua Bengio, Luca Scimeca
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10236
• Pdf URL: https://arxiv.org/pdf/2502.10236
• Copy Paste: [[2502.10236]] Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control(https://arxiv.org/abs/2502.10236)
• Keywords: diffusion, generative
• Abstract: Diffusion Probabilistic Models (DPMs) are powerful generative models that have achieved unparalleled success in a number of generative tasks. In this work, we aim to build inductive biases into the training and sampling of diffusion models to better accommodate the target distribution of the data to model. For topologically structured data, we devise a frequency-based noising operator to purposefully manipulate, and set, these inductive biases. We first show that appropriate manipulations of the noising forward process can lead DPMs to focus on particular aspects of the distribution to learn. We show that different datasets necessitate different inductive biases, and that appropriate frequency-based noise control induces increased generative performance compared to standard diffusion. Finally, we demonstrate the possibility of ignoring information at particular frequencies while learning. We show this in an image corruption and recovery task, where we train a DPM to recover the original target distribution after severe noise corruption.

Title: Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices

• Authors: Mohamed Aboelenien Ahmed, Kilian Pfeiffer, Ramin Khalili, Heba Khdr, Jörg Henkel
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10239
• Pdf URL: https://arxiv.org/pdf/2502.10239
• Copy Paste: [[2502.10239]] Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices(https://arxiv.org/abs/2502.10239)
• Keywords: privacy, federate, large language model
• Abstract: Federated fine-tuning offers a promising approach for tuning Large Language Models (LLMs) on edge devices while preserving data privacy. However, fine-tuning these models on edge devices remains challenging due to high memory, communication, and computational demands. Zero-order optimization with task alignment provides a potential solution, enabling fine-tuning with inference-level memory requirements but requires a longer convergence time. In this paper, we propose Federated Split-Perturbation Zero-order Optimization (FedSPZO) that divides the network into two blocks, applying a different number of perturbations per block in a computationally effective way, achieving faster convergence. Our evaluation shows a $2.5 - 7\times $ reduction in computation overhead compared to zero-order state of the art techniques in federated learning.

Title: Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model

• Authors: Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, Yu Zhou, Deshan Sun, Deyu Zhou, Jian Zhou, Kaijun Tan, Kang An, Mei Chen, Wei Ji, Qiling Wu, Wen Sun, Xin Han, Yanan Wei, Zheng Ge, Aojie Li, Bin Wang, Bizhu Huang, Bo Wang, Brian Li, Changxing Miao, Chen Xu, Chenfei Wu, Chenguang Yu, Dapeng Shi, Dingyuan Hu, Enle Liu, Gang Yu, Ge Yang, Guanzhe Huang, Gulin Yan, Haiyang Feng, Hao Nie, Haonan Jia, Hanpeng Hu, Hanqi Chen, Haolong Yan, Heng Wang, Hongcheng Guo, Huilin Xiong, Huixin Xiong, Jiahao Gong, Jianchang Wu, Jiaoren Wu, Jie Wu, Jie Yang, Jiashuai Liu, Jiashuo Li, Jingyang Zhang, Junjing Guo, Junzhe Lin, Kaixiang Li, Lei Liu, Lei Xia, Liang Zhao, Liguo Tan, Liwen Huang, Liying Shi, Ming Li, Mingliang Li, Muhua Cheng, Na Wang, Qiaohui Chen, Qinglin He, Qiuyan Liang, Quan Sun, Ran Sun, Rui Wang, Shaoliang Pang, Shiliang Yang, Sitong Liu, Siqi Liu, Shuli Gao, Tiancheng Cao, Tianyu Wang, Weipeng Ming, Wenqing He, Xu Zhao, Xuelin Zhang, Xianfang Zeng, Xiaojia Liu, Xuan Yang, Yaqi Dai, Yanbo Yu, Yang Li, Yineng Deng, Yingming Wang, Yilei Wang, Yuanwei Lu, Yu Chen, Yu Luo, Yuchu Luo
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10248
• Pdf URL: https://arxiv.org/pdf/2502.10248
• Copy Paste: [[2502.10248]] Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model(https://arxiv.org/abs/2502.10248)
• Keywords: diffusion
• Abstract: We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model with 30B parameters and the ability to generate videos up to 204 frames in length. A deep compression Variational Autoencoder, Video-VAE, is designed for video generation tasks, achieving 16x16 spatial and 8x temporal compression ratios, while maintaining exceptional video reconstruction quality. User prompts are encoded using two bilingual text encoders to handle both English and Chinese. A DiT with 3D full attention is trained using Flow Matching and is employed to denoise input noise into latent frames. A video-based DPO approach, Video-DPO, is applied to reduce artifacts and improve the visual quality of the generated videos. We also detail our training strategies and share key observations and insights. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, Step-Video-T2V-Eval, demonstrating its state-of-the-art text-to-video quality when compared with both open-source and commercial engines. Additionally, we discuss the limitations of current diffusion-based model paradigm and outline future directions for video foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval available at this https URL. The online version can be accessed from this https URL as well. Our goal is to accelerate the innovation of video foundation models and empower video content creators.

Title: VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

• Authors: Gokul Karthik Kumar, Iheb Chaabane, Kebin Wu
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10250
• Pdf URL: https://arxiv.org/pdf/2502.10250
• Copy Paste: [[2502.10250]] VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models(https://arxiv.org/abs/2502.10250)
• Keywords: large language model
• Abstract: Vision-language models (VLMs) excel in various visual benchmarks but are often constrained by the lack of high-quality visual fine-tuning data. To address this challenge, we introduce VisCon-100K, a novel dataset derived from interleaved image-text web documents. Our approach transforms 45K web documents from the OBELICS dataset into 100K image conversation samples. We utilize GPT-4V to generate image-contextual captions and OpenChat 3.5 model to convert these captions into diverse free-form and multiple-choice question-answer pairs. Integrating this dataset for fine-tuning considerably enhances VLM performance across multiple benchmarks. Unlike methods that focus solely on fine-grained visual content, our approach leverages accompanying web context, yielding superior results. We also discover that a `leaky modality mix,' where conversation samples contain questions answerable from both the image and its contextual caption, outperforms non-leaky combinations of captions and Q\&A pairs. VisCon-100k dataset shows strong performance with two popular VLM approaches: text-only large language model (LLM) aligned with a vision encoder using image captions data (ShareGPT4V-7b) and multimodally pretrained LLM (IDEFICS2-8b) using interleaved image-text data. In addition to releasing the VisCon-100K dataset, we provide a contextual captioner trained on this dataset, facilitating scalable fine-tuning data generation for future research and open-source applications. Using the same pipeline, but substituting our trained contextual captioner for GPT-4V, we also release the larger VisCon-1M dataset.

Title: MITO: Enabling Non-Line-of-Sight Perception using Millimeter-waves through Real-World Datasets and Simulation Tools

• Authors: Laura Dodds, Tara Boroushaki, Fadel Adib
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10259
• Pdf URL: https://arxiv.org/pdf/2502.10259
• Copy Paste: [[2502.10259]] MITO: Enabling Non-Line-of-Sight Perception using Millimeter-waves through Real-World Datasets and Simulation Tools(https://arxiv.org/abs/2502.10259)
• Keywords: segmentation
• Abstract: We present MITO, the first dataset of multi-spectral millimeter-wave (mmWave) images of everyday objects. Unlike visible light, mmWave signals can image through everyday occlusions (e.g., cardboard boxes, fabric, plastic). However, due to the dearth of publicly-available mmWave images and the interdisciplinary challenges in collecting and processing mmWave signals, it remains difficult today for computer vision researchers to develop mmWave-based non-line-of-sight perception algorithms and models. To overcome these challenges, we introduce a real-world dataset and open-source simulation tool for mmWave imaging. The dataset is acquired using a UR5 robotic arm with two mmWave radars operating at different frequencies and an RGB-D camera. Through a signal processing pipeline, we capture and create over 580 real-world 3D mmWave images from over 76 different objects in the YCB dataset, a standard dataset for robotics manipulation. We provide real-world mmWave images in line-of-sight and non-line-of-sight, as well as RGB-D images and ground truth segmentation masks. We also develop an open-source simulation tool that can be used to generate synthetic mmWave images for any 3D triangle mesh, which achieves a median F-Score of 94% when compared to real-world mmWave images. We show the usefulness of this dataset and simulation tool in multiple CV tasks in non-line-of-sight. First, we perform object segmentation for mmWave images using the segment anything model (SAM), and achieve a median precision and recall of 92.6% and 64%. Second, we train a classifier that can recognize objects in non-line-of-sight. It is trained on synthetic images and can classify real-world images with 85% accuracy. We believe MITO will be a valuable resource for computer vision researchers in developing non-line-of-sight perception, similar to how early camera-based datasets shaped the field.

Title: Large Language Models and Synthetic Data for Monitoring Dataset Mentions in Research Papers

• Authors: Aivin V. Solatorio, Rafael Macalaba, James Liounis
• Subjects: cs.CL, cs.AI, cs.CY, cs.DB, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10263
• Pdf URL: https://arxiv.org/pdf/2502.10263
• Copy Paste: [[2502.10263]] Large Language Models and Synthetic Data for Monitoring Dataset Mentions in Research Papers(https://arxiv.org/abs/2502.10263)
• Keywords: extraction, large language model
• Abstract: Tracking how data is mentioned and used in research papers provides critical insights for improving data discoverability, quality, and production. However, manually identifying and classifying dataset mentions across vast academic literature is resource-intensive and not scalable. This paper presents a machine learning framework that automates dataset mention detection across research domains by leveraging large language models (LLMs), synthetic data, and a two-stage fine-tuning process. We employ zero-shot extraction from research papers, an LLM-as-a-Judge for quality assessment, and a reasoning agent for refinement to generate a weakly supervised synthetic dataset. The Phi-3.5-mini instruct model is pre-fine-tuned on this dataset, followed by fine-tuning on a manually annotated subset. At inference, a ModernBERT-based classifier efficiently filters dataset mentions, reducing computational overhead while maintaining high recall. Evaluated on a held-out manually annotated sample, our fine-tuned model outperforms NuExtract-v1.5 and GLiNER-large-v2.1 in dataset extraction accuracy. Our results highlight how LLM-generated synthetic data can effectively address training data scarcity, improving generalization in low-resource settings. This framework offers a pathway toward scalable monitoring of dataset usage, enhancing transparency, and supporting researchers, funders, and policymakers in identifying data gaps and strengthening data accessibility for informed decision-making.

Title: Are Large Language Models the future crowd workers of Linguistics?

• Authors: Iris Ferrazzo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10266
• Pdf URL: https://arxiv.org/pdf/2502.10266
• Copy Paste: [[2502.10266]] Are Large Language Models the future crowd workers of Linguistics?(https://arxiv.org/abs/2502.10266)
• Keywords: large language model
• Abstract: Data elicitation from human participants is one of the core data collection strategies used in empirical linguistic research. The amount of participants in such studies may vary considerably, ranging from a handful to crowdsourcing dimensions. Even if they provide resourceful extensive data, both of these settings come alongside many disadvantages, such as low control of participants' attention during task completion, precarious working conditions in crowdsourcing environments, and time-consuming experimental designs. For these reasons, this research aims to answer the question of whether Large Language Models (LLMs) may overcome those obstacles if included in empirical linguistic pipelines. Two reproduction case studies are conducted to gain clarity into this matter: Cruz (2023) and Lombard et al. (2021). The two forced elicitation tasks, originally designed for human participants, are reproduced in the proposed framework with the help of OpenAI's GPT-4o-mini model. Its performance with our zero-shot prompting baseline shows the effectiveness and high versatility of LLMs, that tend to outperform human informants in linguistic tasks. The findings of the second replication further highlight the need to explore additional prompting techniques, such as Chain-of-Thought (CoT) prompting, which, in a second follow-up experiment, demonstrates higher alignment to human performance on both critical and filler items. Given the limited scale of this study, it is worthwhile to further explore the performance of LLMs in empirical Linguistics and in other future applications in the humanities.

Title: Artificial Intelligence to Assess Dental Findings from Panoramic Radiographs -- A Multinational Study

• Authors: Yin-Chih Chelsea Wang, Tsao-Lun Chen, Shankeeth Vinayahalingam, Tai-Hsien Wu, Chu Wei Chang, Hsuan Hao Chang, Hung-Jen Wei, Mu-Hsiung Chen, Ching-Chang Ko, David Anssari Moin, Bram van Ginneken, Tong Xi, Hsiao-Cheng Tsai, Min-Huey Chen, Tzu-Ming Harry Hsu, Hye Chou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10277
• Pdf URL: https://arxiv.org/pdf/2502.10277
• Copy Paste: [[2502.10277]] Artificial Intelligence to Assess Dental Findings from Panoramic Radiographs -- A Multinational Study(https://arxiv.org/abs/2502.10277)
• Keywords: robust, segmentation
• Abstract: Dental panoramic radiographs (DPRs) are widely used in clinical practice for comprehensive oral assessment but present challenges due to overlapping structures and time constraints in interpretation. This study aimed to establish a solid baseline for the AI-automated assessment of findings in DPRs by developing, evaluating an AI system, and comparing its performance with that of human readers across multinational data sets. We analyzed 6,669 DPRs from three data sets (the Netherlands, Brazil, and Taiwan), focusing on 8 types of dental findings. The AI system combined object detection and semantic segmentation techniques for per-tooth finding identification. Performance metrics included sensitivity, specificity, and area under the receiver operating characteristic curve (AUC-ROC). AI generalizability was tested across data sets, and performance was compared with human dental practitioners. The AI system demonstrated comparable or superior performance to human readers, particularly +67.9% (95% CI: 54.0%-81.9%; p < .001) sensitivity for identifying periapical radiolucencies and +4.7% (95% CI: 1.4%-8.0%; p = .008) sensitivity for identifying missing teeth. The AI achieved a macro-averaged AUC-ROC of 96.2% (95% CI: 94.6%-97.8%) across 8 findings. AI agreements with the reference were comparable to inter-human agreements in 7 of 8 findings except for caries (p = .024). The AI system demonstrated robust generalization across diverse imaging and demographic settings and processed images 79 times faster (95% CI: 75-82) than human readers. The AI system effectively assessed findings in DPRs, achieving performance on par with or better than human experts while significantly reducing interpretation time. These results highlight the potential for integrating AI into clinical workflows to improve diagnostic efficiency and accuracy, and patient management.

Title: Probabilistic Super-Resolution for High-Fidelity Physical System Simulations with Uncertainty Quantification

• Authors: Pengyu Zhang, Connor Duffin, Alex Glyn-Davies, Arnaud Vadeboncoeur, Mark Girolami
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10280
• Pdf URL: https://arxiv.org/pdf/2502.10280
• Copy Paste: [[2502.10280]] Probabilistic Super-Resolution for High-Fidelity Physical System Simulations with Uncertainty Quantification(https://arxiv.org/abs/2502.10280)
• Keywords: generative
• Abstract: Super-resolution (SR) is a promising tool for generating high-fidelity simulations of physical systems from low-resolution data, enabling fast and accurate predictions in engineering applications. However, existing deep-learning based SR methods, require large labeled datasets and lack reliable uncertainty quantification (UQ), limiting their applicability in real-world scenarios. To overcome these challenges, we propose a probabilistic SR framework that leverages the Statistical Finite Element Method and energy-based generative modeling. Our method enables efficient high-resolution predictions with inherent UQ, while eliminating the need for extensive labeled datasets. The method is validated on a 2D Poisson example and compared with bicubic interpolation upscaling. Results demonstrate a computational speed-up over high-resolution numerical solvers while providing reliable uncertainty estimates.

Title: TrustZero - open, verifiable and scalable zero-trust

• Authors: Adrian-Tudor Dumitrescu, Johan Pouwelse
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2502.10281
• Pdf URL: https://arxiv.org/pdf/2502.10281
• Copy Paste: [[2502.10281]] TrustZero - open, verifiable and scalable zero-trust(https://arxiv.org/abs/2502.10281)
• Keywords: secure, security, attack, robust
• Abstract: We present a passport-level trust token for Europe. In an era of escalating cyber threats fueled by global competition in economic, military, and technological domains, traditional security models are proving inadequate. The rise of advanced attacks exploiting zero-day vulnerabilities, supply chain infiltration, and system interdependencies underscores the need for a paradigm shift in cybersecurity. Zero Trust Architecture (ZTA) emerges as a transformative framework that replaces implicit trust with continuous verification of identity and granular access control. This thesis introduces TrustZero, a scalable layer of zero-trust security built around a universal "trust token" - a non-revocable self-sovereign identity with cryptographic signatures to enable robust, mathematically grounded trust attestations. By integrating ZTA principles with cryptography, TrustZero establishes a secure web-of-trust framework adaptable to legacy systems and inter-organisational communication.

Title: Anomaly Detection with LWE Encrypted Control

• Authors: Rijad Alisic, Junsoo Kim, Henrik Sandberg
• Subjects: cs.CR, eess.SY
• Abstract URL: https://arxiv.org/abs/2502.10283
• Pdf URL: https://arxiv.org/pdf/2502.10283
• Copy Paste: [[2502.10283]] Anomaly Detection with LWE Encrypted Control(https://arxiv.org/abs/2502.10283)
• Keywords: secure, attack
• Abstract: Detecting attacks using encrypted signals is challenging since encryption hides its information content. We present a novel mechanism for anomaly detection over Learning with Errors (LWE) encrypted signals without using decryption, secure channels, nor complex communication schemes. Instead, the detector exploits the homomorphic property of LWE encryption to perform hypothesis tests on transformations of the encrypted samples. The specific transformations are determined by solutions to a hard lattice-based minimization problem. While the test's sensitivity deteriorates with suboptimal solutions, similar to the exponential deterioration of the (related) test that breaks the cryptosystem, we show that the deterioration is polynomial for our test. This rate gap can be exploited to pick parameters that lead to somewhat weaker encryption but large gains in detection capability. Finally, we conclude the paper by presenting a numerical example that simulates anomaly detection, demonstrating the effectiveness of our method in identifying attacks.

Title: Adversarial Mixup Unlearning

• Authors: Zhuoyi Peng, Yixuan Tang, Yi Yang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10288
• Pdf URL: https://arxiv.org/pdf/2502.10288
• Copy Paste: [[2502.10288]] Adversarial Mixup Unlearning(https://arxiv.org/abs/2502.10288)
• Keywords: privacy, robust
• Abstract: Machine unlearning is a critical area of research aimed at safeguarding data privacy by enabling the removal of sensitive information from machine learning models. One unique challenge in this field is catastrophic unlearning, where erasing specific data from a well-trained model unintentionally removes essential knowledge, causing the model to deviate significantly from a retrained one. To address this, we introduce a novel approach that regularizes the unlearning process by utilizing synthesized mixup samples, which simulate the data susceptible to catastrophic effects. At the core of our approach is a generator-unlearner framework, MixUnlearn, where a generator adversarially produces challenging mixup examples, and the unlearner effectively forgets target information based on these synthesized data. Specifically, we first introduce a novel contrastive objective to train the generator in an adversarial direction: generating examples that prompt the unlearner to reveal information that should be forgotten, while losing essential knowledge. Then the unlearner, guided by two other contrastive loss terms, processes the synthesized and real data jointly to ensure accurate unlearning without losing critical knowledge, overcoming catastrophic effects. Extensive evaluations across benchmark datasets demonstrate that our method significantly outperforms state-of-the-art approaches, offering a robust solution to machine unlearning. This work not only deepens understanding of unlearning mechanisms but also lays the foundation for effective machine unlearning with mixup augmentation.

Title: Small Loss Bounds for Online Learning Separated Function Classes: A Gaussian Process Perspective

• Authors: Adam Block, Abhishek Shetty
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10292
• Pdf URL: https://arxiv.org/pdf/2502.10292
• Copy Paste: [[2502.10292]] Small Loss Bounds for Online Learning Separated Function Classes: A Gaussian Process Perspective(https://arxiv.org/abs/2502.10292)
• Keywords: privacy
• Abstract: In order to develop practical and efficient algorithms while circumventing overly pessimistic computational lower bounds, recent work has been interested in developing oracle-efficient algorithms in a variety of learning settings. Two such settings of particular interest are online and differentially private learning. While seemingly different, these two fields are fundamentally connected by the requirement that successful algorithms in each case satisfy stability guarantees; in particular, recent work has demonstrated that algorithms for online learning whose performance adapts to beneficial problem instances, attaining the so-called small-loss bounds, require a form of stability similar to that of differential privacy. In this work, we identify the crucial role that separation plays in allowing oracle-efficient algorithms to achieve this strong stability. Our notion, which we term $\rho$-separation, generalizes and unifies several previous approaches to enforcing this strong stability, including the existence of small-separator sets and the recent notion of $\gamma$-approximability. We present an oracle-efficient algorithm that is capable of achieving small-loss bounds with improved rates in greater generality than previous work, as well as a variant for differentially private learning that attains optimal rates, again under our separation condition. In so doing, we prove a new stability result for minimizers of a Gaussian process that strengthens and generalizes previous work.

Title: A Roadmap to Address Burnout in the Cybersecurity Profession: Outcomes from a Multifaceted Workshop

• Authors: Ann Rangarajan, Calvin Nobles, Josiah Dykstra, Margaret Cunningham, Nikki Robinson, Tammie Hollis, Celeste Lyn Paul, Charles Gulotta
• Subjects: cs.CR, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.10293
• Pdf URL: https://arxiv.org/pdf/2502.10293
• Copy Paste: [[2502.10293]] A Roadmap to Address Burnout in the Cybersecurity Profession: Outcomes from a Multifaceted Workshop(https://arxiv.org/abs/2502.10293)
• Keywords: security, defense
• Abstract: This paper addresses the critical issue of burnout among cybersecurity professionals, a growing concern that threatens the effectiveness of digital defense systems. As the industry faces a significant attrition crisis, with nearly 46% of cybersecurity leaders contemplating departure from their roles, it is imperative to explore the causes and consequences of burnout through a socio-technical lens. These challenges were discussed by experts from academia and industry in a multi-disciplinary workshop at the 26th International Conference on Human-Computer Interaction to address broad antecedents of burnout, manifestation and its consequences among cybersecurity professionals, as well as programs to mitigate impacts from burnout. Central to the analysis is an empirical study of former National Security Agency (NSA) tactical cyber operators. This paper presents key insights in the following areas based on discussions in the workshop: lessons for public and private sectors from the NSA study, a comparative review of addressing burnout in the healthcare profession. It also outlines a roadmap for future collaborative research, thereby informing interdisciplinary studies.

Title: QMaxViT-Unet+: A Query-Based MaxViT-Unet with Edge Enhancement for Scribble-Supervised Segmentation of Medical Images

• Authors: Thien B. Nguyen-Tat, Hoang-An Vo, Phuoc-Sang Dang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10294
• Pdf URL: https://arxiv.org/pdf/2502.10294
• Copy Paste: [[2502.10294]] QMaxViT-Unet+: A Query-Based MaxViT-Unet with Edge Enhancement for Scribble-Supervised Segmentation of Medical Images(https://arxiv.org/abs/2502.10294)
• Keywords: robust, transformer, segmentation
• Abstract: The deployment of advanced deep learning models for medical image segmentation is often constrained by the requirement for extensively annotated datasets. Weakly-supervised learning, which allows less precise labels, has become a promising solution to this challenge. Building on this approach, we propose QMaxViT-Unet+, a novel framework for scribble-supervised medical image segmentation. This framework is built on the U-Net architecture, with the encoder and decoder replaced by Multi-Axis Vision Transformer (MaxViT) blocks. These blocks enhance the model's ability to learn local and global features efficiently. Additionally, our approach integrates a query-based Transformer decoder to refine features and an edge enhancement module to compensate for the limited boundary information in the scribble label. We evaluate the proposed QMaxViT-Unet+ on four public datasets focused on cardiac structures, colorectal polyps, and breast cancer: ACDC, MS-CMRSeg, SUN-SEG, and BUSI. Evaluation metrics include the Dice similarity coefficient (DSC) and the 95th percentile of Hausdorff distance (HD95). Experimental results show that QMaxViT-Unet+ achieves 89.1\% DSC and 1.316mm HD95 on ACDC, 88.4\% DSC and 2.226mm HD95 on MS-CMRSeg, 71.4\% DSC and 4.996mm HD95 on SUN-SEG, and 69.4\% DSC and 50.122mm HD95 on BUSI. These results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency while remaining competitive with fully-supervised learning approaches. This makes it ideal for medical image analysis, where high-quality annotations are often scarce and require significant effort and expense. The code is available at: this https URL

Title: DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders

• Authors: Julien Siems, Timur Carstensen, Arber Zela, Frank Hutter, Massimiliano Pontil, Riccardo Grazzi
• Subjects: cs.LG, cs.CL, cs.FL
• Abstract URL: https://arxiv.org/abs/2502.10297
• Pdf URL: https://arxiv.org/pdf/2502.10297
• Copy Paste: [[2502.10297]] DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders(https://arxiv.org/abs/2502.10297)
• Keywords: transformer
• Abstract: Linear Recurrent Neural Networks (linear RNNs) have emerged as competitive alternatives to Transformers for sequence modeling, offering efficient training and linear-time inference. However, existing architectures face a fundamental trade-off between expressivity and efficiency, dictated by the structure of their state-transition matrices. While diagonal matrices used in architectures like Mamba, GLA, or mLSTM yield fast runtime, they suffer from severely limited expressivity. To address this, recent architectures such as (Gated) DeltaNet and RWKVv7 adopted a diagonal plus rank-1 structure, allowing simultaneous token-channel mixing, which overcomes some expressivity limitations with only a slight decrease in training efficiency. Building on the interpretation of DeltaNet's recurrence as performing one step of online gradient descent per token on an associative recall loss, we introduce DeltaProduct, which instead takes multiple ($n_h$) steps per token. This naturally leads to diagonal plus rank-$n_h$ state-transition matrices, formed as products of $n_h$ generalized Householder transformations, providing a tunable mechanism to balance expressivity and efficiency and a stable recurrence. Through extensive experiments, we demonstrate that DeltaProduct achieves superior state-tracking and language modeling capabilities while exhibiting significantly improved length extrapolation compared to DeltaNet. Additionally, we also strengthen the theoretical foundation of DeltaNet's expressivity by proving that it can solve dihedral group word problems in just two layers.

Title: ExplainReduce: Summarising local explanations via proxies

• Authors: Lauri Seppäläinen, Mudong Guo, Kai Puolamäki
• Subjects: cs.LG, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.10311
• Pdf URL: https://arxiv.org/pdf/2502.10311
• Copy Paste: [[2502.10311]] ExplainReduce: Summarising local explanations via proxies(https://arxiv.org/abs/2502.10311)
• Keywords: generative
• Abstract: Most commonly used non-linear machine learning methods are closed-box models, uninterpretable to humans. The field of explainable artificial intelligence (XAI) aims to develop tools to examine the inner workings of these closed boxes. An often-used model-agnostic approach to XAI involves using simple models as local approximations to produce so-called local explanations; examples of this approach include LIME, SHAP, and SLISEMAP. This paper shows how a large set of local explanations can be reduced to a small "proxy set" of simple models, which can act as a generative global explanation. This reduction procedure, ExplainReduce, can be formulated as an optimisation problem and approximated efficiently using greedy heuristics.

Title: Dynamic Fraud Proof

• Authors: Gabriele Picco, Andrea Fortugno
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2502.10321
• Pdf URL: https://arxiv.org/pdf/2502.10321
• Copy Paste: [[2502.10321]] Dynamic Fraud Proof(https://arxiv.org/abs/2502.10321)
• Keywords: security, attack
• Abstract: In this paper, we present a novel fraud-proof mechanism that achieves fast finality and, when combined with optimistic execution, enables real-time transaction processing. State-of-the-art optimistic rollups typically adopt a 7-day challenge window, during which any honest party can raise a challenge in case of fraud. We propose a new assert/challenge construction called "Dynamic Fraud Proofs" that achieves sub-second finality in ideal scenarios, while dynamically delaying settlement in the event of fraud detection and challenge resolution. The system relies on 1) a dynamic challenge period and 2) a configurable number of randomly selected verifier nodes who must interactively approve a state commitment without raising a challenge. If these conditions are not met, the state is not finalized, and the challenge period and approval criteria are dynamically adjusted. We provide a detailed analysis of the system's design, explaining how it maintains the assumption of a single honest node and addresses censorship attacks by inverting the traditional challenge process. Additionally, we formalize the system's probabilistic security model and discuss how bonding, incentives, and slashing mechanisms can encourage honest behavior, thereby increasing the likelihood of fast settlement in ideal scenarios.

Title: DiOpt: Self-supervised Diffusion for Constrained Optimization

• Authors: Shutong Ding, Yimiao Zhou, Ke Hu, Xi Yao, Junchi Yan, Xiaoying Tang, Ye Shi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10330
• Pdf URL: https://arxiv.org/pdf/2502.10330
• Copy Paste: [[2502.10330]] DiOpt: Self-supervised Diffusion for Constrained Optimization(https://arxiv.org/abs/2502.10330)
• Keywords: diffusion
• Abstract: Recent advances in diffusion models show promising potential for learning-based optimization by leveraging their multimodal sampling capability to escape local optima. However, existing diffusion-based optimization approaches, often reliant on supervised training, lacks a mechanism to ensure strict constraint satisfaction which is often required in real-world applications. One resulting observation is the distributional misalignment, i.e. the generated solution distribution often exhibits small overlap with the feasible domain. In this paper, we propose DiOpt, a novel diffusion paradigm that systematically learns near-optimal feasible solution distributions through iterative self-training. Our framework introduces several key innovations: a target distribution specifically designed to maximize overlap with the constrained solution manifold; a bootstrapped self-training mechanism that adaptively weights candidate solutions based on the severity of constraint violations and optimality gaps; and a dynamic memory buffer that accelerates convergence by retaining high-quality solutions over training iterations. To our knowledge, DiOpt represents the first successful integration of self-supervised diffusion with hard constraint satisfaction. Evaluations on diverse tasks, including power grid control, motion retargeting, wireless allocation demonstrate its superiority in terms of both optimality and constraint satisfaction.

Title: Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network

• Authors: Arun Kunwar, Dibakar Raj Pant, Jukka Heikkonen, Rajeev Kanth
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10334
• Pdf URL: https://arxiv.org/pdf/2502.10334
• Copy Paste: [[2502.10334]] Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network(https://arxiv.org/abs/2502.10334)
• Keywords: generative
• Abstract: The Convolutional Neural Network (CNN) has shown impressive performance in image classification because of its strong learning capabilities. However, it demands a substantial and balanced dataset for effective training. Otherwise, networks frequently exhibit over fitting and struggle to generalize to new examples. Publicly available dataset of fundus images of ocular disease is insufficient to train any classification model to achieve satisfactory accuracy. So, we propose Generative Adversarial Network(GAN) based data generation technique to synthesize dataset for training CNN based classification model and later use original disease containing ocular images to test the model. During testing the model classification accuracy with the original ocular image, the model achieves an accuracy rate of 78.6% for myopia, 88.6% for glaucoma, and 84.6% for cataract, with an overall classification accuracy of 84.6%.

Title: Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering

• Authors: Nick Ferguson, Liane Guillou, Alan Bundy, Kwabena Nuamah
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10338
• Pdf URL: https://arxiv.org/pdf/2502.10338
• Copy Paste: [[2502.10338]] Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering(https://arxiv.org/abs/2502.10338)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) excel in natural language tasks but still face challenges in Question Answering (QA) tasks requiring complex, multi-step reasoning. We outline the types of reasoning required in some of these tasks, and reframe them in terms of meta-level reasoning (akin to high-level strategic reasoning or planning) and object-level reasoning (embodied in lower-level tasks such as mathematical reasoning). Franklin, a novel dataset with requirements of meta- and object-level reasoning, is introduced and used along with three other datasets to evaluate four LLMs at question answering tasks requiring multiple steps of reasoning. Results from human annotation studies suggest LLMs demonstrate meta-level reasoning with high frequency, but struggle with object-level reasoning tasks in some of the datasets used. Additionally, evidence suggests that LLMs find the object-level reasoning required for the questions in the Franklin dataset challenging, yet they do exhibit strong performance with respect to the meta-level reasoning requirements.

Title: STAR: Spectral Truncation and Rescale for Model Merging

• Authors: Yu-Ang Lee, Ching-Yun Ko, Tejaswini Pedapati, I-Hsin Chung, Mi-Yen Yeh, Pin-Yu Chen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10339
• Pdf URL: https://arxiv.org/pdf/2502.10339
• Copy Paste: [[2502.10339]] STAR: Spectral Truncation and Rescale for Model Merging(https://arxiv.org/abs/2502.10339)
• Keywords: robust
• Abstract: Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning, and it has gained attention in various domains, including natural language processing (NLP). Despite the efficiency, a key challenge in model merging is the seemingly inevitable decrease in task performance as the number of models increases. In this paper, we propose $\mathbf{S}$pectral $\mathbf{T}$runcation $\mathbf{A}$nd $\mathbf{R}$escale (STAR) that aims at mitigating ``merging conflicts'' by truncating small components in the respective spectral spaces, which is followed by an automatic parameter rescaling scheme to retain the nuclear norm of the original matrix. STAR requires no additional inference on original training data and is robust to hyperparamater choice. We demonstrate the effectiveness of STAR through extensive model merging cases on diverse NLP tasks. Specifically, STAR works robustly across varying model sizes, and can outperform baselines by 4.2$\%$ when merging 12 models on Flan-T5. Our code is publicly available at this https URL.

Title: Organize the Web: Constructing Domains Enhances Pre-Training Data Curation

• Authors: Alexander Wettig, Kyle Lo, Sewon Min, Hannaneh Hajishirzi, Danqi Chen, Luca Soldaini
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10341
• Pdf URL: https://arxiv.org/pdf/2502.10341
• Copy Paste: [[2502.10341]] Organize the Web: Constructing Domains Enhances Pre-Training Data Curation(https://arxiv.org/abs/2502.10341)
• Keywords: large language model
• Abstract: Modern language models are trained on large, unstructured datasets consisting of trillions of tokens and obtained by crawling the web. The unstructured nature makes it difficult to reason about their contents and develop systematic approaches to data curation. In this paper, we unpack monolithic web corpora by developing taxonomies of their contents and organizing them into domains. We introduce WebOrganizer, a framework for organizing web pages in terms of both their topic and format. Using these two complementary notions of domains, we automatically annotate pre-training data by distilling annotations from a large language model into efficient classifiers. This allows us to study how data from different domains should be mixed to improve models on downstream tasks, and we show that we can combine insights about effective topics and formats to further boost performance. We demonstrate that our domain mixing also improves existing methods that select data based on quality. Furthermore, we study and compare how quality-based methods will implicitly change the domain mixture. Overall, our work demonstrates that constructing and mixing domains provides a valuable complement to quality-based data curation methods, opening new avenues for effective and insightful pre-training data curation.

Title: Agentic Verification for Ambiguous Query Disambiguation

• Authors: Youngwon Lee, Seung-won Hwang, Ruofan Wu, Feng Yan, Danmei Xu, Moutasem Akkad, Zhewei Yao, Yuxiong He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10352
• Pdf URL: https://arxiv.org/pdf/2502.10352
• Copy Paste: [[2502.10352]] Agentic Verification for Ambiguous Query Disambiguation(https://arxiv.org/abs/2502.10352)
• Keywords: robust
• Abstract: In this work, we tackle the challenge of disambiguating queries in retrieval-augmented generation (RAG) to diverse yet answerable interpretations. State-of-the-arts follow a Diversify-then-Verify (DtV) pipeline, where diverse interpretations are generated by an LLM, later used as search queries to retrieve supporting passages. Such a process may introduce noise in either interpretations or retrieval, particularly in enterprise settings, where LLMs -- trained on static data -- may struggle with domain-specific disambiguations. Thus, a post-hoc verification phase is introduced to prune noises. Our distinction is to unify diversification with verification by incorporating feedback from retriever and generator early on. This joint approach improves both efficiency and robustness by reducing reliance on multiple retrieval and inference steps, which are susceptible to cascading errors. We validate the efficiency and effectiveness of our method, Verified-Diversification with Consolidation (VERDICT), on the widely adopted ASQA benchmark to achieve diverse yet verifiable interpretations. Empirical results show that VERDICT improves grounding-aware F1 score by an average of 23% over the strongest baseline across different backbone LLMs.

Title: Dimension-free Score Matching and Time Bootstrapping for Diffusion Models

• Authors: Syamantak Kumar, Dheeraj Nagaraj, Purnamrita Sarkar
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10354
• Pdf URL: https://arxiv.org/pdf/2502.10354
• Copy Paste: [[2502.10354]] Dimension-free Score Matching and Time Bootstrapping for Diffusion Models(https://arxiv.org/abs/2502.10354)
• Keywords: diffusion, generative
• Abstract: Diffusion models generate samples by estimating the score function of the target distribution at various noise levels. The model is trained using samples drawn from the target distribution, progressively adding noise. In this work, we establish the first (nearly) dimension-free sample complexity bounds for learning these score functions, achieving a double exponential improvement in dimension over prior results. A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels, a critical feature of diffusion models in practice which enables generalization across timesteps. Our analysis introduces a novel martingale-based error decomposition and sharp variance bounds, enabling efficient learning from dependent data generated by Markov processes, which may be of independent interest. Building on these insights, we propose Bootstrapped Score Matching (BSM), a variance reduction technique that utilizes previously learned scores to improve accuracy at higher noise levels. These results provide crucial insights into the efficiency and effectiveness of diffusion models for generative modeling.

Title: Enhancing Multilingual LLM Pretraining with Model-Based Data Selection

• Authors: Bettina Messmer, Vinko Sabolčec, Martin Jaggi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10361
• Pdf URL: https://arxiv.org/pdf/2502.10361
• Copy Paste: [[2502.10361]] Enhancing Multilingual LLM Pretraining with Model-Based Data Selection(https://arxiv.org/abs/2502.10361)
• Keywords: transformer, large language model
• Abstract: Dataset curation has become a basis for strong large language model (LLM) performance. While various rule-based filtering heuristics exist for English and multilingual datasets, model-based filtering techniques have primarily focused on English. To address the disparity stemming from limited research on non-English languages, we propose a model-based filtering framework for multilingual datasets that aims to identify a diverse set of structured and knowledge-rich samples. Our approach emphasizes transparency, simplicity, and efficiency, leveraging Transformer- and FastText-based classifiers to ensure the broad accessibility of our technique and data. We conduct comprehensive ablation studies on the FineWeb-2 web crawl dataset across diverse language families, scripts, and resource availability to demonstrate the effectiveness of our method. Training a 1B-parameter Llama model for 70B and 119B tokens, our approach can match the baseline MMLU score with as little as 15% of the training tokens, while also improving across other benchmarks. These findings provide strong evidence for the generalizability of our approach to other languages. As a result, we extend our framework to 20 languages for which we release the refined pretraining datasets.

Title: AffinityFlow: Guided Flows for Antibody Affinity Maturation

• Authors: Can Chen, Karla-Luise Herpoldt, Chenchao Zhao, Zichen Wang, Marcus Collins, Shang Shang, Ron Benson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.10365
• Pdf URL: https://arxiv.org/pdf/2502.10365
• Copy Paste: [[2502.10365]] AffinityFlow: Guided Flows for Antibody Affinity Maturation(https://arxiv.org/abs/2502.10365)
• Keywords: generative
• Abstract: Antibodies are widely used as therapeutics, but their development requires costly affinity maturation, involving iterative mutations to enhance binding this http URL paper explores a sequence-only scenario for affinity maturation, using solely antibody and antigen sequences. Recently AlphaFlow wraps AlphaFold within flow matching to generate diverse protein structures, enabling a sequence-conditioned generative model of structure. Building on this, we propose an alternating optimization framework that (1) fixes the sequence to guide structure generation toward high binding affinity using a structure-based affinity predictor, then (2) applies inverse folding to create sequence mutations, refined by a sequence-based affinity predictor for post selection. To address this, we develop a co-teaching module that incorporates valuable information from noisy biophysical energies into predictor refinement. The sequence-based predictor selects consensus samples to teach the structure-based predictor, and vice versa. Our method, AffinityFlow, achieves state-of-the-art performance in affinity maturation experiments. We plan to open-source our code after acceptance.

Title: OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models

• Authors: William Chen, Jinchuan Tian, Yifan Peng, Brian Yan, Chao-Han Huck Yang, Shinji Watanabe
• Subjects: cs.CL, cs.AI, cs.LG, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.10373
• Pdf URL: https://arxiv.org/pdf/2502.10373
• Copy Paste: [[2502.10373]] OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models(https://arxiv.org/abs/2502.10373)
• Keywords: robust
• Abstract: Neural scaling laws offer valuable insights for designing robust sequence processing architectures. While these laws have been extensively characterized in other modalities, their behavior in speech remains comparatively underexplored. In this work, we introduce OWLS, an open-access, reproducible suite of multilingual speech recognition and translation models spanning 0.25B to 18B parameters, with the 18B version being the largest speech model, to the best of our knowledge. OWLS leverages up to 360K hours of public speech data across 150 languages, enabling a systematic investigation into how data, model, and compute scaling each influence performance in multilingual speech tasks. We use OWLS to derive neural scaling laws, showing how final performance can be reliably predicted when scaling. One of our key findings is that scaling enhances performance on low-resource languages/dialects, helping to mitigate bias and improve the accessibility of speech technologies. Finally, we show how OWLS can be used to power new research directions by discovering emergent abilities in large-scale speech models. Model checkpoints will be released on this https URL for future studies.

Title: ReStyle3D: Scene-Level Appearance Transfer with Semantic Correspondences

• Authors: Liyuan Zhu, Shengqu Cai, Shengyu Huang, Gordon Wetzstein, Naji Khosravan, Iro Armeni
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.10377
• Pdf URL: https://arxiv.org/pdf/2502.10377
• Copy Paste: [[2502.10377]] ReStyle3D: Scene-Level Appearance Transfer with Semantic Correspondences(https://arxiv.org/abs/2502.10377)
• Keywords: diffusion, segmentation
• Abstract: We introduce ReStyle3D, a novel framework for scene-level appearance transfer from a single style image to a real-world scene represented by multiple views. The method combines explicit semantic correspondences with multi-view consistency to achieve precise and coherent stylization. Unlike conventional stylization methods that apply a reference style globally, ReStyle3D uses open-vocabulary segmentation to establish dense, instance-level correspondences between the style and real-world images. This ensures that each object is stylized with semantically matched textures. It first transfers the style to a single view using a training-free semantic-attention mechanism in a diffusion model. It then lifts the stylization to additional views via a learned warp-and-refine network guided by monocular depth and pixel-wise correspondences. Experiments show that ReStyle3D consistently outperforms prior methods in structure preservation, perceptual style similarity, and multi-view coherence. User studies further validate its ability to produce photo-realistic, semantically faithful results. Our code, pretrained models, and dataset will be publicly released, to support new applications in interior design, virtual staging, and 3D-consistent stylization.

Title: Simplifying DINO via Coding Rate Regularization

• Authors: Ziyang Wu, Jingyuan Zhang, Druv Pai, XuDong Wang, Chandan Singh, Jianwei Yang, Jianfeng Gao, Yi Ma
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10385
• Pdf URL: https://arxiv.org/pdf/2502.10385
• Copy Paste: [[2502.10385]] Simplifying DINO via Coding Rate Regularization(https://arxiv.org/abs/2502.10385)
• Keywords: robust, segmentation
• Abstract: DINO and DINOv2 are two model families being widely used to learn representations from unlabeled imagery data at large scales. Their learned representations often enable state-of-the-art performance for downstream tasks, such as image classification and segmentation. However, they employ many empirically motivated design choices and their training pipelines are highly complex and unstable -- many hyperparameters need to be carefully tuned to ensure that the representations do not collapse -- which poses considerable difficulty to improving them or adapting them to new domains. In this work, we posit that we can remove most such-motivated idiosyncrasies in the pre-training pipelines, and only need to add an explicit coding rate term in the loss function to avoid collapse of the representations. As a result, we obtain highly simplified variants of the DINO and DINOv2 which we call SimDINO and SimDINOv2, respectively. Remarkably, these simplified models are more robust to different design choices, such as network architecture and hyperparameters, and they learn even higher-quality representations, measured by performance on downstream tasks, offering a Pareto improvement over the corresponding DINO and DINOv2 models. This work highlights the potential of using simplifying design principles to improve the empirical practice of deep learning.

Title: Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction

• Authors: WonJin Yoon, Boyu Ren, Spencer Thomas, Chanwhi Kim, Guergana Savova, Mei-Hua Hall, Timothy Miller
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.10388
• Pdf URL: https://arxiv.org/pdf/2502.10388
• Copy Paste: [[2502.10388]] Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction(https://arxiv.org/abs/2502.10388)
• Keywords: extraction, transformer, large language model
• Abstract: Recent progress in large language models (LLMs) has enabled the automated processing of lengthy documents even without supervised training on a task-specific dataset. Yet, their zero-shot performance in complex tasks as opposed to straightforward information extraction tasks remains suboptimal. One feasible approach for tasks with lengthy, complex input is to first summarize the document and then apply supervised fine-tuning to the summary. However, the summarization process inevitably results in some loss of information. In this study we present a method for processing the summaries of long documents aimed to capture different important aspects of the original document. We hypothesize that LLM summaries generated with different aspect-oriented prompts contain different \textit{information signals}, and we propose methods to measure these differences. We introduce approaches to effectively integrate signals from these different summaries for supervised training of transformer models. We validate our hypotheses on a high-impact task -- 30-day readmission prediction from a psychiatric discharge -- using real-world data from four hospitals, and show that our proposed method increases the prediction performance for the complex task of predicting patient outcome.

Title: Region-Adaptive Sampling for Diffusion Transformers

• Authors: Ziming Liu, Yifan Yang, Chengruidong Zhang, Yiqi Zhang, Lili Qiu, Yang You, Yuqing Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.10389
• Pdf URL: https://arxiv.org/pdf/2502.10389
• Copy Paste: [[2502.10389]] Region-Adaptive Sampling for Diffusion Transformers(https://arxiv.org/abs/2502.10389)
• Keywords: diffusion, transformer, generative
• Abstract: Diffusion models (DMs) have become the leading choice for generative tasks across diverse domains. However, their reliance on multiple sequential forward passes significantly limits real-time performance. Previous acceleration methods have primarily focused on reducing the number of sampling steps or reusing intermediate results, failing to leverage variations across spatial regions within the image due to the constraints of convolutional U-Net structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in handling variable number of tokens, we introduce RAS, a novel, training-free sampling strategy that dynamically assigns different sampling ratios to regions within an image based on the focus of the DiT model. Our key observation is that during each sampling step, the model concentrates on semantically meaningful regions, and these areas of focus exhibit strong continuity across consecutive steps. Leveraging this insight, RAS updates only the regions currently in focus, while other regions are updated using cached noise from the previous step. The model's focus is determined based on the output from the preceding step, capitalizing on the temporal consistency we observed. We evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up to 2.36x and 2.51x, respectively, with minimal degradation in generation quality. Additionally, a user study reveals that RAS delivers comparable qualities under human evaluation while achieving a 1.6x speedup. Our approach makes a significant step towards more efficient diffusion transformers, enhancing their potential for real-time applications.

Title: (How) Can Transformers Predict Pseudo-Random Numbers?

• Authors: Tao Tao, Darshil Doshi, Dayal Singh Kalra, Tianyu He, Maissam Barkeshli
• Subjects: cs.LG, cond-mat.dis-nn, cs.CR, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.10390
• Pdf URL: https://arxiv.org/pdf/2502.10390
• Copy Paste: [[2502.10390]] (How) Can Transformers Predict Pseudo-Random Numbers?(https://arxiv.org/abs/2502.10390)
• Keywords: transformer
• Abstract: Transformers excel at discovering patterns in sequential data, yet their fundamental limitations and learning mechanisms remain crucial topics of investigation. In this paper, we study the ability of Transformers to learn pseudo-random number sequences from linear congruential generators (LCGs), defined by the recurrence relation $x_{t+1} = a x_t + c \;\mathrm{mod}\; m$. Our analysis reveals that with sufficient architectural capacity and training data variety, Transformers can perform in-context prediction of LCG sequences with unseen moduli ($m$) and parameters ($a,c$). Through analysis of embedding layers and attention patterns, we uncover how Transformers develop algorithmic structures to learn these sequences in two scenarios of increasing complexity. First, we analyze how Transformers learn LCG sequences with unseen ($a, c$) but fixed modulus, and we demonstrate successful learning up to $m = 2^{32}$. Our analysis reveals that models learn to factorize the modulus and utilize digit-wise number representations to make sequential predictions. In the second, more challenging scenario of unseen moduli, we show that Transformers can generalize to unseen moduli up to $m_{\text{test}} = 2^{16}$. In this case, the model employs a two-step strategy: first estimating the unknown modulus from the context, then utilizing prime factorizations to generate predictions. For this task, we observe a sharp transition in the accuracy at a critical depth $=3$. We also find that the number of in-context sequence elements needed to reach high accuracy scales sublinearly with the modulus.

Title: MM-RLHF: The Next Step Forward in Multimodal LLM Alignment

• Authors: Yi-Fan Zhang, Tao Yu, Haochen Tian, Chaoyou Fu, Peiyan Li, Jianshu Zeng, Wulin Xie, Yang Shi, Huanyu Zhang, Junkang Wu, Xue Wang, Yibo Hu, Bin Wen, Fan Yang, Zhang Zhang, Tingting Gao, Di Zhang, Liang Wang, Rong Jin, Tieniu Tan
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.10391
• Pdf URL: https://arxiv.org/pdf/2502.10391
• Copy Paste: [[2502.10391]] MM-RLHF: The Next Step Forward in Multimodal LLM Alignment(https://arxiv.org/abs/2502.10391)
• Keywords: interpretability, large language model
• Abstract: Despite notable advancements in Multimodal Large Language Models (MLLMs), most state-of-the-art models have not undergone thorough alignment with human preferences. This gap exists because current alignment research has primarily achieved progress in specific areas (e.g., hallucination reduction), while the broader question of whether aligning models with human preferences can systematically enhance MLLM capability remains largely unexplored. To this end, we introduce MM-RLHF, a dataset containing $\mathbf{120k}$ fine-grained, human-annotated preference comparison pairs. This dataset represents a substantial advancement over existing resources, offering superior size, diversity, annotation granularity, and quality. Leveraging this dataset, we propose several key innovations to improve both the quality of reward models and the efficiency of alignment algorithms. Notably, we introduce a Critique-Based Reward Model, which generates critiques of model outputs before assigning scores, offering enhanced interpretability and more informative feedback compared to traditional scalar reward mechanisms. Additionally, we propose Dynamic Reward Scaling, a method that adjusts the loss weight of each sample according to the reward signal, thereby optimizing the use of high-quality comparison pairs. Our approach is rigorously evaluated across $\mathbf{10}$ distinct dimensions and $\mathbf{27}$ benchmarks, with results demonstrating significant and consistent improvements in model performance. Specifically, fine-tuning LLaVA-ov-7B with MM-RLHF and our alignment algorithm leads to a $\mathbf{19.5}$% increase in conversational abilities and a $\mathbf{60}$% improvement in safety. We have open-sourced the preference dataset, reward model, training and evaluation code, as well as reward modeling and safety benchmarks. For more details, please visit our project page: this https URL.