2025-02-28

Title: On the Interpolation Effect of Score Smoothing

• Authors: Zhengdao Chen
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.19499
• Pdf URL: https://arxiv.org/pdf/2502.19499
• Copy Paste: [[2502.19499]] On the Interpolation Effect of Score Smoothing(https://arxiv.org/abs/2502.19499)
• Keywords: diffusion
• Abstract: Score-based diffusion models have achieved remarkable progress in various domains with the ability to generate new data samples that do not exist in the training set. In this work, we examine the hypothesis that their generalization ability arises from an interpolation effect caused by a smoothing of the empirical score function. Focusing on settings where the training set lies uniformly in a one-dimensional linear subspace, we study the interplay between score smoothing and the denoising dynamics with mathematically solvable models. In particular, we demonstrate how a smoothed score function can lead to the generation of samples that interpolate among the training data within their subspace while avoiding full memorization. We also present evidence that learning score functions with regularized neural networks can have a similar effect on the denoising dynamics as score smoothing.

Title: Models That Are Interpretable But Not Transparent

• Authors: Chudi Zhong, Panyu Chen, Cynthia Rudin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19502
• Pdf URL: https://arxiv.org/pdf/2502.19502
• Copy Paste: [[2502.19502]] Models That Are Interpretable But Not Transparent(https://arxiv.org/abs/2502.19502)
• Keywords: defense, attack
• Abstract: Faithful explanations are essential for machine learning models in high-stakes applications. Inherently interpretable models are well-suited for these applications because they naturally provide faithful explanations by revealing their decision logic. However, model designers often need to keep these models proprietary to maintain their value. This creates a tension: we need models that are interpretable--allowing human decision-makers to understand and justify predictions, but not transparent, so that the model's decision boundary is not easily replicated by attackers. Shielding the model's decision boundary is particularly challenging alongside the requirement of completely faithful explanations, since such explanations reveal the true logic of the model for an entire subspace around each query point. This work provides an approach, FaithfulDefense, that creates model explanations for logical models that are completely faithful, yet reveal as little as possible about the decision boundary. FaithfulDefense is based on a maximum set cover formulation, and we provide multiple formulations for it, taking advantage of submodularity.

Title: Evaluating the Suitability of Different Intraoral Scan Resolutions for Deep Learning-Based Tooth Segmentation

• Authors: Daron Weekley, Jace Duckworth, Anastasiia Sukhanova, Ananya Jana
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19515
• Pdf URL: https://arxiv.org/pdf/2502.19515
• Copy Paste: [[2502.19515]] Evaluating the Suitability of Different Intraoral Scan Resolutions for Deep Learning-Based Tooth Segmentation(https://arxiv.org/abs/2502.19515)
• Keywords: segmentation
• Abstract: Intraoral scans are widely used in digital dentistry for tasks such as dental restoration, treatment planning, and orthodontic procedures. These scans contain detailed topological information, but manual annotation of these scans remains a time-consuming task. Deep learning-based methods have been developed to automate tasks such as tooth segmentation. A typical intraoral scan contains over 200,000 mesh cells, making direct processing computationally expensive. Models are often trained on downsampled versions, typically with 10,000 or 16,000 cells. Previous studies suggest that downsampling may degrade segmentation accuracy, but the extent of this degradation remains unclear. Understanding the extent of degradation is crucial for deploying ML models on edge devices. This study evaluates the extent of performance degradation with decreasing resolution. We train a deep learning model (PointMLP) on intraoral scans decimated to 16K, 10K, 8K, 6K, 4K, and 2K mesh cells. Models trained at lower resolutions are tested on high-resolution scans to assess performance. Our goal is to identify a resolution that balances computational efficiency and segmentation accuracy.

Title: Cognitive networks highlight differences and similarities in the STEM mindsets of human and LLM-simulated trainees, experts and academics

• Authors: Edith Haim, Lars van den Bergh, Cynthia S. Q. Siew, Yoed N. Kenett, Daniele Marinazzo, Massimo Stella
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19529
• Pdf URL: https://arxiv.org/pdf/2502.19529
• Copy Paste: [[2502.19529]] Cognitive networks highlight differences and similarities in the STEM mindsets of human and LLM-simulated trainees, experts and academics(https://arxiv.org/abs/2502.19529)
• Keywords: robust, large language model
• Abstract: Understanding attitudes towards STEM means quantifying the cognitive and emotional ways in which individuals, and potentially large language models too, conceptualise such subjects. This study uses behavioural forma mentis networks (BFMNs) to investigate the STEM-focused mindset, i.e. ways of associating and perceiving ideas, of 177 human participants and 177 artificial humans simulated by GPT-3.5. Participants were split in 3 groups - trainees, experts and academics - to compare the influence of expertise level on their mindset. The results revealed that human forma mentis networks exhibited significantly higher clustering coefficients compared to GPT-3.5, indicating that human mindsets displayed a tendency to form and close triads of conceptual associations while recollecting STEM ideas. Human experts, in particular, demonstrated robust clustering coefficients, reflecting better integration of STEM concepts into their cognitive networks. In contrast, GPT-3.5 produced sparser mindsets. Furthermore, both human and GPT mindsets framed mathematics in neutral or positive terms, differently from STEM high schoolers, researchers and other large language models sampled in other works. This research contributes to understanding how mindset structure can provide cognitive insights about memory structure and machine limitations.

Title: No, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data

• Authors: Joshua Kazdan, Lisa Yu, Rylan Schaeffer, Chris Cundy, Sanmi Koyejo, Dvijotham Krishnamurthy
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19537
• Pdf URL: https://arxiv.org/pdf/2502.19537
• Copy Paste: [[2502.19537]] No, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data(https://arxiv.org/abs/2502.19537)
• Keywords: protect, defense, attack
• Abstract: Leading language model (LM) providers like OpenAI and Google offer fine-tuning APIs that allow customers to adapt LMs for specific use cases. To prevent misuse, these LM providers implement filtering mechanisms to block harmful fine-tuning data. Consequently, adversaries seeking to produce unsafe LMs via these APIs must craft adversarial training data that are not identifiably harmful. We make three contributions in this context: 1. We show that many existing attacks that use harmless data to create unsafe LMs rely on eliminating model refusals in the first few tokens of their responses. 2. We show that such prior attacks can be blocked by a simple defense that pre-fills the first few tokens from an aligned model before letting the fine-tuned model fill in the rest. 3. We describe a new data-poisoning attack, ``No, Of course I Can Execute'' (NOICE), which exploits an LM's formulaic refusal mechanism to elicit harmful responses. By training an LM to refuse benign requests on the basis of safety before fulfilling those requests regardless, we are able to jailbreak several open-source models and a closed-source model (GPT-4o). We show an attack success rate (ASR) of 57% against GPT-4o; our attack earned a Bug Bounty from OpenAI. Against open-source models protected by simple defenses, we improve ASRs by an average of 3.25 times compared to the best performing previous attacks that use only harmless data. NOICE demonstrates the exploitability of repetitive refusal mechanisms and broadens understanding of the threats closed-source models face from harmless data.

Title: Dictionary-based Framework for Interpretable and Consistent Object Parsing

• Authors: Tiezheng Zhang, Qihang Yu, Alan Yuille, Ju He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19540
• Pdf URL: https://arxiv.org/pdf/2502.19540
• Copy Paste: [[2502.19540]] Dictionary-based Framework for Interpretable and Consistent Object Parsing(https://arxiv.org/abs/2502.19540)
• Keywords: transformer, segmentation
• Abstract: In this work, we present CoCal, an interpretable and consistent object parsing framework based on dictionary-based mask transformer. Designed around Contrastive Components and Logical Constraints, CoCal rethinks existing cluster-based mask transformer architectures used in segmentation; Specifically, CoCal utilizes a set of dictionary components, with each component being explicitly linked to a specific semantic class. To advance this concept, CoCal introduces a hierarchical formulation of dictionary components that aligns with the semantic hierarchy. This is achieved through the integration of both within-level contrastive components and cross-level logical constraints. Concretely, CoCal employs a component-wise contrastive algorithm at each semantic level, enabling the contrasting of dictionary components within the same class against those from different classes. Furthermore, CoCal addresses logical concerns by ensuring that the dictionary component representing a particular part is closer to its corresponding object component than to those of other objects through a cross-level contrastive learning objective. To further enhance our logical relation modeling, we implement a post-processing function inspired by the principle that a pixel assigned to a part should also be assigned to its corresponding object. With these innovations, CoCal establishes a new state-of-the-art performance on both PartImageNet and Pascal-Part-108, outperforming previous methods by a significant margin of 2.08% and 0.70% in part mIoU, respectively. Moreover, CoCal exhibits notable enhancements in object-level metrics across these benchmarks, highlighting its capacity to not only refine parsing at a finer level but also elevate the overall quality of object segmentation.

Title: High-fidelity Multiphysics Modelling for Rapid Predictions Using Physics-informed Parallel Neural Operator

• Authors: Biao Yuan, He Wang, Yanjie Song, Ana Heitor, Xiaohui Chen
• Subjects: cs.LG, math.NA, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2502.19543
• Pdf URL: https://arxiv.org/pdf/2502.19543
• Copy Paste: [[2502.19543]] High-fidelity Multiphysics Modelling for Rapid Predictions Using Physics-informed Parallel Neural Operator(https://arxiv.org/abs/2502.19543)
• Keywords: robust, data-free
• Abstract: Modelling complex multiphysics systems governed by nonlinear and strongly coupled partial differential equations (PDEs) is a cornerstone in computational science and engineering. However, it remains a formidable challenge for traditional numerical solvers due to high computational cost, making them impractical for large-scale applications. Neural operators' reliance on data-driven training limits their applicability in real-world scenarios, as data is often scarce or expensive to obtain. Here, we propose a novel paradigm, physics-informed parallel neural operator (PIPNO), a scalable and unsupervised learning framework that enables data-free PDE modelling by leveraging only governing physical laws. The parallel kernel integration design, incorporating ensemble learning, significantly enhances both compatibility and computational efficiency, enabling scalable operator learning for nonlinear and strongly coupled PDEs. PIPNO efficiently captures nonlinear operator mappings across diverse physics, including geotechnical engineering, material science, electromagnetism, quantum mechanics, and fluid dynamics. The proposed method achieves high-fidelity and rapid predictions, outperforming existing operator learning approaches in modelling nonlinear and strongly coupled multiphysics systems. Therefore, PIPNO offers a powerful alternative to conventional solvers, broadening the applicability of neural operators for multiphysics modelling while ensuring efficiency, robustness, and scalability.

Title: Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in QA Agents

• Authors: Ashley Lewis, Michael White, Jing Liu, Toshiaki Koike-Akino, Kieran Parsons, Ye Wang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19545
• Pdf URL: https://arxiv.org/pdf/2502.19545
• Copy Paste: [[2502.19545]] Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in QA Agents(https://arxiv.org/abs/2502.19545)
• Keywords: robust, large language model
• Abstract: The deployment of Large Language Models (LLMs) in customer support is constrained by hallucination-generating false information-and the high cost of proprietary models. To address these challenges, we propose a retrieval-augmented question-answering (QA) pipeline and explore how to balance human input and automation. Using a dataset of questions about a Samsung Smart TV user manual, we demonstrate that synthetic data generated by LLMs outperforms crowdsourced data in reducing hallucination in finetuned models. We also compare self-training (fine-tuning models on their own outputs) and knowledge distillation (fine-tuning on stronger models' outputs, e.g., GPT-4o), and find that self-training achieves comparable hallucination reduction. We conjecture that this surprising finding can be attributed to increased exposure bias issues in the knowledge distillation case and support this conjecture with post hoc analysis. We also improve robustness to unanswerable questions and retrieval failures with contextualized "I don't know" responses. These findings show that scalable, cost-efficient QA systems can be built using synthetic data and self-training with open-source models, reducing reliance on proprietary tools or costly human annotations.

Title: When Large Language Models Meet Speech: A Survey on Integration Approaches

• Authors: Zhengdong Yang, Shuichiro Shimizu, Yahan Yu, Chenhui Chu
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.19548
• Pdf URL: https://arxiv.org/pdf/2502.19548
• Copy Paste: [[2502.19548]] When Large Language Models Meet Speech: A Survey on Integration Approaches(https://arxiv.org/abs/2502.19548)
• Keywords: large language model
• Abstract: Recent advancements in large language models (LLMs) have spurred interest in expanding their application beyond text-based tasks. A large number of studies have explored integrating other modalities with LLMs, notably speech modality, which is naturally related to text. This paper surveys the integration of speech with LLMs, categorizing the methodologies into three primary approaches: text-based, latent-representation-based, and audio-token-based integration. We also demonstrate how these methods are applied across various speech-related applications and highlight the challenges in this field to offer inspiration for

Title: Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?

• Authors: Yudi Zhang, Lu Wang, Meng Fang, Yali Du, Chenghua Huang, Jun Wang, Qingwei Lin, Mykola Pechenizkiy, Dongmei Zhang, Saravan Rajmohan, Qi Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19557
• Pdf URL: https://arxiv.org/pdf/2502.19557
• Copy Paste: [[2502.19557]] Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?(https://arxiv.org/abs/2502.19557)
• Keywords: large language model
• Abstract: Distilling large language models (LLMs) typically involves transferring the teacher model's responses through supervised fine-tuning (SFT). However, this approach neglects the potential to distill both data (output content) and reward signals (quality evaluations). Extracting reliable reward signals directly from teacher models is challenging, as LLMs are optimized for generation rather than evaluation, often resulting in biased or inconsistent assessments. To address this limitation, we propose a novel distillation pipeline that transfers both responses and rewards. Our method generates pseudo-rewards through a self-supervised mechanism that leverages the inherent structure of both teacher and student responses, enabling reward learning without explicit external evaluation. The reward model subsequently guides reinforcement learning (RL), allowing iterative refinement of the student model after an SFT warm-up phase. Experiments on GSM8K and MMLU-PRO demonstrate that our method consistently outperforms traditional SFT-based approaches, enabling student models to surpass the performance of their teachers. This work highlights the potential for scalable, efficient distillation through structured self-supervised reward learning, reducing dependence on external reward supervision.

Title: Stay Focused: Problem Drift in Multi-Agent Debate

• Authors: Jonas Becker, Lars Benedikt Kaesberg, Andreas Stephan, Jan Philip Wahle, Terry Ruas, Bela Gipp
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19559
• Pdf URL: https://arxiv.org/pdf/2502.19559
• Copy Paste: [[2502.19559]] Stay Focused: Problem Drift in Multi-Agent Debate(https://arxiv.org/abs/2502.19559)
• Keywords: generative, large language model
• Abstract: Multi-agent debate - multiple instances of large language models discussing problems in turn-based interaction - has shown promise for solving knowledge and reasoning tasks. However, these methods show limitations, particularly when scaling them to longer reasoning chains. In this study, we unveil a new issue of multi-agent debate: discussions drift away from the initial problem over multiple turns. We define this phenomenon as problem drift and quantify its presence across ten tasks (i.e., three generative, three knowledge, three reasoning, and one instruction-following task). To identify the reasons for this issue, we perform a human study with eight experts on discussions suffering from problem drift, who find the most common issues are a lack of progress (35% of cases), low-quality feedback (26% of cases), and a lack of clarity (25% of cases). To systematically address the issue of problem drift, we propose DRIFTJudge, a method based on LLM-as-a-judge, to detect problem drift at test-time. We further propose DRIFTPolicy, a method to mitigate 31% of problem drift cases. Our study can be seen as a first step to understanding a key limitation of multi-agent debate, highlighting pathways for improving their effectiveness in the future.

Title: Atlas: A Framework for ML Lifecycle Provenance & Transparency

• Authors: Marcin Spoczynski, Marcela S. Melara, Sebastian Szyller
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19567
• Pdf URL: https://arxiv.org/pdf/2502.19567
• Copy Paste: [[2502.19567]] Atlas: A Framework for ML Lifecycle Provenance & Transparency(https://arxiv.org/abs/2502.19567)
• Keywords: attack
• Abstract: The rapid adoption of open source machine learning (ML) datasets and models exposes today's AI applications to critical risks like data poisoning and supply chain attacks across the ML lifecycle. With growing regulatory pressure to address these issues through greater transparency, ML model vendors face challenges balancing these requirements against confidentiality for data and intellectual property needs. We propose Atlas, a framework that enables fully attestable ML pipelines. Atlas leverages open specifications for data and software supply chain provenance to collect verifiable records of model artifact authenticity and end-to-end lineage metadata. Atlas combines trusted hardware and transparency logs to enhance metadata integrity, preserve data confidentiality, and limit unauthorized access during ML pipeline operations, from training through deployment. Our prototype implementation of Atlas integrates several open-source tools to build an ML lifecycle transparency system, and assess the practicality of Atlas through two case study ML pipelines.

Title: PhenoProfiler: Advancing Phenotypic Learning for Image-based Drug Discovery

• Authors: Bo Li, Bob Zhang, Chengyang Zhang, Minghao Zhou, Weiliang Huang, Shihang Wang, Qing Wang, Mengran Li, Yong Zhang, Qianqian Song
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19568
• Pdf URL: https://arxiv.org/pdf/2502.19568
• Copy Paste: [[2502.19568]] PhenoProfiler: Advancing Phenotypic Learning for Image-based Drug Discovery(https://arxiv.org/abs/2502.19568)
• Keywords: robust, interpretability
• Abstract: In the field of image-based drug discovery, capturing the phenotypic response of cells to various drug treatments and perturbations is a crucial step. However, existing methods require computationally extensive and complex multi-step procedures, which can introduce inefficiencies, limit generalizability, and increase potential errors. To address these challenges, we present PhenoProfiler, an innovative model designed to efficiently and effectively extract morphological representations, enabling the elucidation of phenotypic changes induced by treatments. PhenoProfiler is designed as an end-to-end tool that processes whole-slide multi-channel images directly into low-dimensional quantitative representations, eliminating the extensive computational steps required by existing methods. It also includes a multi-objective learning module to enhance robustness, accuracy, and generalization in morphological representation learning. PhenoProfiler is rigorously evaluated on large-scale publicly available datasets, including over 230,000 whole-slide multi-channel images in end-to-end scenarios and more than 8.42 million single-cell images in non-end-to-end settings. Across these benchmarks, PhenoProfiler consistently outperforms state-of-the-art methods by up to 20%, demonstrating substantial improvements in both accuracy and robustness. Furthermore, PhenoProfiler uses a tailored phenotype correction strategy to emphasize relative phenotypic changes under treatments, facilitating the detection of biologically meaningful signals. UMAP visualizations of treatment profiles demonstrate PhenoProfiler ability to effectively cluster treatments with similar biological annotations, thereby enhancing interpretability. These findings establish PhenoProfiler as a scalable, generalizable, and robust tool for phenotypic learning.

Title: LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM

• Authors: Yehonathan Refael, Iftach Arbel, Ofir Lindenbaum, Tom Tirer
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.19571
• Pdf URL: https://arxiv.org/pdf/2502.19571
• Copy Paste: [[2502.19571]] LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM(https://arxiv.org/abs/2502.19571)
• Keywords: robust
• Abstract: We study robust parameter-efficient fine-tuning (PEFT) techniques designed to improve accuracy and generalization while operating within strict computational and memory hardware constraints, specifically focusing on large-language models (LLMs). Existing PEFT methods often lack robustness and fail to generalize effectively across diverse tasks, leading to suboptimal performance in real-world scenarios. To address this, we present a new highly computationally efficient framework called AdaZo-SAM, combining Adam and Sharpness-Aware Minimization (SAM) while requiring only a single-gradient computation in every iteration. This is achieved using a stochastic zeroth-order estimation to find SAM's ascent perturbation. We provide a convergence guarantee for AdaZo-SAM and show that it improves the generalization ability of state-of-the-art PEFT methods. Additionally, we design a low-rank gradient optimization method named LORENZA, which is a memory-efficient version of AdaZo-SAM. LORENZA utilizes a randomized SVD scheme to efficiently compute the subspace projection matrix and apply optimization steps onto the selected subspace. This technique enables full-parameter fine-tuning with adaptive low-rank gradient updates, achieving the same reduced memory consumption as gradient-low-rank-projection methods. We provide a convergence analysis of LORENZA and demonstrate its merits for pre-training and fine-tuning LLMs.

Title: Do Large Language Models Know How Much They Know?

• Authors: Gabriele Prato, Jerry Huang, Prasannna Parthasarathi, Shagun Sodhani, Sarath Chandar
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19573
• Pdf URL: https://arxiv.org/pdf/2502.19573
• Copy Paste: [[2502.19573]] Do Large Language Models Know How Much They Know?(https://arxiv.org/abs/2502.19573)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have emerged as highly capable systems and are increasingly being integrated into various uses. However, the rapid pace of their deployment has outpaced a comprehensive understanding of their internal mechanisms and a delineation of their capabilities and limitations. A desired attribute of an intelligent system is its ability to recognize the scope of its own knowledge. To investigate whether LLMs embody this characteristic, we develop a benchmark designed to challenge these models to enumerate all information they possess on specific topics. This benchmark evaluates whether the models recall excessive, insufficient, or the precise amount of information, thereby indicating their awareness of their own knowledge. Our findings reveal that all tested LLMs, given sufficient scale, demonstrate an understanding of how much they know about specific topics. While different architectures exhibit varying rates of this capability's emergence, the results suggest that awareness of knowledge may be a generalizable attribute of LLMs. Further research is needed to confirm this potential and fully elucidate the underlying mechanisms.

Title: Tell me why: Visual foundation models as self-explainable classifiers

• Authors: Hugues Turbé, Mina Bjelogrlic, Gianmarco Mengaldo, Christian Lovis
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19577
• Pdf URL: https://arxiv.org/pdf/2502.19577
• Copy Paste: [[2502.19577]] Tell me why: Visual foundation models as self-explainable classifiers(https://arxiv.org/abs/2502.19577)
• Keywords: interpretability
• Abstract: Visual foundation models (VFMs) have become increasingly popular due to their state-of-the-art performance. However, interpretability remains crucial for critical applications. In this sense, self-explainable models (SEM) aim to provide interpretable classifiers that decompose predictions into a weighted sum of interpretable concepts. Despite their promise, recent studies have shown that these explanations often lack faithfulness. In this work, we combine VFMs with a novel prototypical architecture and specialized training objectives. By training only a lightweight head (approximately 1M parameters) on top of frozen VFMs, our approach (ProtoFM) offers an efficient and interpretable solution. Evaluations demonstrate that our approach achieves competitive classification performance while outperforming existing models across a range of interpretability metrics derived from the literature. Code is available at this https URL.

Title: Where Are We? Evaluating LLM Performance on African Languages

• Authors: Ife Adebara, Hawau Olamide Toyin, Nahom Tesfu Ghebremichael, AbdelRahim Elmadany, Muhammad Abdul-Mageed
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19582
• Pdf URL: https://arxiv.org/pdf/2502.19582
• Copy Paste: [[2502.19582]] Where Are We? Evaluating LLM Performance on African Languages(https://arxiv.org/abs/2502.19582)
• Keywords: large language model
• Abstract: Africa's rich linguistic heritage remains underrepresented in NLP, largely due to historical policies that favor foreign languages and create significant data inequities. In this paper, we integrate theoretical insights on Africa's language landscape with an empirical evaluation using Sahara - a comprehensive benchmark curated from large-scale, publicly accessible datasets capturing the continent's linguistic diversity. By systematically assessing the performance of leading large language models (LLMs) on Sahara, we demonstrate how policy-induced data variations directly impact model effectiveness across African languages. Our findings reveal that while a few languages perform reasonably well, many Indigenous languages remain marginalized due to sparse data. Leveraging these insights, we offer actionable recommendations for policy reforms and inclusive data practices. Overall, our work underscores the urgent need for a dual approach - combining theoretical understanding with empirical evaluation - to foster linguistic diversity in AI for African communities.

Title: Improving Representation Learning of Complex Critical Care Data with ICU-BERT

• Authors: Ricardo Santos, André V. Carreiro, Xi Peng, Hugo Gamboa, Holger Fröhlich
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19593
• Pdf URL: https://arxiv.org/pdf/2502.19593
• Copy Paste: [[2502.19593]] Improving Representation Learning of Complex Critical Care Data with ICU-BERT(https://arxiv.org/abs/2502.19593)
• Keywords: robust, transformer, generative, large language model
• Abstract: The multivariate, asynchronous nature of real-world clinical data, such as that generated in Intensive Care Units (ICUs), challenges traditional AI-based decision-support systems. These often assume data regularity and feature independence and frequently rely on limited data scopes and manual feature engineering. The potential of generative AI technologies has not yet been fully exploited to analyze clinical data. We introduce ICU-BERT, a transformer-based model pre-trained on the MIMIC-IV database using a multi-task scheme to learn robust representations of complex ICU data with minimal preprocessing. ICU-BERT employs a multi-token input strategy, incorporating dense embeddings from a biomedical Large Language Model to learn a generalizable representation of complex and multivariate ICU data. With an initial evaluation of five tasks and four additional ICU datasets, ICU-BERT results indicate that ICU-BERT either compares to or surpasses current performance benchmarks by leveraging fine-tuning. By integrating structured and unstructured data, ICU-BERT advances the use of foundational models in medical informatics, offering an adaptable solution for clinical decision support across diverse applications.

Title: Introduction to Sequence Modeling with Transformers

• Authors: Joni-Kristian Kämäräinen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19597
• Pdf URL: https://arxiv.org/pdf/2502.19597
• Copy Paste: [[2502.19597]] Introduction to Sequence Modeling with Transformers(https://arxiv.org/abs/2502.19597)
• Keywords: transformer
• Abstract: Understanding the transformer architecture and its workings is essential for machine learning (ML) engineers. However, truly understanding the transformer architecture can be demanding, even if you have a solid background in machine learning or deep learning. The main working horse is attention, which yields to the transformer encoder-decoder structure. However, putting attention aside leaves several programming components that are easy to implement but whose role for the whole is unclear. These components are 'tokenization', 'embedding' ('un-embedding'), 'masking', 'positional encoding', and 'padding'. The focus of this work is on understanding them. To keep things simple, the understanding is built incrementally by adding components one by one, and after each step investigating what is doable and what is undoable with the current model. Simple sequences of zeros (0) and ones (1) are used to study the workings of each step.

Title: Learning Ensembles of Interpretable Simple Structure

• Authors: Gaurav Arwade, Sigurdur Olafsson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19602
• Pdf URL: https://arxiv.org/pdf/2502.19602
• Copy Paste: [[2502.19602]] Learning Ensembles of Interpretable Simple Structure(https://arxiv.org/abs/2502.19602)
• Keywords: robust, explainability
• Abstract: Decision-making in complex systems often relies on machine learning models, yet highly accurate models such as XGBoost and neural networks can obscure the reasoning behind their predictions. In operations research applications, understanding how a decision is made is often as crucial as the decision itself. Traditional interpretable models, such as decision trees and logistic regression, provide transparency but may struggle with datasets containing intricate feature interactions. However, complexity in decision-making stem from interactions that are only relevant within certain subsets of data. Within these subsets, feature interactions may be simplified, forming simple structures where simple interpretable models can perform effectively. We propose a bottom-up simple structure-identifying algorithm that partitions data into interpretable subgroups known as simple structure, where feature interactions are minimized, allowing simple models to be trained within each subgroup. We demonstrate the robustness of the algorithm on synthetic data and show that the decision boundaries derived from simple structures are more interpretable and aligned with the intuition of the domain than those learned from a global model. By improving both explainability and predictive accuracy, our approach provides a principled framework for decision support in applications where model transparency is essential.

Title: Revisiting Word Embeddings in the LLM Era

• Authors: Yash Mahajan, Matthew Freestone, Sathyanarayanan Aakur, Santu Karmaker
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19607
• Pdf URL: https://arxiv.org/pdf/2502.19607
• Copy Paste: [[2502.19607]] Revisiting Word Embeddings in the LLM Era(https://arxiv.org/abs/2502.19607)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have recently shown remarkable advancement in various NLP tasks. As such, a popular trend has emerged lately where NLP researchers extract word/sentence/document embeddings from these large decoder-only models and use them for various inference tasks with promising results. However, it is still unclear whether the performance improvement of LLM-induced embeddings is merely because of scale or whether underlying embeddings they produce significantly differ from classical encoding models like Word2Vec, GloVe, Sentence-BERT (SBERT) or Universal Sentence Encoder (USE). This is the central question we investigate in the paper by systematically comparing classical decontextualized and contextualized word embeddings with the same for LLM-induced embeddings. Our results show that LLMs cluster semantically related words more tightly and perform better on analogy tasks in decontextualized settings. However, in contextualized settings, classical models like SimCSE often outperform LLMs in sentence-level similarity assessment tasks, highlighting their continued relevance for fine-grained semantics.

Title: Evaluation of Hate Speech Detection Using Large Language Models and Geographical Contextualization

• Authors: Anwar Hossain Zahid, Monoshi Kumar Roy, Swarna Das
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19612
• Pdf URL: https://arxiv.org/pdf/2502.19612
• Copy Paste: [[2502.19612]] Evaluation of Hate Speech Detection Using Large Language Models and Geographical Contextualization(https://arxiv.org/abs/2502.19612)
• Keywords: robust, large language model
• Abstract: The proliferation of hate speech on social media is one of the serious issues that is bringing huge impacts to society: an escalation of violence, discrimination, and social fragmentation. The problem of detecting hate speech is intrinsically multifaceted due to cultural, linguistic, and contextual complexities and adversarial manipulations. In this study, we systematically investigate the performance of LLMs on detecting hate speech across multilingual datasets and diverse geographic contexts. Our work presents a new evaluation framework in three dimensions: binary classification of hate speech, geography-aware contextual detection, and robustness to adversarially generated text. Using a dataset of 1,000 comments from five diverse regions, we evaluate three state-of-the-art LLMs: Llama2 (13b), Codellama (7b), and DeepSeekCoder (6.7b). Codellama had the best binary classification recall with 70.6% and an F1-score of 52.18%, whereas DeepSeekCoder had the best performance in geographic sensitivity, correctly detecting 63 out of 265 locations. The tests for adversarial robustness also showed significant weaknesses; Llama2 misclassified 62.5% of manipulated samples. These results bring to light the trade-offs between accuracy, contextual understanding, and robustness in the current versions of LLMs. This work has thus set the stage for developing contextually aware, multilingual hate speech detection systems by underlining key strengths and limitations, therefore offering actionable insights for future research and real-world applications.

Title: Is Your Paper Being Reviewed by an LLM? A New Benchmark Dataset and Approach for Detecting AI Text in Peer Review

• Authors: Sungduk Yu, Man Luo, Avinash Madusu, Vasudev Lal, Phillip Howard
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19614
• Pdf URL: https://arxiv.org/pdf/2502.19614
• Copy Paste: [[2502.19614]] Is Your Paper Being Reviewed by an LLM? A New Benchmark Dataset and Approach for Detecting AI Text in Peer Review(https://arxiv.org/abs/2502.19614)
• Keywords: generative, large language model
• Abstract: Peer review is a critical process for ensuring the integrity of published scientific research. Confidence in this process is predicated on the assumption that experts in the relevant domain give careful consideration to the merits of manuscripts which are submitted for publication. With the recent rapid advancements in large language models (LLMs), a new risk to the peer review process is that negligent reviewers will rely on LLMs to perform the often time consuming process of reviewing a paper. However, there is a lack of existing resources for benchmarking the detectability of AI text in the domain of peer review. To address this deficiency, we introduce a comprehensive dataset containing a total of 788,984 AI-written peer reviews paired with corresponding human reviews, covering 8 years of papers submitted to each of two leading AI research conferences (ICLR and NeurIPS). We use this new resource to evaluate the ability of 18 existing AI text detection algorithms to distinguish between peer reviews written by humans and different state-of-the-art LLMs. Motivated by the shortcomings of existing methods, we propose a new detection approach which surpasses existing methods in the identification of AI written peer reviews. Our work reveals the difficulty of identifying AI-generated text at the individual peer review level, highlighting the urgent need for new tools and methods to detect this unethical use of generative AI.

Title: Comprehensive Digital Forensics and Risk Mitigation Strategy for Modern Enterprises

• Authors: Shamnad Mohamed Shaffi
• Subjects: cs.CR, cs.ET
• Abstract URL: https://arxiv.org/abs/2502.19621
• Pdf URL: https://arxiv.org/pdf/2502.19621
• Copy Paste: [[2502.19621]] Comprehensive Digital Forensics and Risk Mitigation Strategy for Modern Enterprises(https://arxiv.org/abs/2502.19621)
• Keywords: security, protect, robust
• Abstract: Enterprises today face increasing cybersecurity threats that necessitate robust digital forensics and risk mitigation strategies. This paper explores these challenges through an imaginary case study of an organization, a global identity management and data analytics company handling vast customer data. Given the critical nature of its data assets, EP has established a dedicated digital forensics team to detect threats, manage vulnerabilities, and respond to security incidents. This study outlines an approach to cybersecurity, including proactive threat anticipation, forensic investigations, and compliance with regulations like GDPR and CCPA. Key threats such as social engineering, insider risks, phishing, and ransomware are examined, along with mitigation strategies leveraging AI and machine learning. By detailing security framework, this paper highlights best practices in digital forensics, incident response, and enterprise risk management. The findings emphasize the importance of continuous monitoring, policy enforcement, and adaptive security measures to protect sensitive data and ensure business continuity in an evolving threat landscape

Title: Weaker LLMs' Opinions Also Matter: Mixture of Opinions Enhances LLM's Mathematical Reasoning

• Authors: Yanan Chen, Ali Pesaranghader, Tanmana Sadhu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19622
• Pdf URL: https://arxiv.org/pdf/2502.19622
• Copy Paste: [[2502.19622]] Weaker LLMs' Opinions Also Matter: Mixture of Opinions Enhances LLM's Mathematical Reasoning(https://arxiv.org/abs/2502.19622)
• Keywords: large language model
• Abstract: Recent advances in Large Language Models (LLMs) have raised interest in their formal reasoning capabilities, particularly in mathematics. While closed LLMs like GPT-4 perform well on mathematical benchmarks, e.g., GSM8K, it remains unclear whether small to medium-sized open LLMs can achieve similar performance, questioning their reliability. To close this gap, we propose a post-training approach leveraging a mixture of opinions (MoO) from weaker ancillary LLMs to enhance a (relatively) stronger LLM's reasoning. For that, each post-training sample is augmented with Chain-of-Thought (CoT) reasoning steps and answers from ancillary LLMs, enabling the main LLM to learn from diverse perspectives. We compare MoO with standard supervised fine-tuning (SFT), few-shot prompting, and the Mixture of Agents (MoA) method on mathematical reasoning benchmarks. Our results show that incorporating weaker LLMs' opinions improves mathematical reasoning by an average of 5%, highlighting the value of diverse perspectives in reasoning tasks.

Title: 3D Nephrographic Image Synthesis in CT Urography with the Diffusion Model and Swin Transformer

• Authors: Hongkun Yu, Syed Jamal Safdar Gardezi, E. Jason Abel, Daniel Shapiro, Meghan G. Lubner, Joshua Warner, Matthew Smith, Giuseppe Toia, Lu Mao, Pallavi Tiwari, Andrew L. Wentland
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19623
• Pdf URL: https://arxiv.org/pdf/2502.19623
• Copy Paste: [[2502.19623]] 3D Nephrographic Image Synthesis in CT Urography with the Diffusion Model and Swin Transformer(https://arxiv.org/abs/2502.19623)
• Keywords: diffusion, transformer
• Abstract: Purpose: This study aims to develop and validate a method for synthesizing 3D nephrographic phase images in CT urography (CTU) examinations using a diffusion model integrated with a Swin Transformer-based deep learning approach. Materials and Methods: This retrospective study was approved by the local Institutional Review Board. A dataset comprising 327 patients who underwent three-phase CTU (mean $\pm$ SD age, 63 $\pm$ 15 years; 174 males, 153 females) was curated for deep learning model development. The three phases for each patient were aligned with an affine registration algorithm. A custom deep learning model coined dsSNICT (diffusion model with a Swin transformer for synthetic nephrographic phase images in CT) was developed and implemented to synthesize the nephrographic images. Performance was assessed using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Mean Absolute Error (MAE), and Fréchet Video Distance (FVD). Qualitative evaluation by two fellowship-trained abdominal radiologists was performed. Results: The synthetic nephrographic images generated by our proposed approach achieved high PSNR (26.3 $\pm$ 4.4 dB), SSIM (0.84 $\pm$ 0.069), MAE (12.74 $\pm$ 5.22 HU), and FVD (1323). Two radiologists provided average scores of 3.5 for real images and 3.4 for synthetic images (P-value = 0.5) on a Likert scale of 1-5, indicating that our synthetic images closely resemble real images. Conclusion: The proposed approach effectively synthesizes high-quality 3D nephrographic phase images. This model can be used to reduce radiation dose in CTU by 33.3\% without compromising image quality, which thereby enhances the safety and diagnostic utility of CT urography.

Title: Treatment Non-Adherence Bias in Clinical Machine Learning: A Real-World Study on Hypertension Medication

• Authors: Zhongyuan Liang, Arvind Suresh, Irene Y. Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19625
• Pdf URL: https://arxiv.org/pdf/2502.19625
• Copy Paste: [[2502.19625]] Treatment Non-Adherence Bias in Clinical Machine Learning: A Real-World Study on Hypertension Medication(https://arxiv.org/abs/2502.19625)
• Keywords: large language model
• Abstract: Machine learning systems trained on electronic health records (EHRs) increasingly guide treatment decisions, but their reliability depends on the critical assumption that patients follow the prescribed treatments recorded in EHRs. Using EHR data from 3,623 hypertension patients, we investigate how treatment non-adherence introduces implicit bias that can fundamentally distort both causal inference and predictive modeling. By extracting patient adherence information from clinical notes using a large language model, we identify 786 patients (21.7%) with medication non-adherence. We further uncover key demographic and clinical factors associated with non-adherence, as well as patient-reported reasons including side effects and difficulties obtaining refills. Our findings demonstrate that this implicit bias can not only reverse estimated treatment effects, but also degrade model performance by up to 5% while disproportionately affecting vulnerable populations by exacerbating disparities in decision outcomes and model error rates. This highlights the importance of accounting for treatment non-adherence in developing responsible and equitable clinical machine learning systems.

Title: MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning

• Authors: Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, Daniel Rueckert
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19634
• Pdf URL: https://arxiv.org/pdf/2502.19634
• Copy Paste: [[2502.19634]] MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning(https://arxiv.org/abs/2502.19634)
• Keywords: robust
• Abstract: Reasoning is a critical frontier for advancing medical image analysis, where transparency and trustworthiness play a central role in both clinician trust and regulatory approval. Although Medical Visual Language Models (VLMs) show promise for radiological tasks, most existing VLMs merely produce final answers without revealing the underlying reasoning. To address this gap, we introduce MedVLM-R1, a medical VLM that explicitly generates natural language reasoning to enhance transparency and trustworthiness. Instead of relying on supervised fine-tuning (SFT), which often suffers from overfitting to training distributions and fails to foster genuine reasoning, MedVLM-R1 employs a reinforcement learning framework that incentivizes the model to discover human-interpretable reasoning paths without using any reasoning references. Despite limited training data (600 visual question answering samples) and model parameters (2B), MedVLM-R1 boosts accuracy from 55.11% to 78.22% across MRI, CT, and X-ray benchmarks, outperforming larger models trained on over a million samples. It also demonstrates robust domain generalization under out-of-distribution tasks. By unifying medical image analysis with explicit reasoning, MedVLM-R1 marks a pivotal step toward trustworthy and interpretable AI in clinical practice.

Title: Developing robust methods to handle missing data in real-world applications effectively

• Authors: Youran Zhou, Mohamed Reda Bouadjenek, Sunil Arya
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19635
• Pdf URL: https://arxiv.org/pdf/2502.19635
• Copy Paste: [[2502.19635]] Developing robust methods to handle missing data in real-world applications effectively(https://arxiv.org/abs/2502.19635)
• Keywords: robust
• Abstract: Missing data is a pervasive challenge spanning diverse data types, including tabular, sensor data, time-series, images and so on. Its origins are multifaceted, resulting in various missing mechanisms. Prior research in this field has predominantly revolved around the assumption of the Missing Completely At Random (MCAR) mechanism. However, Missing At Random (MAR) and Missing Not At Random (MNAR) mechanisms, though equally prevalent, have often remained underexplored despite their significant influence. This PhD project presents a comprehensive research agenda designed to investigate the implications of diverse missing data mechanisms. The principal aim is to devise robust methodologies capable of effectively handling missing data while accommodating the unique characteristics of MCAR, MAR, and MNAR mechanisms. By addressing these gaps, this research contributes to an enriched understanding of the challenges posed by missing data across various industries and data modalities. It seeks to provide practical solutions that enable the effective management of missing data, empowering researchers and practitioners to leverage incomplete datasets confidently.

Title: cMIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning

• Authors: Micha Livne
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19642
• Pdf URL: https://arxiv.org/pdf/2502.19642
• Copy Paste: [[2502.19642]] cMIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning(https://arxiv.org/abs/2502.19642)
• Keywords: robust, generative
• Abstract: Learning representations that are useful for unknown downstream tasks is a fundamental challenge in representation learning. Prominent approaches in this domain include contrastive learning, self-supervised masking, and denoising auto-encoders. In this paper, we introduce a novel method, termed contrastive Mutual Information Machine (cMIM), which aims to enhance the utility of learned representations for downstream tasks. cMIM integrates a new contrastive learning loss with the Mutual Information Machine (MIM) learning framework, a probabilistic auto-encoder that maximizes the mutual information between inputs and latent representations while clustering the latent codes. Despite MIM's potential, initial experiments indicated that the representations learned by MIM were less effective for discriminative downstream tasks compared to state-of-the-art (SOTA) models. The proposed cMIM method directly addresses this limitation. The main contributions of this work are twofold: (1) We propose a novel contrastive extension to MIM for learning discriminative representations which eliminates the need for data augmentation and is robust to variations in the number of negative examples (i.e., batch size). (2) We introduce a generic method for extracting informative embeddings from encoder-decoder models, which significantly improves performance in discriminative downstream tasks without requiring additional training. This method is applicable to any pre-trained encoder-decoder model. By presenting cMIM, we aim to offer a unified generative model that is effective for both generative and discriminative tasks. Our results demonstrate that the learned representations are valuable for downstream tasks while maintaining the generative capabilities of MIM.

Title: Adaptive Score Alignment Learning for Continual Perceptual Quality Assessment of 360-Degree Videos in Virtual Reality

• Authors: Kanglei Zhou, Zikai Hao, Liyuan Wang, Xiaohui Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19644
• Pdf URL: https://arxiv.org/pdf/2502.19644
• Copy Paste: [[2502.19644]] Adaptive Score Alignment Learning for Continual Perceptual Quality Assessment of 360-Degree Videos in Virtual Reality(https://arxiv.org/abs/2502.19644)
• Keywords: extraction
• Abstract: Virtual Reality Video Quality Assessment (VR-VQA) aims to evaluate the perceptual quality of 360-degree videos, which is crucial for ensuring a distortion-free user experience. Traditional VR-VQA methods trained on static datasets with limited distortion diversity struggle to balance correlation and precision. This becomes particularly critical when generalizing to diverse VR content and continually adapting to dynamic and evolving video distribution variations. To address these challenges, we propose a novel approach for assessing the perceptual quality of VR videos, Adaptive Score Alignment Learning (ASAL). ASAL integrates correlation loss with error loss to enhance alignment with human subjective ratings and precision in predicting perceptual quality. In particular, ASAL can naturally adapt to continually changing distributions through a feature space smoothing process that enhances generalization to unseen content. To further improve continual adaptation to dynamic VR environments, we extend ASAL with adaptive memory replay as a novel Continul Learning (CL) framework. Unlike traditional CL models, ASAL utilizes key frame extraction and feature adaptation to address the unique challenges of non-stationary variations with both the computation and storage restrictions of VR devices. We establish a comprehensive benchmark for VR-VQA and its CL counterpart, introducing new data splits and evaluation metrics. Our experiments demonstrate that ASAL outperforms recent strong baseline models, achieving overall correlation gains of up to 4.78\% in the static joint training setting and 12.19\% in the dynamic CL setting on various datasets. This validates the effectiveness of ASAL in addressing the inherent challenges of this http URL code is available at this https URL.

Title: Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models

• Authors: Jan Wehner, Sahar Abdelnabi, Daniel Tan, David Krueger, Mario Fritz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19649
• Pdf URL: https://arxiv.org/pdf/2502.19649
• Copy Paste: [[2502.19649]] Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models(https://arxiv.org/abs/2502.19649)
• Keywords: large language model
• Abstract: Representation Engineering (RepE) is a novel paradigm for controlling the behavior of LLMs. Unlike traditional approaches that modify inputs or fine-tune the model, RepE directly manipulates the model's internal representations. As a result, it may offer more effective, interpretable, data-efficient, and flexible control over models' behavior. We present the first comprehensive survey of RepE for LLMs, reviewing the rapidly growing literature to address key questions: What RepE methods exist and how do they differ? For what concepts and problems has RepE been applied? What are the strengths and weaknesses of RepE compared to other methods? To answer these, we propose a unified framework describing RepE as a pipeline comprising representation identification, operationalization, and control. We posit that while RepE methods offer significant potential, challenges remain, including managing multiple concepts, ensuring reliability, and preserving models' performance. Towards improving RepE, we identify opportunities for experimental and methodological improvements and construct a guide for best practices.

Title: Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning

• Authors: Shangding Gu, Laixi Shi, Muning Wen, Ming Jin, Eric Mazumdar, Yuejie Chi, Adam Wierman, Costas Spanos
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.19652
• Pdf URL: https://arxiv.org/pdf/2502.19652
• Copy Paste: [[2502.19652]] Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning(https://arxiv.org/abs/2502.19652)
• Keywords: robust
• Abstract: Driven by inherent uncertainty and the sim-to-real gap, robust reinforcement learning (RL) seeks to improve resilience against the complexity and variability in agent-environment sequential interactions. Despite the existence of a large number of RL benchmarks, there is a lack of standardized benchmarks for robust RL. Current robust RL policies often focus on a specific type of uncertainty and are evaluated in distinct, one-off environments. In this work, we introduce Robust-Gymnasium, a unified modular benchmark designed for robust RL that supports a wide variety of disruptions across all key RL components-agents' observed state and reward, agents' actions, and the environment. Offering over sixty diverse task environments spanning control and robotics, safe RL, and multi-agent RL, it provides an open-source and user-friendly tool for the community to assess current methods and foster the development of robust RL algorithms. In addition, we benchmark existing standard and robust RL algorithms within this framework, uncovering significant deficiencies in each and offering new insights.

Title: Investigating Neurons and Heads in Transformer-based LLMs for Typographical Errors

• Authors: Kohei Tsuji, Tatsuya Hiraoka, Yuchang Cheng, Eiji Aramaki, Tomoya Iwakura
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19669
• Pdf URL: https://arxiv.org/pdf/2502.19669
• Copy Paste: [[2502.19669]] Investigating Neurons and Heads in Transformer-based LLMs for Typographical Errors(https://arxiv.org/abs/2502.19669)
• Keywords: transformer
• Abstract: This paper investigates how LLMs encode inputs with typos. We hypothesize that specific neurons and attention heads recognize typos and fix them internally using local and global contexts. We introduce a method to identify typo neurons and typo heads that work actively when inputs contain typos. Our experimental results suggest the following: 1) LLMs can fix typos with local contexts when the typo neurons in either the early or late layers are activated, even if those in the other are not. 2) Typo neurons in the middle layers are responsible for the core of typo-fixing with global contexts. 3) Typo heads fix typos by widely considering the context not focusing on specific tokens. 4) Typo neurons and typo heads work not only for typo-fixing but also for understanding general contexts.

Title: Training Robust Graph Neural Networks by Modeling Noise Dependencies

• Authors: Yeonjun In, Kanghoon Yoon, Sukwon Yun, Kibum Kim, Sungchul Kim, Chanyoung Park
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.19670
• Pdf URL: https://arxiv.org/pdf/2502.19670
• Copy Paste: [[2502.19670]] Training Robust Graph Neural Networks by Modeling Noise Dependencies(https://arxiv.org/abs/2502.19670)
• Keywords: robust
• Abstract: In real-world applications, node features in graphs often contain noise from various sources, leading to significant performance degradation in GNNs. Although several methods have been developed to enhance robustness, they rely on the unrealistic assumption that noise in node features is independent of the graph structure and node labels, thereby limiting their applicability. To this end, we introduce a more realistic noise scenario, dependency-aware noise on graphs (DANG), where noise in node features create a chain of noise dependencies that propagates to the graph structure and node labels. We propose a novel robust GNN, DA-GNN, which captures the causal relationships among variables in the data generating process (DGP) of DANG using variational inference. In addition, we present new benchmark datasets that simulate DANG in real-world applications, enabling more practical research on robust GNNs. Extensive experiments demonstrate that DA-GNN consistently outperforms existing baselines across various noise scenarios, including both DANG and conventional noise models commonly considered in this field.

Title: Test-Time Modality Generalization for Medical Image Segmentation

• Authors: Ju-Hyeon Nam, Sang-Chul Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19671
• Pdf URL: https://arxiv.org/pdf/2502.19671
• Copy Paste: [[2502.19671]] Test-Time Modality Generalization for Medical Image Segmentation(https://arxiv.org/abs/2502.19671)
• Keywords: robust, segmentation
• Abstract: Generalizable medical image segmentation is essential for ensuring consistent performance across diverse unseen clinical settings. However, existing methods often overlook the capability to generalize effectively across arbitrary unseen modalities. In this paper, we introduce a novel Test-Time Modality Generalization (TTMG) framework, which comprises two core components: Modality-Aware Style Projection (MASP) and Modality-Sensitive Instance Whitening (MSIW), designed to enhance generalization in arbitrary unseen modality datasets. The MASP estimates the likelihood of a test instance belonging to each seen modality and maps it onto a distribution using modality-specific style bases, guiding its projection effectively. Furthermore, as high feature covariance hinders generalization to unseen modalities, the MSIW is applied during training to selectively suppress modality-sensitive information while retaining modality-invariant features. By integrating MASP and MSIW, the TTMG framework demonstrates robust generalization capabilities for medical image segmentation in unseen modalities a challenge that current methods have largely neglected. We evaluated TTMG alongside other domain generalization techniques across eleven datasets spanning four modalities (colonoscopy, ultrasound, dermoscopy, and radiology), consistently achieving superior segmentation performance across various modality combinations.

Title: Improving Adversarial Transferability in MLLMs via Dynamic Vision-Language Alignment Attack

• Authors: Chenhe Gu, Jindong Gu, Andong Hua, Yao Qin
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19672
• Pdf URL: https://arxiv.org/pdf/2502.19672
• Copy Paste: [[2502.19672]] Improving Adversarial Transferability in MLLMs via Dynamic Vision-Language Alignment Attack(https://arxiv.org/abs/2502.19672)
• Keywords: attack, large language model
• Abstract: Multimodal Large Language Models (MLLMs), built upon LLMs, have recently gained attention for their capabilities in image recognition and understanding. However, while MLLMs are vulnerable to adversarial attacks, the transferability of these attacks across different models remains limited, especially under targeted attack setting. Existing methods primarily focus on vision-specific perturbations but struggle with the complex nature of vision-language modality alignment. In this work, we introduce the Dynamic Vision-Language Alignment (DynVLA) Attack, a novel approach that injects dynamic perturbations into the vision-language connector to enhance generalization across diverse vision-language alignment of different models. Our experimental results show that DynVLA significantly improves the transferability of adversarial examples across various MLLMs, including BLIP2, InstructBLIP, MiniGPT4, LLaVA, and closed-source models such as Gemini.

Title: SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization

• Authors: Shubhankar Borse, Kartikeya Bhardwaj, Mohammad Reza Karimi Dastjerdi, Hyojin Park, Shreya Kadambi, Shobitha Shivakumar, Prathamesh Mandke, Ankita Nayak, Harris Teague, Munawar Hayat, Fatih Porikli
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19673
• Pdf URL: https://arxiv.org/pdf/2502.19673
• Copy Paste: [[2502.19673]] SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization(https://arxiv.org/abs/2502.19673)
• Keywords: diffusion, generative
• Abstract: Diffusion models are increasingly popular for generative tasks, including personalized composition of subjects and styles. While diffusion models can generate user-specified subjects performing text-guided actions in custom styles, they require fine-tuning and are not feasible for personalization on mobile devices. Hence, tuning-free personalization methods such as IP-Adapters have progressively gained traction. However, for the composition of subjects and styles, these works are less flexible due to their reliance on ControlNet, or show content and style leakage artifacts. To tackle these, we present SubZero, a novel framework to generate any subject in any style, performing any action without the need for fine-tuning. We propose a novel set of constraints to enhance subject and style similarity, while reducing leakage. Additionally, we propose an orthogonalized temporal aggregation scheme in the cross-attention blocks of denoising model, effectively conditioning on a text prompt along with single subject and style images. We also propose a novel method to train customized content and style projectors to reduce content and style leakage. Through extensive experiments, we show that our proposed approach, while suitable for running on-edge, shows significant improvements over state-of-the-art works performing subject, style and action composition.

Title: M-LLM Based Video Frame Selection for Efficient Video Understanding

• Authors: Kai Hu, Feng Gao, Xiaohan Nie, Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, Trishul Chilimbi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19680
• Pdf URL: https://arxiv.org/pdf/2502.19680
• Copy Paste: [[2502.19680]] M-LLM Based Video Frame Selection for Efficient Video Understanding(https://arxiv.org/abs/2502.19680)
• Keywords: attack, large language model
• Abstract: Recent advances in Multi-Modal Large Language Models (M-LLMs) show promising results in video reasoning. Popular Multi-Modal Large Language Model (M-LLM) frameworks usually apply naive uniform sampling to reduce the number of video frames that are fed into an M-LLM, particularly for long context videos. However, it could lose crucial context in certain periods of a video, so that the downstream M-LLM may not have sufficient visual information to answer a question. To attack this pain point, we propose a light-weight M-LLM -based frame selection method that adaptively select frames that are more relevant to users' queries. In order to train the proposed frame selector, we introduce two supervision signals (i) Spatial signal, where single frame importance score by prompting a M-LLM; (ii) Temporal signal, in which multiple frames selection by prompting Large Language Model (LLM) using the captions of all frame candidates. The selected frames are then digested by a frozen downstream video M-LLM for visual reasoning and question answering. Empirical results show that the proposed M-LLM video frame selector improves the performances various downstream video Large Language Model (video-LLM) across medium (ActivityNet, NExT-QA) and long (EgoSchema, LongVideoBench) context video question answering benchmarks.

Title: BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance

• Authors: Xin Ye, Burhaneddin Yaman, Sheng Cheng, Feng Tao, Abhirup Mallik, Liu Ren
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19694
• Pdf URL: https://arxiv.org/pdf/2502.19694
• Copy Paste: [[2502.19694]] BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance(https://arxiv.org/abs/2502.19694)
• Keywords: diffusion
• Abstract: Bird's-eye-view (BEV) representations play a crucial role in autonomous driving tasks. Despite recent advancements in BEV generation, inherent noise, stemming from sensor limitations and the learning process, remains largely unaddressed, resulting in suboptimal BEV representations that adversely impact the performance of downstream tasks. To address this, we propose BEVDiffuser, a novel diffusion model that effectively denoises BEV feature maps using the ground-truth object layout as guidance. BEVDiffuser can be operated in a plug-and-play manner during training time to enhance existing BEV models without requiring any architectural modifications. Extensive experiments on the challenging nuScenes dataset demonstrate BEVDiffuser's exceptional denoising and generation capabilities, which enable significant enhancement to existing BEV models, as evidenced by notable improvements of 12.3\% in mAP and 10.1\% in NDS achieved for 3D object detection without introducing additional computational complexity. Moreover, substantial improvements in long-tail object detection and under challenging weather and lighting conditions further validate BEVDiffuser's effectiveness in denoising and enhancing BEV representations.

Title: Prompt-driven Transferable Adversarial Attack on Person Re-Identification with Attribute-aware Textual Inversion

• Authors: Yuan Bian, Min Liu, Yunqi Yi, Xueping Wang, Yaonan Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19697
• Pdf URL: https://arxiv.org/pdf/2502.19697
• Copy Paste: [[2502.19697]] Prompt-driven Transferable Adversarial Attack on Person Re-Identification with Attribute-aware Textual Inversion(https://arxiv.org/abs/2502.19697)
• Keywords: security, attack
• Abstract: Person re-identification (re-id) models are vital in security surveillance systems, requiring transferable adversarial attacks to explore the vulnerabilities of them. Recently, vision-language models (VLM) based attacks have shown superior transferability by attacking generalized image and textual features of VLM, but they lack comprehensive feature disruption due to the overemphasis on discriminative semantics in integral representation. In this paper, we introduce the Attribute-aware Prompt Attack (AP-Attack), a novel method that leverages VLM's image-text alignment capability to explicitly disrupt fine-grained semantic features of pedestrian images by destroying attribute-specific textual embeddings. To obtain personalized textual descriptions for individual attributes, textual inversion networks are designed to map pedestrian images to pseudo tokens that represent semantic embeddings, trained in the contrastive learning manner with images and a predefined prompt template that explicitly describes the pedestrian attributes. Inverted benign and adversarial fine-grained textual semantics facilitate attacker in effectively conducting thorough disruptions, enhancing the transferability of adversarial examples. Extensive experiments show that AP-Attack achieves state-of-the-art transferability, significantly outperforming previous methods by 22.9% on mean Drop Rate in cross-model&dataset attack scenarios.

Title: You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving

• Authors: Guangfeng Jiang, Jun Liu, Yongxuan Lv, Yuzhi Wu, Xianfei Li, Wenlong Liao, Tao He, Pai Peng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19698
• Pdf URL: https://arxiv.org/pdf/2502.19698
• Copy Paste: [[2502.19698]] You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving(https://arxiv.org/abs/2502.19698)
• Keywords: segmentation
• Abstract: Outdoor LiDAR point cloud 3D instance segmentation is a crucial task in autonomous driving. However, it requires laborious human efforts to annotate the point cloud for training a segmentation model. To address this challenge, we propose a YoCo framework, which generates 3D pseudo labels using minimal coarse click annotations in the bird's eye view plane. It is a significant challenge to produce high-quality pseudo labels from sparse annotations. Our YoCo framework first leverages vision foundation models combined with geometric constraints from point clouds to enhance pseudo label generation. Second, a temporal and spatial-based label updating module is designed to generate reliable updated labels. It leverages predictions from adjacent frames and utilizes the inherent density variation of point clouds (dense near, sparse far). Finally, to further improve label quality, an IoU-guided enhancement module is proposed, replacing pseudo labels with high-confidence and high-IoU predictions. Experiments on the Waymo dataset demonstrate YoCo's effectiveness and generality, achieving state-of-the-art performance among weakly supervised methods and surpassing fully supervised Cylinder3D. Additionally, the YoCo is suitable for various networks, achieving performance comparable to fully supervised methods with minimal fine-tuning using only 0.8% of the fully labeled data, significantly reducing annotation costs.

Title: Spatial-Spectral Diffusion Contrastive Representation Network for Hyperspectral Image Classification

• Authors: Yimin Zhu, Linlin Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19699
• Pdf URL: https://arxiv.org/pdf/2502.19699
• Copy Paste: [[2502.19699]] Spatial-Spectral Diffusion Contrastive Representation Network for Hyperspectral Image Classification(https://arxiv.org/abs/2502.19699)
• Keywords: extraction, diffusion, transformer
• Abstract: Although efficient extraction of discriminative spatial-spectral features is critical for hyperspectral images classification (HSIC), it is difficult to achieve these features due to factors such as the spatial-spectral heterogeneity and noise effect. This paper presents a Spatial-Spectral Diffusion Contrastive Representation Network (DiffCRN), based on denoising diffusion probabilistic model (DDPM) combined with contrastive learning (CL) for HSIC, with the following characteristics. First,to improve spatial-spectral feature representation, instead of adopting the UNets-like structure which is widely used for DDPM, we design a novel staged architecture with spatial self-attention denoising module (SSAD) and spectral group self-attention denoising module (SGSAD) in DiffCRN with improved efficiency for spectral-spatial feature learning. Second, to improve unsupervised feature learning efficiency, we design new DDPM model with logarithmic absolute error (LAE) loss and CL that improve the loss function effectiveness and increase the instance-level and inter-class discriminability. Third, to improve feature selection, we design a learnable approach based on pixel-level spectral angle mapping (SAM) for the selection of time steps in the proposed DDPM model in an adaptive and automatic manner. Last, to improve feature integration and classification, we design an Adaptive weighted addition modul (AWAM) and Cross time step Spectral-Spatial Fusion Module (CTSSFM) to fuse time-step-wise features and perform classification. Experiments conducted on widely used four HSI datasets demonstrate the improved performance of the proposed DiffCRN over the classical backbone models and state-of-the-art GAN, transformer models and other pretrained methods. The source code and pre-trained model will be made available publicly.

Title: Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model

• Authors: Yimin Zhu, Linlin Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19700
• Pdf URL: https://arxiv.org/pdf/2502.19700
• Copy Paste: [[2502.19700]] Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model(https://arxiv.org/abs/2502.19700)
• Keywords: diffusion
• Abstract: Although data augmentation is an effective method to address the imbalanced-small sample data (ISSD) problem in hyperspectral image classification (HSIC), most methodologies extend features in the latent space. Few, however, generate realistic and diverse samples using text information to balance the limited number of annotated samples. Recently, text-driven diffusion models have gained significant attention due to their remarkable ability to generate highly diverse images based on given text prompts in natural image synthesis. Therefore, this paper proposes a novel language-informed hyperspectral image synthesis method (Txt2HSI-LDM(VAE)) for addressing the ISSD problem of HSIC. First, for addressing the high-dimensional hyperspectral data, we use universal varitional autoencoeder (VAE) to map the hyperspectral into a low-dimensional latent space and get stable feature representation, which hugely reduce the inference parameter of diffusion model. Next, a semi-supervised diffusion model is designed for fully taking advantage of unlabeled data, beside, random polygon spatial clipping (RPSC) and uncertainty estimation of latent feature (LF-UE) are also used for simulating the varying degrees of mixing of training data. Then, VAE decodes HSI from latent space generated by diffusion model with the conditional language as input, contributing to more realistic and diverse samples. In our experiments, we fully evaluate the effectiveness of synthetic samples from aspect of statistical characteristic and data distribution in 2D-PCA space. Additionally, cross-attention map is visualized on the pixel-level to prove that our proposed model can capture the spatial layout of and geometry of the generated hyperspectral image depend on the visual-linguistic alignment.

Title: CFTrack: Enhancing Lightweight Visual Tracking through Contrastive Learning and Feature Matching

• Authors: Juntao Liang, Jun Hou, Weijun Zhang, Yong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19705
• Pdf URL: https://arxiv.org/pdf/2502.19705
• Copy Paste: [[2502.19705]] CFTrack: Enhancing Lightweight Visual Tracking through Contrastive Learning and Feature Matching(https://arxiv.org/abs/2502.19705)
• Keywords: robust
• Abstract: Achieving both efficiency and strong discriminative ability in lightweight visual tracking is a challenge, especially on mobile and edge devices with limited computational resources. Conventional lightweight trackers often struggle with robustness under occlusion and interference, while deep trackers, when compressed to meet resource constraints, suffer from performance degradation. To address these issues, we introduce CFTrack, a lightweight tracker that integrates contrastive learning and feature matching to enhance discriminative feature representations. CFTrack dynamically assesses target similarity during prediction through a novel contrastive feature matching module optimized with an adaptive contrastive loss, thereby improving tracking accuracy. Extensive experiments on LaSOT, OTB100, and UAV123 show that CFTrack surpasses many state-of-the-art lightweight trackers, operating at 136 frames per second on the NVIDIA Jetson NX platform. Results on the HOOT dataset further demonstrate CFTrack's strong discriminative ability under heavy occlusion.

Title: Weakly Supervised Segmentation Framework for Thyroid Nodule Based on High-confidence Labels and High-rationality Losses

• Authors: Jianning Chi, Zelan Li, Geng Lin, MingYang Sun, Xiaosheng Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19707
• Pdf URL: https://arxiv.org/pdf/2502.19707
• Copy Paste: [[2502.19707]] Weakly Supervised Segmentation Framework for Thyroid Nodule Based on High-confidence Labels and High-rationality Losses(https://arxiv.org/abs/2502.19707)
• Keywords: segmentation
• Abstract: Weakly supervised segmentation methods can delineate thyroid nodules in ultrasound images efficiently using training data with coarse labels, but suffer from: 1) low-confidence pseudo-labels that follow topological priors, introducing significant label noise, and 2) low-rationality loss functions that rigidly compare segmentation with labels, ignoring discriminative information for nodules with diverse and complex shapes. To solve these issues, we clarify the objective and references for weakly supervised ultrasound image segmentation, presenting a framework with high-confidence pseudo-labels to represent topological and anatomical information and high-rationality losses to capture multi-level discriminative features. Specifically, we fuse geometric transformations of four-point annotations and MedSAM model results prompted by specific annotations to generate high-confidence box, foreground, and background labels. Our high-rationality learning strategy includes: 1) Alignment loss measuring spatial consistency between segmentation and box label, and topological continuity within the foreground label, guiding the network to perceive nodule location; 2) Contrastive loss pulling features from labeled foreground regions while pushing features from labeled foreground and background regions, guiding the network to learn nodule and background feature distribution; 3) Prototype correlation loss measuring consistency between correlation maps derived by comparing features with foreground and background prototypes, refining uncertain regions to accurate nodule edges. Experimental results show that our method achieves state-of-the-art performance on the TN3K and DDTI datasets. The code is available at this https URL.

Title: SAP-DIFF: Semantic Adversarial Patch Generation for Black-Box Face Recognition Models via Diffusion Models

• Authors: Mingsi Wang, Shuaiyin Yao, Chang Yue, Lijie Zhang, Guozhu Meng
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.19710
• Pdf URL: https://arxiv.org/pdf/2502.19710
• Copy Paste: [[2502.19710]] SAP-DIFF: Semantic Adversarial Patch Generation for Black-Box Face Recognition Models via Diffusion Models(https://arxiv.org/abs/2502.19710)
• Keywords: attack, robust, diffusion
• Abstract: Given the need to evaluate the robustness of face recognition (FR) models, many efforts have focused on adversarial patch attacks that mislead FR models by introducing localized perturbations. Impersonation attacks are a significant threat because adversarial perturbations allow attackers to disguise themselves as legitimate users. This can lead to severe consequences, including data breaches, system damage, and misuse of resources. However, research on such attacks in FR remains limited. Existing adversarial patch generation methods exhibit limited efficacy in impersonation attacks due to (1) the need for high attacker capabilities, (2) low attack success rates, and (3) excessive query requirements. To address these challenges, we propose a novel method SAP-DIFF that leverages diffusion models to generate adversarial patches via semantic perturbations in the latent space rather than direct pixel manipulation. We introduce an attention disruption mechanism to generate features unrelated to the original face, facilitating the creation of adversarial samples and a directional loss function to guide perturbations toward the target identity feature space, thereby enhancing attack effectiveness and efficiency. Extensive experiments on popular FR models and datasets demonstrate that our method outperforms state-of-the-art approaches, achieving an average attack success rate improvement of 45.66% (all exceeding 40%), and a reduction in the number of queries by about 40% compared to the SOTA approach

Title: Recent Advances on Generalizable Diffusion-generated Image Detection

• Authors: Qijie Xu, Defang Chen, Jiawei Chen, Siwei Lyu, Can Wang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19716
• Pdf URL: https://arxiv.org/pdf/2502.19716
• Copy Paste: [[2502.19716]] Recent Advances on Generalizable Diffusion-generated Image Detection(https://arxiv.org/abs/2502.19716)
• Keywords: diffusion
• Abstract: The rise of diffusion models has significantly improved the fidelity and diversity of generated images. With numerous benefits, these advancements also introduce new risks. Diffusion models can be exploited to create high-quality Deepfake images, which poses challenges for image authenticity verification. In recent years, research on generalizable diffusion-generated image detection has grown rapidly. However, a comprehensive review of this topic is still lacking. To bridge this gap, we present a systematic survey of recent advances and classify them into two main categories: (1) data-driven detection and (2) feature-driven detection. Existing detection methods are further classified into six fine-grained categories based on their underlying principles. Finally, we identify several open challenges and envision some future directions, with the hope of inspiring more research work on this important topic. Reviewed works in this survey can be found at this https URL.

Title: Learning Mask Invariant Mutual Information for Masked Image Modeling

• Authors: Tao Huang, Yanxiang Ma, Shan You, Chang Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19718
• Pdf URL: https://arxiv.org/pdf/2502.19718
• Copy Paste: [[2502.19718]] Learning Mask Invariant Mutual Information for Masked Image Modeling(https://arxiv.org/abs/2502.19718)
• Keywords: segmentation
• Abstract: Masked autoencoders (MAEs) represent a prominent self-supervised learning paradigm in computer vision. Despite their empirical success, the underlying mechanisms of MAEs remain insufficiently understood. Recent studies have attempted to elucidate the functioning of MAEs through contrastive learning and feature representation analysis, yet these approaches often provide only implicit insights. In this paper, we propose a new perspective for understanding MAEs by leveraging the information bottleneck principle in information theory. Our theoretical analyses reveal that optimizing the latent features to balance relevant and irrelevant information is key to improving MAE performance. Building upon our proofs, we introduce MI-MAE, a novel method that optimizes MAEs through mutual information maximization and minimization. By enhancing latent features to retain maximal relevant information between them and the output, and minimizing irrelevant information between them and the input, our approach achieves better performance. Extensive experiments on standard benchmarks show that MI-MAE significantly outperforms MAE models in tasks such as image classification, object detection, and semantic segmentation. Our findings validate the theoretical framework and highlight the practical advantages of applying the information bottleneck principle to MAEs, offering deeper insights for developing more powerful self-supervised learning models.

Title: Sensing and Steering Stereotypes: Extracting and Applying Gender Representation Vectors in LLMs

• Authors: Hannah Cyberey, Yangfeng Ji, David Evans
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2502.19721
• Pdf URL: https://arxiv.org/pdf/2502.19721
• Copy Paste: [[2502.19721]] Sensing and Steering Stereotypes: Extracting and Applying Gender Representation Vectors in LLMs(https://arxiv.org/abs/2502.19721)
• Keywords: large language model
• Abstract: Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate potential harms that may result from these biases, but most work studies biases in LLMs as a black-box problem without considering how concepts are represented within the model. We adapt techniques from representation engineering to study how the concept of "gender" is represented within LLMs. We introduce a new method that extracts concept representations via probability weighting without labeled data and efficiently selects a steering vector for measuring and manipulating the model's representation. We also present a projection-based method that enables precise steering of model predictions and demonstrate its effectiveness in mitigating gender bias in LLMs.

Title: Few-Shot Multilingual Open-Domain QA from 5 Examples

• Authors: Fan Jiang, Tom Drummond, Trevor Cohn
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.19722
• Pdf URL: https://arxiv.org/pdf/2502.19722
• Copy Paste: [[2502.19722]] Few-Shot Multilingual Open-Domain QA from 5 Examples(https://arxiv.org/abs/2502.19722)
• Keywords: large language model
• Abstract: Recent approaches to multilingual open-domain question answering (MLODQA) have achieved promising results given abundant language-specific training data. However, the considerable annotation cost limits the application of these methods for underrepresented languages. We introduce a \emph{few-shot learning} approach to synthesise large-scale multilingual data from large language models (LLMs). Our method begins with large-scale self-supervised pre-training using WikiData, followed by training on high-quality synthetic multilingual data generated by prompting LLMs with few-shot supervision. The final model, \textsc{FsModQA}, significantly outperforms existing few-shot and supervised baselines in MLODQA and cross-lingual and monolingual retrieval. We further show our method can be extended for effective zero-shot adaptation to new languages through a \emph{cross-lingual prompting} strategy with only English-supervised data, making it a general and applicable solution for MLODQA tasks without costly large-scale annotation.

Title: CNsum:Automatic Summarization for Chinese News Text

• Authors: Yu Zhao, Songping Huang, Dongsheng Zhou, Zhaoyun Ding, Fei Wang, Aixin Nian
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19723
• Pdf URL: https://arxiv.org/pdf/2502.19723
• Copy Paste: [[2502.19723]] CNsum:Automatic Summarization for Chinese News Text(https://arxiv.org/abs/2502.19723)
• Keywords: transformer
• Abstract: Obtaining valuable information from massive data efficiently has become our research goal in the era of Big Data. Text summarization technology has been continuously developed to meet this demand. Recent work has also shown that transformer-based pre-trained language models have achieved great success on various tasks in Natural Language Processing (NLP). Aiming at the problem of Chinese news text summary generation and the application of Transformer structure on Chinese, this paper proposes a Chinese news text summarization model (CNsum) based on Transformer structure, and tests it on Chinese datasets such as THUCNews. The results of the conducted experiments show that CNsum achieves better ROUGE score than the baseline models, which verifies the outperformance of the model.

Title: Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training

• Authors: Toan Tran, Ruixuan Liu, Li Xiong
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19726
• Pdf URL: https://arxiv.org/pdf/2502.19726
• Copy Paste: [[2502.19726]] Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training(https://arxiv.org/abs/2502.19726)
• Keywords: privacy, protect, defense, attack, membership infer, large language model
• Abstract: Large language models (LLMs) have become the backbone of modern natural language processing but pose privacy concerns about leaking sensitive training data. Membership inference attacks (MIAs), which aim to infer whether a sample is included in a model's training dataset, can serve as a foundation for broader privacy threats. Existing defenses designed for traditional classification models do not account for the sequential nature of text data. As a result, they either require significant computational resources or fail to effectively mitigate privacy risks in LLMs. In this work, we propose a lightweight yet effective empirical privacy defense for protecting training data of language modeling by leveraging the token-specific characteristics. By analyzing token dynamics during training, we propose a token selection strategy that categorizes tokens into hard tokens for learning and memorized tokens for unlearning. Subsequently, our training-phase defense optimizes a novel dual-purpose token-level loss to achieve a Pareto-optimal balance between utility and privacy. Extensive experiments demonstrate that our approach not only provides strong protection against MIAs but also improves language modeling performance by around 10\% across various LLM architectures and datasets compared to the baselines.

Title: Preference Learning Unlocks LLMs' Psycho-Counseling Skills

• Authors: Mian Zhang, Shaun M. Eack, Zhiyu Zoey Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19731
• Pdf URL: https://arxiv.org/pdf/2502.19731
• Copy Paste: [[2502.19731]] Preference Learning Unlocks LLMs' Psycho-Counseling Skills(https://arxiv.org/abs/2502.19731)
• Keywords: privacy, robust, large language model
• Abstract: Applying large language models (LLMs) to assist in psycho-counseling is an emerging and meaningful approach, driven by the significant gap between patient needs and the availability of mental health support. However, current LLMs struggle to consistently provide effective responses to client speeches, largely due to the lack of supervision from high-quality real psycho-counseling data, whose content is typically inaccessible due to client privacy concerns. Furthermore, the quality of therapists' responses in available sessions can vary significantly based on their professional training and experience. Assessing the quality of therapists' responses remains an open challenge. In this work, we address these challenges by first proposing a set of professional and comprehensive principles to evaluate therapists' responses to client speeches. Using these principles, we create a preference dataset, PsychoCounsel-Preference, which contains 36k high-quality preference comparison pairs. This dataset aligns with the preferences of professional psychotherapists, providing a robust foundation for evaluating and improving LLMs in psycho-counseling. Experiments on reward modeling and preference learning demonstrate that PsychoCounsel-Preference is an excellent resource for LLMs to acquire essential skills for responding to clients in a counseling session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference, PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to facilitate the research of psycho-counseling with LLMs at: this https URL.

Title: R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning

• Authors: Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, Hao Yang, Boxing Chen, Osamu Yoshie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19735
• Pdf URL: https://arxiv.org/pdf/2502.19735
• Copy Paste: [[2502.19735]] R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning(https://arxiv.org/abs/2502.19735)
• Keywords: large language model
• Abstract: Despite recent breakthroughs in reasoning-enhanced large language models (LLMs) like DeepSeek-R1, incorporating inference-time reasoning into machine translation (MT), where human translators naturally employ structured, multi-layered reasoning chain-of-thoughts (CoTs), is yet underexplored. Existing methods either design a fixed CoT tailored for a specific MT sub-task (e.g., literature translation), or rely on synthesizing CoTs unaligned with humans and supervised fine-tuning (SFT) prone to catastrophic forgetting, limiting their adaptability to diverse translation scenarios. This paper introduces R1-Translator (R1-T1), a novel framework to achieve inference-time reasoning for general MT via reinforcement learning (RL) with human-aligned CoTs comprising six common patterns. Our approach pioneers three innovations: (1) extending reasoning-based translation beyond MT sub-tasks to six languages and diverse tasks (e.g., legal/medical domain adaptation, idiom resolution); (2) formalizing six expert-curated CoT templates that mirror hybrid human strategies like context-aware paraphrasing and back translation; and (3) enabling self-evolving CoT discovery and anti-forgetting adaptation through RL with KL-constrained rewards. Experimental results indicate a steady translation performance improvement in 21 languages and 80 translation directions on Flores-101 test set, especially on the 15 languages unseen from training, with its general multilingual abilities preserved compared with plain SFT.

Title: HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture

• Authors: Taiqiang Wu, Chenchen Ding, Wenyong Zhou, Yuxin Cheng, Xincheng Feng, Shuqi Wang, Chufan Shi, Zhengwu Liu, Ngai Wong
• Subjects: cs.CL, cs.AR
• Abstract URL: https://arxiv.org/abs/2502.19747
• Pdf URL: https://arxiv.org/pdf/2502.19747
• Copy Paste: [[2502.19747]] HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture(https://arxiv.org/abs/2502.19747)
• Keywords: robust, large language model
• Abstract: Low-rank adaptation (LoRA) is a predominant parameter-efficient finetuning method to adapt large language models (LLMs) for downstream tasks. In this paper, we first propose to deploy the LoRA-finetuned LLMs on the hybrid compute-in-memory (CIM) architecture (i.e., pretrained weights onto RRAM and LoRA onto SRAM). To address performance degradation from RRAM's inherent noise, we design a novel Hardware-aware Low-rank Adaption (HaLoRA) method, aiming to train a LoRA branch that is both robust and accurate by aligning the training objectives under both ideal and noisy conditions. Experiments finetuning LLaMA 3.2 1B and 3B demonstrate HaLoRA's effectiveness across multiple reasoning tasks, achieving up to 22.7 improvement in average score while maintaining robustness at various noise levels.

Title: Beneath the Surface: How Large Language Models Reflect Hidden Bias

• Authors: Jinhao Pan, Chahat Raj, Ziyu Yao, Ziwei Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19749
• Pdf URL: https://arxiv.org/pdf/2502.19749
• Copy Paste: [[2502.19749]] Beneath the Surface: How Large Language Models Reflect Hidden Bias(https://arxiv.org/abs/2502.19749)
• Keywords: large language model
• Abstract: The exceptional performance of Large Language Models (LLMs) often comes with the unintended propagation of social biases embedded in their training data. While existing benchmarks evaluate overt bias through direct term associations between bias concept terms and demographic terms, LLMs have become increasingly adept at avoiding biased responses, creating an illusion of neutrality. However, biases persist in subtler, contextually hidden forms that traditional benchmarks fail to capture. We introduce the Hidden Bias Benchmark (HBB), a novel dataset designed to assess hidden bias that bias concepts are hidden within naturalistic, subtly framed contexts in real-world scenarios. We analyze six state-of-the-art LLMs, revealing that while models reduce bias in response to overt bias, they continue to reinforce biases in nuanced settings. Data, code, and results are available at this https URL.

Title: CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer

• Authors: Yang Liu, Zinan Zheng, Jiashun Cheng, Fugee Tsung, Deli Zhao, Yu Rong, Jia Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19750
• Pdf URL: https://arxiv.org/pdf/2502.19750
• Copy Paste: [[2502.19750]] CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer(https://arxiv.org/abs/2502.19750)
• Keywords: transformer
• Abstract: Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.

Title: Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data

• Authors: Pei-Yau Weng, Minh Hoang, Lam M. Nguyen, My T. Thai, Tsui-Wei Weng, Trong Nghia Hoang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19752
• Pdf URL: https://arxiv.org/pdf/2502.19752
• Copy Paste: [[2502.19752]] Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data(https://arxiv.org/abs/2502.19752)
• Keywords: federate
• Abstract: Fine-tuning pre-trained models is a popular approach in machine learning for solving complex tasks with moderate data. However, fine-tuning the entire pre-trained model is ineffective in federated data scenarios where local data distributions are diversely skewed. To address this, we explore integrating federated learning with a more effective prompt-tuning method, optimizing for a small set of input prefixes to reprogram the pre-trained model's behavior. Our approach transforms federated learning into a distributed set modeling task, aggregating diverse sets of prompts to globally fine-tune the pre-trained model. We benchmark various baselines based on direct adaptations of existing federated model aggregation techniques and introduce a new probabilistic prompt aggregation method that substantially outperforms these baselines. Our reported results on a variety of computer vision datasets confirm that the proposed method is most effective to combat extreme data heterogeneity in federated learning.

Title: Finding Local Diffusion Schrödinger Bridge using Kolmogorov-Arnold Network

• Authors: Xingyu Qiu, Mengying Yang, Xinghua Ma, Fanding Li, Dong Liang, Gongning Luo, Wei Wang, Kuanquan Wang, Shuo Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19754
• Pdf URL: https://arxiv.org/pdf/2502.19754
• Copy Paste: [[2502.19754]] Finding Local Diffusion Schrödinger Bridge using Kolmogorov-Arnold Network(https://arxiv.org/abs/2502.19754)
• Keywords: diffusion
• Abstract: In image generation, Schrödinger Bridge (SB)-based methods theoretically enhance the efficiency and quality compared to the diffusion models by finding the least costly path between two distributions. However, they are computationally expensive and time-consuming when applied to complex image data. The reason is that they focus on fitting globally optimal paths in high-dimensional spaces, directly generating images as next step on the path using complex networks through self-supervised training, which typically results in a gap with the global optimum. Meanwhile, most diffusion models are in the same path subspace generated by weights $f_A(t)$ and $f_B(t)$, as they follow the paradigm ($x_t = f_A(t)x_{Img} + f_B(t)\epsilon$). To address the limitations of SB-based methods, this paper proposes for the first time to find local Diffusion Schrödinger Bridges (LDSB) in the diffusion path subspace, which strengthens the connection between the SB problem and diffusion models. Specifically, our method optimizes the diffusion paths using Kolmogorov-Arnold Network (KAN), which has the advantage of resistance to forgetting and continuous output. The experiment shows that our LDSB significantly improves the quality and efficiency of image generation using the same pre-trained denoising network and the KAN for optimising is only less than 0.1MB. The FID metric is reduced by \textbf{more than 15\%}, especially with a reduction of 48.50\% when NFE of DDIM is $5$ for the CelebA dataset. Code is available at this https URL.

Title: HALO: Robust Out-of-Distribution Detection via Joint Optimisation

• Authors: Hugo Lyons Keenan, Sarah Erfani, Christopher Leckie
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19755
• Pdf URL: https://arxiv.org/pdf/2502.19755
• Copy Paste: [[2502.19755]] HALO: Robust Out-of-Distribution Detection via Joint Optimisation(https://arxiv.org/abs/2502.19755)
• Keywords: attack, robust
• Abstract: Effective out-of-distribution (OOD) detection is crucial for the safe deployment of machine learning models in real-world scenarios. However, recent work has shown that OOD detection methods are vulnerable to adversarial attacks, potentially leading to critical failures in high-stakes applications. This discovery has motivated work on robust OOD detection methods that are capable of maintaining performance under various attack settings. Prior approaches have made progress on this problem but face a number of limitations: often only exhibiting robustness to attacks on OOD data or failing to maintain strong clean performance. In this work, we adapt an existing robust classification framework, TRADES, extending it to the problem of robust OOD detection and discovering a novel objective function. Recognising the critical importance of a strong clean/robust trade-off for OOD detection, we introduce an additional loss term which boosts classification and detection performance. Our approach, called HALO (Helper-based AdversariaL OOD detection), surpasses existing methods and achieves state-of-the-art performance across a number of datasets and attack settings. Extensive experiments demonstrate an average AUROC improvement of 3.15 in clean settings and 7.07 under adversarial attacks when compared to the next best method. Furthermore, HALO exhibits resistance to transferred attacks, offers tuneable performance through hyperparameter selection, and is compatible with existing OOD detection frameworks out-of-the-box, leaving open the possibility of future performance gains. Code is available at: this https URL

Title: PolyPrompt: Automating Knowledge Extraction from Multilingual Language Models with Dynamic Prompt Generation

• Authors: Nathan Roll
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19756
• Pdf URL: https://arxiv.org/pdf/2502.19756
• Copy Paste: [[2502.19756]] PolyPrompt: Automating Knowledge Extraction from Multilingual Language Models with Dynamic Prompt Generation(https://arxiv.org/abs/2502.19756)
• Keywords: extraction, large language model
• Abstract: Large language models (LLMs) showcase increasingly impressive English benchmark scores, however their performance profiles remain inconsistent across multilingual settings. To address this gap, we introduce PolyPrompt, a novel, parameter-efficient framework for enhancing the multilingual capabilities of LLMs. Our method learns a set of trigger tokens for each language through a gradient-based search, identifying the input query's language and selecting the corresponding trigger tokens which are prepended to the prompt during inference. We perform experiments on two ~1 billion parameter models, with evaluations on the global MMLU benchmark across fifteen typologically and resource diverse languages, demonstrating accuracy gains of 3.7%-19.9% compared to naive and translation-pipeline baselines.

Title: Snowball Adversarial Attack on Traffic Sign Classification

• Authors: Anthony Etim, Jakub Szefer
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2502.19757
• Pdf URL: https://arxiv.org/pdf/2502.19757
• Copy Paste: [[2502.19757]] Snowball Adversarial Attack on Traffic Sign Classification(https://arxiv.org/abs/2502.19757)
• Keywords: defense, attack, robust
• Abstract: Adversarial attacks on machine learning models often rely on small, imperceptible perturbations to mislead classifiers. Such strategy focuses on minimizing the visual perturbation for humans so they are not confused, and also maximizing the misclassification for machine learning algorithms. An orthogonal strategy for adversarial attacks is to create perturbations that are clearly visible but do not confuse humans, yet still maximize misclassification for machine learning algorithms. This work follows the later strategy, and demonstrates instance of it through the Snowball Adversarial Attack in the context of traffic sign recognition. The attack leverages the human brain's superior ability to recognize objects despite various occlusions, while machine learning algorithms are easily confused. The evaluation shows that the Snowball Adversarial Attack is robust across various images and is able to confuse state-of-the-art traffic sign recognition algorithm. The findings reveal that Snowball Adversarial Attack can significantly degrade model performance with minimal effort, raising important concerns about the vulnerabilities of deep neural networks and highlighting the necessity for improved defenses for image recognition machine learning models.

Title: EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models

• Authors: Che Hyun Lee, Heeseung Kim, Jiheum Yeom, Sungroh Yoon
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19765
• Pdf URL: https://arxiv.org/pdf/2502.19765
• Copy Paste: [[2502.19765]] EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models(https://arxiv.org/abs/2502.19765)
• Keywords: robust, diffusion
• Abstract: We propose EdiText, a controllable text editing method that modify the reference text to desired attributes at various scales. We integrate an SDEdit-based editing technique that allows for broad adjustments in the degree of text editing. Additionally, we introduce a novel fine-level editing method based on self-conditioning, which allows subtle control of reference text. While being capable of editing on its own, this fine-grained method, integrated with the SDEdit approach, enables EdiText to make precise adjustments within the desired range. EdiText demonstrates its controllability to robustly adjust reference text at broad range of levels across various tasks, including toxicity control and sentiment control.

Title: Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint Detection and Temporal Segmentation Approach for Small Datasets

• Authors: Jisoo Lee, Tamim Ahmed, Thanassis Rikakis, Pavan Turaga
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19766
• Pdf URL: https://arxiv.org/pdf/2502.19766
• Copy Paste: [[2502.19766]] Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint Detection and Temporal Segmentation Approach for Small Datasets(https://arxiv.org/abs/2502.19766)
• Keywords: segmentation
• Abstract: Rehabilitation is essential and critical for post-stroke patients, addressing both physical and cognitive aspects. Stroke predominantly affects older adults, with 75% of cases occurring in individuals aged 65 and older, underscoring the urgent need for tailored rehabilitation strategies in aging populations. Despite the critical role therapists play in evaluating rehabilitation progress and ensuring the effectiveness of treatment, current assessment methods can often be subjective, inconsistent, and time-consuming, leading to delays in adjusting therapy protocols. This study aims to address these challenges by providing a solution for consistent and timely analysis. Specifically, we perform temporal segmentation of video recordings to capture detailed activities during stroke patients' rehabilitation. The main application scenario motivating this study is the clinical assessment of daily tabletop object interactions, which are crucial for post-stroke physical rehabilitation. To achieve this, we present a framework that leverages the biomechanics of movement during therapy sessions. Our solution divides the process into two main tasks: 2D keypoint detection to track patients' physical movements, and 1D time-series temporal segmentation to analyze these movements over time. This dual approach enables automated labeling with only a limited set of real-world data, addressing the challenges of variability in patient movements and limited dataset availability. By tackling these issues, our method shows strong potential for practical deployment in physical therapy settings, enhancing the speed and accuracy of rehabilitation assessments.

Title: QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects

• Authors: Elkhan Ismayilzada, MD Khalequzzaman Chowdhury Sayem, Yihalem Yimolal Tiruneh, Mubarrat Tajoar Chowdhury, Muhammadjon Boboev, Seungryul Baek
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19769
• Pdf URL: https://arxiv.org/pdf/2502.19769
• Copy Paste: [[2502.19769]] QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects(https://arxiv.org/abs/2502.19769)
• Keywords: secure, transformer
• Abstract: Significant advancements have been achieved in the realm of understanding poses and interactions of two hands manipulating an object. The emergence of augmented reality (AR) and virtual reality (VR) technologies has heightened the demand for real-time performance in these applications. However, current state-of-the-art models often exhibit promising results at the expense of substantial computational overhead. In this paper, we present a query-optimized real-time Transformer (QORT-Former), the first Transformer-based real-time framework for 3D pose estimation of two hands and an object. We first limit the number of queries and decoders to meet the efficiency requirement. Given limited number of queries and decoders, we propose to optimize queries which are taken as input to the Transformer decoder, to secure better accuracy: (1) we propose to divide queries into three types (a left hand query, a right hand query and an object query) and enhance query features (2) by using the contact information between hands and an object and (3) by using three-step update of enhanced image and query features with respect to one another. With proposed methods, we achieved real-time pose estimation performance using just 108 queries and 1 decoder (53.5 FPS on an RTX 3090TI GPU). Surpassing state-of-the-art results on the H2O dataset by 17.6% (left hand), 22.8% (right hand), and 27.2% (object), as well as on the FPHA dataset by 5.3% (right hand) and 10.4% (object), our method excels in accuracy. Additionally, it sets the state-of-the-art in interaction recognition, maintaining real-time efficiency with an off-the-shelf action recognition module.

Title: TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning

• Authors: Weiqi Wang, Zhiyi Tian, An Liu, Shui Yu
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19770
• Pdf URL: https://arxiv.org/pdf/2502.19770
• Copy Paste: [[2502.19770]] TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning(https://arxiv.org/abs/2502.19770)
• Keywords: privacy
• Abstract: With the increasing prevalence of Web-based platforms handling vast amounts of user data, machine unlearning has emerged as a crucial mechanism to uphold users' right to be forgotten, enabling individuals to request the removal of their specified data from trained models. However, the auditing of machine unlearning processes remains significantly underexplored. Although some existing methods offer unlearning auditing by leveraging backdoors, these backdoor-based approaches are inefficient and impractical, as they necessitate involvement in the initial model training process to embed the backdoors. In this paper, we propose a TAilored Posterior diffErence (TAPE) method to provide unlearning auditing independently of original model training. We observe that the process of machine unlearning inherently introduces changes in the model, which contains information related to the erased data. TAPE leverages unlearning model differences to assess how much information has been removed through the unlearning operation. Firstly, TAPE mimics the unlearned posterior differences by quickly building unlearned shadow models based on first-order influence estimation. Secondly, we train a Reconstructor model to extract and evaluate the private information of the unlearned posterior differences to audit unlearning. Existing privacy reconstructing methods based on posterior differences are only feasible for model updates of a single sample. To enable the reconstruction effective for multi-sample unlearning requests, we propose two strategies, unlearned data perturbation and unlearned influence-based division, to augment the posterior difference. Extensive experimental results indicate the significant superiority of TAPE over the state-of-the-art unlearning verification methods, at least 4.5$\times$ efficiency speedup and supporting the auditing for broader unlearning scenarios.

Title: Do Retrieval-Augmented Language Models Adapt to Varying User Needs?

• Authors: Peilin Wu, Xinlu Zhang, Wenhao Yu, Xingyu Liu, Xinya Du, Zhiyu Zoey Chen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19779
• Pdf URL: https://arxiv.org/pdf/2502.19779
• Copy Paste: [[2502.19779]] Do Retrieval-Augmented Language Models Adapt to Varying User Needs?(https://arxiv.org/abs/2502.19779)
• Keywords: robust
• Abstract: Recent advancements in Retrieval-Augmented Language Models (RALMs) have demonstrated their efficacy in knowledge-intensive tasks. However, existing evaluation benchmarks often assume a single optimal approach to leveraging retrieved information, failing to account for varying user needs. This paper introduces a novel evaluation framework that systematically assesses RALMs under three user need cases-Context-Exclusive, Context-First, and Memory-First-across three distinct context settings: Context Matching, Knowledge Conflict, and Information Irrelevant. By varying both user instructions and the nature of retrieved information, our approach captures the complexities of real-world applications where models must adapt to diverse user requirements. Through extensive experiments on multiple QA datasets, including HotpotQA, DisentQA, and our newly constructed synthetic URAQ dataset, we find that restricting memory usage improves robustness in adversarial retrieval conditions but decreases peak performance with ideal retrieval results and model family dominates behavioral differences. Our findings highlight the necessity of user-centric evaluations in the development of retrieval-augmented systems and provide insights into optimizing model performance across varied retrieval contexts. We will release our code and URAQ dataset upon acceptance of the paper.

Title: Open-Vocabulary Semantic Part Segmentation of 3D Human

• Authors: Keito Suzuki, Bang Du, Girish Krishnan, Kunyao Chen, Runfa Blark Li, Truong Nguyen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19782
• Pdf URL: https://arxiv.org/pdf/2502.19782
• Copy Paste: [[2502.19782]] Open-Vocabulary Semantic Part Segmentation of 3D Human(https://arxiv.org/abs/2502.19782)
• Keywords: segmentation
• Abstract: 3D part segmentation is still an open problem in the field of 3D vision and AR/VR. Due to limited 3D labeled data, traditional supervised segmentation methods fall short in generalizing to unseen shapes and categories. Recently, the advancement in vision-language models' zero-shot abilities has brought a surge in open-world 3D segmentation methods. While these methods show promising results for 3D scenes or objects, they do not generalize well to 3D humans. In this paper, we present the first open-vocabulary segmentation method capable of handling 3D human. Our framework can segment the human category into desired fine-grained parts based on the textual prompt. We design a simple segmentation pipeline, leveraging SAM to generate multi-view proposals in 2D and proposing a novel HumanCLIP model to create unified embeddings for visual and textual inputs. Compared with existing pre-trained CLIP models, the HumanCLIP model yields more accurate embeddings for human-centric contents. We also design a simple-yet-effective MaskFusion module, which classifies and fuses multi-view features into 3D semantic masks without complex voting and grouping mechanisms. The design of decoupling mask proposals and text input also significantly boosts the efficiency of per-prompt inference. Experimental results on various 3D human datasets show that our method outperforms current state-of-the-art open-vocabulary 3D segmentation methods by a large margin. In addition, we show that our method can be directly applied to various 3D representations including meshes, point clouds, and 3D Gaussian Splatting.

Title: In-Context Learning with Hypothesis-Class Guidance

• Authors: Ziqian Lin, Shubham Kumar Bharti, Kangwook Lee
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19787
• Pdf URL: https://arxiv.org/pdf/2502.19787
• Copy Paste: [[2502.19787]] In-Context Learning with Hypothesis-Class Guidance(https://arxiv.org/abs/2502.19787)
• Keywords: transformer
• Abstract: Recent research has investigated the underlying mechanisms of in-context learning (ICL) both theoretically and empirically, often using data generated from simple function classes. However, the existing work often focuses on the sequence consisting solely of labeled examples, while in practice, labeled examples are typically accompanied by an instruction, providing some side information about the task. In this work, we propose ICL with hypothesis-class guidance (ICL-HCG), a novel synthetic data model for ICL where the input context consists of the literal description of a (finite) hypothesis class $\mathcal{H}$ and $(x,y)$ pairs from a hypothesis chosen from $\mathcal{H}$. Under our framework ICL-HCG, we conduct extensive experiments to explore: (i) a variety of generalization abilities to new hypothesis classes; (ii) different model architectures; (iii) sample complexity; (iv) in-context data imbalance; (v) the role of instruction; and (vi) the effect of pretraining hypothesis diversity. As a result, we show that (a) Transformers can successfully learn ICL-HCG and generalize to unseen hypotheses and unseen hypothesis classes, and (b) compared with ICL without instruction, ICL-HCG achieves significantly higher accuracy, demonstrating the role of instructions.

Title: MFSR: Multi-fractal Feature for Super-resolution Reconstruction with Fine Details Recovery

• Authors: Lianping Yang, Peng Jiao, Jinshan Pan, Hegui Zhu, Su Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19797
• Pdf URL: https://arxiv.org/pdf/2502.19797
• Copy Paste: [[2502.19797]] MFSR: Multi-fractal Feature for Super-resolution Reconstruction with Fine Details Recovery(https://arxiv.org/abs/2502.19797)
• Keywords: diffusion
• Abstract: In the process of performing image super-resolution processing, the processing of complex localized information can have a significant impact on the quality of the image generated. Fractal features can capture the rich details of both micro and macro texture structures in an image. Therefore, we propose a diffusion model-based super-resolution method incorporating fractal features of low-resolution images, named MFSR. MFSR leverages these fractal features as reinforcement conditions in the denoising process of the diffusion model to ensure accurate recovery of texture information. MFSR employs convolution as a soft assignment to approximate the fractal features of low-resolution images. This approach is also used to approximate the density feature maps of these images. By using soft assignment, the spatial layout of the image is described hierarchically, encoding the self-similarity properties of the image at different scales. Different processing methods are applied to various types of features to enrich the information acquired by the model. In addition, a sub-denoiser is integrated in the denoising U-Net to reduce the noise in the feature maps during the up-sampling process in order to improve the quality of the generated images. Experiments conducted on various face and natural image datasets demonstrate that MFSR can generate higher quality images.

Title: UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition

• Authors: Xiao Lin, Yuge Huang, Jianqing Xu, Yuxi Mi, Shuigeng Zhou, Shouhong Ding
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19803
• Pdf URL: https://arxiv.org/pdf/2502.19803
• Copy Paste: [[2502.19803]] UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition(https://arxiv.org/abs/2502.19803)
• Keywords: privacy, diffusion
• Abstract: Face recognition (FR) stands as one of the most crucial applications in computer vision. The accuracy of FR models has significantly improved in recent years due to the availability of large-scale human face datasets. However, directly using these datasets can inevitably lead to privacy and legal problems. Generating synthetic data to train FR models is a feasible solution to circumvent these issues. While existing synthetic-based face recognition methods have made significant progress in generating identity-preserving images, they are severely plagued by context overfitting, resulting in a lack of intra-class diversity of generated images and poor face recognition performance. In this paper, we propose a framework to Unleash Inherent capability of the model to enhance intra-class diversity for synthetic face recognition, shortened as UIFace. Our framework first trains a diffusion model that can perform sampling conditioned on either identity contexts or a learnable empty context. The former generates identity-preserving images but lacks variations, while the latter exploits the model's intrinsic ability to synthesize intra-class-diversified images but with random identities. Then we adopt a novel two-stage sampling strategy during inference to fully leverage the strengths of both types of contexts, resulting in images that are diverse as well as identitypreserving. Moreover, an attention injection module is introduced to further augment the intra-class variations by utilizing attention maps from the empty context to guide the sampling process in ID-conditioned generation. Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset. The proposed UIFace even achieves comparable performance with FR models trained on real datasets when we further increase the number of synthetic identities.

Title: Implicit Search via Discrete Diffusion: A Study on Chess

• Authors: Jiacheng Ye, Zhenyu Wu, Jiahui Gao, Zhiyong Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19805
• Pdf URL: https://arxiv.org/pdf/2502.19805
• Copy Paste: [[2502.19805]] Implicit Search via Discrete Diffusion: A Study on Chess(https://arxiv.org/abs/2502.19805)
• Keywords: diffusion, large language model
• Abstract: In the post-AlphaGo era, there has been a renewed interest in search techniques such as Monte Carlo Tree Search (MCTS), particularly in their application to Large Language Models (LLMs). This renewed attention is driven by the recognition that current next-token prediction models often lack the ability for long-term planning. Is it possible to instill search-like abilities within the models to enhance their planning abilities without relying on explicit search? We propose DiffuSearch , a model that does \textit{implicit search} by looking into the future world via discrete diffusion modeling. We instantiate DiffuSearch on a classical board game, Chess, where explicit search is known to be essential. Through extensive controlled experiments, we show DiffuSearch outperforms both the searchless and explicit search-enhanced policies. Specifically, DiffuSearch outperforms the one-step policy by 19.2% and the MCTS-enhanced policy by 14% on action accuracy. Furthermore, DiffuSearch demonstrates a notable 30% enhancement in puzzle-solving abilities compared to explicit search-based policies, along with a significant 540 Elo increase in game-playing strength assessment. These results indicate that implicit search via discrete diffusion is a viable alternative to explicit search over a one-step policy. All codes are publicly available at \href{this https URL}{this https URL}.

Title: Twofold Debiasing Enhances Fine-Grained Learning with Coarse Labels

• Authors: Xin-yang Zhao, Jian Jin, Yang-yang Li, Yazhou Yao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19816
• Pdf URL: https://arxiv.org/pdf/2502.19816
• Copy Paste: [[2502.19816]] Twofold Debiasing Enhances Fine-Grained Learning with Coarse Labels(https://arxiv.org/abs/2502.19816)
• Keywords: extraction
• Abstract: The Coarse-to-Fine Few-Shot (C2FS) task is designed to train models using only coarse labels, then leverages a limited number of subclass samples to achieve fine-grained recognition capabilities. This task presents two main challenges: coarse-grained supervised pre-training suppresses the extraction of critical fine-grained features for subcategory discrimination, and models suffer from overfitting due to biased distributions caused by limited fine-grained samples. In this paper, we propose the Twofold Debiasing (TFB) method, which addresses these challenges through detailed feature enhancement and distribution calibration. Specifically, we introduce a multi-layer feature fusion reconstruction module and an intermediate layer feature alignment module to combat the model's tendency to focus on simple predictive features directly related to coarse-grained supervision, while neglecting complex fine-grained level details. Furthermore, we mitigate the biased distributions learned by the fine-grained classifier using readily available coarse-grained sample embeddings enriched with fine-grained information. Extensive experiments conducted on five benchmark datasets demonstrate the efficacy of our approach, achieving state-of-the-art results that surpass competitive methods.

Title: Foot-In-The-Door: A Multi-turn Jailbreak for LLMs

• Authors: Zixuan Weng, Xiaolong Jin, Jinyuan Jia, Xiangyu Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19820
• Pdf URL: https://arxiv.org/pdf/2502.19820
• Copy Paste: [[2502.19820]] Foot-In-The-Door: A Multi-turn Jailbreak for LLMs(https://arxiv.org/abs/2502.19820)
• Keywords: attack, large language model
• Abstract: Ensuring AI safety is crucial as large language models become increasingly integrated into real-world applications. A key challenge is jailbreak, where adversarial prompts bypass built-in safeguards to elicit harmful disallowed outputs. Inspired by psychological foot-in-the-door principles, we introduce FITD,a novel multi-turn jailbreak method that leverages the phenomenon where minor initial commitments lower resistance to more significant or more unethical this http URL approach progressively escalates the malicious intent of user queries through intermediate bridge prompts and aligns the model's response by itself to induce toxic responses. Extensive experimental results on two jailbreak benchmarks demonstrate that FITD achieves an average attack success rate of 94% across seven widely used models, outperforming existing state-of-the-art methods. Additionally, we provide an in-depth analysis of LLM self-corruption, highlighting vulnerabilities in current alignment strategies and emphasizing the risks inherent in multi-turn this http URL code is available at this https URL .

Title: Analyzing CLIP's Performance Limitations in Multi-Object Scenarios: A Controlled High-Resolution Study

• Authors: Reza Abbasi, Ali Nazari, Aminreza Sefid, Mohammadali Banayeeanzade, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19828
• Pdf URL: https://arxiv.org/pdf/2502.19828
• Copy Paste: [[2502.19828]] Analyzing CLIP's Performance Limitations in Multi-Object Scenarios: A Controlled High-Resolution Study(https://arxiv.org/abs/2502.19828)
• Keywords: diffusion
• Abstract: Contrastive Language-Image Pre-training (CLIP) models have demonstrated remarkable performance in zero-shot classification tasks, yet their efficacy in handling complex multi-object scenarios remains challenging. This study presents a comprehensive analysis of CLIP's performance limitations in multi-object contexts through controlled experiments. We introduce two custom datasets, SimCO and CompCO, to evaluate CLIP's image and text encoders in various multi-object configurations. Our findings reveal significant biases in both encoders: the image encoder favors larger objects, while the text encoder prioritizes objects mentioned first in descriptions. We hypothesize these biases originate from CLIP's training process and provide evidence through analyses of the COCO dataset and CLIP's training progression. Additionally, we extend our investigation to Stable Diffusion models, revealing that biases in the CLIP text encoder significantly impact text-to-image generation tasks. Our experiments demonstrate how these biases affect CLIP's performance in image-caption matching and generation tasks, particularly when manipulating object sizes and their order in captions. This work contributes valuable insights into CLIP's behavior in complex visual environments and highlights areas for improvement in future vision-language models.

Title: Knowledge Bridger: Towards Training-free Missing Multi-modality Completion

• Authors: Guanzhou Ke, Shengfeng He, Xiao Li Wang, Bo Wang, Guoqing Chao, Yuanyang Zhang, Yi Xie, HeXing Su
• Subjects: cs.LG, cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.19834
• Pdf URL: https://arxiv.org/pdf/2502.19834
• Copy Paste: [[2502.19834]] Knowledge Bridger: Towards Training-free Missing Multi-modality Completion(https://arxiv.org/abs/2502.19834)
• Keywords: robust
• Abstract: Previous successful approaches to missing modality completion rely on carefully designed fusion techniques and extensive pre-training on complete data, which can limit their generalizability in out-of-domain (OOD) scenarios. In this study, we pose a new challenge: can we develop a missing modality completion model that is both resource-efficient and robust to OOD generalization? To address this, we present a training-free framework for missing modality completion that leverages large multimodal models (LMMs). Our approach, termed the "Knowledge Bridger", is modality-agnostic and integrates generation and ranking of missing modalities. By defining domain-specific priors, our method automatically extracts structured information from available modalities to construct knowledge graphs. These extracted graphs connect the missing modality generation and ranking modules through the LMM, resulting in high-quality imputations of missing modalities. Experimental results across both general and medical domains show that our approach consistently outperforms competing methods, including in OOD generalization. Additionally, our knowledge-driven generation and ranking techniques demonstrate superiority over variants that directly employ LMMs for generation and ranking, offering insights that may be valuable for applications in other domains.

Title: CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation

• Authors: Reza Abbasi, Ali Nazari, Aminreza Sefid, Mohammadali Banayeeanzade, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19842
• Pdf URL: https://arxiv.org/pdf/2502.19842
• Copy Paste: [[2502.19842]] CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation(https://arxiv.org/abs/2502.19842)
• Keywords: diffusion
• Abstract: Contrastive Language-Image Pre-training (CLIP) models excel in zero-shot classification, yet face challenges in complex multi-object scenarios. This study offers a comprehensive analysis of CLIP's limitations in these contexts using a specialized dataset, ComCO, designed to evaluate CLIP's encoders in diverse multi-object scenarios. Our findings reveal significant biases: the text encoder prioritizes first-mentioned objects, and the image encoder favors larger objects. Through retrieval and classification tasks, we quantify these biases across multiple CLIP variants and trace their origins to CLIP's training process, supported by analyses of the LAION dataset and training progression. Our image-text matching experiments show substantial performance drops when object size or token order changes, underscoring CLIP's instability with rephrased but semantically similar captions. Extending this to longer captions and text-to-image models like Stable Diffusion, we demonstrate how prompt order influences object prominence in generated images. For more details and access to our dataset and analysis code, visit our project repository: this https URL.

Title: ProAPO: Progressively Automatic Prompt Optimization for Visual Classification

• Authors: Xiangyan Qu, Gaopeng Gou, Jiamin Zhuang, Jing Yu, Kun Song, Qihao Wang, Yili Li, Gang Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19844
• Pdf URL: https://arxiv.org/pdf/2502.19844
• Copy Paste: [[2502.19844]] ProAPO: Progressively Automatic Prompt Optimization for Visual Classification(https://arxiv.org/abs/2502.19844)
• Keywords: large language model
• Abstract: Vision-language models (VLMs) have made significant progress in image classification by training with large-scale paired image-text data. Their performances largely depend on the prompt quality. While recent methods show that visual descriptions generated by large language models (LLMs) enhance the generalization of VLMs, class-specific prompts may be inaccurate or lack discrimination due to the hallucination in LLMs. In this paper, we aim to find visually discriminative prompts for fine-grained categories with minimal supervision and no human-in-the-loop. An evolution-based algorithm is proposed to progressively optimize language prompts from task-specific templates to class-specific descriptions. Unlike optimizing templates, the search space shows an explosion in class-specific candidate prompts. This increases prompt generation costs, iterative times, and the overfitting problem. To this end, we first introduce several simple yet effective edit-based and evolution-based operations to generate diverse candidate prompts by one-time query of LLMs. Then, two sampling strategies are proposed to find a better initial search point and reduce traversed categories, saving iteration costs. Moreover, we apply a novel fitness score with entropy constraints to mitigate overfitting. In a challenging one-shot image classification setting, our method outperforms existing textual prompt-based methods and improves LLM-generated description methods across 13 datasets. Meanwhile, we demonstrate that our optimal prompts improve adapter-based methods and transfer effectively across different backbones.

Title: One-for-More: Continual Diffusion Model for Anomaly Detection

• Authors: Xiaofan Li, Xin Tan, Zhuo Chen, Zhizhong Zhang, Ruixin Zhang, Rizen Guo, Guanna Jiang, Yulong Chen, Yanyun Qu, Lizhuang Ma, Yuan Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19848
• Pdf URL: https://arxiv.org/pdf/2502.19848
• Copy Paste: [[2502.19848]] One-for-More: Continual Diffusion Model for Anomaly Detection(https://arxiv.org/abs/2502.19848)
• Keywords: protect, diffusion, generative
• Abstract: With the rise of generative models, there is a growing interest in unifying all tasks within a generative framework. Anomaly detection methods also fall into this scope and utilize diffusion models to generate or reconstruct normal samples when given arbitrary anomaly images. However, our study found that the diffusion model suffers from severe ``faithfulness hallucination'' and ``catastrophic forgetting'', which can't meet the unpredictable pattern increments. To mitigate the above problems, we propose a continual diffusion model that uses gradient projection to achieve stable continual learning. Gradient projection deploys a regularization on the model updating by modifying the gradient towards the direction protecting the learned knowledge. But as a double-edged sword, it also requires huge memory costs brought by the Markov process. Hence, we propose an iterative singular value decomposition method based on the transitive property of linear representation, which consumes tiny memory and incurs almost no performance loss. Finally, considering the risk of ``over-fitting'' to normal images of the diffusion model, we propose an anomaly-masked network to enhance the condition mechanism of the diffusion model. For continual anomaly detection, ours achieves first place in 17/18 settings on MVTec and VisA. Code is available at this https URL

Title: Revisit the Stability of Vanilla Federated Learning Under Diverse Conditions

• Authors: Youngjoon Lee, Jinu Gong, Sun Choi, Joonhyuk Kang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19849
• Pdf URL: https://arxiv.org/pdf/2502.19849
• Copy Paste: [[2502.19849]] Revisit the Stability of Vanilla Federated Learning Under Diverse Conditions(https://arxiv.org/abs/2502.19849)
• Keywords: privacy, robust, federate, transformer
• Abstract: Federated Learning (FL) is a distributed machine learning paradigm enabling collaborative model training across decentralized clients while preserving data privacy. In this paper, we revisit the stability of the vanilla FedAvg algorithm under diverse conditions. Despite its conceptual simplicity, FedAvg exhibits remarkably stable performance compared to more advanced FL techniques. Our experiments assess the performance of various FL methods on blood cell and skin lesion classification tasks using Vision Transformer (ViT). Additionally, we evaluate the impact of different representative classification models and analyze sensitivity to hyperparameter variations. The results consistently demonstrate that, regardless of dataset, classification model employed, or hyperparameter settings, FedAvg maintains robust performance. Given its stability, robust performance without the need for extensive hyperparameter tuning, FedAvg is a safe and efficient choice for FL deployments in resource-constrained hospitals handling medical data. These findings underscore the enduring value of the vanilla FedAvg approach as a trusted baseline for clinical practice.

Title: Team A at SemEval-2025 Task 11: Breaking Language Barriers in Emotion Detection with Multilingual Models

• Authors: P Sam Sahil, Anupam Jamatia
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19856
• Pdf URL: https://arxiv.org/pdf/2502.19856
• Copy Paste: [[2502.19856]] Team A at SemEval-2025 Task 11: Breaking Language Barriers in Emotion Detection with Multilingual Models(https://arxiv.org/abs/2502.19856)
• Keywords: robust
• Abstract: This paper describes the system submitted by Team A to SemEval 2025 Task 11, ``Bridging the Gap in Text-Based Emotion Detection.'' The task involved identifying the perceived emotion of a speaker from text snippets, with each instance annotated with one of six emotions: joy, sadness, fear, anger, surprise, or disgust. A dataset provided by the task organizers served as the foundation for training and evaluating our models. Among the various approaches explored, the best performance was achieved using multilingual embeddings combined with a fully connected layer. This paper details the system architecture, discusses experimental results, and highlights the advantages of leveraging multilingual representations for robust emotion detection in text.

Title: MIND: Towards Immersive Psychological Healing with Multi-agent Inner Dialogue

• Authors: Yujia Chen, Changsong Li, Yiming Wang, Qingqing Xiao, Nan Zhang, Zifan Kong, Peng Wang, Binyu Yan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19860
• Pdf URL: https://arxiv.org/pdf/2502.19860
• Copy Paste: [[2502.19860]] MIND: Towards Immersive Psychological Healing with Multi-agent Inner Dialogue(https://arxiv.org/abs/2502.19860)
• Keywords: generative, large language model
• Abstract: Mental health issues are worsening in today's competitive society, such as depression and anxiety. Traditional healings like counseling and chatbots fail to engage effectively, they often provide generic responses lacking emotional depth. Although large language models (LLMs) have the potential to create more human-like interactions, they still struggle to capture subtle emotions. This requires LLMs to be equipped with human-like adaptability and warmth. To fill this gap, we propose the MIND (Multi-agent INner Dialogue), a novel paradigm that provides more immersive psychological healing environments. Considering the strong generative and role-playing ability of LLM agents, we predefine an interactive healing framework and assign LLM agents different roles within the framework to engage in interactive inner dialogues with users, thereby providing an immersive healing experience. We conduct extensive human experiments in various real-world healing dimensions, and find that MIND provides a more user-friendly experience than traditional paradigms. This demonstrates that MIND effectively leverages the significant potential of LLMs in psychological healing.

Title: LMHLD: A Large-scale Multi-source High-resolution Landslide Dataset for Landslide Detection based on Deep Learning

• Authors: Guanting Liu, Yi Wang, Xi Chen, Baoyu Du, Penglei Li, Yuan Wu, Zhice Fang
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.19866
• Pdf URL: https://arxiv.org/pdf/2502.19866
• Copy Paste: [[2502.19866]] LMHLD: A Large-scale Multi-source High-resolution Landslide Dataset for Landslide Detection based on Deep Learning(https://arxiv.org/abs/2502.19866)
• Keywords: robust, extraction
• Abstract: Landslides are among the most common natural disasters globally, posing significant threats to human society. Deep learning (DL) has proven to be an effective method for rapidly generating landslide inventories in large-scale disaster areas. However, DL models rely heavily on high-quality labeled landslide data for strong feature extraction capabilities. And landslide detection using DL urgently needs a benchmark dataset to evaluate the generalization ability of the latest models. To solve the above problems, we construct a Large-scale Multi-source High-resolution Landslide Dataset (LMHLD) for Landslide Detection based on DL. LMHLD collects remote sensing images from five different satellite sensors across seven study areas worldwide: Wenchuan, China (2008); Rio de Janeiro, Brazil (2011); Gorkha, Nepal (2015); Jiuzhaigou, China (2015); Taiwan, China (2018); Hokkaido, Japan (2018); Emilia-Romagna, Italy (2023). The dataset includes a total of 25,365 patches, with different patch sizes to accommodate different landslide scales. Additionally, a training module, LMHLDpart, is designed to accommodate landslide detection tasks at varying scales and to alleviate the issue of catastrophic forgetting in multi-task learning. Furthermore, the models trained by LMHLD is applied in other datasets to highlight the robustness of LMHLD. Five dataset quality evaluation experiments designed by using seven DL models from the U-Net family demonstrate that LMHLD has the potential to become a benchmark dataset for landslide detection. LMHLD is open access and can be accessed through the link: this https URL. This dataset provides a strong foundation for DL models, accelerates the development of DL in landslide detection, and serves as a valuable resource for landslide prevention and mitigation efforts.

Title: C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation

• Authors: Yuhao Li, Mirana Claire Angel, Salman Khan, Yu Zhu, Jinqiu Sun, Yanning Zhang, Fahad Shahbaz Khan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19868
• Pdf URL: https://arxiv.org/pdf/2502.19868
• Copy Paste: [[2502.19868]] C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation(https://arxiv.org/abs/2502.19868)
• Keywords: diffusion
• Abstract: Trajectory-based motion control has emerged as an intuitive and efficient approach for controllable video generation. However, the existing trajectory-based approaches are usually limited to only generating the motion trajectory of the controlled object and ignoring the dynamic interactions between the controlled object and its surroundings. To address this limitation, we propose a Chain-of-Thought-based motion controller for controllable video generation, named C-Drag. Instead of directly generating the motion of some objects, our C-Drag first performs object perception and then reasons the dynamic interactions between different objects according to the given motion control of the objects. Specifically, our method includes an object perception module and a Chain-of-Thought-based motion reasoning module. The object perception module employs visual language models to capture the position and category information of various objects within the image. The Chain-of-Thought-based motion reasoning module takes this information as input and conducts a stage-wise reasoning process to generate motion trajectories for each of the affected objects, which are subsequently fed to the diffusion model for video synthesis. Furthermore, we introduce a new video object interaction (VOI) dataset to evaluate the generation quality of motion controlled video generation methods. Our VOI dataset contains three typical types of interactions and provides the motion trajectories of objects that can be used for accurate performance evaluation. Experimental results show that C-Drag achieves promising performance across multiple metrics, excelling in object motion control. Our benchmark, codes, and models will be available at this https URL.

Title: MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge

• Authors: Yuntao Du, Kailin Jiang, Zhi Gao, Chenrui Shi, Zilong Zheng, Siyuan Qi, Qing Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19870
• Pdf URL: https://arxiv.org/pdf/2502.19870
• Copy Paste: [[2502.19870]] MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge(https://arxiv.org/abs/2502.19870)
• Keywords: robust, large language model
• Abstract: Knowledge editing techniques have emerged as essential tools for updating the factual knowledge of large language models (LLMs) and multimodal models (LMMs), allowing them to correct outdated or inaccurate information without retraining from scratch. However, existing benchmarks for multimodal knowledge editing primarily focus on entity-level knowledge represented as simple triplets, which fail to capture the complexity of real-world multimodal information. To address this issue, we introduce MMKE-Bench, a comprehensive MultiModal Knowledge Editing Benchmark, designed to evaluate the ability of LMMs to edit diverse visual knowledge in real-world scenarios. MMKE-Bench addresses these limitations by incorporating three types of editing tasks: visual entity editing, visual semantic editing, and user-specific editing. Besides, MMKE-Bench uses free-form natural language to represent and edit knowledge, offering a more flexible and effective format. The benchmark consists of 2,940 pieces of knowledge and 8,363 images across 33 broad categories, with evaluation questions automatically generated and human-verified. We assess five state-of-the-art knowledge editing methods on three prominent LMMs, revealing that no method excels across all criteria, and that visual and user-specific edits are particularly challenging. MMKE-Bench sets a new standard for evaluating the robustness of multimodal knowledge editing techniques, driving progress in this rapidly evolving field.

Title: Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models

• Authors: Sibo Yi, Tianshuo Cong, Xinlei He, Qi Li, Jiaxing Song
• Subjects: cs.CR, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19883
• Pdf URL: https://arxiv.org/pdf/2502.19883
• Copy Paste: [[2502.19883]] Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models(https://arxiv.org/abs/2502.19883)
• Keywords: secure, security, defense, attack, robust, large language model
• Abstract: Small language models (SLMs) have become increasingly prominent in the deployment on edge devices due to their high efficiency and low computational cost. While researchers continue to advance the capabilities of SLMs through innovative training strategies and model compression techniques, the security risks of SLMs have received considerably less attention compared to large language models (LLMs).To fill this gap, we provide a comprehensive empirical study to evaluate the security performance of 13 state-of-the-art SLMs under various jailbreak attacks. Our experiments demonstrate that most SLMs are quite susceptible to existing jailbreak attacks, while some of them are even vulnerable to direct harmful this http URL address the safety concerns, we evaluate several representative defense methods and demonstrate their effectiveness in enhancing the security of SLMs. We further analyze the potential security degradation caused by different SLM techniques including architecture compression, quantization, knowledge distillation, and so on. We expect that our research can highlight the security challenges of SLMs and provide valuable insights to future work in developing more robust and secure SLMs.

Title: High-Fidelity Relightable Monocular Portrait Animation with Lighting-Controllable Video Diffusion Model

• Authors: Mingtao Guo, Guanyu Xing, Yanli Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19894
• Pdf URL: https://arxiv.org/pdf/2502.19894
• Copy Paste: [[2502.19894]] High-Fidelity Relightable Monocular Portrait Animation with Lighting-Controllable Video Diffusion Model(https://arxiv.org/abs/2502.19894)
• Keywords: diffusion
• Abstract: Relightable portrait animation aims to animate a static reference portrait to match the head movements and expressions of a driving video while adapting to user-specified or reference lighting conditions. Existing portrait animation methods fail to achieve relightable portraits because they do not separate and manipulate intrinsic (identity and appearance) and extrinsic (pose and lighting) features. In this paper, we present a Lighting Controllable Video Diffusion model (LCVD) for high-fidelity, relightable portrait animation. We address this limitation by distinguishing these feature types through dedicated subspaces within the feature space of a pre-trained image-to-video diffusion model. Specifically, we employ the 3D mesh, pose, and lighting-rendered shading hints of the portrait to represent the extrinsic attributes, while the reference represents the intrinsic attributes. In the training phase, we employ a reference adapter to map the reference into the intrinsic feature subspace and a shading adapter to map the shading hints into the extrinsic feature subspace. By merging features from these subspaces, the model achieves nuanced control over lighting, pose, and expression in generated animations. Extensive evaluations show that LCVD outperforms state-of-the-art methods in lighting realism, image quality, and video consistency, setting a new benchmark in relightable portrait animation.

Title: GenPC: Zero-shot Point Cloud Completion via 3D Generative Priors

• Authors: An Li, Zhe Zhu, Mingqiang Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19896
• Pdf URL: https://arxiv.org/pdf/2502.19896
• Copy Paste: [[2502.19896]] GenPC: Zero-shot Point Cloud Completion via 3D Generative Priors(https://arxiv.org/abs/2502.19896)
• Keywords: robust, generative
• Abstract: Existing point cloud completion methods, which typically depend on predefined synthetic training datasets, encounter significant challenges when applied to out-of-distribution, real-world scans. To overcome this limitation, we introduce a zero-shot completion framework, termed GenPC, designed to reconstruct high-quality real-world scans by leveraging explicit 3D generative priors. Our key insight is that recent feed-forward 3D generative models, trained on extensive internet-scale data, have demonstrated the ability to perform 3D generation from single-view images in a zero-shot setting. To harness this for completion, we first develop a Depth Prompting module that links partial point clouds with image-to-3D generative models by leveraging depth images as a stepping stone. To retain the original partial structure in the final results, we design the Geometric Preserving Fusion module that aligns the generated shape with input by adaptively adjusting its pose and scale. Extensive experiments on widely used benchmarks validate the superiority and generalizability of our approach, bringing us a step closer to robust real-world scan completion.

Title: SkipPipe: Partial and Reordered Pipelining Framework for Training LLMs in Heterogeneous Networks

• Authors: Nikolay Blagoev, Lydia Yiyu Chen, Oğuzhan Ersoy
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2502.19913
• Pdf URL: https://arxiv.org/pdf/2502.19913
• Copy Paste: [[2502.19913]] SkipPipe: Partial and Reordered Pipelining Framework for Training LLMs in Heterogeneous Networks(https://arxiv.org/abs/2502.19913)
• Keywords: large language model
• Abstract: Data and pipeline parallelism are ubiquitous for training of Large Language Models (LLM) on distributed nodes. Driven by the need for cost-effective training, recent work explores efficient communication arrangement for end to end training. Motivated by LLM's resistance to layer skipping and layer reordering, in this paper, we explore stage (several consecutive layers) skipping in pipeline training, and challenge the conventional practice of sequential pipeline execution. We derive convergence and throughput constraints (guidelines) for pipelining with skipping and swapping pipeline stages. Based on these constraints, we propose SkipPipe, the first partial pipeline framework to reduce the end-to-end training time for LLMs while preserving the convergence. The core of SkipPipe is a path scheduling algorithm that optimizes the paths for individual microbatches and reduces idle time (due to microbatch collisions) on the distributed nodes, complying with the given stage skipping ratio. We extensively evaluate SkipPipe on LLaMa models from 500M to 8B parameters on up to 20 nodes. Our results show that SkipPipe reduces training iteration time by up to $55\%$ compared to full pipeline. Our partial pipeline training also improves resistance to layer omission during inference, experiencing a drop in perplexity of only $7\%$ when running only half the model. Our code is available at this https URL.

Title: Picking the Cream of the Crop: Visual-Centric Data Selection with Collaborative Agents

• Authors: Zhenyu Liu, Yunxin Li, Baotian Hu, Wenhan Luo, Yaowei Wang, Min Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19917
• Pdf URL: https://arxiv.org/pdf/2502.19917
• Copy Paste: [[2502.19917]] Picking the Cream of the Crop: Visual-Centric Data Selection with Collaborative Agents(https://arxiv.org/abs/2502.19917)
• Keywords: large language model
• Abstract: To improve Multimodal Large Language Models' (MLLMs) ability to process images and complex instructions, researchers predominantly curate large-scale visual instruction tuning datasets, which are either sourced from existing vision tasks or synthetically generated using LLMs and image descriptions. However, they often suffer from critical flaws, including misaligned instruction-image pairs and low-quality images. Such issues hinder training efficiency and limit performance improvements, as models waste resources on noisy or irrelevant data with minimal benefit to overall capability. To address this issue, we propose a \textbf{Vi}sual-Centric \textbf{S}election approach via \textbf{A}gents Collaboration (ViSA), which centers on image quality assessment and image-instruction relevance evaluation. Specifically, our approach consists of 1) an image information quantification method via visual agents collaboration to select images with rich visual information, and 2) a visual-centric instruction quality assessment method to select high-quality instruction data related to high-quality images. Finally, we reorganize 80K instruction data from large open-source datasets. Extensive experiments demonstrate that ViSA outperforms or is comparable to current state-of-the-art models on seven benchmarks, using only 2.5\% of the original data, highlighting the efficiency of our data selection approach. Moreover, we conduct ablation studies to validate the effectiveness of each component of our method. The code is available at this https URL.

Title: Playing Pokémon Red via Deep Reinforcement Learning

• Authors: Marco Pleines, Daniel Addis, David Rubinstein, Frank Zimmer, Mike Preuss, Peter Whidden
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19920
• Pdf URL: https://arxiv.org/pdf/2502.19920
• Copy Paste: [[2502.19920]] Playing Pokémon Red via Deep Reinforcement Learning(https://arxiv.org/abs/2502.19920)
• Keywords: large language model
• Abstract: Pokémon Red, a classic Game Boy JRPG, presents significant challenges as a testbed for agents, including multi-tasking, long horizons of tens of thousands of steps, hard exploration, and a vast array of potential policies. We introduce a simplistic environment and a Deep Reinforcement Learning (DRL) training methodology, demonstrating a baseline agent that completes an initial segment of the game up to completing Cerulean City. Our experiments include various ablations that reveal vulnerabilities in reward shaping, where agents exploit specific reward signals. We also discuss limitations and argue that games like Pokémon hold strong potential for future research on Large Language Model agents, hierarchical training algorithms, and advanced exploration methods. Source Code: this https URL

Title: Incremental Learning with Repetition via Pseudo-Feature Projection

• Authors: Benedikt Tscheschner, Eduardo Veas, Marc Masana
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.19922
• Pdf URL: https://arxiv.org/pdf/2502.19922
• Copy Paste: [[2502.19922]] Incremental Learning with Repetition via Pseudo-Feature Projection(https://arxiv.org/abs/2502.19922)
• Keywords: fair
• Abstract: Incremental Learning scenarios do not always represent real-world inference use-cases, which tend to have less strict task boundaries, and exhibit repetition of common classes and concepts in their continual data stream. To better represent these use-cases, new scenarios with partial repetition and mixing of tasks are proposed, where the repetition patterns are innate to the scenario and unknown to the strategy. We investigate how exemplar-free incremental learning strategies are affected by data repetition, and we adapt a series of state-of-the-art approaches to analyse and fairly compare them under both settings. Further, we also propose a novel method (Horde), able to dynamically adjust an ensemble of self-reliant feature extractors, and align them by exploiting class repetition. Our proposed exemplar-free method achieves competitive results in the classic scenario without repetition, and state-of-the-art performance in the one with repetition.

Title: Image Referenced Sketch Colorization Based on Animation Creation Workflow

• Authors: Dingkun Yan, Xinrui Wang, Zhuoru Li, Suguru Saito, Yusuke Iwasawa, Yutaka Matsuo, Jiaxian Guo
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.19937
• Pdf URL: https://arxiv.org/pdf/2502.19937
• Copy Paste: [[2502.19937]] Image Referenced Sketch Colorization Based on Animation Creation Workflow(https://arxiv.org/abs/2502.19937)
• Keywords: diffusion
• Abstract: Sketch colorization plays an important role in animation and digital illustration production tasks. However, existing methods still meet problems in that text-guided methods fail to provide accurate color and style reference, hint-guided methods still involve manual operation, and image-referenced methods are prone to cause artifacts. To address these limitations, we propose a diffusion-based framework inspired by real-world animation production workflows. Our approach leverages the sketch as the spatial guidance and an RGB image as the color reference, and separately extracts foreground and background from the reference image with spatial masks. Particularly, we introduce a split cross-attention mechanism with LoRA (Low-Rank Adaptation) modules. They are trained separately with foreground and background regions to control the corresponding embeddings for keys and values in cross-attention. This design allows the diffusion model to integrate information from foreground and background independently, preventing interference and eliminating the spatial artifacts. During inference, we design switchable inference modes for diverse use scenarios by changing modules activated in the framework. Extensive qualitative and quantitative experiments, along with user studies, demonstrate our advantages over existing methods in generating high-qualigy artifact-free results with geometric mismatched references. Ablation studies further confirm the effectiveness of each component. Codes are available at this https URL tellurion-kanata/colorizeDiffusion.

Title: Flexible Bivariate Beta Mixture Model: A Probabilistic Approach for Clustering Complex Data Structures

• Authors: Yung-Peng Hsu, Hung-Hsuan Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19938
• Pdf URL: https://arxiv.org/pdf/2502.19938
• Copy Paste: [[2502.19938]] Flexible Bivariate Beta Mixture Model: A Probabilistic Approach for Clustering Complex Data Structures(https://arxiv.org/abs/2502.19938)
• Keywords: robust
• Abstract: Clustering is essential in data analysis and machine learning, but traditional algorithms like $k$-means and Gaussian Mixture Models (GMM) often fail with nonconvex clusters. To address the challenge, we introduce the Flexible Bivariate Beta Mixture Model (FBBMM), which utilizes the flexibility of the bivariate beta distribution to handle diverse and irregular cluster shapes. Using the Expectation Maximization (EM) algorithm and Sequential Least Squares Programming (SLSQP) optimizer for parameter estimation, we validate FBBMM on synthetic and real-world datasets, demonstrating its superior performance in clustering complex data structures, offering a robust solution for big data analytics across various domains. We release the experimental code at this https URL.

Title: Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies

• Authors: Yuan-Chih Yang, Hung-Hsuan Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19948
• Pdf URL: https://arxiv.org/pdf/2502.19948
• Copy Paste: [[2502.19948]] Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies(https://arxiv.org/abs/2502.19948)
• Keywords: robust
• Abstract: Dropout and DropConnect are well-known techniques that apply a consistent drop rate to randomly deactivate neurons or edges in a neural network layer during training. This paper introduces a novel methodology that assigns dynamic drop rates to each edge within a layer, uniquely tailoring the dropping process without incorporating additional learning parameters. We perform experiments on synthetic and openly available datasets to validate the effectiveness of our approach. The results demonstrate that our method outperforms Dropout, DropConnect, and Standout, a classic mechanism known for its adaptive dropout capabilities. Furthermore, our approach improves the robustness and generalization of neural network training without increasing computational complexity. The complete implementation of our methodology is publicly accessible for research and replication purposes at this https URL.

Title: GeoEdit: Geometric Knowledge Editing for Large Language Models

• Authors: Yujie Feng, Liming Zhan, Zexin Lu, Yongxin Xu, Xu Chu, Yasha Wang, Jiannong Cao, Philip S. Yu, Xiao-Ming Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19953
• Pdf URL: https://arxiv.org/pdf/2502.19953
• Copy Paste: [[2502.19953]] GeoEdit: Geometric Knowledge Editing for Large Language Models(https://arxiv.org/abs/2502.19953)
• Keywords: large language model
• Abstract: Regular updates are essential for maintaining up-to-date knowledge in large language models (LLMs). Consequently, various model editing methods have been developed to update specific knowledge within LLMs. However, training-based approaches often struggle to effectively incorporate new knowledge while preserving unrelated general knowledge. To address this challenge, we propose a novel framework called Geometric Knowledge Editing (GeoEdit). GeoEdit utilizes the geometric relationships of parameter updates from fine-tuning to differentiate between neurons associated with new knowledge updates and those related to general knowledge perturbations. By employing a direction-aware knowledge identification method, we avoid updating neurons with directions approximately orthogonal to existing knowledge, thus preserving the model's generalization ability. For the remaining neurons, we integrate both old and new knowledge for aligned directions and apply a "forget-then-learn" editing strategy for opposite directions. Additionally, we introduce an importance-guided task vector fusion technique that filters out redundant information and provides adaptive neuron-level weighting, further enhancing model editing performance. Extensive experiments on two publicly available datasets demonstrate the superiority of GeoEdit over existing state-of-the-art methods.

Title: Collaborative Stance Detection via Small-Large Language Model Consistency Verification

• Authors: Yu Yan, Sheng Sun, Zixiang Tang, Teli Liu, Min Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19954
• Pdf URL: https://arxiv.org/pdf/2502.19954
• Copy Paste: [[2502.19954]] Collaborative Stance Detection via Small-Large Language Model Consistency Verification(https://arxiv.org/abs/2502.19954)
• Keywords: large language model
• Abstract: Stance detection on social media aims to identify attitudes expressed in tweets towards specific targets. Current studies prioritize Large Language Models (LLMs) over Small Language Models (SLMs) due to the overwhelming performance improving provided by LLMs. However, heavily relying on LLMs for stance detection, regardless of the cost, is impractical for real-world social media monitoring systems that require vast data analysis. To this end, we propose \textbf{\underline{Co}}llaborative Stance Detection via Small-Large Language Model Consistency \textbf{\underline{Ver}}ification (\textbf{CoVer}) framework, which enhances LLM utilization via context-shared batch reasoning and logical verification between LLM and SLM. Specifically, instead of processing each text individually, CoVer processes texts batch-by-batch, obtaining stance predictions and corresponding explanations via LLM reasoning in a shared context. Then, to exclude the bias caused by context noises, CoVer introduces the SLM for logical consistency verification. Finally, texts that repeatedly exhibit low logical consistency are classified using consistency-weighted aggregation of prior LLM stance predictions. Our experiments show that CoVer outperforms state-of-the-art methods across multiple benchmarks in the zero-shot setting, achieving 0.54 LLM queries per tweet while significantly enhancing performance. Our CoVer offers a more practical solution for LLM deploying for social media stance detection.

Title: SeisMoLLM: Advancing Seismic Monitoring via Cross-modal Transfer with Pre-trained Large Language Model

• Authors: Xinghao Wang, Feng Liu, Rui Su, Zhihui Wang, Lei Bai, Wanli Ouyang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19960
• Pdf URL: https://arxiv.org/pdf/2502.19960
• Copy Paste: [[2502.19960]] SeisMoLLM: Advancing Seismic Monitoring via Cross-modal Transfer with Pre-trained Large Language Model(https://arxiv.org/abs/2502.19960)
• Keywords: large language model
• Abstract: Recent advances in deep learning have revolutionized seismic monitoring, yet developing a foundation model that performs well across multiple complex tasks remains challenging, particularly when dealing with degraded signals or data scarcity. This work presents SeisMoLLM, the first foundation model that utilizes cross-modal transfer for seismic monitoring, to unleash the power of large-scale pre-training from a large language model without requiring direct pre-training on seismic datasets. Through elaborate waveform tokenization and fine-tuning of pre-trained GPT-2 model, SeisMoLLM achieves state-of-the-art performance on the DiTing and STEAD datasets across five critical tasks: back-azimuth estimation, epicentral distance estimation, magnitude estimation, phase picking, and first-motion polarity classification. It attains 36 best results out of 43 task metrics and 12 top scores out of 16 few-shot generalization metrics, with many relative improvements ranging from 10% to 50%. In addition to its superior performance, SeisMoLLM maintains efficiency comparable to or even better than lightweight models in both training and inference. These findings establish SeisMoLLM as a promising foundation model for practical seismic monitoring and highlight cross-modal transfer as an exciting new direction for earthquake studies, showcasing the potential of advanced deep learning techniques to propel seismology research forward.

Title: ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning

• Authors: Quanxing Zha, Xin Liu, Shu-Juan Peng, Yiu-ming Cheung, Xing Xu, Nannan Wang
• Subjects: cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2502.19962
• Pdf URL: https://arxiv.org/pdf/2502.19962
• Copy Paste: [[2502.19962]] ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning(https://arxiv.org/abs/2502.19962)
• Keywords: robust
• Abstract: Can we accurately identify the true correspondences from multimodal datasets containing mismatched data pairs? Existing methods primarily emphasize the similarity matching between the representations of objects across modalities, potentially neglecting the crucial relation consistency within modalities that are particularly important for distinguishing the true and false correspondences. Such an omission often runs the risk of misidentifying negatives as positives, thus leading to unanticipated performance degradation. To address this problem, we propose a general Relation Consistency learning framework, namely ReCon, to accurately discriminate the true correspondences among the multimodal data and thus effectively mitigate the adverse impact caused by mismatches. Specifically, ReCon leverages a novel relation consistency learning to ensure the dual-alignment, respectively of, the cross-modal relation consistency between different modalities and the intra-modal relation consistency within modalities. Thanks to such dual constrains on relations, ReCon significantly enhances its effectiveness for true correspondence discrimination and therefore reliably filters out the mismatched pairs to mitigate the risks of wrong supervisions. Extensive experiments on three widely-used benchmark datasets, including Flickr30K, MS-COCO, and Conceptual Captions, are conducted to demonstrate the effectiveness and superiority of ReCon compared with other SOTAs. The code is available at: this https URL.

Title: Do Sparse Autoencoders Generalize? A Case Study of Answerability

• Authors: Lovis Heindrich, Philip Torr, Fazl Barez, Veronika Thost
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19964
• Pdf URL: https://arxiv.org/pdf/2502.19964
• Copy Paste: [[2502.19964]] Do Sparse Autoencoders Generalize? A Case Study of Answerability(https://arxiv.org/abs/2502.19964)
• Keywords: extraction, interpretability
• Abstract: Sparse autoencoders (SAEs) have emerged as a promising approach in language model interpretability, offering unsupervised extraction of sparse features. For interpretability methods to succeed, they must identify abstract features across domains, and these features can often manifest differently in each context. We examine this through "answerability"-a model's ability to recognize answerable questions. We extensively evaluate SAE feature generalization across diverse answerability datasets for Gemma 2 SAEs. Our analysis reveals that residual stream probes outperform SAE features within domains, but generalization performance differs sharply. SAE features demonstrate inconsistent transfer ability, and residual stream probes similarly show high variance out of distribution. Overall, this demonstrates the need for quantitative methods to predict feature generalization in SAE-based interpretability.

Title: Deterministic or probabilistic? The psychology of LLMs as random number generators

• Authors: Javier Coronado-Blázquez
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19965
• Pdf URL: https://arxiv.org/pdf/2502.19965
• Copy Paste: [[2502.19965]] Deterministic or probabilistic? The psychology of LLMs as random number generators(https://arxiv.org/abs/2502.19965)
• Keywords: transformer, large language model
• Abstract: Large Language Models (LLMs) have transformed text generation through inherently probabilistic context-aware mechanisms, mimicking human natural language. In this paper, we systematically investigate the performance of various LLMs when generating random numbers, considering diverse configurations such as different model architectures, numerical ranges, temperature, and prompt languages. Our results reveal that, despite their stochastic transformers-based architecture, these models often exhibit deterministic responses when prompted for random numerical outputs. In particular, we find significant differences when changing the model, as well as the prompt language, attributing this phenomenon to biases deeply embedded within the training data. Models such as DeepSeek-R1 can shed some light on the internal reasoning process of LLMs, despite arriving to similar results. These biases induce predictable patterns that undermine genuine randomness, as LLMs are nothing but reproducing our own human cognitive biases.

Title: Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios

• Authors: Chao Wang, Luning Zhang, Zheng Wang, Yang Zhou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.19973
• Pdf URL: https://arxiv.org/pdf/2502.19973
• Copy Paste: [[2502.19973]] Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios(https://arxiv.org/abs/2502.19973)
• Keywords: large language model
• Abstract: Combining multiple perceptual inputs and performing combinatorial reasoning in complex scenarios is a sophisticated cognitive function in humans. With advancements in multi-modal large language models, recent benchmarks tend to evaluate visual understanding across multiple images. However, they often overlook the necessity of combinatorial reasoning across multiple perceptual information. To explore the ability of advanced models to integrate multiple perceptual inputs for combinatorial reasoning in complex scenarios, we introduce two benchmarks: Clue-Visual Question Answering (CVQA), with three task types to assess visual comprehension and synthesis, and Clue of Password-Visual Question Answering (CPVQA), with two task types focused on accurate interpretation and application of visual data. For our benchmarks, we present three plug-and-play approaches: utilizing model input for reasoning, enhancing reasoning through minimum margin decoding with randomness generation, and retrieving semantically relevant visual information for effective data integration. The combined results reveal current models' poor performance on combinatorial reasoning benchmarks, even the state-of-the-art (SOTA) closed-source model achieves only 33.04% accuracy on CVQA, and drops to 7.38% on CPVQA. Notably, our approach improves the performance of models on combinatorial reasoning, with a 22.17% boost on CVQA and 9.40% on CPVQA over the SOTA closed-source model, demonstrating its effectiveness in enhancing combinatorial reasoning with multiple perceptual inputs in complex scenarios. The code will be publicly available.

Title: Can Textual Gradient Work in Federated Learning?

• Authors: Minghui Chen, Ruinan Jin, Wenlong Deng, Yuanyuan Chen, Zhi Huang, Han Yu, Xiaoxiao Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19980
• Pdf URL: https://arxiv.org/pdf/2502.19980
• Copy Paste: [[2502.19980]] Can Textual Gradient Work in Federated Learning?(https://arxiv.org/abs/2502.19980)
• Keywords: federate
• Abstract: Recent studies highlight the promise of LLM-based prompt optimization, especially with TextGrad, which automates differentiation'' via texts and backpropagates textual feedback. This approach facilitates training in various real-world applications that do not support numerical gradient propagation or loss calculation. In this paper, we systematically explore the potential and challenges of incorporating textual gradient into Federated Learning (FL). Our contributions are fourfold. Firstly, we introduce a novel FL paradigm, Federated Textual Gradient (FedTextGrad), that allows clients to upload locally optimized prompts derived from textual gradients, while the server aggregates the received prompts. Unlike traditional FL frameworks, which are designed for numerical aggregation, FedTextGrad is specifically tailored for handling textual data, expanding the applicability of FL to a broader range of problems that lack well-defined numerical loss functions. Secondly, building on this design, we conduct extensive experiments to explore the feasibility of FedTextGrad. Our findings highlight the importance of properly tuning key factors (e.g., local steps) in FL training. Thirdly, we highlight a major challenge in FedTextGrad aggregation: retaining essential information from distributed prompt updates. Last but not least, in response to this issue, we improve the vanilla variant of FedTextGrad by providing actionable guidance to the LLM when summarizing client prompts by leveraging the Uniform Information Density principle. Through this principled study, we enable the adoption of textual gradients in FL for optimizing LLMs, identify important issues, and pinpoint future directions, thereby opening up a new research area that warrants further investigation.

Title: The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs

• Authors: Tanja Baeumel, Josef van Genabith, Simon Ostermann
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.19981
• Pdf URL: https://arxiv.org/pdf/2502.19981
• Copy Paste: [[2502.19981]] The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs(https://arxiv.org/abs/2502.19981)
• Keywords: fair, large language model
• Abstract: Autoregressive large language models (LLMs) exhibit impressive performance across various tasks but struggle with simple arithmetic, such as addition of two or more operands. We show that this struggle arises from LLMs' use of a simple one-digit lookahead heuristic, which works fairly well (but not perfect) for two-operand addition but fails in multi-operand cases, where the carry-over logic is more complex. Our probing experiments and digit-wise accuracy evaluation show that LLMs fail precisely where a one-digit lookahead is insufficient to account for cascading carries. We analyze the impact of tokenization strategies on arithmetic performance and show that all investigated models, regardless of tokenization, are inherently limited in the addition of multiple operands due to their reliance on a one-digit lookahead heuristic. Our findings reveal fundamental limitations that prevent LLMs from generalizing to more complex numerical reasoning.

Title: Erasing Without Remembering: Safeguarding Knowledge Forgetting in Large Language Models

• Authors: Huazheng Wang, Yongcheng Jing, Haifeng Sun, Yingjie Wang, Jingyu Wang, Jianxin Liao, Dacheng Tao
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19982
• Pdf URL: https://arxiv.org/pdf/2502.19982
• Copy Paste: [[2502.19982]] Erasing Without Remembering: Safeguarding Knowledge Forgetting in Large Language Models(https://arxiv.org/abs/2502.19982)
• Keywords: robust, large language model
• Abstract: In this paper, we explore machine unlearning from a novel dimension, by studying how to safeguard model unlearning in large language models (LLMs). Our goal is to prevent unlearned models from recalling any related memory of the targeted this http URL begin by uncovering a surprisingly simple yet overlooked fact: existing methods typically erase only the exact expressions of the targeted knowledge, leaving paraphrased or related information intact. To rigorously measure such oversights, we introduce UGBench, the first benchmark tailored for evaluating the generalisation performance across 13 state-of-the-art this http URL reveals that unlearned models can still recall paraphrased answers and retain target facts in intermediate layers. To address this, we propose PERMU, a perturbation-based method that significantly enhances the generalisation capabilities for safeguarding LLM this http URL demonstrate that PERMU delivers up to a 50.13% improvement in unlearning while maintaining a 43.53% boost in robust generalisation. Our code can be found in this https URL.

Title: WaveGAS: Waveform Relaxation for Scaling Graph Neural Networks

• Authors: Jana Vatter, Mykhaylo Zayats, Marcos Martínez Galindo, Vanessa López, Ruben Mayer, Hans-Arno Jacobsen, Hoang Thanh Lam
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.19986
• Pdf URL: https://arxiv.org/pdf/2502.19986
• Copy Paste: [[2502.19986]] WaveGAS: Waveform Relaxation for Scaling Graph Neural Networks(https://arxiv.org/abs/2502.19986)
• Keywords: robust
• Abstract: With the ever-growing size of real-world graphs, numerous techniques to overcome resource limitations when training Graph Neural Networks (GNNs) have been developed. One such approach, GNNAutoScale (GAS), uses graph partitioning to enable training under constrained GPU memory. GAS also stores historical embedding vectors, which are retrieved from one-hop neighbors in other partitions, ensuring critical information is captured across partition boundaries. The historical embeddings which come from the previous training iteration are stale compared to the GAS estimated embeddings, resulting in approximation errors of the training algorithm. Furthermore, these errors accumulate over multiple layers, leading to suboptimal node embeddings. To address this shortcoming, we propose two enhancements: first, WaveGAS, inspired by waveform relaxation, performs multiple forward passes within GAS before the backward pass, refining the approximation of historical embeddings and gradients to improve accuracy; second, a gradient-tracking method that stores and utilizes more accurate historical gradients during training. Empirical results show that WaveGAS enhances GAS and achieves better accuracy, even outperforming methods that train on full graphs, thanks to its robust estimation of node embeddings.

Title: Modern DDoS Threats and Countermeasures: Insights into Emerging Attacks and Detection Strategies

• Authors: Jincheng Wang, Le Yu, John C.S. Lui, Xiapu Luo
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.19996
• Pdf URL: https://arxiv.org/pdf/2502.19996
• Copy Paste: [[2502.19996]] Modern DDoS Threats and Countermeasures: Insights into Emerging Attacks and Detection Strategies(https://arxiv.org/abs/2502.19996)
• Keywords: defense, attack, robust
• Abstract: Distributed Denial of Service (DDoS) attacks persist as significant threats to online services and infrastructure, evolving rapidly in sophistication and eluding traditional detection mechanisms. This evolution demands a comprehensive examination of current trends in DDoS attacks and the efficacy of modern detection strategies. This paper offers an comprehensive survey of emerging DDoS attacks and detection strategies over the past decade. We delve into the diversification of attack targets, extending beyond conventional web services to include newer network protocols and systems, and the adoption of advanced adversarial tactics. Additionally, we review current detection techniques, highlighting essential features that modern systems must integrate to effectively neutralize these evolving threats. Given the technological demands of contemporary network systems, such as high-volume and in-line packet processing capabilities, we also explore how innovative hardware technologies like programmable switches can significantly enhance the development and deployment of robust DDoS detection systems. We conclude by identifying open problems and proposing future directions for DDoS research. In particular, our survey sheds light on the investigation of DDoS attack surfaces for emerging systems, protocols, and adversarial strategies. Moreover, we outlines critical open questions in the development of effective detection systems, e.g., the creation of defense mechanisms independent of control planes.

Title: Offline Reinforcement Learning via Inverse Optimization

• Authors: Ioannis Dimanidis, Tolga Ok, Peyman Mohajerin Esfahani
• Subjects: cs.LG, eess.SY, math.OC
• Abstract URL: https://arxiv.org/abs/2502.20030
• Pdf URL: https://arxiv.org/pdf/2502.20030
• Copy Paste: [[2502.20030]] Offline Reinforcement Learning via Inverse Optimization(https://arxiv.org/abs/2502.20030)
• Keywords: robust
• Abstract: Inspired by the recent successes of Inverse Optimization (IO) across various application domains, we propose a novel offline Reinforcement Learning (ORL) algorithm for continuous state and action spaces, leveraging the convex loss function called ``sub-optimality loss" from the IO literature. To mitigate the distribution shift commonly observed in ORL problems, we further employ a robust and non-causal Model Predictive Control (MPC) expert steering a nominal model of the dynamics using in-hindsight information stemming from the model mismatch. Unlike the existing literature, our robust MPC expert enjoys an exact and tractable convex reformulation. In the second part of this study, we show that the IO hypothesis class, trained by the proposed convex loss function, enjoys ample expressiveness and achieves competitive performance comparing with the state-of-the-art (SOTA) methods in the low-data regime of the MuJoCo benchmark while utilizing three orders of magnitude fewer parameters, thereby requiring significantly fewer computational resources. To facilitate the reproducibility of our results, we provide an open-source package implementing the proposed algorithms and the experiments.

Title: Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Grouping

• Authors: Guannan Lai, Yujie Li, Xiangkun Wang, Junbo Zhang, Tianrui Li, Xin Yang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20032
• Pdf URL: https://arxiv.org/pdf/2502.20032
• Copy Paste: [[2502.20032]] Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Grouping(https://arxiv.org/abs/2502.20032)
• Keywords: robust
• Abstract: Class Incremental Learning (CIL) requires a model to continuously learn new classes without forgetting previously learned ones. While recent studies have significantly alleviated the problem of catastrophic forgetting (CF), more and more research reveals that the order in which classes appear have significant influences on CIL models. Specifically, prioritizing the learning of classes with lower similarity will enhance the model's generalization performance and its ability to mitigate forgetting. Hence, it is imperative to develop an order-robust class incremental learning model that maintains stable performance even when faced with varying levels of class similarity in different orders. In response, we first provide additional theoretical analysis, which reveals that when the similarity among a group of classes is lower, the model demonstrates increased robustness to the class order. Then, we introduce a novel \textbf{G}raph-\textbf{D}riven \textbf{D}ynamic \textbf{S}imilarity \textbf{G}rouping (\textbf{GDDSG}) method, which leverages a graph coloring algorithm for class-based similarity grouping. The proposed approach trains independent CIL models for each group of classes, ultimately combining these models to facilitate joint prediction. Experimental results demonstrate that our method effectively addresses the issue of class order sensitivity while achieving optimal performance in both model accuracy and anti-forgetting capability. Our code is available at this https URL.

Title: AsymLoRA: Harmonizing Data Conflicts and Commonalities in MLLMs

• Authors: Xuyang Wei, Chunlin Tian, Li Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20035
• Pdf URL: https://arxiv.org/pdf/2502.20035
• Copy Paste: [[2502.20035]] AsymLoRA: Harmonizing Data Conflicts and Commonalities in MLLMs(https://arxiv.org/abs/2502.20035)
• Keywords: large language model
• Abstract: Effective instruction fine-tuning on diverse image-text datasets is crucial for developing a versatile Multimodal Large Language Model (MLLM), where dataset composition dictates the model's adaptability across multimodal tasks. However, complex datasets often contain inherent conflicts -- stemming from modality-specific optimization objectives -- and latent commonalities that enable cross-task transfer, which most existing approaches handle separately. To bridge this gap, we introduce AsymLoRA, a parameter-efficient tuning framework that unifies knowledge modularization and cross-modal coordination via asymmetric LoRA: task-specific low-rank projections (matrix B) that preserve distinct adaptation pathways for conflicting objectives, and a shared projection (matrix A) that consolidates cross-modal commonalities. Extensive evaluations demonstrate that AsymLoRA consistently surpasses both vanilla LoRA, which captures only commonalities, and LoRA-MoE, which focuses solely on conflicts, achieving superior model performance and system efficiency across diverse benchmarks.\href{Code}{this https URL}.

Title: A2-GNN: Angle-Annular GNN for Visual Descriptor-free Camera Relocalization

• Authors: Yejun Zhang, Shuzhe Wang, Juho Kannala
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20036
• Pdf URL: https://arxiv.org/pdf/2502.20036
• Copy Paste: [[2502.20036]] A2-GNN: Angle-Annular GNN for Visual Descriptor-free Camera Relocalization(https://arxiv.org/abs/2502.20036)
• Keywords: privacy, robust, extraction
• Abstract: Visual localization involves estimating the 6-degree-of-freedom (6-DoF) camera pose within a known scene. A critical step in this process is identifying pixel-to-point correspondences between 2D query images and 3D models. Most advanced approaches currently rely on extensive visual descriptors to establish these correspondences, facing challenges in storage, privacy issues and model maintenance. Direct 2D-3D keypoint matching without visual descriptors is becoming popular as it can overcome those challenges. However, existing descriptor-free methods suffer from low accuracy or heavy computation. Addressing this gap, this paper introduces the Angle-Annular Graph Neural Network (A2-GNN), a simple approach that efficiently learns robust geometric structural representations with annular feature extraction. Specifically, this approach clusters neighbors and embeds each group's distance information and angle as supplementary information to capture local structures. Evaluation on matching and visual localization datasets demonstrates that our approach achieves state-of-the-art accuracy with low computational overhead among visual description-free methods. Our code will be released on this https URL.

Title: 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds

• Authors: Hengshuo Chu, Xiang Deng, Xiaoyang Chen, Yinchuan Li, Jianye Hao, Liqiang Nie
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.20041
• Pdf URL: https://arxiv.org/pdf/2502.20041
• Copy Paste: [[2502.20041]] 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds(https://arxiv.org/abs/2502.20041)
• Keywords: large language model, segmentation
• Abstract: 3D Affordance detection is a challenging problem with broad applications on various robotic tasks. Existing methods typically formulate the detection paradigm as a label-based semantic segmentation task. This paradigm relies on predefined labels and lacks the ability to comprehend complex natural language, resulting in limited generalization in open-world scene. To address these limitations, we reformulate the traditional affordance detection paradigm into \textit{Instruction Reasoning Affordance Segmentation} (IRAS) task. This task is designed to output a affordance mask region given a query reasoning text, which avoids fixed categories of input labels. We accordingly propose the \textit{3D-AffordanceLLM} (3D-ADLLM), a framework designed for reasoning affordance detection in 3D open-scene. Specifically, 3D-ADLLM introduces large language models (LLMs) to 3D affordance perception with a custom-designed decoder for generating affordance masks, thus achieving open-world reasoning affordance detection. In addition, given the scarcity of 3D affordance datasets for training large models, we seek to extract knowledge from general segmentation data and transfer it to affordance detection. Thus, we propose a multi-stage training strategy that begins with a novel pre-training task, i.e., \textit{Referring Object Part Segmentation}~(ROPS). This stage is designed to equip the model with general recognition and segmentation capabilities at the object-part level. Then followed by fine-tuning with the IRAS task, 3D-ADLLM obtains the reasoning ability for affordance detection. In summary, 3D-ADLLM leverages the rich world knowledge and human-object interaction reasoning ability of LLMs, achieving approximately an 8\% improvement in mIoU on open-vocabulary affordance detection tasks.

Title: Polish-ASTE: Aspect-Sentiment Triplet Extraction Datasets for Polish

• Authors: Marta Lango, Borys Naglik, Mateusz Lango, Iwo Naglik
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20046
• Pdf URL: https://arxiv.org/pdf/2502.20046
• Copy Paste: [[2502.20046]] Polish-ASTE: Aspect-Sentiment Triplet Extraction Datasets for Polish(https://arxiv.org/abs/2502.20046)
• Keywords: extraction, large language model
• Abstract: Aspect-Sentiment Triplet Extraction (ASTE) is one of the most challenging and complex tasks in sentiment analysis. It concerns the construction of triplets that contain an aspect, its associated sentiment polarity, and an opinion phrase that serves as a rationale for the assigned polarity. Despite the growing popularity of the task and the many machine learning methods being proposed to address it, the number of datasets for ASTE is very limited. In particular, no dataset is available for any of the Slavic languages. In this paper, we present two new datasets for ASTE containing customer opinions about hotels and purchased products expressed in Polish. We also perform experiments with two ASTE techniques combined with two large language models for Polish to investigate their performance and the difficulty of the assembled datasets. The new datasets are available under a permissive licence and have the same file format as the English datasets, facilitating their use in future research.

Title: Connecting the Persian-speaking World through Transliteration

• Authors: Rayyan Merchant, Akhilesh Kakolu Ramarao, Kevin Tang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20047
• Pdf URL: https://arxiv.org/pdf/2502.20047
• Copy Paste: [[2502.20047]] Connecting the Persian-speaking World through Transliteration(https://arxiv.org/abs/2502.20047)
• Keywords: transformer
• Abstract: Despite speaking mutually intelligible varieties of the same language, speakers of Tajik Persian, written in a modified Cyrillic alphabet, cannot read Iranian and Afghan texts written in the Perso-Arabic script. As the vast majority of Persian text on the Internet is written in Perso-Arabic, monolingual Tajik speakers are unable to interface with the Internet in any meaningful way. Due to overwhelming similarity between the formal registers of these dialects and the scarcity of Tajik-Farsi parallel data, machine transliteration has been proposed as more a practical and appropriate solution than machine translation. This paper presents a transformer-based G2P approach to Tajik-Farsi transliteration, achieving chrF++ scores of 58.70 (Farsi to Tajik) and 74.20 (Tajik to Farsi) on novel digraphic datasets, setting a comparable baseline metric for future work. Our results also demonstrate the non-trivial difficulty of this task in both directions. We also provide an overview of the differences between the two scripts and the challenges they present, so as to aid future efforts in Tajik-Farsi transliteration.

Title: A Generative Model Enhanced Multi-Agent Reinforcement Learning Method for Electric Vehicle Charging Navigation

• Authors: Tianyang Qi, Shibo Chen, Jun Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20068
• Pdf URL: https://arxiv.org/pdf/2502.20068
• Copy Paste: [[2502.20068]] A Generative Model Enhanced Multi-Agent Reinforcement Learning Method for Electric Vehicle Charging Navigation(https://arxiv.org/abs/2502.20068)
• Keywords: privacy, generative
• Abstract: With the widespread adoption of electric vehicles (EVs), navigating for EV drivers to select a cost-effective charging station has become an important yet challenging issue due to dynamic traffic conditions, fluctuating electricity prices, and potential competition from other EVs. The state-of-the-art deep reinforcement learning (DRL) algorithms for solving this task still require global information about all EVs at the execution stage, which not only increases communication costs but also raises privacy issues among EV drivers. To overcome these drawbacks, we introduce a novel generative model-enhanced multi-agent DRL algorithm that utilizes only the EV's local information while achieving performance comparable to these state-of-the-art algorithms. Specifically, the policy network is implemented on the EV side, and a Conditional Variational Autoencoder-Long Short Term Memory (CVAE-LSTM)-based recommendation model is developed to provide recommendation information. Furthermore, a novel future charging competition encoder is designed to effectively compress global information, enhancing training performance. The multi-gradient descent algorithm (MGDA) is also utilized to adaptively balance the weight between the two parts of the training objective, resulting in a more stable training process. Simulations are conducted based on a practical area in Xián, China. Experimental results show that our proposed algorithm, which relies on local information, outperforms existing local information-based methods and achieves less than 8\% performance loss compared to global information-based methods.

Title: Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents

• Authors: Haochen Sun, Shuwen Zhang, Lei Ren, Hao Xu, Hao Fu, Caixia Yuan, Xiaojie Wang
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2502.20073
• Pdf URL: https://arxiv.org/pdf/2502.20073
• Copy Paste: [[2502.20073]] Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents(https://arxiv.org/abs/2502.20073)
• Keywords: large language model
• Abstract: Large language models (LLMs) based agent systems have made great strides in real-world applications beyond traditional NLP tasks. This paper proposes a new LLM-powered Multi-Agent System (LLM-MAS) benchmark, Collab-Overcooked, built on the popular Overcooked-AI game with more applicable and challenging tasks in interactive environments. Collab-Overcooked extends existing benchmarks from two novel perspectives. First, it provides a multi-agent framework supporting diverse tasks and objectives and encourages collaboration through natural language communication. Second, it introduces a spectrum of process-oriented evaluation metrics to assess the fine-grained collaboration capabilities of different LLM agents, a dimension often overlooked in prior work. We conduct extensive experiments over 10 popular LLMs and show that, while the LLMs present a strong ability in goal interpretation, there is a significant discrepancy in active collaboration and continuous adaption that are critical for efficiently fulfilling complicated tasks. Notably, we highlight the strengths and weaknesses in LLM-MAS and provide insights for improving and evaluating LLM-MAS on a unified and open-sourced benchmark. Environments, 30 open-ended tasks, and an integrated evaluation package are now publicly available at this https URL.

Title: SegLocNet: Multimodal Localization Network for Autonomous Driving via Bird's-Eye-View Segmentation

• Authors: Zijie Zhou, Zhangshuo Qi, Luqi Cheng, Guangming Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20077
• Pdf URL: https://arxiv.org/pdf/2502.20077
• Copy Paste: [[2502.20077]] SegLocNet: Multimodal Localization Network for Autonomous Driving via Bird's-Eye-View Segmentation(https://arxiv.org/abs/2502.20077)
• Keywords: robust, interpretability, segmentation
• Abstract: Robust and accurate localization is critical for autonomous driving. Traditional GNSS-based localization methods suffer from signal occlusion and multipath effects in urban environments. Meanwhile, methods relying on high-definition (HD) maps are constrained by the high costs associated with the construction and maintenance of HD maps. Standard-definition (SD) maps-based methods, on the other hand, often exhibit unsatisfactory performance or poor generalization ability due to overfitting. To address these challenges, we propose SegLocNet, a multimodal GNSS-free localization network that achieves precise localization using bird's-eye-view (BEV) semantic segmentation. SegLocNet employs a BEV segmentation network to generate semantic maps from multiple sensor inputs, followed by an exhaustive matching process to estimate the vehicle's ego pose. This approach avoids the limitations of regression-based pose estimation and maintains high interpretability and generalization. By introducing a unified map representation, our method can be applied to both HD and SD maps without any modifications to the network architecture, thereby balancing localization accuracy and area coverage. Extensive experiments on the nuScenes and Argoverse datasets demonstrate that our method outperforms the current state-of-the-art methods, and that our method can accurately estimate the ego pose in urban environments without relying on GNSS, while maintaining strong generalization ability. Our code and pre-trained model will be released publicly.

Title: LongRoPE2: Near-Lossless LLM Context Window Scaling

• Authors: Ning Shang, Li Lyna Zhang, Siyuan Wang, Gaokai Zhang, Gilsinia Lopez, Fan Yang, Weizhu Chen, Mao Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20082
• Pdf URL: https://arxiv.org/pdf/2502.20082
• Copy Paste: [[2502.20082]] LongRoPE2: Near-Lossless LLM Context Window Scaling(https://arxiv.org/abs/2502.20082)
• Keywords: large language model
• Abstract: LongRoPE2 is a novel approach that extends the effective context window of pre-trained large language models (LLMs) to the target length, while preserving the performance on the original shorter context window. This is achieved by three contributions: (1) a hypothesis that insufficient training in higher RoPE dimensions contributes to the persistent out-of-distribution (OOD) issues observed in existing methods; (2) an effective RoPE rescaling algorithm that adopts evolutionary search guided by "needle-driven" perplexity to address the insufficient training problem; (3) a mixed context window training approach that fine-tunes model weights to adopt rescaled RoPE for long-context sequences while preserving the short-context performance with the original RoPE. Extensive experiments on LLaMA3-8B and Phi3-mini-3.8B across various benchmarks validate the hypothesis and demonstrate the effectiveness of LongRoPE2. Remarkably, LongRoPE2 extends LLaMA3-8B to achieve a 128K effective context length while retaining over 98.5% of short-context performance, using only 10B tokens -- 80x fewer than Meta's approach, which fails to reach the target effective context length. Code will be available at this https URL.

Title: OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels

• Authors: Meng Lou, Yizhou Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20087
• Pdf URL: https://arxiv.org/pdf/2502.20087
• Copy Paste: [[2502.20087]] OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels(https://arxiv.org/abs/2502.20087)
• Keywords: segmentation
• Abstract: In the human vision system, top-down attention plays a crucial role in perception, wherein the brain initially performs an overall but rough scene analysis to extract salient cues (i.e., overview first), followed by a finer-grained examination to make more accurate judgments (i.e., look closely next). However, recent efforts in ConvNet designs primarily focused on increasing kernel size to obtain a larger receptive field without considering this crucial biomimetic mechanism to further improve performance. To this end, we propose a novel pure ConvNet vision backbone, termed OverLoCK, which is carefully devised from both the architecture and mixer perspectives. Specifically, we introduce a biomimetic Deep-stage Decomposition Strategy (DDS) that fuses semantically meaningful context representations into middle and deep layers by providing dynamic top-down context guidance at both feature and kernel weight levels. To fully unleash the power of top-down context guidance, we further propose a novel \textbf{Cont}ext-\textbf{Mix}ing Dynamic Convolution (ContMix) that effectively models long-range dependencies while preserving inherent local inductive biases even when the input resolution increases. These properties are absent in previous convolutions. With the support from both DDS and ContMix, our OverLoCK exhibits notable performance improvement over existing methods. For instance, OverLoCK-T achieves a Top-1 accuracy of 84.2\%, significantly surpassing ConvNeXt-B while only using around one-third of the FLOPs/parameters. On object detection with Cascade Mask R-CNN, our OverLoCK-S surpasses MogaNet-B by a significant 1\% in AP$^b$. On semantic segmentation with UperNet, our OverLoCK-T remarkably improves UniRepLKNet-T by 1.7\% in mIoU. Code is publicly available at this https URL.

Title: VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers

• Authors: Ziang Guo, Konstantin Gubernatorov, Selamawit Asfaw, Zakhar Yagudin, Dzmitry Tsetserukou
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.20108
• Pdf URL: https://arxiv.org/pdf/2502.20108
• Copy Paste: [[2502.20108]] VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers(https://arxiv.org/abs/2502.20108)
• Keywords: robust, diffusion, transformer
• Abstract: In autonomous driving, dynamic environment and corner cases pose significant challenges to the robustness of ego vehicle's decision-making. To address these challenges, commencing with the representation of state-action mapping in the end-to-end autonomous driving paradigm, we introduce a novel pipeline, VDT-Auto. Leveraging the advancement of the state understanding of Visual Language Model (VLM), incorporating with diffusion Transformer-based action generation, our VDT-Auto parses the environment geometrically and contextually for the conditioning of the diffusion process. Geometrically, we use a bird's-eye view (BEV) encoder to extract feature grids from the surrounding images. Contextually, the structured output of our fine-tuned VLM is processed into textual embeddings and noisy paths. During our diffusion process, the added noise for the forward process is sampled from the noisy path output of the fine-tuned VLM, while the extracted BEV feature grids and embedded texts condition the reverse process of our diffusion Transformers. Our VDT-Auto achieved 0.52m on average L2 errors and 21% on average collision rate in the nuScenes open-loop planning evaluation. Moreover, the real-world demonstration exhibited prominent generalizability of our VDT-Auto. The code and dataset will be released after acceptance.

Title: Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworks

• Authors: Jeripothula Prudviraj, Vikram Jamwal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20119
• Pdf URL: https://arxiv.org/pdf/2502.20119
• Copy Paste: [[2502.20119]] Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworks(https://arxiv.org/abs/2502.20119)
• Keywords: robust, extraction
• Abstract: Understanding the stroke-based evolution of visual artworks is useful for advancing artwork learning, appreciation, and interactive display. While the stroke sequence of renowned artworks remains largely unknown, formulating this sequence for near-natural image drawing processes can significantly enhance our understanding of artistic techniques. This paper introduces a novel method for approximating artwork stroke evolution through a proximity-based clustering mechanism. We first convert pixel images into vector images via parametric curves and then explore the clustering approach to determine the sequence order of extracted strokes. Our proposed algorithm demonstrates the potential to infer stroke sequences in unknown artworks. We evaluate the performance of our method using WikiArt data and qualitatively demonstrate the plausible stroke sequences. Additionally, we demonstrate the robustness of our approach to handle a wide variety of input image types such as line art, face sketches, paintings, and photographic images. By exploring stroke extraction and sequence construction, we aim to improve our understanding of the intricacies of the art development techniques and the step-by-step reconstruction process behind visual artworks, thereby enriching our understanding of the creative journey from the initial sketch to the final artwork.

Title: Self-Training Elicits Concise Reasoning in Large Language Models

• Authors: Tergel Munkhbat, Namgyu Ho, Seohyun Kim, Yongjin Yang, Yujin Kim, Se-Young Yun
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20122
• Pdf URL: https://arxiv.org/pdf/2502.20122
• Copy Paste: [[2502.20122]] Self-Training Elicits Concise Reasoning in Large Language Models(https://arxiv.org/abs/2502.20122)
• Keywords: robust, large language model
• Abstract: Chain-of-thought (CoT) reasoning has enabled large language models (LLMs) to utilize additional computation through intermediate tokens to solve complex tasks. However, we posit that typical reasoning traces contain many redundant tokens, incurring extraneous inference costs. Upon examination of the output distribution of current LLMs, we find evidence on their latent ability to reason more concisely, relative to their default behavior. To elicit this capability, we propose simple fine-tuning methods which leverage self-generated concise reasoning paths obtained by best-of-N sampling and few-shot conditioning, in task-specific settings. Our combined method achieves a 30% reduction in output tokens on average, across five model families on GSM8K and MATH, while maintaining average accuracy. By exploiting the fundamental stochasticity and in-context learning capabilities of LLMs, our self-training approach robustly elicits concise reasoning on a wide range of models, including those with extensive post-training. Code is available at this https URL

Title: Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer

• Authors: Yujie Li, Guannan Lai, Xin Yang, Yonghao Li, Marcello Bonsangue, Tianrui Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20124
• Pdf URL: https://arxiv.org/pdf/2502.20124
• Copy Paste: [[2502.20124]] Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer(https://arxiv.org/abs/2502.20124)
• Keywords: robust
• Abstract: Open-World Continual Learning (OWCL) is a challenging paradigm where models must incrementally learn new knowledge without forgetting while operating under an open-world assumption. This requires handling incomplete training data and recognizing unknown samples during inference. However, existing OWCL methods often treat open detection and continual learning as separate tasks, limiting their ability to integrate open-set detection and incremental classification in OWCL. Moreover, current approaches primarily focus on transferring knowledge from known samples, neglecting the insights derived from unknown/open samples. To address these limitations, we formalize four distinct OWCL scenarios and conduct comprehensive empirical experiments to explore potential challenges in OWCL. Our findings reveal a significant interplay between the open detection of unknowns and incremental classification of knowns, challenging a widely held assumption that unknown detection and known classification are orthogonal processes. Building on our insights, we propose \textbf{HoliTrans} (Holistic Knowns-Unknowns Knowledge Transfer), a novel OWCL framework that integrates nonlinear random projection (NRP) to create a more linearly separable embedding space and distribution-aware prototypes (DAPs) to construct an adaptive knowledge space. Particularly, our HoliTrans effectively supports knowledge transfer for both known and unknown samples while dynamically updating representations of open samples during OWCL. Extensive experiments across various OWCL scenarios demonstrate that HoliTrans outperforms 22 competitive baselines, bridging the gap between OWCL theory and practice and providing a robust, scalable framework for advancing open-world learning paradigms.

Title: FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute

• Authors: Sotiris Anagnostidis, Gregor Bachmann, Yeongmin Kim, Jonas Kohler, Markos Georgopoulos, Artsiom Sanakoyeu, Yuming Du, Albert Pumarola, Ali Thabet, Edgar Schönfeld
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20126
• Pdf URL: https://arxiv.org/pdf/2502.20126
• Copy Paste: [[2502.20126]] FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute(https://arxiv.org/abs/2502.20126)
• Keywords: diffusion, transformer
• Abstract: Despite their remarkable performance, modern Diffusion Transformers are hindered by substantial resource requirements during inference, stemming from the fixed and large amount of compute needed for each denoising step. In this work, we revisit the conventional static paradigm that allocates a fixed compute budget per denoising iteration and propose a dynamic strategy instead. Our simple and sample-efficient framework enables pre-trained DiT models to be converted into \emph{flexible} ones -- dubbed FlexiDiT -- allowing them to process inputs at varying compute budgets. We demonstrate how a single \emph{flexible} model can generate images without any drop in quality, while reducing the required FLOPs by more than $40$\% compared to their static counterparts, for both class-conditioned and text-conditioned image generation. Our method is general and agnostic to input and conditioning modalities. We show how our approach can be readily extended for video generation, where FlexiDiT models generate samples with up to $75$\% less compute without compromising performance.

Title: CLIP-driven Dual Feature Enhancing Network for Gaze Estimation

• Authors: Lin Zhang, Yi Tian, Wanru Xu, Yi Jin, Yaping Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20128
• Pdf URL: https://arxiv.org/pdf/2502.20128
• Copy Paste: [[2502.20128]] CLIP-driven Dual Feature Enhancing Network for Gaze Estimation(https://arxiv.org/abs/2502.20128)
• Keywords: robust
• Abstract: The complex application scenarios have raised critical requirements for precise and generalizable gaze estimation methods. Recently, the pre-trained CLIP has achieved remarkable performance on various vision tasks, but its potentials have not been fully exploited in gaze estimation. In this paper, we propose a novel CLIP-driven Dual Feature Enhancing Network (CLIP-DFENet), which boosts gaze estimation performance with the help of CLIP under a novel `main-side' collaborative enhancing strategy. Accordingly, a Language-driven Differential Module (LDM) is designed on the basis of the CLIP's text encoder to reveal the semantic difference of gaze. This module could empower our Core Feature Extractor with the capability of characterizing the gaze-related semantic information. Moreover, a Vision-driven Fusion Module (VFM) is introduced to strengthen the generalized and valuable components of visual embeddings obtained via CLIP's image encoder, and utilizes them to further improve the generalization of the features captured by Core Feature Extractor. Finally, a robust Double-head Gaze Regressor is adopted to map the enhanced features to gaze directions. Extensive experimental results on four challenging datasets over within-domain and cross-domain tasks demonstrate the discriminability and generalizability of our CLIP-DFENet.

Title: Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking

• Authors: Yifan Zhang, Wenyu Du, Dongming Jin, Jie Fu, Zhi Jin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20129
• Pdf URL: https://arxiv.org/pdf/2502.20129
• Copy Paste: [[2502.20129]] Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking(https://arxiv.org/abs/2502.20129)
• Keywords: robust, transformer, large language model
• Abstract: Chain-of-Thought (CoT) significantly enhances the performance of large language models (LLMs) across a wide range of tasks, and prior research shows that CoT can theoretically increase expressiveness. However, there is limited mechanistic understanding of the algorithms that Transformer+CoT can learn. In this work, we (1) evaluate the state tracking capabilities of Transformer+CoT and its variants, confirming the effectiveness of CoT. (2) Next, we identify the circuit, a subset of model components, responsible for tracking the world state, finding that late-layer MLP neurons play a key role. We propose two metrics, compression and distinction, and show that the neuron sets for each state achieve nearly 100% accuracy, providing evidence of an implicit finite state automaton (FSA) embedded within the model. (3) Additionally, we explore three realistic settings: skipping intermediate steps, introducing data noise, and testing length generalization. Our results demonstrate that Transformer+CoT learns robust algorithms (FSA), highlighting its resilience in challenging scenarios.

Title: QPM: Discrete Optimization for Globally Interpretable Image Classification

• Authors: Thomas Norrenbrock, Timo Kaiser, Sovan Biswas, Ramesh Manuvinakurike, Bodo Rosenhahn
• Subjects: cs.CV, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20130
• Pdf URL: https://arxiv.org/pdf/2502.20130
• Copy Paste: [[2502.20130]] QPM: Discrete Optimization for Globally Interpretable Image Classification(https://arxiv.org/abs/2502.20130)
• Keywords: interpretability
• Abstract: Understanding the classifications of deep neural networks, e.g. used in safety-critical situations, is becoming increasingly important. While recent models can locally explain a single decision, to provide a faithful global explanation about an accurate model's general behavior is a more challenging open task. Towards that goal, we introduce the Quadratic Programming Enhanced Model (QPM), which learns globally interpretable class representations. QPM represents every class with a binary assignment of very few, typically 5, features, that are also assigned to other classes, ensuring easily comparable contrastive class representations. This compact binary assignment is found using discrete optimization based on predefined similarity measures and interpretability constraints. The resulting optimal assignment is used to fine-tune the diverse features, so that each of them becomes the shared general concept between the assigned classes. Extensive evaluations show that QPM delivers unprecedented global interpretability across small and large-scale datasets while setting the state of the art for the accuracy of interpretable models.

Title: Regional climate projections using a deep learning--based model-ranking and downscaling framework: Application to European climate zones

• Authors: Parthiban Loganathan, Elias Zea, Ricardo Vinuesa, Evelyn Otero
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2502.20132
• Pdf URL: https://arxiv.org/pdf/2502.20132
• Copy Paste: [[2502.20132]] Regional climate projections using a deep learning--based model-ranking and downscaling framework: Application to European climate zones(https://arxiv.org/abs/2502.20132)
• Keywords: transformer
• Abstract: Accurate regional climate forecast calls for high-resolution downscaling of Global Climate Models (GCMs). This work presents a deep-learning-based multi-model evaluation and downscaling framework ranking 32 Coupled Model Intercomparison Project Phase 6 (CMIP6) models using a Deep Learning-TOPSIS (DL-TOPSIS) mechanism and so refines outputs using advanced deep-learning models. Using nine performance criteria, five Köppen-Geiger climate zones -- Tropical, Arid, Temperate, Continental, and Polar -- are investigated over four seasons. While TaiESM1 and CMCC-CM2-SR5 show notable biases, ranking results show that NorESM2-LM, GISS-E2-1-G, and HadGEM3-GC31-LL outperform other models. Four models contribute to downscaling the top-ranked GCMs to 0.1$^{\circ}$ resolution: Vision Transformer (ViT), Geospatial Spatiotemporal Transformer with Attention and Imbalance-Aware Network (GeoSTANet), CNN-LSTM, and CNN-Long Short-Term Memory (ConvLSTM). Effectively capturing temperature extremes (TXx, TNn), GeoSTANet achieves the highest accuracy (Root Mean Square Error (RMSE) = 1.57$^{\circ}$C, Kling-Gupta Efficiency (KGE) = 0.89, Nash-Sutcliffe Efficiency (NSE) = 0.85, Correlation ($r$) = 0.92), so reducing RMSE by 20\% over ConvLSTM. CNN-LSTM and ConvLSTM do well in Continental and Temperate zones; ViT finds fine-scale temperature fluctuations difficult. These results confirm that multi-criteria ranking improves GCM selection for regional climate studies and transformer-based downscaling exceeds conventional deep-learning methods. This framework offers a scalable method to enhance high-resolution climate projections, benefiting impact assessments and adaptation plans.

Title: Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models

• Authors: Itay Benou, Tammy Riklin-Raviv
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20134
• Pdf URL: https://arxiv.org/pdf/2502.20134
• Copy Paste: [[2502.20134]] Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models(https://arxiv.org/abs/2502.20134)
• Keywords: segmentation
• Abstract: Modern deep neural networks have now reached human-level performance across a variety of tasks. However, unlike humans they lack the ability to explain their decisions by showing where and telling what concepts guided them. In this work, we present a unified framework for transforming any vision neural network into a spatially and conceptually interpretable model. We introduce a spatially-aware concept bottleneck layer that projects "black-box" features of pre-trained backbone models into interpretable concept maps, without requiring human labels. By training a classification layer over this bottleneck, we obtain a self-explaining model that articulates which concepts most influenced its prediction, along with heatmaps that ground them in the input image. Accordingly, we name this method "Spatially-Aware and Label-Free Concept Bottleneck Model" (SALF-CBM). Our results show that the proposed SALF-CBM: (1) Outperforms non-spatial CBM methods, as well as the original backbone, on a variety of classification tasks; (2) Produces high-quality spatial explanations, outperforming widely used heatmap-based methods on a zero-shot segmentation task; (3) Facilitates model exploration and debugging, enabling users to query specific image regions and refine the model's decisions by locally editing its concept maps.

Title: Robust sensitivity control in digital pathology via tile score distribution matching

• Authors: Arthur Pignet, John Klein, Genevieve Robin, Antoine Olivier
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20144
• Pdf URL: https://arxiv.org/pdf/2502.20144
• Copy Paste: [[2502.20144]] Robust sensitivity control in digital pathology via tile score distribution matching(https://arxiv.org/abs/2502.20144)
• Keywords: robust
• Abstract: Deploying digital pathology models across medical centers is challenging due to distribution shifts. Recent advances in domain generalization improve model transferability in terms of aggregated performance measured by the Area Under Curve (AUC). However, clinical regulations often require to control the transferability of other metrics, such as prescribed sensitivity levels. We introduce a novel approach to control the sensitivity of whole slide image (WSI) classification models, based on optimal transport and Multiple Instance Learning (MIL). Validated across multiple cohorts and tasks, our method enables robust sensitivity control with only a handful of calibration samples, providing a practical solution for reliable deployment of computational pathology systems.

Title: Adaptive H&E-IHC information fusion staining framework based on feature extra

• Authors: Yifan Jia, Xingda Yu, Zhengyang Ji, Songning Lai, Yutao Yue
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20156
• Pdf URL: https://arxiv.org/pdf/2502.20156
• Copy Paste: [[2502.20156]] Adaptive H&E-IHC information fusion staining framework based on feature extra(https://arxiv.org/abs/2502.20156)
• Keywords: extraction, generative
• Abstract: Immunohistochemistry (IHC) staining plays a significant role in the evaluation of diseases such as breast cancer. The H&E-to-IHC transformation based on generative models provides a simple and cost-effective method for obtaining IHC images. Although previous models can perform digital coloring well, they still suffer from (i) coloring only through the pixel features that are not prominent in HE, which is easy to cause information loss in the coloring process; (ii) The lack of pixel-perfect H&E-IHC groundtruth pairs poses a challenge to the classical L1 this http URL address the above challenges, we propose an adaptive information enhanced coloring framework based on feature extractors. We first propose the VMFE module to effectively extract the color information features using multi-scale feature extraction and wavelet transform convolution, while combining the shared decoder for feature fusion. The high-performance dual feature extractor of H&E-IHC is trained by contrastive learning, which can effectively perform feature alignment of HE-IHC in high latitude space. At the same time, the trained feature encoder is used to enhance the features and adaptively adjust the loss in the HE section staining process to solve the problems related to unclear and asymmetric information. We have tested on different datasets and achieved excellent this http URL code is available at this https URL

Title: Learning to Generalize without Bias for Open-Vocabulary Action Recognition

• Authors: Yating Yu, Congqi Cao, Yifan Zhang, Yanning Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20158
• Pdf URL: https://arxiv.org/pdf/2502.20158
• Copy Paste: [[2502.20158]] Learning to Generalize without Bias for Open-Vocabulary Action Recognition(https://arxiv.org/abs/2502.20158)
• Keywords: robust
• Abstract: Leveraging the effective visual-text alignment and static generalizability from CLIP, recent video learners adopt CLIP initialization with further regularization or recombination for generalization in open-vocabulary action recognition in-context. However, due to the static bias of CLIP, such video learners tend to overfit on shortcut static features, thereby compromising their generalizability, especially to novel out-of-context actions. To address this issue, we introduce Open-MeDe, a novel Meta-optimization framework with static Debiasing for Open-vocabulary action recognition. From a fresh perspective of generalization, Open-MeDe adopts a meta-learning approach to improve known-to-open generalizing and image-to-video debiasing in a cost-effective manner. Specifically, Open-MeDe introduces a cross-batch meta-optimization scheme that explicitly encourages video learners to quickly generalize to arbitrary subsequent data via virtual evaluation, steering a smoother optimization landscape. In effect, the free of CLIP regularization during optimization implicitly mitigates the inherent static bias of the video meta-learner. We further apply self-ensemble over the optimization trajectory to obtain generic optimal parameters that can achieve robust generalization to both in-context and out-of-context novel data. Extensive evaluations show that Open-MeDe not only surpasses state-of-the-art regularization methods tailored for in-context open-vocabulary action recognition but also substantially excels in out-of-context scenarios.

Title: Gradient-Guided Annealing for Domain Generalization

• Authors: Aristotelis Ballas, Christos Diou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20162
• Pdf URL: https://arxiv.org/pdf/2502.20162
• Copy Paste: [[2502.20162]] Gradient-Guided Annealing for Domain Generalization(https://arxiv.org/abs/2502.20162)
• Keywords: robust
• Abstract: Domain Generalization (DG) research has gained considerable traction as of late, since the ability to generalize to unseen data distributions is a requirement that eludes even state-of-the-art training algorithms. In this paper we observe that the initial iterations of model training play a key role in domain generalization effectiveness, since the loss landscape may be significantly different across the training and test distributions, contrary to the case of i.i.d. data. Conflicts between gradients of the loss components of each domain lead the optimization procedure to undesirable local minima that do not capture the domain-invariant features of the target classes. We propose alleviating domain conflicts in model optimization, by iteratively annealing the parameters of a model in the early stages of training and searching for points where gradients align between domains. By discovering a set of parameter values where gradients are updated towards the same direction for each data distribution present in the training set, the proposed Gradient-Guided Annealing (GGA) algorithm encourages models to seek out minima that exhibit improved robustness against domain shifts. The efficacy of GGA is evaluated on five widely accepted and challenging image classification domain generalization benchmarks, where its use alone is able to establish highly competitive or even state-of-the-art performance. Moreover, when combined with previously proposed domain-generalization algorithms it is able to consistently improve their effectiveness by significant margins.

Title: Similarity-Distance-Magnitude Universal Verification

• Authors: Allen Schmaltz
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20167
• Pdf URL: https://arxiv.org/pdf/2502.20167
• Copy Paste: [[2502.20167]] Similarity-Distance-Magnitude Universal Verification(https://arxiv.org/abs/2502.20167)
• Keywords: robust, interpretability
• Abstract: We solve the neural network robustness problem by adding Similarity (i.e., correctly predicted depth-matches into training)-awareness and Distance-to-training-distribution-awareness to the existing output Magnitude (i.e., decision-boundary)-awareness of the softmax function. The resulting sdm activation function provides strong signals of the relative epistemic (reducible) predictive uncertainty. We use this novel behavior to further address the complementary HCI problem of mapping the output to human-interpretable summary statistics over relevant partitions of a held-out calibration set. Estimates of prediction-conditional uncertainty are obtained via a parsimonious learned transform over the class-conditional empirical CDFs of the output of a final-layer sdm activation function. For decision-making and as an intrinsic model check, estimates of class-conditional accuracy are obtained by further partitioning the high-probability regions of this calibrated output into class-conditional, region-specific CDFs. The uncertainty estimates from sdm calibration are remarkably robust to test-time distribution shifts and out-of-distribution inputs; incorporate awareness of the effective sample size; provide estimates of uncertainty from the learning and data splitting processes; and are well-suited for selective classification and conditional branching for additional test-time compute based on the predictive uncertainty, as for selective LLM generation, routing, and composition over multiple models and retrieval. Finally, we construct sdm networks, LLMs with uncertainty-aware verification and interpretability-by-exemplar as intrinsic properties. We provide open-source software implementing these results.

Title: Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies

• Authors: Toon Vandendriessche, Mathieu De Coster, Annelies Lejon, Joni Dambre
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20171
• Pdf URL: https://arxiv.org/pdf/2502.20171
• Copy Paste: [[2502.20171]] Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies(https://arxiv.org/abs/2502.20171)
• Keywords: robust
• Abstract: Isolated Sign Language Recognition (ISLR) is crucial for scalable sign language technology, yet language-specific approaches limit current models. To address this, we propose a one-shot learning approach that generalises across languages and evolving vocabularies. Our method involves pretraining a model to embed signs based on essential features and using a dense vector search for rapid, accurate recognition of unseen signs. We achieve state-of-the-art results, including 50.8% one-shot MRR on a large dictionary containing 10,235 unique signs from a different language than the training set. Our approach is robust across languages and support sets, offering a scalable, adaptable solution for ISLR. Co-created with the Deaf and Hard of Hearing (DHH) community, this method aligns with real-world needs, and advances scalable sign language recognition.

Title: Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think

• Authors: Liang Chen, Shuai Bai, Wenhao Chai, Weichu Xie, Haozhe Zhao, Leon Vinci, Junyang Lin, Baobao Chang
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20172
• Pdf URL: https://arxiv.org/pdf/2502.20172
• Copy Paste: [[2502.20172]] Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think(https://arxiv.org/abs/2502.20172)
• Keywords: diffusion, transformer
• Abstract: The field of advanced text-to-image generation is witnessing the emergence of unified frameworks that integrate powerful text encoders, such as CLIP and T5, with Diffusion Transformer backbones. Although there have been efforts to control output images with additional conditions, like canny and depth map, a comprehensive framework for arbitrary text-image interleaved control is still lacking. This gap is especially evident when attempting to merge concepts or visual elements from multiple images in the generation process. To mitigate the gap, we conducted preliminary experiments showing that large multimodal models (LMMs) offer an effective shared representation space, where image and text can be well-aligned to serve as a condition for external diffusion models. Based on this discovery, we propose Dream Engine, an efficient and unified framework designed for arbitrary text-image interleaved control in image generation models. Building on powerful text-to-image models like SD3.5, we replace the original text-only encoders by incorporating versatile multimodal information encoders such as QwenVL. Our approach utilizes a two-stage training paradigm, consisting of joint text-image alignment and multimodal interleaved instruction tuning. Our experiments demonstrate that this training method is effective, achieving a 0.69 overall score on the GenEval benchmark, and matching the performance of state-of-the-art text-to-image models like SD3.5 and FLUX.

Title: SSD: A State-based Stealthy Backdoor Attack For Navigation System in UAV Route Planning

• Authors: Zhaoxuan Wang, Yang Li, Jie Zhang, Xingshuo Han, Kangbo Liu, Lyu Yang, yuan Zhou, Tianwei Zhang, Quan Pan
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.20178
• Pdf URL: https://arxiv.org/pdf/2502.20178
• Copy Paste: [[2502.20178]] SSD: A State-based Stealthy Backdoor Attack For Navigation System in UAV Route Planning(https://arxiv.org/abs/2502.20178)
• Keywords: security, attack, steal
• Abstract: Unmanned aerial vehicles (UAVs) are increasingly employed to perform high-risk tasks that require minimal human intervention. However, UAVs face escalating cybersecurity threats, particularly from GNSS spoofing attacks. While previous studies have extensively investigated the impacts of GNSS spoofing on UAVs, few have focused on its effects on specific tasks. Moreover, the influence of UAV motion states on the assessment of network security risks is often overlooked. To address these gaps, we first provide a detailed evaluation of how motion states affect the effectiveness of network attacks. We demonstrate that nonlinear motion states not only enhance the effectiveness of position spoofing in GNSS spoofing attacks but also reduce the probability of speed-related attack detection. Building upon this, we propose a state-triggered backdoor attack method (SSD) to deceive GNSS systems and assess its risk to trajectory planning tasks. Extensive validation of SSD's effectiveness and stealthiness is conducted. Experimental results show that, with appropriately tuned hyperparameters, SSD significantly increases positioning errors and the risk of task failure, while maintaining 100% stealth across three state-of-the-art detectors.

Title: Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge

• Authors: Yan-Lun Chen, Yi-Ru Wei, Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Ying-Dar Lin, Yu-Sung Wu, Wei-Bin Lee
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20186
• Pdf URL: https://arxiv.org/pdf/2502.20186
• Copy Paste: [[2502.20186]] Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge(https://arxiv.org/abs/2502.20186)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) demonstrate strong task-specific capabilities through fine-tuning, but merging multiple fine-tuned models often leads to degraded performance due to overlapping instruction-following components. Task Arithmetic (TA), which combines task vectors derived from fine-tuning, enables multi-task learning and task forgetting but struggles to isolate task-specific knowledge from general instruction-following behavior. To address this, we propose Layer-Aware Task Arithmetic (LATA), a novel approach that assigns layer-specific weights to task vectors based on their alignment with instruction-following or task-specific components. By amplifying task-relevant layers and attenuating instruction-following layers, LATA improves task learning and forgetting performance while preserving overall model utility. Experiments on multiple benchmarks, including WikiText-2, GSM8K, and HumanEval, demonstrate that LATA outperforms existing methods in both multi-task learning and selective task forgetting, achieving higher task accuracy and alignment with minimal degradation in output quality. Our findings highlight the importance of layer-wise analysis in disentangling task-specific and general-purpose knowledge, offering a robust framework for efficient model merging and editing.

Title: ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models

• Authors: Haibin Chen, Kangtao Lv, Chengwei Hu, Yanshi Li, Yujin Yuan, Yancheng He, Xingyao Zhang, Langming Liu, Shilei Liu, Wenbo Su, Bo Zheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20196
• Pdf URL: https://arxiv.org/pdf/2502.20196
• Copy Paste: [[2502.20196]] ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models(https://arxiv.org/abs/2502.20196)
• Keywords: large language model
• Abstract: With the increasing use of Large Language Models (LLMs) in fields such as e-commerce, domain-specific concept evaluation benchmarks are crucial for assessing their domain capabilities. Existing LLMs may generate factually incorrect information within the complex e-commerce applications. Therefore, it is necessary to build an e-commerce concept benchmark. Existing benchmarks encounter two primary challenges: (1) handle the heterogeneous and diverse nature of tasks, (2) distinguish between generality and specificity within the e-commerce field. To address these problems, we propose \textbf{ChineseEcomQA}, a scalable question-answering benchmark focused on fundamental e-commerce concepts. ChineseEcomQA is built on three core characteristics: \textbf{Focus on Fundamental Concept}, \textbf{E-commerce Generality} and \textbf{E-commerce Expertise}. Fundamental concepts are designed to be applicable across a diverse array of e-commerce tasks, thus addressing the challenge of heterogeneity and diversity. Additionally, by carefully balancing generality and specificity, ChineseEcomQA effectively differentiates between broad e-commerce concepts, allowing for precise validation of domain capabilities. We achieve this through a scalable benchmark construction process that combines LLM validation, Retrieval-Augmented Generation (RAG) validation, and rigorous manual annotation. Based on ChineseEcomQA, we conduct extensive evaluations on mainstream LLMs and provide some valuable insights. We hope that ChineseEcomQA could guide future domain-specific evaluations, and facilitate broader LLM adoption in e-commerce applications.

Title: 4Deform: Neural Surface Deformation for Robust Shape Interpolation

• Authors: Lu Sang, Zehranaz Canfes, Dongliang Cao, Riccardo Marin, Florian Bernard, Daniel Cremers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20208
• Pdf URL: https://arxiv.org/pdf/2502.20208
• Copy Paste: [[2502.20208]] 4Deform: Neural Surface Deformation for Robust Shape Interpolation(https://arxiv.org/abs/2502.20208)
• Keywords: robust
• Abstract: Generating realistic intermediate shapes between non-rigidly deformed shapes is a challenging task in computer vision, especially with unstructured data (e.g., point clouds) where temporal consistency across frames is lacking, and topologies are changing. Most interpolation methods are designed for structured data (i.e., meshes) and do not apply to real-world point clouds. In contrast, our approach, 4Deform, leverages neural implicit representation (NIR) to enable free topology changing shape deformation. Unlike previous mesh-based methods that learn vertex-based deformation fields, our method learns a continuous velocity field in Euclidean space. Thus, it is suitable for less structured data such as point clouds. Additionally, our method does not require intermediate-shape supervision during training; instead, we incorporate physical and geometrical constraints to regularize the velocity field. We reconstruct intermediate surfaces using a modified level-set equation, directly linking our NIR with the velocity field. Experiments show that our method significantly outperforms previous NIR approaches across various scenarios (e.g., noisy, partial, topology-changing, non-isometric shapes) and, for the first time, enables new applications like 4D Kinect sequence upsampling and real-world high-resolution mesh deformation.

Title: Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars

• Authors: Tobias Kirschstein, Javier Romero, Artem Sevastopolsky, Matthias Nießner, Shunsuke Saito
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20220
• Pdf URL: https://arxiv.org/pdf/2502.20220
• Copy Paste: [[2502.20220]] Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars(https://arxiv.org/abs/2502.20220)
• Keywords: robust
• Abstract: Traditionally, creating photo-realistic 3D head avatars requires a studio-level multi-view capture setup and expensive optimization during test-time, limiting the use of digital human doubles to the VFX industry or offline renderings. To address this shortcoming, we present Avat3r, which regresses a high-quality and animatable 3D head avatar from just a few input images, vastly reducing compute requirements during inference. More specifically, we make Large Reconstruction Models animatable and learn a powerful prior over 3D human heads from a large multi-view video dataset. For better 3D head reconstructions, we employ position maps from DUSt3R and generalized feature maps from the human foundation model Sapiens. To animate the 3D head, our key discovery is that simple cross-attention to an expression code is already sufficient. Finally, we increase robustness by feeding input images with different expressions to our model during training, enabling the reconstruction of 3D head avatars from inconsistent inputs, e.g., an imperfect phone capture with accidental movement, or frames from a monocular video. We compare Avat3r with current state-of-the-art methods for few-input and single-input scenarios, and find that our method has a competitive advantage in both tasks. Finally, we demonstrate the wide applicability of our proposed model, creating 3D head avatars from images of different sources, smartphone captures, single images, and even out-of-domain inputs like antique busts. Project website: this https URL

Title: URL Inspection Tasks: Helping Users Detect Phishing Links in Emails

• Authors: Daniele Lain, Yoshimichi Nakatsuka, Kari Kostiainen, Gene Tsudik, Srdjan Capkun
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.20234
• Pdf URL: https://arxiv.org/pdf/2502.20234
• Copy Paste: [[2502.20234]] URL Inspection Tasks: Helping Users Detect Phishing Links in Emails(https://arxiv.org/abs/2502.20234)
• Keywords: security, defense, attack
• Abstract: The most widespread type of phishing attack involves email messages with links pointing to malicious content. Despite user training and the use of detection techniques, these attacks are still highly effective. Recent studies show that it is user inattentiveness, rather than lack of education, that is one of the key factors in successful phishing attacks. To this end, we develop a novel phishing defense mechanism based on URL inspection tasks: small challenges (loosely inspired by CAPTCHAs) that, to be solved, require users to interact with, and understand, the basic URL structure. We implemented and evaluated three tasks that act as ``barriers'' to visiting the website: (1) correct click-selection from a list of URLs, (2) mouse-based highlighting of the domain-name URL component, and (3) re-typing the domain-name. These tasks follow best practices in security interfaces and warning design. We assessed the efficacy of these tasks through an extensive on-line user study with 2,673 participants from three different cultures, native languages, and alphabets. Results show that these tasks significantly decrease the rate of successful phishing attempts, compared to the baseline case. Results also showed the highest efficacy for difficult URLs, such as typo-squats, with which participants struggled the most. This highlights the importance of (1) slowing down users while focusing their attention and (2) helping them understand the URL structure (especially, the domain-name component thereof) and matching it to their intent.

Title: Attention Distillation: A Unified Approach to Visual Characteristics Transfer

• Authors: Yang Zhou, Xu Gao, Zichong Chen, Hui Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20235
• Pdf URL: https://arxiv.org/pdf/2502.20235
• Copy Paste: [[2502.20235]] Attention Distillation: A Unified Approach to Visual Characteristics Transfer(https://arxiv.org/abs/2502.20235)
• Keywords: diffusion, generative
• Abstract: Recent advances in generative diffusion models have shown a notable inherent understanding of image style and semantics. In this paper, we leverage the self-attention features from pretrained diffusion networks to transfer the visual characteristics from a reference to generated images. Unlike previous work that uses these features as plug-and-play attributes, we propose a novel attention distillation loss calculated between the ideal and current stylization results, based on which we optimize the synthesized image via backpropagation in latent space. Next, we propose an improved Classifier Guidance that integrates attention distillation loss into the denoising sampling process, further accelerating the synthesis and enabling a broad range of image generation applications. Extensive experiments have demonstrated the extraordinary performance of our approach in transferring the examples' style, appearance, and texture to new images in synthesis. Code is available at this https URL.

Title: Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks

• Authors: Gianluca Bencomo, Max Gupta, Ioana Marinescu, R. Thomas McCoy, Thomas L. Griffiths
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20237
• Pdf URL: https://arxiv.org/pdf/2502.20237
• Copy Paste: [[2502.20237]] Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks(https://arxiv.org/abs/2502.20237)
• Keywords: robust, transformer
• Abstract: Artificial neural networks can acquire many aspects of human knowledge from data, making them promising as models of human learning. But what those networks can learn depends upon their inductive biases -- the factors other than the data that influence the solutions they discover -- and the inductive biases of neural networks remain poorly understood, limiting our ability to draw conclusions about human learning from the performance of these systems. Cognitive scientists and machine learning researchers often focus on the architecture of a neural network as a source of inductive bias. In this paper we explore the impact of another source of inductive bias -- the initial weights of the network -- using meta-learning as a tool for finding initial weights that are adapted for specific problems. We evaluate four widely-used architectures -- MLPs, CNNs, LSTMs, and Transformers -- by meta-training 430 different models across three tasks requiring different biases and forms of generalization. We find that meta-learning can substantially reduce or entirely eliminate performance differences across architectures and data representations, suggesting that these factors may be less important as sources of inductive bias than is typically assumed. When differences are present, architectures and data representations that perform well without meta-learning tend to meta-train more effectively. Moreover, all architectures generalize poorly on problems that are far from their meta-training experience, underscoring the need for stronger inductive biases for robust generalization.

Title: FINEREASON: Evaluating and Improving LLMs' Deliberate Reasoning through Reflective Puzzle Solving

• Authors: Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Chaoqun Liu, Lidong Bing, Deli Zhao, Anh Tuan Luu, Yu Rong
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20238
• Pdf URL: https://arxiv.org/pdf/2502.20238
• Copy Paste: [[2502.20238]] FINEREASON: Evaluating and Improving LLMs' Deliberate Reasoning through Reflective Puzzle Solving(https://arxiv.org/abs/2502.20238)
• Keywords: large language model
• Abstract: Many challenging reasoning tasks require not just rapid, intuitive responses, but a more deliberate, multi-step approach. Recent progress in large language models (LLMs) highlights an important shift from the "System 1" way of quick reactions to the "System 2" style of reflection-and-correction problem solving. However, current benchmarks heavily rely on the final-answer accuracy, leaving much of a model's intermediate reasoning steps unexamined. This fails to assess the model's ability to reflect and rectify mistakes within the reasoning process. To bridge this gap, we introduce FINEREASON, a logic-puzzle benchmark for fine-grained evaluation of LLMs' reasoning capabilities. Each puzzle can be decomposed into atomic steps, making it ideal for rigorous validation of intermediate correctness. Building on this, we introduce two tasks: state checking, and state transition, for a comprehensive evaluation of how models assess the current situation and plan the next move. To support broader research, we also provide a puzzle training set aimed at enhancing performance on general mathematical tasks. We show that models trained on our state checking and transition data demonstrate gains in math reasoning by up to 5.1% on GSM8K.

Title: The Impact of Transparency in AI Systems on Users' Data-Sharing Intentions: A Scenario-Based Experiment

• Authors: Julian Rosenberger, Sophie Kuhlemann, Verena Tiefenbeck, Mathias Kraus, Patrick Zschech
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2502.20243
• Pdf URL: https://arxiv.org/pdf/2502.20243
• Copy Paste: [[2502.20243]] The Impact of Transparency in AI Systems on Users' Data-Sharing Intentions: A Scenario-Based Experiment(https://arxiv.org/abs/2502.20243)
• Keywords: privacy
• Abstract: Artificial Intelligence (AI) systems are frequently employed in online services to provide personalized experiences to users based on large collections of data. However, AI systems can be designed in different ways, with black-box AI systems appearing as complex data-processing engines and white-box AI systems appearing as fully transparent data-processors. As such, it is reasonable to assume that these different design choices also affect user perception and thus their willingness to share data. To this end, we conducted a pre-registered, scenario-based online experiment with 240 participants and investigated how transparent and non-transparent data-processing entities influenced data-sharing intentions. Surprisingly, our results revealed no significant difference in willingness to share data across entities, challenging the notion that transparency increases data-sharing willingness. Furthermore, we found that a general attitude of trust towards AI has a significant positive influence, especially in the transparent AI condition, whereas privacy concerns did not significantly affect data-sharing decisions.

Title: From Retrieval to Generation: Comparing Different Approaches

• Authors: Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Mohammed Ali, Adam Jatowt
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20245
• Pdf URL: https://arxiv.org/pdf/2502.20245
• Copy Paste: [[2502.20245]] From Retrieval to Generation: Comparing Different Approaches(https://arxiv.org/abs/2502.20245)
• Keywords: generative
• Abstract: Knowledge-intensive tasks, particularly open-domain question answering (ODQA), document reranking, and retrieval-augmented language modeling, require a balance between retrieval accuracy and generative flexibility. Traditional retrieval models such as BM25 and Dense Passage Retrieval (DPR), efficiently retrieve from large corpora but often lack semantic depth. Generative models like GPT-4-o provide richer contextual understanding but face challenges in maintaining factual consistency. In this work, we conduct a systematic evaluation of retrieval-based, generation-based, and hybrid models, with a primary focus on their performance in ODQA and related retrieval-augmented tasks. Our results show that dense retrievers, particularly DPR, achieve strong performance in ODQA with a top-1 accuracy of 50.17\% on NQ, while hybrid models improve nDCG@10 scores on BEIR from 43.42 (BM25) to 52.59, demonstrating their strength in document reranking. Additionally, we analyze language modeling tasks using WikiText-103, showing that retrieval-based approaches like BM25 achieve lower perplexity compared to generative and hybrid methods, highlighting their utility in retrieval-augmented generation. By providing detailed comparisons and practical insights into the conditions where each approach excels, we aim to facilitate future optimizations in retrieval, reranking, and generative models for ODQA and related knowledge-intensive applications.

Title: Beyond Natural Language Perplexity: Detecting Dead Code Poisoning in Code Generation Datasets

• Authors: Chichien Tsai, Chiamu Yu, Yingdar Lin, Yusung Wu, Weibin Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20246
• Pdf URL: https://arxiv.org/pdf/2502.20246
• Copy Paste: [[2502.20246]] Beyond Natural Language Perplexity: Detecting Dead Code Poisoning in Code Generation Datasets(https://arxiv.org/abs/2502.20246)
• Keywords: secure, security, attack, robust, large language model
• Abstract: The increasing adoption of large language models (LLMs) for code-related tasks has raised concerns about the security of their training datasets. One critical threat is dead code poisoning, where syntactically valid but functionally redundant code is injected into training data to manipulate model behavior. Such attacks can degrade the performance of neural code search systems, leading to biased or insecure code suggestions. Existing detection methods, such as token-level perplexity analysis, fail to effectively identify dead code due to the structural and contextual characteristics of programming languages. In this paper, we propose DePA (Dead Code Perplexity Analysis), a novel line-level detection and cleansing method tailored to the structural properties of code. DePA computes line-level perplexity by leveraging the contextual relationships between code lines and identifies anomalous lines by comparing their perplexity to the overall distribution within the file. Our experiments on benchmark datasets demonstrate that DePA significantly outperforms existing methods, achieving 0.14-0.19 improvement in detection F1-score and a 44-65% increase in poisoned segment localization precision. Furthermore, DePA enhances detection speed by 0.62-23x, making it practical for large-scale dataset cleansing. Overall, by addressing the unique challenges of dead code poisoning, DePA provides a robust and efficient solution for safeguarding the integrity of code generation model training datasets.

Title: Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels

• Authors: Pierre Vuillecard, Jean-Marc Odobez
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20249
• Pdf URL: https://arxiv.org/pdf/2502.20249
• Copy Paste: [[2502.20249]] Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels(https://arxiv.org/abs/2502.20249)
• Keywords: transformer
• Abstract: Accurate 3D gaze estimation in unconstrained real-world environments remains a significant challenge due to variations in appearance, head pose, occlusion, and the limited availability of in-the-wild 3D gaze datasets. To address these challenges, we introduce a novel Self-Training Weakly-Supervised Gaze Estimation framework (ST-WSGE). This two-stage learning framework leverages diverse 2D gaze datasets, such as gaze-following data, which offer rich variations in appearances, natural scenes, and gaze distributions, and proposes an approach to generate 3D pseudo-labels and enhance model generalization. Furthermore, traditional modality-specific models, designed separately for images or videos, limit the effective use of available training data. To overcome this, we propose the Gaze Transformer (GaT), a modality-agnostic architecture capable of simultaneously learning static and dynamic gaze information from both image and video datasets. By combining 3D video datasets with 2D gaze target labels from gaze following tasks, our approach achieves the following key contributions: (i) Significant state-of-the-art improvements in within-domain and cross-domain generalization on unconstrained benchmarks like Gaze360 and GFIE, with notable cross-modal gains in video gaze estimation; (ii) Superior cross-domain performance on datasets such as MPIIFaceGaze and Gaze360 compared to frontal face methods. Code and pre-trained models will be released to the community.

Title: Do computer vision foundation models learn the low-level characteristics of the human visual system?

• Authors: Yancheng Cai, Fei Yin, Dounia Hammou, Rafal Mantiuk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20256
• Pdf URL: https://arxiv.org/pdf/2502.20256
• Copy Paste: [[2502.20256]] Do computer vision foundation models learn the low-level characteristics of the human visual system?(https://arxiv.org/abs/2502.20256)
• Keywords: generative
• Abstract: Computer vision foundation models, such as DINO or OpenCLIP, are trained in a self-supervised manner on large image datasets. Analogously, substantial evidence suggests that the human visual system (HVS) is influenced by the statistical distribution of colors and patterns in the natural world, characteristics also present in the training data of foundation models. The question we address in this paper is whether foundation models trained on natural images mimic some of the low-level characteristics of the human visual system, such as contrast detection, contrast masking, and contrast constancy. Specifically, we designed a protocol comprising nine test types to evaluate the image encoders of 45 foundation and generative models. Our results indicate that some foundation models (e.g., DINO, DINOv2, and OpenCLIP), share some of the characteristics of human vision, but other models show little resemblance. Foundation models tend to show smaller sensitivity to low contrast and rather irregular responses to contrast across frequencies. The foundation models show the best agreement with human data in terms of contrast masking. Our findings suggest that human vision and computer vision may take both similar and different paths when learning to interpret images of the real world. Overall, while differences remain, foundation models trained on vision tasks start to align with low-level human vision, with DINOv2 showing the closest resemblance.

Title: LLM as a Broken Telephone: Iterative Generation Distorts Information

• Authors: Amr Mohamed, Mingmeng Geng, Michalis Vazirgiannis, Guokan Shang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20258
• Pdf URL: https://arxiv.org/pdf/2502.20258
• Copy Paste: [[2502.20258]] LLM as a Broken Telephone: Iterative Generation Distorts Information(https://arxiv.org/abs/2502.20258)
• Keywords: large language model
• Abstract: As large language models are increasingly responsible for online content, concerns arise about the impact of repeatedly processing their own outputs. Inspired by the "broken telephone" effect in chained human communication, this study investigates whether LLMs similarly distort information through iterative generation. Through translation-based experiments, we find that distortion accumulates over time, influenced by language choice and chain complexity. While degradation is inevitable, it can be mitigated through strategic prompting techniques. These findings contribute to discussions on the long-term effects of AI-mediated information propagation, raising important questions about the reliability of LLM-generated content in iterative workflows.

Title: Understanding the Limits of Deep Tabular Methods with Temporal Shift

• Authors: Hao-Run Cai, Han-Jia Ye
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20260
• Pdf URL: https://arxiv.org/pdf/2502.20260
• Copy Paste: [[2502.20260]] Understanding the Limits of Deep Tabular Methods with Temporal Shift(https://arxiv.org/abs/2502.20260)
• Keywords: robust
• Abstract: Deep tabular models have demonstrated remarkable success on i.i.d. data, excelling in a variety of structured data tasks. However, their performance often deteriorates under temporal distribution shifts, where trends and periodic patterns are present in the evolving data distribution over time. In this paper, we explore the underlying reasons for this failure in capturing temporal dependencies. We begin by investigating the training protocol, revealing a key issue in how model selection perform. While existing approaches use temporal ordering for splitting validation set, we show that even a random split can significantly improve model performance. By minimizing the time lag between training data and test time, while reducing the bias in validation, our proposed training protocol significantly improves generalization across various methods. Furthermore, we analyze how temporal data affects deep tabular representations, uncovering that these models often fail to capture crucial periodic and trend information. To address this gap, we introduce a plug-and-play temporal embedding method based on Fourier series expansion to learn and incorporate temporal patterns, offering an adaptive approach to handle temporal shifts. Our experiments demonstrate that this temporal embedding, combined with the improved training protocol, provides a more effective and robust framework for learning from temporal tabular data.

Title: On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+($λ$,$λ$))-GA

• Authors: Tai Nguyen, Phong Le, André Biendenkapp, Carola Doerr, Nguyen Dang
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2502.20265
• Pdf URL: https://arxiv.org/pdf/2502.20265
• Copy Paste: [[2502.20265]] On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+($λ$,$λ$))-GA(https://arxiv.org/abs/2502.20265)
• Keywords: robust
• Abstract: Dynamic Algorithm Configuration (DAC) has garnered significant attention in recent years, particularly in the prevalence of machine learning and deep learning algorithms. Numerous studies have leveraged the robustness of decision-making in Reinforcement Learning (RL) to address the optimization challenges associated with algorithm configuration. However, making an RL agent work properly is a non-trivial task, especially in reward design, which necessitates a substantial amount of handcrafted knowledge based on domain expertise. In this work, we study the importance of reward design in the context of DAC via a case study on controlling the population size of the $(1+(\lambda,\lambda))$-GA optimizing OneMax. We observed that a poorly designed reward can hinder the RL agent's ability to learn an optimal policy because of a lack of exploration, leading to both scalability and learning divergence issues. To address those challenges, we propose the application of a reward shaping mechanism to facilitate enhanced exploration of the environment by the RL agent. Our work not only demonstrates the ability of RL in dynamically configuring the $(1+(\lambda,\lambda))$-GA, but also confirms the advantages of reward shaping in the scalability of RL agents across various sizes of OneMax problems.

Title: Large Language Models as Attribution Regularizers for Efficient Model Training

• Authors: Davor Vukadin, Marin Šilić, Goran Delač
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20268
• Pdf URL: https://arxiv.org/pdf/2502.20268
• Copy Paste: [[2502.20268]] Large Language Models as Attribution Regularizers for Efficient Model Training(https://arxiv.org/abs/2502.20268)
• Keywords: robust, interpretability, large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across diverse domains. However, effectively leveraging their vast knowledge for training smaller downstream models remains an open challenge, especially in domains like tabular data learning, where simpler models are often preferred due to interpretability and efficiency. In this paper, we introduce a novel yet straightforward method for incorporating LLM-generated global task feature attributions into the training process of smaller networks. Specifically, we propose an attribution-matching regularization term that aligns the training dynamics of the smaller model with the insights provided by the LLM. By doing so, our approach yields superior performance in few-shot learning scenarios. Notably, our method requires only black-box API access to the LLM, making it easy to integrate into existing training pipelines with minimal computational overhead. Furthermore, we demonstrate how this method can be used to address common issues in real-world datasets, such as skewness and bias. By integrating high-level knowledge from LLMs, our approach improves generalization, even when training data is limited or imbalanced. We validate its effectiveness through extensive experiments across multiple tasks, demonstrating improved learning efficiency and model robustness.

Title: Explainable, Multi-modal Wound Infection Classification from Images Augmented with Generated Captions

• Authors: Palawat Busaranuvong, Emmanuel Agu, Reza Saadati Fard, Deepak Kumar, Shefalika Gautam, Bengisu Tulu, Diane Strong
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20277
• Pdf URL: https://arxiv.org/pdf/2502.20277
• Copy Paste: [[2502.20277]] Explainable, Multi-modal Wound Infection Classification from Images Augmented with Generated Captions(https://arxiv.org/abs/2502.20277)
• Keywords: interpretability, diffusion
• Abstract: Infections in Diabetic Foot Ulcers (DFUs) can cause severe complications, including tissue death and limb amputation, highlighting the need for accurate, timely diagnosis. Previous machine learning methods have focused on identifying infections by analyzing wound images alone, without utilizing additional metadata such as medical notes. In this study, we aim to improve infection detection by introducing Synthetic Caption Augmented Retrieval for Wound Infection Detection (SCARWID), a novel deep learning framework that leverages synthetic textual descriptions to augment DFU images. SCARWID consists of two components: (1) Wound-BLIP, a Vision-Language Model (VLM) fine-tuned on GPT-4o-generated descriptions to synthesize consistent captions from images; and (2) an Image-Text Fusion module that uses cross-attention to extract cross-modal embeddings from an image and its corresponding Wound-BLIP caption. Infection status is determined by retrieving the top-k similar items from a labeled support set. To enhance the diversity of training data, we utilized a latent diffusion model to generate additional wound images. As a result, SCARWID outperformed state-of-the-art models, achieving average sensitivity, specificity, and accuracy of 0.85, 0.78, and 0.81, respectively, for wound infection classification. Displaying the generated captions alongside the wound images and infection detection results enhances interpretability and trust, enabling nurses to align SCARWID outputs with their medical knowledge. This is particularly valuable when wound notes are unavailable or when assisting novice nurses who may find it difficult to identify visual attributes of wound infection.

Title: Conformal Tail Risk Control for Large Language Model Alignment

• Authors: Catherine Yu-Chi Chen, Jingyan Shen, Zhun Deng, Lihua Lei
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.20285
• Pdf URL: https://arxiv.org/pdf/2502.20285
• Copy Paste: [[2502.20285]] Conformal Tail Risk Control for Large Language Model Alignment(https://arxiv.org/abs/2502.20285)
• Keywords: large language model
• Abstract: Recent developments in large language models (LLMs) have led to their widespread usage for various tasks. The prevalence of LLMs in society implores the assurance on the reliability of their performance. In particular, risk-sensitive applications demand meticulous attention to unexpectedly poor outcomes, i.e., tail events, for instance, toxic answers, humiliating language, and offensive outputs. Due to the costly nature of acquiring human annotations, general-purpose scoring models have been created to automate the process of quantifying these tail events. This phenomenon introduces potential human-machine misalignment between the respective scoring mechanisms. In this work, we present a lightweight calibration framework for blackbox models that ensures the alignment of humans and machines with provable guarantees. Our framework provides a rigorous approach to controlling any distortion risk measure that is characterized by a weighted average of quantiles of the loss incurred by the LLM with high confidence. The theoretical foundation of our method relies on the connection between conformal risk control and a traditional family of statistics, i.e., L-statistics. To demonstrate the utility of our framework, we conduct comprehensive experiments that address the issue of human-machine misalignment.

Title: Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription

• Authors: Benjamin Gutteridge, Matthew Thomas Jackson, Toni Kukurin, Xiaowen Dong
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20295
• Pdf URL: https://arxiv.org/pdf/2502.20295
• Copy Paste: [[2502.20295]] Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription(https://arxiv.org/abs/2502.20295)
• Keywords: large language model
• Abstract: Handwritten text recognition (HTR) remains a challenging task, particularly for multi-page documents where pages share common formatting and contextual features. While modern optical character recognition (OCR) engines are proficient with printed text, their performance on handwriting is limited, often requiring costly labeled data for fine-tuning. In this paper, we explore the use of multi-modal large language models (MLLMs) for transcribing multi-page handwritten documents in a zero-shot setting. We investigate various configurations of commercial OCR engines and MLLMs, utilizing the latter both as end-to-end transcribers and as post-processors, with and without image components. We propose a novel method, '+first page', which enhances MLLM transcription by providing the OCR output of the entire document along with just the first page image. This approach leverages shared document features without incurring the high cost of processing all images. Experiments on a multi-page version of the IAM Handwriting Database demonstrate that '+first page' improves transcription accuracy, balances cost with performance, and even enhances results on out-of-sample text by extrapolating formatting and OCR error patterns from a single page.

Title: An exploration of features to improve the generalisability of fake news detection models

• Authors: Nathaniel Hoy, Theodora Koulouri
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20299
• Pdf URL: https://arxiv.org/pdf/2502.20299
• Copy Paste: [[2502.20299]] An exploration of features to improve the generalisability of fake news detection models(https://arxiv.org/abs/2502.20299)
• Keywords: robust, large language model
• Abstract: Fake news poses global risks by influencing elections and spreading misinformation, making detection critical. Existing NLP and supervised Machine Learning methods perform well under cross-validation but struggle to generalise across datasets, even within the same domain. This issue stems from coarsely labelled training data, where articles are labelled based on their publisher, introducing biases that token-based models like TF-IDF and BERT are sensitive to. While Large Language Models (LLMs) offer promise, their application in fake news detection remains limited. This study demonstrates that meaningful features can still be extracted from coarsely labelled data to improve real-world robustness. Stylistic features-lexical, syntactic, and semantic-are explored due to their reduced sensitivity to dataset biases. Additionally, novel social-monetisation features are introduced, capturing economic incentives behind fake news, such as advertisements, external links, and social media elements. The study trains on the coarsely labelled NELA 2020-21 dataset and evaluates using the manually labelled Facebook URLs dataset, a gold standard for generalisability. Results highlight the limitations of token-based models trained on biased data and contribute to the scarce evidence on LLMs like LLaMa in this field. Findings indicate that stylistic and social-monetisation features offer more generalisable predictions than token-based methods and LLMs. Statistical and permutation feature importance analyses further reveal their potential to enhance performance and mitigate dataset biases, providing a path forward for improving fake news detection.

Title: M^3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging

• Authors: Jinghao Feng, Qiaoyu Zheng, Chaoyi Wu, Ziheng Zhao, Ya Zhang, Yanfeng Wang, Weidi Xie
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20301
• Pdf URL: https://arxiv.org/pdf/2502.20301
• Copy Paste: [[2502.20301]] M^3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging(https://arxiv.org/abs/2502.20301)
• Keywords: large language model
• Abstract: Agentic AI systems have gained significant attention for their ability to autonomously perform complex tasks. However, their reliance on well-prepared tools limits their applicability in the medical domain, which requires to train specialized models. In this paper, we make three contributions: (i) We present M3Builder, a novel multi-agent system designed to automate machine learning (ML) in medical imaging. At its core, M3Builder employs four specialized agents that collaborate to tackle complex, multi-step medical ML workflows, from automated data processing and environment configuration to self-contained auto debugging and model training. These agents operate within a medical imaging ML workspace, a structured environment designed to provide agents with free-text descriptions of datasets, training codes, and interaction tools, enabling seamless communication and task execution. (ii) To evaluate progress in automated medical imaging ML, we propose M3Bench, a benchmark comprising four general tasks on 14 training datasets, across five anatomies and three imaging modalities, covering both 2D and 3D data. (iii) We experiment with seven state-of-the-art large language models serving as agent cores for our system, such as Claude series, GPT-4o, and DeepSeek-V3. Compared to existing ML agentic designs, M3Builder shows superior performance on completing ML tasks in medical imaging, achieving a 94.29% success rate using Claude-3.7-Sonnet as the agent core, showing huge potential towards fully automated machine learning in medical imaging.

Title: SecureGaze: Defending Gaze Estimation Against Backdoor Attacks

• Authors: Lingyu Du, Yupei Liu, Jinyuan Jia, Guohao Lan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20306
• Pdf URL: https://arxiv.org/pdf/2502.20306
• Copy Paste: [[2502.20306]] SecureGaze: Defending Gaze Estimation Against Backdoor Attacks(https://arxiv.org/abs/2502.20306)
• Keywords: secure, security, protect, defense, attack
• Abstract: Gaze estimation models are widely used in applications such as driver attention monitoring and human-computer interaction. While many methods for gaze estimation exist, they rely heavily on data-hungry deep learning to achieve high performance. This reliance often forces practitioners to harvest training data from unverified public datasets, outsource model training, or rely on pre-trained models. However, such practices expose gaze estimation models to backdoor attacks. In such attacks, adversaries inject backdoor triggers by poisoning the training data, creating a backdoor vulnerability: the model performs normally with benign inputs, but produces manipulated gaze directions when a specific trigger is present. This compromises the security of many gaze-based applications, such as causing the model to fail in tracking the driver's attention. To date, there is no defense that addresses backdoor attacks on gaze estimation models. In response, we introduce SecureGaze, the first solution designed to protect gaze estimation models from such attacks. Unlike classification models, defending gaze estimation poses unique challenges due to its continuous output space and globally activated backdoor behavior. By identifying distinctive characteristics of backdoored gaze estimation models, we develop a novel and effective approach to reverse-engineer the trigger function for reliable backdoor detection. Extensive evaluations in both digital and physical worlds demonstrate that SecureGaze effectively counters a range of backdoor attacks and outperforms seven state-of-the-art defenses adapted from classification models.

Title: Mobius: Text to Seamless Looping Video Generation via Latent Shift

• Authors: Xiuli Bi, Jianfei Yuan, Bo Liu, Yong Zhang, Xiaodong Cun, Chi-Man Pun, Bin Xiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20307
• Pdf URL: https://arxiv.org/pdf/2502.20307
• Copy Paste: [[2502.20307]] Mobius: Text to Seamless Looping Video Generation via Latent Shift(https://arxiv.org/abs/2502.20307)
• Keywords: diffusion
• Abstract: We present Mobius, a novel method to generate seamlessly looping videos from text descriptions directly without any user annotations, thereby creating new visual materials for the multi-media presentation. Our method repurposes the pre-trained video latent diffusion model for generating looping videos from text prompts without any training. During inference, we first construct a latent cycle by connecting the starting and ending noise of the videos. Given that the temporal consistency can be maintained by the context of the video diffusion model, we perform multi-frame latent denoising by gradually shifting the first-frame latent to the end in each step. As a result, the denoising context varies in each step while maintaining consistency throughout the inference process. Moreover, the latent cycle in our method can be of any length. This extends our latent-shifting approach to generate seamless looping videos beyond the scope of the video diffusion model's context. Unlike previous cinemagraphs, the proposed method does not require an image as appearance, which will restrict the motions of the generated results. Instead, our method can produce more dynamic motion and better visual quality. We conduct multiple experiments and comparisons to verify the effectiveness of the proposed method, demonstrating its efficacy in different scenarios. All the code will be made available.

Title: Adapting Automatic Speech Recognition for Accented Air Traffic Control Communications

• Authors: Marcus Yu Zhe Wee, Justin Juin Hng Wong, Lynus Lim, Joe Yu Wei Tan, Prannaya Gupta, Dillion Lim, En Hao Tew, Aloysius Keng Siew Han, Yong Zhi Lim
• Subjects: cs.LG, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2502.20311
• Pdf URL: https://arxiv.org/pdf/2502.20311
• Copy Paste: [[2502.20311]] Adapting Automatic Speech Recognition for Accented Air Traffic Control Communications(https://arxiv.org/abs/2502.20311)
• Keywords: robust
• Abstract: Effective communication in Air Traffic Control (ATC) is critical to maintaining aviation safety, yet the challenges posed by accented English remain largely unaddressed in Automatic Speech Recognition (ASR) systems. Existing models struggle with transcription accuracy for Southeast Asian-accented (SEA-accented) speech, particularly in noisy ATC environments. This study presents the development of ASR models fine-tuned specifically for Southeast Asian accents using a newly created dataset. Our research achieves significant improvements, achieving a Word Error Rate (WER) of 0.0982 or 9.82% on SEA-accented ATC speech. Additionally, the paper highlights the importance of region-specific datasets and accent-focused training, offering a pathway for deploying ASR systems in resource-constrained military operations. The findings emphasize the need for noise-robust training techniques and region-specific datasets to improve transcription accuracy for non-Western accents in ATC communications.

Title: FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction

• Authors: Siyu Jiao, Gengwei Zhang, Yinlong Qian, Jiancheng Huang, Yao Zhao, Humphrey Shi, Lin Ma, Yunchao Wei, Zequn Jie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20313
• Pdf URL: https://arxiv.org/pdf/2502.20313
• Copy Paste: [[2502.20313]] FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction(https://arxiv.org/abs/2502.20313)
• Keywords: diffusion
• Abstract: This work challenges the residual prediction paradigm in visual autoregressive modeling and presents FlexVAR, a new Flexible Visual AutoRegressive image generation paradigm. FlexVAR facilitates autoregressive learning with ground-truth prediction, enabling each step to independently produce plausible images. This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable. Trained solely on low-resolution images ($\leq$ 256px), FlexVAR can: (1) Generate images of various resolutions and aspect ratios, even exceeding the resolution of the training images. (2) Support various image-to-image tasks, including image refinement, in/out-painting, and image expansion. (3) Adapt to various autoregressive steps, allowing for faster inference with fewer steps or enhancing image quality with more steps. Our 1.0B model outperforms its VAR counterpart on the ImageNet 256$\times$256 benchmark. Moreover, when zero-shot transfer the image generation process with 13 steps, the performance further improves to 2.08 FID, outperforming state-of-the-art autoregressive models AiM/VAR by 0.25/0.28 FID and popular diffusion models LDM/DiT by 1.52/0.19 FID, respectively. When transferring our 1.0B model to the ImageNet 512$\times$512 benchmark in a zero-shot manner, FlexVAR achieves competitive results compared to the VAR 2.3B model, which is a fully supervised model trained at 512$\times$512 resolution.

Title: Adversarial Robustness in Parameter-Space Classifiers

• Authors: Tamir Shor, Ethan Fetaya, Chaim Baskin, Alex Bronstein
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20314
• Pdf URL: https://arxiv.org/pdf/2502.20314
• Copy Paste: [[2502.20314]] Adversarial Robustness in Parameter-Space Classifiers(https://arxiv.org/abs/2502.20314)
• Keywords: attack, robust
• Abstract: Implicit Neural Representations (INRs) have been recently garnering increasing interest in various research fields, mainly due to their ability to represent large, complex data in a compact and continuous manner. Past work further showed that numerous popular downstream tasks can be performed directly in the INR parameter-space. Doing so can substantially reduce the computational resources required to process the represented data in their native domain. A major difficulty in using modern machine-learning approaches, is their high susceptibility to adversarial attacks, which have been shown to greatly limit the reliability and applicability of such methods in a wide range of settings. In this work, we show that parameter-space models trained for classification are inherently robust to adversarial attacks -- without the need of any robust training. To support our claims, we develop a novel suite of adversarial attacks targeting parameter-space classifiers, and furthermore analyze practical considerations of attacking parameter-space classifiers. Code for reproducing all experiments and implementation of all proposed methods will be released upon publication.

Title: Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds

• Authors: Mohamed Abdelsamad, Michael Ulrich, Claudius Gläser, Abhinav Valada
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.20316
• Pdf URL: https://arxiv.org/pdf/2502.20316
• Copy Paste: [[2502.20316]] Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds(https://arxiv.org/abs/2502.20316)
• Keywords: generative, segmentation
• Abstract: Masked autoencoders (MAE) have shown tremendous potential for self-supervised learning (SSL) in vision and beyond. However, point clouds from LiDARs used in automated driving are particularly challenging for MAEs since large areas of the 3D volume are empty. Consequently, existing work suffers from leaking occupancy information into the decoder and has significant computational complexity, thereby limiting the SSL pre-training to only 2D bird's eye view encoders in practice. In this work, we propose the novel neighborhood occupancy MAE (NOMAE) that overcomes the aforementioned challenges by employing masked occupancy reconstruction only in the neighborhood of non-masked voxels. We incorporate voxel masking and occupancy reconstruction at multiple scales with our proposed hierarchical mask generation technique to capture features of objects of different sizes in the point cloud. NOMAEs are extremely flexible and can be directly employed for SSL in existing 3D architectures. We perform extensive evaluations on the nuScenes and Waymo Open datasets for the downstream perception tasks of semantic segmentation and 3D object detection, comparing with both discriminative and generative SSL methods. The results demonstrate that NOMAE sets the new state-of-the-art on multiple benchmarks for multiple point cloud perception tasks.

Title: ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model

• Authors: Xuangeng Chu, Nabarun Goswami, Ziteng Cui, Hanqin Wang, Tatsuya Harada
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20323
• Pdf URL: https://arxiv.org/pdf/2502.20323
• Copy Paste: [[2502.20323]] ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model(https://arxiv.org/abs/2502.20323)
• Keywords: diffusion
• Abstract: Speech-driven 3D facial animation aims to generate realistic lip movements and facial expressions for 3D head models from arbitrary audio clips. Although existing diffusion-based methods are capable of producing natural motions, their slow generation speed limits their application potential. In this paper, we introduce a novel autoregressive model that achieves real-time generation of highly synchronized lip movements and realistic head poses and eye blinks by learning a mapping from speech to a multi-scale motion codebook. Furthermore, our model can adapt to unseen speaking styles using sample motion sequences, enabling the creation of 3D talking avatars with unique personal styles beyond the identities seen during training. Extensive evaluations and user studies demonstrate that our method outperforms existing approaches in lip synchronization accuracy and perceived quality.

Title: Long-Context Inference with Retrieval-Augmented Speculative Decoding

• Authors: Guanzheng Chen, Qilong Feng, Jinjie Ni, Xin Li, Michael Qizhe Shieh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20330
• Pdf URL: https://arxiv.org/pdf/2502.20330
• Copy Paste: [[2502.20330]] Long-Context Inference with Retrieval-Augmented Speculative Decoding(https://arxiv.org/abs/2502.20330)
• Keywords: robust, large language model
• Abstract: The emergence of long-context large language models (LLMs) offers a promising alternative to traditional retrieval-augmented generation (RAG) for processing extensive documents. However, the computational overhead of long-context inference, particularly in managing key-value (KV) caches, presents significant efficiency challenges. While Speculative Decoding (SD) traditionally accelerates inference using smaller draft models, its effectiveness diminishes substantially in long-context scenarios due to memory-bound KV cache operations. We present Retrieval-Augmented Speculative Decoding (RAPID), which leverages RAG for both accelerating and enhancing generation quality in long-context inference. RAPID introduces the RAG drafter-a draft LLM operating on shortened retrieval contexts-to speculate on the generation of long-context target LLMs. Our approach enables a new paradigm where same-scale or even larger LLMs can serve as RAG drafters while maintaining computational efficiency. To fully leverage the potentially superior capabilities from stronger RAG drafters, we develop an inference-time knowledge transfer dynamic that enriches the target distribution by RAG. Extensive experiments on the LLaMA-3.1 and Qwen2.5 backbones demonstrate that RAPID effectively integrates the strengths of both approaches, achieving significant performance improvements (e.g., from 39.33 to 42.83 on InfiniteBench for LLaMA-3.1-8B) with more than 2x speedups. Our analyses reveal that RAPID achieves robust acceleration beyond 32K context length and demonstrates superior generation quality in real-world applications.

Title: Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models

• Authors: Yukang Yang, Declan Campbell, Kaixuan Huang, Mengdi Wang, Jonathan Cohen, Taylor Webb
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20332
• Pdf URL: https://arxiv.org/pdf/2502.20332
• Copy Paste: [[2502.20332]] Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models(https://arxiv.org/abs/2502.20332)
• Keywords: robust, large language model
• Abstract: Many recent studies have found evidence for emergent reasoning capabilities in large language models, but debate persists concerning the robustness of these capabilities, and the extent to which they depend on structured reasoning mechanisms. To shed light on these issues, we perform a comprehensive study of the internal mechanisms that support abstract rule induction in an open-source language model (Llama3-70B). We identify an emergent symbolic architecture that implements abstract reasoning via a series of three computations. In early layers, symbol abstraction heads convert input tokens to abstract variables based on the relations between those tokens. In intermediate layers, symbolic induction heads perform sequence induction over these abstract variables. Finally, in later layers, retrieval heads predict the next token by retrieving the value associated with the predicted abstract variable. These results point toward a resolution of the longstanding debate between symbolic and neural network approaches, suggesting that emergent reasoning in neural networks depends on the emergence of symbolic mechanisms.

Title: Expertise Is What We Want

• Authors: Alan Ashworth, Munir Al-Dajani, Keegan Duchicela, Kiril Kafadarov, Allison Kurian, Othman Laraki, Amina Lazrak, Divneet Mandair, Wendy McKennon, Rebecca Miksad, Jayodita Sanghvi, Travis Zack
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20335
• Pdf URL: https://arxiv.org/pdf/2502.20335
• Copy Paste: [[2502.20335]] Expertise Is What We Want(https://arxiv.org/abs/2502.20335)
• Keywords: interpretability, explainability, large language model
• Abstract: Clinical decision-making depends on expert reasoning, which is guided by standardized, evidence-based guidelines. However, translating these guidelines into automated clinical decision support systems risks inaccuracy and importantly, loss of nuance. We share an application architecture, the Large Language Expert (LLE), that combines the flexibility and power of Large Language Models (LLMs) with the interpretability, explainability, and reliability of Expert Systems. LLMs help address key challenges of Expert Systems, such as integrating and codifying knowledge, and data normalization. Conversely, an Expert System-like approach helps overcome challenges with LLMs, including hallucinations, atomic and inexpensive updates, and testability. To highlight the power of the Large Language Expert (LLE) system, we built an LLE to assist with the workup of patients newly diagnosed with cancer. Timely initiation of cancer treatment is critical for optimal patient outcomes. However, increasing complexity in diagnostic recommendations has made it difficult for primary care physicians to ensure their patients have completed the necessary workup before their first visit with an oncologist. As with many real-world clinical tasks, these workups require the analysis of unstructured health records and the application of nuanced clinical decision logic. In this study, we describe the design & evaluation of an LLE system built to rapidly identify and suggest the correct diagnostic workup. The system demonstrated a high degree of clinical-level accuracy (>95%) and effectively addressed gaps identified in real-world data from breast and colon cancer patients at a large academic center.

Title: Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners

• Authors: Daniele Paliotta, Junxiong Wang, Matteo Pagliardini, Kevin Y. Li, Aviv Bick, J. Zico Kolter, Albert Gu, François Fleuret, Tri Dao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20339
• Pdf URL: https://arxiv.org/pdf/2502.20339
• Copy Paste: [[2502.20339]] Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners(https://arxiv.org/abs/2502.20339)
• Keywords: transformer, large language model
• Abstract: Recent advancements have demonstrated that the performance of large language models (LLMs) can be significantly enhanced by scaling computational resources at test time. A common strategy involves generating multiple Chain-of-Thought (CoT) trajectories and aggregating their outputs through various selection mechanisms. This raises a fundamental question: can models with lower complexity leverage their superior generation throughput to outperform similarly sized Transformers for a fixed computational budget? To address this question and overcome the lack of strong subquadratic reasoners, we distill pure and hybrid Mamba models from pretrained Transformers. Trained on only 8 billion tokens, our distilled models show strong performance and scaling on mathematical reasoning datasets while being much faster at inference for large batches and long sequences. Despite the zero-shot performance hit due to distillation, both pure and hybrid Mamba models can scale their coverage and accuracy performance past their Transformer teacher models under fixed time budgets, opening a new direction for scaling inference compute.

Title: Sparse Auto-Encoder Interprets Linguistic Features in Large Language Models

• Authors: Yi Jing, Zijun Yao, Lingxu Ran, Hongzhu Guo, Xiaozhi Wang, Lei Hou, Juanzi Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20344
• Pdf URL: https://arxiv.org/pdf/2502.20344
• Copy Paste: [[2502.20344]] Sparse Auto-Encoder Interprets Linguistic Features in Large Language Models(https://arxiv.org/abs/2502.20344)
• Keywords: large language model
• Abstract: Large language models (LLMs) excel in tasks that require complex linguistic abilities, such as reference disambiguation and metaphor recognition/generation. Although LLMs possess impressive capabilities, their internal mechanisms for processing and representing linguistic knowledge remain largely opaque. Previous work on linguistic mechanisms has been limited by coarse granularity, insufficient causal analysis, and a narrow focus. In this study, we present a systematic and comprehensive causal investigation using sparse auto-encoders (SAEs). We extract a wide range of linguistic features from six dimensions: phonetics, phonology, morphology, syntax, semantics, and pragmatics. We extract, evaluate, and intervene on these features by constructing minimal contrast datasets and counterfactual sentence datasets. We introduce two indices-Feature Representation Confidence (FRC) and Feature Intervention Confidence (FIC)-to measure the ability of linguistic features to capture and control linguistic phenomena. Our results reveal inherent representations of linguistic knowledge in LLMs and demonstrate the potential for controlling model outputs. This work provides strong evidence that LLMs possess genuine linguistic knowledge and lays the foundation for more interpretable and controllable language modeling in future research.

Title: KEDRec-LM: A Knowledge-distilled Explainable Drug Recommendation Large Language Model

• Authors: Kai Zhang, Rui Zhu, Shutian Ma, Jingwei Xiong, Yejin Kim, Fabricio Murai, Xiaozhong Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20350
• Pdf URL: https://arxiv.org/pdf/2502.20350
• Copy Paste: [[2502.20350]] KEDRec-LM: A Knowledge-distilled Explainable Drug Recommendation Large Language Model(https://arxiv.org/abs/2502.20350)
• Keywords: large language model
• Abstract: Drug discovery is a critical task in biomedical natural language processing (NLP), yet explainable drug discovery remains underexplored. Meanwhile, large language models (LLMs) have shown remarkable abilities in natural language understanding and generation. Leveraging LLMs for explainable drug discovery has the potential to improve downstream tasks and real-world applications. In this study, we utilize open-source drug knowledge graphs, clinical trial data, and PubMed publications to construct a comprehensive dataset for the explainable drug discovery task, named \textbf{expRxRec}. Furthermore, we introduce \textbf{KEDRec-LM}, an instruction-tuned LLM which distills knowledge from rich medical knowledge corpus for drug recommendation and rationale generation. To encourage further research in this area, we will publicly release\footnote{A copy is attached with this submission} both the dataset and KEDRec-LM.

Title: Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs

• Authors: Kuan Lok Zhou, Jiayi Chen, Siddharth Suresh, Reuben Narad, Timothy T. Rogers, Lalit K Jain, Robert D Nowak, Bob Mankoff, Jifan Zhang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20356
• Pdf URL: https://arxiv.org/pdf/2502.20356
• Copy Paste: [[2502.20356]] Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs(https://arxiv.org/abs/2502.20356)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have shown significant limitations in understanding creative content, as demonstrated by Hessel et al. (2023)'s influential work on the New Yorker Cartoon Caption Contest (NYCCC). Their study exposed a substantial gap between LLMs and humans in humor comprehension, establishing that understanding and evaluating creative content is key challenge in AI development. We revisit this challenge by decomposing humor understanding into three components and systematically improve each: enhancing visual understanding through improved annotation, utilizing LLM-generated humor reasoning and explanations, and implementing targeted alignment with human preference data. Our refined approach achieves 82.4% accuracy in caption ranking, singificantly improving upon the previous 67% benchmark and matching the performance of world-renowned human experts in this domain. Notably, while attempts to mimic subgroup preferences through various persona prompts showed minimal impact, model finetuning with crowd preferences proved remarkably effective. These findings reveal that LLM limitations in creative judgment can be effectively addressed through focused alignment to specific subgroups and individuals. Lastly, we propose the position that achieving artificial general intelligence necessitates systematic collection of human preference data across creative domains. We advocate that just as human creativity is deeply influenced by individual and cultural preferences, training LLMs with diverse human preference data may be essential for developing true creative understanding.

Title: Evaluating the long-term viability of eye-tracking for continuous authentication in virtual reality

• Authors: Sai Ganesh Grandhi, Saeed Samet
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2502.20359
• Pdf URL: https://arxiv.org/pdf/2502.20359
• Copy Paste: [[2502.20359]] Evaluating the long-term viability of eye-tracking for continuous authentication in virtual reality(https://arxiv.org/abs/2502.20359)
• Keywords: secure, biometric, transformer
• Abstract: Traditional authentication methods, such as passwords and biometrics, verify a user's identity only at the start of a session, leaving systems vulnerable to session hijacking. Continuous authentication, however, ensures ongoing verification by monitoring user behavior. This study investigates the long-term feasibility of eye-tracking as a behavioral biometric for continuous authentication in virtual reality (VR) environments, using data from the GazebaseVR dataset. Our approach evaluates three architectures, Transformer Encoder, DenseNet, and XGBoost, on short and long-term data to determine their efficacy in user identification tasks. Initial results indicate that both Transformer Encoder and DenseNet models achieve high accuracy rates of up to 97% in short-term settings, effectively capturing unique gaze patterns. However, when tested on data collected 26 months later, model accuracy declined significantly, with rates as low as 1.78% for some tasks. To address this, we propose periodic model updates incorporating recent data, restoring accuracy to over 95%. These findings highlight the adaptability required for gaze-based continuous authentication systems and underscore the need for model retraining to manage evolving user behavior. Our study provides insights into the efficacy and limitations of eye-tracking as a biometric for VR authentication, paving the way for adaptive, secure VR user experiences.

Title: OpenTAD: A Unified Framework and Comprehensive Study of Temporal Action Detection

• Authors: Shuming Liu, Chen Zhao, Fatimah Zohra, Mattia Soldan, Alejandro Pardo, Mengmeng Xu, Lama Alssum, Merey Ramazanova, Juan León Alcázar, Anthony Cioppa, Silvio Giancola, Carlos Hinojosa, Bernard Ghanem
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20361
• Pdf URL: https://arxiv.org/pdf/2502.20361
• Copy Paste: [[2502.20361]] OpenTAD: A Unified Framework and Comprehensive Study of Temporal Action Detection(https://arxiv.org/abs/2502.20361)
• Keywords: fair
• Abstract: Temporal action detection (TAD) is a fundamental video understanding task that aims to identify human actions and localize their temporal boundaries in videos. Although this field has achieved remarkable progress in recent years, further progress and real-world applications are impeded by the absence of a standardized framework. Currently, different methods are compared under different implementation settings, evaluation protocols, etc., making it difficult to assess the real effectiveness of a specific technique. To address this issue, we propose \textbf{OpenTAD}, a unified TAD framework consolidating 16 different TAD methods and 9 standard datasets into a modular codebase. In OpenTAD, minimal effort is required to replace one module with a different design, train a feature-based TAD model in end-to-end mode, or switch between the two. OpenTAD also facilitates straightforward benchmarking across various datasets and enables fair and in-depth comparisons among different methods. With OpenTAD, we comprehensively study how innovations in different network components affect detection performance and identify the most effective design choices through extensive experiments. This study has led to a new state-of-the-art TAD method built upon existing techniques for each component. We have made our code and models available at this https URL.

Title: Bridging Legal Knowledge and AI: Retrieval-Augmented Generation with Vector Stores, Knowledge Graphs, and Hierarchical Non-negative Matrix Factorization

• Authors: Ryan C. Barron, Maksim E. Eren, Olga M. Serafimova, Cynthia Matuszek, Boian S. Alexandrov
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.20364
• Pdf URL: https://arxiv.org/pdf/2502.20364
• Copy Paste: [[2502.20364]] Bridging Legal Knowledge and AI: Retrieval-Augmented Generation with Vector Stores, Knowledge Graphs, and Hierarchical Non-negative Matrix Factorization(https://arxiv.org/abs/2502.20364)
• Keywords: security, generative, large language model
• Abstract: Agentic Generative AI, powered by Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG), Knowledge Graphs (KGs), and Vector Stores (VSs), represents a transformative technology applicable to specialized domains such as legal systems, research, recommender systems, cybersecurity, and global security, including proliferation research. This technology excels at inferring relationships within vast unstructured or semi-structured datasets. The legal domain here comprises complex data characterized by extensive, interrelated, and semi-structured knowledge systems with complex relations. It comprises constitutions, statutes, regulations, and case law. Extracting insights and navigating the intricate networks of legal documents and their relations is crucial for effective legal research. Here, we introduce a generative AI system that integrates RAG, VS, and KG, constructed via Non-Negative Matrix Factorization (NMF), to enhance legal information retrieval and AI reasoning and minimize hallucinations. In the legal system, these technologies empower AI agents to identify and analyze complex connections among cases, statutes, and legal precedents, uncovering hidden relationships and predicting legal trends-challenging tasks that are essential for ensuring justice and improving operational efficiency. Our system employs web scraping techniques to systematically collect legal texts, such as statutes, constitutional provisions, and case law, from publicly accessible platforms like Justia. It bridges the gap between traditional keyword-based searches and contextual understanding by leveraging advanced semantic representations, hierarchical relationships, and latent topic discovery. This framework supports legal document clustering, summarization, and cross-referencing, for scalable, interpretable, and accurate retrieval for semi-structured data while advancing computational law and AI.

Title: Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation

• Authors: Zhi Cen, Huaijin Pi, Sida Peng, Qing Shuai, Yujun Shen, Hujun Bao, Xiaowei Zhou, Ruizhen Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20370
• Pdf URL: https://arxiv.org/pdf/2502.20370
• Copy Paste: [[2502.20370]] Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation(https://arxiv.org/abs/2502.20370)
• Keywords: diffusion
• Abstract: This paper addresses the task of generating two-character online interactions. Previously, two main settings existed for two-character interaction generation: (1) generating one's motions based on the counterpart's complete motion sequence, and (2) jointly generating two-character motions based on specific conditions. We argue that these settings fail to model the process of real-life two-character interactions, where humans will react to their counterparts in real time and act as independent individuals. In contrast, we propose an online reaction policy, called Ready-to-React, to generate the next character pose based on past observed motions. Each character has its own reaction policy as its "brain", enabling them to interact like real humans in a streaming manner. Our policy is implemented by incorporating a diffusion head into an auto-regressive model, which can dynamically respond to the counterpart's motions while effectively mitigating the error accumulation throughout the generation process. We conduct comprehensive experiments using the challenging boxing task. Experimental results demonstrate that our method outperforms existing baselines and can generate extended motion sequences. Additionally, we show that our approach can be controlled by sparse signals, making it well-suited for VR and other online interactive environments.

Title: Constrained Generative Modeling with Manually Bridged Diffusion Models

• Authors: Saeid Naderiparizi, Xiaoxuan Liang, Berend Zwartsenberg, Frank Wood
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20371
• Pdf URL: https://arxiv.org/pdf/2502.20371
• Copy Paste: [[2502.20371]] Constrained Generative Modeling with Manually Bridged Diffusion Models(https://arxiv.org/abs/2502.20371)
• Keywords: diffusion, generative
• Abstract: In this paper we describe a novel framework for diffusion-based generative modeling on constrained spaces. In particular, we introduce manual bridges, a framework that expands the kinds of constraints that can be practically used to form so-called diffusion bridges. We develop a mechanism for combining multiple such constraints so that the resulting multiply-constrained model remains a manual bridge that respects all constraints. We also develop a mechanism for training a diffusion model that respects such multiple constraints while also adapting it to match a data distribution. We develop and extend theory demonstrating the mathematical validity of our mechanisms. Additionally, we demonstrate our mechanism in constrained generative modeling tasks, highlighting a particular high-value application in modeling trajectory initializations for path planning and control in autonomous vehicles.

Title: When does a predictor know its own loss?

• Authors: Aravind Gollakota, Parikshit Gopalan, Aayush Karan, Charlotte Peale, Udi Wieder
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20375
• Pdf URL: https://arxiv.org/pdf/2502.20375
• Copy Paste: [[2502.20375]] When does a predictor know its own loss?(https://arxiv.org/abs/2502.20375)
• Keywords: robust, fair
• Abstract: Given a predictor and a loss function, how well can we predict the loss that the predictor will incur on an input? This is the problem of loss prediction, a key computational task associated with uncertainty estimation for a predictor. In a classification setting, a predictor will typically predict a distribution over labels and hence have its own estimate of the loss that it will incur, given by the entropy of the predicted distribution. Should we trust this estimate? In other words, when does the predictor know what it knows and what it does not know? In this work we study the theoretical foundations of loss prediction. Our main contribution is to establish tight connections between nontrivial loss prediction and certain forms of multicalibration, a multigroup fairness notion that asks for calibrated predictions across computationally identifiable subgroups. Formally, we show that a loss predictor that is able to improve on the self-estimate of a predictor yields a witness to a failure of multicalibration, and vice versa. This has the implication that nontrivial loss prediction is in effect no easier or harder than auditing for multicalibration. We support our theoretical results with experiments that show a robust positive correlation between the multicalibration error of a predictor and the efficacy of training a loss predictor.

Title: PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation

• Authors: Albert Gong, Kamilė Stankevičiūtė, Chao Wan, Anmol Kabra, Raphael Thesmar, Johann Lee, Julius Klenke, Carla P. Gomes, Kilian Q. Weinberger
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20377
• Pdf URL: https://arxiv.org/pdf/2502.20377
• Copy Paste: [[2502.20377]] PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation(https://arxiv.org/abs/2502.20377)
• Keywords: large language model
• Abstract: High-quality benchmarks are essential for evaluating reasoning and retrieval capabilities of large language models (LLMs). However, curating datasets for this purpose is not a permanent solution as they are prone to data leakage and inflated performance results. To address these challenges, we propose PhantomWiki: a pipeline to generate unique, factually consistent document corpora with diverse question-answer pairs. Unlike prior work, PhantomWiki is neither a fixed dataset, nor is it based on any existing data. Instead, a new PhantomWiki instance is generated on demand for each evaluation. We vary the question difficulty and corpus size to disentangle reasoning and retrieval capabilities respectively, and find that PhantomWiki datasets are surprisingly challenging for frontier LLMs. Thus, we contribute a scalable and data leakage-resistant framework for disentangled evaluation of reasoning, retrieval, and tool-use abilities. Our code is available at this https URL.

Title: Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling

• Authors: Hanyang Kong, Xingyi Yang, Xinchao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20378
• Pdf URL: https://arxiv.org/pdf/2502.20378
• Copy Paste: [[2502.20378]] Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling(https://arxiv.org/abs/2502.20378)
• Keywords: robust
• Abstract: Rendering dynamic scenes from monocular videos is a crucial yet challenging task. The recent deformable Gaussian Splatting has emerged as a robust solution to represent real-world dynamic scenes. However, it often leads to heavily redundant Gaussians, attempting to fit every training view at various time steps, leading to slower rendering speeds. Additionally, the attributes of Gaussians in static areas are time-invariant, making it unnecessary to model every Gaussian, which can cause jittering in static regions. In practice, the primary bottleneck in rendering speed for dynamic scenes is the number of Gaussians. In response, we introduce Efficient Dynamic Gaussian Splatting (EDGS), which represents dynamic scenes via sparse time-variant attribute modeling. Our approach formulates dynamic scenes using a sparse anchor-grid representation, with the motion flow of dense Gaussians calculated via a classical kernel representation. Furthermore, we propose an unsupervised strategy to efficiently filter out anchors corresponding to static areas. Only anchors associated with deformable objects are input into MLPs to query time-variant attributes. Experiments on two real-world datasets demonstrate that our EDGS significantly improves the rendering speed with superior rendering quality compared to previous state-of-the-art methods.

Title: Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis

• Authors: Jeffrey Yang Fan Chiang, Seungjae Lee, Jia-Bin Huang, Furong Huang, Yizheng Chen
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.20383
• Pdf URL: https://arxiv.org/pdf/2502.20383
• Copy Paste: [[2502.20383]] Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis(https://arxiv.org/abs/2502.20383)
• Keywords: security, defense, robust, large language model
• Abstract: Recent advancements in Web AI agents have demonstrated remarkable capabilities in addressing complex web navigation tasks. However, emerging research shows that these agents exhibit greater vulnerability compared to standalone Large Language Models (LLMs), despite both being built upon the same safety-aligned models. This discrepancy is particularly concerning given the greater flexibility of Web AI Agent compared to standalone LLMs, which may expose them to a wider range of adversarial user inputs. To build a scaffold that addresses these concerns, this study investigates the underlying factors that contribute to the increased vulnerability of Web AI agents. Notably, this disparity stems from the multifaceted differences between Web AI agents and standalone LLMs, as well as the complex signals - nuances that simple evaluation metrics, such as success rate, often fail to capture. To tackle these challenges, we propose a component-level analysis and a more granular, systematic evaluation framework. Through this fine-grained investigation, we identify three critical factors that amplify the vulnerability of Web AI agents; (1) embedding user goals into the system prompt, (2) multi-step action generation, and (3) observational capabilities. Our findings highlights the pressing need to enhance security and robustness in AI agent design and provide actionable insights for targeted defense strategies.

Title: InsTaG: Learning Personalized 3D Talking Head from Few-Second Video

• Authors: Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Jun Zhou, Lin Gu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20387
• Pdf URL: https://arxiv.org/pdf/2502.20387
• Copy Paste: [[2502.20387]] InsTaG: Learning Personalized 3D Talking Head from Few-Second Video(https://arxiv.org/abs/2502.20387)
• Keywords: robust
• Abstract: Despite exhibiting impressive performance in synthesizing lifelike personalized 3D talking heads, prevailing methods based on radiance fields suffer from high demands for training data and time for each new identity. This paper introduces InsTaG, a 3D talking head synthesis framework that allows a fast learning of realistic personalized 3D talking head from few training data. Built upon a lightweight 3DGS person-specific synthesizer with universal motion priors, InsTaG achieves high-quality and fast adaptation while preserving high-level personalization and efficiency. As preparation, we first propose an Identity-Free Pre-training strategy that enables the pre-training of the person-specific model and encourages the collection of universal motion priors from long-video data corpus. To fully exploit the universal motion priors to learn an unseen new identity, we then present a Motion-Aligned Adaptation strategy to adaptively align the target head to the pre-trained field, and constrain a robust dynamic head structure under few training data. Experiments demonstrate our outstanding performance and efficiency under various data scenarios to render high-quality personalized talking heads.

Title: Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation

• Authors: Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan Yuille, Liang-Chieh Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.20388
• Pdf URL: https://arxiv.org/pdf/2502.20388
• Copy Paste: [[2502.20388]] Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation(https://arxiv.org/abs/2502.20388)
• Keywords: generative
• Abstract: Autoregressive (AR) modeling, known for its next-token prediction paradigm, underpins state-of-the-art language and visual generative models. Traditionally, a ``token'' is treated as the smallest prediction unit, often a discrete symbol in language or a quantized patch in vision. However, the optimal token definition for 2D image structures remains an open question. Moreover, AR models suffer from exposure bias, where teacher forcing during training leads to error accumulation at inference. In this paper, we propose xAR, a generalized AR framework that extends the notion of a token to an entity X, which can represent an individual patch token, a cell (a $k\times k$ grouping of neighboring patches), a subsample (a non-local grouping of distant patches), a scale (coarse-to-fine resolution), or even a whole image. Additionally, we reformulate discrete token classification as \textbf{continuous entity regression}, leveraging flow-matching methods at each AR step. This approach conditions training on noisy entities instead of ground truth tokens, leading to Noisy Context Learning, which effectively alleviates exposure bias. As a result, xAR offers two key advantages: (1) it enables flexible prediction units that capture different contextual granularity and spatial structures, and (2) it mitigates exposure bias by avoiding reliance on teacher forcing. On ImageNet-256 generation benchmark, our base model, xAR-B (172M), outperforms DiT-XL/SiT-XL (675M) while achieving 20$\times$ faster inference. Meanwhile, xAR-H sets a new state-of-the-art with an FID of 1.24, running 2.2$\times$ faster than the previous best-performing model without relying on vision foundation modules (\eg, DINOv2) or advanced guidance interval sampling.

Title: InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions

• Authors: Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, Liang-Yan Gui
• Subjects: cs.CV, cs.GR, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.20390
• Pdf URL: https://arxiv.org/pdf/2502.20390
• Copy Paste: [[2502.20390]] InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions(https://arxiv.org/abs/2502.20390)
• Keywords: robust, generative
• Abstract: Achieving realistic simulations of humans interacting with a wide range of objects has long been a fundamental goal. Extending physics-based motion imitation to complex human-object interactions (HOIs) is challenging due to intricate human-object coupling, variability in object geometries, and artifacts in motion capture data, such as inaccurate contacts and limited hand detail. We introduce InterMimic, a framework that enables a single policy to robustly learn from hours of imperfect MoCap data covering diverse full-body interactions with dynamic and varied objects. Our key insight is to employ a curriculum strategy -- perfect first, then scale up. We first train subject-specific teacher policies to mimic, retarget, and refine motion capture data. Next, we distill these teachers into a student policy, with the teachers acting as online experts providing direct supervision, as well as high-quality references. Notably, we incorporate RL fine-tuning on the student policy to surpass mere demonstration replication and achieve higher-quality solutions. Our experiments demonstrate that InterMimic produces realistic and diverse interactions across multiple HOI datasets. The learned policy generalizes in a zero-shot manner and seamlessly integrates with kinematic generators, elevating the framework from mere imitation to generative modeling of complex human-object interactions.

Title: R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts

• Authors: Zhongyang Li, Ziyue Li, Tianyi Zhou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.20395
• Pdf URL: https://arxiv.org/pdf/2502.20395
• Copy Paste: [[2502.20395]] R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts(https://arxiv.org/abs/2502.20395)
• Keywords: large language model
• Abstract: In large multimodal models (LMMs), the perception of non-language modalities (e.g., visual representations) is usually not on par with the large language models (LLMs)' powerful reasoning capabilities, deterring LMMs' performance on challenging downstream tasks. This weakness has been recently mitigated by replacing the vision encoder with a mixture-of-experts (MoE), which provides rich, multi-granularity, and diverse representations required by diverse downstream tasks. The performance of multimodal MoE largely depends on its router, which reweights and mixes the representations of different experts for each input. However, we find that the end-to-end trained router does not always produce the optimal routing weights for every test sample. To bridge the gap, we propose a novel and efficient method "Re-Routing in Test-Time(R2-T2) that locally optimizes the vector of routing weights in test-time by moving it toward those vectors of the correctly predicted samples in a neighborhood of the test sample. We propose three R2-T2 strategies with different optimization objectives and neighbor-search spaces. R2-T2 consistently and greatly improves state-of-the-art LMMs' performance on challenging benchmarks of diverse tasks, without training any base-model parameters.