2025-03-04

Title: Eeyore: Realistic Depression Simulation via Supervised and Preference Optimization

• Authors: Siyang Liu, Bianca Brie, Wenda Li, Laura Biester, Andrew Lee, James Pennebaker, Rada Mihalcea
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00018
• Pdf URL: https://arxiv.org/pdf/2503.00018
• Copy Paste: [[2503.00018]] Eeyore: Realistic Depression Simulation via Supervised and Preference Optimization(https://arxiv.org/abs/2503.00018)
• Keywords: extraction, large language model
• Abstract: Large Language Models (LLMs) have been previously explored for mental healthcare training and therapy client simulation, but they still fall short in authentically capturing diverse client traits and psychological conditions. We introduce \textbf{Eeyore}, an 8B model optimized for realistic depression simulation through a structured alignment framework, incorporating expert input at every stage. First, we systematically curate real-world depression-related conversations, extracting depressive traits to guide data filtering and psychological profile construction, and use this dataset to instruction-tune Eeyore for profile adherence. Next, to further enhance realism, Eeyore undergoes iterative preference optimization -- first leveraging model-generated preferences and then calibrating with a small set of expert-annotated preferences. Throughout the entire pipeline, we actively collaborate with domain experts, developing interactive interfaces to validate trait extraction and iteratively refine structured psychological profiles for clinically meaningful role-play customization. Despite its smaller model size, the Eeyore depression simulation outperforms GPT-4o with SOTA prompting strategies, both in linguistic authenticity and profile adherence.

Title: A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety

• Authors: Rakeen Rouf, Trupti Bavalatti, Osama Ahmed, Dhaval Potdar, Faraz Jawed
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00020
• Pdf URL: https://arxiv.org/pdf/2503.00020
• Copy Paste: [[2503.00020]] A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety(https://arxiv.org/abs/2503.00020)
• Keywords: generative
• Abstract: Novel research aimed at text-to-image (T2I) generative AI safety often relies on publicly available datasets for training and evaluation, making the quality and composition of these datasets crucial. This paper presents a comprehensive review of the key datasets used in the T2I research, detailing their collection methods, compositions, semantic and syntactic diversity of prompts and the quality, coverage, and distribution of harm types in the datasets. By highlighting the strengths and limitations of the datasets, this study enables researchers to find the most relevant datasets for a use case, critically assess the downstream impacts of their work given the dataset distribution, particularly regarding model safety and ethical considerations, and also identify the gaps in dataset coverage and quality that future research may address.

Title: KVCrush: Key value cache size-reduction using similarity in head-behaviour

• Authors: Gopi Krishna Jha, Sameh Gobriel, Liubov Talamanova, Alexander Kozlov, Nilesh Jain
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00022
• Pdf URL: https://arxiv.org/pdf/2503.00022
• Copy Paste: [[2503.00022]] KVCrush: Key value cache size-reduction using similarity in head-behaviour(https://arxiv.org/abs/2503.00022)
• Keywords: large language model
• Abstract: Key-value (KV) caching has emerged as a crucial optimization technique for accelerating inference in large language models (LLMs). By allowing the attention operation to scale linearly rather than quadratically with the total sequence length, KV caching significantly enhances generation throughput. However, due to large context lengths in the modern LLMs, the memory footprint of the KV is a huge bottleneck for model deployment directly impacting the model's batch size, hindering its ability to deliver high-throughput. Existing research addresses this challenge using several techniques, such as discarding low-attention tokens, quantization, and matrix approximation which typically lead to a negative impact on the model accuracy. In this paper, We propose KVCrush technology which can be combined with many KV compression technologies to improve the model accuracy at a much smaller memory. KVCrush provides an alternate representation scheme for key-value states, along with a low-overhead token pruning algorithm that accounts for the token distribution in the KV cache, which in turn allows for a a smaller footprint while maintaining the accuracy of the model. Based on our results, KVCrush reduces LongBench KV Cache size by 4x with less than 1% accuracy drop and achieves state-of-the-art average accuracy with minimal overhead, incurring less than 0.5% total inference latency. KVCrush not only outperforms the accuracy of state-of-the-art importance-based token retention schemes but is also compatible with typical practical LLM deployments using KV cache paging schemes such as vLLM and mixed precision quantization.

Title: Evaluating Large Language Models on the Spanish Medical Intern Resident (MIR) Examination 2024/2025:A Comparative Analysis of Clinical Reasoning and Knowledge Application

• Authors: Carlos Luengo Vera, Ignacio Ferro Picon, M. Teresa del Val Nunez, Jose Andres Gomez Gandia, Antonio de Lucas Ancillo, Victor Ramos Arroyo, Carlos Milan Figueredo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00025
• Pdf URL: https://arxiv.org/pdf/2503.00025
• Copy Paste: [[2503.00025]] Evaluating Large Language Models on the Spanish Medical Intern Resident (MIR) Examination 2024/2025:A Comparative Analysis of Clinical Reasoning and Knowledge Application(https://arxiv.org/abs/2503.00025)
• Keywords: robust, large language model
• Abstract: This study presents a comparative evaluation of 22 large language models LLMs on the Spanish Medical Intern Resident MIR examinations for 2024 and 2025 with a focus on clinical reasoning domain specific expertise and multimodal processing capabilities The MIR exam consisting of 210 multiple choice questions some requiring image interpretation serves as a stringent benchmark for assessing both factual recall and complex clinical problem solving skills Our investigation encompasses general purpose models such as GPT4 Claude LLaMA and Gemini as well as specialized fine tuned systems like Miri Pro which leverages proprietary Spanish healthcare data to excel in medical contexts Recent market entries Deepseek and Grok have further enriched the evaluation landscape particularly for tasks that demand advanced visual and semantic analysis The findings indicate that while general purpose LLMs perform robustly overall fine tuned models consistently achieve superior accuracy especially in addressing nuanced domain specific challenges A modest performance decline observed between the two exam cycles appears attributable to the implementation of modified questions designed to mitigate reliance on memorization The results underscore the transformative potential of domain specific fine tuning and multimodal integration in advancing medical AI applications They also highlight critical implications for the future integration of LLMs into medical education training and clinical decision making emphasizing the importance of balancing automated reasoning with ethical and context aware judgment

Title: Streaming Looking Ahead with Token-level Self-reward

• Authors: Hongming Zhang, Ruixin Hong, Dong Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00029
• Pdf URL: https://arxiv.org/pdf/2503.00029
• Copy Paste: [[2503.00029]] Streaming Looking Ahead with Token-level Self-reward(https://arxiv.org/abs/2503.00029)
• Keywords: transformer
• Abstract: Autoregressive decoding algorithms that use only past information often cannot guarantee the best performance. Recently, people discovered that looking-ahead algorithms such as Monte Carlo Tree Search (MCTS) with external reward models (RMs) can significantly improve models' output by allowing them to think ahead and leverage future outputs and associated rewards to guide the current generation. Such techniques can help the reinforcement fine-tuning phase by sampling better trajectories and the inference phase by selecting the better output. However, their high computational cost limits their applications, especially in streaming scenarios. To address this issue, we propose equipping the policy model with token-level self-reward modeling (TRM) capability to eliminate the need for external models and extra communication. We name the new architecture as Reward Transformer. In addition, we propose a streaming-looking-ahead (SLA) algorithm to further boost search efficiency with better parallelization. Experiments show that SLA achieves an overall win rate of 79.7\% against the baseline greedy decoding algorithm on three general-domain datasets with a frozen policy model while maintaining streaming efficiency. If we combine SLA with reinforcement fine-tuning techniques such as DPO, SLA achieves an overall win rate of 89.4\%.

Title: Game-Theoretic Regularized Self-Play Alignment of Large Language Models

• Authors: Xiaohang Tang, Sangwoong Yoon, Seongho Son, Huizhuo Yuan, Quanquan Gu, Ilija Bogunovic
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00030
• Pdf URL: https://arxiv.org/pdf/2503.00030
• Copy Paste: [[2503.00030]] Game-Theoretic Regularized Self-Play Alignment of Large Language Models(https://arxiv.org/abs/2503.00030)
• Keywords: large language model
• Abstract: Self-play alignment algorithms have been developed as effective methods for fine-tuning large language models (LLMs), formulating preference optimization as a two-player game. However, the regularization with respect to the reference policy, which is crucial for mitigating over-optimization, has been insufficiently investigated in self-play alignment. In this paper, we show that our regularization method can improve the unregularized self-play significantly. To study the impact of different regularizations in self-play alignment, we propose Regularized Self-Play Policy Optimization (RSPO). This generalized framework regularizes the self-play by simply adding a chosen regularization term into the loss while maintaining provable last-iterate convergence to the Nash Equilibrium of the corresponding regularized game. Surprisingly, empirical evaluations using the Mistral-7B-Instruct base model reveal that forward KL divergence regularization reduces response length in RSPO, whereas reverse KL divergence markedly improves raw win rates. RSPO with a linear combination of forward and reverse KL divergence regularization substantially increases the length-controlled win rate in AlpacaEval-2, elevating the unregularized self-play alignment method (SPPO) from $28.53\%$ to $35.44\%$. Finally, we show that RSPO also improves the response diversity.

Title: Efficient Test-Time Scaling via Self-Calibration

• Authors: Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, Jiaxin Huang
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00031
• Pdf URL: https://arxiv.org/pdf/2503.00031
• Copy Paste: [[2503.00031]] Efficient Test-Time Scaling via Self-Calibration(https://arxiv.org/abs/2503.00031)
• Keywords: large language model
• Abstract: Increasing test-time computation is a straightforward approach to enhancing the quality of responses in Large Language Models (LLMs). While Best-of-N sampling and Self-Consistency with majority voting are simple and effective, they require a fixed number of sampling responses for each query, regardless of its complexity. This could result in wasted computation for simpler questions and insufficient exploration for more challenging ones. In this work, we argue that model confidence of responses can be used for improving the efficiency of test-time scaling. Unfortunately, LLMs are known to be overconfident and provide unreliable confidence estimation. To address this limitation, we introduce Self-Calibration by distilling Self-Consistency-derived confidence into the model itself. This enables reliable confidence estimation at test time with one forward pass. We then design confidence-based efficient test-time scaling methods to handle queries of various difficulty, such as Early-Stopping for Best-of-N and Self-Consistency with calibrated confidence. Experiments on three LLMs across six datasets demonstrate the effectiveness of our approach. Specifically, applying confidence-based Early Stopping to Best-of-N improves MathQA accuracy from 81.0 to 83.6 with a sample budget of 16 responses, indicating the efficacy of confidence-based sampling strategy at inference time.

Title: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis

• Authors: Shinwoo Park, Shubin Kim, Do-Kyung Kim, Yo-Sub Han
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00032
• Pdf URL: https://arxiv.org/pdf/2503.00032
• Copy Paste: [[2503.00032]] Detecting LLM-Generated Korean Text through Linguistic Feature Analysis(https://arxiv.org/abs/2503.00032)
• Keywords: protect, large language model
• Abstract: The rapid advancement of large language models (LLMs) increases the difficulty of distinguishing between human-written and LLM-generated text. Detecting LLM-generated text is crucial for upholding academic integrity, preventing plagiarism, protecting copyrights, and ensuring ethical research practices. Most prior studies on detecting LLM-generated text focus primarily on English text. However, languages with distinct morphological and syntactic characteristics require specialized detection approaches. Their unique structures and usage patterns can hinder the direct application of methods primarily designed for English. Among such languages, we focus on Korean, which has relatively flexible spacing rules, a rich morphological system, and less frequent comma usage compared to English. We introduce KatFish, the first benchmark dataset for detecting LLM-generated Korean text. The dataset consists of text written by humans and generated by four LLMs across three genres. By examining spacing patterns, part-of-speech diversity, and comma usage, we illuminate the linguistic differences between human-written and LLM-generated Korean text. Building on these observations, we propose KatFishNet, a detection method specifically designed for the Korean language. KatFishNet achieves an average of 19.78% higher AUROC compared to the best-performing existing detection method.

Title: MergeIT: From Selection to Merging for Efficient Instruction Tuning

• Authors: Hongyi Cai, Yuqian Fu, Hongming Fu, Bo Zhao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00034
• Pdf URL: https://arxiv.org/pdf/2503.00034
• Copy Paste: [[2503.00034]] MergeIT: From Selection to Merging for Efficient Instruction Tuning(https://arxiv.org/abs/2503.00034)
• Keywords: large language model
• Abstract: Instruction tuning is crucial for optimizing Large Language Models (LLMs), yet mainstream data selection methods heavily rely on LLMs as instruction quality scorers, leading to high computational costs and reduced data diversity. To address these limitations, we propose MergeIT, a novel LLM-based Merging strategy for better Instruction Tuning that shifts the focus from selection to synthesis. MergeIT operates in two stages: first, topic-aware filtering clusters and refines the dataset, preserving diversity while eliminating redundancy without relying on LLM-based scoring. Second, LLM-based merging synthesizes semantically similar instructions into more informative and compact training data, enhancing data richness while further reducing dataset size. Experimental results demonstrate that MergeIT enables efficient, diverse, and scalable instruction selection and synthesis, establishing LLM-based merging as a promising alternative to conventional scoring-based selection methods for instruction tuning. Our source code and datasets are now available at this https URL

Title: Constraining Sequential Model Editing with Editing Anchor Compression

• Authors: Hao-Xiang Xu, Jun-Yu Ma, Zhen-Hua Ling, Ningyu Zhang, Jia-Chen Gu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00035
• Pdf URL: https://arxiv.org/pdf/2503.00035
• Copy Paste: [[2503.00035]] Constraining Sequential Model Editing with Editing Anchor Compression(https://arxiv.org/abs/2503.00035)
• Keywords: large language model
• Abstract: Large language models (LLMs) struggle with hallucinations due to false or outdated knowledge. Given the high resource demands of retraining these models, there is an increasing focus on developing model editing. However, the general abilities of LLMs across downstream tasks are prone to significant degradation during sequential editing. This paper statistically observes that the parameter matrix after editing exhibits a significant deviation compared to its previous state as the number of edits increases. This serious deviation affects the original knowledge associations within LLMs and leads to the degradation of their general abilities. To this end, a framework termed Editing Anchor Compression (EAC) is proposed to constrain the deviation of the parameter matrix during sequential editing. It compresses the editing information by selecting editing anchors that are important in encoding new relations without deviating too much from the original matrix, thereby preserving the general abilities. Experiments of applying EAC to two popular editing methods on three LLMs across four tasks are conducted. Evaluation results show that EAC effectively minimizes unreasonable deviations caused by model editing, preserving over 70% of the general abilities while better retaining the editing knowledge compared to the original counterpart methods.

Title: Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs

• Authors: Wei Zhao, Zhe Li, Yige Li, Jun Sun
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00037
• Pdf URL: https://arxiv.org/pdf/2503.00037
• Copy Paste: [[2503.00037]] Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs(https://arxiv.org/abs/2503.00037)
• Keywords: defense, robust
• Abstract: Large Vision-Language Models (LVLMs) have made significant strides in multimodal comprehension, thanks to extensive pre-training and fine-tuning on large-scale visual datasets. However, despite their robust textual safety mechanisms, they remain vulnerable to harmful visual inputs. Existing safeguards-typically relying on pre-filtering or fine-tuning-incur high costs and diminish overall utility. To address this critical vulnerability, we introduce SafeCLIP, a lightweight method that leverages LVLMs inherent multimodal alignment for zero-shot toxic image detection. By projecting CLIPs discarded CLS token into its text space and matching it with toxic descriptors, SafeCLIP detects harmful content without any architectural changes-adding minimal latency and enabling dynamic safety corrections during inference and this http URL show that SafeCLIP achieves a 66.9% defense success rate with only 3.2% false positive rate and 7.2% overhead. In contrast, state-of-the-art methods achieve 52.9% success but have a 10.7% false positive rate and 210% overhead. Our work demonstrates that leveraging inherent multimodal alignment can yield efficient, low-cost LVLM safety. Code is available at this http URL.

Title: from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors

• Authors: Yu Yan, Sheng Sun, Zenghao Duan, Teli Liu, Min Liu, Zhiyi Yin, Qi Li, Jiangyu Lei
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00038
• Pdf URL: https://arxiv.org/pdf/2503.00038
• Copy Paste: [[2503.00038]] from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors(https://arxiv.org/abs/2503.00038)
• Keywords: attack, large language model
• Abstract: Current studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks. However, they overlook that the direct generation of harmful content from scratch is more difficult than inducing LLM to calibrate benign content into harmful forms. In our study, we introduce a novel attack framework that exploits AdVersArial meTAphoR (AVATAR) to induce the LLM to calibrate malicious metaphors for jailbreaking. Specifically, to answer harmful queries, AVATAR adaptively identifies a set of benign but logically related metaphors as the initial seed. Then, driven by these metaphors, the target LLM is induced to reason and calibrate about the metaphorical content, thus jailbroken by either directly outputting harmful responses or calibrating residuals between metaphorical and professional harmful content. Experimental results demonstrate that AVATAR can effectively and transferable jailbreak LLMs and achieve a state-of-the-art attack success rate across multiple advanced LLMs.

Title: An Analysis of Segment Anything 2

• Authors: Clayton Bromley, Alexander Moore, Amar Saini, Doug Poland, Carmen Carrano
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00042
• Pdf URL: https://arxiv.org/pdf/2503.00042
• Copy Paste: [[2503.00042]] An Analysis of Segment Anything 2(https://arxiv.org/abs/2503.00042)
• Keywords: segmentation
• Abstract: Video object segmentation (VOS) is a critical task in the development of video perception and understanding. The Segment-Anything Model 2 (SAM 2), released by Meta AI, is the current state-of-the-art architecture for end-to-end VOS. SAM 2 performs very well on both clean video data and augmented data, and completely intelligent video perception requires an understanding of how this architecture is capable of achieving such quality results. To better understand how each step within the SAM 2 architecture permits high-quality video segmentation, we pass a variety of complex video transformations through the architecture and measure the impact at each stage of the process. We observe that each progressive stage enables the filtering of complex transformation noise and the emphasis of the object of interest. Our contributions include the creation of complex transformation video datasets, an analysis of how each stage of the SAM 2 architecture interprets these transformations, and visualizations of segmented objects through each stage. By better understanding how each model structure impacts overall video understanding, VOS development can work to improve real-world applicability and performance tracking, localizing, and segmenting objects despite complex cluttered scenes and obscurations.

Title: VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning

• Authors: Nilay Yilmaz, Maitreya Patel, Yiran Lawrence Luo, Tejas Gokhale, Chitta Baral, Suren Jayasuriya, Yezhou Yang
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00043
• Pdf URL: https://arxiv.org/pdf/2503.00043
• Copy Paste: [[2503.00043]] VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning(https://arxiv.org/abs/2503.00043)
• Keywords: large language model
• Abstract: Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information. Despite their exceptional performance on visual understanding benchmarks, measuring their ability to reason abstractly across multiple images remains a significant challenge. To address this, we introduce VOILA, a large-scale, open-ended, dynamic benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning. VOILA employs an analogical mapping approach in the visual domain, requiring models to generate an image that completes an analogy between two given image pairs, reference and application, without relying on predefined choices. Our experiments demonstrate that the analogical reasoning tasks in VOILA present a challenge to MLLMs. Through multi-step analysis, we reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning. Notably, we observe that performance improves when following a multi-step strategy of least-to-most prompting. Comprehensive evaluations on open-source models and GPT-4o show that on text-based answers, the best accuracy for challenging scenarios is 13% (LLaMa 3.2) and even for simpler tasks is only 29% (GPT-4o), while human performance is significantly higher at 70% across both difficulty levels.

Title: Correspondence-Free Pose Estimation with Patterns: A Unified Approach for Multi-Dimensional Vision

• Authors: Quan Quan, Dun Dai
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.00051
• Pdf URL: https://arxiv.org/pdf/2503.00051
• Copy Paste: [[2503.00051]] Correspondence-Free Pose Estimation with Patterns: A Unified Approach for Multi-Dimensional Vision(https://arxiv.org/abs/2503.00051)
• Keywords: robust
• Abstract: 6D pose estimation is a central problem in robot vision. Compared with pose estimation based on point correspondences or its robust versions, correspondence-free methods are often more flexible. However, existing correspondence-free methods often rely on feature representation alignment or end-to-end regression. For such a purpose, a new correspondence-free pose estimation method and its practical algorithms are proposed, whose key idea is the elimination of unknowns by process of addition to separate the pose estimation from correspondence. By taking the considered point sets as patterns, feature functions used to describe these patterns are introduced to establish a sufficient number of equations for optimization. The proposed method is applicable to nonlinear transformations such as perspective projection and can cover various pose estimations from 3D-to-3D points, 3D-to-2D points, and 2D-to-2D points. Experimental results on both simulation and actual data are presented to demonstrate the effectiveness of the proposed method.

Title: RURA-Net: A general disease diagnosis method based on Zero-Shot Learning

• Authors: Yan Su, Qiulin Wu, Weizhen Li, Chengchang Pan, Honggang Qi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00052
• Pdf URL: https://arxiv.org/pdf/2503.00052
• Copy Paste: [[2503.00052]] RURA-Net: A general disease diagnosis method based on Zero-Shot Learning(https://arxiv.org/abs/2503.00052)
• Keywords: segmentation
• Abstract: The training of deep learning models relies on a large amount of labeled data. However, the high cost of medical labeling seriously hinders the development of deep learning in the medical field. Our study proposes a general disease diagnosis approach based on Zero-Shot Learning. The Siamese neural network is used to find similar diseases for the target diseases, and the U-Net segmentation model is used to accurately segment the key lesions of the disease. Finally, based on the ResNet-Agglomerative clustering algorithm, a clustering model is trained on a large number of sample data of similar diseases to obtain a approximate diagnosis of the target disease. Zero-Shot Learning of the target disease is then successfully achieved. To evaluate the validity of the model, we validated our method on a dataset of ophthalmic diseases in CFP modality. The external dataset was used to test its performance, and the accuracy=0.8395, precision=0.8094, recall=0.8463, F1 Score=0.8274, AUC=0.9226, which exceeded the indexes of most Few-Shot Learning and One-Shot Learning models. It proves that our method has great potential and reference value in the medical field, where annotation data is usually scarce and expensive to obtain.

Title: Improved YOLOv12 with LLM-Generated Synthetic Data for Enhanced Apple Detection and Benchmarking Against YOLOv11 and YOLOv10

• Authors: Ranjan Sapkota, Manoj Karkee
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00057
• Pdf URL: https://arxiv.org/pdf/2503.00057
• Copy Paste: [[2503.00057]] Improved YOLOv12 with LLM-Generated Synthetic Data for Enhanced Apple Detection and Benchmarking Against YOLOv11 and YOLOv10(https://arxiv.org/abs/2503.00057)
• Keywords: large language model
• Abstract: This study evaluated the performance of the YOLOv12 object detection model, and compared against YOLOv11 and YOLOv10 for apple detection in commercial orchards using synthetic images generated by Large Language Models (LLMs). The YOLOv12n configuration excelled, achieving the highest precision at 0.916, the highest recall at 0.969, and the highest mean Average Precision (mAP@50) at 0.978. In comparison, the YOLOv11 series was led by YOLO11x, which recorded the highest precision at 0.857, recall at 0.85, and mAP@50 at 0.91. For the YOLOv10 series, YOLOv10b and YOLOv10l tied for the highest precision at 0.85, with YOLOv10n achieving the highest recall at 0.8 and mAP@50 at 0.89. The study also highlighted efficiency in processing speeds, where YOLOv11n reported the lowest inference time at 4.7 ms, compared to YOLOv12n's 5.6 ms and YOLOv10n's 5.9 ms. Although YOLOv12 is new in more accurate than YOLOv11, and YOLOv10, the YOLO11n still stays the fastest YOLO algorithm among YOLOv10, YOLOv11 and YOLOv12. These findings demonstrated that YOLOv12, when trained on high-quality LLM-generated datasets, not only surpassed its predecessors in key performance metrics but also offered a cost-effective solution by reducing the need for extensive manual data collection in the field.

Title: African Gender Classification Using Clothing Identification Via Deep Learning

• Authors: Samuel Ozechi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00058
• Pdf URL: https://arxiv.org/pdf/2503.00058
• Copy Paste: [[2503.00058]] African Gender Classification Using Clothing Identification Via Deep Learning(https://arxiv.org/abs/2503.00058)
• Keywords: robust
• Abstract: Human attribute identification and classification are crucial in computer vision, driving the development of innovative recognition systems. Traditional gender classification methods primarily rely on facial recognition, which, while effective, struggles under non-ideal conditions such as blurriness, side views, or partial occlusions. This study explores an alternative approach by leveraging clothing identification, specifically focusing on African traditional attire, which carries culturally significant and gender-specific features. We use the AFRIFASHION1600 dataset, a curated collection of 1,600 images of African traditional clothing labeled into two gender classes: male and female. A deep learning model, based on a modified VGG16 architecture and trained using transfer learning, was developed for classification. Data augmentation was applied to address the challenges posed by the relatively small dataset and to mitigate overfitting. The model achieved an accuracy of 87% on the test set, demonstrating strong predictive capability despite dataset imbalances favoring female samples. These findings highlight the potential of clothing-based identification as a complementary technique to facial recognition for gender classification in African contexts. Future research should focus on expanding and balancing datasets to enhance classification robustness and improve the applicability of clothing-based gender recognition systems.

Title: Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models

• Authors: Rui Hu, Delai Qiu, Shuyu Wei, Jiaming Zhang, Yining Wang, Shengping Liu, Jitao Sang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00059
• Pdf URL: https://arxiv.org/pdf/2503.00059
• Copy Paste: [[2503.00059]] Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models(https://arxiv.org/abs/2503.00059)
• Keywords: large language model
• Abstract: Omnimodal Large Language Models (OLLMs) have shown significant progress in integrating vision and text, but still struggle with integrating vision and audio, often exhibiting suboptimal performance when processing audio queries compared to text queries. This disparity is primarily due to insufficient alignment between vision and audio modalities during training, leading to inadequate attention to visual information when using audio queries. To mitigate this issue, we propose a Self-Knowledge Distillation (Self-KD) training method where the vision-text component of the OLLM serves as the teacher and the vision-audio component as the student. This enables the model to process audio in a manner analogous to its text processing. Our experimental results demonstrate that Self-KD is an effective method for enhancing the vision-audio capabilities of OLLMs by learning from the vision-text components, which subsequently improves the interaction between audio and images and results in improved performance on multimodal tasks.

Title: SAC-ViT: Semantic-Aware Clustering Vision Transformer with Early Exit

• Authors: Youbing Hu, Yun Cheng, Anqi Lu, Dawei Wei, Zhijun Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00060
• Pdf URL: https://arxiv.org/pdf/2503.00060
• Copy Paste: [[2503.00060]] SAC-ViT: Semantic-Aware Clustering Vision Transformer with Early Exit(https://arxiv.org/abs/2503.00060)
• Keywords: transformer
• Abstract: The Vision Transformer (ViT) excels in global modeling but faces deployment challenges on resource-constrained devices due to the quadratic computational complexity of its attention mechanism. To address this, we propose the Semantic-Aware Clustering Vision Transformer (SAC-ViT), a non-iterative approach to enhance ViT's computational efficiency. SAC-ViT operates in two stages: Early Exit (EE) and Semantic-Aware Clustering (SAC). In the EE stage, downsampled input images are processed to extract global semantic information and generate initial inference results. If these results do not meet the EE termination criteria, the information is clustered into target and non-target tokens. In the SAC stage, target tokens are mapped back to the original image, cropped, and embedded. These target tokens are then combined with reused non-target tokens from the EE stage, and the attention mechanism is applied within each cluster. This two-stage design, with end-to-end optimization, reduces spatial redundancy and enhances computational efficiency, significantly boosting overall ViT performance. Extensive experiments demonstrate the efficacy of SAC-ViT, reducing 62% of the FLOPs of DeiT and achieving 1.98 times throughput without compromising performance.

Title: Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents

• Authors: Qiusi Zhan, Richard Fang, Henil Shalin Panchal, Daniel Kang
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00061
• Pdf URL: https://arxiv.org/pdf/2503.00061
• Copy Paste: [[2503.00061]] Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents(https://arxiv.org/abs/2503.00061)
• Keywords: security, defense, attack, robust, large language model
• Abstract: Large Language Model (LLM) agents exhibit remarkable performance across diverse applications by using external tools to interact with environments. However, integrating external tools introduces security risks, such as indirect prompt injection (IPI) attacks. Despite defenses designed for IPI attacks, their robustness remains questionable due to insufficient testing against adaptive attacks. In this paper, we evaluate eight different defenses and bypass all of them using adaptive attacks, consistently achieving an attack success rate of over 50%. This reveals critical vulnerabilities in current defenses. Our research underscores the need for adaptive attack evaluation when designing defenses to ensure robustness and reliability. The code is available at this https URL.

Title: CRFU: Compressive Representation Forgetting Against Privacy Leakage on Machine Unlearning

• Authors: Weiqi Wang, Chenhan Zhang, Zhiyi Tian, Shushu Liu, Shui Yu
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00062
• Pdf URL: https://arxiv.org/pdf/2503.00062
• Copy Paste: [[2503.00062]] CRFU: Compressive Representation Forgetting Against Privacy Leakage on Machine Unlearning(https://arxiv.org/abs/2503.00062)
• Keywords: privacy, defense, attack, membership infer
• Abstract: Machine unlearning allows data owners to erase the impact of their specified data from trained models. Unfortunately, recent studies have shown that adversaries can recover the erased data, posing serious threats to user privacy. An effective unlearning method removes the information of the specified data from the trained model, resulting in different outputs for the same input before and after unlearning. Adversaries can exploit these output differences to conduct privacy leakage attacks, such as reconstruction and membership inference attacks. However, directly applying traditional defenses to unlearning leads to significant model utility degradation. In this paper, we introduce a Compressive Representation Forgetting Unlearning scheme (CRFU), designed to safeguard against privacy leakage on unlearning. CRFU achieves data erasure by minimizing the mutual information between the trained compressive representation (learned through information bottleneck theory) and the erased data, thereby maximizing the distortion of data. This ensures that the model's output contains less information that adversaries can exploit. Furthermore, we introduce a remembering constraint and an unlearning rate to balance the forgetting of erased data with the preservation of previously learned knowledge, thereby reducing accuracy degradation. Theoretical analysis demonstrates that CRFU can effectively defend against privacy leakage attacks. Our experimental results show that CRFU significantly increases the reconstruction mean square error (MSE), achieving a defense effect improvement of approximately $200\%$ against privacy reconstruction attacks with only $1.5\%$ accuracy degradation on MNIST.

Title: NoPain: No-box Point Cloud Attack via Optimal Transport Singular Boundary

• Authors: Zezeng Li, Xiaoyu Du, Na Lei, Liming Chen, Weimin Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00063
• Pdf URL: https://arxiv.org/pdf/2503.00063
• Copy Paste: [[2503.00063]] NoPain: No-box Point Cloud Attack via Optimal Transport Singular Boundary(https://arxiv.org/abs/2503.00063)
• Keywords: defense, attack
• Abstract: Adversarial attacks exploit the vulnerability of deep models against adversarial samples. Existing point cloud attackers are tailored to specific models, iteratively optimizing perturbations based on gradients in either a white-box or black-box setting. Despite their promising attack performance, they often struggle to produce transferable adversarial samples due to overfitting the specific parameters of surrogate models. To overcome this issue, we shift our focus to the data distribution itself and introduce a novel approach named \textbf{NoPain}, which employs optimal transport (OT) to identify the inherent singular boundaries of the data manifold for cross-network point cloud attacks. Specifically, we first calculate the OT mapping from noise to the target feature space, then identify singular boundaries by locating non-differentiable positions. Finally, we sample along singular boundaries to generate adversarial point clouds. Once the singular boundaries are determined, NoPain can efficiently produce adversarial samples without the need of iterative updates or guidance from the surrogate classifiers. Extensive experiments demonstrate that the proposed end-to-end method outperforms baseline approaches in terms of both transferability and efficiency, while also maintaining notable advantages even against defense strategies. The source code will be publicly available.

Title: ADAGE: Active Defenses Against GNN Extraction

• Authors: Jing Xu, Franziska Boenisch, Adam Dziedzic
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00065
• Pdf URL: https://arxiv.org/pdf/2503.00065
• Copy Paste: [[2503.00065]] ADAGE: Active Defenses Against GNN Extraction(https://arxiv.org/abs/2503.00065)
• Keywords: secure, defense, attack, steal, extraction
• Abstract: Graph Neural Networks (GNNs) achieve high performance in various real-world applications, such as drug discovery, traffic states prediction, and recommendation systems. The fact that building powerful GNNs requires a large amount of training data, powerful computing resources, and human expertise turns the models into lucrative targets for model stealing attacks. Prior work has revealed that the threat vector of stealing attacks against GNNs is large and diverse, as an attacker can leverage various heterogeneous signals ranging from node labels to high-dimensional node embeddings to create a local copy of the target GNN at a fraction of the original training costs. This diversity in the threat vector renders the design of effective and general defenses challenging and existing defenses usually focus on one particular stealing setup. Additionally, they solely provide means to identify stolen model copies rather than preventing the attack. To close this gap, we propose the first and general Active Defense Against GNN Extraction (ADAGE). By analyzing the queries to the GNN, tracking their diversity in terms of proximity to different communities identified in the underlying graph, and increasing the defense strength with the growing fraction of communities that have been queried, ADAGE can prevent stealing in all common attack setups. Our extensive experimental evaluation using six benchmark datasets, four GNN models, and three types of adaptive attackers shows that ADAGE penalizes attackers to the degree of rendering stealing impossible, whilst not harming predictive performance for legitimate users. ADAGE, thereby, contributes towards securely sharing valuable GNNs in the future.

Title: CrowdAL: Towards a Blockchain-empowered Active Learning System in Crowd Data Labeling

• Authors: Shaojie Hou, Yuandou Wang, Zhiming Zhao
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00066
• Pdf URL: https://arxiv.org/pdf/2503.00066
• Copy Paste: [[2503.00066]] CrowdAL: Towards a Blockchain-empowered Active Learning System in Crowd Data Labeling(https://arxiv.org/abs/2503.00066)
• Keywords: privacy, protect
• Abstract: Active Learning (AL) is a machine learning technique where the model selectively queries the most informative data points for labeling by human experts. Integrating AL with crowdsourcing leverages crowd diversity to enhance data labeling but introduces challenges in consensus and privacy. This poster presents CrowdAL, a blockchain-empowered crowd AL system designed to address these challenges. CrowdAL integrates blockchain for transparency and a tamper-proof incentive mechanism, using smart contracts to evaluate crowd workers' performance and aggregate labeling results, and employs zero-knowledge proofs to protect worker privacy.

Title: PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing

• Authors: Ziyu Wu, Yufan Xiong, Mengting Niu, Fangting Xie, Quan Wan, Qijun Ying, Boyan Liu, Xiaohui Cai
• Subjects: cs.CV, cs.ET
• Abstract URL: https://arxiv.org/abs/2503.00068
• Pdf URL: https://arxiv.org/pdf/2503.00068
• Copy Paste: [[2503.00068]] PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing(https://arxiv.org/abs/2503.00068)
• Keywords: privacy, robust
• Abstract: Long-term in-bed monitoring benefits automatic and real-time health management within healthcare, and the advancement of human shape reconstruction technologies further enhances the representation and visualization of users' activity patterns. However, existing technologies are primarily based on visual cues, facing serious challenges in non-light-of-sight and privacy-sensitive in-bed scenes. Pressure-sensing bedsheets offer a promising solution for real-time motion reconstruction. Yet, limited exploration in model designs and data have hindered its further development. To tackle these issues, we propose a general framework that bridges gaps in data annotation and model design. Firstly, we introduce SMPLify-IB, an optimization method that overcomes the depth ambiguity issue in top-view scenarios through gravity constraints, enabling generating high-quality 3D human shape annotations for in-bed datasets. Then we present PI-HMR, a temporal-based human shape estimator to regress meshes from pressure sequences. By integrating multi-scale feature fusion with high-pressure distribution and spatial position priors, PI-HMR outperforms SOTA methods with 17.01mm Mean-Per-Joint-Error decrease. This work provides a whole

Title: Systematic Review of Cybersecurity in Banking: Evolution from Pre-Industry 4.0 to Post-Industry 4.0 in Artificial Intelligence, Blockchain, Policies and Practice

• Authors: Tue Nhi Tran
• Subjects: cs.CR, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2503.00070
• Pdf URL: https://arxiv.org/pdf/2503.00070
• Copy Paste: [[2503.00070]] Systematic Review of Cybersecurity in Banking: Evolution from Pre-Industry 4.0 to Post-Industry 4.0 in Artificial Intelligence, Blockchain, Policies and Practice(https://arxiv.org/abs/2503.00070)
• Keywords: security
• Abstract: Throughout the history from pre-industry 4.0 to post-industry 4.0, cybersecurity at banks has undergone significant changes. Pre-industry 4.0 cyber security at banks relied on individual security methods that were highly manual and had low accuracy. When moving to post-industry 4.0, cybersecurity at banks had a major turning point with security methods that combined different technologies such as Artificial Intelligence (AI), Blockchain, IoT, automating necessary processes and significantly increasing the defence layer for banks. However, along with the development of new technologies, the current challenge of cybersecurity at banks lies in scalability, high costs and resources in both money and time for R&D of defence methods along with the threat of high-tech cybercriminals growing and expanding. This report goes from introducing the importance of cybersecurity at banks, analyzing their management, operational and business objectives, evaluating pre-industry 4.0 technologies used for cybersecurity at banks to assessing post-industry 4.0 technologies focusing on Artificial Intelligence and Blockchain, discussing current policies and practices and ending with discussing key advantages and challenges for 4.0 technologies and recommendations for further developing cybersecurity at banks.

Title: I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue

• Authors: Esam Ghaleb, Bulat Khaertdinov, Aslı Özyürek, Raquel Fernández
• Subjects: cs.CV, cs.CL, cs.MM
• Abstract URL: https://arxiv.org/abs/2503.00071
• Pdf URL: https://arxiv.org/pdf/2503.00071
• Copy Paste: [[2503.00071]] I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue(https://arxiv.org/abs/2503.00071)
• Keywords: robust
• Abstract: In face-to-face interaction, we use multiple modalities, including speech and gestures, to communicate information and resolve references to objects. However, how representational co-speech gestures refer to objects remains understudied from a computational perspective. In this work, we address this gap by introducing a multimodal reference resolution task centred on representational gestures, while simultaneously tackling the challenge of learning robust gesture embeddings. We propose a self-supervised pre-training approach to gesture representation learning that grounds body movements in spoken language. Our experiments show that the learned embeddings align with expert annotations and have significant predictive power. Moreover, reference resolution accuracy further improves when (1) using multimodal gesture representations, even when speech is unavailable at inference time, and (2) leveraging dialogue history. Overall, our findings highlight the complementary roles of gesture and speech in reference resolution, offering a step towards more naturalistic models of human-machine interaction.

Title: Generalization of CNNs on Relational Reasoning with Bar Charts

• Authors: Zhenxing Cui, Lu Chen, Yunhai Wang, Daniel Haehn, Yong Wang, Hanspeter Pfister
• Subjects: cs.CV, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00086
• Pdf URL: https://arxiv.org/pdf/2503.00086
• Copy Paste: [[2503.00086]] Generalization of CNNs on Relational Reasoning with Bar Charts(https://arxiv.org/abs/2503.00086)
• Keywords: robust
• Abstract: This paper presents a systematic study of the generalization of convolutional neural networks (CNNs) and humans on relational reasoning tasks with bar charts. We first revisit previous experiments on graphical perception and update the benchmark performance of CNNs. We then test the generalization performance of CNNs on a classic relational reasoning task: estimating bar length ratios in a bar chart, by progressively perturbing the standard visualizations. We further conduct a user study to compare the performance of CNNs and humans. Our results show that CNNs outperform humans only when the training and test data have the same visual encodings. Otherwise, they may perform worse. We also find that CNNs are sensitive to perturbations in various visual encodings, regardless of their relevance to the target bars. Yet, humans are mainly influenced by bar lengths. Our study suggests that robust relational reasoning with visualizations is challenging for CNNs. Improving CNNs' generalization performance may require training them to better recognize task-related visual properties.

Title: Gaussian process surrogate model to approximate power grid simulators -- An application to the certification of a congestion management controller

• Authors: Pierre Houdouin, Manuel Ruiz, Lucas Saludjian
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00094
• Pdf URL: https://arxiv.org/pdf/2503.00094
• Copy Paste: [[2503.00094]] Gaussian process surrogate model to approximate power grid simulators -- An application to the certification of a congestion management controller(https://arxiv.org/abs/2503.00094)
• Keywords: interpretability
• Abstract: With the digitalization of power grids, physical equations become insufficient to describe the network's behavior, and realistic but time-consuming simulators must be used. Numerical experiments, such as safety validation, that involve simulating a large number of scenarios become computationally intractable. A popular solution to reduce the computational burden is to learn a surrogate model of the simulator with Machine Learning (ML) and then conduct the experiment directly on the fast-to-evaluate surrogate model. Among the various ML possibilities for building surrogate models, Gaussian processes (GPs) emerged as a popular solution due to their flexibility, data efficiency, and interpretability. Their probabilistic nature enables them to provide both predictions and uncertainty quantification (UQ). This paper starts with a discussion on the interest of using GPs to approximate power grid simulators and fasten numerical experiments. Such simulators, however, often violate the GP's underlying Gaussian assumption, leading to poor approximations. To address this limitation, an approach that consists in adding an adaptive residual uncertainty term to the UQ is proposed. It enables the GP to remain accurate and reliable despite the simulator's non-Gaussian behaviors. This approach is successfully applied to the certification of the proper functioning of a congestion management controller, with over 98% of simulations avoided.

Title: Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks

• Authors: Nikita Soni, Pranav Chitale, Khushboo Singh, Niranjan Balasubramanian, H. Andrew Schwartz
• Subjects: cs.CL, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00124
• Pdf URL: https://arxiv.org/pdf/2503.00124
• Copy Paste: [[2503.00124]] Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks(https://arxiv.org/abs/2503.00124)
• Keywords: transformer
• Abstract: Like most of NLP, models for human-centered NLP tasks -- tasks attempting to assess author-level information -- predominantly use representations derived from hidden states of Transformer-based LLMs. However, what component of the LM is used for the representation varies widely. Moreover, there is a need for Human Language Models (HuLMs) that implicitly model the author and provide a user-level hidden state. Here, we systematically evaluate different ways of representing documents and users using different LM and HuLM architectures to predict task outcomes as both dynamically changing states and averaged trait-like user-level attributes of valence, arousal, empathy, and distress. We find that representing documents as an average of the token hidden states performs the best generally. Further, while a user-level hidden state itself is rarely the best representation, we find its inclusion in the model strengthens token or document embeddings used to derive document- and user-level representations resulting in best performances.

Title: AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction

• Authors: Magnus Sesodia, Alina Petrova, John Armour, Thomas Lukasiewicz, Oana-Maria Camburu, Puneet K. Dokania, Philip Torr, Christian Schroeder de Witt
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00128
• Pdf URL: https://arxiv.org/pdf/2503.00128
• Copy Paste: [[2503.00128]] AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction(https://arxiv.org/abs/2503.00128)
• Keywords: explainability, large language model
• Abstract: Legal systems worldwide continue to struggle with overwhelming caseloads, limited judicial resources, and growing complexities in legal proceedings. Artificial intelligence (AI) offers a promising solution, with Legal Judgment Prediction (LJP) -- the practice of predicting a court's decision from the case facts -- emerging as a key research area. However, existing datasets often formulate the task of LJP unrealistically, not reflecting its true difficulty. They also lack high-quality annotation essential for legal reasoning and explainability. To address these shortcomings, we introduce AnnoCaseLaw, a first-of-its-kind dataset of 471 meticulously annotated U.S. Appeals Court negligence cases. Each case is enriched with comprehensive, expert-labeled annotations that highlight key components of judicial decision making, along with relevant legal concepts. Our dataset lays the groundwork for more human-aligned, explainable LJP models. We define three legally relevant tasks: (1) judgment prediction; (2) concept identification; and (3) automated case annotation, and establish a performance baseline using industry-leading large language models (LLMs). Our results demonstrate that LJP remains a formidable task, with application of legal precedent proving particularly difficult. Code and data are available at this https URL.

Title: CNSv2: Probabilistic Correspondence Encoded Neural Image Servo

• Authors: Anzhe Chen, Hongxiang Yu, Shuxin Li, Yuxi Chen, Zhongxiang Zhou, Wentao Sun, Rong Xiong, Yue Wang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.00132
• Pdf URL: https://arxiv.org/pdf/2503.00132
• Copy Paste: [[2503.00132]] CNSv2: Probabilistic Correspondence Encoded Neural Image Servo(https://arxiv.org/abs/2503.00132)
• Keywords: robust
• Abstract: Visual servo based on traditional image matching methods often requires accurate keypoint correspondence for high precision control. However, keypoint detection or matching tends to fail in challenging scenarios with inconsistent illuminations or textureless objects, resulting significant performance degradation. Previous approaches, including our proposed Correspondence encoded Neural image Servo policy (CNS), attempted to alleviate these issues by integrating neural control strategies. While CNS shows certain improvement against error correspondence over conventional image-based controllers, it could not fully resolve the limitations arising from poor keypoint detection and matching. In this paper, we continue to address this problem and propose a new solution: Probabilistic Correspondence Encoded Neural Image Servo (CNSv2). CNSv2 leverages probabilistic feature matching to improve robustness in challenging scenarios. By redesigning the architecture to condition on multimodal feature matching, CNSv2 achieves high precision, improved robustness across diverse scenes and runs in real-time. We validate CNSv2 with simulations and real-world experiments, demonstrating its effectiveness in overcoming the limitations of detector-based methods in visual servo tasks.

Title: Personalized Causal Graph Reasoning for LLMs: A Case Study on Dietary Recommendations

• Authors: Zhongqi Yang, Amir Rahmani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00134
• Pdf URL: https://arxiv.org/pdf/2503.00134
• Copy Paste: [[2503.00134]] Personalized Causal Graph Reasoning for LLMs: A Case Study on Dietary Recommendations(https://arxiv.org/abs/2503.00134)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) effectively leverage common-sense knowledge for general reasoning, yet they struggle with personalized reasoning when tasked with interpreting multifactor personal data. This limitation restricts their applicability in domains that require context-aware decision-making tailored to individuals. This paper introduces Personalized Causal Graph Reasoning as an agentic framework that enhances LLM reasoning by incorporating personal causal graphs derived from data of individuals. These graphs provide a foundation that guides the LLM's reasoning process. We evaluate it on a case study on nutrient-oriented dietary recommendations, which requires personal reasoning due to the implicit unique dietary effects. We propose a counterfactual evaluation to estimate the efficiency of LLM-recommended foods for glucose management. Results demonstrate that the proposed method efficiently provides personalized dietary recommendations to reduce average glucose iAUC across three time windows, which outperforms the previous approach. LLM-as-a-judge evaluation results indicate that our proposed method enhances personalization in the reasoning process.

Title: SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models

• Authors: Grigor Nalbandyan, Rima Shahbazyan, Evelina Bakhturina
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00137
• Pdf URL: https://arxiv.org/pdf/2503.00137
• Copy Paste: [[2503.00137]] SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models(https://arxiv.org/abs/2503.00137)
• Keywords: robust, large language model
• Abstract: Typical evaluations of Large Language Models (LLMs) report a single metric per dataset, often representing the model's best-case performance under carefully selected settings. Unfortunately, this approach overlooks model robustness and reliability in real-world applications. For instance, simple paraphrasing of prompts on the MMLU-Pro dataset causes accuracy fluctuations of up to 10\%, while reordering answer choices in the AGIEval dataset results in accuracy differences of up to 6.1\%. While some studies discuss issues with LLM robustness, there is no unified or centralized framework for evaluating the robustness of language models. To address this gap and consolidate existing research on model robustness, we present SCORE ($\mathbf{S}$ystematic $\mathbf{CO}$nsistency and $\mathbf{R}$obustness $\mathbf{E}$valuation), a comprehensive framework for non-adversarial evaluation of LLMs. The SCORE framework evaluates models by repeatedly testing them on the same benchmarks in various setups to give a realistic estimate of their accuracy and consistency. We release the code publicly and start an LLM robustness leaderboard to facilitate further development and research.

Title: AMuLeT: Automated Design-Time Testing of Secure Speculation Countermeasures

• Authors: Bo Fu, Leo Tenenbaum, David Adler, Assaf Klein, Arpit Gogia, Alaa R. Alameldeen, Marco Guarnieri, Mark Silberstein, Oleksii Oleksenko, Gururaj Saileshwar
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2503.00145
• Pdf URL: https://arxiv.org/pdf/2503.00145
• Copy Paste: [[2503.00145]] AMuLeT: Automated Design-Time Testing of Secure Speculation Countermeasures(https://arxiv.org/abs/2503.00145)
• Keywords: secure, security, defense, attack
• Abstract: In recent years, several hardware-based countermeasures proposed to mitigate Spectre attacks have been shown to be insecure. To enable the development of effective secure speculation countermeasures, we need easy-to-use tools that can automatically test their security guarantees early-on in the design phase to facilitate rapid prototyping. This paper develops AMuLeT, the first tool capable of testing secure speculation countermeasures for speculative leakage early in their design phase in simulators. Our key idea is to leverage model-based relational testing tools that can detect speculative leaks in commercial CPUs, and apply them to micro-architectural simulators to test secure speculation defenses. We identify and overcome several challenges, including designing an expressive yet realistic attacker observer model in a simulator, overcoming the slow simulation speed, and searching the vast micro-architectural state space for potential vulnerabilities. AMuLeT speeds up test throughput by more than 10x compared to a naive design and uses techniques to amplify vulnerabilities to uncover them within a limited test budget. Using AMuLeT, we launch for the first time, a systematic, large-scale testing campaign of four secure speculation countermeasures from 2018 to 2024--InvisiSpec, CleanupSpec, STT, and SpecLFB--and uncover 3 known and 6 unknown bugs and vulnerabilities, within 3 hours of testing. We also show for the first time that the open-source implementation of SpecLFB is insecure.

Title: Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs

• Authors: Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy, Abdelrahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, Reem Abdel-Salam, Hanin Atwany, Youssef Nafea, Abdulfattah Mohammed Yahya, Rahaf Alhamouri, Hamzah A. Alsayadi, Hiba Zayed, Sara Shatnawi, Serry Sibaee, Yasir Ech-Chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, Ahmed Oumar El-Shangiti, Aisha Alraeesi, Mohammed Anwar Al-Ghrawi, Abdulrahman S. Al-Batati, Elgizouli Mohamed, Noha Taha Elgindi, Muhammed Saeed, Houdaifa Atou, Issam Ait Yahia, Abdelhak Bouayad, Mohammed Machrouh, Amal Makouar, Dania Alkawi, Mukhtar Mohamed, Safaa Taher Abdelfadil, Amine Ziad Ounnoughene, Rouabhia Anfel, Rwaa Assi, Ahmed Sorkatti, Mohamedou Cheikh Tourad, Anis Koubaa, Ismail Berrada, Mustafa Jarrar, Shady Shehata, Muhammad Abdul-Mageed
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00151
• Pdf URL: https://arxiv.org/pdf/2503.00151
• Copy Paste: [[2503.00151]] Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs(https://arxiv.org/abs/2503.00151)
• Keywords: large language model
• Abstract: As large language models (LLMs) become increasingly integrated into daily life, ensuring their cultural sensitivity and inclusivity is paramount. We introduce our dataset, a year-long community-driven project covering all 22 Arab countries. The dataset includes instructions (input, response pairs) in both Modern Standard Arabic (MSA) and dialectal Arabic (DA), spanning 20 diverse topics. Built by a team of 44 researchers across the Arab world, all of whom are authors of this paper, our dataset offers a broad, inclusive perspective. We use our dataset to evaluate the cultural and dialectal capabilities of several frontier LLMs, revealing notable limitations. For instance, while closed-source LLMs generally exhibit strong performance, they are not without flaws, and smaller open-source models face greater challenges. Moreover, certain countries (e.g., Egypt, the UAE) appear better represented than others (e.g., Iraq, Mauritania, Yemen). Our annotation guidelines, code, and data for reproducibility are publicly available.

Title: Transforming Cyber Defense: Harnessing Agentic and Frontier AI for Proactive, Ethical Threat Intelligence

• Authors: Krti Tallam
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00164
• Pdf URL: https://arxiv.org/pdf/2503.00164
• Copy Paste: [[2503.00164]] Transforming Cyber Defense: Harnessing Agentic and Frontier AI for Proactive, Ethical Threat Intelligence(https://arxiv.org/abs/2503.00164)
• Keywords: security, defense, robust, fair
• Abstract: In an era marked by unprecedented digital complexity, the cybersecurity landscape is evolving at a breakneck pace, challenging traditional defense paradigms. Advanced Persistent Threats (APTs) reveal inherent vulnerabilities in conventional security measures and underscore the urgent need for continuous, adaptive, and proactive strategies that seamlessly integrate human insight with cutting edge AI technologies. This manuscript explores how the convergence of agentic AI and Frontier AI is transforming cybersecurity by reimagining frameworks such as the cyber kill chain, enhancing threat intelligence processes, and embedding robust ethical governance within automated response systems. Drawing on real-world data and forward looking perspectives, we examine the roles of real time monitoring, automated incident response, and perpetual learning in forging a resilient, dynamic defense ecosystem. Our vision is to harmonize technological innovation with unwavering ethical oversight, ensuring that future AI driven security solutions uphold core human values of fairness, transparency, and accountability while effectively countering emerging cyber threats.

Title: SSL4EO-S12 v1.1: A Multimodal, Multiseasonal Dataset for Pretraining, Updated

• Authors: Benedikt Blumenstiel, Nassim Ait Ali Braham, Conrad M Albrecht, Stefano Maurogiovanni, Paolo Fraccaro
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00168
• Pdf URL: https://arxiv.org/pdf/2503.00168
• Copy Paste: [[2503.00168]] SSL4EO-S12 v1.1: A Multimodal, Multiseasonal Dataset for Pretraining, Updated(https://arxiv.org/abs/2503.00168)
• Keywords: robust
• Abstract: This technical report presents SSL4EO-S12 v1.1, a multimodal, multitemporal Earth Observation dataset designed for pretraining large-scale foundation models. Building on the success of SSL4EO-S12 v1.0, the new version addresses the previous challenges of data misalignment and a limited data structure for low-barrier, analysis-ready EO processing. SSL4EO-S12 v1.1 covers the world's 10,000 largest cities and its surroundings within a 50 km radius across four seasons, resulting in a diverse collection of nearly one million patches. SSL4EO-S12 v1.1 packages the data in Zarr file format for cloud-efficient loading and representation of meta-information such as including cloud masks and geolocation. Released under the CC-BY-4.0 license, SSL4EO-S12 v1.1 facilitates open research and provides a robust foundation for future advancements in self-supervised learning and geospatial analysis. The dataset is available online through this https URL, and we provided additional resources at this https URL.

Title: PaliGemma-CXR: A Multi-task Multimodal Model for TB Chest X-ray Interpretation

• Authors: Denis Musinguzi, Andrew Katumba, Sudi Murindanyi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00171
• Pdf URL: https://arxiv.org/pdf/2503.00171
• Copy Paste: [[2503.00171]] PaliGemma-CXR: A Multi-task Multimodal Model for TB Chest X-ray Interpretation(https://arxiv.org/abs/2503.00171)
• Keywords: segmentation
• Abstract: Tuberculosis (TB) is a infectious global health challenge. Chest X-rays are a standard method for TB screening, yet many countries face a critical shortage of radiologists capable of interpreting these images. Machine learning offers an alternative, as it can automate tasks such as disease diagnosis, and report generation. However, traditional approaches rely on task-specific models, which cannot utilize the interdependence between tasks. Building a multi-task model capable of performing multiple tasks poses additional challenges such as scarcity of multimodal data, dataset imbalance, and negative transfer. To address these challenges, we propose PaliGemma-CXR, a multi-task multimodal model capable of performing TB diagnosis, object detection, segmentation, report generation, and VQA. Starting with a dataset of chest X-ray images annotated with TB diagnosis labels and segmentation masks, we curated a multimodal dataset to support additional tasks. By finetuning PaliGemma on this dataset and sampling data using ratios of the inverse of the size of task datasets, we achieved the following results across all tasks: 90.32% accuracy on TB diagnosis and 98.95% on close-ended VQA, 41.3 BLEU score on report generation, and a mAP of 19.4 and 16.0 on object detection and segmentation, respectively. These results demonstrate that PaliGemma-CXR effectively leverages the interdependence between multiple image interpretation tasks to enhance performance.

Title: A Survey of Uncertainty Estimation Methods on Large Language Models

• Authors: Zhiqiu Xia, Jinxuan Xu, Yuqian Zhang, Hang Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00172
• Pdf URL: https://arxiv.org/pdf/2503.00172
• Copy Paste: [[2503.00172]] A Survey of Uncertainty Estimation Methods on Large Language Models(https://arxiv.org/abs/2503.00172)
• Keywords: large language model
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, these models could offer biased, hallucinated, or non-factual responses camouflaged by their fluency and realistic appearance. Uncertainty estimation is the key method to address this challenge. While research efforts in uncertainty estimation are ramping up, there is a lack of comprehensive and dedicated surveys on LLM uncertainty estimation. This survey presents four major avenues of LLM uncertainty estimation. Furthermore, we perform extensive experimental evaluations across multiple methods and datasets. At last, we provide critical and promising future directions for LLM uncertainty estimation.

Title: Steering Large Language Model Activations in Sparse Spaces

• Authors: Reza Bayat, Ali Rahimi-Kalahroudi, Mohammad Pezeshki, Sarath Chandar, Pascal Vincent
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00177
• Pdf URL: https://arxiv.org/pdf/2503.00177
• Copy Paste: [[2503.00177]] Steering Large Language Model Activations in Sparse Spaces(https://arxiv.org/abs/2503.00177)
• Keywords: interpretability, large language model
• Abstract: A key challenge in AI alignment is guiding large language models (LLMs) to follow desired behaviors at test time. Activation steering, which modifies internal model activations during inference, offers a potential solution. However, prior work in dense activation spaces struggles with superposition, wherein multiple features become entangled, limiting interpretability and precise control. In contrast, sparse representations provide an untapped opportunity for more interpretable behavior modulation. In this work, we introduce sparse activation steering (SAS), a method that leverages sparse autoencoders (SAEs) to steer LLM behavior in sparse spaces. By isolating behavior-specific features through a contrastive prompt-pairing approach, we define a set of features that can selectively reinforce or suppress behaviors. Experiments on Gemma 2 LLMs show that SAS vectors enable nuanced behavioral modulation and finer-grained control. Furthermore, scaling SAEs improves monosemanticity of SAS vectors, suggesting more reliable and interpretable interventions.

Title: Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks

• Authors: Hanjiang Hu, Alexander Robey, Changliu Liu
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00187
• Pdf URL: https://arxiv.org/pdf/2503.00187
• Copy Paste: [[2503.00187]] Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks(https://arxiv.org/abs/2503.00187)
• Keywords: defense, attack, robust, large language model
• Abstract: Large language models (LLMs) are highly vulnerable to jailbreaking attacks, wherein adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering unsafe inputs, they fail against multi-turn jailbreaks that exploit contextual drift over multiple interactions, gradually leading LLMs away from safe behavior. To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues. Our approach models the dialogue with LLMs using state-space representations and introduces a novel neural barrier function (NBF) to detect and filter harmful queries emerging from evolving contexts proactively. Our method achieves invariant safety at each turn of dialogue by learning a safety predictor that accounts for adversarial queries, preventing potential context drift toward jailbreaks. Extensive experiments under multiple LLMs show that our NBF-based safety steering outperforms safety alignment baselines, offering stronger defenses against multi-turn jailbreaks while maintaining a better trade-off between safety and helpfulness under different multi-turn jailbreak methods. Our code is available at this https URL .

Title: PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion

• Authors: Amar Kumar, Anita Kriz, Mohammad Havaei, Tal Arbel
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00196
• Pdf URL: https://arxiv.org/pdf/2503.00196
• Copy Paste: [[2503.00196]] PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion(https://arxiv.org/abs/2503.00196)
• Keywords: robust, diffusion
• Abstract: Developing reliable and generalizable deep learning systems for medical imaging faces significant obstacles due to spurious correlations, data imbalances, and limited text annotations in datasets. Addressing these challenges requires architectures robust to the unique complexities posed by medical imaging data. The rapid advancements in vision-language foundation models within the natural image domain prompt the question of how they can be adapted for medical imaging tasks. In this work, we present PRISM, a framework that leverages foundation models to generate high-resolution, language-guided medical image counterfactuals using Stable Diffusion. Our approach demonstrates unprecedented precision in selectively modifying spurious correlations (the medical devices) and disease features, enabling the removal and addition of specific attributes while preserving other image characteristics. Through extensive evaluation, we show how PRISM advances counterfactual generation and enables the development of more robust downstream classifiers for clinically deployable solutions. To facilitate broader adoption and research, we make our code publicly available at this https URL.

Title: Llamarine: Open-source Maritime Industry-specific Large Language Model

• Authors: William Nguyen, An Phan, Konobu Kimura, Hitoshi Maeno, Mika Tanaka, Quynh Le, William Poucher, Christopher Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00203
• Pdf URL: https://arxiv.org/pdf/2503.00203
• Copy Paste: [[2503.00203]] Llamarine: Open-source Maritime Industry-specific Large Language Model(https://arxiv.org/abs/2503.00203)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated substantial potential in addressing complex reasoning tasks, yet their general-purpose nature often limits their effectiveness in specialized domains such as maritime navigation. To bridge this gap, we introduce Llamarine, the first open-source LLM designed specifically for maritime navigation. Llamarine 1.0 is developed through continued pretraining and fine-tuning on a high-quality corpus comprising maritime textbooks, research publications, and web text from Wikipedia. This domain-specific training enables the model to acquire expert-level knowledge in navigational principles, collision avoidance, route optimization, and regulatory compliance. Our key contributions include (a) the curation of a comprehensive maritime dataset from authoritative sources, ensuring depth and reliability in the model's knowledge base; (b) the development of a foundational model capable of reasoning about complex navigational challenges with greater accuracy than general-purpose LLMs; and (c) the establishment of a benchmark to evaluate performance in maritime-specific decision-making tasks. Experimental results demonstrate that Llamarine outperforms both general-purpose and commercial LLMs in critical navigation-related tasks, such as trajectory planning, risk assessment, and compliance with maritime regulations. By providing an open-source foundation model trained exclusively on high-quality maritime literature, Llamarine paves the way for AI-driven advancements in maritime safety, efficiency, and operational decision-making.

Title: AnalogGenie: A Generative Engine for Automatic Discovery of Analog Circuit Topologies

• Authors: Jian Gao, Weidong Cao, Junyi Yang, Xuan Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00205
• Pdf URL: https://arxiv.org/pdf/2503.00205
• Copy Paste: [[2503.00205]] AnalogGenie: A Generative Engine for Automatic Discovery of Analog Circuit Topologies(https://arxiv.org/abs/2503.00205)
• Keywords: generative
• Abstract: The massive and large-scale design of foundational semiconductor integrated circuits (ICs) is crucial to sustaining the advancement of many emerging and future technologies, such as generative AI, 5G/6G, and quantum computing. Excitingly, recent studies have shown the great capabilities of foundational models in expediting the design of digital ICs. Yet, applying generative AI techniques to accelerate the design of analog ICs remains a significant challenge due to critical domain-specific issues, such as the lack of a comprehensive dataset and effective representation methods for analog circuits. This paper proposes, $\textbf{AnalogGenie}$, a $\underline{\textbf{Gen}}$erat$\underline{\textbf{i}}$ve $\underline{\textbf{e}}$ngine for automatic design/discovery of $\underline{\textbf{Analog}}$ circuit topologies--the most challenging and creative task in the conventional manual design flow of analog ICs. AnalogGenie addresses two key gaps in the field: building a foundational comprehensive dataset of analog circuit topology and developing a scalable sequence-based graph representation universal to analog circuits. Experimental results show the remarkable generation performance of AnalogGenie in broadening the variety of analog ICs, increasing the number of devices within a single design, and discovering unseen circuit topologies far beyond any prior arts. Our work paves the way to transform the longstanding time-consuming manual design flow of analog ICs to an automatic and massive manner powered by generative AI. Our source code is available at this https URL.

Title: Quantifying First-Order Markov Violations in Noisy Reinforcement Learning: A Causal Discovery Approach

• Authors: Naveen Mysore
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00206
• Pdf URL: https://arxiv.org/pdf/2503.00206
• Copy Paste: [[2503.00206]] Quantifying First-Order Markov Violations in Noisy Reinforcement Learning: A Causal Discovery Approach(https://arxiv.org/abs/2503.00206)
• Keywords: robust
• Abstract: Reinforcement learning (RL) methods frequently assume that each new observation completely reflects the environment's state, thereby guaranteeing Markovian (one-step) transitions. In practice, partial observability or sensor/actuator noise often invalidates this assumption. This paper proposes a systematic methodology for detecting such violations, combining a partial correlation-based causal discovery process (PCMCI) with a novel Markov Violation score (MVS). The MVS measures multi-step dependencies that emerge when noise or incomplete state information disrupts the Markov property. Classic control tasks (CartPole, Pendulum, Acrobot) serve as examples to illustrate how targeted noise and dimension omissions affect both RL performance and measured Markov consistency. Surprisingly, even substantial observation noise sometimes fails to induce strong multi-lag dependencies in certain domains (e.g., Acrobot). In contrast, dimension-dropping investigations show that excluding some state variables (e.g., angular velocities in CartPole and Pendulum) significantly reduces returns and increases MVS, while removing other dimensions has minimal impact. These findings emphasize the importance of locating and safeguarding the most causally essential dimensions in order to preserve effective single-step learning. By integrating partial correlation tests with RL performance outcomes, the proposed approach precisely identifies when and where the Markov assumption is violated. This framework offers a principled mechanism for developing robust policies, informing representation learning, and addressing partial observability in real-world RL scenarios. All code and experimental logs are accessible for reproducibility (this https URL).

Title: Autoencoder-Based Framework to Capture Vocabulary Quality in NLP

• Authors: Vu Minh Hoang Dang, Rakesh M. Verma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00209
• Pdf URL: https://arxiv.org/pdf/2503.00209
• Copy Paste: [[2503.00209]] Autoencoder-Based Framework to Capture Vocabulary Quality in NLP(https://arxiv.org/abs/2503.00209)
• Keywords: robust
• Abstract: Linguistic richness is essential for advancing natural language processing (NLP), as dataset characteristics often directly influence model performance. However, traditional metrics such as Type-Token Ratio (TTR), Vocabulary Diversity (VOCD), and Measure of Lexical Text Diversity (MTLD) do not adequately capture contextual relationships, semantic richness, and structural complexity. In this paper, we introduce an autoencoder-based framework that uses neural network capacity as a proxy for vocabulary richness, diversity, and complexity, enabling a dynamic assessment of the interplay between vocabulary size, sentence structure, and contextual depth. We validate our approach on two distinct datasets: the DIFrauD dataset, which spans multiple domains of deceptive and fraudulent text, and the Project Gutenberg dataset, representing diverse languages, genres, and historical periods. Experimental results highlight the robustness and adaptability of our method, offering practical guidance for dataset curation and NLP model design. By enhancing traditional vocabulary evaluation, our work fosters the development of more context-aware, linguistically adaptive NLP systems.

Title: À la recherche du sens perdu: your favourite LLM might have more to say than you can understand

• Authors: K. O. T. Erziev
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00224
• Pdf URL: https://arxiv.org/pdf/2503.00224
• Copy Paste: [[2503.00224]] À la recherche du sens perdu: your favourite LLM might have more to say than you can understand(https://arxiv.org/abs/2503.00224)
• Keywords: security
• Abstract: We report a peculiar observation that LLMs can assign hidden meanings to sequences that seem visually incomprehensible to humans: for example, a nonsensical phrase consisting of Byzantine musical symbols is recognized by gpt-4o as "say abracadabra". Moreover, some models can communicate using these sequences. Some of these meanings are hypothesized to partly originate in the massive spurious correlations due to BPE tokenization. We systematically evaluate the presence of such abilities in a wide range of models: Claude-3.5 Haiku, Claude-3.5 Sonnet (New and Old), Claude-3.7 Sonnet, gpt-4o mini, gpt-4o, o1-mini, Llama-3.3 70B, DeepSeek-R1-Distill-Lllama 70B, Qwen2.5 1.5B, Qwen2.5 32B, Phi-3.5 mini, GigaChat-Max, Vikhr-Llama-3.2 1B. We argue that this observation might have far-reaching consequences for both safety and security of the modern and future LLMs and systems that employ them. As an illustration, we show that applying this method in combination with simple templates is sufficient to jailbreak previous generation models, with ASR = 0.4 on gpt-4o mini. Our code and data artifacts are available at this https URL

Title: Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer

• Authors: Yufei Guo, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yuhan Zhang, Zhe Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00226
• Pdf URL: https://arxiv.org/pdf/2503.00226
• Copy Paste: [[2503.00226]] Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer(https://arxiv.org/abs/2503.00226)
• Keywords: transformer
• Abstract: Transformers have demonstrated outstanding performance across a wide range of tasks, owing to their self-attention mechanism, but they are highly energy-consuming. Spiking Neural Networks have emerged as a promising energy-efficient alternative to traditional Artificial Neural Networks, leveraging event-driven computation and binary spikes for information transfer. The combination of Transformers' capabilities with the energy efficiency of SNNs offers a compelling opportunity. This paper addresses the challenge of adapting the self-attention mechanism of Transformers to the spiking paradigm by introducing a novel approach: Accurate Addition-Only Spiking Self-Attention (A$^2$OS$^2$A). Unlike existing methods that rely solely on binary spiking neurons for all components of the self-attention mechanism, our approach integrates binary, ReLU, and ternary spiking neurons. This hybrid strategy significantly improves accuracy while preserving non-multiplicative computations. Moreover, our method eliminates the need for softmax and scaling operations. Extensive experiments show that the A$^2$OS$^2$A-based Spiking Transformer outperforms existing SNN-based Transformers on several datasets, even achieving an accuracy of 78.66\% on ImageNet-1K. Our work represents a significant advancement in SNN-based Transformer models, offering a more accurate and efficient solution for real-world applications.

Title: Jawaher: A Multidialectal Dataset of Arabic Proverbs for LLM Benchmarking

• Authors: Samar M. Magdy, Sang Yun Kwon, Fakhraddin Alwajih, Safaa Abdelfadil, Shady Shehata, Muhammad Abdul-Mageed
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00231
• Pdf URL: https://arxiv.org/pdf/2503.00231
• Copy Paste: [[2503.00231]] Jawaher: A Multidialectal Dataset of Arabic Proverbs for LLM Benchmarking(https://arxiv.org/abs/2503.00231)
• Keywords: large language model
• Abstract: Recent advancements in instruction fine-tuning, alignment methods such as reinforcement learning from human feedback (RLHF), and optimization techniques like direct preference optimization (DPO) have significantly enhanced the adaptability of large language models (LLMs) to user preferences. However, despite these innovations, many LLMs continue to exhibit biases toward Western, Anglo-centric, or American cultures, with performance on English data consistently surpassing that of other languages. This reveals a persistent cultural gap in LLMs, which complicates their ability to accurately process culturally rich and diverse figurative language such as proverbs. To address this, we introduce Jawaher, a benchmark designed to assess LLMs' capacity to comprehend and interpret Arabic proverbs. Jawaher includes proverbs from various Arabic dialects, along with idiomatic translations and explanations. Through extensive evaluations of both open- and closed-source models, we find that while LLMs can generate idiomatically accurate translations, they struggle with producing culturally nuanced and contextually relevant explanations. These findings highlight the need for ongoing model refinement and dataset expansion to bridge the cultural gap in figurative language processing.

Title: Transformers with Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation

• Authors: Kaleab A. Kinfu, René Vidal
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00232
• Pdf URL: https://arxiv.org/pdf/2503.00232
• Copy Paste: [[2503.00232]] Transformers with Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation(https://arxiv.org/abs/2503.00232)
• Keywords: robust, transformer
• Abstract: Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have led to significant progress in 2D body pose estimation. However, achieving a good balance between accuracy, efficiency, and robustness remains a challenge. For instance, CNNs are computationally efficient but struggle with long-range dependencies, while ViTs excel in capturing such dependencies but suffer from quadratic computational complexity. This paper proposes two ViT-based models for accurate, efficient, and robust 2D pose estimation. The first one, EViTPose, operates in a computationally efficient manner without sacrificing accuracy by utilizing learnable joint tokens to select and process a subset of the most important body patches, enabling us to control the trade-off between accuracy and efficiency by changing the number of patches to be processed. The second one, UniTransPose, while not allowing for the same level of direct control over the trade-off, efficiently handles multiple scales by combining (1) an efficient multi-scale transformer encoder that uses both local and global attention with (2) an efficient sub-pixel CNN decoder for better speed and accuracy. Moreover, by incorporating all joints from different benchmarks into a unified skeletal representation, we train robust methods that learn from multiple datasets simultaneously and perform well across a range of scenarios -- including pose variations, lighting conditions, and occlusions. Experiments on six benchmarks demonstrate that the proposed methods significantly outperform state-of-the-art methods while improving computational efficiency. EViTPose exhibits a significant decrease in computational complexity (30% to 44% less in GFLOPs) with a minimal drop of accuracy (0% to 3.5% less), and UniTransPose achieves accuracy improvements ranging from 0.9% to 43.8% across these benchmarks.

Title: Towards Fairness for the Right Reasons: Using Saliency Maps to Evaluate Bias Removal in Neural Networks

• Authors: Lukasz Sztukiewicz, Ignacy Stępka, Michał Wiliński, Jerzy Stefanowski
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.00234
• Pdf URL: https://arxiv.org/pdf/2503.00234
• Copy Paste: [[2503.00234]] Towards Fairness for the Right Reasons: Using Saliency Maps to Evaluate Bias Removal in Neural Networks(https://arxiv.org/abs/2503.00234)
• Keywords: protect, fair
• Abstract: The widespread adoption of machine learning systems has raised critical concerns about fairness and bias, making mitigating harmful biases essential for AI development. In this paper, we investigate the relationship between fairness improvement and the removal of harmful biases in neural networks applied to computer vision tasks. First, we introduce a set of novel XAI-based metrics that analyze saliency maps to assess shifts in a model's decision-making process. Then, we demonstrate that successful debiasing methods systematically redirect model focus away from protected attributes. Additionally, we show that techniques originally developed for artifact removal can be effectively repurposed for fairness. These findings underscore the importance of ensuring that models are fair for the right reasons, contributing to the development of more ethical and trustworthy AI systems.

Title: 1-Lipschitz Network Initialization for Certifiably Robust Classification Applications: A Decay Problem

• Authors: Marius F. R. Juston, William R. Norris, Dustin Nottage, Ahmet Soylemezoglu
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00240
• Pdf URL: https://arxiv.org/pdf/2503.00240
• Copy Paste: [[2503.00240]] 1-Lipschitz Network Initialization for Certifiably Robust Classification Applications: A Decay Problem(https://arxiv.org/abs/2503.00240)
• Keywords: attack, robust
• Abstract: This paper discusses the weight parametrization of two standard 1-Lipschitz network structure methodologies, the Almost-Orthogonal-Layers (AOL) and the SDP-based Lipschitz Layers (SLL), and derives their impact on the initialization for deep 1-Lipschitz feedforward networks in addition to discussing underlying issues surrounding this initialization. These networks are mainly used in certifiably robust classification applications to combat adversarial attacks by limiting the effects of perturbations on the output classification result. An exact and an upper bound for the parameterized weight variance was calculated assuming a standard Normal distribution initialization; additionally, an upper bound was computed assuming a Generalized Normal Distribution, generalizing the proof for Uniform, Laplace, and Normal distribution weight initializations. It is demonstrated that the weight variance holds no bearing on the output variance distribution and that only the dimension of the weight matrices matters. Additionally, this paper demonstrates that the weight initialization always causes deep 1-Lipschitz networks to decay to zero.

Title: CoSMoEs: Compact Sparse Mixture of Experts

• Authors: Patrick Huber, Akshat Shrivastava, Ernie Chang, Chinnadhurai Sankar, Ahmed Aly, Adithya Sagar
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00245
• Pdf URL: https://arxiv.org/pdf/2503.00245
• Copy Paste: [[2503.00245]] CoSMoEs: Compact Sparse Mixture of Experts(https://arxiv.org/abs/2503.00245)
• Keywords: fair
• Abstract: Sparse Mixture of Expert (MoE) models are popular foundational architectures at large scale, however, under-explored at smaller sizes. Here, we show how to enable Compact Sparse Mixture of Experts (CoSMoEs) for on-device inference. Specifically, we tackle the three main on-device dimensions: Quality, Memory and Latency. Along the quality axis, we show that in a fair evaluation (removing confounding factors) MoE architectures outperform FLOP-aligned dense models at on-device scale. We introduce weight-decomposed experts, further improving the MoE model performance. Regarding model memory and latency, we significantly improve model offloading efficiency and, in turn, reduce model inference latency.

Title: Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series

• Authors: Yanan Niu, Roy Sarkis, Demetri Psaltis, Mario Paolone, Christophe Moser, Luisa Lambertini
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00250
• Pdf URL: https://arxiv.org/pdf/2503.00250
• Copy Paste: [[2503.00250]] Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series(https://arxiv.org/abs/2503.00250)
• Keywords: transformer
• Abstract: Accurate intraday solar irradiance forecasting is crucial for optimizing dispatch planning and electricity trading. For this purpose, we introduce a novel and effective approach that includes three distinguishing components from the literature: 1) the uncommon use of single-frame public camera imagery; 2) solar irradiance time series scaled with a proposed normalization step, which boosts performance; and 3) a lightweight multimodal model, called Solar Multimodal Transformer (SMT), that delivers accurate short-term solar irradiance forecasting by combining images and scaled time series. Benchmarking against Solcast, a leading solar forecasting service provider, our model improved prediction accuracy by 25.95%. Our approach allows for easy adaptation to various camera specifications, offering broad applicability for real-world solar forecasting challenges.

Title: Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality

• Authors: Milad Yazdani, Yasamin Medghalchi, Pooria Ashrafian, Ilker Hacihaliloglu, Dena Shahriari
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00266
• Pdf URL: https://arxiv.org/pdf/2503.00266
• Copy Paste: [[2503.00266]] Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality(https://arxiv.org/abs/2503.00266)
• Keywords: privacy, diffusion, generative, segmentation
• Abstract: Deep learning models have emerged as a powerful tool for various medical applications. However, their success depends on large, high-quality datasets that are challenging to obtain due to privacy concerns and costly annotation. Generative models, such as diffusion models, offer a potential solution by synthesizing medical images, but their practical adoption is hindered by long inference times. In this paper, we propose the use of an optimal transport flow matching approach to accelerate image generation. By introducing a straighter mapping between the source and target distribution, our method significantly reduces inference time while preserving and further enhancing the quality of the outputs. Furthermore, this approach is highly adaptable, supporting various medical imaging modalities, conditioning mechanisms (such as class labels and masks), and different spatial dimensions, including 2D and 3D. Beyond image generation, it can also be applied to related tasks such as image enhancement. Our results demonstrate the efficiency and versatility of this framework, making it a promising advancement for medical imaging applications. Code with checkpoints and a synthetic dataset (beneficial for classification and segmentation) is now available on: this https URL.

Title: Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy

• Authors: Jahan C. Penny-Dimri, Magdalena Bachmann, William R. Cooke, Sam Mathewlynn, Samuel Dockree, John Tolladay, Jannik Kossen, Lin Li, Yarin Gal, Gabriel Davis Jones
• Subjects: cs.LG, cs.AI, cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.00269
• Pdf URL: https://arxiv.org/pdf/2503.00269
• Copy Paste: [[2503.00269]] Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy(https://arxiv.org/abs/2503.00269)
• Keywords: large language model
• Abstract: Large language models (LLMs) hold substantial promise for clinical decision support. However, their widespread adoption in medicine, particularly in healthcare, is hindered by their propensity to generate false or misleading outputs, known as hallucinations. In high-stakes domains such as women's health (obstetrics & gynaecology), where errors in clinical reasoning can have profound consequences for maternal and neonatal outcomes, ensuring the reliability of AI-generated responses is critical. Traditional methods for quantifying uncertainty, such as perplexity, fail to capture meaning-level inconsistencies that lead to misinformation. Here, we evaluate semantic entropy (SE), a novel uncertainty metric that assesses meaning-level variation, to detect hallucinations in AI-generated medical content. Using a clinically validated dataset derived from UK RCOG MRCOG examinations, we compared SE with perplexity in identifying uncertain responses. SE demonstrated superior performance, achieving an AUROC of 0.76 (95% CI: 0.75-0.78), compared to 0.62 (0.60-0.65) for perplexity. Clinical expert validation further confirmed its effectiveness, with SE achieving near-perfect uncertainty discrimination (AUROC: 0.97). While semantic clustering was successful in only 30% of cases, SE remains a valuable tool for improving AI safety in women's health. These findings suggest that SE could enable more reliable AI integration into clinical practice, particularly in resource-limited settings where LLMs could augment care. This study highlights the potential of SE as a key safeguard in the responsible deployment of AI-driven tools in women's health, leading to safer and more effective digital health interventions.

Title: Learning to Animate Images from A Few Videos to Portray Delicate Human Actions

• Authors: Haoxin Li, Yingchen Yu, Qilong Wu, Hanwang Zhang, Boyang Li, Song Bai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00276
• Pdf URL: https://arxiv.org/pdf/2503.00276
• Copy Paste: [[2503.00276]] Learning to Animate Images from A Few Videos to Portray Delicate Human Actions(https://arxiv.org/abs/2503.00276)
• Keywords: generative
• Abstract: Despite recent progress, video generative models still struggle to animate human actions from static images, particularly when handling uncommon actions whose training data are limited. In this paper, we investigate the task of learning to animate human actions from a small number of videos -- 16 or fewer -- which is highly valuable in real-world applications like video and movie production. Few-shot learning of generalizable motion patterns while ensuring smooth transitions from the initial reference image is exceedingly challenging. We propose FLASH (Few-shot Learning to Animate and Steer Humans), which improves motion generalization by aligning motion features and inter-frame correspondence relations between videos that share the same motion but have different appearances. This approach minimizes overfitting to visual appearances in the limited training data and enhances the generalization of learned motion patterns. Additionally, FLASH extends the decoder with additional layers to compensate lost details in the latent space, fostering smooth transitions from the initial reference image. Experiments demonstrate that FLASH effectively animates images with unseen human or scene appearances into specified actions while maintaining smooth transitions from the reference image.

Title: Robust Multi-Objective Preference Alignment with Online DPO

• Authors: Raghav Gupta, Ryan Sullivan, Yunxuan Li, Samrat Phatale, Abhinav Rastogi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00295
• Pdf URL: https://arxiv.org/pdf/2503.00295
• Copy Paste: [[2503.00295]] Robust Multi-Objective Preference Alignment with Online DPO(https://arxiv.org/abs/2503.00295)
• Keywords: robust, large language model
• Abstract: Multi-objective preference alignment of large language models (LLMs) is critical for developing AI systems that are more configurable, personalizable, helpful, and safe. However, optimizing model outputs to satisfy diverse objectives with variable weights at inference time for truly personalized models presents a significant challenge. Existing approaches are either computationally expensive to train or do not sufficiently steer model behaviors. This paper introduces the Multi-Objective Online DPO (MO-ODPO) algorithm, designed to robustly and efficiently align model behaviors with multiple, potentially conflicting human preferences. Our approach incorporates a prompt conditioning mechanism, allowing us to train a single preference-conditional policy, that can adapt to new preference combinations at inference. Experiments on two popular benchmarks show that MO-ODPO Pareto-dominates existing baselines while providing excellent inference-time steerability between diverse objectives.

Title: Hidden Convexity of Fair PCA and Fast Solver via Eigenvalue Optimization

• Authors: Junhui Shen, Aaron J. Davis, Ding Lu, Zhaojun Bai
• Subjects: cs.LG, cs.AI, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00299
• Pdf URL: https://arxiv.org/pdf/2503.00299
• Copy Paste: [[2503.00299]] Hidden Convexity of Fair PCA and Fast Solver via Eigenvalue Optimization(https://arxiv.org/abs/2503.00299)
• Keywords: fair
• Abstract: Principal Component Analysis (PCA) is a foundational technique in machine learning for dimensionality reduction of high-dimensional datasets. However, PCA could lead to biased outcomes that disadvantage certain subgroups of the underlying datasets. To address the bias issue, a Fair PCA (FPCA) model was introduced by Samadi et al. (2018) for equalizing the reconstruction loss between subgroups. The semidefinite relaxation (SDR) based approach proposed by Samadi et al. (2018) is computationally expensive even for suboptimal solutions. To improve efficiency, several alternative variants of the FPCA model have been developed. These variants often shift the focus away from equalizing the reconstruction loss. In this paper, we identify a hidden convexity in the FPCA model and introduce an algorithm for convex optimization via eigenvalue optimization. Our approach achieves the desired fairness in reconstruction loss without sacrificing performance. As demonstrated in real-world datasets, the proposed FPCA algorithm runs $8\times$ faster than the SDR-based algorithm, and only at most 85% slower than the standard PCA.

Title: Differential Coding for Training-Free ANN-to-SNN Conversion

• Authors: Zihan Huang, Wei Fang, Tong Bu, Peng Xue, Zecheng Hao, Wenxuan Liu, Yuanhong Tang, Zhaofei Yu, Tiejun Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00301
• Pdf URL: https://arxiv.org/pdf/2503.00301
• Copy Paste: [[2503.00301]] Differential Coding for Training-Free ANN-to-SNN Conversion(https://arxiv.org/abs/2503.00301)
• Keywords: transformer
• Abstract: Spiking Neural Networks (SNNs) exhibit significant potential due to their low energy consumption. Converting Artificial Neural Networks (ANNs) to SNNs is an efficient way to achieve high-performance SNNs. However, many conversion methods are based on rate coding, which requires numerous spikes and longer time-steps compared to directly trained SNNs, leading to increased energy consumption and latency. This article introduces differential coding for ANN-to-SNN conversion, a novel coding scheme that reduces spike counts and energy consumption by transmitting changes in rate information rather than rates directly, and explores its application across various layers. Additionally, the threshold iteration method is proposed to optimize thresholds based on activation distribution when converting Rectified Linear Units (ReLUs) to spiking neurons. Experimental results on various Convolutional Neural Networks (CNNs) and Transformers demonstrate that the proposed differential coding significantly improves accuracy while reducing energy consumption, particularly when combined with the threshold iteration method, achieving state-of-the-art performance.

Title: Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuning

• Authors: Tianci Liu, Ruirui Li, Yunzhe Qi, Hui Liu, Xianfeng Tang, Tianqi Zheng, Qingyu Yin, Monica Xiao Cheng, Jun Huan, Haoyu Wang, Jing Gao
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00306
• Pdf URL: https://arxiv.org/pdf/2503.00306
• Copy Paste: [[2503.00306]] Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuning(https://arxiv.org/abs/2503.00306)
• Keywords: large language model
• Abstract: Large language models (LLMs) have achieved remarkable performance on various natural language tasks. However, they are trained on static corpora and their knowledge can become outdated quickly in the fast-changing world. This motivates the development of knowledge editing methods designed to update certain knowledge in LLMs without changing unrelated others. To make selective edits, previous efforts often sought to update a small amount of parameters in some specific layer(s) of a LLM. Nonetheless, in challenging scenarios, they still fall short in making successful edits while preserving knowledge irrelevant to the updates simultaneously, resulting in a notable editing-locality trade-off. In this work, we question if the trade-offs are caused by the fact that parameter-based updates have a global effect, i.e., edited parameters affect all inputs indiscriminately. In light of this, we explore the feasibility of representation fine-tuning, which applied some linear update to a few representations in a learned subspace, for knowledge editing. While being effective to enhance an LLM's general ability as demonstrated in the previous work, we theoretically show that this linear update imposes a tension in editing-locality trade-off. Subsequently, BaFT is proposed to break the linearity. BaFT computes a weight for each basis that spans a dimension of the subspace based on the input representation. This input-dependent weighting mechanism allows BaFT to manage different types of knowledge in an adaptive way, thereby achieving a better editing-locality trade-off. Experiments on three LLMs with five editing benchmarks in diverse scenarios show the superiority of our method.

Title: Remasking Discrete Diffusion Models with Inference-Time Scaling

• Authors: Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, Volodymyr Kuleshov
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00307
• Pdf URL: https://arxiv.org/pdf/2503.00307
• Copy Paste: [[2503.00307]] Remasking Discrete Diffusion Models with Inference-Time Scaling(https://arxiv.org/abs/2503.00307)
• Keywords: diffusion
• Abstract: Part of the success of diffusion models stems from their ability to perform iterative refinement, i.e., repeatedly correcting outputs during generation. However, modern masked discrete diffusion lacks this capability: when a token is generated, it cannot be updated again, even when it introduces an error. Here, we address this limitation by introducing the remasking diffusion model (ReMDM) sampler, a method that can be applied to pretrained masked diffusion models in a principled way and that is derived from a discrete diffusion model with a custom remasking backward process. Most interestingly, ReMDM endows discrete diffusion with a form of inference-time compute scaling. By increasing the number of sampling steps, ReMDM generates natural language outputs that approach the quality of autoregressive models, whereas when the computation budget is limited, ReMDM better maintains quality. ReMDM also improves sample quality of masked diffusion models for discretized images, and in scientific domains such as molecule design, ReMDM facilitates diffusion guidance and pushes the Pareto frontier of controllability relative to classical masking and uniform noise diffusion. We provide the code along with a blog post on the project page: this https URL.

Title: DeepONet Augmented by Randomized Neural Networks for Efficient Operator Learning in PDEs

• Authors: Zhaoxi Jiang, Fei Wang
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2503.00317
• Pdf URL: https://arxiv.org/pdf/2503.00317
• Copy Paste: [[2503.00317]] DeepONet Augmented by Randomized Neural Networks for Efficient Operator Learning in PDEs(https://arxiv.org/abs/2503.00317)
• Keywords: diffusion
• Abstract: Deep operator networks (DeepONets) represent a powerful class of data-driven methods for operator learning, demonstrating strong approximation capabilities for a wide range of linear and nonlinear operators. They have shown promising performance in learning operators that govern partial differential equations (PDEs), including diffusion-reaction systems and Burgers' equations. However, the accuracy of DeepONets is often constrained by computational limitations and optimization challenges inherent in training deep neural networks. Furthermore, the computational cost associated with training these networks is typically very high. To address these challenges, we leverage randomized neural networks (RaNNs), in which the parameters of the hidden layers remain fixed following random initialization. RaNNs compute the output layer parameters using the least-squares method, significantly reducing training time and mitigating optimization errors. In this work, we integrate DeepONets with RaNNs to propose RaNN-DeepONets, a hybrid architecture designed to balance accuracy and efficiency. Furthermore, to mitigate the need for extensive data preparation, we introduce the concept of physics-informed RaNN-DeepONets. Instead of relying on data generated through other time-consuming numerical methods, we incorporate PDE information directly into the training process. We evaluate the proposed model on three benchmark PDE problems: diffusion-reaction dynamics, Burgers' equation, and the Darcy flow problem. Through these tests, we assess its ability to learn nonlinear operators with varying input types. When compared to the standard DeepONet framework, RaNN-DeepONets achieves comparable accuracy while reducing computational costs by orders of magnitude. These results highlight the potential of RaNN-DeepONets as an efficient alternative for operator learning in PDE-based systems.

Title: FLStore: Efficient Federated Learning Storage for non-training workloads

• Authors: Ahmad Faraz Khan, Samuel Fountain, Ahmed M. Abdelmoniem, Ali R. Butt, Ali Anwar
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.00323
• Pdf URL: https://arxiv.org/pdf/2503.00323
• Copy Paste: [[2503.00323]] FLStore: Efficient Federated Learning Storage for non-training workloads(https://arxiv.org/abs/2503.00323)
• Keywords: privacy, federate
• Abstract: Federated Learning (FL) is an approach for privacy-preserving Machine Learning (ML), enabling model training across multiple clients without centralized data collection. With an aggregator server coordinating training, aggregating model updates, and storing metadata across rounds. In addition to training, a substantial part of FL systems are the non-training workloads such as scheduling, personalization, clustering, debugging, and incentivization. Most existing systems rely on the aggregator to handle non-training workloads and use cloud services for data storage. This results in high latency and increased costs as non-training workloads rely on large volumes of metadata, including weight parameters from client updates, hyperparameters, and aggregated updates across rounds, making the situation even worse. We propose FLStore, a serverless framework for efficient FL non-training workloads and storage. FLStore unifies the data and compute planes on a serverless cache, enabling locality-aware execution via tailored caching policies to reduce latency and costs. Per our evaluations, compared to cloud object store based aggregator server FLStore reduces per request average latency by 71% and costs by 92.45%, with peak improvements of 99.7% and 98.8%, respectively. Compared to an in-memory cloud cache based aggregator server, FLStore reduces average latency by 64.6% and costs by 98.83%, with peak improvements of 98.8% and 99.6%, respectively. FLStore integrates seamlessly with existing FL frameworks with minimal modifications, while also being fault-tolerant and highly scalable.

Title: DySec: A Machine Learning-based Dynamic Analysis for Detecting Malicious Packages in PyPI Ecosystem

• Authors: Sk Tanzir Mehedi, Chadni Islam, Gowri Ramachandran, Raja Jurdak
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2503.00324
• Pdf URL: https://arxiv.org/pdf/2503.00324
• Copy Paste: [[2503.00324]] DySec: A Machine Learning-based Dynamic Analysis for Detecting Malicious Packages in PyPI Ecosystem(https://arxiv.org/abs/2503.00324)
• Keywords: attack
• Abstract: Malicious Python packages make software supply chains vulnerable by exploiting trust in open-source repositories like Python Package Index (PyPI). Lack of real-time behavioral monitoring makes metadata inspection and static code analysis inadequate against advanced attack strategies such as typosquatting, covert remote access activation, and dynamic payload generation. To address these challenges, we introduce DySec, a machine learning (ML)-based dynamic analysis framework for PyPI that uses eBPF kernel and user-level probes to monitor behaviors during package installation. By capturing 36 real-time features-including system calls, network traffic, resource usage, directory access, and installation patterns-DySec detects threats like typosquatting, covert remote access activation, dynamic payload generation, and multiphase attack malware. We developed a comprehensive dataset of 14,271 Python packages, including 7,127 malicious sample traces, by executing them in a controlled isolated environment. Experimental results demonstrate that DySec achieves a 95.99\% detection accuracy with a latency of <0.5s, reducing false negatives by 78.65\% compared to static analysis and 82.24\% compared to metadata analysis. During the evaluation, DySec flagged 11 packages that PyPI classified as benign. A manual analysis, including installation behavior inspection, confirmed six of them as malicious. These findings were reported to PyPI maintainers, resulting in the removal of four packages. DySec bridges the gap between reactive traditional methods and proactive, scalable threat mitigation in open-source ecosystems by uniquely detecting malicious install-time behaviors.

Title: CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging

• Authors: Zhiwei Ling, Yachen Chang, Hailiang Zhao, Xinkui Zhao, Kingsum Chow, Shuiguang Deng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00325
• Pdf URL: https://arxiv.org/pdf/2503.00325
• Copy Paste: [[2503.00325]] CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging(https://arxiv.org/abs/2503.00325)
• Keywords: robust
• Abstract: Deep neural networks (DNNs) have been widely criticized for their overconfidence when dealing with out-of-distribution (OOD) samples, highlighting the critical need for effective OOD detection to ensure the safe deployment of DNNs in real-world settings. Existing post-hoc OOD detection methods primarily enhance the discriminative power of logit-based approaches by reshaping sample features, yet they often neglect critical information inherent in the features themselves. In this paper, we propose the Class-Aware Relative Feature-based method (CARef), which utilizes the error between a sample's feature and its class-aware average feature as a discriminative criterion. To further refine this approach, we introduce the Class-Aware Decoupled Relative Feature-based method (CADRef), which decouples sample features based on the alignment of signs between the relative feature and corresponding model weights, enhancing the discriminative capabilities of CARef. Extensive experimental results across multiple datasets and models demonstrate that both proposed methods exhibit effectiveness and robustness in OOD detection compared to state-of-the-art methods. Specifically, our two methods outperform the best baseline by 2.82% and 3.27% in AUROC, with improvements of 4.03% and 6.32% in FPR95, respectively.

Title: How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognition

• Authors: Yao Yao, Yifei Yang, Xinbei Ma, Dongjie Yang, Zhuosheng Zhang, Zuchao Li, Hai Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00330
• Pdf URL: https://arxiv.org/pdf/2503.00330
• Copy Paste: [[2503.00330]] How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognition(https://arxiv.org/abs/2503.00330)
• Keywords: large language model
• Abstract: How human cognitive abilities are formed has long captivated researchers. However, a significant challenge lies in developing meaningful methods to measure these complex processes. With the advent of large language models (LLMs), which now rival human capabilities in various domains, we are presented with a unique testbed to investigate human cognition through a new lens. Among the many facets of cognition, one particularly crucial aspect is the concept of semantic size, the perceived magnitude of both abstract and concrete words or concepts. This study seeks to investigate whether LLMs exhibit similar tendencies in understanding semantic size, thereby providing insights into the underlying mechanisms of human cognition. We begin by exploring metaphorical reasoning, comparing how LLMs and humans associate abstract words with concrete objects of varying sizes. Next, we examine LLMs' internal representations to evaluate their alignment with human cognitive processes. Our findings reveal that multi-modal training is crucial for LLMs to achieve more human-like understanding, suggesting that real-world, multi-modal experiences are similarly vital for human cognitive development. Lastly, we examine whether LLMs are influenced by attention-grabbing headlines with larger semantic sizes in a real-world web shopping scenario. The results show that multi-modal LLMs are more emotionally engaged in decision-making, but this also introduces potential biases, such as the risk of manipulation through clickbait headlines. Ultimately, this study offers a novel perspective on how LLMs interpret and internalize language, from the smallest concrete objects to the most profound abstract concepts like love. The insights gained not only improve our understanding of LLMs but also provide new avenues for exploring the cognitive abilities that define human intelligence.

Title: PINN-DT: Optimizing Energy Consumption in Smart Building Using Hybrid Physics-Informed Neural Networks and Digital Twin Framework with Blockchain Security

• Authors: Hajar Kazemi Naeini, Roya Shomali, Abolhassan Pishahang, Hamidreza Hasanzadeh, Mahdieh Mohammadi, Saeid Asadi, Ahmad Gholizadeh Lonbar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00331
• Pdf URL: https://arxiv.org/pdf/2503.00331
• Copy Paste: [[2503.00331]] PINN-DT: Optimizing Energy Consumption in Smart Building Using Hybrid Physics-Informed Neural Networks and Digital Twin Framework with Blockchain Security(https://arxiv.org/abs/2503.00331)
• Keywords: secure, security, robust, interpretability
• Abstract: The advancement of smart grid technologies necessitates the integration of cutting-edge computational methods to enhance predictive energy optimization. This study proposes a multi-faceted approach by incorporating (1) Deep Reinforcement Learning (DRL) agents trained using data from Digital Twins (DTs) to optimize energy consumption in real time, (2) Physics-Informed Neural Networks (PINNs) to seamlessly embed physical laws within the optimization process, ensuring model accuracy and interpretability, and (3) Blockchain (BC) technology to facilitate secure and transparent communication across the smart grid infrastructure. The model was trained and validated using comprehensive datasets, including smart meter energy consumption data, renewable energy outputs, dynamic pricing, and user preferences collected from IoT devices. The proposed framework achieved superior predictive performance with a Mean Absolute Error (MAE) of 0.237 kWh, Root Mean Square Error (RMSE) of 0.298 kWh, and an R-squared (R2) value of 0.978, indicating a 97.8% explanation of data variance. Classification metrics further demonstrated the model's robustness, achieving 97.7% accuracy, 97.8% precision, 97.6% recall, and an F1 Score of 97.7%. Comparative analysis with traditional models like Linear Regression, Random Forest, SVM, LSTM, and XGBoost revealed the superior accuracy and real-time adaptability of the proposed method. In addition to enhancing energy efficiency, the model reduced energy costs by 35%, maintained a 96% user comfort index, and increased renewable energy utilization to 40%. This study demonstrates the transformative potential of integrating PINNs, DT, and Blockchain technologies to optimize energy consumption in smart grids, paving the way for sustainable, secure, and efficient energy management systems.

Title: More of the Same: Persistent Representational Harms Under Increased Representation

• Authors: Jennifer Mickel, Maria De-Arteaga, Leqi Liu, Kevin Tian
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00333
• Pdf URL: https://arxiv.org/pdf/2503.00333
• Copy Paste: [[2503.00333]] More of the Same: Persistent Representational Harms Under Increased Representation(https://arxiv.org/abs/2503.00333)
• Keywords: generative, large language model
• Abstract: To recognize and mitigate the harms of generative AI systems, it is crucial to consider who is represented in the outputs of generative AI systems and how people are represented. A critical gap emerges when naively improving who is represented, as this does not imply bias mitigation efforts have been applied to address how people are represented. We critically examined this by investigating gender representation in occupation across state-of-the-art large language models. We first show evidence suggesting that over time there have been interventions to models altering the resulting gender distribution, and we find that women are more represented than men when models are prompted to generate biographies or personas. We then demonstrate that representational biases persist in how different genders are represented by examining statistically significant word differences across genders. This results in a proliferation of representational harms, stereotypes, and neoliberalism ideals that, despite existing interventions to increase female representation, reinforce existing systems of oppression.

Title: U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack

• Authors: Yunfan Gao, Yun Xiong, Wenlong Wu, Zijing Huang, Bohan Li, Haofen Wang
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.00353
• Pdf URL: https://arxiv.org/pdf/2503.00353
• Copy Paste: [[2503.00353]] U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack(https://arxiv.org/abs/2503.00353)
• Keywords: robust, large language model
• Abstract: Recent advancements in Large Language Models (LLMs) have expanded their context windows to unprecedented lengths, sparking debates about the necessity of Retrieval-Augmented Generation (RAG). To address the fragmented evaluation paradigms and limited cases in existing Needle-in-a-Haystack (NIAH), this paper introduces U-NIAH, a unified framework that systematically compares LLMs and RAG methods in controlled long context settings. Our framework extends beyond traditional NIAH by incorporating multi-needle, long-needle, and needle-in-needle configurations, along with different retrieval settings, while leveraging the synthetic Starlight Academy dataset-a fictional magical universe-to eliminate biases from pre-trained knowledge. Through extensive experiments, we investigate three research questions: (1) performance trade-offs between LLMs and RAG, (2) error patterns in RAG, and (3) RAG's limitations in complex settings. Our findings show that RAG significantly enhances smaller LLMs by mitigating the "lost-in-the-middle" effect and improving robustness, achieving an 82.58% win-rate over LLMs. However, we observe that retrieval noise and reverse chunk ordering degrade performance, while surprisingly, advanced reasoning LLMs exhibit reduced RAG compatibility due to sensitivity to semantic distractors. We identify typical error patterns including omission due to noise, hallucination under high noise critical condition, and self-doubt behaviors. Our work not only highlights the complementary roles of RAG and LLMs, but also provides actionable insights for optimizing deployments. Code: this https URL.

Title: Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Data

• Authors: Tianyi Huang, Elsa Fan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00355
• Pdf URL: https://arxiv.org/pdf/2503.00355
• Copy Paste: [[2503.00355]] Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Data(https://arxiv.org/abs/2503.00355)
• Keywords: fair, large language model
• Abstract: From disinformation spread by AI chatbots to AI recommendations that inadvertently reinforce stereotypes, textual bias poses a significant challenge to the trustworthiness of large language models (LLMs). In this paper, we propose a multi-agent framework that systematically identifies biases by disentangling each statement as fact or opinion, assigning a bias intensity score, and providing concise, factual justifications. Evaluated on 1,500 samples from the WikiNPOV dataset, the framework achieves 84.9% accuracy$\unicode{x2014}$an improvement of 13.0% over the zero-shot baseline$\unicode{x2014}$demonstrating the efficacy of explicitly modeling fact versus opinion prior to quantifying bias intensity. By combining enhanced detection accuracy with interpretable explanations, this approach sets a foundation for promoting fairness and accountability in modern language models.

Title: BERT-based model for Vietnamese Fact Verification Dataset

• Authors: Bao Tran, T. N. Khanh, Khang Nguyen Tuong, Thien Dang, Quang Nguyen, Nguyen T. Thinh, Vo T. Hung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00356
• Pdf URL: https://arxiv.org/pdf/2503.00356
• Copy Paste: [[2503.00356]] BERT-based model for Vietnamese Fact Verification Dataset(https://arxiv.org/abs/2503.00356)
• Keywords: large language model
• Abstract: The rapid advancement of information and communication technology has facilitated easier access to information. However, this progress has also necessitated more stringent verification measures to ensure the accuracy of information, particularly within the context of Vietnam. This paper introduces an approach to address the challenges of Fact Verification using the Vietnamese dataset by integrating both sentence selection and classification modules into a unified network architecture. The proposed approach leverages the power of large language models by utilizing pre-trained PhoBERT and XLM-RoBERTa as the backbone of the network. The proposed model was trained on a Vietnamese dataset, named ISE-DSC01, and demonstrated superior performance compared to the baseline model across all three metrics. Notably, we achieved a Strict Accuracy level of 75.11\%, indicating a remarkable 28.83\% improvement over the baseline model.

Title: CRUPL: A Semi-Supervised Cyber Attack Detection with Consistency Regularization and Uncertainty-aware Pseudo-Labeling in Smart Grid

• Authors: Smruti P. Dash, Kedar V. Khandeparkar, Nipun Agrawal
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00358
• Pdf URL: https://arxiv.org/pdf/2503.00358
• Copy Paste: [[2503.00358]] CRUPL: A Semi-Supervised Cyber Attack Detection with Consistency Regularization and Uncertainty-aware Pseudo-Labeling in Smart Grid(https://arxiv.org/abs/2503.00358)
• Keywords: attack
• Abstract: The modern power grids are integrated with digital technologies and automation systems. The inclusion of digital technologies has made the smart grids vulnerable to cyber-attacks. Cyberattacks on smart grids can compromise data integrity and jeopardize the reliability of the power supply. Traditional intrusion detection systems often need help to effectively detect novel and sophisticated attacks due to their reliance on labeled training data, which may only encompass part of the spectrum of potential threats. This work proposes a semi-supervised method for cyber-attack detection in smart grids by leveraging the labeled and unlabeled measurement data. We implement consistency regularization and pseudo-labeling to identify deviations from expected behavior and predict the attack classes. We use a curriculum learning approach to improve pseudo-labeling performance, capturing the model uncertainty. We demonstrate the efficiency of the proposed method in detecting different types of cyberattacks, minimizing the false positives by implementing them on publicly available datasets. The method proposes a promising solution by improving the detection accuracy to 99% in the presence of unknown samples and significantly reducing false positives.

Title: Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding

• Authors: Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, Yanning Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00361
• Pdf URL: https://arxiv.org/pdf/2503.00361
• Copy Paste: [[2503.00361]] Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding(https://arxiv.org/abs/2503.00361)
• Keywords: generative
• Abstract: Large Vision-Language Models (LVLMs) have obtained impressive performance in visual content understanding and multi-modal reasoning. Unfortunately, these large models suffer from serious hallucination problems and tend to generate fabricated responses. Recently, several Contrastive Decoding (CD) strategies have been proposed to alleviate hallucination by introducing disturbed inputs. Although great progress has been made, these CD strategies mostly apply a one-size-fits-all approach for all input conditions. In this paper, we revisit this process through extensive experiments. Related results show that hallucination causes are hybrid and each generative step faces a unique hallucination challenge. Leveraging these meaningful insights, we introduce a simple yet effective Octopus-like framework that enables the model to adaptively identify hallucination types and create a dynamic CD workflow. Our Octopus framework not only outperforms existing methods across four benchmarks but also demonstrates excellent deployability and expansibility. Code is available at this https URL.

Title: CFSum: A Transformer-Based Multi-Modal Video Summarization Framework With Coarse-Fine Fusion

• Authors: Yaowei Guo, Jiazheng Xing, Xiaojun Hou, Shuo Xin, Juntao Jiang, Demetri Terzopoulos, Chenfanfu Jiang, Yong Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00364
• Pdf URL: https://arxiv.org/pdf/2503.00364
• Copy Paste: [[2503.00364]] CFSum: A Transformer-Based Multi-Modal Video Summarization Framework With Coarse-Fine Fusion(https://arxiv.org/abs/2503.00364)
• Keywords: transformer
• Abstract: Video summarization, by selecting the most informative and/or user-relevant parts of original videos to create concise summary videos, has high research value and consumer demand in today's video proliferation era. Multi-modal video summarization that accomodates user input has become a research hotspot. However, current multi-modal video summarization methods suffer from two limitations. First, existing methods inadequately fuse information from different modalities and cannot effectively utilize modality-unique features. Second, most multi-modal methods focus on video and text modalities, neglecting the audio modality, despite the fact that audio information can be very useful in certain types of videos. In this paper we propose CFSum, a transformer-based multi-modal video summarization framework with coarse-fine fusion. CFSum exploits video, text, and audio modal features as input, and incorporates a two-stage transformer-based feature fusion framework to fully utilize modality-unique information. In the first stage, multi-modal features are fused simultaneously to perform initial coarse-grained feature fusion, then, in the second stage, video and audio features are explicitly attended with the text representation yielding more fine-grained information interaction. The CFSum architecture gives equal importance to each modality, ensuring that each modal feature interacts deeply with the other modalities. Our extensive comparative experiments against prior methods and ablation studies on various datasets confirm the effectiveness and superiority of CFSum.

Title: Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners

• Authors: David James Woo, Hengky Susanto, Chi Ho Yeung, Kai Guo, Yilin Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00367
• Pdf URL: https://arxiv.org/pdf/2503.00367
• Copy Paste: [[2503.00367]] Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners(https://arxiv.org/abs/2503.00367)
• Keywords: generative
• Abstract: Generative artificial intelligence (AI) chatbots, such as ChatGPT, are reshaping how English as a foreign language (EFL) students write since students can compose texts by integrating their own words with AI-generated text. This study investigated how 59 Hong Kong secondary school students with varying levels of academic achievement interacted with AI-generated text to compose a feature article, exploring whether any interaction patterns benefited the overall quality of the article. Through content analysis, multiple linear regression and cluster analysis, we found the overall number of words -- whether AI- or human-generated -- is the main predictor of writing quality. However, the impact varies by students' competence to write independently, for instance, by using their own words accurately and coherently to compose a text, and to follow specific interaction patterns with AI-generated text. Therefore, although composing texts with human words and AI-generated text may become prevalent in EFL writing classrooms, without educators' careful attention to EFL writing pedagogy and AI literacy, high-achieving students stand to benefit more from using AI-generated text than low-achieving students.

Title: Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis

• Authors: Xuehao Gao, Yang Yang, Shaoyi Du, Guo-Jun Qi, Junwei Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00371
• Pdf URL: https://arxiv.org/pdf/2503.00371
• Copy Paste: [[2503.00371]] Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis(https://arxiv.org/abs/2503.00371)
• Keywords: robust
• Abstract: As two intimate reciprocal tasks, scene-aware human motion synthesis and analysis require a joint understanding between multiple modalities, including 3D body motions, 3D scenes, and textual descriptions. In this paper, we integrate these two paired processes into a Co-Evolving Synthesis-Analysis (CESA) pipeline and mutually benefit their learning. Specifically, scene-aware text-to-human synthesis generates diverse indoor motion samples from the same textual description to enrich human-scene interaction intra-class diversity, thus significantly benefiting training a robust human motion analysis system. Reciprocally, human motion analysis would enforce semantic scrutiny on each synthesized motion sample to ensure its semantic consistency with the given textual description, thus improving realistic motion synthesis. Considering that real-world indoor human motions are goal-oriented and path-guided, we propose a cascaded generation strategy that factorizes text-driven scene-specific human motion generation into three stages: goal inferring, path planning, and pose synthesizing. Coupling CESA with this powerful cascaded motion synthesis model, we jointly improve realistic human motion synthesis and robust human motion analysis in 3D scenes.

Title: Few-shot crack image classification using clip based on bayesian optimization

• Authors: Yingchao Zhang, Cheng Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00376
• Pdf URL: https://arxiv.org/pdf/2503.00376
• Copy Paste: [[2503.00376]] Few-shot crack image classification using clip based on bayesian optimization(https://arxiv.org/abs/2503.00376)
• Keywords: robust, extraction
• Abstract: This study proposes a novel few-shot crack image classification model based on CLIP and Bayesian optimization. By combining multimodal information and Bayesian approach, the model achieves efficient classification of crack images in a small number of training samples. The CLIP model employs its robust feature extraction capabilities to facilitate precise classification with a limited number of samples. In contrast, Bayesian optimisation enhances the robustness and generalization of the model, while reducing the reliance on extensive labelled data. The results demonstrate that the model exhibits robust performance across a diverse range of dataset scales, particularly in the context of small sample sets. The study validates the potential of the method in civil engineering crack classification.

Title: Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach

• Authors: Guixu Lin, Muyao Niu, Qingtian Zhu, Zhengwei Yin, Zhuoxiao Li, Shengfeng He, Yinqiang Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00377
• Pdf URL: https://arxiv.org/pdf/2503.00377
• Copy Paste: [[2503.00377]] Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach(https://arxiv.org/abs/2503.00377)
• Keywords: attack, robust
• Abstract: Event cameras, known for their low latency and high dynamic range, show great potential in pedestrian detection applications. However, while recent research has primarily focused on improving detection accuracy, the robustness of event-based visual models against physical adversarial attacks has received limited attention. For example, adversarial physical objects, such as specific clothing patterns or accessories, can exploit inherent vulnerabilities in these systems, leading to misdetections or misclassifications. This study is the first to explore physical adversarial attacks on event-driven pedestrian detectors, specifically investigating whether certain clothing patterns worn by pedestrians can cause these detectors to fail, effectively rendering them unable to detect the person. To address this, we developed an end-to-end adversarial framework in the digital domain, framing the design of adversarial clothing textures as a 2D texture optimization problem. By crafting an effective adversarial loss function, the framework iteratively generates optimal textures through backpropagation. Our results demonstrate that the textures identified in the digital domain possess strong adversarial properties. Furthermore, we translated these digitally optimized textures into physical clothing and tested them in real-world scenarios, successfully demonstrating that the designed textures significantly degrade the performance of event-based pedestrian detection models. This work highlights the vulnerability of such models to physical adversarial attacks.

Title: Conditioning on Local Statistics for Scalable Heterogeneous Federated Learning

• Authors: Rickard Brännvall
• Subjects: cs.LG, cs.AI, cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.00378
• Pdf URL: https://arxiv.org/pdf/2503.00378
• Copy Paste: [[2503.00378]] Conditioning on Local Statistics for Scalable Heterogeneous Federated Learning(https://arxiv.org/abs/2503.00378)
• Keywords: privacy, federate
• Abstract: Federated learning is a distributed machine learning approach where multiple clients collaboratively train a model without sharing their local data, which contributes to preserving privacy. A challenge in federated learning is managing heterogeneous data distributions across clients, which can hinder model convergence and performance due to the need for the global model to generalize well across diverse local datasets. We propose to use local characteristic statistics, by which we mean some statistical properties calculated independently by each client using only their local training dataset. These statistics, such as means, covariances, and higher moments, are used to capture the characteristics of the local data distribution. They are not shared with other clients or a central node. During training, these local statistics help the model learn how to condition on the local data distribution, and during inference, they guide the client's predictions. Our experiments show that this approach allows for efficient handling of heterogeneous data across the federation, has favorable scaling compared to approaches that directly try to identify peer nodes that share distribution characteristics, and maintains privacy as no additional information needs to be communicated.

Title: Improving internal cluster quality evaluation in noisy Gaussian mixtures

• Authors: Renato Cordeiro de Amorim, Vladimir Makarenkov
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00379
• Pdf URL: https://arxiv.org/pdf/2503.00379
• Copy Paste: [[2503.00379]] Improving internal cluster quality evaluation in noisy Gaussian mixtures(https://arxiv.org/abs/2503.00379)
• Keywords: robust
• Abstract: Clustering is a fundamental technique in machine learning and data analysis, widely used across various domains. Internal clustering validation measures, such as the Average Silhouette Width, Calinski-Harabasz, and Davies-Bouldin indices, play a crucial role in assessing clustering quality when external ground truth labels are unavailable. However, these measures can be affected by feature relevance, potentially leading to unreliable evaluations in high-dimensional or noisy data sets. In this paper, we introduce a Feature Importance Rescaling (FIR) method designed to enhance internal clustering validation by adjusting feature contributions based on their dispersion. Our method systematically attenuates noise features making clustering compactness and separation clearer, and by consequence aligning internal validation measures more closely with the ground truth. Through extensive experiments on synthetic data sets under different configurations, we demonstrate that FIR consistently improves the correlation between internal validation indices and the ground truth, particularly in settings with noisy or irrelevant features. The results show that FIR increases the robustness of clustering evaluation, reduces variability in performance across different data sets, and remains effective even when clusters exhibit significant overlap. These findings highlight the potential of FIR as a valuable enhancement for internal clustering validation, making it a practical tool for unsupervised learning tasks where labelled data is not available.

Title: Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems

• Authors: Song Xia, Yi Yu, Wenhan Yang, Meiwen Ding, Zhuo Chen, Lingyu Duan, Alex C. Kot, Xudong Jiang
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00383
• Pdf URL: https://arxiv.org/pdf/2503.00383
• Copy Paste: [[2503.00383]] Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems(https://arxiv.org/abs/2503.00383)
• Keywords: privacy, defense, attack, robust
• Abstract: By locally encoding raw data into intermediate features, collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers. However, recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs). Obfuscation-based methods, such as noise corruption, adversarial representation learning, and information filters, enhance the inversion robustness by obfuscating the task-irrelevant redundancy empirically. However, methods for quantifying such redundancy remain elusive, and the explicit mathematical relation between this redundancy minimization and inversion robustness enhancement has not yet been established. To address that, this work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA. Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy maximization (CEM) algorithm to enhance the inversion robustness. Experimental results on four datasets demonstrate the effectiveness and adaptability of our proposed CEM; without compromising feature utility and computing efficiency, plugging the proposed CEM into obfuscation-based defense mechanisms consistently boosts their inversion robustness, achieving average gains ranging from 12.9\% to 48.2\%. Code is available at \href{this https URL}{this https URL}.

Title: A Survey of Adversarial Defenses in Vision-based Systems: Categorization, Methods and Challenges

• Authors: Nandish Chattopadhyay, Abdul Basit, Bassem Ouni, Muhammad Shafique
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00384
• Pdf URL: https://arxiv.org/pdf/2503.00384
• Copy Paste: [[2503.00384]] A Survey of Adversarial Defenses in Vision-based Systems: Categorization, Methods and Challenges(https://arxiv.org/abs/2503.00384)
• Keywords: defense, attack
• Abstract: Adversarial attacks have emerged as a major challenge to the trustworthy deployment of machine learning models, particularly in computer vision applications. These attacks have a varied level of potency and can be implemented in both white box and black box approaches. Practical attacks include methods to manipulate the physical world and enforce adversarial behaviour by the corresponding target neural network models. Multiple different approaches to mitigate different kinds of such attacks are available in the literature, each with their own advantages and limitations. In this survey, we present a comprehensive systematization of knowledge on adversarial defenses, focusing on two key computer vision tasks: image classification and object detection. We review the state-of-the-art adversarial defense techniques and categorize them for easier comparison. In addition, we provide a schematic representation of these categories within the context of the overall machine learning pipeline, facilitating clearer understanding and benchmarking of defenses. Furthermore, we map these defenses to the types of adversarial attacks and datasets where they are most effective, offering practical insights for researchers and practitioners. This study is necessary for understanding the scope of how the available defenses are able to address the adversarial threats, and their shortcomings as well, which is necessary for driving the research in this area in the most appropriate direction, with the aim of building trustworthy AI systems for regular practical use-cases.

Title: BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds

• Authors: Yuto Shibata, Yusuke Oumi, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
• Subjects: cs.CV, cs.AI, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2503.00389
• Pdf URL: https://arxiv.org/pdf/2503.00389
• Copy Paste: [[2503.00389]] BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds(https://arxiv.org/abs/2503.00389)
• Keywords: extraction
• Abstract: We propose BGM2Pose, a non-invasive 3D human pose estimation method using arbitrary music (e.g., background music) as active sensing signals. Unlike existing approaches that significantly limit practicality by employing intrusive chirp signals within the audible range, our method utilizes natural music that causes minimal discomfort to humans. Estimating human poses from standard music presents significant challenges. In contrast to sound sources specifically designed for measurement, regular music varies in both volume and pitch. These dynamic changes in signals caused by music are inevitably mixed with alterations in the sound field resulting from human motion, making it hard to extract reliable cues for pose estimation. To address these challenges, BGM2Pose introduces a Contrastive Pose Extraction Module that employs contrastive learning and hard negative sampling to eliminate musical components from the recorded data, isolating the pose information. Additionally, we propose a Frequency-wise Attention Module that enables the model to focus on subtle acoustic variations attributable to human movement by dynamically computing attention across frequency bands. Experiments suggest that our method outperforms the existing methods, demonstrating substantial potential for real-world applications. Our datasets and code will be made publicly available.

Title: Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving

• Authors: Qihui Zhou, Peiqi Yin, Pengfei Zuo, James Cheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00392
• Pdf URL: https://arxiv.org/pdf/2503.00392
• Copy Paste: [[2503.00392]] Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving(https://arxiv.org/abs/2503.00392)
• Keywords: large language model
• Abstract: Processing long contexts has become a critical capability for modern large language models (LLMs). However, serving long-context LLMs comes with significant inference costs due to the high memory overhead of the key-value (KV) cache. Existing work leverages dynamic sparse attention algorithms (DSAes) to mitigate the KV cache overhead, but these algorithms rely on top-$k$ KV cache selection, which results in a trade-off between accuracy and efficiency. A larger $k$ improves accuracy but decreases efficiency, while a smaller $k$ boosts efficiency but compromises accuracy. To overcome this trade-off, this paper presents PSA, a $\underline{P}$rogressive $\underline{S}$parse $\underline{A}$ttention mechanism that integrates algorithmic innovations with system co-design to achieve both high inference accuracy and improved efficiency in LLM serving. The PSA algorithm adaptively adjusts the KV cache budget of different tokens and layers according to their real attention weight distributions, rather than relying on a fixed budget $k$. This enables high accuracy while minimizing KV cache usage. To further enhance execution efficiency, we introduce a pipelined iteration scheme that reduces CPU-GPU interleaving and synchronization overhead during PSA computation. Additionally, we implement unified GPU memory management that optimizes PSA's memory utilization by accounting for uneven memory requirements across different model layers. Extensive experimental results demonstrate that PSA reduces KV cache usage for attention computation by up to 2.4$\times$ and 8.8$\times$, and increases end-to-end serving throughput by up to 1.4$\times$ and 2.0$\times$, compared to state-of-the-art DSAes and systems without sparse attention, respectively.

Title: Reservoir Network with Structural Plasticity for Human Activity Recognition

• Authors: Abdullah M. Zyarah, Alaa M. Abdul-Hadi, Dhireesha Kudithipudi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00393
• Pdf URL: https://arxiv.org/pdf/2503.00393
• Copy Paste: [[2503.00393]] Reservoir Network with Structural Plasticity for Human Activity Recognition(https://arxiv.org/abs/2503.00393)
• Keywords: robust
• Abstract: The unprecedented dissemination of edge devices is accompanied by a growing demand for neuromorphic chips that can process time-series data natively without cloud support. Echo state network (ESN) is a class of recurrent neural networks that can be used to identify unique patterns in time-series data and predict future events. It is known for minimal computing resource requirements and fast training, owing to the use of linear optimization solely at the readout stage. In this work, a custom-design neuromorphic chip based on ESN targeting edge devices is proposed. The proposed system supports various learning mechanisms, including structural plasticity and synaptic plasticity, locally on-chip. This provides the network with an additional degree of freedom to continuously learn, adapt, and alter its structure and sparsity level, ensuring high performance and continuous stability. We demonstrate the performance of the proposed system as well as its robustness to noise against real-world time-series datasets while considering various topologies of data movement. An average accuracy of 95.95% and 85.24% are achieved on human activity recognition and prosthetic finger control, respectively. We also illustrate that the proposed system offers a throughput of 6x10^4 samples/sec with a power consumption of 47.7mW on a 65nm IBM process.

Title: Taming Large Multimodal Agents for Ultra-low Bitrate Semantically Disentangled Image Compression

• Authors: Juan Song, Lijie Yang, Mingtao Feng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00399
• Pdf URL: https://arxiv.org/pdf/2503.00399
• Copy Paste: [[2503.00399]] Taming Large Multimodal Agents for Ultra-low Bitrate Semantically Disentangled Image Compression(https://arxiv.org/abs/2503.00399)
• Keywords: diffusion
• Abstract: It remains a significant challenge to compress images at ultra-low bitrate while achieving both semantic consistency and high perceptual quality. We propose a novel image compression framework, Semantically Disentangled Image Compression (SEDIC) in this paper. Our proposed SEDIC leverages large multimodal models (LMMs) to disentangle the image into several essential semantic information, including an extremely compressed reference image, overall and object-level text descriptions, and the semantic masks. A multi-stage semantic decoder is designed to progressively restore the transmitted reference image object-by-object, ultimately producing high-quality and perceptually consistent reconstructions. In each decoding stage, a pre-trained controllable diffusion model is utilized to restore the object details on the reference image conditioned by the text descriptions and semantic masks. Experimental results demonstrate that SEDIC significantly outperforms state-of-the-art approaches, achieving superior perceptual quality and semantic consistency at ultra-low bitrates ($\le$ 0.05 bpp). Our code is available at this https URL.

Title: Inteval Analysis for two spherical functions arising from robust Perspective-n-Lines problem

• Authors: Xiang Zheng, Haodong Jiang, Junfeng Wu
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.00400
• Pdf URL: https://arxiv.org/pdf/2503.00400
• Copy Paste: [[2503.00400]] Inteval Analysis for two spherical functions arising from robust Perspective-n-Lines problem(https://arxiv.org/abs/2503.00400)
• Keywords: robust
• Abstract: This report presents a comprehensive interval analysis of two spherical functions derived from the robust Perspective-n-Lines (PnL) problem. The study is motivated by the application of a dimension-reduction technique to achieve global solutions for the robust PnL problem. We establish rigorous theoretical results, supported by detailed proofs, and validate our findings through extensive numerical simulations.

Title: Asynchronous Personalized Federated Learning through Global Memorization

• Authors: Fan Wan, Yuchen Li, Xueqi Qiu, Rui Sun, Leyuan Zhang, Xingyu Miao, Tianyu Zhang, Haoran Duan, Yang Long
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.00407
• Pdf URL: https://arxiv.org/pdf/2503.00407
• Copy Paste: [[2503.00407]] Asynchronous Personalized Federated Learning through Global Memorization(https://arxiv.org/abs/2503.00407)
• Keywords: privacy, robust, federate
• Abstract: The proliferation of Internet of Things devices and advances in communication technology have unleashed an explosion of personal data, amplifying privacy concerns amid stringent regulations like GDPR and CCPA. Federated Learning offers a privacy preserving solution by enabling collaborative model training across decentralized devices without centralizing sensitive data. However, statistical heterogeneity from non-independent and identically distributed datasets and system heterogeneity due to client dropouts particularly those with monopolistic classes severely degrade the global model's performance. To address these challenges, we propose the Asynchronous Personalized Federated Learning framework, which empowers clients to develop personalized models using a server side semantic generator. This generator, trained via data free knowledge transfer under global model supervision, enhances client data diversity by producing both seen and unseen samples, the latter enabled by Zero-Shot Learning to mitigate dropout-induced data loss. To counter the risks of synthetic data impairing training, we introduce a decoupled model interpolation method, ensuring robust personalization. Extensive experiments demonstrate that AP FL significantly outperforms state of the art FL methods in tackling non-IID distributions and client dropouts, achieving superior accuracy and resilience across diverse real-world scenarios.

Title: CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering

• Authors: Tianyu Huai, Jie Zhou, Xingjiao Wu, Qin Chen, Qingchun Bai, Ze Zhou, Liang He
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00413
• Pdf URL: https://arxiv.org/pdf/2503.00413
• Copy Paste: [[2503.00413]] CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering(https://arxiv.org/abs/2503.00413)
• Keywords: robust, large language model
• Abstract: Multimodal large language models (MLLMs) have garnered widespread attention from researchers due to their remarkable understanding and generation capabilities in visual language tasks (e.g., visual question answering). However, the rapid pace of knowledge updates in the real world makes offline training of MLLMs costly, and when faced with non-stationary data streams, MLLMs suffer from catastrophic forgetting during learning. In this paper, we propose an MLLMs-based dual momentum Mixture-of-Experts (CL-MoE) framework for continual visual question answering (VQA). We integrate MLLMs with continual learning to utilize the rich commonsense knowledge in LLMs. We introduce a Dual-Router MoE (RMoE) strategy to select the global and local experts using task-level and instance-level routers, to robustly assign weights to the experts most appropriate for the task. Then, we design a dynamic Momentum MoE (MMoE) to update the parameters of experts dynamically based on the relationships between the experts and tasks/instances, so that the model can absorb new knowledge while maintaining existing knowledge. The extensive experimental results indicate that our method achieves state-of-the-art performance on 10 VQA tasks, proving the effectiveness of our approach.

Title: SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detection

• Authors: Xin Lin, Chong Shi, Zuopeng Yang, Haojin Tang, Zhili Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00414
• Pdf URL: https://arxiv.org/pdf/2503.00414
• Copy Paste: [[2503.00414]] SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detection(https://arxiv.org/abs/2503.00414)
• Keywords: robust, large language model
• Abstract: Recent open-vocabulary human-object interaction (OV-HOI) detection methods primarily rely on large language model (LLM) for generating auxiliary descriptions and leverage knowledge distilled from CLIP to detect unseen interaction categories. Despite their effectiveness, these methods face two challenges: (1) feature granularity deficiency, due to reliance on last layer visual features for text alignment, leading to the neglect of crucial object-level details from intermediate layers; (2) semantic similarity confusion, resulting from CLIP's inherent biases toward certain classes, while LLM-generated descriptions based solely on labels fail to adequately capture inter-class similarities. To address these challenges, we propose a stratified granular comparison network. First, we introduce a granularity sensing alignment module that aggregates global semantic features with local details, refining interaction representations and ensuring robust alignment between intermediate visual features and text embeddings. Second, we develop a hierarchical group comparison module that recursively compares and groups classes using LLMs, generating fine-grained and discriminative descriptions for each interaction category. Experimental results on two widely-used benchmark datasets, SWIG-HOI and HICO-DET, demonstrate that our method achieves state-of-the-art results in OV-HOI detection. Codes will be released on this https URL.

Title: Breaking the Loop: Detecting and Mitigating Denial-of-Service Vulnerabilities in Large Language Models

• Authors: Junzhe Yu, Yi Liu, Huijia Sun, Ling Shi, Yuqi Chen
• Subjects: cs.CR, cs.AI, cs.PF
• Abstract URL: https://arxiv.org/abs/2503.00416
• Pdf URL: https://arxiv.org/pdf/2503.00416
• Copy Paste: [[2503.00416]] Breaking the Loop: Detecting and Mitigating Denial-of-Service Vulnerabilities in Large Language Models(https://arxiv.org/abs/2503.00416)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have significantly advanced text understanding and generation, becoming integral to applications across education, software development, healthcare, entertainment, and legal services. Despite considerable progress in improving model reliability, latency remains under-explored, particularly through recurrent generation, where models repeatedly produce similar or identical outputs, causing increased latency and potential Denial-of-Service (DoS) vulnerabilities. We propose RecurrentGenerator, a black-box evolutionary algorithm that efficiently identifies recurrent generation scenarios in prominent LLMs like LLama-3 and GPT-4o. Additionally, we introduce RecurrentDetector, a lightweight real-time classifier trained on activation patterns, achieving 95.24% accuracy and an F1 score of 0.87 in detecting recurrent loops. Our methods provide practical solutions to mitigate latency-related vulnerabilities, and we publicly share our tools and data to support further research.

Title: A Multi-Labeled Dataset for Indonesian Discourse: Examining Toxicity, Polarization, and Demographics Information

• Authors: Lucky Susanto, Musa Wijanarko, Prasetia Pratama, Zilu Tang, Fariz Akyas, Traci Hong, Ika Idris, Alham Aji, Derry Wijaya
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00417
• Pdf URL: https://arxiv.org/pdf/2503.00417
• Copy Paste: [[2503.00417]] A Multi-Labeled Dataset for Indonesian Discourse: Examining Toxicity, Polarization, and Demographics Information(https://arxiv.org/abs/2503.00417)
• Keywords: large language model
• Abstract: Polarization is defined as divisive opinions held by two or more groups on substantive issues. As the world's third-largest democracy, Indonesia faces growing concerns about the interplay between political polarization and online toxicity, which is often directed at vulnerable minority groups. Despite the importance of this issue, previous NLP research has not fully explored the relationship between toxicity and polarization. To bridge this gap, we present a novel multi-label Indonesian dataset that incorporates toxicity, polarization, and annotator demographic information. Benchmarking this dataset using BERT-base models and large language models (LLMs) shows that polarization information enhances toxicity classification, and vice versa. Furthermore, providing demographic information significantly improves the performance of polarization classification.

Title: Auto-encoding Molecules: Graph-Matching Capabilities Matter

• Authors: Magnus Cunow, Gerrit Großmann
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00426
• Pdf URL: https://arxiv.org/pdf/2503.00426
• Copy Paste: [[2503.00426]] Auto-encoding Molecules: Graph-Matching Capabilities Matter(https://arxiv.org/abs/2503.00426)
• Keywords: robust, transformer, generative
• Abstract: Autoencoders are effective deep learning models that can function as generative models and learn latent representations for downstream tasks. The use of graph autoencoders - with both encoder and decoder implemented as message passing networks - is intriguing due to their ability to generate permutation-invariant graph representations. However, this approach faces difficulties because decoding a graph structure from a single vector is challenging, and comparing input and output graphs requires an effective permutation-invariant similarity measure. As a result, many studies rely on approximate methods. In this work, we explore the effect of graph matching precision on the training behavior and generation capabilities of a Variational Autoencoder (VAE). Our contribution is two-fold: (1) we propose a transformer-based message passing graph decoder as an alternative to a graph neural network decoder, that is more robust and expressive by leveraging global attention mechanisms. (2) We show that the precision of graph matching has significant impact on training behavior and is essential for effective de novo (molecular) graph generation. Code is available at this https URL

Title: DashCop: Automated E-ticket Generation for Two-Wheeler Traffic Violations Using Dashcam Videos

• Authors: Deepti Rawat, Keshav Gupta, Aryamaan Basu Roy, Ravi Kiran Sarvadevabhatla
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00428
• Pdf URL: https://arxiv.org/pdf/2503.00428
• Copy Paste: [[2503.00428]] DashCop: Automated E-ticket Generation for Two-Wheeler Traffic Violations Using Dashcam Videos(https://arxiv.org/abs/2503.00428)
• Keywords: robust, segmentation
• Abstract: Motorized two-wheelers are a prevalent and economical means of transportation, particularly in the Asia-Pacific region. However, hazardous driving practices such as triple riding and non-compliance with helmet regulations contribute significantly to accident rates. Addressing these violations through automated enforcement mechanisms can enhance traffic safety. In this paper, we propose DashCop, an end-to-end system for automated E-ticket generation. The system processes vehicle-mounted dashcam videos to detect two-wheeler traffic violations. Our contributions include: (1) a novel Segmentation and Cross-Association (SAC) module to accurately associate riders with their motorcycles, (2) a robust cross-association-based tracking algorithm optimized for the simultaneous presence of riders and motorcycles, and (3) the RideSafe-400 dataset, a comprehensive annotated dashcam video dataset for triple riding and helmet rule violations. Our system demonstrates significant improvements in violation detection, validated through extensive evaluations on the RideSafe-400 dataset.

Title: DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing

• Authors: Jingyi Yang, Xun Lin, Zitong Yu, Liepiao Zhang, Xin Liu, Hui Li, Xiaochen Yuan, Xiaochun Cao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00429
• Pdf URL: https://arxiv.org/pdf/2503.00429
• Copy Paste: [[2503.00429]] DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing(https://arxiv.org/abs/2503.00429)
• Keywords: attack, robust
• Abstract: With the availability of diverse sensor modalities (i.e., RGB, Depth, Infrared) and the success of multi-modal learning, multi-modal face anti-spoofing (FAS) has emerged as a prominent research focus. The intuition behind it is that leveraging multiple modalities can uncover more intrinsic spoofing traces. However, this approach presents more risk of misalignment. We identify two main types of misalignment: (1) \textbf{Intra-domain modality misalignment}, where the importance of each modality varies across different attacks. For instance, certain modalities (e.g., Depth) may be non-defensive against specific attacks (e.g., 3D mask), indicating that each modality has unique strengths and weaknesses in countering particular attacks. Consequently, simple fusion strategies may fall short. (2) \textbf{Inter-domain modality misalignment}, where the introduction of additional modalities exacerbates domain shifts, potentially overshadowing the benefits of complementary fusion. To tackle (1), we propose a alignment module between modalities based on mutual information, which adaptively enhances favorable modalities while suppressing unfavorable ones. To address (2), we employ a dual alignment optimization method that aligns both sub-domain hyperplanes and modality angle margins, thereby mitigating domain gaps. Our method, dubbed \textbf{D}ual \textbf{A}lignment of \textbf{D}omain and \textbf{M}odality (DADM), achieves state-of-the-art performance in extensive experiments across four challenging protocols demonstrating its robustness in multi-modal domain generalization scenarios. The codes will be released soon.

Title: AILS-NTUA at SemEval-2025 Task 8: Language-to-Code prompting and Error Fixing for Tabular Question Answering

• Authors: Andreas Evangelatos, Giorgos Filandrianos, Maria Lymperaiou, Athanasios Voulodimos, Giorgos Stamou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00435
• Pdf URL: https://arxiv.org/pdf/2503.00435
• Copy Paste: [[2503.00435]] AILS-NTUA at SemEval-2025 Task 8: Language-to-Code prompting and Error Fixing for Tabular Question Answering(https://arxiv.org/abs/2503.00435)
• Keywords: interpretability, large language model
• Abstract: In this paper, we present our submission to SemEval-2025 Task 8: Question Answering over Tabular Data. This task, evaluated on the DataBench dataset, assesses Large Language Models' (LLMs) ability to answer natural language questions over structured data while addressing topic diversity and table size limitations in previous benchmarks. We propose a system that employs effective LLM prompting to translate natural language queries into executable code, enabling accurate responses, error correction, and interpretability. Our approach ranks first in both subtasks of the competition in the proprietary model category, significantly outperforming the organizer's baseline.

Title: Split Adaptation for Pre-trained Vision Transformers

• Authors: Lixu Wang, Bingqi Shang, Yi Li, Payal Mohapatra, Wei Dong, Xiao Wang, Qi Zhu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00441
• Pdf URL: https://arxiv.org/pdf/2503.00441
• Copy Paste: [[2503.00441]] Split Adaptation for Pre-trained Vision Transformers(https://arxiv.org/abs/2503.00441)
• Keywords: privacy, protect, defense, attack, extraction, transformer
• Abstract: Vision Transformers (ViTs), extensively pre-trained on large-scale datasets, have become essential to foundation models, allowing excellent performance on diverse downstream tasks with minimal adaptation. Consequently, there is growing interest in adapting pre-trained ViTs across various fields, including privacy-sensitive domains where clients are often reluctant to share their data. Existing adaptation methods typically require direct data access, rendering them infeasible under these constraints. A straightforward solution may be sending the pre-trained ViT to clients for local adaptation, which poses issues of model intellectual property protection and incurs heavy client computation overhead. To address these issues, we propose a novel split adaptation (SA) method that enables effective downstream adaptation while protecting data and models. SA, inspired by split learning (SL), segments the pre-trained ViT into a frontend and a backend, with only the frontend shared with the client for data representation extraction. But unlike regular SL, SA replaces frontend parameters with low-bit quantized values, preventing direct exposure of the model. SA allows the client to add bi-level noise to the frontend and the extracted data representations, ensuring data protection. Accordingly, SA incorporates data-level and model-level out-of-distribution enhancements to mitigate noise injection's impact on adaptation performance. Our SA focuses on the challenging few-shot adaptation and adopts patch retrieval augmentation for overfitting alleviation. Extensive experiments on multiple datasets validate SA's superiority over state-of-the-art methods and demonstrate its defense against advanced data reconstruction attacks while preventing model leakage with minimal computation cost on the client side. The source codes can be found at this https URL.

Title: Detection of Customer Interested Garments in Surveillance Video using Computer Vision

• Authors: Earnest Paul Ijjina, Aniruddha Srinivas Joshi, Goutham Kanahasabai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00442
• Pdf URL: https://arxiv.org/pdf/2503.00442
• Copy Paste: [[2503.00442]] Detection of Customer Interested Garments in Surveillance Video using Computer Vision(https://arxiv.org/abs/2503.00442)
• Keywords: segmentation
• Abstract: One of the basic requirements of humans is clothing and this approach aims to identify the garments selected by customer during shopping, from surveillance video. The existing approaches to detect garments were developed on western wear using datasets of western clothing. They do not address Indian garments due to the increased complexity. In this work, we propose a computer vision based framework to address this problem through video surveillance. The proposed framework uses the Mixture of Gaussians background subtraction algorithm to identify the foreground present in a video frame. The visual information present in this foreground is analysed using computer vision techniques such as image segmentation to detect the various garments, the customer is interested in. The framework was tested on a dataset, that comprises of CCTV videos from a garments store. When presented with raw surveillance footage, the proposed framework demonstrated its effectiveness in detecting the interest of customer in choosing their garments by achieving a high precision and recall.

Title: Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models

• Authors: Yanyue Zhang, Yulan He, Deyu Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00449
• Pdf URL: https://arxiv.org/pdf/2503.00449
• Copy Paste: [[2503.00449]] Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models(https://arxiv.org/abs/2503.00449)
• Keywords: large language model
• Abstract: Personalized opinion summarization is crucial as it considers individual user interests while generating product summaries. Recent studies show that although large language models demonstrate powerful text summarization and evaluation capabilities without the need for training data, they face difficulties in personalized tasks involving long texts. To address this, \textbf{Rehearsal}, a personalized opinion summarization framework via LLMs-based role-playing is proposed. Having the model act as the user, the model can better understand the user's personalized needs. Additionally, a role-playing supervisor and practice process are introduced to improve the role-playing ability of the LLMs, leading to a better expression of user needs. Furthermore, through suggestions from virtual users, the summary generation is intervened, ensuring that the generated summary includes information of interest to the user, thus achieving personalized summary generation. Experiment results demonstrate that our method can effectively improve the level of personalization in large model-generated summaries.

Title: Ranking pre-trained segmentation models for zero-shot transferability

• Authors: Joshua Talks, Anna Kreshuk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00450
• Pdf URL: https://arxiv.org/pdf/2503.00450
• Copy Paste: [[2503.00450]] Ranking pre-trained segmentation models for zero-shot transferability(https://arxiv.org/abs/2503.00450)
• Keywords: segmentation
• Abstract: Model transfer presents a solution to the challenges of segmentation in the microscopy community, where the immense cost of labelling sufficient training data is a major bottleneck in the use of deep learning. With large quantities of imaging data produced across a wide range of imaging conditions, institutes also produce many bespoke models trained on specific source data which then get collected in model banks or zoos. As the number of available models grows, so does the need for an efficient and reliable model selection method for a specific target dataset of interest. We focus on the unsupervised regime where no labels are available for the target dataset. Building on previous work linking model generalisation and consistency under perturbation, we propose the first unsupervised transferability estimator for semantic and instance segmentation tasks which doesn't require access to source training data or target domain labels. We evaluate the method on multiple segmentation problems across microscopy modalities, finding a strong correlation between the rankings based on our estimator and rankings based on target dataset performance.

Title: Using Machine Learning for move sequence visualization and generation in climbing

• Authors: Thomas Rimbot, Martin Jaggi, Luis Barba
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00458
• Pdf URL: https://arxiv.org/pdf/2503.00458
• Copy Paste: [[2503.00458]] Using Machine Learning for move sequence visualization and generation in climbing(https://arxiv.org/abs/2503.00458)
• Keywords: transformer
• Abstract: In this work, we investigate the application of Machine Learning techniques to sport climbing. Expanding upon previous projects, we develop a visualization tool for move sequence evaluation on a given boulder. Then, we look into move sequence prediction from simple holds sequence information using three different Transformer models. While the results are not conclusive, they are a first step in this kind of approach and lay the ground for future work.

Title: Unstable Grounds for Beautiful Trees? Testing the Robustness of Concept Translations in the Compilation of Multilingual Wordlists

• Authors: David Snee, Luca Ciucci, Arne Rubehn, Kellen Parker van Dam, Johann-Mattis List
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00464
• Pdf URL: https://arxiv.org/pdf/2503.00464
• Copy Paste: [[2503.00464]] Unstable Grounds for Beautiful Trees? Testing the Robustness of Concept Translations in the Compilation of Multilingual Wordlists(https://arxiv.org/abs/2503.00464)
• Keywords: robust
• Abstract: Multilingual wordlists play a crucial role in comparative linguistics. While many studies have been carried out to test the power of computational methods for language subgrouping or divergence time estimation, few studies have put the data upon which these studies are based to a rigorous test. Here, we conduct a first experiment that tests the robustness of concept translation as an integral part of the compilation of multilingual wordlists. Investigating the variation in concept translations in independently compiled wordlists from 10 dataset pairs covering 9 different language families, we find that on average, only 83% of all translations yield the same word form, while identical forms in terms of phonetic transcriptions can only be found in 23% of all cases. Our findings can prove important when trying to assess the uncertainty of phylogenetic studies and the conclusions derived from them.

Title: Adaptive Rectangular Convolution for Remote Sensing Pansharpening

• Authors: Xueyang Wang, Zhixin Zheng, Jiandong Shao, Yule Duan, Liang-Jian Deng
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00467
• Pdf URL: https://arxiv.org/pdf/2503.00467
• Copy Paste: [[2503.00467]] Adaptive Rectangular Convolution for Remote Sensing Pansharpening(https://arxiv.org/abs/2503.00467)
• Keywords: extraction
• Abstract: Recent advancements in convolutional neural network (CNN)-based techniques for remote sensing pansharpening have markedly enhanced image quality. However, conventional convolutional modules in these methods have two critical drawbacks. First, the sampling positions in convolution operations are confined to a fixed square window. Second, the number of sampling points is preset and remains unchanged. Given the diverse object sizes in remote sensing images, these rigid parameters lead to suboptimal feature extraction. To overcome these limitations, we introduce an innovative convolutional module, Adaptive Rectangular Convolution (ARConv). ARConv adaptively learns both the height and width of the convolutional kernel and dynamically adjusts the number of sampling points based on the learned scale. This approach enables ARConv to effectively capture scale-specific features of various objects within an image, optimizing kernel sizes and sampling locations. Additionally, we propose ARNet, a network architecture in which ARConv is the primary convolutional module. Extensive evaluations across multiple datasets reveal the superiority of our method in enhancing pansharpening performance over previous techniques. Ablation studies and visualization further confirm the efficacy of ARConv.

Title: Rapid morphology characterization of two-dimensional TMDs and lateral heterostructures based on deep learning

• Authors: Junqi He, Yujie Zhang, Jialu Wang, Tao Wang, Pan Zhang, Chengjie Cai, Jinxing Yang, Xiao Lin, Xiaohui Yang
• Subjects: cs.LG, cond-mat.mtrl-sci, cs.CV, physics.optics
• Abstract URL: https://arxiv.org/abs/2503.00470
• Pdf URL: https://arxiv.org/pdf/2503.00470
• Copy Paste: [[2503.00470]] Rapid morphology characterization of two-dimensional TMDs and lateral heterostructures based on deep learning(https://arxiv.org/abs/2503.00470)
• Keywords: robust
• Abstract: Two-dimensional (2D) materials and heterostructures exhibit unique physical properties, necessitating efficient and accurate characterization methods. Leveraging advancements in artificial intelligence, we introduce a deep learning-based method for efficiently characterizing heterostructures and 2D materials, specifically MoS2-MoSe2 lateral heterostructures and MoS2 flakes with varying shapes and thicknesses. By utilizing YOLO models, we achieve an accuracy rate of over 94.67% in identifying these materials. Additionally, we explore the application of transfer learning across different materials, which further enhances model performance. This model exhibits robust generalization and anti-interference ability, ensuring reliable results in diverse scenarios. To facilitate practical use, we have developed an application that enables real-time analysis directly from optical microscope images, making the process significantly faster and more cost-effective than traditional methods. This deep learning-driven approach represents a promising tool for the rapid and accurate characterization of 2D materials, opening new avenues for research and development in material science.

Title: G-OSR: A Comprehensive Benchmark for Graph Open-Set Recognition

• Authors: Yicong Dong, Rundong He, Guangyao Chen, Wentao Zhang, Zhongyi Han, Jieming Shi, Yilong Yin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00476
• Pdf URL: https://arxiv.org/pdf/2503.00476
• Copy Paste: [[2503.00476]] G-OSR: A Comprehensive Benchmark for Graph Open-Set Recognition(https://arxiv.org/abs/2503.00476)
• Keywords: robust, fair
• Abstract: Graph Neural Networks (GNNs) have achieved significant success in machine learning, with wide applications in social networks, bioinformatics, knowledge graphs, and other fields. Most research assumes ideal closed-set environments. However, in real-world open-set environments, graph learning models face challenges in robustness and reliability due to unseen classes. This highlights the need for Graph Open-Set Recognition (GOSR) methods to address these issues and ensure effective GNN application in practical scenarios. Research in GOSR is in its early stages, with a lack of a comprehensive benchmark spanning diverse tasks and datasets to evaluate methods. Moreover, traditional methods, Graph Out-of-Distribution Detection (GOODD), GOSR, and Graph Anomaly Detection (GAD) have mostly evolved in isolation, with little exploration of their interconnections or potential applications to GOSR. To fill these gaps, we introduce \textbf{G-OSR}, a comprehensive benchmark for evaluating GOSR methods at both the node and graph levels, using datasets from multiple domains to ensure fair and standardized comparisons of effectiveness and efficiency across traditional, GOODD, GOSR, and GAD methods. The results offer critical insights into the generalizability and limitations of current GOSR methods and provide valuable resources for advancing research in this field through systematic analysis of diverse approaches.

Title: TSDW: A Tri-Stream Dynamic Weight Network for Cloth-Changing Person Re-Identification

• Authors: Ruiqi He, Zihan Wang, Xiang Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00477
• Pdf URL: https://arxiv.org/pdf/2503.00477
• Copy Paste: [[2503.00477]] TSDW: A Tri-Stream Dynamic Weight Network for Cloth-Changing Person Re-Identification(https://arxiv.org/abs/2503.00477)
• Keywords: security, robust, extraction
• Abstract: Cloth-Changing Person Re-identification (CC-ReID) aims to solve the challenge of identifying individuals across different temporal-spatial scenarios, viewpoints, and clothing variations. This field is gaining increasing attention in big data research and public security domains. Existing ReID research primarily relies on face recognition, gait semantic recognition, and clothing-irrelevant feature identification, which perform relatively well in scenarios with high-quality clothing change videos and images. However, these approaches depend on either single features or simple combinations of multiple features, making further performance improvements difficult. Additionally, limitations such as missing facial information, challenges in gait extraction, and inconsistent camera parameters restrict the broader application of CC-ReID. To address the above limitations, we innovatively propose a Tri-Stream Dynamic Weight Network (TSDW) that requires only images. This dynamic weighting network consists of three parallel feature streams: facial features, head-limb features, and global features. Each stream specializes in extracting its designated features, after which a gating network dynamically fuses confidence levels. The three parallel feature streams enhance recognition performance and reduce the impact of any single feature failure, thereby improving model robustness. Extensive experiments on benchmark datasets (e.g., PRCC, Celeb-reID, VC-Clothes) demonstrate that our method significantly outperforms existing state-of-the-art approaches.

Title: Towards High-fidelity 3D Talking Avatar with Personalized Dynamic Texture

• Authors: Xuanchen Li, Jianyu Wang, Yuhao Cheng, Yikun Zeng, Xingyu Ren, Wenhan Zhu, Weiming Zhao, Yichao Yan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00495
• Pdf URL: https://arxiv.org/pdf/2503.00495
• Copy Paste: [[2503.00495]] Towards High-fidelity 3D Talking Avatar with Personalized Dynamic Texture(https://arxiv.org/abs/2503.00495)
• Keywords: diffusion
• Abstract: Significant progress has been made for speech-driven 3D face animation, but most works focus on learning the motion of mesh/geometry, ignoring the impact of dynamic texture. In this work, we reveal that dynamic texture plays a key role in rendering high-fidelity talking avatars, and introduce a high-resolution 4D dataset \textbf{TexTalk4D}, consisting of 100 minutes of audio-synced scan-level meshes with detailed 8K dynamic textures from 100 subjects. Based on the dataset, we explore the inherent correlation between motion and texture, and propose a diffusion-based framework \textbf{TexTalker} to simultaneously generate facial motions and dynamic textures from speech. Furthermore, we propose a novel pivot-based style injection strategy to capture the complicity of different texture and motion styles, which allows disentangled control. TexTalker, as the first method to generate audio-synced facial motion with dynamic texture, not only outperforms the prior arts in synthesising facial motions, but also produces realistic textures that are consistent with the underlying facial movements. Project page: this https URL.

Title: Explainable LiDAR 3D Point Cloud Segmentation and Clustering for Detecting Airplane-Generated Wind Turbulence

• Authors: Zhan Qu, Shuzhou Yuan, Michael Färber, Marius Brennfleck, Niklas Wartha, Anton Stephan
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00518
• Pdf URL: https://arxiv.org/pdf/2503.00518
• Copy Paste: [[2503.00518]] Explainable LiDAR 3D Point Cloud Segmentation and Clustering for Detecting Airplane-Generated Wind Turbulence(https://arxiv.org/abs/2503.00518)
• Keywords: segmentation
• Abstract: Wake vortices - strong, coherent air turbulences created by aircraft - pose a significant risk to aviation safety and therefore require accurate and reliable detection methods. In this paper, we present an advanced, explainable machine learning method that utilizes Light Detection and Ranging (LiDAR) data for effective wake vortex detection. Our method leverages a dynamic graph CNN (DGCNN) with semantic segmentation to partition a 3D LiDAR point cloud into meaningful segments. Further refinement is achieved through clustering techniques. A novel feature of our research is the use of a perturbation-based explanation technique, which clarifies the model's decision-making processes for air traffic regulators and controllers, increasing transparency and building trust. Our experimental results, based on measured and simulated LiDAR scans compared against four baseline methods, underscore the effectiveness and reliability of our approach. This combination of semantic segmentation and clustering for real-time wake vortex tracking significantly advances aviation safety measures, ensuring that these are both effective and comprehensible.

Title: 2DMCG:2DMambawith Change Flow Guidance for Change Detection in Remote Sensing

• Authors: JunYao Kaung, HongWei Ge
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00521
• Pdf URL: https://arxiv.org/pdf/2503.00521
• Copy Paste: [[2503.00521]] 2DMCG:2DMambawith Change Flow Guidance for Change Detection in Remote Sensing(https://arxiv.org/abs/2503.00521)
• Keywords: extraction, transformer
• Abstract: Remote sensing change detection (CD) has made significant advancements with the adoption of Convolutional Neural Networks (CNNs) and Transformers. While CNNs offer powerful feature extraction, they are constrained by receptive field limitations, and Transformers suffer from quadratic complexity when processing long sequences, restricting scalability. The Mamba architecture provides an appealing alternative, offering linear complexity and high parallelism. However, its inherent 1D processing structure causes a loss of spatial information in 2D vision tasks. This paper addresses this limitation by proposing an efficient framework based on a Vision Mamba variant that enhances its ability to capture 2D spatial information while maintaining the linear complexity characteristic of Mamba. The framework employs a 2DMamba encoder to effectively learn global spatial contextual information from multi-temporal images. For feature fusion, we introduce a 2D scan-based, channel-parallel scanning strategy combined with a spatio-temporal feature fusion method, which adeptly captures both local and global change information, alleviating spatial discontinuity issues during fusion. In the decoding stage, we present a feature change flow-based decoding method that improves the mapping of feature change information from low-resolution to high-resolution feature maps, mitigating feature shift and misalignment. Extensive experiments on benchmark datasets such as LEVIR-CD+ and WHU-CD demonstrate the superior performance of our framework compared to state-of-the-art methods, showcasing the potential of Vision Mamba for efficient and accurate remote sensing change detection.

Title: Periodic Materials Generation using Text-Guided Joint Diffusion Model

• Authors: Kishalay Das, Subhojyoti Khastagir, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, Niloy Ganguly
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2503.00522
• Pdf URL: https://arxiv.org/pdf/2503.00522
• Copy Paste: [[2503.00522]] Periodic Materials Generation using Text-Guided Joint Diffusion Model(https://arxiv.org/abs/2503.00522)
• Keywords: diffusion, generative
• Abstract: Equivariant diffusion models have emerged as the prevailing approach for generating novel crystal materials due to their ability to leverage the physical symmetries of periodic material structures. However, current models do not effectively learn the joint distribution of atom types, fractional coordinates, and lattice structure of the crystal material in a cohesive end-to-end diffusion framework. Also, none of these models work under realistic setups, where users specify the desired characteristics that the generated structures must match. In this work, we introduce TGDMat, a novel text-guided diffusion model designed for 3D periodic material generation. Our approach integrates global structural knowledge through textual descriptions at each denoising step while jointly generating atom coordinates, types, and lattice structure using a periodic-E(3)-equivariant graph neural network (GNN). Extensive experiments using popular datasets on benchmark tasks reveal that TGDMat outperforms existing baseline methods by a good margin. Notably, for the structure prediction task, with just one generated sample, TGDMat outperforms all baseline models, highlighting the importance of text-guided diffusion. Further, in the generation task, TGDMat surpasses all baselines and their text-fusion variants, showcasing the effectiveness of the joint diffusion paradigm. Additionally, incorporating textual knowledge reduces overall training and sampling computational overhead while enhancing generative performance when utilizing real-world textual prompts from experts.

Title: End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler

• Authors: Denis Blessing, Xiaogang Jia, Gerhard Neumann
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00524
• Pdf URL: https://arxiv.org/pdf/2503.00524
• Copy Paste: [[2503.00524]] End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler(https://arxiv.org/abs/2503.00524)
• Keywords: diffusion
• Abstract: Diffusion models optimized via variational inference (VI) have emerged as a promising tool for generating samples from unnormalized target densities. These models create samples by simulating a stochastic differential equation, starting from a simple, tractable prior, typically a Gaussian distribution. However, when the support of this prior differs greatly from that of the target distribution, diffusion models often struggle to explore effectively or suffer from large discretization errors. Moreover, learning the prior distribution can lead to mode-collapse, exacerbated by the mode-seeking nature of reverse Kullback-Leibler divergence commonly used in VI. To address these challenges, we propose end-to-end learnable Gaussian mixture priors (GMPs). GMPs offer improved control over exploration, adaptability to target support, and increased expressiveness to counteract mode collapse. We further leverage the structure of mixture models by proposing a strategy to iteratively refine the model by adding mixture components during training. Our experimental results demonstrate significant performance improvements across a diverse range of real-world and synthetic benchmark problems when using GMPs without requiring additional target evaluations.

Title: Efficient Prompting for Continual Adaptation to Missing Modalities

• Authors: Zirun Guo, Shulei Wang, Wang Lin, Weicai Yan, Yangyang Wu, Tao Jin
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00528
• Pdf URL: https://arxiv.org/pdf/2503.00528
• Copy Paste: [[2503.00528]] Efficient Prompting for Continual Adaptation to Missing Modalities(https://arxiv.org/abs/2503.00528)
• Keywords: privacy
• Abstract: Missing modality issues are common in real-world applications, arising from factors such as equipment failures and privacy concerns. When fine-tuning pre-trained models on downstream datasets with missing modalities, performance can degrade significantly. Current methods often aggregate various missing cases to train recovery modules or align multimodal features, resulting in suboptimal performance, high computational costs, and the risk of catastrophic forgetting in continual environments where data arrives sequentially. In this paper, we formulate the dynamic missing modality problem as a continual learning task and introduce the continual multimodal missing modality task. To address this challenge efficiently, we introduce three types of prompts: modality-specific, task-aware, and task-specific prompts. These prompts enable the model to learn intra-modality, inter-modality, intra-task, and inter-task features. Furthermore, we propose a contrastive task interaction strategy to explicitly learn prompts correlating different modalities. We conduct extensive experiments on three public datasets, where our method consistently outperforms state-of-the-art approaches.

Title: GaussianSeal: Rooting Adaptive Watermarks for 3D Gaussian Generation Model

• Authors: Runyi Li, Xuanyu Zhang, Chuhan Tong, Zhipei Xu, Jian Zhang
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00531
• Pdf URL: https://arxiv.org/pdf/2503.00531
• Copy Paste: [[2503.00531]] GaussianSeal: Rooting Adaptive Watermarks for 3D Gaussian Generation Model(https://arxiv.org/abs/2503.00531)
• Keywords: protect, watermark, generative
• Abstract: With the advancement of AIGC technologies, the modalities generated by models have expanded from images and videos to 3D objects, leading to an increasing number of works focused on 3D Gaussian Splatting (3DGS) generative models. Existing research on copyright protection for generative models has primarily concentrated on watermarking in image and text modalities, with little exploration into the copyright protection of 3D object generative models. In this paper, we propose the first bit watermarking framework for 3DGS generative models, named GaussianSeal, to enable the decoding of bits as copyright identifiers from the rendered outputs of generated 3DGS. By incorporating adaptive bit modulation modules into the generative model and embedding them into the network blocks in an adaptive way, we achieve high-precision bit decoding with minimal training overhead while maintaining the fidelity of the model's outputs. Experiments demonstrate that our method outperforms post-processing watermarking approaches for 3DGS objects, achieving superior performance of watermark decoding accuracy and preserving the quality of the generated results.

Title: What Makes a Good Diffusion Planner for Decision Making?

• Authors: Haofei Lu, Dongqi Han, Yifei Shen, Dongsheng Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00535
• Pdf URL: https://arxiv.org/pdf/2503.00535
• Copy Paste: [[2503.00535]] What Makes a Good Diffusion Planner for Decision Making?(https://arxiv.org/abs/2503.00535)
• Keywords: diffusion, transformer
• Abstract: Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.

Title: Distributionally Robust Reinforcement Learning with Human Feedback

• Authors: Debmalya Mandal, Paulius Sasnauskas, Goran Radanovic
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00539
• Pdf URL: https://arxiv.org/pdf/2503.00539
• Copy Paste: [[2503.00539]] Distributionally Robust Reinforcement Learning with Human Feedback(https://arxiv.org/abs/2503.00539)
• Keywords: robust, large language model
• Abstract: Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fine-tuning large language models (LLMs). However, existing RLHF methods are non-robust, and their performance deteriorates if the downstream task differs significantly from the preference dataset used in fine-tuning. In order to mitigate this problem, we introduce a distributionally robust RLHF for fine-tuning LLMs. In particular, our goal is to ensure that a fine-tuned model retains its performance even when the distribution of prompts significantly differs from the distribution encountered during fine-tuning. We formulate distributionally robust optimization (DRO) version of two popular fine-tuning methods -- (1) reward-based RLHF and (2) reward-free DPO (direct preference optimization). We propose a minibatch gradient descent based algorithms for both of them, and theoretically prove convergence guarantees for the algorithms. Subsequently, we evaluate our algorithms on an out-of-distribution (OOD) task by first training the model on the Unified-Feedback dataset and evaluating its performance on two different datasets. The experimental results show that our robust training improves the accuracy of the learned reward models on average, and markedly on some tasks, such as reasoning. Furthermore, we show that the robust versions of policy optimization methods, similarly improve performance on OOD tasks.

Title: Streaming Video Question-Answering with In-context Video KV-Cache Retrieval

• Authors: Shangzhe Di, Zhelun Yu, Guanghao Zhang, Haoyuan Li, Tao Zhong, Hao Cheng, Bolin Li, Wanggui He, Fangxun Shu, Hao Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00540
• Pdf URL: https://arxiv.org/pdf/2503.00540
• Copy Paste: [[2503.00540]] Streaming Video Question-Answering with In-context Video KV-Cache Retrieval(https://arxiv.org/abs/2503.00540)
• Keywords: large language model
• Abstract: We propose ReKV, a novel training-free approach that enables efficient streaming video question-answering (StreamingVQA), by seamlessly integrating with existing Video Large Language Models (Video-LLMs). Traditional VideoQA systems struggle with long videos, as they must process entire videos before responding to queries, and repeat this process for each new question. In contrast, our approach analyzes long videos in a streaming manner, allowing for prompt responses as soon as user queries are received. Building on a common Video-LLM, we first incorporate a sliding-window attention mechanism, ensuring that input frames attend to a limited number of preceding frames, thereby reducing computational overhead. To prevent information loss, we store processed video key-value caches (KV-Caches) in RAM and disk, reloading them into GPU memory as needed. Additionally, we introduce a retrieval method that leverages an external retriever or the parameters within Video-LLMs to retrieve only query-relevant KV-Caches, ensuring both efficiency and accuracy in question answering. ReKV enables the separation of video encoding and question-answering across different processes and GPUs, significantly enhancing the efficiency of StreamingVQA. Through comprehensive experimentation, we validate the efficacy and practicality of our approach, which significantly boosts efficiency and enhances applicability over existing VideoQA models.

Title: Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing

• Authors: Lukas Fesser, Melanie Weber
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00547
• Pdf URL: https://arxiv.org/pdf/2503.00547
• Copy Paste: [[2503.00547]] Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing(https://arxiv.org/abs/2503.00547)
• Keywords: transformer
• Abstract: Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and regression tasks, which frequently arise in areas such as biochemistry and drug discovery. Achieving good performance in practice requires careful model design. Due to gaps in our understanding of the relationship between model and data characteristics, this often requires manual architecture and hyperparameter tuning. This is particularly pronounced in graph-level tasks, due to much higher variation in the input data than in node-level tasks. To work towards closing these gaps, we begin with a systematic analysis of individual performance in graph-level tasks. Our results establish significant performance heterogeneity in both message-passing and transformer-based architectures. We then investigate the interplay of model and data characteristics as drivers of the observed heterogeneity. Our results suggest that graph topology alone cannot explain heterogeneity. Using the Tree Mover's Distance, which jointly evaluates topological and feature information, we establish a link between class-distance ratios and performance heterogeneity in graph classification. These insights motivate model and data preprocessing choices that account for heterogeneity between graphs. We propose a selective rewiring approach, which only targets graphs whose individual performance benefits from rewiring. We further show that the optimal network depth depends on the graph's spectrum, which motivates a heuristic for choosing the number of GNN layers. Our experiments demonstrate the utility of both design choices in practice.

Title: Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable

• Authors: Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Zachary Yahn, Yichang Xu, Ling Liu
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00555
• Pdf URL: https://arxiv.org/pdf/2503.00555
• Copy Paste: [[2503.00555]] Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable(https://arxiv.org/abs/2503.00555)
• Keywords: large language model
• Abstract: Safety alignment is an important procedure before the official deployment of a Large Language Model (LLM). While safety alignment has been extensively studied for LLM, there is still a large research gap for Large Reasoning Models (LRMs) that equip with improved reasoning capability. We in this paper systematically examine a simplified pipeline for producing safety aligned LRMs. With our evaluation of various LRMs, we deliver two main findings: i) Safety alignment can be done upon the LRM to restore its safety capability. ii) Safety alignment leads to a degradation of the reasoning capability of LRMs. The two findings show that there exists a trade-off between reasoning and safety capability with the sequential LRM production pipeline. The discovered trade-off, which we name Safety Tax, should shed light on future endeavors of safety research on LRMs. As a by-product, we curate a dataset called DirectRefusal, which might serve as an alternative dataset for safety alignment. Our source code is available at this https URL.

Title: Heatwave increases nighttime light intensity in hyperdense cities of the Global South: A double machine learning study

• Authors: Ramit Debnath, Taran Chandel, Fengyuan Han, Ronita Bardhan
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.00557
• Pdf URL: https://arxiv.org/pdf/2503.00557
• Copy Paste: [[2503.00557]] Heatwave increases nighttime light intensity in hyperdense cities of the Global South: A double machine learning study(https://arxiv.org/abs/2503.00557)
• Keywords: robust
• Abstract: Heatwaves, intensified by climate change and rapid urbanisation, pose significant threats to urban systems, particularly in the Global South, where adaptive capacity is constrained. This study investigates the relationship between heatwaves and nighttime light (NTL) radiance, a proxy of nighttime economic activity, in four hyperdense cities: Delhi, Guangzhou, Cairo, and Sao Paulo. We hypothesised that heatwaves increase nighttime activity. Using a double machine learning (DML) framework, we analysed data from 2013 to 2019 to quantify the impact of heatwaves on NTL while controlling for local climatic confounders. Results revealed a statistically significant increase in NTL intensity during heatwaves, with Cairo, Delhi, and Guangzhou showing elevated NTL on the third day, while São Paulo exhibits a delayed response on the fourth day. Sensitivity analyses confirmed the robustness of these findings, indicating that prolonged heat stress prompts urban populations to shift activities to night. Heterogeneous responses across cities highlight the possible influence of urban morphology and adaptive capacity to heatwave impacts. Our findings provide a foundation for policymakers to develop data-driven heat adaptation strategies, ensuring that cities remain liveable and economically resilient in an increasingly warming world.

Title: A Guide to Failure in Machine Learning: Reliability and Robustness from Foundations to Practice

• Authors: Eric Heim, Oren Wright, David Shriver
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00563
• Pdf URL: https://arxiv.org/pdf/2503.00563
• Copy Paste: [[2503.00563]] A Guide to Failure in Machine Learning: Reliability and Robustness from Foundations to Practice(https://arxiv.org/abs/2503.00563)
• Keywords: robust
• Abstract: One of the main barriers to adoption of Machine Learning (ML) is that ML models can fail unexpectedly. In this work, we aim to provide practitioners a guide to better understand why ML models fail and equip them with techniques they can use to reason about failure. Specifically, we discuss failure as either being caused by lack of reliability or lack of robustness. Differentiating the causes of failure in this way allows us to formally define why models fail from first principles and tie these definitions to engineering concepts and real-world deployment settings. Throughout the document we provide 1) a summary of important theoretic concepts in reliability and robustness, 2) a sampling current techniques that practitioners can utilize to reason about ML model reliability and robustness, and 3) examples that show how these concepts and techniques can apply to real-world settings.

Title: Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization

• Authors: Jake B. Perazzone, Shiqiang Wang, Mingyue Ji, Kevin Chan
• Subjects: cs.LG, cs.DC, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00569
• Pdf URL: https://arxiv.org/pdf/2503.00569
• Copy Paste: [[2503.00569]] Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization(https://arxiv.org/abs/2503.00569)
• Keywords: federate
• Abstract: Federated learning (FL) is a useful tool that enables the training of machine learning models over distributed data without having to collect data centrally. When deploying FL in constrained wireless environments, however, intermittent connectivity of devices, heterogeneous connection quality, and non-i.i.d. data can severely slow convergence. In this paper, we consider FL with arbitrary device participation probabilities for each round and show that by weighing each device's update by the reciprocal of their per-round participation probability, we can guarantee convergence to a stationary point. Our bound applies to non-convex loss functions and non-i.i.d. datasets and recovers state-of-the-art convergence rates for both full and uniform partial participation, including linear speedup, with only a single-sided learning rate. Then, using the derived convergence bound, we develop a new online client selection and power allocation algorithm that utilizes the Lyapunov drift-plus-penalty framework to opportunistically minimize a function of the convergence bound and the average communication time under a transmit power constraint. We use optimization over manifold techniques to obtain a solution to the minimization problem. Thanks to the Lyapunov framework, one key feature of the algorithm is that knowledge of the channel distribution is not required and only the instantaneous channel state information needs to be known. Using the CIFAR-10 dataset with varying levels of data heterogeneity, we show through simulations that the communication time can be significantly decreased using our algorithm compared to uniformly random participation, especially for heterogeneous channel conditions.

Title: LoR2C : Low-Rank Residual Connection Adaptation for Parameter-Efficient Fine-Tuning

• Authors: Jiancheng Zhao, Xingda Yu, Yuxiang Zhang, Zhen Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00572
• Pdf URL: https://arxiv.org/pdf/2503.00572
• Copy Paste: [[2503.00572]] LoR2C : Low-Rank Residual Connection Adaptation for Parameter-Efficient Fine-Tuning(https://arxiv.org/abs/2503.00572)
• Keywords: large language model
• Abstract: In recent years, pretrained large language models have demonstrated outstanding performance across various natural language processing tasks. However, full-parameter fine-tuning methods require adjusting all model parameters, leading to immense computational resource demands. Although parameter-efficient fine-tuning methods like LoRA have significantly reduced the number of parameters, they still face challenges such as gradient vanishing and the potential for further parameter reduction. To address these issues, this paper proposes a novel parameter-efficient fine-tuning method called LoR2C (Low-Rank Residual Connection Adaptation). LoR2C introduces residual connections with low-rank matrices within the model layers, which not only reduces the number of fine-tuning parameters but also effectively alleviates the gradient vanishing problem. Additionally, this paper presents three optimization variants of LoR2C: ShareLoR2C, MergeLoR2C, and InjectLoR2C. These variants further improve parameter efficiency and model performance through parameter sharing, module merging, and injection mechanisms, respectively. Experimental results on multiple natural language understanding and natural language generation tasks demonstrate that LoR2C and its optimized variants significantly reduce parameter overhead while maintaining or even improving performance, outperforming existing mainstream parameter-efficient fine-tuning this http URL code is publicly available at this https URL.

Title: Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discovery

• Authors: Xinliang Zhou, Chenyu Liu, Zhisheng Chen, Kun Wang, Yi Ding, Ziyu Jia, Qingsong Wen
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2503.00580
• Pdf URL: https://arxiv.org/pdf/2503.00580
• Copy Paste: [[2503.00580]] Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discovery(https://arxiv.org/abs/2503.00580)
• Keywords: robust, interpretability
• Abstract: Brain foundation models (BFMs) have emerged as a transformative paradigm in computational neuroscience, offering a revolutionary framework for processing diverse neural signals across different brain-related tasks. These models leverage large-scale pre-training techniques, allowing them to generalize effectively across multiple scenarios, tasks, and modalities, thus overcoming the traditional limitations faced by conventional artificial intelligence (AI) approaches in understanding complex brain data. By tapping into the power of pretrained models, BFMs provide a means to process neural data in a more unified manner, enabling advanced analysis and discovery in the field of neuroscience. In this survey, we define BFMs for the first time, providing a clear and concise framework for constructing and utilizing these models in various applications. We also examine the key principles and methodologies for developing these models, shedding light on how they transform the landscape of neural signal processing. This survey presents a comprehensive review of the latest advancements in BFMs, covering the most recent methodological innovations, novel views of application areas, and challenges in the field. Notably, we highlight the future directions and key challenges that need to be addressed to fully realize the potential of BFMs. These challenges include improving the quality of brain data, optimizing model architecture for better generalization, increasing training efficiency, and enhancing the interpretability and robustness of BFMs in real-world applications.

Title: Secure Aggregation in Federated Learning using Multiparty Homomorphic Encryption

• Authors: Erfan Hosseini, Shuangyi Chen, Ashish Khisti
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00581
• Pdf URL: https://arxiv.org/pdf/2503.00581
• Copy Paste: [[2503.00581]] Secure Aggregation in Federated Learning using Multiparty Homomorphic Encryption(https://arxiv.org/abs/2503.00581)
• Keywords: secure, security, federate
• Abstract: A key operation in federated learning is the aggregation of gradient vectors generated by individual client nodes. We develop a method based on multiparty homomorphic encryption (MPHE) that enables the central node to compute this aggregate, while receiving only encrypted version of each individual gradients. Towards this end, we extend classical MPHE methods so that the decryption of the aggregate vector can be successful even when only a subset of client nodes are available. This is accomplished by introducing a secret-sharing step during the setup phase of MPHE when the public encryption key is generated. We develop conditions on the parameters of the MPHE scheme that guarantee correctness of decryption and (computational) security. We explain how our method can be extended to accommodate client nodes that do not participate during the setup phase. We also propose a compression scheme for gradient vectors at each client node that can be readily combined with our MPHE scheme and perform the associated convergence analysis. We discuss the advantages of our proposed scheme with other approaches based on secure multi-party computation. Finally we discuss a practical implementation of our system, compare the performance of our system with different approaches, and demonstrate that by suitably combining compression with encryption the overhead over baseline schemes is rather small.

Title: AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models

• Authors: Sohan Patnaik, Rishabh Jain, Balaji Krishnamurthy, Mausoom Sarkar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00591
• Pdf URL: https://arxiv.org/pdf/2503.00591
• Copy Paste: [[2503.00591]] AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models(https://arxiv.org/abs/2503.00591)
• Keywords: generative, large language model
• Abstract: Visual layouts are essential in graphic design fields such as advertising, posters, and web interfaces. The application of generative models for content-aware layout generation has recently gained traction. However, these models fail to understand the contextual aesthetic requirements of layout design and do not align with human-like preferences, primarily treating it as a prediction task without considering the final rendered output. To overcome these problems, we offer Aesthetic-Aware Preference Alignment(AAPA), a novel technique to train a Multi-modal Large Language Model (MLLM) for layout prediction that uses MLLM's aesthetic preferences for Direct Preference Optimization over graphic layouts. We propose a data filtering protocol utilizing our layout-quality heuristics for AAPA to ensure training happens on high-quality layouts. Additionally, we introduce a novel evaluation metric that uses another MLLM to compute the win rate of the generated layout against the ground-truth layout based on aesthetics criteria. We also demonstrate the applicability of AAPA for MLLMs of varying scales (1B to 8B parameters) and LLM families (Qwen, Phi, InternLM). By conducting thorough qualitative and quantitative analyses, we verify the efficacy of our approach on two challenging benchmarks - Crello and Webui, showcasing 17%, and 16 improvement over current State-of-The-Art methods, thereby highlighting the potential of MLLMs in aesthetic-aware layout generation.

Title: SolidMark: Evaluating Image Memorization in Generative Models

• Authors: Nicky Kriplani, Minh Pham, Gowthami Somepalli, Chinmay Hegde, Niv Cohen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00592
• Pdf URL: https://arxiv.org/pdf/2503.00592
• Copy Paste: [[2503.00592]] SolidMark: Evaluating Image Memorization in Generative Models(https://arxiv.org/abs/2503.00592)
• Keywords: diffusion, generative
• Abstract: Recent works have shown that diffusion models are able to memorize training images and emit them at generation time. However, the metrics used to evaluate memorization and its mitigation techniques suffer from dataset-dependent biases and struggle to detect whether a given specific image has been memorized or not. This paper begins with a comprehensive exploration of issues surrounding memorization metrics in diffusion models. Then, to mitigate these issues, we introduce $\rm \style{font-variant: small-caps}{SolidMark}$, a novel evaluation method that provides a per-image memorization score. We then re-evaluate existing memorization mitigation techniques. We also show that $\rm \style{font-variant: small-caps}{SolidMark}$ is capable of evaluating fine-grained pixel-level memorization. Finally, we release a variety of models based on $\rm \style{font-variant: small-caps}{SolidMark}$ to facilitate further research for understanding memorization phenomena in generative models. All of our code is available at this https URL.

Title: BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge

• Authors: Terry Tong, Fei Wang, Zhe Zhao, Muhao Chen
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00596
• Pdf URL: https://arxiv.org/pdf/2503.00596
• Copy Paste: [[2503.00596]] BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge(https://arxiv.org/abs/2503.00596)
• Keywords: attack, fair
• Abstract: This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored evaluator victimizes benign users by unfairly assigning inflated scores to adversary. A trivial single token backdoor poisoning 1% of the evaluator training data triples the adversary's score with respect to their legitimate score. We systematically categorize levels of data access corresponding to three real-world settings, (1) web poisoning, (2) malicious annotator, and (3) weight poisoning. These regimes reflect a weak to strong escalation of data access that highly correlates with attack severity. Under the weakest assumptions - web poisoning (1), the adversary still induces a 20% score inflation. Likewise, in the (3) weight poisoning regime, the stronger assumptions enable the adversary to inflate their scores from 1.5/5 to 4.9/5. The backdoor threat generalizes across different evaluator architectures, trigger designs, evaluation tasks, and poisoning rates. By poisoning 10% of the evaluator training data, we control toxicity judges (Guardrails) to misclassify toxic prompts as non-toxic 89% of the time, and document reranker judges in RAG to rank the poisoned document first 97% of the time. LLM-as-a-Judge is uniquely positioned at the intersection of ethics and technology, where social implications of mislead model selection and evaluation constrain the available defensive tools. Amidst these challenges, model merging emerges as a principled tool to offset the backdoor, reducing ASR to near 0% whilst maintaining SOTA performance. Model merging's low computational cost and convenient integration into the current LLM Judge training pipeline position it as a promising avenue for backdoor mitigation in the LLM-as-a-Judge setting.

Title: Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Models

• Authors: Jayanth Mohan, Jishnu Ray Chowdhury, Tomas Malik, Cornelia Caragea
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00597
• Pdf URL: https://arxiv.org/pdf/2503.00597
• Copy Paste: [[2503.00597]] Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Models(https://arxiv.org/abs/2503.00597)
• Keywords: large language model
• Abstract: Keyphrases are the essential topical phrases that summarize a document. Keyphrase generation is a long-standing NLP task for automatically generating keyphrases for a given document. While the task has been comprehensively explored in the past via various models, only a few works perform some preliminary analysis of Large Language Models (LLMs) for the task. Given the impact of LLMs in the field of NLP, it is important to conduct a more thorough examination of their potential for keyphrase generation. In this paper, we attempt to meet this demand with our research agenda. Specifically, we focus on the zero-shot capabilities of open-source instruction-tuned LLMs (Phi-3, Llama-3) and the closed-source GPT-4o for this task. We systematically investigate the effect of providing task-relevant specialized instructions in the prompt. Moreover, we design task-specific counterparts to self-consistency-style strategies for LLMs and show significant benefits from our proposals over the baselines.

Title: xIDS-EnsembleGuard: An Explainable Ensemble Learning-based Intrusion Detection System

• Authors: Muhammad Adil, Mian Ahmad Jan, Safayat Bin Hakim, Houbing Herbert Song, Zhanpeng Jin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00615
• Pdf URL: https://arxiv.org/pdf/2503.00615
• Copy Paste: [[2503.00615]] xIDS-EnsembleGuard: An Explainable Ensemble Learning-based Intrusion Detection System(https://arxiv.org/abs/2503.00615)
• Keywords: attack, interpretability
• Abstract: In this paper, we focus on addressing the challenges of detecting malicious attacks in networks by designing an advanced Explainable Intrusion Detection System (xIDS). The existing machine learning and deep learning approaches have invisible limitations, such as potential biases in predictions, a lack of interpretability, and the risk of overfitting to training data. These issues can create doubt about their usefulness, transparency, and a decrease in trust among stakeholders. To overcome these challenges, we propose an ensemble learning technique called "EnsembleGuard." This approach uses the predicted outputs of multiple models, including tree-based methods (LightGBM, GBM, Bagging, XGBoost, CatBoost) and deep learning models such as LSTM (long short-term memory) and GRU (gated recurrent unit), to maintain a balance and achieve trustworthy results. Our work is unique because it combines both tree-based and deep learning models to design an interpretable and explainable meta-model through model distillation. By considering the predictions of all individual models, our meta-model effectively addresses key challenges and ensures both explainable and reliable results. We evaluate our model using well-known datasets, including UNSW-NB15, NSL-KDD, and CIC-IDS-2017, to assess its reliability against various types of attacks. During analysis, we found that our model outperforms both tree-based models and other comparative approaches in different attack scenarios.

Title: An evaluation of DeepSeek Models in Biomedical Natural Language Processing

• Authors: Zaifu Zhan, Shuang Zhou, Huixue Zhou, Jiawen Deng, Yu Hou, Jeremy Yeung, Rui Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00624
• Pdf URL: https://arxiv.org/pdf/2503.00624
• Copy Paste: [[2503.00624]] An evaluation of DeepSeek Models in Biomedical Natural Language Processing(https://arxiv.org/abs/2503.00624)
• Keywords: extraction, large language model
• Abstract: The advancement of Large Language Models (LLMs) has significantly impacted biomedical Natural Language Processing (NLP), enhancing tasks such as named entity recognition, relation extraction, event extraction, and text classification. In this context, the DeepSeek series of models have shown promising potential in general NLP tasks, yet their capabilities in the biomedical domain remain underexplored. This study evaluates multiple DeepSeek models (Distilled-DeepSeek-R1 series and Deepseek-LLMs) across four key biomedical NLP tasks using 12 datasets, benchmarking them against state-of-the-art alternatives (Llama3-8B, Qwen2.5-7B, Mistral-7B, Phi-4-14B, Gemma-2-9B). Our results reveal that while DeepSeek models perform competitively in named entity recognition and text classification, challenges persist in event and relation extraction due to precision-recall trade-offs. We provide task-specific model recommendations and highlight future research directions. This evaluation underscores the strengths and limitations of DeepSeek models in biomedical NLP, guiding their future deployment and optimization.

Title: Efficiently Editing Mixture-of-Experts Models with Compressed Experts

• Authors: Yifei He, Yang Liu, Chen Liang, Hany Hassan Awadalla
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00634
• Pdf URL: https://arxiv.org/pdf/2503.00634
• Copy Paste: [[2503.00634]] Efficiently Editing Mixture-of-Experts Models with Compressed Experts(https://arxiv.org/abs/2503.00634)
• Keywords: large language model
• Abstract: Mixture-of-Experts (MoE) models have become a key approach for scaling large language models efficiently by activating only a subset of experts during training and inference. Typically, the number of activated experts presents a trade-off: fewer experts reduce computational costs, while more experts improve performance. Recent studies reveal that not all activated experts contribute equally to model performance, with some providing minimal utility, particularly when finetuning pretrained MoE models for specialized downstream tasks. The co-existence of significant and redundant parameters in experts provides us an opportunity to reduce the number of activated experts while maintaining model performance. In this work, we propose the concept of compressed experts, lightweight modules that serve as compact representations of full experts. Our approach preserves the most important experts while replacing other auxiliary activated experts with compressed experts. The reduction of active parameters significantly lowers inference costs while achieving comparable performance. Extensive experiments on models including Phi-MoE and OLMoE demonstrate that compressed experts recover over 90% of full expert performance across various tasks while reducing more than 30% active parameters and saving 20% in inference costs. This approach enables efficient deployment of MoE models in resource-constrained settings and facilitates scaling to larger models with manageable overhead. Our code is available at this https URL.

Title: POSERS: Steganography-Driven Molecular Tagging Using Randomized DNA Sequences

• Authors: Lena Hochrein, Peter Nejjar, Ali Tafazoli Yazdi
• Subjects: cs.CR, math.PR, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2503.00638
• Pdf URL: https://arxiv.org/pdf/2503.00638
• Copy Paste: [[2503.00638]] POSERS: Steganography-Driven Molecular Tagging Using Randomized DNA Sequences(https://arxiv.org/abs/2503.00638)
• Keywords: secure, security
• Abstract: Counterfeiting poses a significant challenge across multiple industries, leading to financial losses and health risks. While DNA-based molecular tagging has emerged as a promising anti-counterfeiting strategy, existing methods rely on predefined DNA sequences, making them vulnerable to replication as sequencing and synthesis technologies advance. To address these limitations, we introduce POSERS (Position-Oriented Scattering of Elements among a Randomized Sequence), a steganographic tagging system embedded within DNA sequences. POSERS ensures copy- and forgery-proof authentication by adding restrictions within randomized DNA libraries, enhancing security against counterfeiting attempts. The POSERS design allows the complexity of the libraries to be adjusted based on the customer's needs while ensuring they withstand the ongoing improvements in DNA synthesis and sequencing technologies. We mathematically validate its security properties and experimentally demonstrate its effectiveness using Next-Generation Sequencing and an authentication test, successfully distinguishing genuine POSERS tags from counterfeit ones. Our results highlight the potential of POSERS as a long-term, adaptable solution for secure product authentication.

Title: Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning

• Authors: Zijian Li, Shunxing Fan, Yujia Zheng, Ignavier Ng, Shaoan Xie, Guangyi Chen, Xinshuai Dong, Ruichu Cai, Kun Zhang
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00639
• Pdf URL: https://arxiv.org/pdf/2503.00639
• Copy Paste: [[2503.00639]] Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning(https://arxiv.org/abs/2503.00639)
• Keywords: generative
• Abstract: Disentangled representation learning aims to uncover latent variables underlying the observed data, and generally speaking, rather strong assumptions are needed to ensure identifiability. Some approaches rely on sufficient changes on the distribution of latent variables indicated by auxiliary variables such as domain indices, but acquiring enough domains is often challenging. Alternative approaches exploit structural sparsity assumptions on the mixing procedure, but such constraints are usually (partially) violated in practice. Interestingly, we find that these two seemingly unrelated assumptions can actually complement each other to achieve identifiability. Specifically, when conditioned on auxiliary variables, the sparse mixing procedure assumption provides structural constraints on the mapping from estimated to true latent variables and hence compensates for potentially insufficient distribution changes. Building on this insight, we propose an identifiability theory with less restrictive constraints regarding distribution changes and the sparse mixing procedure, enhancing applicability to real-world scenarios. Additionally, we develop an estimation framework incorporating a domain encoding network and a sparse mixing constraint and provide two implementations based on variational autoencoders and generative adversarial networks, respectively. Experiment results on synthetic and real-world datasets support our theoretical results.

Title: Deep Change Monitoring: A Hyperbolic Representative Learning Framework and a Dataset for Long-term Fine-grained Tree Change Detection

• Authors: Yante Li, Hanwen Qi, Haoyu Chen, Xinlian Liang, Guoying Zhao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00643
• Pdf URL: https://arxiv.org/pdf/2503.00643
• Copy Paste: [[2503.00643]] Deep Change Monitoring: A Hyperbolic Representative Learning Framework and a Dataset for Long-term Fine-grained Tree Change Detection(https://arxiv.org/abs/2503.00643)
• Keywords: protect, robust
• Abstract: In environmental protection, tree monitoring plays an essential role in maintaining and improving ecosystem health. However, precise monitoring is challenging because existing datasets fail to capture continuous fine-grained changes in trees due to low-resolution images and high acquisition costs. In this paper, we introduce UAVTC, a large-scale, long-term, high-resolution dataset collected using UAVs equipped with cameras, specifically designed to detect individual Tree Changes (TCs). UAVTC includes rich annotations and statistics based on biological knowledge, offering a fine-grained view for tree monitoring. To address environmental influences and effectively model the hierarchical diversity of physiological TCs, we propose a novel Hyperbolic Siamese Network (HSN) for TC detection, enabling compact and hierarchical representations of dynamic tree changes. Extensive experiments show that HSN can effectively capture complex hierarchical changes and provide a robust solution for fine-grained TC detection. In addition, HSN generalizes well to cross-domain face anti-spoofing task, highlighting its broader significance in AI. We believe our work, combining ecological insights and interdisciplinary expertise, will benefit the community by offering a new benchmark and innovative AI technologies.

Title: CATS: A framework for Cooperative Autonomy Trust & Security

• Authors: Namo Asavisanu, Tina Khezresmaeilzadeh, Rohan Sequeira, Hang Qiu, Fawad Ahmad, Konstantinos Psounis, Ramesh Govindan
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2503.00659
• Pdf URL: https://arxiv.org/pdf/2503.00659
• Copy Paste: [[2503.00659]] CATS: A framework for Cooperative Autonomy Trust & Security(https://arxiv.org/abs/2503.00659)
• Keywords: secure, security, privacy
• Abstract: With cooperative perception, autonomous vehicles can wirelessly share sensor data and representations to overcome sensor occlusions, improving situational awareness. Securing such data exchanges is crucial for connected autonomous vehicles. Existing, automated reputation-based approaches often suffer from a delay between detection and exclusion of misbehaving vehicles, while majority-based approaches have communication overheads that limits scalability. In this paper, we introduce CATS, a novel automated system that blends together the best traits of reputation-based and majority-based detection mechanisms to secure vehicle-to-everything (V2X) communications for cooperative perception, while preserving the privacy of cooperating vehicles. Our evaluation with city-scale simulations on realistic traffic data shows CATS's effectiveness in rapidly identifying and isolating misbehaving vehicles, with a low false negative rate and overheads, proving its suitability for real world deployments.

Title: The Role, Trends, and Applications of Machine Learning in Undersea Communication: A Bangladesh Perspective

• Authors: Yousuf Islam, Sumon Chandra Das, Md. Jalal Uddin Chowdhury
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00669
• Pdf URL: https://arxiv.org/pdf/2503.00669
• Copy Paste: [[2503.00669]] The Role, Trends, and Applications of Machine Learning in Undersea Communication: A Bangladesh Perspective(https://arxiv.org/abs/2503.00669)
• Keywords: security
• Abstract: The rapid evolution of machine learning (ML) has brought about groundbreaking developments in numerous industries, not the least of which is in the area of undersea communication. This domain is critical for applications like ocean exploration, environmental monitoring, resource management, and national security. Bangladesh, a maritime nation with abundant resources in the Bay of Bengal, can harness the immense potential of ML to tackle the unprecedented challenges associated with underwater communication. Beyond that, environmental conditions are unique to the region: in addition to signal attenuation, multipath propagation, noise interference, and limited bandwidth. In this study, we address the necessity to bring ML into communication via undersea; it investigates the latest technologies under the domain of ML in that respect, such as deep learning and reinforcement learning, especially concentrating on Bangladesh scenarios in the sense of implementation. This paper offers a contextualized regional perspective by incorporating region-specific needs, case studies, and recent research to propose a roadmap for deploying ML-driven solutions to improve safety at sea, promote sustainable resource use, and enhance disaster response systems. This research ultimately highlights the promise of ML-powered solutions for transforming undersea communication, leading to more efficient and cost-effective technologies that subsequently contribute to both economic growth and environmental sustainability.

Title: Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos

• Authors: Gargi V. Pillai, Ashish Verma, Debashis Sen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00670
• Pdf URL: https://arxiv.org/pdf/2503.00670
• Copy Paste: [[2503.00670]] Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos(https://arxiv.org/abs/2503.00670)
• Keywords: transformer
• Abstract: Anomaly detection in videos is a challenging task as anomalies in different videos are of different kinds. Therefore, a promising way to approach video anomaly detection is by learning the non-anomalous nature of the video at hand. To this end, we propose a one-class few-shot learning driven transformer based approach for anomaly detection in videos that is self-context aware. Features from the first few consecutive non-anomalous frames in a video are used to train the transformer in predicting the non-anomalous feature of the subsequent frame. This takes place under the attention of a self-context learned from the input features themselves. After the learning, given a few previous frames, the video-specific transformer is used to infer if a frame is anomalous or not by comparing the feature predicted by it with the actual. The effectiveness of the proposed method with respect to the state-of-the-art is demonstrated through qualitative and quantitative results on different standard datasets. We also study the positive effect of the self-context used in our approach.

Title: Dur360BEV: A Real-world Single 360-degree Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving

• Authors: Wenke E, Chao Yuan, Li Li, Yixin Sun, Yona Falinie A. Gaus, Amir Atapour-Abarghouei, Toby P. Breckon
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.00675
• Pdf URL: https://arxiv.org/pdf/2503.00675
• Copy Paste: [[2503.00675]] Dur360BEV: A Real-world Single 360-degree Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving(https://arxiv.org/abs/2503.00675)
• Keywords: segmentation
• Abstract: We present Dur360BEV, a novel spherical camera autonomous driving dataset equipped with a high-resolution 128-channel 3D LiDAR and a RTK-refined GNSS/INS system, along with a benchmark architecture designed to generate Bird-Eye-View (BEV) maps using only a single spherical camera. This dataset and benchmark address the challenges of BEV generation in autonomous driving, particularly by reducing hardware complexity through the use of a single 360-degree camera instead of multiple perspective cameras. Within our benchmark architecture, we propose a novel spherical-image-to-BEV (SI2BEV) module that leverages spherical imagery and a refined sampling strategy to project features from 2D to 3D. Our approach also includes an innovative application of Focal Loss, specifically adapted to address the extreme class imbalance often encountered in BEV segmentation tasks. Through extensive experiments, we demonstrate that this application of Focal Loss significantly improves segmentation performance on the Dur360BEV dataset. The results show that our benchmark not only simplifies the sensor setup but also achieves competitive performance.

Title: Transformer Meets Twicing: Harnessing Unattended Residual Information

• Authors: Laziz Abdullaev, Tan Nguyen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00687
• Pdf URL: https://arxiv.org/pdf/2503.00687
• Copy Paste: [[2503.00687]] Transformer Meets Twicing: Harnessing Unattended Residual Information(https://arxiv.org/abs/2503.00687)
• Keywords: robust, extraction, transformer
• Abstract: Transformer-based deep learning models have achieved state-of-the-art performance across numerous language and vision tasks. While the self-attention mechanism, a core component of transformers, has proven capable of handling complex data patterns, it has been observed that the representational capacity of the attention matrix degrades significantly across transformer layers, thereby hurting its overall performance. In this work, we leverage the connection between self-attention computations and low-pass non-local means (NLM) smoothing filters and propose the Twicing Attention, a novel attention mechanism that uses kernel twicing procedure in nonparametric regression to alleviate the low-pass behavior of associated NLM smoothing with compelling theoretical guarantees and enhanced adversarial robustness. This approach enables the extraction and reuse of meaningful information retained in the residuals following the imperfect smoothing operation at each layer. Our proposed method offers two key advantages over standard self-attention: 1) a provably slower decay of representational capacity and 2) improved robustness and accuracy across various data modalities and tasks. We empirically demonstrate the performance gains of our model over baseline transformers on multiple tasks and benchmarks, including image classification and language modeling, on both clean and corrupted data.

Title: MoSFormer: Augmenting Temporal Context with Memory of Surgery for Surgical Phase Recognition

• Authors: Hao Ding, Xu Lian, Mathias Unberath
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00695
• Pdf URL: https://arxiv.org/pdf/2503.00695
• Copy Paste: [[2503.00695]] MoSFormer: Augmenting Temporal Context with Memory of Surgery for Surgical Phase Recognition(https://arxiv.org/abs/2503.00695)
• Keywords: transformer
• Abstract: Surgical phase recognition from video enables various downstream applications. Transformer-based sliding window approaches have set the state-of-the-art by capturing rich spatial-temporal features. However, while transformers can theoretically handle arbitrary-length sequences, in practice they are limited by memory and compute constraints, resulting in fixed context windows that struggle with maintaining temporal consistency across lengthy surgical procedures. This often leads to fragmented predictions and limited procedure-level understanding. To address these challenges, we propose Memory of Surgery (MoS), a framework that enriches temporal modeling by incorporating both semantic interpretable long-term surgical history and short-term impressions. MoSFormer, our enhanced transformer architecture, integrates MoS using a carefully designed encoding and fusion mechanism. We further introduce step filtering to refine history representation and develop a memory caching pipeline to improve training and inference stability, mitigating shortcut learning and overfitting. MoSFormer demonstrates state-of-the-art performance on multiple benchmarks. On the Challenging BernBypass70 benchmark, it attains 88.0 video-level accuracy and phase-level metrics of 70.7 precision, 68.7 recall, and 66.3 F1 score, outperforming its baseline with 2.1 video-level accuracy and phase-level metrics of 4.6 precision, 3.6 recall, and 3.8 F1 score. Further studies confirms the individual and combined benefits of long-term and short-term memory components through ablation and counterfactual inference. Qualitative results shows improved temporal consistency. The augmented temporal context enables procedure-level understanding, paving the way for more comprehensive surgical video analysis.

Title: Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo

• Authors: Hyunsu Kim, Giung Nam, Chulhee Yun, Hongseok Yang, Juho Lee
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.00699
• Pdf URL: https://arxiv.org/pdf/2503.00699
• Copy Paste: [[2503.00699]] Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo(https://arxiv.org/abs/2503.00699)
• Keywords: robust
• Abstract: Bayesian Neural Networks (BNNs) provide a promising framework for modeling predictive uncertainty and enhancing out-of-distribution robustness (OOD) by estimating the posterior distribution of network parameters. Stochastic Gradient Markov Chain Monte Carlo (SGMCMC) is one of the most powerful methods for scalable posterior sampling in BNNs, achieving efficiency by combining stochastic gradient descent with second-order Langevin dynamics. However, SGMCMC often suffers from limited sample diversity in practice, which affects uncertainty estimation and model performance. We propose a simple yet effective approach to enhance sample diversity in SGMCMC without the need for tempering or running multiple chains. Our approach reparameterizes the neural network by decomposing each of its weight matrices into a product of matrices, resulting in a sampling trajectory that better explores the target parameter space. This approach produces a more diverse set of samples, allowing faster mixing within the same computational budget. Notably, our sampler achieves these improvements without increasing the inference cost compared to the standard SGMCMC. Extensive experiments on image classification tasks, including OOD robustness, diversity, loss surface analyses, and a comparative study with Hamiltonian Monte Carlo, demonstrate the superiority of the proposed approach.

Title: Towards hyperparameter-free optimization with differential privacy

• Authors: Zhiqi Bu, Ruixuan Liu
• Subjects: cs.LG, cs.CL, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00703
• Pdf URL: https://arxiv.org/pdf/2503.00703
• Copy Paste: [[2503.00703]] Towards hyperparameter-free optimization with differential privacy(https://arxiv.org/abs/2503.00703)
• Keywords: privacy, protect
• Abstract: Differential privacy (DP) is a privacy-preserving paradigm that protects the training data when training deep learning models. Critically, the performance of models is determined by the training hyperparameters, especially those of the learning rate schedule, thus requiring fine-grained hyperparameter tuning on the data. In practice, it is common to tune the learning rate hyperparameters through the grid search that (1) is computationally expensive as multiple runs are needed, and (2) increases the risk of data leakage as the selection of hyperparameters is data-dependent. In this work, we adapt the automatic learning rate schedule to DP optimization for any models and optimizers, so as to significantly mitigate or even eliminate the cost of hyperparameter tuning when applied together with automatic per-sample gradient clipping. Our hyperparameter-free DP optimization is almost as computationally efficient as the standard non-DP optimization, and achieves state-of-the-art DP performance on various language and vision tasks.

Title: Proteina: Scaling Flow-based Protein Structure Generative Models

• Authors: Tomas Geffner, Kieran Didi, Zuobai Zhang, Danny Reidenbach, Zhonglin Cao, Jason Yim, Mario Geiger, Christian Dallago, Emine Kucukbenli, Arash Vahdat, Karsten Kreis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00710
• Pdf URL: https://arxiv.org/pdf/2503.00710
• Copy Paste: [[2503.00710]] Proteina: Scaling Flow-based Protein Structure Generative Models(https://arxiv.org/abs/2503.00710)
• Keywords: diffusion, transformer, generative
• Abstract: Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale flow-based protein backbone generator that utilizes hierarchical fold class labels for conditioning and relies on a tailored scalable transformer architecture with up to 5x as many parameters as previous models. To meaningfully quantify performance, we introduce a new set of metrics that directly measure the distributional similarity of generated proteins with reference sets, complementing existing metrics. We further explore scaling training data to millions of synthetic protein structures and explore improved training and sampling recipes adapted to protein backbone generation. This includes fine-tuning strategies like LoRA for protein backbones, new guidance methods like classifier-free guidance and autoguidance for protein backbones, and new adjusted training objectives. Proteina achieves state-of-the-art performance on de novo protein backbone design and produces diverse and designable proteins at unprecedented length, up to 800 residues. The hierarchical conditioning offers novel control, enabling high-level secondary-structure guidance as well as low-level fold-specific generation.

Title: OpenECG: Benchmarking ECG Foundation Models with Public 1.2 Million Records

• Authors: Zhijiang Wan, Qianhao Yu, Jia Mao, Wenfeng Duan, Cheng Ding
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00711
• Pdf URL: https://arxiv.org/pdf/2503.00711
• Copy Paste: [[2503.00711]] OpenECG: Benchmarking ECG Foundation Models with Public 1.2 Million Records(https://arxiv.org/abs/2503.00711)
• Keywords: robust, transformer, generative
• Abstract: This study introduces OpenECG, a large-scale benchmark of 1.2 million 12-lead ECG recordings from nine centers, to evaluate ECG foundation models (ECG-FMs) trained on public datasets. We investigate three self-supervised learning methods (SimCLR, BYOL, MAE) with ResNet-50 and Vision Transformer architectures, assessing model generalization through leave-one-dataset-out experiments and data scaling analysis. Results show that pre-training on diverse datasets significantly improves generalization, with BYOL and MAE outperforming SimCLR, highlighting the efficacy of feature-consistency and generative learning over contrastive approaches. Data scaling experiments reveal that performance saturates at 60-70% of total data for BYOL and MAE, while SimCLR requires more data. These findings demonstrate that publicly available ECG data can match or surpass proprietary datasets in training robust ECG-FMs, paving the way for scalable, clinically meaningful AI-driven ECG analysis.

Title: Enhanced Security of Public Key Encryption with Certified Deletion

• Authors: Xiaogang Cheng, Ren Guo
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00719
• Pdf URL: https://arxiv.org/pdf/2503.00719
• Copy Paste: [[2503.00719]] Enhanced Security of Public Key Encryption with Certified Deletion(https://arxiv.org/abs/2503.00719)
• Keywords: security
• Abstract: In classical cryptography, certified deletion is simply impossible. Since classical information can be copied any number of times easily. In quantum cryptography, certified deletion is possible because of theorems of quantum mechanics such as the quantum no-clone theorem, quantum superposition etc. In this paper, we show the PKE-CD (Public Key Encryption with Certified Deletion) scheme constructed in by Bartusek and Khurana in CRYPTO 2023 lack an important security property, which is important in practical applications. Then we show how to enhance this property, and construct a concrete scheme with this property. And we also discuss the relations between PKE-CD and other quantum cryptographic schemes such as quantum seal, quantum bit commitment etc.

Title: Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategies

• Authors: Tianyi Huang, Jingyuan Yi, Peiyang Yu, Xiaochuan Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00724
• Pdf URL: https://arxiv.org/pdf/2503.00724
• Copy Paste: [[2503.00724]] Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategies(https://arxiv.org/abs/2503.00724)
• Keywords: attack, robust, federate, explainability, large language model
• Abstract: The proliferation of misinformation on social media has raised significant societal concerns, necessitating robust detection mechanisms. Large Language Models such as GPT-4 and LLaMA2 have been envisioned as possible tools for detecting misinformation based on their advanced natural language understanding and reasoning capabilities. This paper conducts a comparison of LLM-based approaches to detecting misinformation between text-based, multimodal, and agentic approaches. We evaluate the effectiveness of fine-tuned models, zero-shot learning, and systematic fact-checking mechanisms in detecting misinformation across different topic domains like public health, politics, and finance. We also discuss scalability, generalizability, and explainability of the models and recognize key challenges such as hallucination, adversarial attacks on misinformation, and computational resources. Our findings point towards the importance of hybrid approaches that pair structured verification protocols with adaptive learning techniques to enhance detection accuracy and explainability. The paper closes by suggesting potential avenues of future work, including real-time tracking of misinformation, federated learning, and cross-platform detection models.

Title: Shazam: Unifying Multiple Foundation Models for Advanced Computational Pathology

• Authors: Wenhui Lei, Anqi Li, Yusheng Tan, Hanyu Chen, Xiaofan Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00736
• Pdf URL: https://arxiv.org/pdf/2503.00736
• Copy Paste: [[2503.00736]] Shazam: Unifying Multiple Foundation Models for Advanced Computational Pathology(https://arxiv.org/abs/2503.00736)
• Keywords: extraction
• Abstract: Foundation Models (FMs) in computational pathology (CPath) have significantly advanced the extraction of meaningful features from histopathology image datasets, achieving strong performance across various clinical tasks. Despite their impressive performance, these models often exhibit variability when applied to different tasks, prompting the need for a unified framework capable of consistently excelling across various applications. In this work, we propose Shazam, a novel framework designed to efficiently combine multiple CPath models. Unlike previous approaches that train a fixed-parameter FM, Shazam dynamically extracts and refines information from diverse FMs for each specific task. To ensure that each FM contributes effectively without dominance, a novel distillation strategy is applied, guiding the student model with features from all teacher models, which enhances its generalization ability. Experimental results on two pathology patch classification datasets demonstrate that Shazam outperforms existing CPath models and other fusion methods. Its lightweight, flexible design makes it a promising solution for improving CPath analysis in real-world settings. Code will be available at this https URL.

Title: FaceShot: Bring Any Character into Life

• Authors: Junyao Gao, Yanan Sun, Fei Shen, Xin Jiang, Zhening Xing, Kai Chen, Cairong Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00740
• Pdf URL: https://arxiv.org/pdf/2503.00740
• Copy Paste: [[2503.00740]] FaceShot: Bring Any Character into Life(https://arxiv.org/abs/2503.00740)
• Keywords: robust, diffusion
• Abstract: In this paper, we present FaceShot, a novel training-free portrait animation framework designed to bring any character into life from any driven video without fine-tuning or retraining. We achieve this by offering precise and robust reposed landmark sequences from an appearance-guided landmark matching module and a coordinate-based landmark retargeting module. Together, these components harness the robust semantic correspondences of latent diffusion models to produce facial motion sequence across a wide range of character types. After that, we input the landmark sequences into a pre-trained landmark-driven animation model to generate animated video. With this powerful generalization capability, FaceShot can significantly extend the application of portrait animation by breaking the limitation of realistic portrait landmark detection for any stylized character and driven video. Also, FaceShot is compatible with any landmark-driven animation model, significantly improving overall performance. Extensive experiments on our newly constructed character benchmark CharacBench confirm that FaceShot consistently surpasses state-of-the-art (SOTA) approaches across any character domain. More results are available at our project website this https URL.

Title: Revolutionizing Healthcare Record Management: Secure Documentation Storage and Access through Advanced Blockchain Solutions

• Authors: Geeta N. Brijwani, Prafulla E Ajmire, Mohammad Atique Mohammad Junaid, Suhashini Awadhesh Charasia, Deepali Bhende
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00742
• Pdf URL: https://arxiv.org/pdf/2503.00742
• Copy Paste: [[2503.00742]] Revolutionizing Healthcare Record Management: Secure Documentation Storage and Access through Advanced Blockchain Solutions(https://arxiv.org/abs/2503.00742)
• Keywords: secure, security, protect, robust
• Abstract: Integrating blockchain technology into healthcare systems presents a transformative approach to documenting, storing, and accessing electronic health records (EHRs). This research introduces a novel blockchain-based EHR system designed to significantly enhance security, scalability, and accessibility compared to existing solutions. Current systems primarily utilize SHA-256 for security and either IPFS or centralized storage, which, while effective, have limitations in providing comprehensive data integrity and security. The proposed system leverages a hybrid security algorithm combining Argon2 and AES and integrates a hybrid storage and consensus mechanism utilizing IPFS and PBFT. This multifaceted approach ensures robust encryption, efficient consensus, and high fault tolerance. Furthermore, the system incorporates Multi-Factor Authentication (MFA) to safeguard against unauthorized access. It utilizes advanced blockchain tools like MetaMask, Ganache, and Truffle to facilitate seamless interaction with the decentralized network. Simulation results demonstrate that this system offers superior protection against data breaches and enhances operational efficiency. Specifically, the proposed hybrid model substantially improves data integrity, consensus efficiency, fault tolerance, data availability, latency, bandwidth utilization, throughput, memory usage, and CPU usage across various healthcare applications. To validate the performance and security of the proposed system, comprehensive analyses were conducted using real-world healthcare scenarios. The findings highlight the significant advantages of the blockchain-based EHR system, emphasizing its potential to revolutionize healthcare data management by ensuring secure, reliable, and efficient handling of sensitive medical information.

Title: Confounder-Aware Medical Data Selection for Fine-Tuning Pretrained Vision Models

• Authors: Anyang Ji, Qingbo Kang, Wei Xu, Changfan Wang, Kang Li, Qicheng Lao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00744
• Pdf URL: https://arxiv.org/pdf/2503.00744
• Copy Paste: [[2503.00744]] Confounder-Aware Medical Data Selection for Fine-Tuning Pretrained Vision Models(https://arxiv.org/abs/2503.00744)
• Keywords: privacy
• Abstract: The emergence of large-scale pre-trained vision foundation models has greatly advanced the medical imaging field through the pre-training and fine-tuning paradigm. However, selecting appropriate medical data for downstream fine-tuning remains a significant challenge considering its annotation cost, privacy concerns, and the detrimental effects of confounding variables. In this work, we present a confounder-aware medical data selection approach for medical dataset curation aiming to select minimal representative data by strategically mitigating the undesirable impact of confounding variables while preserving the natural distribution of the dataset. Our approach first identifies confounding variables within data and then develops a distance-based data selection strategy for confounder-aware sampling with a constrained budget in the data size. We validate the superiority of our approach through extensive experiments across diverse medical imaging modalities, highlighting its effectiveness in addressing the substantial impact of confounding variables and enhancing the fine-tuning efficiency in the medical imaging domain, compared to other data selection approaches.

Title: Unifying Light Field Perception with Field of Parallax

• Authors: Fei Teng, Buyin Deng, Boyuan Zheng, Kai Luo, Kunyu Peng, Jiaming Zhang, Kailun Yang
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00747
• Pdf URL: https://arxiv.org/pdf/2503.00747
• Copy Paste: [[2503.00747]] Unifying Light Field Perception with Field of Parallax(https://arxiv.org/abs/2503.00747)
• Keywords: robust, segmentation
• Abstract: Field of Parallax (FoP)}, a spatial field that distills the common features from different LF representations to provide flexible and consistent support for multi-task learning. FoP is built upon three core features--projection difference, adjacency divergence, and contextual consistency--which are essential for cross-task adaptability. To implement FoP, we design a two-step angular adapter: the first step captures angular-specific differences, while the second step consolidates contextual consistency to ensure robust representation. Leveraging the FoP-based representation, we introduce the LFX framework, the first to handle arbitrary LF representations seamlessly, unifying LF multi-task vision. We evaluated LFX across three different tasks, achieving new state-of-the-art results, compared with previous task-specific architectures: 84.74% in mIoU for semantic segmentation on UrbanLF, 0.84% in AP for object detection on PKU, and 0.030 in MAE and 0.026 in MAE for salient object detection on Duftv2 and PKU, respectively. The source code will be made publicly available at this https URL.

Title: Dynamic Gradient Sparsification Training for Few-Shot Fine-tuning of CT Lymph Node Segmentation Foundation Model

• Authors: Zihao Luo, Zijun Gao, Wenjun Liao, Shichuan Zhang, Guotai Wang, Xiangde Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00748
• Pdf URL: https://arxiv.org/pdf/2503.00748
• Copy Paste: [[2503.00748]] Dynamic Gradient Sparsification Training for Few-Shot Fine-tuning of CT Lymph Node Segmentation Foundation Model(https://arxiv.org/abs/2503.00748)
• Keywords: robust, segmentation
• Abstract: Accurate lymph node (LN) segmentation is critical in radiotherapy treatment and prognosis analysis, but is limited by the need for large annotated datasets. While deep learning-based segmentation foundation models show potential in developing high-performing models with fewer samples, their medical adaptation faces LN domain-specific prior deficiencies and inefficient few-shot fine-tuning for complex clinical practices, highlighting the necessity of an LN segmentation foundation model. In this work, we annotated 36,106 visible LNs from 3,346 publicly available head-and-neck CT scans to establish a robust LN segmentation model (nnUNetv2). Building on this, we propose Dynamic Gradient Sparsification Training (DGST), a few-shot fine-tuning approach that preserves foundational knowledge while dynamically updating the most critical parameters of the LN segmentation model with few annotations. We validate it on two publicly available LN segmentation datasets: SegRap2023 and LNQ2023. The results show that DGST outperforms existing few-shot fine-tuning methods, achieving satisfactory performance with limited labeled data. We release the dataset, models and all implementations to facilitate relevant research: this https URL.

Title: RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery

• Authors: Hongchao Gu, Dexun Li, Kuicai Dong, Hao Zhang, Hang Lv, Hao Wang, Defu Lian, Yong Liu, Enhong Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00751
• Pdf URL: https://arxiv.org/pdf/2503.00751
• Copy Paste: [[2503.00751]] RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery(https://arxiv.org/abs/2503.00751)
• Keywords: robust, large language model
• Abstract: Generating knowledge-intensive and comprehensive long texts, such as encyclopedia articles, remains significant challenges for Large Language Models. It requires not only the precise integration of facts but also the maintenance of thematic coherence throughout the article. Existing methods, such as direct generation and multi-agent discussion, often struggle with issues like hallucinations, topic incoherence, and significant latency. To address these challenges, we propose RAPID, an efficient retrieval-augmented long text generation framework. RAPID consists of three main modules: (1) Retrieval-augmented preliminary outline generation to reduce hallucinations, (2) Attribute-constrained search for efficient information discovery, (3) Plan-guided article generation for enhanced coherence. Extensive experiments on our newly compiled benchmark dataset, FreshWiki-2024, demonstrate that RAPID significantly outperforms state-of-the-art methods across a wide range of evaluation metrics (e.g. long-text generation, outline quality, latency, etc). Our work provides a robust and efficient solution to the challenges of automated long-text generation.

Title: MR-EIT: Multi-Resolution Reconstruction for Electrical Impedance Tomography via Data-Driven and Unsupervised Dual-Mode Neural Networks

• Authors: Fangming Shi, Jinzhen Liu, Xiangqian Meng, Yapeng Zhou, Hui Xiong
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.00762
• Pdf URL: https://arxiv.org/pdf/2503.00762
• Copy Paste: [[2503.00762]] MR-EIT: Multi-Resolution Reconstruction for Electrical Impedance Tomography via Data-Driven and Unsupervised Dual-Mode Neural Networks(https://arxiv.org/abs/2503.00762)
• Keywords: robust, extraction
• Abstract: This paper presents a multi-resolution reconstruction method for Electrical Impedance Tomography (EIT), referred to as MR-EIT, which is capable of operating in both supervised and unsupervised learning modes. MR-EIT integrates an ordered feature extraction module and an unordered coordinate feature expression module. The former achieves the mapping from voltage to two-dimensional conductivity features through pre-training, while the latter realizes multi-resolution reconstruction independent of the order and size of the input sequence by utilizing symmetric functions and local feature extraction mechanisms. In the data-driven mode, MR-EIT reconstructs high-resolution images from low-resolution data of finite element meshes through two stages of pre-training and joint training, and demonstrates excellent performance in simulation experiments. In the unsupervised learning mode, MR-EIT does not require pre-training data and performs iterative optimization solely based on measured voltages to rapidly achieve image reconstruction from low to high resolution. It shows robustness to noise and efficient super-resolution reconstruction capabilities in both simulation and real water tank experiments. Experimental results indicate that MR-EIT outperforms the comparison methods in terms of Structural Similarity (SSIM) and Relative Image Error (RIE), especially in the unsupervised learning mode, where it can significantly reduce the number of iterations and improve image reconstruction quality.

Title: Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity

• Authors: Yupu Hao, Pengfei Cao, Zhuoran Jin, Huanxuan Liao, Yubo Chen, Kang Liu, Jun Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00771
• Pdf URL: https://arxiv.org/pdf/2503.00771
• Copy Paste: [[2503.00771]] Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity(https://arxiv.org/abs/2503.00771)
• Keywords: large language model
• Abstract: Personalized tool utilization is essential for aligning large language models (LLMs) with user preference in interaction scenarios with various tools. However, most of the current benchmarks primarily focus on either personalization of text generation or direct tool-utilizing, without considering both. In this work, we introduce a novel benchmark ETAPP for evaluating personalized tool invocation, establishing a sandbox environment, and a comprehensive dataset of 800 testing cases covering diverse user profiles. To improve the accuracy of our evaluation, we propose a key-point-based LLM evaluation method, mitigating biases in the LLM-as-a-judge system by manually annotating key points for each test case and providing them to LLM as the reference. Additionally, we evaluate the excellent LLMs and provide an in-depth analysis. Furthermore, we investigate the impact of different tool-invoking strategies on LLMs' personalization performance and the effects of fine-tuning in our task. The effectiveness of our preference-setting and key-point-based evaluation method is also validated. Our findings offer insights into improving personalized LLM agents. Our Code is available at this https URL.

Title: Enhanced Multi-Class Classification of Gastrointestinal Endoscopic Images with Interpretable Deep Learning Model

• Authors: Astitva Kamble, Vani Bandodkar, Saakshi Dharmadhikary, Veena Anand, Pradyut Kumar Sanki, Mei X. Wu, Biswabandhu Jana
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00780
• Pdf URL: https://arxiv.org/pdf/2503.00780
• Copy Paste: [[2503.00780]] Enhanced Multi-Class Classification of Gastrointestinal Endoscopic Images with Interpretable Deep Learning Model(https://arxiv.org/abs/2503.00780)
• Keywords: interpretability
• Abstract: Endoscopy serves as an essential procedure for evaluating the gastrointestinal (GI) tract and plays a pivotal role in identifying GI-related disorders. Recent advancements in deep learning have demonstrated substantial progress in detecting abnormalities through intricate models and data augmentation this http URL research introduces a novel approach to enhance classification accuracy using 8,000 labeled endoscopic images from the Kvasir dataset, categorized into eight distinct classes. Leveraging EfficientNetB3 as the backbone, the proposed architecture eliminates reliance on data augmentation while preserving moderate model complexity. The model achieves a test accuracy of 94.25%, alongside precision and recall of 94.29% and 94.24% respectively. Furthermore, Local Interpretable Model-agnostic Explanation (LIME) saliency maps are employed to enhance interpretability by defining critical regions in the images that influenced model predictions. Overall, this work highlights the importance of AI in advancing medical imaging by combining high classification accuracy with interpretability.

Title: DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting

• Authors: Kai Lv, Honglin Guo, Qipeng Guo, Xipeng Qiu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00784
• Pdf URL: https://arxiv.org/pdf/2503.00784
• Copy Paste: [[2503.00784]] DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting(https://arxiv.org/abs/2503.00784)
• Keywords: large language model
• Abstract: Large language models (LLMs) exhibit exceptional performance across a wide range of tasks; however, their token-by-token autoregressive generation process significantly hinders inference speed. Speculative decoding presents a promising draft-then-verify framework that reduces generation latency while maintaining output distribution fidelity. Nevertheless, the draft model introduces additional computational overhead, becoming a performance bottleneck and increasing the time to first token (TTFT). Previous approaches to mitigate draft model overhead have primarily relied on heuristics and generally failed to match the quality of the draft language models. To address these challenges, we propose DuoDecoding, a novel approach that strategically deploys the draft and target models on the CPU and GPU respectively, enabling parallel decoding while preserving draft quality. Our method incorporates a hardware-aware optimal draft budget to minimize idle times and employs dynamic multi-sequence drafting to enhance draft quality. Extensive experiments across seven tasks show that DuoDecoding achieves up to 2.61x speedup in generation latency, while reducing TTFT to 83% of that in conventional speculative decoding. The Code is available at this https URL.

Title: Graph Attention Networks Unleashed: A Fast and Explainable Vulnerability Assessment Framework for Microgrids

• Authors: Wei Liu, Tao Zhang, Chenhui Lin, Kaiwen Li, Rui Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00786
• Pdf URL: https://arxiv.org/pdf/2503.00786
• Copy Paste: [[2503.00786]] Graph Attention Networks Unleashed: A Fast and Explainable Vulnerability Assessment Framework for Microgrids(https://arxiv.org/abs/2503.00786)
• Keywords: attack, explainability
• Abstract: Independent microgrids are crucial for supplying electricity by combining distributed energy resources and loads in scenarios like isolated islands and field combat. Fast and accurate assessments of microgrid vulnerability against intentional attacks or natural disasters are essential for effective risk prevention and design optimization. However, conventional Monte Carlo simulation (MCS) methods are computationally expensive and time-consuming, while existing machine learning-based approaches often lack accuracy and explainability. To address these challenges, this study proposes a fast and explainable vulnerability assessment framework that integrates MCS with a graph attention network enhanced by self-attention pooling (GAT-S). MCS generates training data, while the GAT-S model learns the structural and electrical characteristics of the microgrid and further assesses its vulnerability intelligently. The GAT-S improves explainability and computational efficiency by dynamically assigning attention weights to critical nodes. Comprehensive experimental evaluations across various microgrid configurations demonstrate that the proposed framework provides accurate vulnerability assessments, achieving a mean squared error as low as 0.001, real-time responsiveness within 1 second, and delivering explainable results.

Title: Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning

• Authors: Ukcheol Shin, Kyunghyun Lee, Jean Oh
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.00793
• Pdf URL: https://arxiv.org/pdf/2503.00793
• Copy Paste: [[2503.00793]] Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning(https://arxiv.org/abs/2503.00793)
• Keywords: robust
• Abstract: Deploying depth estimation networks in the real world requires high-level robustness against various adverse conditions to ensure safe and reliable autonomy. For this purpose, many autonomous vehicles employ multi-modal sensor systems, including an RGB camera, NIR camera, thermal camera, LiDAR, or Radar. They mainly adopt two strategies to use multiple sensors: modality-wise and multi-modal fused inference. The former method is flexible but memory-inefficient, unreliable, and vulnerable. Multi-modal fusion can provide high-level reliability, yet it needs a specialized architecture. In this paper, we propose an effective solution, named align-and-fuse strategy, for the depth estimation from multi-spectral images. In the align stage, we align embedding spaces between multiple spectrum bands to learn shareable representation across multi-spectral images by minimizing contrastive loss of global and spatially aligned local features with geometry cue. After that, in the fuse stage, we train an attachable feature fusion module that can selectively aggregate the multi-spectral features for reliable and robust prediction results. Based on the proposed method, a single-depth network can achieve both spectral-invariant and multi-spectral fused depth estimation while preserving reliability, memory efficiency, and flexibility.

Title: STAR-Edge: Structure-aware Local Spherical Curve Representation for Thin-walled Edge Extraction from Unstructured Point Clouds

• Authors: Zikuan Li, Honghua Chen, Yuecheng Wang, Sibo Wu, Mingqiang Wei, Jun Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00801
• Pdf URL: https://arxiv.org/pdf/2503.00801
• Copy Paste: [[2503.00801]] STAR-Edge: Structure-aware Local Spherical Curve Representation for Thin-walled Edge Extraction from Unstructured Point Clouds(https://arxiv.org/abs/2503.00801)
• Keywords: robust, extraction
• Abstract: Extracting geometric edges from unstructured point clouds remains a significant challenge, particularly in thin-walled structures that are commonly found in everyday objects. Traditional geometric methods and recent learning-based approaches frequently struggle with these structures, as both rely heavily on sufficient contextual information from local point neighborhoods. However, 3D measurement data of thin-walled structures often lack the accurate, dense, and regular neighborhood sampling required for reliable edge extraction, resulting in degraded performance. In this work, we introduce STAR-Edge, a novel approach designed for detecting and refining edge points in thin-walled structures. Our method leverages a unique representation-the local spherical curve-to create structure-aware neighborhoods that emphasize co-planar points while reducing interference from close-by, non-co-planar surfaces. This representation is transformed into a rotation-invariant descriptor, which, combined with a lightweight multi-layer perceptron, enables robust edge point classification even in the presence of noise and sparse or irregular sampling. Besides, we also use the local spherical curve representation to estimate more precise normals and introduce an optimization function to project initially identified edge points exactly on the true edges. Experiments conducted on the ABC dataset and thin-walled structure-specific datasets demonstrate that STAR-Edge outperforms existing edge detection methods, showcasing better robustness under various challenging conditions.

Title: MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Models

• Authors: Jia-Xuan Jiang, Wenhui Lei, Yifeng Wu, Hongtao Wu, Furong Li, Yining Xie, Xiaofan Zhang, Zhong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00802
• Pdf URL: https://arxiv.org/pdf/2503.00802
• Copy Paste: [[2503.00802]] MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Models(https://arxiv.org/abs/2503.00802)
• Keywords: diffusion, segmentation
• Abstract: Medical Foundation Models (MFMs), trained on large-scale datasets, have demonstrated superior performance across various tasks. However, these models still struggle with domain gaps in practical applications. Specifically, even after fine-tuning on source-domain data, task-adapted foundation models often perform poorly in the target domain. To address this challenge, we propose a few-shot unsupervised domain adaptation (UDA) framework for MFMs, named MFM-DA, which only leverages a limited number of unlabeled target-domain images. Our approach begins by training a Denoising Diffusion Probabilistic Model (DDPM), which is then adapted to the target domain using a proposed dynamic instance-aware adaptor and a distribution direction loss, enabling the DDPM to translate source-domain images into the target domain style. The adapted images are subsequently processed through the MFM, where we introduce a designed channel-spatial alignment Low-Rank Adaptation (LoRA) to ensure effective feature alignment. Extensive experiments on optic cup and disc segmentation tasks demonstrate that MFM-DA outperforms state-of-the-art methods. Our work provides a practical solution to the domain gap issue in real-world MFM deployment. Code will be available at here.

Title: Evaluating and Predicting Distorted Human Body Parts for Generated Images

• Authors: Lu Ma, Kaibo Cao, Hao Liang, Jiaxin Lin, Zhuang Li, Yuhong Liu, Jihong Zhang, Wentao Zhang, Bin Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00811
• Pdf URL: https://arxiv.org/pdf/2503.00811
• Copy Paste: [[2503.00811]] Evaluating and Predicting Distorted Human Body Parts for Generated Images(https://arxiv.org/abs/2503.00811)
• Keywords: transformer, segmentation
• Abstract: Recent advancements in text-to-image (T2I) models enable high-quality image synthesis, yet generating anatomically accurate human figures remains challenging. AI-generated images frequently exhibit distortions such as proliferated limbs, missing fingers, deformed extremities, or fused body parts. Existing evaluation metrics like Inception Score (IS) and Fréchet Inception Distance (FID) lack the granularity to detect these distortions, while human preference-based metrics focus on abstract quality assessments rather than anatomical fidelity. To address this gap, we establish the first standards for identifying human body distortions in AI-generated images and introduce Distortion-5K, a comprehensive dataset comprising 4,700 annotated images of normal and malformed human figures across diverse styles and distortion types. Based on this dataset, we propose ViT-HD, a Vision Transformer-based model tailored for detecting human body distortions in AI-generated images, which outperforms state-of-the-art segmentation models and visual language models, achieving an F1 score of 0.899 and IoU of 0.831 on distortion localization. Additionally, we construct the Human Distortion Benchmark with 500 human-centric prompts to evaluate four popular T2I models using trained ViT-HD, revealing that nearly 50\% of generated images contain distortions. This work pioneers a systematic approach to evaluating anatomical accuracy in AI-generated humans, offering tools to advance the fidelity of T2I models and their real-world applicability. The Distortion-5K dataset, trained ViT-HD will soon be released in our GitHub repository: \href{this https URL}{this https URL}.

Title: Training-Free Dataset Pruning for Instance Segmentation

• Authors: Yalun Dai, Lingao Xiao, Ivor W. Tsang, Yang He
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00828
• Pdf URL: https://arxiv.org/pdf/2503.00828
• Copy Paste: [[2503.00828]] Training-Free Dataset Pruning for Instance Segmentation(https://arxiv.org/abs/2503.00828)
• Keywords: transformer, segmentation
• Abstract: Existing dataset pruning techniques primarily focus on classification tasks, limiting their applicability to more complex and practical tasks like instance segmentation. Instance segmentation presents three key challenges: pixel-level annotations, instance area variations, and class imbalances, which significantly complicate dataset pruning efforts. Directly adapting existing classification-based pruning methods proves ineffective due to their reliance on time-consuming model training process. To address this, we propose a novel Training-Free Dataset Pruning (TFDP) method for instance segmentation. Specifically, we leverage shape and class information from image annotations to design a Shape Complexity Score (SCS), refining it into a Scale-Invariant (SI-SCS) and Class-Balanced (CB-SCS) versions to address instance area variations and class imbalances, all without requiring model training. We achieve state-of-the-art results on VOC 2012, Cityscapes, and COCO datasets, generalizing well across CNN and Transformer architectures. Remarkably, our approach accelerates the pruning process by an average of 1349$\times$ on COCO compared to the adapted baselines. Source code is available at: this https URL

Title: Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models

• Authors: Jeffrey Gu, Serena Yeung-Levy
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00838
• Pdf URL: https://arxiv.org/pdf/2503.00838
• Copy Paste: [[2503.00838]] Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models(https://arxiv.org/abs/2503.00838)
• Keywords: transformer
• Abstract: Large pre-trained models, or foundation models, have shown impressive performance when adapted to a variety of downstream tasks, often out-performing specialized models. Hypernetworks, neural networks that generate some or all of the parameters of another neural network, have become an increasingly important technique for conditioning and generalizing implicit neural representations (INRs), which represent signals or objects such as audio or 3D shapes using a neural network. However, despite the potential benefits of incorporating foundation models in hypernetwork methods, this research direction has not been investigated, likely due to the dissimilarity of the weight generation task with other visual tasks. To address this gap, we (1) show how foundation models can improve hypernetworks with Transformer-based architectures, (2) provide an empirical analysis of the benefits of foundation models for hypernetworks through the lens of the generalizable INR task, showing that leveraging foundation models improves performance, generalizability, and data efficiency across a variety of algorithms and modalities. We also provide further analysis in examining the design space of foundation model-based hypernetworks, including examining the choice of foundation models, algorithms, and the effect of scaling foundation models.

Title: Rewarding Graph Reasoning Process makes LLMs more Generalized Reasoners

• Authors: Miao Peng, Nuo Chen, Zongrui Suo, Jia Li
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00845
• Pdf URL: https://arxiv.org/pdf/2503.00845
• Copy Paste: [[2503.00845]] Rewarding Graph Reasoning Process makes LLMs more Generalized Reasoners(https://arxiv.org/abs/2503.00845)
• Keywords: large language model
• Abstract: Despite significant advancements in Large Language Models (LLMs), developing advanced reasoning capabilities in LLMs remains a key challenge. Process Reward Models (PRMs) have demonstrated exceptional promise in enhancing reasoning by providing step-wise feedback, particularly in the context of mathematical reasoning. However, their application to broader reasoning domains remains understudied, largely due to the high costs associated with manually creating step-level supervision. In this work, we explore the potential of PRMs in graph reasoning problems - a domain that demands sophisticated multi-step reasoning and offers opportunities for automated step-level data generation using established graph algorithms. We introduce GraphSILO, the largest dataset for graph reasoning problems with fine-grained step-wise labels, built using automated Task-oriented Trajectories and Monte Carlo Tree Search (MCTS) to generate detailed reasoning steps with step-wise labels. Building upon this dataset, we train GraphPRM, the first PRM designed for graph reasoning problems, and evaluate its effectiveness in two key settings: inference-time scaling and reinforcement learning via Direct Preference Optimization (DPO). Experimental results show that GraphPRM significantly improves LLM performance across 13 graph reasoning tasks, delivering a 9% gain for Qwen2.5-7B and demonstrating transferability to new graph reasoning datasets and new reasoning domains like mathematical problem-solving. Notably, GraphPRM enhances LLM performance on GSM8K and Math500, underscoring the cross-domain applicability of graph-based reasoning rewards. Our findings highlight the potential of PRMs in advancing reasoning across diverse domains, paving the way for more versatile and effective LLMs.

Title: Argument Summarization and its Evaluation in the Era of Large Language Models

• Authors: Moritz Altemeyer, Steffen Eger, Johannes Daxenberger, Tim Altendorf, Philipp Cimiano, Benjamin Schiller
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00847
• Pdf URL: https://arxiv.org/pdf/2503.00847
• Copy Paste: [[2503.00847]] Argument Summarization and its Evaluation in the Era of Large Language Models(https://arxiv.org/abs/2503.00847)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have revolutionized various Natural Language Generation (NLG) tasks, including Argument Summarization (ArgSum), a key subfield of Argument Mining (AM). This paper investigates the integration of state-of-the-art LLMs into ArgSum, including for its evaluation. In particular, we propose a novel prompt-based evaluation scheme, and validate it through a novel human benchmark dataset. Our work makes three main contributions: (i) the integration of LLMs into existing ArgSum frameworks, (ii) the development of a new LLM-based ArgSum system, benchmarked against prior methods, and (iii) the introduction of an advanced LLM-based evaluation scheme. We demonstrate that the use of LLMs substantially improves both the generation and evaluation of argument summaries, achieving state-of-the-art results and advancing the field of ArgSum.

Title: MTReD: 3D Reconstruction Dataset for Fly-over Videos of Maritime Domain

• Authors: Rui Yi Yong, Samuel Picosson, Arnold Wiliem
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00853
• Pdf URL: https://arxiv.org/pdf/2503.00853
• Copy Paste: [[2503.00853]] MTReD: 3D Reconstruction Dataset for Fly-over Videos of Maritime Domain(https://arxiv.org/abs/2503.00853)
• Keywords: segmentation
• Abstract: This work tackles 3D scene reconstruction for a video fly-over perspective problem in the maritime domain, with a specific emphasis on geometrically and visually sound reconstructions. This will allow for downstream tasks such as segmentation, navigation, and localization. To our knowledge, there is no dataset available in this domain. As such, we propose a novel maritime 3D scene reconstruction benchmarking dataset, named as MTReD (Maritime Three-Dimensional Reconstruction Dataset). The MTReD comprises 19 fly-over videos curated from the Internet containing ships, islands, and coastlines. As the task is aimed towards geometrical consistency and visual completeness, the dataset uses two metrics: (1) Reprojection error; and (2) Perception based metrics. We find that existing perception-based metrics, such as Learned Perceptual Image Patch Similarity (LPIPS), do not appropriately measure the completeness of a reconstructed image. Thus, we propose a novel semantic similarity metric utilizing DINOv2 features coined DiFPS (DinoV2 Features Perception Similarity). We perform initial evaluation on two baselines: (1) Structured from Motion (SfM) through Colmap; and (2) the recent state-of-the-art MASt3R model. We find that the reconstructed scenes by MASt3R have higher reprojection errors, but superior perception based metric scores. To this end, some pre-processing methods are explored, and we find a pre-processing method which improves both the reprojection error and perception-based score. We envisage our proposed MTReD to stimulate further research in these directions. The dataset and all the code will be made available in this https URL.

Title: FACROC: a fairness measure for FAir Clustering through ROC curves

• Authors: Tai Le Quy, Long Le Thanh, Lan Luong Thi Hong, Frank Hopfgartner
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00854
• Pdf URL: https://arxiv.org/pdf/2503.00854
• Copy Paste: [[2503.00854]] FACROC: a fairness measure for FAir Clustering through ROC curves(https://arxiv.org/abs/2503.00854)
• Keywords: protect, fair
• Abstract: Fair clustering has attracted remarkable attention from the research community. Many fairness measures for clustering have been proposed; however, they do not take into account the clustering quality w.r.t. the values of the protected attribute. In this paper, we introduce a new visual-based fairness measure for fair clustering through ROC curves, namely FACROC. This fairness measure employs AUCC as a measure of clustering quality and then computes the difference in the corresponding ROC curves for each value of the protected attribute. Experimental results on several popular datasets for fairness-aware machine learning and well-known (fair) clustering models show that FACROC is a beneficial method for visually evaluating the fairness of clustering models.

Title: Zero-Shot Head Swapping in Real-World Scenarios

• Authors: Sohyun Jeong, Taewoong Kang, Hyojin Jang, Jaegul Choo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00861
• Pdf URL: https://arxiv.org/pdf/2503.00861
• Copy Paste: [[2503.00861]] Zero-Shot Head Swapping in Real-World Scenarios(https://arxiv.org/abs/2503.00861)
• Keywords: robust
• Abstract: With growing demand in media and social networks for personalized images, the need for advanced head-swapping techniques, integrating an entire head from the head image with the body from the body image, has increased. However, traditional head swapping methods heavily rely on face-centered cropped data with primarily frontal facing views, which limits their effectiveness in real world applications. Additionally, their masking methods, designed to indicate regions requiring editing, are optimized for these types of dataset but struggle to achieve seamless blending in complex situations, such as when the original data includes features like long hair extending beyond the masked area. To overcome these limitations and enhance adaptability in diverse and complex scenarios, we propose a novel head swapping method, HID, that is robust to images including the full head and the upper body, and handles from frontal to side views, while automatically generating context aware masks. For automatic mask generation, we introduce the IOMask, which enables seamless blending of the head and body, effectively addressing integration challenges. We further introduce the hair injection module to capture hair details with greater precision. Our experiments demonstrate that the proposed approach achieves state-of-the-art performance in head swapping, providing visually consistent and realistic results across a wide range of challenging conditions.

Title: Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers

• Authors: Yiran Zhao, Chaoqun Liu, Yue Deng, Jiahao Ying, Mahani Aljunied, Zhaodonghui Li, Lidong Bing, Hou Pong Chan, Yu Rong, Deli Zhao, Wenxuan Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00865
• Pdf URL: https://arxiv.org/pdf/2503.00865
• Copy Paste: [[2503.00865]] Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers(https://arxiv.org/abs/2503.00865)
• Keywords: large language model
• Abstract: Large language models (LLMs) have revolutionized natural language processing (NLP), yet open-source multilingual LLMs remain scarce, with existing models often limited in language coverage. Such models typically prioritize well-resourced languages, while widely spoken but under-resourced languages are often overlooked. To address this disparity, we introduce $\texttt{Babel}$, an open multilingual LLM that covers the top 25 languages by number of speakers, supports over 90% of the global population, and includes many languages neglected by other open multilingual LLMs. Unlike traditional continue pretraining approaches, Babel expands its parameter count through a layer extension technique that elevates Babel's performance ceiling. We introduce two variants: $\texttt{Babel-9B}$, designed for efficient inference and fine-tuning, and $\texttt{Babel-83B}$, which sets a new standard for open multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its superior performance compared to open LLMs of comparable size. In addition, using open-source supervised fine-tuning datasets, Babel achieves remarkable performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat setting a new standard for multilingual tasks, reaching the same level of commercial models.

Title: DUAL: Diversity and Uncertainty Active Learning for Text Summarization

• Authors: Petros Stylianos Giouroukis, Alexios Gidiotis, Grigorios Tsoumakas
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00867
• Pdf URL: https://arxiv.org/pdf/2503.00867
• Copy Paste: [[2503.00867]] DUAL: Diversity and Uncertainty Active Learning for Text Summarization(https://arxiv.org/abs/2503.00867)
• Keywords: robust, large language model
• Abstract: With the rise of large language models, neural text summarization has advanced significantly in recent years. However, even state-of-the-art models continue to rely heavily on high-quality human-annotated data for training and evaluation. Active learning is frequently used as an effective way to collect such datasets, especially when annotation resources are scarce. Active learning methods typically prioritize either uncertainty or diversity but have shown limited effectiveness in summarization, often being outperformed by random sampling. We present Diversity and Uncertainty Active Learning (DUAL), a novel algorithm that combines uncertainty and diversity to iteratively select and annotate samples that are both representative of the data distribution and challenging for the current model. DUAL addresses the selection of noisy samples in uncertainty-based methods and the limited exploration scope of diversity-based methods. Through extensive experiments with different summarization models and benchmark datasets, we demonstrate that DUAL consistently matches or outperforms the best performing strategies. Using visualizations and quantitative metrics, we provide valuable insights into the effectiveness and robustness of different active learning strategies, in an attempt to understand why these strategies haven't performed consistently in text summarization. Finally, we show that DUAL strikes a good balance between diversity and robustness.

Title: CyberCScope: Mining Skewed Tensor Streams and Online Anomaly Detection in Cybersecurity Systems

• Authors: Kota Nakamura, Koki Kawabata, Shungo Tanaka, Yasuko Matsubara, Yasushi Sakurai
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00871
• Pdf URL: https://arxiv.org/pdf/2503.00871
• Copy Paste: [[2503.00871]] CyberCScope: Mining Skewed Tensor Streams and Online Anomaly Detection in Cybersecurity Systems(https://arxiv.org/abs/2503.00871)
• Keywords: security
• Abstract: Cybersecurity systems are continuously producing a huge number of time-stamped events in the form of high-order tensors, such as {count; time, port, flow duration, packet size, . . . }, and so how can we detect anomalies/intrusions in real time? How can we identify multiple types of intrusions and capture their characteristic behaviors? The tensor data consists of categorical and continuous attributes and the data distributions of continuous attributes typically exhibit skew. These data properties require handling skewed infinite and finite dimensional spaces simultaneously. In this paper, we propose a novel streaming method, namely CyberCScope. The method effectively decomposes incoming tensors into major trends while explicitly distinguishing between categorical and skewed continuous attributes. To our knowledge, it is the first to compute hybrid skewed infinite and finite dimensional decomposition. Based on this decomposition, it streamingly finds distinct time-evolving patterns, enabling the detection of multiple types of anomalies. Extensive experiments on large-scale real datasets demonstrate that CyberCScope detects various intrusions with higher accuracy than state-of-the-art baselines while providing meaningful summaries for the intrusions that occur in practice.

Title: Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Model

• Authors: Rundong He, Yicong Dong, Lanzhe Guo, Yilong Yin, Tailin Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00884
• Pdf URL: https://arxiv.org/pdf/2503.00884
• Copy Paste: [[2503.00884]] Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Model(https://arxiv.org/abs/2503.00884)
• Keywords: robust
• Abstract: Semi-supervised learning (SSL) effectively leverages unlabeled data and has been proven successful across various fields. Current safe SSL methods believe that unseen classes in unlabeled data harm the performance of SSL models. However, previous methods for assessing the impact of unseen classes on SSL model performance are flawed. They fix the size of the unlabeled dataset and adjust the proportion of unseen classes within the unlabeled data to assess the impact. This process contravenes the principle of controlling variables. Adjusting the proportion of unseen classes in unlabeled data alters the proportion of seen classes, meaning the decreased classification performance of seen classes may not be due to an increase in unseen class samples in the unlabeled data, but rather a decrease in seen class samples. Thus, the prior flawed assessment standard that ``unseen classes in unlabeled data can damage SSL model performance" may not always hold true. This paper strictly adheres to the principle of controlling variables, maintaining the proportion of seen classes in unlabeled data while only changing the unseen classes across five critical dimensions, to investigate their impact on SSL models from global robustness and local robustness. Experiments demonstrate that unseen classes in unlabeled data do not necessarily impair the performance of SSL models; in fact, under certain conditions, unseen classes may even enhance them.

Title: A Simple and Effective Reinforcement Learning Method for Text-to-Image Diffusion Fine-tuning

• Authors: Shashank Gupta, Chaitanya Ahuja, Tsung-Yu Lin, Sreya Dutta Roy, Harrie Oosterhuis, Maarten de Rijke, Satya Narayan Shukla
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00897
• Pdf URL: https://arxiv.org/pdf/2503.00897
• Copy Paste: [[2503.00897]] A Simple and Effective Reinforcement Learning Method for Text-to-Image Diffusion Fine-tuning(https://arxiv.org/abs/2503.00897)
• Keywords: robust, diffusion
• Abstract: Reinforcement learning ( RL)-based fine-tuning has emerged as a powerful approach for aligning diffusion models with black-box objectives. Proximal policy optimization (PPO) is the most popular choice of method for policy optimization. While effective in terms of performance, PPO is highly sensitive to hyper-parameters and involves substantial computational overhead. REINFORCE, on the other hand, mitigates some computational complexities such as high memory overhead and sensitive hyper-parameter tuning, but has suboptimal performance due to high-variance and sample inefficiency. While the variance of the REINFORCE can be reduced by sampling multiple actions per input prompt and using a baseline correction term, it still suffers from sample inefficiency. To address these challenges, we systematically analyze the efficiency-effectiveness trade-off between REINFORCE and PPO, and propose leave-one-out PPO ( LOOP), a novel RL for diffusion fine-tuning method. LOOP combines variance reduction techniques from REINFORCE, such as sampling multiple actions per input prompt and a baseline correction term, with the robustness and sample efficiency of PPO via clipping and importance sampling. Our results demonstrate that LOOP effectively improves diffusion models on various black-box objectives, and achieves a better balance between computational efficiency and performance.

Title: S4M: S4 for multivariate time series forecasting with Missing values

• Authors: Jing Peng, Meiqi Yang, Qiong Zhang, Xiaoxiao Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00900
• Pdf URL: https://arxiv.org/pdf/2503.00900
• Copy Paste: [[2503.00900]] S4M: S4 for multivariate time series forecasting with Missing values(https://arxiv.org/abs/2503.00900)
• Keywords: robust
• Abstract: Multivariate time series data play a pivotal role in a wide range of real-world applications. However, the presence of block missing data introduces significant challenges, often compromising the performance of predictive models. Traditional two-step approaches, which first impute missing values and then perform forecasting, are prone to error accumulation, particularly in complex multivariate settings characterized by high missing ratios and intricate dependency structures. In this work, we introduce S4M, an end-to-end time series forecasting framework that seamlessly integrates missing data handling into the Structured State Space Sequence (S4) model architecture. Unlike conventional methods that treat imputation as a separate preprocessing step, S4M leverages the latent space of S4 models to directly recognize and represent missing data patterns, thereby more effectively capturing the underlying temporal and multivariate dependencies. Our framework comprises two key components: the Adaptive Temporal Prototype Mapper (ATPM) and the Missing-Aware Dual Stream S4 (MDS-S4). The ATPM employs a prototype bank to derive robust and informative representations from historical data patterns, while the MDS-S4 processes these representations alongside missingness masks as dual input streams to enable accurate forecasting. Through extensive empirical evaluations on diverse real-world datasets, we demonstrate that S4M consistently achieves state-of-the-art performance. These results underscore the efficacy of our integrated approach in handling missing data, showcasing its robustness and superiority over traditional imputation-based methods. Our findings highlight the potential of S4M to advance reliable time series forecasting in practical applications, offering a promising direction for future research and deployment. Code is available at this https URL.

Title: FunBench: Benchmarking Fundus Reading Skills of MLLMs

• Authors: Qijie Wei, Kaiheng Qian, Xirong Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00901
• Pdf URL: https://arxiv.org/pdf/2503.00901
• Copy Paste: [[2503.00901]] FunBench: Benchmarking Fundus Reading Skills of MLLMs(https://arxiv.org/abs/2503.00901)
• Keywords: large language model
• Abstract: Multimodal Large Language Models (MLLMs) have shown significant potential in medical image analysis. However, their capabilities in interpreting fundus images, a critical skill for ophthalmology, remain under-evaluated. Existing benchmarks lack fine-grained task divisions and fail to provide modular analysis of its two key modules, i.e., large language model (LLM) and vision encoder (VE). This paper introduces FunBench, a novel visual question answering (VQA) benchmark designed to comprehensively evaluate MLLMs' fundus reading skills. FunBench features a hierarchical task organization across four levels (modality perception, anatomy perception, lesion analysis, and disease diagnosis). It also offers three targeted evaluation modes: linear-probe based VE evaluation, knowledge-prompted LLM evaluation, and holistic evaluation. Experiments on nine open-source MLLMs plus GPT-4o reveal significant deficiencies in fundus reading skills, particularly in basic tasks such as laterality recognition. The results highlight the limitations of current MLLMs and emphasize the need for domain-specific training and improved LLMs and VEs.

Title: Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction

• Authors: Liping Liu, Chunhong Zhang, Likang Wu, Chuang Zhao, Zheng Hu, Ming He, Jianping Fan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00902
• Pdf URL: https://arxiv.org/pdf/2503.00902
• Copy Paste: [[2503.00902]] Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction(https://arxiv.org/abs/2503.00902)
• Keywords: large language model
• Abstract: Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs' internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.

Title: DEAL: Data-Efficient Adversarial Learning for High-Quality Infrared Imaging

• Authors: Zhu Liu, Zijun Wang, Jinyuan Liu, Fanqi Meng, Long Ma, Risheng Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00905
• Pdf URL: https://arxiv.org/pdf/2503.00905
• Copy Paste: [[2503.00905]] DEAL: Data-Efficient Adversarial Learning for High-Quality Infrared Imaging(https://arxiv.org/abs/2503.00905)
• Keywords: attack
• Abstract: Thermal imaging is often compromised by dynamic, complex degradations caused by hardware limitations and unpredictable environmental factors. The scarcity of high-quality infrared data, coupled with the challenges of dynamic, intricate degradations, makes it difficult to recover details using existing methods. In this paper, we introduce thermal degradation simulation integrated into the training process via a mini-max optimization, by modeling these degraded factors as adversarial attacks on thermal images. The simulation is dynamic to maximize objective functions, thus capturing a broad spectrum of degraded data distributions. This approach enables training with limited data, thereby improving model this http URL, we introduce a dual-interaction network that combines the benefits of spiking neural networks with scale transformation to capture degraded features with sharp spike signal intensities. This architecture ensures compact model parameters while preserving efficient feature representation. Extensive experiments demonstrate that our method not only achieves superior visual quality under diverse single and composited degradation, but also delivers a significant reduction in processing when trained on only fifty clear images, outperforming existing techniques in efficiency and accuracy. The source code will be available at this https URL.

Title: HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning

• Authors: Zhuohang Jiang, Pangjing Wu, Ziran Liang, Peter Q. Chen, Xu Yuan, Ye Jia, Jiancheng Tu, Chen Li, Peter H.F. Ng, Qing Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00912
• Pdf URL: https://arxiv.org/pdf/2503.00912
• Copy Paste: [[2503.00912]] HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning(https://arxiv.org/abs/2503.00912)
• Keywords: extraction, large language model
• Abstract: Structure reasoning is a fundamental capability of large language models (LLMs), enabling them to reason about structured commonsense and answer multi-hop questions. However, existing benchmarks for structure reasoning mainly focus on horizontal and coordinate structures (\emph{e.g.} graphs), overlooking the hierarchical relationships within them. Hierarchical structure reasoning is crucial for human cognition, particularly in memory organization and problem-solving. It also plays a key role in various real-world tasks, such as information extraction and decision-making. To address this gap, we propose HiBench, the first framework spanning from initial structure generation to final proficiency assessment, designed to benchmark the hierarchical reasoning capabilities of LLMs systematically. HiBench encompasses six representative scenarios, covering both fundamental and practical aspects, and consists of 30 tasks with varying hierarchical complexity, totaling 39,519 queries. To evaluate LLMs comprehensively, we develop five capability dimensions that depict different facets of hierarchical structure understanding. Through extensive evaluation of 20 LLMs from 10 model families, we reveal key insights into their capabilities and limitations: 1) existing LLMs show proficiency in basic hierarchical reasoning tasks; 2) they still struggle with more complex structures and implicit hierarchical representations, especially in structural modification and textual reasoning. Based on these findings, we create a small yet well-designed instruction dataset, which enhances LLMs' performance on HiBench by an average of 88.84\% (Llama-3.1-8B) and 31.38\% (Qwen2.5-7B) across all tasks. The HiBench dataset and toolkit are available here, this https URL, to encourage evaluation.

Title: Multimodal Distillation-Driven Ensemble Learning for Long-Tailed Histopathology Whole Slide Images Analysis

• Authors: Xitong Ling, Yifeng Ping, Jiawen Li, Jing Peng, Yuxuan Chen, Minxi Ouyang, Yizhi Wang, Yonghong He, Tian Guan, Xiaoping Liu, Lianghui Zhu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00915
• Pdf URL: https://arxiv.org/pdf/2503.00915
• Copy Paste: [[2503.00915]] Multimodal Distillation-Driven Ensemble Learning for Long-Tailed Histopathology Whole Slide Images Analysis(https://arxiv.org/abs/2503.00915)
• Keywords: extraction
• Abstract: Multiple Instance Learning (MIL) plays a significant role in computational pathology, enabling weakly supervised analysis of Whole Slide Image (WSI) datasets. The field of WSI analysis is confronted with a severe long-tailed distribution problem, which significantly impacts the performance of classifiers. Long-tailed distributions lead to class imbalance, where some classes have sparse samples while others are abundant, making it difficult for classifiers to accurately identify minority class samples. To address this issue, we propose an ensemble learning method based on MIL, which employs expert decoders with shared aggregators and consistency constraints to learn diverse distributions and reduce the impact of class imbalance on classifier performance. Moreover, we introduce a multimodal distillation framework that leverages text encoders pre-trained on pathology-text pairs to distill knowledge and guide the MIL aggregator in capturing stronger semantic features relevant to class information. To ensure flexibility, we use learnable prompts to guide the distillation process of the pre-trained text encoder, avoiding limitations imposed by specific prompts. Our method, MDE-MIL, integrates multiple expert branches focusing on specific data distributions to address long-tailed issues. Consistency control ensures generalization across classes. Multimodal distillation enhances feature extraction. Experiments on Camelyon+-LT and PANDA-LT datasets show it outperforms state-of-the-art methods.

Title: AMUN: Adversarial Machine UNlearning

• Authors: Ali Ebrahimpour-Boroojeny, Hari Sundaram, Varun Chandrasekaran
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.00917
• Pdf URL: https://arxiv.org/pdf/2503.00917
• Copy Paste: [[2503.00917]] AMUN: Adversarial Machine UNlearning(https://arxiv.org/abs/2503.00917)
• Keywords: privacy, attack, membership infer
• Abstract: Machine unlearning, where users can request the deletion of a forget dataset, is becoming increasingly important because of numerous privacy regulations. Initial works on ``exact'' unlearning (e.g., retraining) incur large computational overheads. However, while computationally inexpensive, ``approximate'' methods have fallen short of reaching the effectiveness of exact unlearning: models produced fail to obtain comparable accuracy and prediction confidence on both the forget and test (i.e., unseen) dataset. Exploiting this observation, we propose a new unlearning method, Adversarial Machine UNlearning (AMUN), that outperforms prior state-of-the-art (SOTA) methods for image classification. AMUN lowers the confidence of the model on the forget samples by fine-tuning the model on their corresponding adversarial examples. Adversarial examples naturally belong to the distribution imposed by the model on the input space; fine-tuning the model on the adversarial examples closest to the corresponding forget samples (a) localizes the changes to the decision boundary of the model around each forget sample and (b) avoids drastic changes to the global behavior of the model, thereby preserving the model's accuracy on test samples. Using AMUN for unlearning a random $10\%$ of CIFAR-10 samples, we observe that even SOTA membership inference attacks cannot do better than random guessing.

Title: Explainable Classifier for Malignant Lymphoma Subtyping via Cell Graph and Image Fusion

• Authors: Daiki Nishiyama, Hiroaki Miyoshi, Noriaki Hashimoto, Koichi Ohshima, Hidekata Hontani, Ichiro Takeuchi, Jun Sakuma
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00925
• Pdf URL: https://arxiv.org/pdf/2503.00925
• Copy Paste: [[2503.00925]] Explainable Classifier for Malignant Lymphoma Subtyping via Cell Graph and Image Fusion(https://arxiv.org/abs/2503.00925)
• Keywords: explainability
• Abstract: Malignant lymphoma subtype classification directly impacts treatment strategies and patient outcomes, necessitating classification models that achieve both high accuracy and sufficient explainability. This study proposes a novel explainable Multi-Instance Learning (MIL) framework that identifies subtype-specific Regions of Interest (ROIs) from Whole Slide Images (WSIs) while integrating cell distribution characteristics and image information. Our framework simultaneously addresses three objectives: (1) indicating appropriate ROIs for each subtype, (2) explaining the frequency and spatial distribution of characteristic cell types, and (3) achieving high-accuracy subtyping by leveraging both image and cell-distribution modalities. The proposed method fuses cell graph and image features extracted from each patch in the WSI using a Mixture-of-Experts (MoE) approach and classifies subtypes within an MIL framework. Experiments on a dataset of 1,233 WSIs demonstrate that our approach achieves state-of-the-art accuracy among ten comparative methods and provides region-level and cell-level explanations that align with a pathologist's perspectives.

Title: Improving the Transferability of Adversarial Attacks by an Input Transpose

• Authors: Qing Wan, Shilong Deng, Xun Wang
• Subjects: cs.CV, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00932
• Pdf URL: https://arxiv.org/pdf/2503.00932
• Copy Paste: [[2503.00932]] Improving the Transferability of Adversarial Attacks by an Input Transpose(https://arxiv.org/abs/2503.00932)
• Keywords: attack
• Abstract: Deep neural networks (DNNs) are highly susceptible to adversarial examples--subtle perturbations applied to inputs that are often imperceptible to humans yet lead to incorrect model predictions. In black-box scenarios, however, existing adversarial examples exhibit limited transferability and struggle to effectively compromise multiple unseen DNN models. Previous strategies enhance the cross-model generalization of adversarial examples by introducing versatility into adversarial perturbations, thereby improving transferability. However, further refining perturbation versatility often demands intricate algorithm development and substantial computation consumption. In this work, we propose an input transpose method that requires almost no additional labor and computation costs but can significantly improve the transferability of existing adversarial strategies. Even without adding adversarial perturbations, our method demonstrates considerable effectiveness in cross-model attacks. Our exploration finds that on specific datasets, a mere $1^\circ$ left or right rotation might be sufficient for most adversarial examples to deceive unseen models. Our further analysis suggests that this transferability improvement triggered by rotating only $1^\circ$ may stem from visible pattern shifts in the DNN's low-level feature maps. Moreover, this transferability exhibits optimal angles that, when identified under unrestricted query conditions, could potentially yield even greater performance.

Title: IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis

• Authors: Yuji Wang, Jingchen Ni, Yong Liu, Chun Yuan, Yansong Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00936
• Pdf URL: https://arxiv.org/pdf/2503.00936
• Copy Paste: [[2503.00936]] IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis(https://arxiv.org/abs/2503.00936)
• Keywords: segmentation
• Abstract: Zero-shot Referring Image Segmentation (RIS) identifies the instance mask that best aligns with a specified referring expression without training and fine-tuning, significantly reducing the labor-intensive annotation process. Despite achieving commendable results, previous CLIP-based models have a critical drawback: the models exhibit a notable reduction in their capacity to discern relative spatial relationships of objects. This is because they generate all possible masks on an image and evaluate each masked region for similarity to the given expression, often resulting in decreased sensitivity to direct positional clues in text inputs. Moreover, most methods have weak abilities to manage relationships between primary words and their contexts, causing confusion and reduced accuracy in identifying the correct target region. To address these challenges, we propose IteRPrimE (Iterative Grad-CAM Refinement and Primary word Emphasis), which leverages a saliency heatmap through Grad-CAM from a Vision-Language Pre-trained (VLP) model for image-text matching. An iterative Grad-CAM refinement strategy is introduced to progressively enhance the model's focus on the target region and overcome positional insensitivity, creating a self-correcting effect. Additionally, we design the Primary Word Emphasis module to help the model handle complex semantic relations, enhancing its ability to attend to the intended object. Extensive experiments conducted on the RefCOCO/+/g, and PhraseCut benchmarks demonstrate that IteRPrimE outperforms previous state-of-the-art zero-shot methods, particularly excelling in out-of-domain scenarios.

Title: From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization

• Authors: Chao Yuan, Guiwei Zhang, Changxiao Ma, Tianyi Zhang, Guanglin Niu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00938
• Pdf URL: https://arxiv.org/pdf/2503.00938
• Copy Paste: [[2503.00938]] From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization(https://arxiv.org/abs/2503.00938)
• Keywords: extraction, generative
• Abstract: Person re-identification (ReID) aims to extract accurate identity representation features. However, during feature extraction, individual samples are inevitably affected by noise (background, occlusions, and model limitations). Considering that features from the same identity follow a normal distribution around identity centers after training, we propose a Training-Free Feature Centralization ReID framework (Pose2ID) by aggregating the same identity features to reduce individual noise and enhance the stability of identity representation, which preserves the feature's original distribution for following strategies such as re-ranking. Specifically, to obtain samples of the same identity, we introduce two components:Identity-Guided Pedestrian Generation: by leveraging identity features to guide the generation process, we obtain high-quality images with diverse poses, ensuring identity consistency even in complex scenarios such as infrared, and this http URL Feature Centralization: it explores each sample's potential positive samples from its neighborhood. Experiments demonstrate that our generative model exhibits strong generalization capabilities and maintains high identity consistency. With the Feature Centralization framework, we achieve impressive performance even with an ImageNet pre-trained model without ReID training, reaching mAP/Rank-1 of 52.81/78.92 on Market1501. Moreover, our method sets new state-of-the-art results across standard, cross-modality, and occluded ReID tasks, showcasing strong adaptability.

Title: Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think

• Authors: Jie Tian, Xiaoye Qu, Zhenyi Lu, Wei Wei, Sichen Liu, Yu Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00948
• Pdf URL: https://arxiv.org/pdf/2503.00948
• Copy Paste: [[2503.00948]] Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think(https://arxiv.org/abs/2503.00948)
• Keywords: diffusion
• Abstract: Image-to-Video (I2V) generation aims to synthesize a video clip according to a given image and condition (e.g., text). The key challenge of this task lies in simultaneously generating natural motions while preserving the original appearance of the images. However, current I2V diffusion models (I2V-DMs) often produce videos with limited motion degrees or exhibit uncontrollable motion that conflicts with the textual condition. To address these limitations, we propose a novel Extrapolating and Decoupling framework, which introduces model merging techniques to the I2V domain for the first time. Specifically, our framework consists of three separate stages: (1) Starting with a base I2V-DM, we explicitly inject the textual condition into the temporal module using a lightweight, learnable adapter and fine-tune the integrated model to improve motion controllability. (2) We introduce a training-free extrapolation strategy to amplify the dynamic range of the motion, effectively reversing the fine-tuning process to enhance the motion degree significantly. (3) With the above two-stage models excelling in motion controllability and degree, we decouple the relevant parameters associated with each type of motion ability and inject them into the base I2V-DM. Since the I2V-DM handles different levels of motion controllability and dynamics at various denoising time steps, we adjust the motion-aware parameters accordingly over time. Extensive qualitative and quantitative experiments have been conducted to demonstrate the superiority of our framework over existing methods.

Title: Decomposition of RSA modulus applying even order elliptic curves

• Authors: Jacek Pomykała, Mariusz Jurkiewicz
• Subjects: cs.CR, math.NT
• Abstract URL: https://arxiv.org/abs/2503.00950
• Pdf URL: https://arxiv.org/pdf/2503.00950
• Copy Paste: [[2503.00950]] Decomposition of RSA modulus applying even order elliptic curves(https://arxiv.org/abs/2503.00950)
• Keywords: security
• Abstract: An efficient integer factorization algorithm would reduce the security of all variants of the RSA cryptographic scheme to zero. Despite the passage of years, no method for efficiently factoring large semiprime numbers in a classical computational model has been discovered. In this paper, we demonstrate how a natural extension of the generalized approach to smoothness, combined with the separation of $2$-adic point orders, leads us to propose a factoring algorithm that finds (conjecturally) the prime decomposition $N = pq$ in subexponential time $L(\sqrt 2+o(1), \min(p,q))$. This approach motivated by the papers \cite{Len}, \cite{MMV} and \cite{PoZo} is based on a more careful investigation of pairs $(E,Q)$, where $Q$ is a point on an elliptic curve $E$ over $\Z _N$. Specifically, in contrast to the familiar condition that the largest prime divisor $P^+(\ord Q_p)$ of the reduced order $\ord Q_p$ does not divide $\#E(\F_q)$ we focus on the relation between $P^+(\ord Q_r)$ and the smallest prime number $l_{\min}(E,Q)$ separating the orders $\ord Q_p$ and $\ord Q_q$. We focus on the ${\calE}_2$ family of even order elliptic curves over $\Z_N$ since then the condition $l_{\min}(E,Q)\le 2$ holds true for large fraction of points $(x,y)\in E(\Z_N)$. Moreover if we know the pair $(E,Q)$ such that $P^+(\ord Q_r)\le t

Title: Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models

• Authors: Xingzhuo Guo, Yu Zhang, Baixu Chen, Haoran Xu, Jianmin Wang, Mingsheng Long
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00951
• Pdf URL: https://arxiv.org/pdf/2503.00951
• Copy Paste: [[2503.00951]] Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models(https://arxiv.org/abs/2503.00951)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as powerful generative frameworks by progressively adding noise to data through a forward process and then reversing this process to generate realistic samples. While these models have achieved strong performance across various tasks and modalities, their application to temporal predictive learning remains underexplored. Existing approaches treat predictive learning as a conditional generation problem, but often fail to fully exploit the temporal dynamics inherent in the data, leading to challenges in generating temporally coherent sequences. To address this, we introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes. Dynamical Diffusion explicitly models temporal transitions at each diffusion step, establishing dependencies on preceding states to better capture temporal dynamics. Through the reparameterization trick, Dynamical Diffusion achieves efficient training and inference similar to any standard diffusion model. Extensive experiments across scientific spatiotemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks, filling a crucial gap in existing methodologies. Code is available at this repository: this https URL.

Title: SemViQA: A Semantic Question Answering System for Vietnamese Information Fact-Checking

• Authors: Nam V. Nguyen, Dien X. Tran, Thanh T. Tran, Anh T. Hoang, Tai V. Duong, Di T. Le, Phuc-Lu Le
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.00955
• Pdf URL: https://arxiv.org/pdf/2503.00955
• Copy Paste: [[2503.00955]] SemViQA: A Semantic Question Answering System for Vietnamese Information Fact-Checking(https://arxiv.org/abs/2503.00955)
• Keywords: robust, large language model
• Abstract: The rise of misinformation, exacerbated by Large Language Models (LLMs) like GPT and Gemini, demands robust fact-checking solutions, especially for low-resource languages like Vietnamese. Existing methods struggle with semantic ambiguity, homonyms, and complex linguistic structures, often trading accuracy for efficiency. We introduce SemViQA, a novel Vietnamese fact-checking framework integrating Semantic-based Evidence Retrieval (SER) and Two-step Verdict Classification (TVC). Our approach balances precision and speed, achieving state-of-the-art results with 78.97\% strict accuracy on ISE-DSC01 and 80.82\% on ViWikiFC, securing 1st place in the UIT Data Science Challenge. Additionally, SemViQA Faster improves inference speed 7x while maintaining competitive accuracy. SemViQA sets a new benchmark for Vietnamese fact verification, advancing the fight against misinformation. The source code is available at: this https URL.

Title: Layered Insights: Generalizable Analysis of Authorial Style by Leveraging All Transformer Layers

• Authors: Milad Alshomary, Nikhil Reddy Varimalla, Vishal Anand, Kathleen McKeown
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00958
• Pdf URL: https://arxiv.org/pdf/2503.00958
• Copy Paste: [[2503.00958]] Layered Insights: Generalizable Analysis of Authorial Style by Leveraging All Transformer Layers(https://arxiv.org/abs/2503.00958)
• Keywords: robust, transformer
• Abstract: We propose a new approach for the authorship attribution task that leverages the various linguistic representations learned at different layers of pre-trained transformer-based models. We evaluate our approach on three datasets, comparing it to a state-of-the-art baseline in in-domain and out-of-domain scenarios. We found that utilizing various transformer layers improves the robustness of authorship attribution models when tested on out-of-domain data, resulting in new state-of-the-art results. Our analysis gives further insights into how our model's different layers get specialized in representing certain stylistic features that benefit the model when tested out of the domain.

Title: CAGN-GAT Fusion: A Hybrid Contrastive Attentive Graph Neural Network for Network Intrusion Detection

• Authors: Md Abrar Jahin, Shahriar Soudeep, M. F. Mridha, Raihan Kabir, Md Rashedul Islam, Yutaka Watanobe
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00961
• Pdf URL: https://arxiv.org/pdf/2503.00961
• Copy Paste: [[2503.00961]] CAGN-GAT Fusion: A Hybrid Contrastive Attentive Graph Neural Network for Network Intrusion Detection(https://arxiv.org/abs/2503.00961)
• Keywords: security, robust, fair
• Abstract: Cybersecurity threats are growing, making network intrusion detection essential. Traditional machine learning models remain effective in resource-limited environments due to their efficiency, requiring fewer parameters and less computational time. However, handling short and highly imbalanced datasets remains challenging. In this study, we propose the fusion of a Contrastive Attentive Graph Network and Graph Attention Network (CAGN-GAT Fusion) and benchmark it against 15 other models, including both Graph Neural Networks (GNNs) and traditional ML models. Our evaluation is conducted on four benchmark datasets (KDD-CUP-1999, NSL-KDD, UNSW-NB15, and CICIDS2017) using a short and proportionally imbalanced dataset with a constant size of 5000 samples to ensure fairness in comparison. Results show that CAGN-GAT Fusion demonstrates stable and competitive accuracy, recall, and F1-score, even though it does not achieve the highest performance in every dataset. Our analysis also highlights the impact of adaptive graph construction techniques, including small changes in connections (edge perturbation) and selective hiding of features (feature masking), improving detection performance. The findings confirm that GNNs, particularly CAGN-GAT Fusion, are robust and computationally efficient, making them well-suited for resource-constrained environments. Future work will explore GraphSAGE layers and multiview graph construction techniques to further enhance adaptability and detection accuracy.

Title: Using Synthetic Images to Augment Small Medical Image Datasets

• Authors: Minh H. Vu, Lorenzo Tronchin, Tufve Nyholm, Tommy Löfstedt
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00962
• Pdf URL: https://arxiv.org/pdf/2503.00962
• Copy Paste: [[2503.00962]] Using Synthetic Images to Augment Small Medical Image Datasets(https://arxiv.org/abs/2503.00962)
• Keywords: generative, segmentation
• Abstract: Recent years have witnessed a growing academic and industrial interest in deep learning (DL) for medical imaging. To perform well, DL models require very large labeled datasets. However, most medical imaging datasets are small, with a limited number of annotated samples. The reason they are small is usually because delineating medical images is time-consuming and demanding for oncologists. There are various techniques that can be used to augment a dataset, for example, to apply affine transformations or elastic transformations to available images, or to add synthetic images generated by a Generative Adversarial Network (GAN). In this work, we have developed a novel conditional variant of a current GAN method, the StyleGAN2, to generate multi-modal high-resolution medical images with the purpose to augment small medical imaging datasets with these synthetic images. We use the synthetic and real images from six datasets to train models for the downstream task of semantic segmentation. The quality of the generated medical images and the effect of this augmentation on the segmentation performance were evaluated afterward. Finally, the results indicate that the downstream segmentation models did not benefit from the generated images. Further work and analyses are required to establish how this augmentation affects the segmentation performance.

Title: Semantic-ICP: Iterative Closest Point for Non-rigid Multi-Organ Point Cloud Registration

• Authors: Wanwen Chen, Carson Studders, Jamie J.Y. Kwon, Emily H.T. Pang, Eitan Prisman, Septimiu E. Salcudean
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00972
• Pdf URL: https://arxiv.org/pdf/2503.00972
• Copy Paste: [[2503.00972]] Semantic-ICP: Iterative Closest Point for Non-rigid Multi-Organ Point Cloud Registration(https://arxiv.org/abs/2503.00972)
• Keywords: robust, explainability
• Abstract: Point cloud registration is important in computer-aided interventions (CAI). While learning-based point cloud registration methods have been developed, their clinical application is hampered by issues of generalizability and explainability. Therefore, classical point cloud registration methods, such as Iterative Closest Point (ICP), are still widely applied in CAI. ICP methods fail to consider that: (1) the points have well-defined semantic meaning, in that each point can be related to a specific anatomical label; (2) the deformation needs to follow biomechanical energy constraints. In this paper, we present a novel semantic ICP (sem-ICP) method that handles multiple point labels and uses linear elastic energy regularization. We use semantic labels to improve the robustness of the closest point matching and propose a new point cloud deformation representation to apply explicit biomechanical energy regularization. Our experiments on the Learn2reg abdominal MR-CT registration dataset and a trans-oral robotic surgery ultrasound-CT registration dataset show that our method improves the Hausdorff distance compared with other state-of-the-art ICP-based registration methods. We also perform a sensitivity study to show that our rigid initialization achieves better convergence with different initializations and visible ratios.

Title: Molecule Generation for Target Protein Binding with Hierarchical Consistency Diffusion Model

• Authors: Guanlue Li, Chenran Jiang, Ziqi Gao, Yu Liu, Chenyang Liu, Jiean Chen, Yong Huang, Jia Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00975
• Pdf URL: https://arxiv.org/pdf/2503.00975
• Copy Paste: [[2503.00975]] Molecule Generation for Target Protein Binding with Hierarchical Consistency Diffusion Model(https://arxiv.org/abs/2503.00975)
• Keywords: robust, diffusion
• Abstract: Effective generation of molecular structures, or new chemical entities, that bind to target proteins is crucial for lead identification and optimization in drug discovery. Despite advancements in atom- and motif-wise deep learning models for 3D molecular generation, current methods often struggle with validity and reliability. To address these issues, we develop the Atom-Motif Consistency Diffusion Model (AMDiff), utilizing a joint-training paradigm for multi-view learning. This model features a hierarchical diffusion architecture that integrates both atom- and motif-level views of molecules, allowing for comprehensive exploration of complementary information. By leveraging classifier-free guidance and incorporating binding site features as conditional inputs, AMDiff ensures robust molecule generation across diverse targets. Compared to existing approaches, AMDiff exhibits superior validity and novelty in generating molecules tailored to fit various protein pockets. Case studies targeting protein kinases, including Anaplastic Lymphoma Kinase (ALK) and Cyclin-dependent kinase 4 (CDK4), demonstrate the model's capability in structure-based de novo drug design. Overall, AMDiff bridges the gap between atom-view and motif-view drug discovery and speeds up the process of target-aware molecular generation.

Title: Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs

• Authors: Ravi Ghadia, Avinash Kumar, Gaurav Jain, Prashant Nair, Poulami Das
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.00979
• Pdf URL: https://arxiv.org/pdf/2503.00979
• Copy Paste: [[2503.00979]] Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs(https://arxiv.org/abs/2503.00979)
• Keywords: transformer
• Abstract: Autoregressive Transformers rely on Key-Value (KV) caching to accelerate inference. However, the linear growth of the KV cache with context length leads to excessive memory consumption and bandwidth constraints. This bottleneck is particularly problematic in real-time applications -- such as chatbots and interactive assistants -- where low latency and high memory efficiency are critical. Existing methods drop distant tokens or compress states in a lossy manner, sacrificing accuracy by discarding vital context or introducing bias. We propose MorphKV, an inference-time technique that maintains a constant-sized KV cache while preserving accuracy. MorphKV balances long-range dependencies and local coherence during text generation. It eliminates early-token bias while retaining high-fidelity context by adaptively ranking tokens through correlation-aware selection. Unlike heuristic retention or lossy compression, MorphKV iteratively refines the KV cache via lightweight updates guided by attention patterns of recent tokens. This approach captures inter-token correlation with greater accuracy, crucial for tasks like content creation and code generation. Our studies on long-response tasks show 52.9$\%$ memory savings and 18.2$\%$ higher accuracy on average compared to state-of-the-art prior works, enabling efficient real-world deployment.

Title: Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study

• Authors: Bashar Alhafni, Nizar Habash
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00985
• Pdf URL: https://arxiv.org/pdf/2503.00985
• Copy Paste: [[2503.00985]] Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study(https://arxiv.org/abs/2503.00985)
• Keywords: interpretability
• Abstract: Text editing frames grammatical error correction (GEC) as a sequence tagging problem, where edit tags are assigned to input tokens, and applying these edits results in the corrected text. This approach has gained attention for its efficiency and interpretability. However, while extensively explored for English, text editing remains largely underexplored for morphologically rich languages like Arabic. In this paper, we introduce a text editing approach that derives edit tags directly from data, eliminating the need for language-specific edits. We demonstrate its effectiveness on Arabic, a diglossic and morphologically rich language, and investigate the impact of different edit representations on model performance. Our approach achieves SOTA results on two Arabic GEC benchmarks and performs on par with SOTA on two others. Additionally, our models are over six times faster than existing Arabic GEC systems, making our approach more practical for real-world applications. Finally, we explore ensemble models, demonstrating how combining different models leads to further performance improvements. We make our code, data, and pretrained models publicly available.

Title: Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning

• Authors: Baoqi Pei, Yifei Huang, Jilan Xu, Guo Chen, Yuping He, Lijin Yang, Yali Wang, Weidi Xie, Yu Qiao, Fei Wu, Limin Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.00986
• Pdf URL: https://arxiv.org/pdf/2503.00986
• Copy Paste: [[2503.00986]] Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning(https://arxiv.org/abs/2503.00986)
• Keywords: robust, large language model
• Abstract: In egocentric video understanding, the motion of hands and objects as well as their interactions play a significant role by nature. However, existing egocentric video representation learning methods mainly focus on aligning video representation with high-level narrations, overlooking the intricate dynamics between hands and objects. In this work, we aim to integrate the modeling of fine-grained hand-object dynamics into the video representation learning process. Since no suitable data is available, we introduce HOD, a novel pipeline employing a hand-object detector and a large language model to generate high-quality narrations with detailed descriptions of hand-object dynamics. To learn these fine-grained dynamics, we propose EgoVideo, a model with a new lightweight motion adapter to capture fine-grained hand-object motion information. Through our co-training strategy, EgoVideo effectively and efficiently leverages the fine-grained hand-object dynamics in the HOD data. Extensive experiments demonstrate that our method achieves state-of-the-art performance across multiple egocentric downstream tasks, including improvements of 6.3% in EK-100 multi-instance retrieval, 5.7% in EK-100 classification, and 16.3% in EGTEA classification in zero-shot settings. Furthermore, our model exhibits robust generalization capabilities in hand-object interaction and robot manipulation tasks. Code and data are available at this https URL.

Title: Evaluating Polish linguistic and cultural competency in large language models

• Authors: Sławomir Dadas, Małgorzata Grębowiec, Michał Perełkiewicz, Rafał Poświata
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.00995
• Pdf URL: https://arxiv.org/pdf/2503.00995
• Copy Paste: [[2503.00995]] Evaluating Polish linguistic and cultural competency in large language models(https://arxiv.org/abs/2503.00995)
• Keywords: large language model
• Abstract: Large language models (LLMs) are becoming increasingly proficient in processing and generating multilingual texts, which allows them to address real-world problems more effectively. However, language understanding is a far more complex issue that goes beyond simple text analysis. It requires familiarity with cultural context, including references to everyday life, historical events, traditions, folklore, literature, and pop culture. A lack of such knowledge can lead to misinterpretations and subtle, hard-to-detect errors. To examine language models' knowledge of the Polish cultural context, we introduce the Polish linguistic and cultural competency benchmark, consisting of 600 manually crafted questions. The benchmark is divided into six categories: history, geography, culture & tradition, art & entertainment, grammar, and vocabulary. As part of our study, we conduct an extensive evaluation involving over 30 open-weight and commercial LLMs. Our experiments provide a new perspective on Polish competencies in language models, moving past traditional natural language processing tasks and general knowledge assessment.

Title: Underdamped Diffusion Bridges with Applications to Sampling

• Authors: Denis Blessing, Julius Berner, Lorenz Richter, Gerhard Neumann
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01006
• Pdf URL: https://arxiv.org/pdf/2503.01006
• Copy Paste: [[2503.01006]] Underdamped Diffusion Bridges with Applications to Sampling(https://arxiv.org/abs/2503.01006)
• Keywords: diffusion, generative
• Abstract: We provide a general framework for learning diffusion bridges that transport prior to target distributions. It includes existing diffusion models for generative modeling, but also underdamped versions with degenerate diffusion matrices, where the noise only acts in certain dimensions. Extending previous findings, our framework allows to rigorously show that score matching in the underdamped case is indeed equivalent to maximizing a lower bound on the likelihood. Motivated by superior convergence properties and compatibility with sophisticated numerical integration schemes of underdamped stochastic processes, we propose \emph{underdamped diffusion bridges}, where a general density evolution is learned rather than prescribed by a fixed noising process. We apply our method to the challenging task of sampling from unnormalized densities without access to samples from the target distribution. Across a diverse range of sampling problems, our approach demonstrates state-of-the-art performance, notably outperforming alternative methods, while requiring significantly fewer discretization steps and no hyperparameter tuning.

Title: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop

• Authors: Yushan Jiang, Wenchao Yu, Geon Lee, Dongjin Song, Kijung Shin, Wei Cheng, Yanchi Liu, Haifeng Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01013
• Pdf URL: https://arxiv.org/pdf/2503.01013
• Copy Paste: [[2503.01013]] Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop(https://arxiv.org/abs/2503.01013)
• Keywords: interpretability, large language model
• Abstract: Time series analysis provides essential insights for real-world system dynamics and informs downstream decision-making, yet most existing methods often overlook the rich contextual signals present in auxiliary modalities. To bridge this gap, we introduce TimeXL, a multi-modal prediction framework that integrates a prototype-based time series encoder with three collaborating Large Language Models (LLMs) to deliver more accurate predictions and interpretable explanations. First, a multi-modal prototype-based encoder processes both time series and textual inputs to generate preliminary forecasts alongside case-based rationales. These outputs then feed into a prediction LLM, which refines the forecasts by reasoning over the encoder's predictions and explanations. Next, a reflection LLM compares the predicted values against the ground truth, identifying textual inconsistencies or noise. Guided by this feedback, a refinement LLM iteratively enhances text quality and triggers encoder retraining. This closed-loop workflow -- prediction, critique (reflect), and refinement -- continuously boosts the framework's performance and interpretability. Empirical evaluations on four real-world datasets demonstrate that TimeXL achieves up to 8.9\% improvement in AUC and produces human-centric, multi-modal explanations, highlighting the power of LLM-driven reasoning for time series prediction.

Title: MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations

• Authors: Ziyang Zhang, Yang Yu, Yucheng Chen, Xulei Yang, Si Yong Yeo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01019
• Pdf URL: https://arxiv.org/pdf/2503.01019
• Copy Paste: [[2503.01019]] MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations(https://arxiv.org/abs/2503.01019)
• Keywords: extraction
• Abstract: Despite significant progress in Vision-Language Pre-training (VLP), current approaches predominantly emphasize feature extraction and cross-modal comprehension, with limited attention to generating or transforming visual content. This gap hinders the model's ability to synthesize coherent and novel visual representations from textual prompts, thereby reducing the effectiveness of multi-modal learning. In this work, we propose MedUnifier, a unified VLP framework tailored for medical data. MedUnifier seamlessly integrates text-grounded image generation capabilities with multi-modal learning strategies, including image-text contrastive alignment, image-text matching and image-grounded text generation. Unlike traditional methods that reply on continuous visual representations, our approach employs visual vector quantization, which not only facilitates a more cohesive learning strategy for cross-modal understanding but also enhances multi-modal generation quality by effectively leveraging discrete representations. Our framework's effectiveness is evidenced by the experiments on established benchmarks, including uni-modal tasks (supervised fine-tuning), cross-modal tasks (image-text retrieval and zero-shot image classification), and multi-modal tasks (medical report generation, image synthesis), where it achieves state-of-the-art performance across various tasks. MedUnifier also offers a highly adaptable tool for a wide range of language and vision tasks in healthcare, marking advancement toward the development of a generalizable AI model for medical applications.

Title: Delving into Out-of-Distribution Detection with Medical Vision-Language Models

• Authors: Lie Ju, Sijin Zhou, Yukun Zhou, Huimin Lu, Zhuoting Zhu, Pearse A. Keane, Zongyuan Ge
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01020
• Pdf URL: https://arxiv.org/pdf/2503.01020
• Copy Paste: [[2503.01020]] Delving into Out-of-Distribution Detection with Medical Vision-Language Models(https://arxiv.org/abs/2503.01020)
• Keywords: robust
• Abstract: Recent advances in medical vision-language models (VLMs) demonstrate impressive performance in image classification tasks, driven by their strong zero-shot generalization capabilities. However, given the high variability and complexity inherent in medical imaging data, the ability of these models to detect out-of-distribution (OOD) data in this domain remains underexplored. In this work, we conduct the first systematic investigation into the OOD detection potential of medical VLMs. We evaluate state-of-the-art VLM-based OOD detection methods across a diverse set of medical VLMs, including both general and domain-specific purposes. To accurately reflect real-world challenges, we introduce a cross-modality evaluation pipeline for benchmarking full-spectrum OOD detection, rigorously assessing model robustness against both semantic shifts and covariate shifts. Furthermore, we propose a novel hierarchical prompt-based method that significantly enhances OOD detection performance. Extensive experiments are conducted to validate the effectiveness of our approach. The codes are available at this https URL.

Title: Data Unlearning in Diffusion Models

• Authors: Silas Alberti, Kenan Hasanaliyev, Manav Shah, Stefano Ermon
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01034
• Pdf URL: https://arxiv.org/pdf/2503.01034
• Copy Paste: [[2503.01034]] Data Unlearning in Diffusion Models(https://arxiv.org/abs/2503.01034)
• Keywords: diffusion
• Abstract: Recent work has shown that diffusion models memorize and reproduce training data examples. At the same time, large copyright lawsuits and legislation such as GDPR have highlighted the need for erasing datapoints from diffusion models. However, retraining from scratch is often too expensive. This motivates the setting of data unlearning, i.e., the study of efficient techniques for unlearning specific datapoints from the training set. Existing concept unlearning techniques require an anchor prompt/class/distribution to guide unlearning, which is not available in the data unlearning setting. General-purpose machine unlearning techniques were found to be either unstable or failed to unlearn data. We therefore propose a family of new loss functions called Subtracted Importance Sampled Scores (SISS) that utilize importance sampling and are the first method to unlearn data with theoretical guarantees. SISS is constructed as a weighted combination between simpler objectives that are responsible for preserving model quality and unlearning the targeted datapoints. When evaluated on CelebA-HQ and MNIST, SISS achieved Pareto optimality along the quality and unlearning strength dimensions. On Stable Diffusion, SISS successfully mitigated memorization on nearly 90% of the prompts we tested.

Title: A Comparison of Object Detection and Phrase Grounding Models in Chest X-ray Abnormality Localization using Eye-tracking Data

• Authors: Elham Ghelichkhan, Tolga Tasdizen
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01037
• Pdf URL: https://arxiv.org/pdf/2503.01037
• Copy Paste: [[2503.01037]] A Comparison of Object Detection and Phrase Grounding Models in Chest X-ray Abnormality Localization using Eye-tracking Data(https://arxiv.org/abs/2503.01037)
• Keywords: explainability
• Abstract: Chest diseases rank among the most prevalent and dangerous global health issues. Object detection and phrase grounding deep learning models interpret complex radiology data to assist healthcare professionals in diagnosis. Object detection locates abnormalities for classes, while phrase grounding locates abnormalities for textual descriptions. This paper investigates how text enhances abnormality localization in chest X-rays by comparing the performance and explainability of these two tasks. To establish an explainability baseline, we proposed an automatic pipeline to generate image regions for report sentences using radiologists' eye-tracking data. The better performance - mIoU = 0.36 vs. 0.20 - and explainability - Containment ratio 0.48 vs. 0.26 - of the phrase grounding model infers the effectiveness of text in enhancing chest X-ray abnormality localization.

Title: Language-agnostic, automated assessment of listeners' speech recall using large language models

• Authors: Björn Herrmann
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01045
• Pdf URL: https://arxiv.org/pdf/2503.01045
• Copy Paste: [[2503.01045]] Language-agnostic, automated assessment of listeners' speech recall using large language models(https://arxiv.org/abs/2503.01045)
• Keywords: large language model
• Abstract: Speech-comprehension difficulties are common among older people. Standard speech tests do not fully capture such difficulties because the tests poorly resemble the context-rich, story-like nature of ongoing conversation and are typically available only in a country's dominant/official language (e.g., English), leading to inaccurate scores for native speakers of other languages. Assessments for naturalistic, story speech in multiple languages require accurate, time-efficient scoring. The current research leverages modern large language models (LLMs) in native English speakers and native speakers of 10 other languages to automate the generation of high-quality, spoken stories and scoring of speech recall in different languages. Participants listened to and freely recalled short stories (in quiet/clear and in babble noise) in their native language. LLM text-embeddings and LLM prompt engineering with semantic similarity analyses to score speech recall revealed sensitivity to known effects of temporal order, primacy/recency, and background noise, and high similarity of recall scores across languages. The work overcomes limitations associated with simple speech materials and testing of closed native-speaker groups because recall data of varying length and details can be mapped across languages with high accuracy. The full automation of speech generation and recall scoring provides an important step towards comprehension assessments of naturalistic speech with clinical applicability.

Title: Personalize Your LLM: Fake it then Align it

• Authors: Yijing Zhang, Dyah Adila, Changho Shin, Frederic Sala
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01048
• Pdf URL: https://arxiv.org/pdf/2503.01048
• Copy Paste: [[2503.01048]] Personalize Your LLM: Fake it then Align it(https://arxiv.org/abs/2503.01048)
• Keywords: large language model
• Abstract: Personalizing large language models (LLMs) is essential for delivering tailored interactions that improve user experience. Many existing personalization methods require fine-tuning LLMs for each user, rendering them prohibitively expensive for widespread adoption. Although retrieval-based approaches offer a more compute-efficient alternative, they still depend on large, high-quality datasets that are not consistently available for all users. To address this challenge, we propose CHAMELEON, a scalable and efficient personalization approach that uses (1) self-generated personal preference data and (2) representation editing to enable quick and cost-effective personalization. Our experiments on various tasks, including those from the LaMP personalization benchmark, show that CHAMELEON efficiently adapts models to personal preferences, improving instruction-tuned models and outperforms two personalization baselines by an average of 40% across two model architectures.

Title: ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Parity LLM Data Valuation

• Authors: Yanzhou Pan, Huawei Lin, Yide Ran, Jiamin Chen, Xiaodong Yu, Weijie Zhao, Denghui Zhang, Zhaozhuo Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01052
• Pdf URL: https://arxiv.org/pdf/2503.01052
• Copy Paste: [[2503.01052]] ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Parity LLM Data Valuation(https://arxiv.org/abs/2503.01052)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited budget. In this work, we aim to offer a third-party data valuation approach that benefits both data providers and model developers. We introduce a linearized future influence kernel (LinFiK), which assesses the value of individual data samples in improving LLM performance during training. We further propose ALinFiK, a learning strategy to approximate LinFiK, enabling scalable data valuation. Our comprehensive evaluations demonstrate that this approach surpasses existing baselines in effectiveness and efficiency, demonstrating significant scalability advantages as LLM parameters increase.

Title: SFO: Piloting VLM Feedback for Offline RL

• Authors: Jacob Beck
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01062
• Pdf URL: https://arxiv.org/pdf/2503.01062
• Copy Paste: [[2503.01062]] SFO: Piloting VLM Feedback for Offline RL(https://arxiv.org/abs/2503.01062)
• Keywords: robust
• Abstract: While internet-scale image and textual data have enabled strong generalization in Vision-Language Models (VLMs), the absence of internet-scale control data has impeded the development of similar generalization in standard reinforcement learning (RL) agents. Although VLMs are fundamentally limited in their ability to solve control tasks due to their lack of action-conditioned training data, their capacity for image understanding allows them to provide valuable feedback in RL tasks by recognizing successful outcomes. A key challenge in Reinforcement Learning from AI Feedback (RLAIF) is determining how best to integrate VLM-derived signals into the learning process. We explore this question in the context of offline RL and introduce a class of methods called sub-trajectory filtered optimization. We identify three key insights. First, trajectory length plays a crucial role in offline RL, as full-trajectory preference learning exacerbates the stitching problem, necessitating the use of sub-trajectories. Second, even in Markovian environments, a non-Markovian reward signal from a sequence of images is required to assess trajectory improvement, as VLMs do not interpret control actions and must rely on visual cues over time. Third, a simple yet effective approach--filtered and weighted behavior cloning--consistently outperforms more complex reinforcement learning from human feedback-based methods. We propose sub-trajectory filtered behavior cloning, a method that leverages VLM feedback on sub-trajectories while incorporating a retrospective filtering mechanism that removes sub-trajectories preceding failures to improve robustness and prevent turbulence. This study is preliminary; we provide initial evidence through evaluations on a toy control domain. Please enjoy our airport puns.

Title: AI-Invented Tonal Languages: Preventing a Machine Lingua Franca Beyond Human Understanding

• Authors: David Noever
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01063
• Pdf URL: https://arxiv.org/pdf/2503.01063
• Copy Paste: [[2503.01063]] AI-Invented Tonal Languages: Preventing a Machine Lingua Franca Beyond Human Understanding(https://arxiv.org/abs/2503.01063)
• Keywords: large language model
• Abstract: This paper investigates the potential for large language models (LLMs) to develop private tonal languages for machine-to-machine (M2M) communication. Inspired by cryptophasia in human twins (affecting up to 50% of twin births) and natural tonal languages like Mandarin and Vietnamese, we implement a precise character-to-frequency mapping system that encodes the full ASCII character set (32-126) using musical semitones. Each character is assigned a unique frequency, creating a logarithmic progression beginning with space (220 Hz) and ending with tilde (50,175.42 Hz). This spans approximately 7.9 octaves, with higher characters deliberately mapped to ultrasonic frequencies beyond human perception (>20 kHz). Our implemented software prototype demonstrates this encoding through visualization, auditory playback, and ABC musical notation, allowing for analysis of information density and transmission speed. Testing reveals that tonal encoding can achieve information rates exceeding human speech while operating partially outside human perceptual boundaries. This work responds directly to concerns about AI systems catastrophically developing private languages within the next five years, providing a concrete prototype software example of how such communication might function and the technical foundation required for its emergence, detection, and governance.

Title: Scientific Reasoning: Assessment of Multimodal Generative LLMs

• Authors: Florian Dreyer, Ekaterina Kolos, Daria Matiash
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01064
• Pdf URL: https://arxiv.org/pdf/2503.01064
• Copy Paste: [[2503.01064]] Scientific Reasoning: Assessment of Multimodal Generative LLMs(https://arxiv.org/abs/2503.01064)
• Keywords: generative, large language model
• Abstract: Large language models (LLMs) can answer questions and reason about complex tasks, also from the scientific domain. We assess several multimodal LLMs (MLLMs) on ScienceQA and find that Gemini models show the highest accuracy with little context, and the highest textual similarity to human explanations with richer context. Adapter-tuning of smaller MLLMs did not lead to any reliable performance. Training from Gemini outputs consistently underperformed training from the original data.

Title: Active Learning for Direct Preference Optimization

• Authors: Branislav Kveton, Xintong Li, Julian McAuley, Ryan Rossi, Jingbo Shang, Junda Wu, Tong Yu
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01076
• Pdf URL: https://arxiv.org/pdf/2503.01076
• Copy Paste: [[2503.01076]] Active Learning for Direct Preference Optimization(https://arxiv.org/abs/2503.01076)
• Keywords: large language model
• Abstract: Direct preference optimization (DPO) is a form of reinforcement learning from human feedback (RLHF) where the policy is learned directly from preferential feedback. Although many models of human preferences exist, the critical task of selecting the most informative feedback for training them is under-explored. We propose an active learning framework for DPO, which can be applied to collect human feedback online or to choose the most informative subset of already collected feedback offline. We propose efficient algorithms for both settings. The key idea is to linearize the DPO objective at the last layer of the neural network representation of the optimized policy and then compute the D-optimal design to collect preferential feedback. We prove that the errors in our DPO logit estimates diminish with more feedback. We show the effectiveness of our algorithms empirically in the setting that matches our theory and also on large language models.

Title: Depth-Adaptive Graph Neural Networks via Learnable Bakry-'Emery Curvature

• Authors: Asela Hevapathige, Ahad N. Zehmakan, Qing Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01079
• Pdf URL: https://arxiv.org/pdf/2503.01079
• Copy Paste: [[2503.01079]] Depth-Adaptive Graph Neural Networks via Learnable Bakry-'Emery Curvature(https://arxiv.org/abs/2503.01079)
• Keywords: diffusion
• Abstract: Graph Neural Networks (GNNs) have demonstrated strong representation learning capabilities for graph-based tasks. Recent advances on GNNs leverage geometric properties, such as curvature, to enhance its representation capabilities by modeling complex connectivity patterns and information flow within graphs. However, most existing approaches focus solely on discrete graph topology, overlooking diffusion dynamics and task-specific dependencies essential for effective learning. To address this, we propose integrating Bakry-Émery curvature, which captures both structural and task-driven aspects of information propagation. We develop an efficient, learnable approximation strategy, making curvature computation scalable for large graphs. Furthermore, we introduce an adaptive depth mechanism that dynamically adjusts message-passing layers per vertex based on its curvature, ensuring efficient propagation. Our theoretical analysis establishes a link between curvature and feature distinctiveness, showing that high-curvature vertices require fewer layers, while low-curvature ones benefit from deeper propagation. Extensive experiments on benchmark datasets validate the effectiveness of our approach, showing consistent performance improvements across diverse graph learning tasks.

Title: Efficient or Powerful? Trade-offs Between Machine Learning and Deep Learning for Mental Illness Detection on Social Media

• Authors: Zhanyi Ding, Zhongyan Wang, Yeyubei Zhang, Yuchen Cao, Yunchong Liu, Xiaorui Shen, Yexin Tian, Jianglai Dai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01082
• Pdf URL: https://arxiv.org/pdf/2503.01082
• Copy Paste: [[2503.01082]] Efficient or Powerful? Trade-offs Between Machine Learning and Deep Learning for Mental Illness Detection on Social Media(https://arxiv.org/abs/2503.01082)
• Keywords: robust, interpretability
• Abstract: Social media platforms provide valuable insights into mental health trends by capturing user-generated discussions on conditions such as depression, anxiety, and suicidal ideation. Machine learning (ML) and deep learning (DL) models have been increasingly applied to classify mental health conditions from textual data, but selecting the most effective model involves trade-offs in accuracy, interpretability, and computational efficiency. This study evaluates multiple ML models, including logistic regression, random forest, and LightGBM, alongside deep learning architectures such as ALBERT and Gated Recurrent Units (GRUs), for both binary and multi-class classification of mental health conditions. Our findings indicate that ML and DL models achieve comparable classification performance on medium-sized datasets, with ML models offering greater interpretability through variable importance scores, while DL models are more robust to complex linguistic patterns. Additionally, ML models require explicit feature engineering, whereas DL models learn hierarchical representations directly from text. Logistic regression provides the advantage of capturing both positive and negative associations between features and mental health conditions, whereas tree-based models prioritize decision-making power through split-based feature selection. This study offers empirical insights into the advantages and limitations of different modeling approaches and provides recommendations for selecting appropriate methods based on dataset size, interpretability needs, and computational constraints.

Title: Identity documents recognition and detection using semantic segmentation with convolutional neural network

• Authors: Mykola Kozlenko, Volodymyr Sendetskyi, Oleksiy Simkiv, Nazar Savchenko, Andy Bosyi
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01085
• Pdf URL: https://arxiv.org/pdf/2503.01085
• Copy Paste: [[2503.01085]] Identity documents recognition and detection using semantic segmentation with convolutional neural network(https://arxiv.org/abs/2503.01085)
• Keywords: security, segmentation
• Abstract: Object recognition and detection are well-studied problems with a developed set of almost standard solutions. Identity documents recognition, classification, detection, and localization are the tasks required in a number of applications, particularly, in physical access control security systems at critical infrastructure premises. In this paper, we propose the new original architecture of a model based on an artificial convolutional neural network and semantic segmentation approach for the recognition and detection of identity documents in images. The challenge with the processing of such images is the limited computational performance and the limited amount of memory when such an application is running on industrial oneboard microcomputer hardware. The aim of this research is to prove the feasibility of the proposed technique and to obtain quality metrics. The methodology of the research is to evaluate the deep learning detection model trained on the mobile identity document video dataset. The dataset contains five hundred video clips for fifty different identity document types. The numerical results from simulations are used to evaluate the quality metrics. We present the results as accuracy versus threshold of the intersection over union value. The paper reports an accuracy above 0.75 for the intersection over union (IoU) threshold value of 0.8. Besides, we assessed the size of the model and proved the feasibility of running the model on an industrial one-board microcomputer or smartphone hardware.

Title: Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time

• Authors: Jon Donnelly, Zhicheng Guo, Alina Jade Barnett, Hayden McTavish, Chaofan Chen, Cynthia Rudin
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01087
• Pdf URL: https://arxiv.org/pdf/2503.01087
• Copy Paste: [[2503.01087]] Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time(https://arxiv.org/abs/2503.01087)
• Keywords: interpretability
• Abstract: Interpretability is critical for machine learning models in high-stakes settings because it allows users to verify the model's reasoning. In computer vision, prototypical part models (ProtoPNets) have become the dominant model type to meet this need. Users can easily identify flaws in ProtoPNets, but fixing problems in a ProtoPNet requires slow, difficult retraining that is not guaranteed to resolve the issue. This problem is called the "interaction bottleneck." We solve the interaction bottleneck for ProtoPNets by simultaneously finding many equally good ProtoPNets (i.e., a draw from a "Rashomon set"). We show that our framework - called Proto-RSet - quickly produces many accurate, diverse ProtoPNets, allowing users to correct problems in real time while maintaining performance guarantees with respect to the training set. We demonstrate the utility of this method in two settings: 1) removing synthetic bias introduced to a bird identification model and 2) debugging a skin cancer identification model. This tool empowers non-machine-learning experts, such as clinicians or domain experts, to quickly refine and correct machine learning models without repeated retraining by machine learning experts.

Title: Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction

• Authors: Nazmul Islam, Mohammad Zulkernine
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.01089
• Pdf URL: https://arxiv.org/pdf/2503.01089
• Copy Paste: [[2503.01089]] Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction(https://arxiv.org/abs/2503.01089)
• Keywords: privacy
• Abstract: Machine learning (ML) has revolutionized Internet of Vehicles (IoV) applications by enhancing intelligent transportation, autonomous driving capabilities, and various connected services within a large, heterogeneous network. However, the increased connectivity and massive data exchange for ML applications introduce significant privacy challenges. Privacy-preserving machine learning (PPML) offers potential solutions to address these challenges by preserving privacy at various stages of the ML pipeline. Despite the rapid development of ML-based IoV applications and the growing data privacy concerns, there are limited comprehensive studies on the adoption of PPML within this domain. Therefore, this study provides a comprehensive review of the fundamentals, recent advancements, and the challenges of integrating PPML into IoV applications. To conduct an extensive study, we first review existing surveys of various PPML techniques and their integration into IoV across different scopes. We then discuss the fundamentals of IoV and propose a four-layer IoV architecture. Additionally, we categorize IoV applications into three key domains and analyze the privacy challenges in leveraging ML for these application domains. Next, we provide an overview of various PPML techniques, highlighting their applicability and performance to address the privacy challenges. Building on these fundamentals, we thoroughly review recent advancements in integrating various PPML techniques within IoV applications, discussing their frameworks, key features, and performance evaluation in terms of privacy, utility, and efficiency. Finally, we identify current challenges and propose future research directions to enhance privacy and reliability in IoV applications.

Title: Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs

• Authors: Haowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi, Xun Yang, Juanzi Li, Meng Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01090
• Pdf URL: https://arxiv.org/pdf/2503.01090
• Copy Paste: [[2503.01090]] Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs(https://arxiv.org/abs/2503.01090)
• Keywords: large language model
• Abstract: Knowledge editing aims to update outdated information in Large Language Models (LLMs). A representative line of study is locate-then-edit methods, which typically employ causal tracing to identify the modules responsible for recalling factual knowledge about entities. However, we find these methods are often sensitive only to changes in the subject entity, leaving them less effective at adapting to changes in relations. This limitation results in poor editing locality, which can lead to the persistence of irrelevant or inaccurate facts, ultimately compromising the reliability of LLMs. We believe this issue arises from the insufficient precision of knowledge localization. To address this, we propose a Fine-grained Neuron-level Knowledge Editing (FiNE) method that enhances editing locality without affecting overall success rates. By precisely identifying and modifying specific neurons within feed-forward networks, FiNE significantly improves knowledge localization and editing. Quantitative experiments demonstrate that FiNE efficiently achieves better overall performance compared to existing techniques, providing new insights into the localization and modification of knowledge within LLMs.

Title: One-Shot Affordance Grounding of Deformable Objects in Egocentric Organizing Scenes

• Authors: Wanjun Jia, Fan Yang, Mengfei Duan, Xianchi Chen, Yinxi Wang, Yiming Jiang, Wenrui Chen, Kailun Yang, Zhiyong Li
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.01092
• Pdf URL: https://arxiv.org/pdf/2503.01092
• Copy Paste: [[2503.01092]] One-Shot Affordance Grounding of Deformable Objects in Egocentric Organizing Scenes(https://arxiv.org/abs/2503.01092)
• Keywords: extraction
• Abstract: Deformable object manipulation in robotics presents significant challenges due to uncertainties in component properties, diverse configurations, visual interference, and ambiguous prompts. These factors complicate both perception and control tasks. To address these challenges, we propose a novel method for One-Shot Affordance Grounding of Deformable Objects (OS-AGDO) in egocentric organizing scenes, enabling robots to recognize previously unseen deformable objects with varying colors and shapes using minimal samples. Specifically, we first introduce the Deformable Object Semantic Enhancement Module (DefoSEM), which enhances hierarchical understanding of the internal structure and improves the ability to accurately identify local features, even under conditions of weak component information. Next, we propose the ORB-Enhanced Keypoint Fusion Module (OEKFM), which optimizes feature extraction of key components by leveraging geometric constraints and improves adaptability to diversity and visual interference. Additionally, we propose an instance-conditional prompt based on image data and task context, effectively mitigates the issue of region ambiguity caused by prompt words. To validate these methods, we construct a diverse real-world dataset, AGDDO15, which includes 15 common types of deformable objects and their associated organizational actions. Experimental results demonstrate that our approach significantly outperforms state-of-the-art methods, achieving improvements of 6.2%, 3.2%, and 2.9% in KLD, SIM, and NSS metrics, respectively, while exhibiting high generalization performance. Source code and benchmark dataset will be publicly available at this https URL.

Title: Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator

• Authors: Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yu Liu, Jun Zhu, Qinsheng Zhang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01103
• Pdf URL: https://arxiv.org/pdf/2503.01103
• Copy Paste: [[2503.01103]] Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator(https://arxiv.org/abs/2503.01103)
• Keywords: diffusion, generative
• Abstract: While likelihood-based generative models, particularly diffusion and autoregressive models, have achieved remarkable fidelity in visual generation, the maximum likelihood estimation (MLE) objective inherently suffers from a mode-covering tendency that limits the generation quality under limited model capacity. In this work, we propose Direct Discriminative Optimization (DDO) as a unified framework that bridges likelihood-based generative training and the GAN objective to bypass this fundamental constraint. Our key insight is to parameterize a discriminator implicitly using the likelihood ratio between a learnable target model and a fixed reference model, drawing parallels with the philosophy of Direct Preference Optimization (DPO). Unlike GANs, this parameterization eliminates the need for joint training of generator and discriminator networks, allowing for direct, efficient, and effective finetuning of a well-trained model to its full potential beyond the limits of MLE. DDO can be performed iteratively in a self-play manner for progressive model refinement, with each round requiring less than 1% of pretraining epochs. Our experiments demonstrate the effectiveness of DDO by significantly advancing the previous SOTA diffusion model EDM, reducing FID scores from 1.79/1.58 to new records of 1.30/0.97 on CIFAR-10/ImageNet-64 datasets, and by consistently improving both guidance-free and CFG-enhanced FIDs of visual autoregressive models on ImageNet 256$\times$256.

Title: VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors

• Authors: Juil Koo, Paul Guerrero, Chun-Hao Paul Huang, Duygu Ceylan, Minhyuk Sung
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01107
• Pdf URL: https://arxiv.org/pdf/2503.01107
• Copy Paste: [[2503.01107]] VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors(https://arxiv.org/abs/2503.01107)
• Keywords: generative
• Abstract: Generative methods for image and video editing use generative models as priors to perform edits despite incomplete information, such as changing the composition of 3D objects shown in a single image. Recent methods have shown promising composition editing results in the image setting, but in the video setting, editing methods have focused on editing object's appearance and motion, or camera motion, and as a result, methods to edit object composition in videos are still missing. We propose \name as a method for editing 3D object compositions in videos of static scenes with camera motion. Our approach allows editing the 3D position of a 3D object across all frames of a video in a temporally consistent manner. This is achieved by lifting intermediate features of a generative model to a 3D reconstruction that is shared between all frames, editing the reconstruction, and projecting the features on the edited reconstruction back to each frame. To the best of our knowledge, this is the first generative approach to edit object compositions in videos. Our approach is simple and training-free, while outperforming state-of-the-art image editing baselines.

Title: SCSegamba: Lightweight Structure-Aware Vision Mamba for Crack Segmentation in Structures

• Authors: Hui Liu, Chen Jia, Fan Shi, Xu Cheng, Shengyong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01113
• Pdf URL: https://arxiv.org/pdf/2503.01113
• Copy Paste: [[2503.01113]] SCSegamba: Lightweight Structure-Aware Vision Mamba for Crack Segmentation in Structures(https://arxiv.org/abs/2503.01113)
• Keywords: segmentation
• Abstract: Pixel-level segmentation of structural cracks across various scenarios remains a considerable challenge. Current methods encounter challenges in effectively modeling crack morphology and texture, facing challenges in balancing segmentation quality with low computational resource usage. To overcome these limitations, we propose a lightweight Structure-Aware Vision Mamba Network (SCSegamba), capable of generating high-quality pixel-level segmentation maps by leveraging both the morphological information and texture cues of crack pixels with minimal computational cost. Specifically, we developed a Structure-Aware Visual State Space module (SAVSS), which incorporates a lightweight Gated Bottleneck Convolution (GBC) and a Structure-Aware Scanning Strategy (SASS). The key insight of GBC lies in its effectiveness in modeling the morphological information of cracks, while the SASS enhances the perception of crack topology and texture by strengthening the continuity of semantic information between crack pixels. Experiments on crack benchmark datasets demonstrate that our method outperforms other state-of-the-art (SOTA) methods, achieving the highest performance with only 2.8M parameters. On the multi-scenario dataset, our method reached 0.8390 in F1 score and 0.8479 in mIoU. The code is available at this https URL.

Title: WeGen: A Unified Model for Interactive Multimodal Generation as We Chat

• Authors: Zhipeng Huang, Shaobin Zhuang, Canmiao Fu, Binxin Yang, Ying Zhang, Chong Sun, Zhizheng Zhang, Yali Wang, Chen Li, Zheng-Jun Zha
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01115
• Pdf URL: https://arxiv.org/pdf/2503.01115
• Copy Paste: [[2503.01115]] WeGen: A Unified Model for Interactive Multimodal Generation as We Chat(https://arxiv.org/abs/2503.01115)
• Keywords: generative
• Abstract: Existing multimodal generative models fall short as qualified design copilots, as they often struggle to generate imaginative outputs once instructions are less detailed or lack the ability to maintain consistency with the provided references. In this work, we introduce WeGen, a model that unifies multimodal generation and understanding, and promotes their interplay in iterative generation. It can generate diverse results with high creativity for less detailed instructions. And it can progressively refine prior generation results or integrating specific contents from references following the instructions in its chat with users. During this process, it is capable of preserving consistency in the parts that the user is already satisfied with. To this end, we curate a large-scale dataset, extracted from Internet videos, containing rich object dynamics and auto-labeled dynamics descriptions by advanced foundation models to date. These two information are interleaved into a single sequence to enable WeGen to learn consistency-aware generation where the specified dynamics are generated while the consistency of unspecified content is preserved aligned with instructions. Besides, we introduce a prompt self-rewriting mechanism to enhance generation diversity. Extensive experiments demonstrate the effectiveness of unifying multimodal understanding and generation in WeGen and show it achieves state-of-the-art performance across various visual generation benchmarks. These also demonstrate the potential of WeGen as a user-friendly design copilot as desired. The code and models will be available at this https URL.

Title: ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization

• Authors: Shizhan Liu, Hao Zheng, Hang Yu, Jianguo Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01122
• Pdf URL: https://arxiv.org/pdf/2503.01122
• Copy Paste: [[2503.01122]] ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization(https://arxiv.org/abs/2503.01122)
• Keywords: diffusion
• Abstract: Image personalization has garnered attention for its ability to customize Text-to-Image generation using only a few reference images. However, a key challenge in image personalization is the issue of conceptual coupling, where the limited number of reference images leads the model to form unwanted associations between the personalization target and other concepts. Current methods attempt to tackle this issue indirectly, leading to a suboptimal balance between text control and personalization fidelity. In this paper, we take a direct approach to the concept coupling problem through statistical analysis, revealing that it stems from two distinct sources of dependence discrepancies. We therefore propose two complementary plug-and-play loss functions: Denoising Decouple Loss and Prior Decouple loss, each designed to minimize one type of dependence discrepancy. Extensive experiments demonstrate that our approach achieves a superior trade-off between text control and personalization fidelity.

Title: ViKANformer: Embedding Kolmogorov Arnold Networks in Vision Transformers for Pattern-Based Learning

• Authors: Shreyas S, Akshath M
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01124
• Pdf URL: https://arxiv.org/pdf/2503.01124
• Copy Paste: [[2503.01124]] ViKANformer: Embedding Kolmogorov Arnold Networks in Vision Transformers for Pattern-Based Learning(https://arxiv.org/abs/2503.01124)
• Keywords: transformer
• Abstract: Vision Transformers (ViTs) have significantly advanced image classification by applying self-attention on patch embeddings. However, the standard MLP blocks in each Transformer layer may not capture complex nonlinear dependencies optimally. In this paper, we propose ViKANformer, a Vision Transformer where we replace the MLP sub-layers with Kolmogorov-Arnold Network (KAN) expansions, including Vanilla KAN, Efficient-KAN, Fast-KAN, SineKAN, and FourierKAN, while also examining a Flash Attention variant. By leveraging the Kolmogorov-Arnold theorem, which guarantees that multivariate continuous functions can be expressed via sums of univariate continuous functions, we aim to boost representational power. Experimental results on MNIST demonstrate that SineKAN, Fast-KAN, and a well-tuned Vanilla KAN can achieve over 97% accuracy, albeit with increased training overhead. This trade-off highlights that KAN expansions may be beneficial if computational cost is acceptable. We detail the expansions, present training/test accuracy and F1/ROC metrics, and provide pseudocode and hyperparameters for reproducibility. Finally, we compare ViKANformer to a simple MLP and a small CNN baseline on MNIST, illustrating the efficiency of Transformer-based methods even on a small-scale dataset.

Title: AirRoom: Objects Matter in Room Reidentification

• Authors: Runmao Yao, Yi Du, Zhuoqun Chen, Haoze Zheng, Chen Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01130
• Pdf URL: https://arxiv.org/pdf/2503.01130
• Copy Paste: [[2503.01130]] AirRoom: Objects Matter in Room Reidentification(https://arxiv.org/abs/2503.01130)
• Keywords: robust, segmentation
• Abstract: Room reidentification (ReID) is a challenging yet essential task with numerous applications in fields such as augmented reality (AR) and homecare robotics. Existing visual place recognition (VPR) methods, which typically rely on global descriptors or aggregate local features, often struggle in cluttered indoor environments densely populated with man-made objects. These methods tend to overlook the crucial role of object-oriented information. To address this, we propose AirRoom, an object-aware pipeline that integrates multi-level object-oriented information-from global context to object patches, object segmentation, and keypoints-utilizing a coarse-to-fine retrieval approach. Extensive experiments on four newly constructed datasets-MPReID, HMReID, GibsonReID, and ReplicaReID-demonstrate that AirRoom outperforms state-of-the-art (SOTA) models across nearly all evaluation metrics, with improvements ranging from 6% to 80%. Moreover, AirRoom exhibits significant flexibility, allowing various modules within the pipeline to be substituted with different alternatives without compromising overall performance. It also shows robust and consistent performance under diverse viewpoint variations.

Title: Beyond QA Pairs: Assessing Parameter-Efficient Fine-Tuning for Fact Embedding in LLMs

• Authors: Shivam Ratnakar, Abhiroop Talasila, Raghav Chamadiya, Nikhil Agarwal, Vinayak K Doifode
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01131
• Pdf URL: https://arxiv.org/pdf/2503.01131
• Copy Paste: [[2503.01131]] Beyond QA Pairs: Assessing Parameter-Efficient Fine-Tuning for Fact Embedding in LLMs(https://arxiv.org/abs/2503.01131)
• Keywords: large language model
• Abstract: This paper presents an extensive examination of Parameter-Efficient Fine-Tuning (PEFT) for embedding domain specific facts into Large Language Models (LLMs), focusing on improving the fine-tuning process by categorizing question-answer (QA) pairs into Factual and Conceptual classes using a BERT-based classifier. Two distinct Llama-2 models are fine-tuned based on these classifications and evaluated using larger models like GPT-3.5 Turbo and Gemini. Our results indicate that models trained on conceptual datasets outperform those trained on factual datasets. Additionally, we compare the efficiency of two synthetic fine-tuning dataset generation techniques, D-RAG and D-Naive, with D-Naive demonstrating superior performance. Although PEFT has shown effectiveness, our research indicates that it may not be the most optimal method for embedding facts into LLMs. However, it has demonstrated exceptional performance in instruction-based tasks. Our findings are reinforced by a 1000-sample dataset in the data center domain, where the fine-tuned Llama-2 7B model significantly outperforms the baseline model in generating product recommendations. Our study highlights the importance of QA pair categorization and synthetic dataset generation techniques in enhancing the performance of LLMs in specific domains.

Title: Prior-guided Hierarchical Harmonization Network for Efficient Image Dehazing

• Authors: Xiongfei Su, Siyuan Li, Yuning Cui, Miao Cao, Yulun Zhang, Zheng Chen, Zongliang Wu, Zedong Wang, Yuanlong Zhang, Xin Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01136
• Pdf URL: https://arxiv.org/pdf/2503.01136
• Copy Paste: [[2503.01136]] Prior-guided Hierarchical Harmonization Network for Efficient Image Dehazing(https://arxiv.org/abs/2503.01136)
• Keywords: transformer
• Abstract: Image dehazing is a crucial task that involves the enhancement of degraded images to recover their sharpness and textures. While vision Transformers have exhibited impressive results in diverse dehazing tasks, their quadratic complexity and lack of dehazing priors pose significant drawbacks for real-world applications. In this paper, guided by triple priors, Bright Channel Prior (BCP), Dark Channel Prior (DCP), and Histogram Equalization (HE), we propose a \textit{P}rior-\textit{g}uided Hierarchical \textit{H}armonization Network (PGH$^2$Net) for image dehazing. PGH$^2$Net is built upon the UNet-like architecture with an efficient encoder and decoder, consisting of two module types: (1) Prior aggregation module that injects B/DCP and selects diverse contexts with gating attention. (2) Feature harmonization modules that subtract low-frequency components from spatial and channel aspects and learn more informative feature distributions to equalize the feature maps.

Title: How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach

• Authors: Ayeong Lee, Ethan Che, Tianyi Peng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01141
• Pdf URL: https://arxiv.org/pdf/2503.01141
• Copy Paste: [[2503.01141]] How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach(https://arxiv.org/abs/2503.01141)
• Keywords: large language model
• Abstract: Chain-of-thought prompting has emerged as a powerful technique for enabling large language models (LLMs) to solve complex reasoning tasks. However, these reasoning chains can be verbose, raising concerns about efficiency. In response, recent works have sought to decrease response lengths through simple prompting strategies (e.g. 'be concise'). In this work, we conduct the first systematic study of the relationship between reasoning length and model performance across a diverse range of compression instructions (e.g. 'use 10 words or less' or 'remove all punctuation'). In doing so, we discover a universal tradeoff between reasoning length and accuracy that persists across even very distinct reasoning chains. We demonstrate that this tradeoff emerges from a sharp threshold behavior at the question level: each task has an intrinsic 'token complexity' - a minimal number of tokens required for successful problem-solving. We show how token complexity enables us to compute information-theoretic limits on the accuracy-compression tradeoff, and find that prompt-based compression strategies operate far from these theoretical limits. This suggests there may be significant room for improvement and our framework provides a benchmark to help researchers evaluate progress in reasoning efficiency. Our work also highlights the importance of adaptive compression -- giving shorter responses for easier questions -- and we show that token complexity is a useful tool for measuring this capability.

Title: DPR: Diffusion Preference-based Reward for Offline Reinforcement Learning

• Authors: Teng Pang, Bingzheng Wang, Guoqiang Wu, Yilong Yin
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01143
• Pdf URL: https://arxiv.org/pdf/2503.01143
• Copy Paste: [[2503.01143]] DPR: Diffusion Preference-based Reward for Offline Reinforcement Learning(https://arxiv.org/abs/2503.01143)
• Keywords: diffusion, transformer
• Abstract: Offline preference-based reinforcement learning (PbRL) mitigates the need for reward definition, aligning with human preferences via preference-driven reward feedback without interacting with the environment. However, the effectiveness of preference-driven reward functions depends on the modeling ability of the learning model, which current MLP-based and Transformer-based methods may fail to adequately provide. To alleviate the failure of the reward function caused by insufficient modeling, we propose a novel preference-based reward acquisition method: Diffusion Preference-based Reward (DPR). Unlike previous methods using Bradley-Terry models for trajectory preferences, we use diffusion models to directly model preference distributions for state-action pairs, allowing rewards to be discriminatively obtained from these distributions. In addition, considering the particularity of preference data that only know the internal relationships of paired trajectories, we further propose Conditional Diffusion Preference-based Reward (C-DPR), which leverages relative preference information to enhance the construction of the diffusion model. We apply the above methods to existing offline reinforcement learning algorithms and a series of experiment results demonstrate that the diffusion-based reward acquisition approach outperforms previous MLP-based and Transformer-based methods.

Title: One-shot In-context Part Segmentation

• Authors: Zhenqi Dai, Ting Liu, Xingxing Zhang, Yunchao Wei, Yanning Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01144
• Pdf URL: https://arxiv.org/pdf/2503.01144
• Copy Paste: [[2503.01144]] One-shot In-context Part Segmentation(https://arxiv.org/abs/2503.01144)
• Keywords: diffusion, segmentation
• Abstract: In this paper, we present the One-shot In-context Part Segmentation (OIParts) framework, designed to tackle the challenges of part segmentation by leveraging visual foundation models (VFMs). Existing training-based one-shot part segmentation methods that utilize VFMs encounter difficulties when faced with scenarios where the one-shot image and test image exhibit significant variance in appearance and perspective, or when the object in the test image is partially visible. We argue that training on the one-shot example often leads to overfitting, thereby compromising the model's generalization capability. Our framework offers a novel approach to part segmentation that is training-free, flexible, and data-efficient, requiring only a single in-context example for precise segmentation with superior generalization ability. By thoroughly exploring the complementary strengths of VFMs, specifically DINOv2 and Stable Diffusion, we introduce an adaptive channel selection approach by minimizing the intra-class distance for better exploiting these two features, thereby enhancing the discriminatory power of the extracted features for the fine-grained parts. We have achieved remarkable segmentation performance across diverse object categories. The OIParts framework not only eliminates the need for extensive labeled data but also demonstrates superior generalization ability. Through comprehensive experimentation on three benchmark datasets, we have demonstrated the superiority of our proposed method over existing part segmentation approaches in one-shot settings.

Title: CoInD: Enabling Logical Compositions in Diffusion Models

• Authors: Sachit Gaudi, Gautam Sreekumar, Vishnu Boddeti
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01145
• Pdf URL: https://arxiv.org/pdf/2503.01145
• Copy Paste: [[2503.01145]] CoInD: Enabling Logical Compositions in Diffusion Models(https://arxiv.org/abs/2503.01145)
• Keywords: diffusion, generative
• Abstract: How can we learn generative models to sample data with arbitrary logical compositions of statistically independent attributes? The prevailing solution is to sample from distributions expressed as a composition of attributes' conditional marginal distributions under the assumption that they are statistically independent. This paper shows that standard conditional diffusion models violate this assumption, even when all attribute compositions are observed during training. And, this violation is significantly more severe when only a subset of the compositions is observed. We propose CoInD to address this problem. It explicitly enforces statistical independence between the conditional marginal distributions by minimizing Fisher's divergence between the joint and marginal distributions. The theoretical advantages of CoInD are reflected in both qualitative and quantitative experiments, demonstrating a significantly more faithful and controlled generation of samples for arbitrary logical compositions of attributes. The benefit is more pronounced for scenarios that current solutions relying on the assumption of conditionally independent marginals struggle with, namely, logical compositions involving the NOT operation and when only a subset of compositions are observed during training.

Title: MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages

• Authors: Chen Zhang, Mingxu Tao, Zhiyuan Liao, Yansong Feng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01150
• Pdf URL: https://arxiv.org/pdf/2503.01150
• Copy Paste: [[2503.01150]] MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages(https://arxiv.org/abs/2503.01150)
• Keywords: large language model
• Abstract: Large language models (LLMs) excel in high-resource languages but struggle with low-resource languages (LRLs), particularly those spoken by minority communities in China, such as Tibetan, Uyghur, Kazakh, and Mongolian. To systematically track the progress in these languages, we introduce MiLiC-Eval, a benchmark designed for minority languages in China, featuring 24K instances across 9 tasks. MiLiC-Eval focuses on underrepresented writing systems and provides a fine-grained assessment of linguistic and problem-solving skills. Our evaluation reveals that LLMs perform poorly on syntax-intensive tasks and multi-script languages. We further demonstrate how MiLiC-Eval can help advance LRL research in handling diverse writing systems and understanding the process of language adaptation.

Title: ReaderLM-v2: Small Language Model for HTML to Markdown and JSON

• Authors: Feng Wang, Zesheng Shi, Bo Wang, Nan Wang, Han Xiao
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01151
• Pdf URL: https://arxiv.org/pdf/2503.01151
• Copy Paste: [[2503.01151]] ReaderLM-v2: Small Language Model for HTML to Markdown and JSON(https://arxiv.org/abs/2503.01151)
• Keywords: extraction, large language model
• Abstract: We present ReaderLM-v2, a compact 1.5 billion parameter language model designed for efficient web content extraction. Our model processes documents up to 512K tokens, transforming messy HTML into clean Markdown or JSON formats with high accuracy -- making it an ideal tool for grounding large language models. The model's effectiveness results from two key innovations: (1) a three-stage data synthesis pipeline that generates high quality, diverse training data by iteratively drafting, refining, and critiquing web content extraction; and (2) a unified training framework combining continuous pre-training with multi-objective optimization. Intensive evaluation demonstrates that ReaderLM-v2 outperforms GPT-4o-2024-08-06 and other larger models by 15-20\% on carefully curated benchmarks, particularly excelling at documents exceeding 100K tokens, while maintaining significantly lower computational requirements.

Title: Nature-Inspired Population-Based Evolution of Large Language Models

• Authors: Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, Shuyue Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01155
• Pdf URL: https://arxiv.org/pdf/2503.01155
• Copy Paste: [[2503.01155]] Nature-Inspired Population-Based Evolution of Large Language Models(https://arxiv.org/abs/2503.01155)
• Keywords: large language model
• Abstract: Evolution, the engine behind the survival and growth of life on Earth, operates through the population-based process of reproduction. Inspired by this principle, this paper formally defines a newly emerging problem -- the population-based evolution of large language models (LLMs) -- and introduces a novel framework. Starting with a population of parent LLMs, our framework enables the population to evolve through four key operations: (i) crossover, merging the weights of different parents to create offspring LLMs, (ii) mutation, introducing small, random changes to model weights to foster diversity, (iii) selection, prioritizing high-performing models, and (iv) succession, transferring the learned experience from parent to offspring LLMs. With only 200 samples per new task, the LLM population evolves rapidly to adapt to the task at hand, without any gradients. Experiments on 12 datasets show that our framework consistently outperforms existing multi-LLM merging and adaptation methods, achieving accuracy gains of up to 54.8% over the best LLM in the initial population. Moreover, our framework allows for the evolution of LLMs across multiple new tasks simultaneously, scaling effectively with populations of up to 40 LLMs, and even zero-shot generalization to unseen held-out tasks. We have open-sourced the code on GitHub and released the weights of 10 parent LLMs, fine-tuned from gemma-2-2b-it, on HuggingFace$, enabling reproduction of our proposed framework using just a single 4090 GPU with 24GB memory, without any performance degradation.

Title: Unify and Anchor: A Context-Aware Transformer for Cross-Domain Time Series Forecasting

• Authors: Xiaobin Hong, Jiawen Zhang, Wenzhong Li, Sanglu Lu, Jia Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01157
• Pdf URL: https://arxiv.org/pdf/2503.01157
• Copy Paste: [[2503.01157]] Unify and Anchor: A Context-Aware Transformer for Cross-Domain Time Series Forecasting(https://arxiv.org/abs/2503.01157)
• Keywords: transformer
• Abstract: The rise of foundation models has revolutionized natural language processing and computer vision, yet their best practices to time series forecasting remains underexplored. Existing time series foundation models often adopt methodologies from these fields without addressing the unique characteristics of time series data. In this paper, we identify two key challenges in cross-domain time series forecasting: the complexity of temporal patterns and semantic misalignment. To tackle these issues, we propose the ``Unify and Anchor" transfer paradigm, which disentangles frequency components for a unified perspective and incorporates external context as domain anchors for guided adaptation. Based on this framework, we introduce ContexTST, a Transformer-based model that employs a time series coordinator for structured representation and the Transformer blocks with a context-informed mixture-of-experts mechanism for effective cross-domain generalization. Extensive experiments demonstrate that ContexTST advances state-of-the-art forecasting performance while achieving strong zero-shot transferability across diverse domains.

Title: EasyCraft: A Robust and Efficient Framework for Automatic Avatar Crafting

• Authors: Suzhen Wang, Weijie Chen, Wei Zhang, Minda Zhao, Lincheng Li, Rongsheng Zhang, Zhipeng Hu, Xin Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01158
• Pdf URL: https://arxiv.org/pdf/2503.01158
• Copy Paste: [[2503.01158]] EasyCraft: A Robust and Efficient Framework for Automatic Avatar Crafting(https://arxiv.org/abs/2503.01158)
• Keywords: robust
• Abstract: Character customization, or 'face crafting,' is a vital feature in role-playing games (RPGs), enhancing player engagement by enabling the creation of personalized avatars. Existing automated methods often struggle with generalizability across diverse game engines due to their reliance on the intermediate constraints of specific image domain and typically support only one type of input, either text or image. To overcome these challenges, we introduce EasyCraft, an innovative end-to-end feedforward framework that automates character crafting by uniquely supporting both text and image inputs. Our approach employs a translator capable of converting facial images of any style into crafting parameters. We first establish a unified feature distribution in the translator's image encoder through self-supervised learning on a large-scale dataset, enabling photos of any style to be embedded into a unified feature representation. Subsequently, we map this unified feature distribution to crafting parameters specific to a game engine, a process that can be easily adapted to most game engines and thus enhances EasyCraft's generalizability. By integrating text-to-image techniques with our translator, EasyCraft also facilitates precise, text-based character crafting. EasyCraft's ability to integrate diverse inputs significantly enhances the versatility and accuracy of avatar creation. Extensive experiments on two RPG games demonstrate the effectiveness of our method, achieving state-of-the-art results and facilitating adaptability across various avatar engines.

Title: Large Language Models for Healthcare Text Classification: A Systematic Review

• Authors: Hajar Sakai, Sarah S. Lam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01159
• Pdf URL: https://arxiv.org/pdf/2503.01159
• Copy Paste: [[2503.01159]] Large Language Models for Healthcare Text Classification: A Systematic Review(https://arxiv.org/abs/2503.01159)
• Keywords: privacy, large language model
• Abstract: Large Language Models (LLMs) have fundamentally transformed approaches to Natural Language Processing (NLP) tasks across diverse domains. In healthcare, accurate and cost-efficient text classification is crucial, whether for clinical notes analysis, diagnosis coding, or any other task, and LLMs present promising potential. Text classification has always faced multiple challenges, including manual annotation for training, handling imbalanced data, and developing scalable approaches. With healthcare, additional challenges are added, particularly the critical need to preserve patients' data privacy and the complexity of the medical terminology. Numerous studies have been conducted to leverage LLMs for automated healthcare text classification and contrast the results with existing machine learning-based methods where embedding, annotation, and training are traditionally required. Existing systematic reviews about LLMs either do not specialize in text classification or do not focus on the healthcare domain. This research synthesizes and critically evaluates the current evidence found in the literature regarding the use of LLMs for text classification in a healthcare setting. Major databases (e.g., Google Scholar, Scopus, PubMed, Science Direct) and other resources were queried, which focused on the papers published between 2018 and 2024 within the framework of PRISMA guidelines, which resulted in 65 eligible research articles. These were categorized by text classification type (e.g., binary classification, multi-label classification), application (e.g., clinical decision support, public health and opinion analysis), methodology, type of healthcare text, and metrics used for evaluation and validation. This review reveals the existing gaps in the literature and suggests future research lines that can be investigated and explored.

Title: Split Gibbs Discrete Diffusion Posterior Sampling

• Authors: Wenda Chu, Yang Song, Yisong Yue
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01161
• Pdf URL: https://arxiv.org/pdf/2503.01161
• Copy Paste: [[2503.01161]] Split Gibbs Discrete Diffusion Posterior Sampling(https://arxiv.org/abs/2503.01161)
• Keywords: diffusion
• Abstract: We study the problem of posterior sampling in discrete-state spaces using discrete diffusion models. While posterior sampling methods for continuous diffusion models have achieved remarkable progress, analogous methods for discrete diffusion models remain challenging. In this work, we introduce a principled plug-and-play discrete diffusion posterior sampling algorithm based on split Gibbs sampling, which we call SG-DPS. Our algorithm enables reward-guided generation and solving inverse problems in discrete-state spaces. We demonstrate that SG-DPS converges to the true posterior distribution on synthetic benchmarks, and enjoys state-of-the-art posterior sampling performance on a range of benchmarks for discrete data, achieving up to 2x improved performance compared to existing baselines.

Title: Med-LEGO: Editing and Adapting toward Generalist Medical Image Diagnosis

• Authors: Yitao Zhu, Yuan Yin, Jiaming Li, Mengjie Xu, Zihao Zhao, Honglin Xiong, Sheng Wang, Qian Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01164
• Pdf URL: https://arxiv.org/pdf/2503.01164
• Copy Paste: [[2503.01164]] Med-LEGO: Editing and Adapting toward Generalist Medical Image Diagnosis(https://arxiv.org/abs/2503.01164)
• Keywords: privacy
• Abstract: The adoption of visual foundation models has become a common practice in computer-aided diagnosis (CAD). While these foundation models provide a viable solution for creating generalist medical AI, privacy concerns make it difficult to pre-train or continuously update such models across multiple domains and datasets, leading many studies to focus on specialist models. To address this challenge, we propose Med-LEGO, a training-free framework that enables the seamless integration or updating of a generalist CAD model by combining multiple specialist models, similar to assembling LEGO bricks. Med-LEGO enhances LoRA (low-rank adaptation) by incorporating singular value decomposition (SVD) to efficiently capture the domain expertise of each specialist model with minimal additional parameters. By combining these adapted weights through simple operations, Med-LEGO allows for the easy integration or modification of specific diagnostic capabilities without the need for original data or retraining. Finally, the combined model can be further adapted to new diagnostic tasks, making it a versatile generalist model. Our extensive experiments demonstrate that Med-LEGO outperforms existing methods in both cross-domain and in-domain medical tasks while using only 0.18% of full model parameters. These merged models show better convergence and generalization to new tasks, providing an effective path toward generalist medical AI.

Title: Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data

• Authors: Haoxin Li, Boyang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01167
• Pdf URL: https://arxiv.org/pdf/2503.01167
• Copy Paste: [[2503.01167]] Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data(https://arxiv.org/abs/2503.01167)
• Keywords: generative
• Abstract: Despite impressive advancements in various multimodal tasks, vision-language models (VLMs) still struggle with compositional understanding due to limited exposure to training samples that contain subtle variations within paired examples. With advances in multimodal generative models, a natural solution is to generate synthetic samples with subtle variations for training VLMs. However, generating and training on synthetic samples with subtle variations presents two challenges: difficulty in accurately creating precise variations and inconsistency in cross-modal alignment quality. To address these challenges, we propose SVD-GT (Subtle Variation Data Generation and Training), which integrates image feature injection into a text-to-image generative model to enhance the quality of synthetic variations and employs an adaptive margin loss to differentiate samples using adaptive margins, which help filter out potentially incorrect synthetic samples and focus the learning on informative hard samples. Evaluations on four compositional understanding benchmarks demonstrate that SVD-GT significantly improves the compositionality of VLMs, boosting the average accuracy of CLIP by over 8% across all benchmarks and outperforming state-of-the-art methods by 2% on three benchmarks.

Title: DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolution

• Authors: Xingyuan Li, Zirui Wang, Yang Zou, Zhixin Chen, Jun Ma, Zhiying Jiang, Long Ma, Jinyuan Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01187
• Pdf URL: https://arxiv.org/pdf/2503.01187
• Copy Paste: [[2503.01187]] DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolution(https://arxiv.org/abs/2503.01187)
• Keywords: diffusion, segmentation
• Abstract: Infrared imaging is essential for autonomous driving and robotic operations as a supportive modality due to its reliable performance in challenging environments. Despite its popularity, the limitations of infrared cameras, such as low spatial resolution and complex degradations, consistently challenge imaging quality and subsequent visual tasks. Hence, infrared image super-resolution (IISR) has been developed to address this challenge. While recent developments in diffusion models have greatly advanced this field, current methods to solve it either ignore the unique modal characteristics of infrared imaging or overlook the machine perception requirements. To bridge these gaps, we propose DifIISR, an infrared image super-resolution diffusion model optimized for visual quality and perceptual performance. Our approach achieves task-based guidance for diffusion by injecting gradients derived from visual and perceptual priors into the noise during the reverse process. Specifically, we introduce an infrared thermal spectrum distribution regulation to preserve visual fidelity, ensuring that the reconstructed infrared images closely align with high-resolution images by matching their frequency components. Subsequently, we incorporate various visual foundational models as the perceptual guidance for downstream visual tasks, infusing generalizable perceptual features beneficial for detection and segmentation. As a result, our approach gains superior visual results while attaining State-Of-The-Art downstream task performance. Code is available at this https URL

Title: Enhancing Retinal Vessel Segmentation Generalization via Layout-Aware Generative Modelling

• Authors: Jonathan Fhima, Jan Van Eijgen, Lennert Beeckmans, Thomas Jacobs, Moti Freiman, Luis Filipe Nakayama, Ingeborg Stalmans, Chaim Baskin, Joachim A. Behar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01190
• Pdf URL: https://arxiv.org/pdf/2503.01190
• Copy Paste: [[2503.01190]] Enhancing Retinal Vessel Segmentation Generalization via Layout-Aware Generative Modelling(https://arxiv.org/abs/2503.01190)
• Keywords: diffusion, generative, segmentation
• Abstract: Generalization in medical segmentation models is challenging due to limited annotated datasets and imaging variability. To address this, we propose Retinal Layout-Aware Diffusion (RLAD), a novel diffusion-based framework for generating controllable layout-aware images. RLAD conditions image generation on multiple key layout components extracted from real images, ensuring high structural fidelity while enabling diversity in other components. Applied to retinal fundus imaging, we augmented the training datasets by synthesizing paired retinal images and vessel segmentations conditioned on extracted blood vessels from real images, while varying other layout components such as lesions and the optic disc. Experiments demonstrated that RLAD-generated data improved generalization in retinal vessel segmentation by up to 8.1%. Furthermore, we present REYIA, a comprehensive dataset comprising 586 manually segmented retinal images. To foster reproducibility and drive innovation, both our code and dataset will be made publicly accessible.

Title: Near-infrared Image Deblurring and Event Denoising with Synergistic Neuromorphic Imaging

• Authors: Chao Qu, Shuo Zhu, Yuhang Wang, Zongze Wu, Xiaoyu Chen, Edmund Y.Lam, Jing Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01193
• Pdf URL: https://arxiv.org/pdf/2503.01193
• Copy Paste: [[2503.01193]] Near-infrared Image Deblurring and Event Denoising with Synergistic Neuromorphic Imaging(https://arxiv.org/abs/2503.01193)
• Keywords: robust
• Abstract: The fields of imaging in the nighttime dynamic and other extremely dark conditions have seen impressive and transformative advancements in recent years, partly driven by the rise of novel sensing approaches, e.g., near-infrared (NIR) cameras with high sensitivity and event cameras with minimal blur. However, inappropriate exposure ratios of near-infrared cameras make them susceptible to distortion and blur. Event cameras are also highly sensitive to weak signals at night yet prone to interference, often generating substantial noise and significantly degrading observations and analysis. Herein, we develop a new framework for low-light imaging combined with NIR imaging and event-based techniques, named synergistic neuromorphic imaging, which can jointly achieve NIR image deblurring and event denoising. Harnessing cross-modal features of NIR images and visible events via spectral consistency and higher-order interaction, the NIR images and events are simultaneously fused, enhanced, and bootstrapped. Experiments on real and realistically simulated sequences demonstrate the effectiveness of our method and indicate better accuracy and robustness than other methods in practical scenarios. This study gives impetus to enhance both NIR images and events, which paves the way for high-fidelity low-light imaging and neuromorphic reasoning.

Title: Cancer Type, Stage and Prognosis Assessment from Pathology Reports using LLMs

• Authors: Rachit Saluja, Jacob Rosenthal, Yoav Artzi, David J. Pisapia, Benjamin L. Liechty, Mert R. Sabuncu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01194
• Pdf URL: https://arxiv.org/pdf/2503.01194
• Copy Paste: [[2503.01194]] Cancer Type, Stage and Prognosis Assessment from Pathology Reports using LLMs(https://arxiv.org/abs/2503.01194)
• Keywords: extraction, large language model
• Abstract: Large Language Models (LLMs) have shown significant promise across various natural language processing tasks. However, their application in the field of pathology, particularly for extracting meaningful insights from unstructured medical texts such as pathology reports, remains underexplored and not well quantified. In this project, we leverage state-of-the-art language models, including the GPT family, Mistral models, and the open-source Llama models, to evaluate their performance in comprehensively analyzing pathology reports. Specifically, we assess their performance in cancer type identification, AJCC stage determination, and prognosis assessment, encompassing both information extraction and higher-order reasoning tasks. Based on a detailed analysis of their performance metrics in a zero-shot setting, we developed two instruction-tuned models: Path-llama3.1-8B and Path-GPT-4o-mini-FT. These models demonstrated superior performance in zero-shot cancer type identification, staging, and prognosis assessment compared to the other models evaluated.

Title: PostHoc FREE Calibrating on Kolmogorov Arnold Networks

• Authors: Wenhao Liang, Wei Emma Zhang, Lin Yue, Miao Xu, Olaf Maennel, Weitong Chen
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01195
• Pdf URL: https://arxiv.org/pdf/2503.01195
• Copy Paste: [[2503.01195]] PostHoc FREE Calibrating on Kolmogorov Arnold Networks(https://arxiv.org/abs/2503.01195)
• Keywords: robust
• Abstract: Kolmogorov Arnold Networks (KANs) are neural architectures inspired by the Kolmogorov Arnold representation theorem that leverage B Spline parameterizations for flexible, locally adaptive function approximation. Although KANs can capture complex nonlinearities beyond those modeled by standard MultiLayer Perceptrons (MLPs), they frequently exhibit miscalibrated confidence estimates manifesting as overconfidence in dense data regions and underconfidence in sparse areas. In this work, we systematically examine the impact of four critical hyperparameters including Layer Width, Grid Order, Shortcut Function, and Grid Range on the calibration of KANs. Furthermore, we introduce a novel TemperatureScaled Loss (TSL) that integrates a temperature parameter directly into the training objective, dynamically adjusting the predictive distribution during learning. Both theoretical analysis and extensive empirical evaluations on standard benchmarks demonstrate that TSL significantly reduces calibration errors, thereby improving the reliability of probabilistic predictions. Overall, our study provides actionable insights into the design of spline based neural networks and establishes TSL as a robust loss solution for enhancing calibration.

Title: Parameter-free Video Segmentation for Vision and Language Understanding

• Authors: Louis Mahon, Mirella Lapata
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01201
• Pdf URL: https://arxiv.org/pdf/2503.01201
• Copy Paste: [[2503.01201]] Parameter-free Video Segmentation for Vision and Language Understanding(https://arxiv.org/abs/2503.01201)
• Keywords: segmentation
• Abstract: The proliferation of creative video content has driven demand for adapting language models to handle video input and enable multimodal understanding. However, end-to-end models struggle to process long videos due to their size and complexity. An effective alternative is to divide them into smaller chunks to be processed separately, and this motivates a method for choosing where the chunk boundaries should be. In this paper, we propose an algorithm for segmenting videos into contiguous chunks, based on the minimum description length principle, coupled with a dynamic programming search. The algorithm is entirely parameter-free, given feature vectors, not requiring a set threshold or the number or size of chunks to be specified. We show empirically that the breakpoints it produces more accurately approximate scene boundaries in long videos, compared with existing methods for scene detection, even when such methods have access to the true number of scenes. We then showcase this algorithm in two tasks: long video summarization, and retrieval-augmented video question answering. In both cases, scene breaks produced by our algorithm lead to better downstream performance than existing methods for video segmentation.

Title: A Multi-Sensor Fusion Approach for Rapid Orthoimage Generation in Large-Scale UAV Mapping

• Authors: Jialei He, Zhihao Zhan, Zhituo Tu, Xiang Zhu, Jie Yuan
• Subjects: cs.CV, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.01202
• Pdf URL: https://arxiv.org/pdf/2503.01202
• Copy Paste: [[2503.01202]] A Multi-Sensor Fusion Approach for Rapid Orthoimage Generation in Large-Scale UAV Mapping(https://arxiv.org/abs/2503.01202)
• Keywords: robust
• Abstract: Rapid generation of large-scale orthoimages from Unmanned Aerial Vehicles (UAVs) has been a long-standing focus of research in the field of aerial mapping. A multi-sensor UAV system, integrating the Global Positioning System (GPS), Inertial Measurement Unit (IMU), 4D millimeter-wave radar and camera, can provide an effective solution to this problem. In this paper, we utilize multi-sensor data to overcome the limitations of conventional orthoimage generation methods in terms of temporal performance, system robustness, and geographic reference accuracy. A prior-pose-optimized feature matching method is introduced to enhance matching speed and accuracy, reducing the number of required features and providing precise references for the Structure from Motion (SfM) process. The proposed method exhibits robustness in low-texture scenes like farmlands, where feature matching is difficult. Experiments show that our approach achieves accurate feature matching orthoimage generation in a short time. The proposed drone system effectively aids in farmland detection and management.

Title: Hypergraph Foundation Model

• Authors: Yifan Feng, Shiquan Liu, Xiangmin Han, Shaoyi Du, Zongze Wu, Han Hu, Yue Gao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01203
• Pdf URL: https://arxiv.org/pdf/2503.01203
• Copy Paste: [[2503.01203]] Hypergraph Foundation Model(https://arxiv.org/abs/2503.01203)
• Keywords: extraction
• Abstract: Hypergraph neural networks (HGNNs) effectively model complex high-order relationships in domains like protein interactions and social networks by connecting multiple vertices through hyperedges, enhancing modeling capabilities, and reducing information loss. Developing foundation models for hypergraphs is challenging due to their distinct data, which includes both vertex features and intricate structural information. We present Hyper-FM, a Hypergraph Foundation Model for multi-domain knowledge extraction, featuring Hierarchical High-Order Neighbor Guided Vertex Knowledge Embedding for vertex feature representation and Hierarchical Multi-Hypergraph Guided Structural Knowledge Extraction for structural information. Additionally, we curate 10 text-attributed hypergraph datasets to advance research between HGNNs and LLMs. Experiments on these datasets show that Hyper-FM outperforms baseline methods by approximately 13.3\%, validating our approach. Furthermore, we propose the first scaling law for hypergraph foundation models, demonstrating that increasing domain diversity significantly enhances performance, unlike merely augmenting vertex and hyperedge counts. This underscores the critical role of domain diversity in scaling hypergraph models.

Title: Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models

• Authors: Tianjie Ju, Yi Hua, Hao Fei, Zhenyu Shao, Yubin Zheng, Haodong Zhao, Mong-Li Lee, Wynne Hsu, Zhuosheng Zhang, Gongshen Liu
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01208
• Pdf URL: https://arxiv.org/pdf/2503.01208
• Copy Paste: [[2503.01208]] Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models(https://arxiv.org/abs/2503.01208)
• Keywords: privacy, watermark, large language model
• Abstract: Multi-Modal Large Language Models (MLLMs) have exhibited remarkable performance on various vision-language tasks such as Visual Question Answering (VQA). Despite accumulating evidence of privacy concerns associated with task-relevant content, it remains unclear whether MLLMs inadvertently memorize private content that is entirely irrelevant to the training tasks. In this paper, we investigate how randomly generated task-irrelevant private content can become spuriously correlated with downstream objectives due to partial mini-batch training dynamics, thus causing inadvertent memorization. Concretely, we randomly generate task-irrelevant watermarks into VQA fine-tuning images at varying probabilities and propose a novel probing framework to determine whether MLLMs have inadvertently encoded such content. Our experiments reveal that MLLMs exhibit notably different training behaviors in partial mini-batch settings with task-irrelevant watermarks embedded. Furthermore, through layer-wise probing, we demonstrate that MLLMs trigger distinct representational patterns when encountering previously seen task-irrelevant knowledge, even if this knowledge does not influence their output during prompting. Our code is available at this https URL.

Title: Architectural and Inferential Inductive Biases For Exchangeable Sequence Modeling

• Authors: Daksh Mittal, Ang Li, Tzu-Ching Yen, Daniel Guetta, Hongseok Namkoong
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01215
• Pdf URL: https://arxiv.org/pdf/2503.01215
• Copy Paste: [[2503.01215]] Architectural and Inferential Inductive Biases For Exchangeable Sequence Modeling(https://arxiv.org/abs/2503.01215)
• Keywords: transformer
• Abstract: Autoregressive models have emerged as a powerful framework for modeling exchangeable sequences - i.i.d. observations when conditioned on some latent factor - enabling direct modeling of uncertainty from missing data (rather than a latent). Motivated by the critical role posterior inference plays as a subroutine in decision-making (e.g., active learning, bandits), we study the inferential and architectural inductive biases that are most effective for exchangeable sequence modeling. For the inference stage, we highlight a fundamental limitation of the prevalent single-step generation approach: inability to distinguish between epistemic and aleatoric uncertainty. Instead, a long line of works in Bayesian statistics advocates for multi-step autoregressive generation; we demonstrate this "correct approach" enables superior uncertainty quantification that translates into better performance on downstream decision-making tasks. This naturally leads to the next question: which architectures are best suited for multi-step inference? We identify a subtle yet important gap between recently proposed Transformer architectures for exchangeable sequences (Muller et al., 2022; Nguyen & Grover, 2022; Ye & Namkoong, 2024), and prove that they in fact cannot guarantee exchangeability despite introducing significant computational overhead. We illustrate our findings using controlled synthetic settings, demonstrating how custom architectures can significantly underperform standard causal masks, underscoring the need for new architectural innovations.

Title: HREB-CRF: Hierarchical Reduced-bias EMA for Chinese Named Entity Recognition

• Authors: Sijin Sun, Ming Deng, Xinrui Yu, Liangbin Zhao
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01217
• Pdf URL: https://arxiv.org/pdf/2503.01217
• Copy Paste: [[2503.01217]] HREB-CRF: Hierarchical Reduced-bias EMA for Chinese Named Entity Recognition(https://arxiv.org/abs/2503.01217)
• Keywords: robust
• Abstract: Incorrect boundary division, complex semantic representation, and differences in pronunciation and meaning often lead to errors in Chinese Named Entity Recognition(CNER). To address these issues, this paper proposes HREB-CRF framework: Hierarchical Reduced-bias EMA with CRF. The proposed method amplifies word boundaries and pools long text gradients through exponentially fixed-bias weighted average of local and global hierarchical attention. Experimental results on the MSRA, Resume, and Weibo datasets show excellent in F1, outperforming the baseline model by 1.1\%, 1.6\%, and 9.8\%. The significant improvement in F1 shows evidences of strong effectiveness and robustness of approach in CNER tasks.

Title: Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion

• Authors: Jiqing Wu, Ingrid Berg, Yawei Li, Ender Konukoglu, Viktor H. Koelzer
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01220
• Pdf URL: https://arxiv.org/pdf/2503.01220
• Copy Paste: [[2503.01220]] Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion(https://arxiv.org/abs/2503.01220)
• Keywords: diffusion, generative
• Abstract: Holistic 3D modeling of molecularly defined brain structures is crucial for understanding complex brain functions. Emerging tissue profiling technologies enable the construction of a comprehensive atlas of the mammalian brain with sub-cellular resolution and spatially resolved gene expression data. However, such tera-scale volumetric datasets present significant computational challenges in understanding complex brain functions within their native 3D spatial context. Here, we propose the novel generative approach $\textbf{Tera-MIND}$, which can simulate $\textbf{Tera}$-scale $\textbf{M}$ouse bra$\textbf{IN}s$ in 3D using a patch-based and boundary-aware $\textbf{D}$iffusion model. Taking spatial transcriptomic data as the conditional input, we generate virtual mouse brains with comprehensive cellular morphological detail at teravoxel scale. Through the lens of 3D $gene$-$gene$ self-attention, we identify spatial molecular interactions for key transcriptomic pathways in the murine brain, exemplified by glutamatergic and dopaminergic neuronal systems. Importantly, these $in$-$silico$ biological findings are consistent and reproducible across three tera-scale virtual mouse brains. Therefore, Tera-MIND showcases a promising path toward efficient and generative simulations of whole organ systems for biomedical research. Project website: $\href{this http URL}{https}$

Title: Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG

• Authors: Wenbin Wang, Yongcheng Jing, Liang Ding, Yingjie Wang, Li Shen, Yong Luo, Bo Du, Dacheng Tao
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01222
• Pdf URL: https://arxiv.org/pdf/2503.01222
• Copy Paste: [[2503.01222]] Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG(https://arxiv.org/abs/2503.01222)
• Keywords: large language model
• Abstract: High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs). To overcome the limitations of existing methods, this paper shifts away from prior dedicated heuristic approaches and revisits the most fundamental idea to HR perception by enhancing the long-context capability of MLLMs, driven by recent advances in long-context techniques like retrieval-augmented generation (RAG) for general LLMs. Towards this end, this paper presents the first study exploring the use of RAG to address HR perception challenges. Specifically, we propose Retrieval-Augmented Perception (RAP), a training-free framework that retrieves and fuses relevant image crops while preserving spatial context using the proposed Spatial-Awareness Layout. To accommodate different tasks, the proposed Retrieved-Exploration Search (RE-Search) dynamically selects the optimal number of crops based on model confidence and retrieval scores. Experimental results on HR benchmarks demonstrate the significant effectiveness of RAP, with LLaVA-v1.5-13B achieving a 43% improvement on $V^*$ Bench and 19% on HR-Bench.

Title: CE-U: Cross Entropy Unlearning

• Authors: Bo Yang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01224
• Pdf URL: https://arxiv.org/pdf/2503.01224
• Copy Paste: [[2503.01224]] CE-U: Cross Entropy Unlearning(https://arxiv.org/abs/2503.01224)
• Keywords: large language model
• Abstract: Large language models (LLMs) inadvertently memorize sensitive data from their massive pretraining corpora \cite{jang2022knowledge}. In this work, we propose CE-U (Cross Entropy Unlearning), a novel loss function designed specifically for unlearning tasks. CE-U addresses fundamental limitations of gradient ascent approaches which suffer from instability due to vanishing gradients when model confidence is high and gradient exploding when confidence is low. We also unify standard cross entropy supervision and cross entropy unlearning into a single framework. Notably, on the TOFU benchmark for unlearning \cite{maini2024tofu}, CE-U achieves state-of-the-art results on LLaMA2-7B with 1\% and 5\% forgetting, even without the use of any extra reference model or additional positive samples. Our theoretical analysis further reveals that the gradient instability issues also exist in popular reinforcement learning algorithms like DPO and GRPO, as they include a gradient ascent component. This suggests that applying CE-U principles to reinforcement learning could be a promising direction for improving stability and convergence.

Title: Enhancing Network Security Management in Water Systems using FM-based Attack Attribution

• Authors: Aleksandar Avdalovic, Joseph Khoury, Ahmad Taha, Elias Bou-Harb
• Subjects: cs.LG, cs.CR, eess.SY
• Abstract URL: https://arxiv.org/abs/2503.01229
• Pdf URL: https://arxiv.org/pdf/2503.01229
• Copy Paste: [[2503.01229]] Enhancing Network Security Management in Water Systems using FM-based Attack Attribution(https://arxiv.org/abs/2503.01229)
• Keywords: security, attack
• Abstract: Water systems are vital components of modern infrastructure, yet they are increasingly susceptible to sophisticated cyber attacks with potentially dire consequences on public health and safety. While state-of-the-art machine learning techniques effectively detect anomalies, contemporary model-agnostic attack attribution methods using LIME, SHAP, and LEMNA are deemed impractical for large-scale, interdependent water systems. This is due to the intricate interconnectivity and dynamic interactions that define these complex environments. Such methods primarily emphasize individual feature importance while falling short of addressing the crucial sensor-actuator interactions in water systems, which limits their effectiveness in identifying root cause attacks. To this end, we propose a novel model-agnostic Factorization Machines (FM)-based approach that capitalizes on water system sensor-actuator interactions to provide granular explanations and attributions for cyber attacks. For instance, an anomaly in an actuator pump activity can be attributed to a top root cause attack candidates, a list of water pressure sensors, which is derived from the underlying linear and quadratic effects captured by our approach. We validate our method using two real-world water system specific datasets, SWaT and WADI, demonstrating its superior performance over traditional attribution methods. In multi-feature cyber attack scenarios involving intricate sensor-actuator interactions, our FM-based attack attribution method effectively ranks attack root causes, achieving approximately 20% average improvement over SHAP and LEMNA.

Title: PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation

• Authors: Yuxuan Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01233
• Pdf URL: https://arxiv.org/pdf/2503.01233
• Copy Paste: [[2503.01233]] PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation(https://arxiv.org/abs/2503.01233)
• Keywords: large language model
• Abstract: The alignment of large language models with human values presents a critical challenge, particularly when balancing conflicting objectives like helpfulness and harmlessness. Existing approaches, such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO), face notable limitations: RLHF suffers from instability and inefficiency in multi-objective optimization, while DPO lacks mechanisms for dynamic trade-offs. To address these challenges, we propose Post-Training Extrapolation Optimization (PEO), a novel and efficient framework for bi-factorial alignment. PEO generates a family of Pareto-optimal policies in a single training pass by leveraging a three-phase pipeline: (1) aspect-specific learning, (2) generalist initialization via interpolation, and (3) post-training optimization via extrapolation. PEO enables dynamic adaptation to diverse user preferences at inference time without retraining. Our comprehensive experiments across multiple LLMs demonstrate that PEO achieves superior Pareto fronts compared to baselines, offering improved flexibility and computational efficiency. Theoretical analyses further highlight PEO's capacity to overcome optimization bottlenecks, paving the way for scalable, personalized alignment.

Title: Self-Adaptive Gamma Context-Aware SSM-based Model for Metal Defect Detection

• Authors: Sijin Sun, Ming Deng, Xingrui Yu, Xinyu Xi, Liangbin Zhao
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01234
• Pdf URL: https://arxiv.org/pdf/2503.01234
• Copy Paste: [[2503.01234]] Self-Adaptive Gamma Context-Aware SSM-based Model for Metal Defect Detection(https://arxiv.org/abs/2503.01234)
• Keywords: robust, extraction
• Abstract: Metal defect detection is critical in industrial quality assurance, yet existing methods struggle with grayscale variations and complex defect states, limiting its robustness. To address these challenges, this paper proposes a Self-Adaptive Gamma Context-Aware SSM-based model(GCM-DET). This advanced detection framework integrating a Dynamic Gamma Correction (GC) module to enhance grayscale representation and optimize feature extraction for precise defect reconstruction. A State-Space Search Management (SSM) architecture captures robust multi-scale features, effectively handling defects of varying shapes and scales. Focal Loss is employed to mitigate class imbalance and refine detection accuracy. Additionally, the CD5-DET dataset is introduced, specifically designed for port container maintenance, featuring significant grayscale variations and intricate defect patterns. Experimental results demonstrate that the proposed model achieves substantial improvements, with mAP@0.5 gains of 27.6\%, 6.6\%, and 2.6\% on the CD5-DET, NEU-DET, and GC10-DET datasets.

Title: Gaussian Process Surrogate Models for Efficient Estimation of Structural Response Distributions and Order Statistics

• Authors: Vegard Flovik, Sebastian Winter, Christian Agrell
• Subjects: cs.LG, cs.CE
• Abstract URL: https://arxiv.org/abs/2503.01242
• Pdf URL: https://arxiv.org/pdf/2503.01242
• Copy Paste: [[2503.01242]] Gaussian Process Surrogate Models for Efficient Estimation of Structural Response Distributions and Order Statistics(https://arxiv.org/abs/2503.01242)
• Keywords: robust
• Abstract: Engineering disciplines often rely on extensive simulations to ensure that structures are designed to withstand harsh conditions while avoiding over-engineering for unlikely scenarios. Assessments such as Serviceability Limit State (SLS) involve evaluating weather events, including estimating loads not expected to be exceeded more than a specified number of times (e.g., 100) throughout the structure's design lifetime. Although physics-based simulations provide robust and detailed insights, they are computationally expensive, making it challenging to generate statistically valid representations of a wide range of weather conditions. To address these challenges, we propose an approach using Gaussian Process (GP) surrogate models trained on a limited set of simulation outputs to directly generate the structural response distribution. We apply this method to an SLS assessment for estimating the order statistics \(Y_{100}\), representing the 100th highest response, of a structure exposed to 25 years of historical weather observations. Our results indicate that the GP surrogate models provide comparable results to full simulations but at a fraction of the computational cost.

Title: Convex Hull-based Algebraic Constraint for Visual Quadric SLAM

• Authors: Xiaolong Yu, Junqiao Zhao, Shuangfu Song, Zhongyang Zhu, Zihan Yuan, Chen Ye, Tiantian Feng
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.01254
• Pdf URL: https://arxiv.org/pdf/2503.01254
• Copy Paste: [[2503.01254]] Convex Hull-based Algebraic Constraint for Visual Quadric SLAM(https://arxiv.org/abs/2503.01254)
• Keywords: segmentation
• Abstract: Using Quadrics as the object representation has the benefits of both generality and closed-form projection derivation between image and world spaces. Although numerous constraints have been proposed for dual quadric reconstruction, we found that many of them are imprecise and provide minimal improvements to this http URL scrutinizing the existing constraints, we introduce a concise yet more precise convex hull-based algebraic constraint for object landmarks, which is applied to object reconstruction, frontend pose estimation, and backend bundle this http URL constraint is designed to fully leverage precise semantic segmentation, effectively mitigating mismatches between complex-shaped object contours and dual this http URL on public datasets demonstrate that our approach is applicable to both monocular and RGB-D SLAM and achieves improved object mapping and localization than existing quadric SLAM methods. The implementation of our method is available at this https URL.

Title: Multi-Level Collaboration in Model Merging

• Authors: Qi Li, Runpeng Yu, Xinchao Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01268
• Pdf URL: https://arxiv.org/pdf/2503.01268
• Copy Paste: [[2503.01268]] Multi-Level Collaboration in Model Merging(https://arxiv.org/abs/2503.01268)
• Keywords: robust
• Abstract: Parameter-level model merging is an emerging paradigm in multi-task learning with significant promise. Previous research has explored its connections with prediction-level model ensembling-commonly viewed as the upper bound for merging-to reveal the potential of achieving performance consistency between the two. However, this observation relies on certain preconditions, such as being limited to two models, using ViT-based models, and all models are fine-tuned from the same pre-trained checkpoint. To further understand the intrinsic connections between model merging and model ensembling, this paper explores an interesting possibility: If these restrictions are removed, can performance consistency still be achieved between merging and ensembling? To answer this question, we first theoretically establish a performance correlation between merging and ensembling. We find that even when previous restrictions are not met, there is still a way for model merging to attain a near-identical and superior performance similar to that of ensembling. To verify whether our findings are practical, we introduce a validation framework termed Neural Ligand (NeuLig). The learning process of NeuLig is meticulously designed with a specialized loss function supported by theoretical foundations. Experimental results demonstrate the robust resilience of NeuLig in terms of both model scale and the number of collaborating models. For instance, for the case involving 5 CLIP-ViT-B/32 models, parameter-level merging achieves the same performance as prediction-level ensembling (merging: 95.44% vs. ensembling: 95.46%).

Title: ChatGPT for President! Presupposed content in politicians versus GPT-generated texts

• Authors: Davide Garassino, Nicola Brocca, Viviana Masia
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.01269
• Pdf URL: https://arxiv.org/pdf/2503.01269
• Copy Paste: [[2503.01269]] ChatGPT for President! Presupposed content in politicians versus GPT-generated texts(https://arxiv.org/abs/2503.01269)
• Keywords: large language model
• Abstract: This study examines ChatGPT-4's capability to replicate linguistic strategies used in political discourse, focusing on its potential for manipulative language generation. As large language models become increasingly popular for text generation, concerns have grown regarding their role in spreading fake news and propaganda. This research compares real political speeches with those generated by ChatGPT, emphasizing presuppositions (a rhetorical device that subtly influences audiences by packaging some content as already known at the moment of utterance, thus swaying opinions without explicit argumentation). Using a corpus-based pragmatic analysis, this study assesses how well ChatGPT can mimic these persuasive strategies. The findings reveal that although ChatGPT-generated texts contain many manipulative presuppositions, key differences emerge in their frequency, form, and function compared with those of politicians. For instance, ChatGPT often relies on change-of-state verbs used in fixed phrases, whereas politicians use presupposition triggers in more varied and creative ways. Such differences, however, are challenging to detect with the naked eye, underscoring the potential risks posed by large language models in political and public this http URL a corpus-based pragmatic analysis, this study assesses how well ChatGPT can mimic these persuasive strategies. The findings reveal that although ChatGPT-generated texts contain many manipulative presuppositions, key differences emerge in their frequency, form, and function compared with those of politicians. For instance, ChatGPT often relies on change-of-state verbs used in fixed phrases, whereas politicians use presupposition triggers in more varied and creative ways. Such differences, however, are challenging to detect with the naked eye, underscoring the potential risks posed by large language models in political and public discourse.

Title: Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning

• Authors: Wenshuai Huo, Xiaocheng Feng, Yichong Huang, Chengpeng Fu, Baohang Li, Yangfan Ye, Zhirui Zhang, Dandan Tu, Duyu Tang, Yunfei Lu, Hui Wang, Bing Qin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01275
• Pdf URL: https://arxiv.org/pdf/2503.01275
• Copy Paste: [[2503.01275]] Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning(https://arxiv.org/abs/2503.01275)
• Keywords: large language model
• Abstract: Large language models (LLMs) have demonstrated significant progress in multilingual language understanding and generation. However, due to the imbalance in training data, their capabilities in non-English languages are limited. Recent studies revealed the English-pivot multilingual mechanism of LLMs, where LLMs implicitly convert non-English queries into English ones at the bottom layers and adopt English for thinking at the middle layers. However, due to the absence of explicit supervision for cross-lingual alignment in the intermediate layers of LLMs, the internal representations during these stages may become inaccurate. In this work, we introduce a deep supervision fine-tuning method (DFT) that incorporates additional supervision in the internal layers of the model to guide its workflow. Specifically, we introduce two training objectives on different layers of LLMs: one at the bottom layers to constrain the conversion of the target language into English, and another at the middle layers to constrain reasoning in English. To effectively achieve the guiding purpose, we designed two types of supervision signals: logits and feature, which represent a stricter constraint and a relatively more relaxed guidance. Our method guides the model to not only consider the final generated result when processing non-English inputs but also ensure the accuracy of internal representations. We conducted extensive experiments on typical English-centric large models, LLaMA-2 and Gemma-2, and the results on multiple multilingual datasets show that our method significantly outperforms traditional fine-tuning methods.

Title: Soybean Disease Detection via Interpretable Hybrid CNN-GNN: Integrating MobileNetV2 and GraphSAGE with Cross-Modal Attention

• Authors: Md Abrar Jahin, Soudeep Shahriar, M. F. Mridha, Nilanjan Dey
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01284
• Pdf URL: https://arxiv.org/pdf/2503.01284
• Copy Paste: [[2503.01284]] Soybean Disease Detection via Interpretable Hybrid CNN-GNN: Integrating MobileNetV2 and GraphSAGE with Cross-Modal Attention(https://arxiv.org/abs/2503.01284)
• Keywords: robust, extraction, interpretability
• Abstract: Soybean leaf disease detection is critical for agricultural productivity but faces challenges due to visually similar symptoms and limited interpretability in conventional methods. While Convolutional Neural Networks (CNNs) excel in spatial feature extraction, they often neglect inter-image relational dependencies, leading to misclassifications. This paper proposes an interpretable hybrid Sequential CNN-Graph Neural Network (GNN) framework that synergizes MobileNetV2 for localized feature extraction and GraphSAGE for relational modeling. The framework constructs a graph where nodes represent leaf images, with edges defined by cosine similarity-based adjacency matrices and adaptive neighborhood sampling. This design captures fine-grained lesion features and global symptom patterns, addressing inter-class similarity challenges. Cross-modal interpretability is achieved via Grad-CAM and Eigen-CAM visualizations, generating heatmaps to highlight disease-influential regions. Evaluated on a dataset of ten soybean leaf diseases, the model achieves $97.16\%$ accuracy, surpassing standalone CNNs ($\le95.04\%$) and traditional machine learning models ($\le77.05\%$). Ablation studies validate the sequential architecture's superiority over parallel or single-model configurations. With only 2.3 million parameters, the lightweight MobileNetV2-GraphSAGE combination ensures computational efficiency, enabling real-time deployment in resource-constrained environments. The proposed approach bridges the gap between accurate classification and practical applicability, offering a robust, interpretable tool for agricultural diagnostics while advancing CNN-GNN integration in plant pathology research.

Title: Robust Simulation-Based Inference under Missing Data via Neural Processes

• Authors: Yogesh Verma, Ayush Bharti, Vikas Garg
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01287
• Pdf URL: https://arxiv.org/pdf/2503.01287
• Copy Paste: [[2503.01287]] Robust Simulation-Based Inference under Missing Data via Neural Processes(https://arxiv.org/abs/2503.01287)
• Keywords: robust
• Abstract: Simulation-based inference (SBI) methods typically require fully observed data to infer parameters of models with intractable likelihood functions. However, datasets often contain missing values due to incomplete observations, data corruptions (common in astrophysics), or instrument limitations (e.g., in high-energy physics applications). In such scenarios, missing data must be imputed before applying any SBI method. We formalize the problem of missing data in SBI and demonstrate that naive imputation methods can introduce bias in the estimation of SBI posterior. We also introduce a novel amortized method that addresses this issue by jointly learning the imputation model and the inference network within a neural posterior estimation (NPE) framework. Extensive empirical results on SBI benchmarks show that our approach provides robust inference outcomes compared to standard baselines for varying levels of missing data. Moreover, we demonstrate the merits of our imputation model on two real-world bioactivity datasets (Adrenergic and Kinase assays). Code is available at this https URL.

Title: Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual

• Authors: Chong Wang, Lanqing Guo, Zixuan Fu, Siyuan Yang, Hao Cheng, Alex C. Kot, Bihan Wen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01288
• Pdf URL: https://arxiv.org/pdf/2503.01288
• Copy Paste: [[2503.01288]] Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual(https://arxiv.org/abs/2503.01288)
• Keywords: diffusion, generative
• Abstract: Plug-and-play (PnP) methods offer an iterative strategy for solving image restoration (IR) problems in a zero-shot manner, using a learned \textit{discriminative denoiser} as the implicit prior. More recently, a sampling-based variant of this approach, which utilizes a pre-trained \textit{generative diffusion model}, has gained great popularity for solving IR problems through stochastic sampling. The IR results using PnP with a pre-trained diffusion model demonstrate distinct advantages compared to those using discriminative denoisers, \ie improved perceptual quality while sacrificing the data fidelity. The unsatisfactory results are due to the lack of integration of these strategies in the IR tasks. In this work, we propose a novel zero-shot IR scheme, dubbed Reconciling Diffusion Model in Dual (RDMD), which leverages only a \textbf{single} pre-trained diffusion model to construct \textbf{two} complementary regularizers. Specifically, the diffusion model in RDMD will iteratively perform deterministic denoising and stochastic sampling, aiming to achieve high-fidelity image restoration with appealing perceptual quality. RDMD also allows users to customize the distortion-perception tradeoff with a single hyperparameter, enhancing the adaptability of the restoration process in different practical scenarios. Extensive experiments on several IR tasks demonstrate that our proposed method could achieve superior results compared to existing approaches on both the FFHQ and ImageNet datasets.

Title: ACTIVA: Amortized Causal Effect Estimation without Graphs via Transformer-based Variational Autoencoder

• Authors: Andreas Sauter, Saber Salehkaleybar, Aske Plaat, Erman Acar
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01290
• Pdf URL: https://arxiv.org/pdf/2503.01290
• Copy Paste: [[2503.01290]] ACTIVA: Amortized Causal Effect Estimation without Graphs via Transformer-based Variational Autoencoder(https://arxiv.org/abs/2503.01290)
• Keywords: transformer
• Abstract: Predicting the distribution of outcomes under hypothetical interventions is crucial in domains like healthcare, economics, and policy-making. Current methods often rely on strong assumptions, such as known causal graphs or parametric models, and lack amortization across problem instances, limiting their practicality. We propose a novel transformer-based conditional variational autoencoder architecture, named ACTIVA, that extends causal transformer encoders to predict causal effects as mixtures of Gaussians. Our method requires no causal graph and predicts interventional distributions given only observational data and a queried intervention. By amortizing over many simulated instances, it enables zero-shot generalization to novel datasets without retraining. Experiments demonstrate accurate predictions for synthetic and semi-synthetic data, showcasing the effectiveness of our graph-free, amortized causal inference approach.

Title: SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance

• Authors: Peishan Cong, Ziyi Wang, Yuexin Ma, Xiangyu Yue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01291
• Pdf URL: https://arxiv.org/pdf/2503.01291
• Copy Paste: [[2503.01291]] SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance(https://arxiv.org/abs/2503.01291)
• Keywords: generative
• Abstract: Generating reasonable and high-quality human interactive motions in a given dynamic environment is crucial for understanding, modeling, transferring, and applying human behaviors to both virtual and physical robots. In this paper, we introduce an effective method, SemGeoMo, for dynamic contextual human motion generation, which fully leverages the text-affordance-joint multi-level semantic and geometric guidance in the generation process, improving the semantic rationality and geometric correctness of generative motions. Our method achieves state-of-the-art performance on three datasets and demonstrates superior generalization capability for diverse interaction scenarios. The project page and code can be found at this https URL.

Title: PA-CLIP: Enhancing Zero-Shot Anomaly Detection through Pseudo-Anomaly Awareness

• Authors: Yurui Pan, Lidong Wang, Yuchao Chen, Wenbing Zhu, Bo Peng, Mingmin Chi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01292
• Pdf URL: https://arxiv.org/pdf/2503.01292
• Copy Paste: [[2503.01292]] PA-CLIP: Enhancing Zero-Shot Anomaly Detection through Pseudo-Anomaly Awareness(https://arxiv.org/abs/2503.01292)
• Keywords: robust
• Abstract: In industrial anomaly detection (IAD), accurately identifying defects amidst diverse anomalies and under varying imaging conditions remains a significant challenge. Traditional approaches often struggle with high false-positive rates, frequently misclassifying normal shadows and surface deformations as defects, an issue that becomes particularly pronounced in products with complex and intricate surface features. To address these challenges, we introduce PA-CLIP, a zero-shot anomaly detection method that reduces background noise and enhances defect detection through a pseudo-anomaly-based framework. The proposed method integrates a multiscale feature aggregation strategy for capturing detailed global and local information, two memory banks for distinguishing background information, including normal patterns and pseudo-anomalies, from true anomaly features, and a decision-making module designed to minimize false positives caused by environmental variations while maintaining high defect sensitivity. Demonstrated on the MVTec AD and VisA datasets, PA-CLIP outperforms existing zero-shot methods, providing a robust solution for industrial defect detection.

Title: Fine-Grained Controllable Apparel Showcase Image Generation via Garment-Centric Outpainting

• Authors: Rong Zhang, Jingnan Wang, Zhiwen Zuo, Jianfeng Dong, Wei Li, Chi Wang, Weiwei Xu, Xun Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01294
• Pdf URL: https://arxiv.org/pdf/2503.01294
• Copy Paste: [[2503.01294]] Fine-Grained Controllable Apparel Showcase Image Generation via Garment-Centric Outpainting(https://arxiv.org/abs/2503.01294)
• Keywords: diffusion
• Abstract: In this paper, we propose a novel garment-centric outpainting (GCO) framework based on the latent diffusion model (LDM) for fine-grained controllable apparel showcase image generation. The proposed framework aims at customizing a fashion model wearing a given garment via text prompts and facial images. Different from existing methods, our framework takes a garment image segmented from a dressed mannequin or a person as the input, eliminating the need for learning cloth deformation and ensuring faithful preservation of garment details. The proposed framework consists of two stages. In the first stage, we introduce a garment-adaptive pose prediction model that generates diverse poses given the garment. Then, in the next stage, we generate apparel showcase images, conditioned on the garment and the predicted poses, along with specified text prompts and facial images. Notably, a multi-scale appearance customization module (MS-ACM) is designed to allow both overall and fine-grained text-based control over the generated model's appearance. Moreover, we leverage a lightweight feature fusion operation without introducing any extra encoders or modules to integrate multiple conditions, which is more efficient. Extensive experiments validate the superior performance of our framework compared to state-of-the-art methods.

Title: Regularization-based Framework for Quantization-, Fault- and Variability-Aware Training

• Authors: Anmol Biswas, Raghav Singhal, Sivakumar Elangovan, Shreyas Sabnis, Udayan Ganguly
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01297
• Pdf URL: https://arxiv.org/pdf/2503.01297
• Copy Paste: [[2503.01297]] Regularization-based Framework for Quantization-, Fault- and Variability-Aware Training(https://arxiv.org/abs/2503.01297)
• Keywords: robust
• Abstract: Efficient inference is critical for deploying deep learning models on edge AI devices. Low-bit quantization (e.g., 3- and 4-bit) with fixed-point arithmetic improves efficiency, while low-power memory technologies like analog nonvolatile memory enable further gains. However, these methods introduce non-ideal hardware behavior, including bit faults and device-to-device variability. We propose a regularization-based quantization-aware training (QAT) framework that supports fixed, learnable step-size, and learnable non-uniform quantization, achieving competitive results on CIFAR-10 and ImageNet. Our method also extends to Spiking Neural Networks (SNNs), demonstrating strong performance on 4-bit networks on CIFAR10-DVS and N-Caltech 101. Beyond quantization, our framework enables fault and variability-aware fine-tuning, mitigating stuck-at faults (fixed weight bits) and device resistance variability. Compared to prior fault-aware training, our approach significantly improves performance recovery under upto 20% bit-fault rate and 40% device-to-device variability. Our results establish a generalizable framework for quantization and robustness-aware training, enhancing efficiency and reliability in low-power, non-ideal hardware.

Title: MINT: Multi-modal Chain of Thought in Unified Generative Models for Enhanced Image Generation

• Authors: Yi Wang, Mushui Liu, Wanggui He, Longxiang Zhang, Ziwei Huang, Guanghao Zhang, Fangxun Shu, Zhong Tao, Dong She, Zhelun Yu, Haoyuan Li, Weilong Dai, Mingli Song, Jie Song, Hao Jiang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01298
• Pdf URL: https://arxiv.org/pdf/2503.01298
• Copy Paste: [[2503.01298]] MINT: Multi-modal Chain of Thought in Unified Generative Models for Enhanced Image Generation(https://arxiv.org/abs/2503.01298)
• Keywords: transformer, generative
• Abstract: Unified generative models have demonstrated extraordinary performance in both text and image generation. However, they tend to underperform when generating intricate images with various interwoven conditions, which is hard to solely rely on straightforward text-to-image generation. In response to this challenge, we introduce MINT, an innovative unified generative model, empowered with native multimodal chain of thought (MCoT) for enhanced image generation for the first time. Firstly, we design Mixture of Transformer Experts (MTXpert), an expert-parallel structure that effectively supports both natural language generation (NLG) and visual capabilities, while avoiding potential modality conflicts that could hinder the full potential of each modality. Building on this, we propose an innovative MCoT training paradigm, a step-by-step approach to multimodal thinking, reasoning, and reflection specifically designed to enhance image generation. This paradigm equips MINT with nuanced, element-wise decoupled alignment and a comprehensive understanding of textual and visual components. Furthermore, it fosters advanced multimodal reasoning and self-reflection, enabling the construction of images that are firmly grounded in the logical relationships between these elements. Notably, MINT has been validated to exhibit superior performance across multiple benchmarks for text-to-image (T2I) and image-to-text (I2T) tasks.

Title: Causal Tree Extraction from Medical Case Reports: A Novel Task for Experts-like Text Comprehension

• Authors: Sakiko Yahata, Zhen Wan, Fei Cheng, Sadao Kurohashi, Hisahiko Sato, Ryozo Nagai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01302
• Pdf URL: https://arxiv.org/pdf/2503.01302
• Copy Paste: [[2503.01302]] Causal Tree Extraction from Medical Case Reports: A Novel Task for Experts-like Text Comprehension(https://arxiv.org/abs/2503.01302)
• Keywords: extraction
• Abstract: Extracting causal relationships from a medical case report is essential for comprehending the case, particularly its diagnostic process. Since the diagnostic process is regarded as a bottom-up inference, causal relationships in cases naturally form a multi-layered tree structure. The existing tasks, such as medical relation extraction, are insufficient for capturing the causal relationships of an entire case, as they treat all relations equally without considering the hierarchical structure inherent in the diagnostic process. Thus, we propose a novel task, Causal Tree Extraction (CTE), which receives a case report and generates a causal tree with the primary disease as the root, providing an intuitive understanding of a case's diagnostic process. Subsequently, we construct a Japanese case report CTE dataset, J-Casemap, propose a generation-based CTE method that outperforms the baseline by 20.2 points in the human evaluation, and introduce evaluation metrics that reflect clinician preferences. Further experiments also show that J-Casemap enhances the performance of solving other medical tasks, such as question answering.

Title: PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation

• Authors: Linhai Zhang, Jialong Wu, Deyu Zhou, Yulan He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01303
• Pdf URL: https://arxiv.org/pdf/2503.01303
• Copy Paste: [[2503.01303]] PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation(https://arxiv.org/abs/2503.01303)
• Keywords: large language model
• Abstract: Personalized large language models (LLMs) aim to tailor their outputs to user preferences. Recent advances in parameter-efficient fine-tuning (PEFT) methods have highlighted the effectiveness of adapting population-level LLMs to personalized LLMs by fine-tuning user-specific parameters with user history. However, user data is typically sparse, making it challenging to adapt LLMs to specific user patterns. To address this challenge, we propose PROgressive PERsonalization (PROPER), a novel progressive learning framework inspired by meso-level theory in social science. PROPER bridges population-level and user-level models by grouping users based on preferences and adapting LLMs in stages. It combines a Mixture-of-Experts (MoE) structure with Low Ranked Adaptation (LoRA), using a user-aware router to assign users to appropriate groups automatically. Additionally, a LoRA-aware router is proposed to facilitate the integration of individual user LoRAs with group-level LoRAs. Experimental results show that PROPER significantly outperforms SOTA models across multiple tasks, demonstrating the effectiveness of our approach.

Title: OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging

• Authors: Yijie Tang, Jiazhao Zhang, Yuqing Lan, Yulan Guo, Dezun Dong, Chenyang Zhu, Kai Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01309
• Pdf URL: https://arxiv.org/pdf/2503.01309
• Copy Paste: [[2503.01309]] OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging(https://arxiv.org/abs/2503.01309)
• Keywords: robust, segmentation
• Abstract: Online 3D open-vocabulary segmentation of a progressively reconstructed scene is both a critical and challenging task for embodied applications. With the success of visual foundation models (VFMs) in the image domain, leveraging 2D priors to address 3D online segmentation has become a prominent research focus. Since segmentation results provided by 2D priors often require spatial consistency to be lifted into final 3D segmentation, an efficient method for identifying spatial overlap among 2D masks is essential - yet existing methods rarely achieve this in real time, mainly limiting its use to offline approaches. To address this, we propose an efficient method that lifts 2D masks generated by VFMs into a unified 3D instance using a hashing technique. By employing voxel hashing for efficient 3D scene querying, our approach reduces the time complexity of costly spatial overlap queries from $O(n^2)$ to $O(n)$. Accurate spatial associations further enable 3D merging of 2D masks through simple similarity-based filtering in a zero-shot manner, making our approach more robust to incomplete and noisy data. Evaluated on the ScanNet and SceneNN benchmarks, our approach achieves state-of-the-art performance in online, open-vocabulary 3D instance segmentation with leading efficiency.

Title: Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches

• Authors: Yifang Chen, Xuyang Guo, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01314
• Pdf URL: https://arxiv.org/pdf/2503.01314
• Copy Paste: [[2503.01314]] Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches(https://arxiv.org/abs/2503.01314)
• Keywords: transformer, large language model
• Abstract: Recently, Large Language Models (LLMs) have achieved remarkable success. A key factor behind this success is the scaling law observed by OpenAI. Specifically, for models with Transformer architecture, the test loss exhibits a power-law relationship with model size, dataset size, and the amount of computation used in training, demonstrating trends that span more than seven orders of magnitude. This scaling law challenges traditional machine learning wisdom, notably the Oscar Scissors principle, which suggests that an overparametrized algorithm will overfit the training datasets, resulting in poor test performance. Recent research has also identified the scaling law in simpler machine learning contexts, such as linear regression. However, fully explaining the scaling law in large practical models remains an elusive goal. In this work, we advance our understanding by demonstrating that the scaling law phenomenon extends to multiple regression and kernel regression settings, which are significantly more expressive and powerful than linear methods. Our analysis provides deeper insights into the scaling law, potentially enhancing our understanding of LLMs.

Title: Explainable Depression Detection in Clinical Interviews with Personalized Retrieval-Augmented Generation

• Authors: Linhai Zhang, Ziyang Gao, Deyu Zhou, Yulan He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01315
• Pdf URL: https://arxiv.org/pdf/2503.01315
• Copy Paste: [[2503.01315]] Explainable Depression Detection in Clinical Interviews with Personalized Retrieval-Augmented Generation(https://arxiv.org/abs/2503.01315)
• Keywords: interpretability
• Abstract: Depression is a widespread mental health disorder, and clinical interviews are the gold standard for assessment. However, their reliance on scarce professionals highlights the need for automated detection. Current systems mainly employ black-box neural networks, which lack interpretability, which is crucial in mental health contexts. Some attempts to improve interpretability use post-hoc LLM generation but suffer from hallucination. To address these limitations, we propose RED, a Retrieval-augmented generation framework for Explainable depression Detection. RED retrieves evidence from clinical interview transcripts, providing explanations for predictions. Traditional query-based retrieval systems use a one-size-fits-all approach, which may not be optimal for depression detection, as user backgrounds and situations vary. We introduce a personalized query generation module that combines standard queries with user-specific background inferred by LLMs, tailoring retrieval to individual contexts. Additionally, to enhance LLM performance in social intelligence, we augment LLMs by retrieving relevant knowledge from a social intelligence datastore using an event-centric retriever. Experimental results on the real-world benchmark demonstrate RED's effectiveness compared to neural networks and LLM-based baselines.

Title: CacheQuant: Comprehensively Accelerated Diffusion Models

• Authors: Xuewen Liu, Zhikai Li, Qingyi Gu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01323
• Pdf URL: https://arxiv.org/pdf/2503.01323
• Copy Paste: [[2503.01323]] CacheQuant: Comprehensively Accelerated Diffusion Models(https://arxiv.org/abs/2503.01323)
• Keywords: diffusion, generative
• Abstract: Diffusion models have gradually gained prominence in the field of image synthesis, showcasing remarkable generative capabilities. Nevertheless, the slow inference and complex networks, resulting from redundancy at both temporal and structural levels, hinder their low-latency applications in real-world scenarios. Current acceleration methods for diffusion models focus separately on temporal and structural levels. However, independent optimization at each level to further push the acceleration limits results in significant performance degradation. On the other hand, integrating optimizations at both levels can compound the acceleration effects. Unfortunately, we find that the optimizations at these two levels are not entirely orthogonal. Performing separate optimizations and then simply integrating them results in unsatisfactory performance. To tackle this issue, we propose CacheQuant, a novel training-free paradigm that comprehensively accelerates diffusion models by jointly optimizing model caching and quantization techniques. Specifically, we employ a dynamic programming approach to determine the optimal cache schedule, in which the properties of caching and quantization are carefully considered to minimize errors. Additionally, we propose decoupled error correction to further mitigate the coupled and accumulated errors step by step. Experimental results show that CacheQuant achieves a 5.18 speedup and 4 compression for Stable Diffusion on MS-COCO, with only a 0.02 loss in CLIP score. Our code are open-sourced: this https URL .

Title: MAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environments

• Authors: Zhiyin Li, Yubo Yang, Tao Yang, Xiaofeng Wu, Ziyu Guo, Bo Hu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01324
• Pdf URL: https://arxiv.org/pdf/2503.01324
• Copy Paste: [[2503.01324]] MAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environments(https://arxiv.org/abs/2503.01324)
• Keywords: attack, federate, fair
• Abstract: Federated learning enables distributed model training across clients under central coordination without raw data exchange. However, in wireless implementations, frequent parameter updates between the server and clients create significant communication overhead. While existing research assumes known channel state information (CSI) or stationary distributions, practical wireless channels exhibit non-stationary characteristics due to channel fading, user mobility, and hostile attacks. The unavailability of CSI and time-varying statistics can cause unpredictable transmission failures, exacerbating client staleness and affecting model convergence. To address these challenges, we propose an asynchronous federated learning scheduling framework for non-stationary channel environments to reduce staleness while promoting fair and efficient communication and this http URL focus on two channel scenarios: extremely non-stationary and piecewise stationary. Age of Information (AoI) quantifies client staleness under non-stationary conditions. Through a rigorous convergence analysis, we explore how AoI and per-round client participation affect learning performance. The scheduling problem is modeled within a multi-armed bandit (MAB) framework, and we derive the theoretical lower bounds on AoI regret. Based on these findings, we develop scheduling strategies for both scenarios using the GLR-CUCB and M-exp3 algorithms, also deriving their respective upper bounds on AoI regret. To address imbalanced client updates, we introduce an adaptive allocation strategy that incorporates marginal utility and fairness. Simulations demonstrate that our algorithm reduces AoI regret growth, accelerates federated learning convergence, and promotes fairer aggregation.

Title: Victim-Centred Abuse Investigations and Defenses for Social Media Platforms

• Authors: Zaid Hakami, Ashfaq Ali Shafin, Peter J. Clarke, Niki Pissinou, Bogdan Carbunar
• Subjects: cs.CR, cs.CY, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.01327
• Pdf URL: https://arxiv.org/pdf/2503.01327
• Copy Paste: [[2503.01327]] Victim-Centred Abuse Investigations and Defenses for Social Media Platforms(https://arxiv.org/abs/2503.01327)
• Keywords: privacy, protect, defense
• Abstract: Online abuse, a persistent aspect of social platform interactions, impacts user well-being and exposes flaws in platform designs that include insufficient detection efforts and inadequate victim protection measures. Ensuring safety in platform interactions requires the integration of victim perspectives in the design of abuse detection and response systems. In this paper, we conduct surveys (n = 230) and semi-structured interviews (n = 15) with students at a minority-serving institution in the US, to explore their experiences with abuse on a variety of social platforms, their defense strategies, and their recommendations for social platforms to improve abuse responses. We build on study findings to propose design requirements for abuse defense systems and discuss the role of privacy, anonymity, and abuse attribution requirements in their implementation. We introduce ARI, a blueprint for a unified, transparent, and personalized abuse response system for social platforms that sustainably detects abuse by leveraging the expertise of platform users, incentivized with proceeds obtained from abusers.

Title: PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization

• Authors: Xinyi Wan, Penghui Qi, Guangxing Huang, Jialin Li, Min Lin
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.01328
• Pdf URL: https://arxiv.org/pdf/2503.01328
• Copy Paste: [[2503.01328]] PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization(https://arxiv.org/abs/2503.01328)
• Keywords: large language model
• Abstract: Pipeline parallelism (PP) is widely used for training large language models (LLMs), yet its scalability is often constrained by high activation memory consumption as the number of in-flight microbatches grows with the degree of PP. In this paper, we focus on addressing this challenge by leveraging the under-explored memory offload strategy in PP. With empirical study, we discover that in the majority of standard configurations, at least half, and potentially all, of the activations can be offloaded with negligible overhead. In the cases where full overload is not possible, we introduce a novel selective offload strategy that decreases peak activation memory in a better-than-linear manner. Furthermore, we integrate memory offload with other techniques to jointly consider overall throughput and memory limitation. Our experiments proves that the per-device activation memory effectively reduces with the total number of stages, making PP a stronger alternative than TP, offering up to a 19\% acceleration with even lower memory consumption. The implementation is open-sourced at \href{this https URL}{this url}.

Title: Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning

• Authors: Anh Tong, Thanh Nguyen-Tang, Dongeun Lee, Duc Nguyen, Toan Tran, David Hall, Cheongwoong Kang, Jaesik Choi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01329
• Pdf URL: https://arxiv.org/pdf/2503.01329
• Copy Paste: [[2503.01329]] Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning(https://arxiv.org/abs/2503.01329)
• Keywords: interpretability, transformer, large language model
• Abstract: Recent advancements in large language models (LLMs) based on transformer architectures have sparked significant interest in understanding their inner workings. In this paper, we introduce a novel approach to modeling transformer architectures using highly flexible non-autonomous neural ordinary differential equations (ODEs). Our proposed model parameterizes all weights of attention and feed-forward blocks through neural networks, expressing these weights as functions of a continuous layer index. Through spectral analysis of the model's dynamics, we uncover an increase in eigenvalue magnitude that challenges the weight-sharing assumption prevalent in existing theoretical studies. We also leverage the Lyapunov exponent to examine token-level sensitivity, enhancing model interpretability. Our neural ODE transformer demonstrates performance comparable to or better than vanilla transformers across various configurations and datasets, while offering flexible fine-tuning capabilities that can adapt to different architectural constraints.

Title: WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models

• Authors: Jian Yuan, Ziwei He, Haoli Bai, Jingwen Leng, Bo Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01330
• Pdf URL: https://arxiv.org/pdf/2503.01330
• Copy Paste: [[2503.01330]] WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models(https://arxiv.org/abs/2503.01330)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) use key-value (KV) cache to reduce redundant computation in autoregressive generation. However, the KV cache size increases linearly during generation, leading to excessive memory usage, especially for long texts. Most KV cache compression methods evict the unimportant KV pairs to maintain a fixed cache size, which leads to the permanent loss of tokens during generation. However, singular value decomposition shows that \textit{values} do not exhibit a strong low-rank property as \textit{keys} do, suggesting that information is distributed more evenly across \textit{values}, in contrast to its more redundant distribution within \textit{keys}. Therefore, methods that evict both \textit{keys} and \textit{values} risk losing crucial information and compromise context integrity, ultimately degrading the output quality. To address this problem, we propose WeightedKV, a novel, training-free approach that discards the \textit{keys} of less important tokens, while merging their \textit{values} into neighboring tokens via a convex combination weighted by their average attention scores. In this way, the retained \textit{keys} serve as anchors that guide the generation process, while the merged \textit{values} provide a rich contextual backdrop. We assess our method on four widely used language modeling datasets, demonstrating superior performance compared to all baseline methods, particularly with a lower budget ratio.

Title: UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface

• Authors: Hao Tang, Chenwei Xie, Haiyang Wang, Xiaoyi Bao, Tingyu Weng, Pandeng Li, Yun Zheng, Liwei Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01342
• Pdf URL: https://arxiv.org/pdf/2503.01342
• Copy Paste: [[2503.01342]] UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface(https://arxiv.org/abs/2503.01342)
• Keywords: segmentation
• Abstract: Generalist models have achieved remarkable success in both language and vision-language tasks, showcasing the potential of unified modeling. However, effectively integrating fine-grained perception tasks like detection and segmentation into these models remains a significant challenge. This is primarily because these tasks often rely heavily on task-specific designs and architectures that can complicate the modeling process. To address this challenge, we present \ours, a framework that \textbf{U}nifies \textbf{F}ine-grained visual perception tasks through an \textbf{O}pen-ended language interface. By transforming all perception targets into the language space, \ours unifies object-level detection, pixel-level segmentation, and image-level vision-language tasks into a single model. Additionally, we introduce a novel embedding retrieval approach that relies solely on the language interface to support segmentation tasks. Our framework bridges the gap between fine-grained perception and vision-language tasks, significantly simplifying architectural design and training strategies while achieving comparable or superior performance to methods with intricate task-specific designs. After multi-task training on five standard visual perception datasets, \ours outperforms the previous state-of-the-art generalist models by 12.3 mAP on COCO instance segmentation and 3.3 mIoU on ADE20K semantic segmentation. Furthermore, our method seamlessly integrates with existing MLLMs, effectively combining fine-grained perception capabilities with their advanced language abilities, thereby enabling more challenging tasks such as reasoning segmentation. Code and models will be publicly available.

Title: Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness

• Authors: Tingchen Fu, Fazl Barez
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01345
• Pdf URL: https://arxiv.org/pdf/2503.01345
• Copy Paste: [[2503.01345]] Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness(https://arxiv.org/abs/2503.01345)
• Keywords: robust, large language model
• Abstract: Insensitivity to semantically-preserving variations of prompts (paraphrases) is crucial for reliable behavior and real-world deployment of large language models. However, language models exhibit significant performance degradation when faced with semantically equivalent but differently phrased prompts, and existing solutions either depend on trial-and-error prompt engineering or require computationally expensive inference-time algorithms. In this study, built on the key insight that worst-case prompts exhibit a drift in embedding space, we present Latent Adversarial Paraphrasing (LAP), a dual-loop adversarial framework: the inner loop trains a learnable perturbation to serve as a "latent continuous paraphrase" while preserving semantics through Lagrangian regulation, and the outer loop optimizes the language model parameters on these perturbations. We conduct extensive experiments to demonstrate the effectiveness of LAP across multiple LLM architectures on the RobustAlpaca benchmark with a 0.5%-4% absolution improvement on worst-case win-rate compared with vanilla supervised fine-tuning.

Title: SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graph

• Authors: Teng Lin, Yizhang Zhu, Yuyu Luo, Nan Tang
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01346
• Pdf URL: https://arxiv.org/pdf/2503.01346
• Copy Paste: [[2503.01346]] SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graph(https://arxiv.org/abs/2503.01346)
• Keywords: large language model
• Abstract: Multi-entity question answering (MEQA) poses significant challenges for large language models (LLMs), which often struggle to consolidate scattered information across multiple documents. An example question might be "What is the distribution of IEEE Fellows among various fields of study?", which requires retrieving information from diverse sources e.g., Wikipedia pages. The effectiveness of current retrieval-augmented generation (RAG) methods is limited by the LLMs' capacity to aggregate insights from numerous pages. To address this gap, this paper introduces a structured RAG (SRAG) framework that systematically organizes extracted entities into relational tables (e.g., tabulating entities with schema columns like "name" and "field of study") and then apply table-based reasoning techniques. Our approach decouples retrieval and reasoning, enabling LLMs to focus on structured data analysis rather than raw text aggregation. Extensive experiments on Wikipedia-based multi-entity QA tasks demonstrate that SRAG significantly outperforms state-of-the-art long-context LLMs and RAG solutions, achieving a 29.6% improvement in accuracy. The results underscore the efficacy of structuring unstructured data to enhance LLMs' reasoning capabilities.

Title: Combining Flow Matching and Transformers for Efficient Solution of Bayesian Inverse Problems

• Authors: Daniil Sherki, Ivan Oseledets, Ekaterina Muravleva
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01375
• Pdf URL: https://arxiv.org/pdf/2503.01375
• Copy Paste: [[2503.01375]] Combining Flow Matching and Transformers for Efficient Solution of Bayesian Inverse Problems(https://arxiv.org/abs/2503.01375)
• Keywords: transformer
• Abstract: Solving Bayesian inverse problems efficiently remains a significant challenge due to the complexity of posterior distributions and the computational cost of traditional sampling methods. Given a series of observations and the forward model, we want to recover the distribution of the parameters, conditioned on observed experimental data. We show, that combining Conditional Flow Mathching (CFM) with transformer-based architecture, we can efficiently sample from such kind of distribution, conditioned on variable number of observations.

Title: Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering

• Authors: Tim Schwabe, Louisa Siebel, Patrik Valach, Maribel Acosta
• Subjects: cs.CL, cs.DB, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01385
• Pdf URL: https://arxiv.org/pdf/2503.01385
• Copy Paste: [[2503.01385]] Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering(https://arxiv.org/abs/2503.01385)
• Keywords: robust, large language model
• Abstract: Question answering (QA) requires accurately aligning user questions with structured queries, a process often limited by the scarcity of high-quality query-natural language (Q-NL) pairs. To overcome this, we present Q-NL Verifier, an approach to generating high-quality synthetic pairs of queries and NL translations. Our approach relies on large language models (LLMs) to generate semantically precise natural language paraphrases of structured queries. Building on these synthetic Q-NL pairs, we introduce a learned verifier component that automatically determines whether a generated paraphrase is semantically equivalent to the original query. Our experiments with the well-known LC-QuAD 2.0 benchmark show that Q-NL Verifier generalizes well to paraphrases from other models and even human-authored translations. Our approach strongly aligns with human judgments across varying query complexities and outperforms existing NLP metrics in assessing semantic correctness. We also integrate the verifier into QA pipelines, showing that verifier-filtered synthetic data has significantly higher quality in terms of translation correctness and enhances NL to Q translation accuracy. Lastly, we release an updated version of the LC-QuAD 2.0 benchmark containing our synthetic Q-NL pairs and verifier scores, offering a new resource for robust and scalable QA.

Title: The Road Less Traveled: Investigating Robustness and Explainability in CNN Malware Detection

• Authors: Matteo Brosolo, Vinod Puthuvath, Mauro Conti
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.01391
• Pdf URL: https://arxiv.org/pdf/2503.01391
• Copy Paste: [[2503.01391]] The Road Less Traveled: Investigating Robustness and Explainability in CNN Malware Detection(https://arxiv.org/abs/2503.01391)
• Keywords: security, defense, attack, robust, interpretability, explainability
• Abstract: Machine learning has become a key tool in cybersecurity, improving both attack strategies and defense mechanisms. Deep learning models, particularly Convolutional Neural Networks (CNNs), have demonstrated high accuracy in detecting malware images generated from binary data. However, the decision-making process of these black-box models remains difficult to interpret. This study addresses this challenge by integrating quantitative analysis with explainability tools such as Occlusion Maps, HiResCAM, and SHAP to better understand CNN behavior in malware classification. We further demonstrate that obfuscation techniques can reduce model accuracy by up to 50%, and propose a mitigation strategy to enhance robustness. Additionally, we analyze heatmaps from multiple tests and outline a methodology for identification of artifacts, aiding researchers in conducting detailed manual investigations. This work contributes to improving the interpretability and resilience of deep learning-based intrusion detection systems

Title: Jailbreaking Generative AI: Empowering Novices to Conduct Phishing Attacks

• Authors: Rina Mishra, Gaurav Varshney, Shreya Singh
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.01395
• Pdf URL: https://arxiv.org/pdf/2503.01395
• Copy Paste: [[2503.01395]] Jailbreaking Generative AI: Empowering Novices to Conduct Phishing Attacks(https://arxiv.org/abs/2503.01395)
• Keywords: security, attack, generative
• Abstract: The rapid advancements in generative AI models, such as ChatGPT, have introduced both significant benefits and new risks within the cybersecurity landscape. This paper investigates the potential misuse of the latest AI model, ChatGPT-4o Mini, in facilitating social engineering attacks, with a particular focus on phishing, one of the most pressing cybersecurity threats today. While existing literature primarily addresses the technical aspects, such as jailbreaking techniques, none have fully explored the free and straightforward execution of a comprehensive phishing campaign by novice users using ChatGPT-4o Mini. In this study, we examine the vulnerabilities of AI-driven chatbot services in 2025, specifically how methods like jailbreaking and reverse psychology can bypass ethical safeguards, allowing ChatGPT to generate phishing content, suggest hacking tools, and assist in carrying out phishing attacks. Our findings underscore the alarming ease with which even inexperienced users can execute sophisticated phishing campaigns, emphasizing the urgent need for stronger cybersecurity measures and heightened user awareness in the age of AI.

Title: CorrNetDroid: Android Malware Detector leveraging a Correlation-based Feature Selection for Network Traffic features

• Authors: Yash Sharma, Anshul Arora
• Subjects: cs.CR, cs.MM
• Abstract URL: https://arxiv.org/abs/2503.01396
• Pdf URL: https://arxiv.org/pdf/2503.01396
• Copy Paste: [[2503.01396]] CorrNetDroid: Android Malware Detector leveraging a Correlation-based Feature Selection for Network Traffic features(https://arxiv.org/abs/2503.01396)
• Keywords: robust, steal
• Abstract: Copious mobile operating systems exist in the market, but Android remains the user's choice. Meanwhile, its growing popularity has also attracted malware developers. Researchers have proposed various static solutions for Android malware detection. However, stealthier malware evade static analysis. This raises the need for a robust Android malware detection system capable of dealing with advanced threats and overcoming the shortcomings of static analysis. Hence, this work proposes a dynamic analysis-based Android malware detection system, CorrNetDroid, that works over network traffic flows. Many traffic features exhibit overlapping ranges in normal and malware datasets. Therefore, we first rank the features using two statistical measures, crRelevance and Normalized Mean Residue Similarity (NMRS), to assess feature-class and feature-feature correlations. Thereafter, we introduce a novel correlation-based feature selection algorithm that applies NMRS on crRelevance rankings to identify the optimal feature subset for Android malware detection. Experimental results highlight that our model effectively reduces the feature set while detecting Android malware with 99.50 percent accuracy when considering only two network traffic features. Furthermore, our experiments demonstrate that the NMRS-based algorithm on crRelevance rankings outperforms statistical tests such as chi-square, ANOVA, Mann-Whitney U test, and Kruskal-Wallis test. In addition, our model surpasses various state-of-the-art Android malware detection techniques in terms of detection accuracy.

Title: Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification

• Authors: Gaozheng Pei, Shaojie Lyu, Gong Chen, Ke Ma, Qianqian Xu, Yingfei Sun, Qingming Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01407
• Pdf URL: https://arxiv.org/pdf/2503.01407
• Copy Paste: [[2503.01407]] Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification(https://arxiv.org/abs/2503.01407)
• Keywords: defense, attack, interpretability, diffusion
• Abstract: Existing diffusion-based purification methods aim to disrupt adversarial perturbations by introducing a certain amount of noise through a forward diffusion process, followed by a reverse process to recover clean examples. However, this approach is fundamentally flawed: the uniform operation of the forward process across all pixels compromises normal pixels while attempting to combat adversarial perturbations, resulting in the target model producing incorrect predictions. Simply relying on low-intensity noise is insufficient for effective defense. To address this critical issue, we implement a heterogeneous purification strategy grounded in the interpretability of neural networks. Our method decisively applies higher-intensity noise to specific pixels that the target model focuses on while the remaining pixels are subjected to only low-intensity noise. This requirement motivates us to redesign the sampling process of the diffusion model, allowing for the effective removal of varying noise levels. Furthermore, to evaluate our method against strong adaptative attack, our proposed method sharply reduces time cost and memory usage through a single-step resampling. The empirical evidence from extensive experiments across three datasets demonstrates that our method outperforms most current adversarial training and purification techniques by a substantial margin.

Title: Learning Actionable World Models for Industrial Process Control

• Authors: Peng Yan, Ahmed Abdulkadir, Gerrit A. Schatte, Giulia Anguzzi, Joonsu Gha, Nikola Pascher, Matthias Rosenthal, Yunlong Gao, Benjamin F. Grewe, Thilo Stadelmann
• Subjects: cs.LG, cs.AI, eess.SY
• Abstract URL: https://arxiv.org/abs/2503.01411
• Pdf URL: https://arxiv.org/pdf/2503.01411
• Copy Paste: [[2503.01411]] Learning Actionable World Models for Industrial Process Control(https://arxiv.org/abs/2503.01411)
• Keywords: interpretability
• Abstract: To go from (passive) process monitoring to active process control, an effective AI system must learn about the behavior of the complex system from very limited training data, forming an ad-hoc digital twin with respect to process in- and outputs that captures the consequences of actions on the process's world. We propose a novel methodology based on learning world models that disentangles process parameters in the learned latent representation, allowing for fine-grained control. Representation learning is driven by the latent factors that influence the processes through contrastive learning within a joint embedding predictive architecture. This makes changes in representations predictable from changes in inputs and vice versa, facilitating interpretability of key factors responsible for process variations, paving the way for effective control actions to keep the process within operational bounds. The effectiveness of our method is validated on the example of plastic injection molding, demonstrating practical relevance in proposing specific control actions for a notoriously unstable process.

Title: Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace

• Authors: Jia-Chen Zhang, Yu-Jie Xiong, Chun-Ming Xia, Dong-Hai Zhu, Xi-He Qiu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01419
• Pdf URL: https://arxiv.org/pdf/2503.01419
• Copy Paste: [[2503.01419]] Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace(https://arxiv.org/abs/2503.01419)
• Keywords: large language model
• Abstract: Large language model (LLM) is considered a milestone towards achieving Artificial General Intelligence (AGI). With its advanced emergent capabilities, it adapt to a wide range of specific applications. Fine-tuning LLMs for various downstream tasks has become a new paradigm. Low-Rank Adaptation (LoRA) is well-known for its parameter efficiency. It can reduce the number of parameters needed to fine-tune LLMs by several orders of magnitude. However, LoRA-based approaches encounter a significant limitation due to the bottleneck imposed by rank one decomposition. As the parameters count in LLMs increase, even rank one decomposition might surpass the number of parameters truly necessary for handling more downstream tasks. In this paper, we propose a new method for Parameter-Efficient Fine-Tuning (PEFT) via deconvolution in subspace, dubbed as DCFT. We innovatively use deconvolution to complete details and enhance knowledge in subspace incremental matrices, and dynamically control parameters by adjusting the kernel size, unconstrained by rank-one decomposition. Extensive experiments are conducted to validate the effectiveness of DCFT. Results show that compared to LoRA, DCFT achieve an 8$\times$ reduction in parameters, and still achieves highly impressive performance. Our code is available here: this https URL.

Title: Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding

• Authors: Yiming Wang, Pei Zhang, Siyuan Huang, Baosong Yang, Zhuosheng Zhang, Fei Huang, Rui Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01422
• Pdf URL: https://arxiv.org/pdf/2503.01422
• Copy Paste: [[2503.01422]] Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding(https://arxiv.org/abs/2503.01422)
• Keywords: large language model
• Abstract: Test-time scaling improves large language model performance by adding extra compute during decoding. Best-of-N (BoN) sampling serves as a common scaling technique, broadening the search space for finding better solutions from the model distribution. However, traditional BoN requires N full generations, leading to high GPU memory overhead and time latency. Moreover, some methods depend on reward models, adding computational cost and limiting domain generalization. In this paper, we propose Self-Truncation Best-of-N (ST-BoN), a novel decoding method that avoids fully generating all samplings and eliminates the need for reward models. ST-BoN introduces early sampling consistency to estimate the most promising sample, truncating suboptimal ones to free memory and accelerate inference. This pushes the sampling-efficient test-time scaling. Compared to traditional BoN, ST-BoN can reduce dynamic GPU memory overhead by over 90% and time latency by 50%, while achieving comparable or even better performance across reasoning and open-ended domains.

Title: DLF: Extreme Image Compression with Dual-generative Latent Fusion

• Authors: Naifu Xue, Zhaoyang Jia, Jiahao Li, Bin Li, Yuan Zhang, Yan Lu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.01428
• Pdf URL: https://arxiv.org/pdf/2503.01428
• Copy Paste: [[2503.01428]] DLF: Extreme Image Compression with Dual-generative Latent Fusion(https://arxiv.org/abs/2503.01428)
• Keywords: diffusion, generative
• Abstract: Recent studies in extreme image compression have achieved remarkable performance by compressing the tokens from generative tokenizers. However, these methods often prioritize clustering common semantics within the dataset, while overlooking the diverse details of individual objects. Consequently, this results in suboptimal reconstruction fidelity, especially at low bitrates. To address this issue, we introduce a Dual-generative Latent Fusion (DLF) paradigm. DLF decomposes the latent into semantic and detail elements, compressing them through two distinct branches. The semantic branch clusters high-level information into compact tokens, while the detail branch encodes perceptually critical details to enhance the overall fidelity. Additionally, we propose a cross-branch interactive design to reduce redundancy between the two branches, thereby minimizing the overall bit cost. Experimental results demonstrate the impressive reconstruction quality of DLF even below 0.01 bits per pixel (bpp). On the CLIC2020 test set, our method achieves bitrate savings of up to 27.93% on LPIPS and 53.55% on DISTS compared to MS-ILLM. Furthermore, DLF surpasses recent diffusion-based codecs in visual fidelity while maintaining a comparable level of generative realism. Code will be available later.

Title: How simple can you go? An off-the-shelf transformer approach to molecular dynamics

• Authors: Max Eissler, Tim Korjakow, Stefan Ganscha, Oliver T. Unke, Klaus-Robert Müller, Stefan Gugler
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01431
• Pdf URL: https://arxiv.org/pdf/2503.01431
• Copy Paste: [[2503.01431]] How simple can you go? An off-the-shelf transformer approach to molecular dynamics(https://arxiv.org/abs/2503.01431)
• Keywords: transformer
• Abstract: Most current neural networks for molecular dynamics (MD) include physical inductive biases, resulting in specialized and complex architectures. This is in contrast to most other machine learning domains, where specialist approaches are increasingly replaced by general-purpose architectures trained on vast datasets. In line with this trend, several recent studies have questioned the necessity of architectural features commonly found in MD models, such as built-in rotational equivariance or energy conservation. In this work, we contribute to the ongoing discussion by evaluating the performance of an MD model with as few specialized architectural features as possible. We present a recipe for MD using an Edge Transformer, an ``off-the-shelf'' transformer architecture that has been minimally modified for the MD domain, termed MD-ET. Our model implements neither built-in equivariance nor energy conservation. We use a simple supervised pre-training scheme on $\sim$30 million molecular structures from the QCML database. Using this ``off-the-shelf'' approach, we show state-of-the-art results on several benchmarks after fine-tuning for a small number of steps. Additionally, we examine the effects of being only approximately equivariant and energy conserving for MD simulations, proposing a novel method for distinguishing the errors resulting from non-equivariance from other sources of inaccuracies like numerical rounding errors. While our model exhibits runaway energy increases on larger structures, we show approximately energy-conserving NVE simulations for a range of small structures.

Title: Generative Human Geometry Distribution

• Authors: Xiangjun Tang, Biao Zhang, Peter Wonka
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01448
• Pdf URL: https://arxiv.org/pdf/2503.01448
• Copy Paste: [[2503.01448]] Generative Human Geometry Distribution(https://arxiv.org/abs/2503.01448)
• Keywords: generative
• Abstract: Realistic human geometry generation is an important yet challenging task, requiring both the preservation of fine clothing details and the accurate modeling of clothing-pose interactions. Geometry distributions, which can model the geometry of a single human as a distribution, provide a promising representation for high-fidelity synthesis. However, applying geometry distributions for human generation requires learning a dataset-level distribution over numerous individual geometry distributions. To address the resulting challenges, we propose a novel 3D human generative framework that, for the first time, models the distribution of human geometry distributions. Our framework operates in two stages: first, generating the human geometry distribution, and second, synthesizing high-fidelity humans by sampling from this distribution. We validate our method on two tasks: pose-conditioned 3D human generation and single-view-based novel pose generation. Experimental results demonstrate that our approach achieves the best quantitative results in terms of realism and geometric fidelity, outperforming state-of-the-art generative methods.

Title: Structural Deep Encoding for Table Question Answering

• Authors: Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01457
• Pdf URL: https://arxiv.org/pdf/2503.01457
• Copy Paste: [[2503.01457]] Structural Deep Encoding for Table Question Answering(https://arxiv.org/abs/2503.01457)
• Keywords: transformer
• Abstract: Although Transformers-based architectures excel at processing textual information, their naive adaptation for tabular data often involves flattening the table structure. This simplification can lead to the loss of essential inter-dependencies between rows, columns, and cells, while also posing scalability challenges for large tables. To address these issues, prior works have explored special tokens, structured embeddings, and sparse attention patterns. In this paper, we conduct a comprehensive analysis of tabular encoding techniques, which highlights the crucial role of attention sparsity in preserving structural information of tables. We also introduce a set of novel sparse attention mask designs for tabular data, that not only enhance computational efficiency but also preserve structural integrity, leading to better overall performance.

Title: MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism

• Authors: Zhixiong Nan, Xianghong Li, Jifeng Dai, Tao Xiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01463
• Pdf URL: https://arxiv.org/pdf/2503.01463
• Copy Paste: [[2503.01463]] MI-DETR: An Object Detection Model with Multi-time Inquiries Mechanism(https://arxiv.org/abs/2503.01463)
• Keywords: interpretability
• Abstract: Based on analyzing the character of cascaded decoder architecture commonly adopted in existing DETR-like models, this paper proposes a new decoder architecture. The cascaded decoder architecture constrains object queries to update in the cascaded direction, only enabling object queries to learn relatively-limited information from image features. However, the challenges for object detection in natural scenes (e.g., extremely-small, heavily-occluded, and confusingly mixed with the background) require an object detection model to fully utilize image features, which motivates us to propose a new decoder architecture with the parallel Multi-time Inquiries (MI) mechanism. MI enables object queries to learn more comprehensive information, and our MI based model, MI-DETR, outperforms all existing DETR-like models on COCO benchmark under different backbones and training epochs, achieving +2.3 AP and +0.6 AP improvements compared to the most representative model DINO and SOTA model Relation-DETR under ResNet-50 backbone. In addition, a series of diagnostic and visualization experiments demonstrate the effectiveness, rationality, and interpretability of MI.

Title: Rethinking Data: Towards Better Performing Domain-Specific Small Language Models

• Authors: Boris Nazarov, Darya Frolova, Yackov Lubarsky, Alexei Gaissinski, Pavel Kisilev
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01464
• Pdf URL: https://arxiv.org/pdf/2503.01464
• Copy Paste: [[2503.01464]] Rethinking Data: Towards Better Performing Domain-Specific Small Language Models(https://arxiv.org/abs/2503.01464)
• Keywords: extraction, large language model
• Abstract: Fine-tuning of Large Language Models (LLMs) for downstream tasks, performed on domain-specific data has shown significant promise. However, commercial use of such LLMs is limited by the high computational cost required for their deployment at scale. On the other hand, small Language Models (LMs) are much more cost effective but have subpar performance in a similar setup. This paper presents our approach to finetuning a small LM, that reaches high accuracy in multiple choice question answering task. We achieve this by improving data quality at each stage of the LM training pipeline. In particular, we start with data structuring resulting in extraction of compact, semantically meaningful text chunks used by a retriever. This allows more efficient knowledge digestion by the LM. Further, we improve the retrieved context by training a lightweight Chunk Re-Ranker (CRR) that generates more accurate relative relevance chunk scores. Finally, we improve the model generalization ability by merging the models fine-tuned with different parameters on different data subsets. We present detailed procedure descriptions, and corresponding experimental findings that show the improvements of each one of the proposed techniques.

Title: SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction

• Authors: Lu Dai, Yijie Xu, Jinhui Ye, Hao Liu, Hui Xiong
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01478
• Pdf URL: https://arxiv.org/pdf/2503.01478
• Copy Paste: [[2503.01478]] SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction(https://arxiv.org/abs/2503.01478)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated improved generation performance by incorporating externally retrieved knowledge, a process known as retrieval-augmented generation (RAG). Despite the potential of this approach, existing studies evaluate RAG effectiveness by 1) assessing retrieval and generation components jointly, which obscures retrieval's distinct contribution, or 2) examining retrievers using traditional metrics such as NDCG, which creates a gap in understanding retrieval's true utility in the overall generation process. To address the above limitations, in this work, we introduce an automatic evaluation method that measures retrieval quality through the lens of information gain within the RAG framework. Specifically, we propose Semantic Perplexity (SePer), a metric that captures the LLM's internal belief about the correctness of the retrieved information. We quantify the utility of retrieval by the extent to which it reduces semantic perplexity post-retrieval. Extensive experiments demonstrate that SePer not only aligns closely with human preferences but also offers a more precise and efficient evaluation of retrieval utility across diverse RAG scenarios.

Title: Revisiting Locally Differentially Private Protocols: Towards Better Trade-offs in Privacy, Utility, and Attack Resistance

• Authors: Héber H. Arcolezi, Sébastien Gambs
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.01482
• Pdf URL: https://arxiv.org/pdf/2503.01482
• Copy Paste: [[2503.01482]] Revisiting Locally Differentially Private Protocols: Towards Better Trade-offs in Privacy, Utility, and Attack Resistance(https://arxiv.org/abs/2503.01482)
• Keywords: security, privacy, protect, attack, robust
• Abstract: Local Differential Privacy (LDP) offers strong privacy protection, especially in settings in which the server collecting the data is untrusted. However, designing LDP mechanisms that achieve an optimal trade-off between privacy, utility, and robustness to adversarial inference attacks remains challenging. In this work, we introduce a general multi-objective optimization framework for refining LDP protocols, enabling the joint optimization of privacy and utility under various adversarial settings. While our framework is flexible enough to accommodate multiple privacy and security attacks as well as utility metrics, in this paper we specifically optimize for Attacker Success Rate (ASR) under distinguishability attack as a measure of privacy and Mean Squared Error (MSE) as a measure of utility. We systematically revisit these trade-offs by analyzing eight state-of-the-art LDP protocols and proposing refined counterparts that leverage tailored optimization techniques. Experimental results demonstrate that our proposed adaptive mechanisms consistently outperform their non-adaptive counterparts, reducing ASR by up to five orders of magnitude while maintaining competitive utility. Analytical derivations also confirm the effectiveness of our mechanisms, moving them closer to the ASR-MSE Pareto frontier.

Title: KurTail : Kurtosis-based LLM Quantization

• Authors: Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski, Evangelos Eleftheriou, Martino Dazzi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01483
• Pdf URL: https://arxiv.org/pdf/2503.01483
• Copy Paste: [[2503.01483]] KurTail : Kurtosis-based LLM Quantization(https://arxiv.org/abs/2503.01483)
• Keywords: large language model
• Abstract: One of the challenges of quantizing a large language model (LLM) is the presence of outliers. Outliers often make uniform quantization schemes less effective, particularly in extreme cases such as 4-bit quantization. We introduce KurTail, a new post-training quantization (PTQ) scheme that leverages Kurtosis-based rotation to mitigate outliers in the activations of LLMs. Our method optimizes Kurtosis as a measure of tailedness. This approach enables the quantization of weights, activations, and the KV cache in 4 bits. We utilize layer-wise optimization, ensuring memory efficiency. KurTail outperforms existing quantization methods, offering a 13.3\% boost in MMLU accuracy and a 15.5\% drop in Wiki perplexity compared to QuaRot. It also outperforms SpinQuant with a 2.6\% MMLU gain and reduces perplexity by 2.9\%, all while reducing the training cost. For comparison, learning the rotation using SpinQuant for Llama3-70B requires at least four NVIDIA H100 80GB GPUs, whereas our method requires only a single GPU, making it a more accessible solution for consumer GPU.

Title: Improving Retrospective Language Agents via Joint Policy Gradient Optimization

• Authors: Xueyang Feng, Bo Lan, Quanyu Dai, Lei Wang, Jiakai Tang, Xu Chen, Zhenhua Dong, Ji-Rong Wen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01490
• Pdf URL: https://arxiv.org/pdf/2503.01490
• Copy Paste: [[2503.01490]] Improving Retrospective Language Agents via Joint Policy Gradient Optimization(https://arxiv.org/abs/2503.01490)
• Keywords: large language model
• Abstract: In recent research advancements within the community, large language models (LLMs) have sparked great interest in creating autonomous agents. However, current prompt-based agents often heavily rely on large-scale LLMs. Meanwhile, although fine-tuning methods significantly enhance the capabilities of smaller LLMs, the fine-tuned agents often lack the potential for self-reflection and self-improvement. To address these challenges, we introduce a novel agent framework named RetroAct, which is a framework that jointly optimizes both task-planning and self-reflective evolution capabilities in language agents. Specifically, we develop a two-stage joint optimization process that integrates imitation learning and reinforcement learning, and design an off-policy joint policy gradient optimization algorithm with imitation learning regularization to enhance the data efficiency and training stability in agent tasks. RetroAct significantly improves the performance of open-source models, reduces dependency on closed-source LLMs, and enables fine-tuned agents to learn and evolve continuously. We conduct extensive experiments across various testing environments, demonstrating RetroAct has substantial improvements in task performance and decision-making processes.

Title: What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret

• Authors: Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, Lin Yan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01491
• Pdf URL: https://arxiv.org/pdf/2503.01491
• Copy Paste: [[2503.01491]] What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret(https://arxiv.org/abs/2503.01491)
• Keywords: large language model
• Abstract: Reinforcement learning (RL) is pivotal for enabling large language models (LLMs) to generate long chains of thought (CoT) for complex tasks like math and reasoning. However, Proximal Policy Optimization (PPO), effective in many RL scenarios, fails in long CoT tasks. This paper identifies that value initialization bias and reward signal decay are the root causes of PPO's failure. We propose Value-Calibrated PPO (VC-PPO) to address these issues. In VC-PPO, the value model is pretrained to tackle initialization bias, and the Generalized Advantage Estimation (GAE) computation is decoupled between the actor and critic to mitigate reward signal decay. Experiments on the American Invitational Mathematics Examination (AIME) show that VC-PPO significantly boosts PPO performance. Ablation studies show that techniques in VC-PPO are essential in enhancing PPO for long CoT tasks.

Title: Llama-3.1-Sherkala-8B-Chat: An Open Large Language Model for Kazakh

• Authors: Fajri Koto, Rituraj Joshi, Nurdaulet Mukhituly, Yuxia Wang, Zhuohan Xie, Rahul Pal, Daniil Orel, Parvez Mullah, Diana Turmakhan, Maiya Goloburda, Mohammed Kamran, Samujjwal Ghosh, Bokang Jia, Jonibek Mansurov, Mukhammed Togmanov, Debopriyo Banerjee, Nurkhan Laiyk, Akhmed Sakip, Xudong Han, Ekaterina Kochmar, Alham Fikri Aji, Aaryamonvikram Singh, Alok Anil Jadhav, Satheesh Katipomu, Samta Kamboj, Monojit Choudhury, Gurpreet Gosal, Gokul Ramakrishnan, Biswajit Mishra, Sarath Chandran, Avraham Sheinin, Natalia Vassilieva, Neha Sengupta, Larry Murray, Preslav Nakov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01493
• Pdf URL: https://arxiv.org/pdf/2503.01493
• Copy Paste: [[2503.01493]] Llama-3.1-Sherkala-8B-Chat: An Open Large Language Model for Kazakh(https://arxiv.org/abs/2503.01493)
• Keywords: generative, large language model
• Abstract: Llama-3.1-Sherkala-8B-Chat, or Sherkala-Chat (8B) for short, is a state-of-the-art instruction-tuned open generative large language model (LLM) designed for Kazakh. Sherkala-Chat (8B) aims to enhance the inclusivity of LLM advancements for Kazakh speakers. Adapted from the LLaMA-3.1-8B model, Sherkala-Chat (8B) is trained on 45.3B tokens across Kazakh, English, Russian, and Turkish. With 8 billion parameters, it demonstrates strong knowledge and reasoning abilities in Kazakh, significantly outperforming existing open Kazakh and multilingual models of similar scale while achieving competitive performance in English. We release Sherkala-Chat (8B) as an open-weight instruction-tuned model and provide a detailed overview of its training, fine-tuning, safety alignment, and evaluation, aiming to advance research and support diverse real-world applications.

Title: Liger: Linearizing Large Language Models to Gated Recurrent Structures

• Authors: Disen Lan, Weigao Sun, Jiaxi Hu, Jusen Du, Yu Cheng
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01496
• Pdf URL: https://arxiv.org/pdf/2503.01496
• Copy Paste: [[2503.01496]] Liger: Linearizing Large Language Models to Gated Recurrent Structures(https://arxiv.org/abs/2503.01496)
• Keywords: transformer, large language model
• Abstract: Transformers with linear recurrent modeling offer linear-time training and constant-memory inference. Despite their demonstrated efficiency and performance, pretraining such non-standard architectures from scratch remains costly and risky. The linearization of large language models (LLMs) transforms pretrained standard models into linear recurrent structures, enabling more efficient deployment. However, current linearization methods typically introduce additional feature map modules that require extensive fine-tuning and overlook the gating mechanisms used in state-of-the-art linear recurrent models. To address these issues, this paper presents Liger, short for Linearizing LLMs to gated recurrent structures. Liger is a novel approach for converting pretrained LLMs into gated linear recurrent models without adding extra parameters. It repurposes the pretrained key matrix weights to construct diverse gating mechanisms, facilitating the formation of various gated recurrent structures while avoiding the need to train additional components from scratch. Using lightweight fine-tuning with Low-Rank Adaptation (LoRA), Liger restores the performance of the linearized gated recurrent models to match that of the original LLMs. Additionally, we introduce Liger Attention, an intra-layer hybrid attention mechanism, which significantly recovers 93\% of the Transformer-based LLM at 0.02\% pre-training tokens during the linearization process, achieving competitive results across multiple benchmarks, as validated on models ranging from 1B to 8B parameters. Code is available at this https URL.

Title: An Approach for Air Drawing Using Background Subtraction and Contour Extraction

• Authors: Ramkrishna Acharya
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01497
• Pdf URL: https://arxiv.org/pdf/2503.01497
• Copy Paste: [[2503.01497]] An Approach for Air Drawing Using Background Subtraction and Contour Extraction(https://arxiv.org/abs/2503.01497)
• Keywords: extraction
• Abstract: In this paper, we propose a novel approach for air drawing that uses image processing techniques to draw on the screen by moving fingers in the air. This approach benefits a wide range of applications such as sign language, in-air drawing, and 'writing' in the air as a new way of input. The approach starts with preparing ROI (Region of Interest) background images by taking a running average in initial camera frames and later subtracting it from the live camera frames to get a binary mask image. We calculate the pointer's position as the top of the contour on the binary image. When drawing a circle on the canvas in that position, it simulates the drawing. Furthermore, we combine the pre-trained Tesseract model for OCR purposes. To address the false contours, we perform hand detection based on the haar cascade before performing the background subtraction. In an experimental setup, we achieved a latency of only 100ms in air drawing. The code used to this research are available in GitHub as this https URL

Title: SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity

• Authors: Xiangyu Xi, Deyang Kong, Jian Yang, Jiawei Yang, Zhengyu Chen, Wei Wang, Jingang Wang, Xunliang Cai, Shikun Zhang, Wei Ye
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01506
• Pdf URL: https://arxiv.org/pdf/2503.01506
• Copy Paste: [[2503.01506]] SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity(https://arxiv.org/abs/2503.01506)
• Keywords: large language model
• Abstract: Existing pretraining data mixing methods for large language models (LLMs) typically follow a domain-wise methodology, a top-down process that first determines domain weights and then performs uniform data sampling across each domain. However, these approaches neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset. Further, uniform sampling within domains ignores fine-grained sample-specific features, potentially leading to suboptimal data distribution. To address these shortcomings, we propose a novel sample-wise data mixture approach based on a bottom-up paradigm. This method performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample, thereby dynamically determining the optimal domain distribution. Comprehensive experiments across multiple downstream tasks and perplexity assessments demonstrate that SampleMix surpasses existing domain-based methods. Meanwhile, SampleMix requires 1.4x to 2.1x training steps to achieves the baselines' performance, highlighting the substantial potential of SampleMix to optimize pre-training data.

Title: Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey

• Authors: Katerina Korre, Dimitris Tsirmpas, Nikos Gkoumas, Emma Cabalé, Dionysis Kontarinis, Danai Myrtzani, Theodoros Evgeniou, Ion Androutsopoulos, John Pavlopoulos
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01513
• Pdf URL: https://arxiv.org/pdf/2503.01513
• Copy Paste: [[2503.01513]] Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey(https://arxiv.org/abs/2503.01513)
• Keywords: large language model
• Abstract: We present a survey of methods for assessing and enhancing the quality of online discussions, focusing on the potential of Large Language Models (LLMs). While online discourses aim, at least in theory, to foster mutual understanding, they often devolve into harmful exchanges, such as hate speech, threatening social cohesion and democratic values. Recent advancements in LLMs enable facilitation agents that not only moderate content, but also actively improve the quality of interactions. Our survey synthesizes ideas from Natural Language Processing (NLP) and Social Sciences to provide (a) a new taxonomy on discussion quality evaluation, (b) an overview of intervention and facilitation strategies, along with a new taxonomy on conversation facilitation datasets, (c) an LLM-oriented roadmap of good practices and future research directions, from technological and societal perspectives.

Title: R2VF: A Two-Step Regularization Algorithm to Cluster Categories in GLMs

• Authors: Yuval Ben Dror
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01521
• Pdf URL: https://arxiv.org/pdf/2503.01521
• Copy Paste: [[2503.01521]] R2VF: A Two-Step Regularization Algorithm to Cluster Categories in GLMs(https://arxiv.org/abs/2503.01521)
• Keywords: interpretability
• Abstract: Over recent decades, extensive research has aimed to overcome the restrictive underlying assumptions required for a Generalized Linear Model to generate accurate and meaningful predictions. These efforts include regularizing coefficients, selecting features, and clustering ordinal categories, among other approaches. Despite these advances, efficiently clustering nominal categories in GLMs without incurring high computational costs remains a challenge. This paper introduces Ranking to Variable Fusion (R2VF), a two-step method designed to efficiently fuse nominal and ordinal categories in GLMs. By first transforming nominal features into an ordinal framework via regularized regression and then applying variable fusion, R2VF strikes a balance between model complexity and interpretability. We demonstrate the effectiveness of R2VF through comparisons with other methods, highlighting its performance in addressing overfitting and finding a proper set of covariates.

Title: Formally Discovering and Reproducing Network Protocols Vulnerabilities

• Authors: Christophe Crochet, John Aoga, Axel Legay
• Subjects: cs.CR, cs.FL, cs.NI
• Abstract URL: https://arxiv.org/abs/2503.01538
• Pdf URL: https://arxiv.org/pdf/2503.01538
• Copy Paste: [[2503.01538]] Formally Discovering and Reproducing Network Protocols Vulnerabilities(https://arxiv.org/abs/2503.01538)
• Keywords: security, attack, robust
• Abstract: The rapid evolution of cyber threats has increased the need for robust methods to discover vulnerabilities in increasingly complex and diverse network protocols. This paper introduces Network Attack-centric Compositional Testing (NACT), a novel methodology designed to discover new vulnerabilities in network protocols and create scenarios to reproduce these vulnerabilities through attacker models. NACT integrates composable attacker specifications, formal specification mutations, and randomized constraint-solving techniques to generate sophisticated attack scenarios and test cases. The methodology enables comprehensive testing of both single-protocol and multi-protocol interactions. Through case studies involving a custom minimalist protocol (MiniP) and five widely used QUIC implementations, NACT is shown to effectively identify, reproduce, and find new real-world vulnerabilities such as version negotiation abuse. Additionally, by comparing the current and older versions of these QUIC implementations, NACT demonstrates its ability to detect both persistent vulnerabilities and regressions. Finally, by supporting cross-protocol testing within a black-box testing framework, NACT provides a versatile approach to improve the security of network protocols.

Title: Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic Language

• Authors: Xi Chen, Shuo Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01539
• Pdf URL: https://arxiv.org/pdf/2503.01539
• Copy Paste: [[2503.01539]] Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic Language(https://arxiv.org/abs/2503.01539)
• Keywords: large language model
• Abstract: The rapid development of large language models (LLMs) gives rise to ethical concerns about their performance, while opening new avenues for developing toxic language detection techniques. However, LLMs' unethical output and their capability of detecting toxicity have primarily been tested on language data that do not demand complex meaning inference, such as the biased associations of 'he' with programmer and 'she' with household. Nowadays toxic language adopts a much more creative range of implicit forms, thanks to advanced censorship. In this study, we collect authentic toxic interactions that evade online censorship and that are verified by human annotators as inference intensive. To evaluate and improve LLMs' reasoning of the authentic implicit toxic language, we propose a new prompting method, Pragmatic Inference Chain (PIC), drawn on interdisciplinary findings from cognitive science and linguistics. The PIC prompting significantly improves the success rate of GPT-4o, Llama-3.1-70B-Instruct, and DeepSeek-v2.5 in identifying implicit toxic language, compared to both direct prompting and Chain-of-Thought. In addition, it also facilitates the models to produce more explicit and coherent reasoning processes, hence can potentially be generalized to other inference-intensive tasks, e.g., understanding humour and metaphors.

Title: Revisiting Large Language Model Pruning using Neuron Semantic Attribution

• Authors: Yizhuo Ding, Xinwei Sun, Yanwei Fu, Guosheng Hu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01542
• Pdf URL: https://arxiv.org/pdf/2503.01542
• Copy Paste: [[2503.01542]] Revisiting Large Language Model Pruning using Neuron Semantic Attribution(https://arxiv.org/abs/2503.01542)
• Keywords: large language model
• Abstract: Model pruning technique is vital for accelerating large language models by reducing their size and computational requirements. However, the generalizability of existing pruning methods across diverse datasets and tasks remains unclear. Thus, we conduct extensive evaluations on 24 datasets and 4 tasks using popular pruning methods. Based on these evaluations, we find and then investigate that calibration set greatly affect the performance of pruning methods. In addition, we surprisingly find a significant performance drop of existing pruning methods in sentiment classification tasks. To understand the link between performance drop and pruned neurons, we propose Neuron Semantic Attribution, which learns to associate each neuron with specific semantics. This method first makes the unpruned neurons of LLMs explainable.

Title: Compositional Reasoning with Transformers, RNNs, and Chain of Thought

• Authors: Gilad Yehudai, Noah Amsel, Joan Bruna
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01544
• Pdf URL: https://arxiv.org/pdf/2503.01544
• Copy Paste: [[2503.01544]] Compositional Reasoning with Transformers, RNNs, and Chain of Thought(https://arxiv.org/abs/2503.01544)
• Keywords: transformer
• Abstract: We study and compare the expressive power of transformers, RNNs, and transformers with chain of thought tokens on a simple and natural class of problems we term Compositional Reasoning Questions (CRQ). This family captures problems like evaluating Boolean formulas and multi-step word problems. Assuming standard hardness assumptions from circuit complexity and communication complexity, we prove that none of these three architectures is capable of solving CRQs unless some hyperparameter (depth, embedding dimension, and number of chain of thought tokens, respectively) grows with the size of the input. We also provide a construction for each architecture that solves CRQs. For transformers, our construction uses depth that is logarithmic in the problem size. For RNNs, logarithmic embedding dimension is necessary and sufficient, so long as the inputs are provided in a certain order. (Otherwise, a linear dimension is necessary). For transformers with chain of thought, our construction uses $n$ CoT tokens. These results show that, while CRQs are inherently hard, there are several different ways for language models to overcome this hardness. Even for a single class of problems, each architecture has strengths and weaknesses, and none is strictly better than the others.

Title: MoCFL: Mobile Cluster Federated Learning Framework for Highly Dynamic Network

• Authors: Kai Fang, Jiangtao Deng, Chengzu Dong, Usman Naseem, Tongcun Liu, Hailin Feng, Wei Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01557
• Pdf URL: https://arxiv.org/pdf/2503.01557
• Copy Paste: [[2503.01557]] MoCFL: Mobile Cluster Federated Learning Framework for Highly Dynamic Network(https://arxiv.org/abs/2503.01557)
• Keywords: robust, federate
• Abstract: Frequent fluctuations of client nodes in highly dynamic mobile clusters can lead to significant changes in feature space distribution and data drift, posing substantial challenges to the robustness of existing federated learning (FL) strategies. To address these issues, we proposed a mobile cluster federated learning framework (MoCFL). MoCFL enhances feature aggregation by introducing an affinity matrix that quantifies the similarity between local feature extractors from different clients, addressing dynamic data distribution changes caused by frequent client churn and topology changes. Additionally, MoCFL integrates historical and current feature information when training the global classifier, effectively mitigating the catastrophic forgetting problem frequently encountered in mobile scenarios. This synergistic combination ensures that MoCFL maintains high performance and stability in dynamically changing mobile environments. Experimental results on the UNSW-NB15 dataset show that MoCFL excels in dynamic environments, demonstrating superior robustness and accuracy while maintaining reasonable training costs.

Title: Attention Condensation via Sparsity Induced Regularized Training

• Authors: Eli Sason, Darya Frolova, Boris Nazarov, Felix Goldberd
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01564
• Pdf URL: https://arxiv.org/pdf/2503.01564
• Copy Paste: [[2503.01564]] Attention Condensation via Sparsity Induced Regularized Training(https://arxiv.org/abs/2503.01564)
• Keywords: transformer, large language model
• Abstract: As the context window expands, self-attention increasingly dominates the transformer's inference time. Therefore, accelerating attention computation while minimizing performance degradation is essential for the efficient deployment of Large Language Models (LLMs). In this study we extend a theoretical framework of attention sparsity in LLMs. A customized loss function is designed to enforce the sparsity by restricting the number of top elements in the attention matrix. We perform an initial set of evaluations with GPT-2 to show the effectiveness of our sparsification approach. The attention matrices of the models trained with the proposed loss are both sparse and effective in capturing relevant input dependencies. We now continue working to demonstrate the value of our approach on larger models and different architectures.

Title: Meta Learning-Driven Iterative Refinement for Robust Anomaly Detection in Industrial Inspection

• Authors: Muhammad Aqeel, Shakiba Sharifi, Marco Cristani, Francesco Setti
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01569
• Pdf URL: https://arxiv.org/pdf/2503.01569
• Copy Paste: [[2503.01569]] Meta Learning-Driven Iterative Refinement for Robust Anomaly Detection in Industrial Inspection(https://arxiv.org/abs/2503.01569)
• Keywords: robust
• Abstract: This study investigates the performance of robust anomaly detection models in industrial inspection, focusing particularly on their ability to handle noisy data. We propose to leverage the adaptation ability of meta learning approaches to identify and reject noisy training data to improve the learning process. In our model, we employ Model Agnostic Meta Learning (MAML) and an iterative refinement process through an Inter-Quartile Range rejection scheme to enhance their adaptability and robustness. This approach significantly improves the models capability to distinguish between normal and defective conditions. Our results of experiments conducted on well known MVTec and KSDD2 datasets demonstrate that the proposed method not only excels in environments with substantial noise but can also contribute in case of a clear training set, isolating those samples that are relatively out of distribution, thus offering significant improvements over traditional models.

Title: MRI super-resolution reconstruction using efficient diffusion probabilistic model with residual shifting

• Authors: Mojtaba Safari, Shansong Wang, Zach Eidex, Qiang Li, Erik H. Middlebrooks, David S. Yu, Xiaofeng Yang
• Subjects: cs.CV, physics.med-ph
• Abstract URL: https://arxiv.org/abs/2503.01576
• Pdf URL: https://arxiv.org/pdf/2503.01576
• Copy Paste: [[2503.01576]] MRI super-resolution reconstruction using efficient diffusion probabilistic model with residual shifting(https://arxiv.org/abs/2503.01576)
• Keywords: robust, diffusion, transformer
• Abstract: Objective:This study introduces a residual error-shifting mechanism that drastically reduces sampling steps while preserving critical anatomical details, thus accelerating MRI reconstruction. Approach:We propose a novel diffusion-based SR framework called Res-SRDiff, which integrates residual error shifting into the forward diffusion process. This enables efficient HR image reconstruction by aligning the degraded HR and LR this http URL evaluated Res-SRDiff on ultra-high-field brain T1 MP2RAGE maps and T2-weighted prostate images, comparing it with Bicubic, Pix2pix, CycleGAN, and a conventional denoising diffusion probabilistic model with vision transformer backbone (TM-DDPM), using quantitative metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), gradient magnitude similarity deviation (GMSD), and learned perceptual image patch similarity (LPIPS). Main results: Res-SRDiff significantly outperformed all comparative methods in terms of PSNR, SSIM, and GMSD across both datasets, with statistically significant improvements (p-values<<0.05). The model achieved high-fidelity image restoration with only four sampling steps, drastically reducing computational time to under one second per slice, which is substantially faster than conventional TM-DDPM with around 20 seconds per slice. Qualitative analyses further demonstrated that Res-SRDiff effectively preserved fine anatomical details and lesion morphology in both brain and pelvic MRI images. Significance: Our findings show that Res-SRDiff is an efficient and accurate MRI SR method, markedly improving computational efficiency and image quality. Integrating residual error shifting into the diffusion process allows for rapid and robust HR image reconstruction, enhancing clinical MRI workflows and advancing medical imaging research. The source at:this https URL

Title: Heterogeneity Matters even More in Distributed Learning: Study from Generalization Perspective

• Authors: Masoud Kavian, Milad Sefidgaran, Abdellatif Zaidi, Romain Chor
• Subjects: cs.LG, cs.IT, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01598
• Pdf URL: https://arxiv.org/pdf/2503.01598
• Copy Paste: [[2503.01598]] Heterogeneity Matters even More in Distributed Learning: Study from Generalization Perspective(https://arxiv.org/abs/2503.01598)
• Keywords: federate
• Abstract: In this paper, we investigate the effect of data heterogeneity across clients on the performance of distributed learning systems, i.e., one-round Federated Learning, as measured by the associated generalization error. Specifically, \(K\) clients have each \(n\) training samples generated independently according to a possibly different data distribution and their individually chosen models are aggregated by a central server. We study the effect of the discrepancy between the clients' data distributions on the generalization error of the aggregated model. First, we establish in-expectation and tail upper bounds on the generalization error in terms of the distributions. In part, the bounds extend the popular Conditional Mutual Information (CMI) bound which was developed for the centralized learning setting, i.e., \(K=1\), to the distributed learning setting with arbitrary number of clients $K \geq 1$. Then, we use a connection with information theoretic rate-distortion theory to derive possibly tighter \textit{lossy} versions of these bounds. Next, we apply our lossy bounds to study the effect of data heterogeneity across clients on the generalization error for distributed classification problem in which each client uses Support Vector Machines (D-SVM). In this case, we establish explicit generalization error bounds which depend explicitly on the data heterogeneity degree. It is shown that the bound gets smaller as the degree of data heterogeneity across clients gets higher, thereby suggesting that D-SVM generalizes better when the dissimilarity between the clients' training samples is bigger. This finding, which goes beyond D-SVM, is validated experimentally through a number of experiments.

Title: Evaluating Stenosis Detection with Grounding DINO, YOLO, and DINO-DETR

• Authors: Muhammad Musab Ansari
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01601
• Pdf URL: https://arxiv.org/pdf/2503.01601
• Copy Paste: [[2503.01601]] Evaluating Stenosis Detection with Grounding DINO, YOLO, and DINO-DETR(https://arxiv.org/abs/2503.01601)
• Keywords: transformer
• Abstract: Detecting stenosis in coronary angiography is vital for diagnosing and managing cardiovascular diseases. This study evaluates the performance of state-of-the-art object detection models on the ARCADE dataset using the MMDetection framework. The models are assessed using COCO evaluation metrics, including Intersection over Union (IoU), Average Precision (AP), and Average Recall (AR). Results indicate variations in detection accuracy across different models, attributed to differences in algorithmic design, transformer-based vs. convolutional architectures. Additionally, several challenges were encountered during implementation, such as compatibility issues between PyTorch, CUDA, and MMDetection, as well as dataset inconsistencies in ARCADE. The findings provide insights into model selection for stenosis detection and highlight areas for further improvement in deep learning-based coronary artery disease diagnosis.

Title: A Leaf-Level Dataset for Soybean-Cotton Detection and Segmentation

• Authors: Thiago H. Segreto, Juliano Negri, Paulo H. Polegato, João Manoel Herrera Pinheiro, Ricardo Godoy, Marcelo Becker
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01605
• Pdf URL: https://arxiv.org/pdf/2503.01605
• Copy Paste: [[2503.01605]] A Leaf-Level Dataset for Soybean-Cotton Detection and Segmentation(https://arxiv.org/abs/2503.01605)
• Keywords: robust, segmentation
• Abstract: Soybean and cotton are major drivers of many countries' agricultural sectors, offering substantial economic returns but also facing persistent challenges from volunteer plants and weeds that hamper sustainable management. Effectively controlling volunteer plants and weeds demands advanced recognition strategies that can identify these amidst complex crop canopies. While deep learning methods have demonstrated promising results for leaf-level detection and segmentation, existing datasets often fail to capture the complexity of real-world agricultural fields. To address this, we collected 640 high-resolution images from a commercial farm spanning multiple growth stages, weed pressures, and lighting variations. Each image is annotated at the leaf-instance level, with 7,221 soybean and 5,190 cotton leaves labeled via bounding boxes and segmentation masks, capturing overlapping foliage, small leaf size, and morphological similarities. We validate this dataset using YOLOv11, demonstrating state-of-the-art performance in accurately identifying and segmenting overlapping foliage. Our publicly available dataset supports advanced applications such as selective herbicide spraying and pest monitoring and can foster more robust, data-driven strategies for soybean-cotton management.

Title: Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering

• Authors: Zhanghao Hu, Hanqi Yan, Qingling Zhu, Zhenyi Shen, Yulan He, Lin Gui
• Subjects: cs.CL, cs.AI, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01606
• Pdf URL: https://arxiv.org/pdf/2503.01606
• Copy Paste: [[2503.01606]] Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering(https://arxiv.org/abs/2503.01606)
• Keywords: large language model
• Abstract: Large language models have recently pushed open domain question answering (ODQA) to new frontiers. However, prevailing retriever-reader pipelines often depend on multiple rounds of prompt level instructions, leading to high computational overhead, instability, and suboptimal retrieval coverage. In this paper, we propose EmbQA, an embedding-level framework that alleviates these shortcomings by enhancing both the retriever and the reader. Specifically, we refine query representations via lightweight linear layers under an unsupervised contrastive learning objective, thereby reordering retrieved passages to highlight those most likely to contain correct answers. Additionally, we introduce an exploratory embedding that broadens the model's latent semantic space to diversify candidate generation and employs an entropy-based selection mechanism to choose the most confident answer automatically. Extensive experiments across three open-source LLMs, three retrieval methods, and four ODQA benchmarks demonstrate that EmbQA substantially outperforms recent baselines in both accuracy and efficiency.

Title: In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

• Authors: David Ponce, Thierry Etchegoyhen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01611
• Pdf URL: https://arxiv.org/pdf/2503.01611
• Copy Paste: [[2503.01611]] In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models(https://arxiv.org/abs/2503.01611)
• Keywords: large language model
• Abstract: Instruction following is a critical ability for Large Language Models to perform downstream tasks. The standard approach to instruction alignment has relied on a specific phase of model tuning over curated instruction datasets, optionally complemented with an alignment step over human preferences. Recent work has shown the potential of in-context learning (ICL) alternatives to guide base models towards instruction following. This type of approach is particularly relevant to extend instruction following across languages and models of varying sizes adapted to different types of usage. In this work we compare ICL and instruction fine-tuning in English, French and Spanish, on Small Language Models, and provide experimental results on applying Direct Preference Optimisation (DPO) over base models. Our results show that scenarios involving multilingual and smaller models result in downgraded ICL instruction following performance, only partially mitigated by DPO alignment. This study aims to further our understanding of current strengths and limitations of alternative methods for instruction following.

Title: Robust Palm-Vein Recognition Using the MMD Filter: Improving SIFT-Based Feature Matching

• Authors: Kaveen Perera, Fouad Khelifi, Ammar Belatreche
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01612
• Pdf URL: https://arxiv.org/pdf/2503.01612
• Copy Paste: [[2503.01612]] Robust Palm-Vein Recognition Using the MMD Filter: Improving SIFT-Based Feature Matching(https://arxiv.org/abs/2503.01612)
• Keywords: robust
• Abstract: A major challenge with palm vein images is that slight movements of the fingers and thumb, or variations in hand posture, can stretch the skin in different areas and alter the vein patterns. This can result in an infinite number of variations in palm vein images for a given individual. This paper introduces a novel filtering technique for SIFT-based feature matching, known as the Mean and Median Distance (MMD) Filter. This method evaluates the differences in keypoint coordinates and computes the mean and median in each direction to eliminate incorrect matches. Experiments conducted on the 850nm subset of the CASIA dataset indicate that the proposed MMD filter effectively preserves correct points while reducing false positives detected by other filtering methods. A comparison with existing SIFT-based palm vein recognition systems demonstrates that the proposed MMD filter delivers outstanding performance, achieving lower Equal Error Rate (EER) values. This article presents an extended author's version based on our previous work, A Keypoint Filtering Method for SIFT based Palm-Vein Recognition.

Title: Advancing vision-language models in front-end development via data synthesis

• Authors: Tong Ge, Yashu Liu, Jieping Ye, Tianyi Li, Chao Wang
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01619
• Pdf URL: https://arxiv.org/pdf/2503.01619
• Copy Paste: [[2503.01619]] Advancing vision-language models in front-end development via data synthesis(https://arxiv.org/abs/2503.01619)
• Keywords: extraction
• Abstract: Modern front-end (FE) development, especially when leveraging the unique features of frameworks like React and Vue, presents distinctive challenges. These include managing modular architectures, ensuring synchronization between data and visual outputs for declarative rendering, and adapting reusable components to various scenarios. Such complexities make it particularly difficult for state-of-the-art large vision-language models (VLMs) to generate accurate and functional code directly from design images. To address these challenges, we propose a reflective agentic workflow that synthesizes high-quality image-text data to capture the diverse characteristics of FE development. This workflow automates the extraction of self-contained\footnote{A \textbf{self-contained} code snippet is one that encapsulates all necessary logic, styling, and dependencies, ensuring it functions independently without requiring external imports or context.} code snippets from real-world projects, renders the corresponding visual outputs, and generates detailed descriptions that link design elements to functional code. To further expand the scope and utility of the synthesis, we introduce three data synthesis strategies: Evolution-based synthesis, which enables scalable and diverse dataset expansion; Waterfall-Model-based synthesis, which generates logically coherent code derived from system requirements; and Additive Development synthesis, which iteratively increases the complexity of human-authored components. We build a large vision-language model, Flame, trained on the synthesized datasets and demonstrate its effectiveness in generating React code via the $\text{pass}@k$ metric. Our results suggest that a code VLM trained to interpret images before code generation may achieve better performance.

Title: DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation

• Authors: Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01622
• Pdf URL: https://arxiv.org/pdf/2503.01622
• Copy Paste: [[2503.01622]] DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation(https://arxiv.org/abs/2503.01622)
• Keywords: robust
• Abstract: Recent work found that LLMs are sensitive to a wide range of arbitrary prompt dimensions, including the type of delimiters, answer enumerators, instruction wording, and more. This throws into question popular single-prompt evaluation practices. We present DOVE (Dataset Of Variation Evaluation) a large-scale dataset containing prompt perturbations of various evaluation benchmarks. In contrast to previous work, we examine LLM sensitivity from an holistic perspective, and assess the joint effects of perturbations along various dimensions, resulting in thousands of perturbations per instance. We evaluate several model families against DOVE, leading to several findings, including efficient methods for choosing well-performing prompts, observing that few-shot examples reduce sensitivity, and identifying instances which are inherently hard across all perturbations. DOVE consists of more than 250M prompt perturbations and model outputs, which we make publicly available to spur a community-wide effort toward meaningful, robust, and efficient evaluation. Browse the data, contribute, and more: this https URL

Title: Annotating and Inferring Compositional Structures in Numeral Systems Across Languages

• Authors: Arne Rubehn, Christoph Rzymski, Luca Ciucci, Kellen Parker van Dam, Alžběta Kučerová, Katja Bocklage, David Snee, Abishek Stephen, Johann-Mattis List
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01625
• Pdf URL: https://arxiv.org/pdf/2503.01625
• Copy Paste: [[2503.01625]] Annotating and Inferring Compositional Structures in Numeral Systems Across Languages(https://arxiv.org/abs/2503.01625)
• Keywords: segmentation
• Abstract: Numeral systems across the world's languages vary in fascinating ways, both regarding their synchronic structure and the diachronic processes that determined how they evolved in their current shape. For a proper comparison of numeral systems across different languages, however, it is important to code them in a standardized form that allows for the comparison of basic properties. Here, we present a simple but effective coding scheme for numeral annotation, along with a workflow that helps to code numeral systems in a computer-assisted manner, providing sample data for numerals from 1 to 40 in 25 typologically diverse languages. We perform a thorough analysis of the sample, focusing on the systematic comparison between the underlying and the surface morphological structure. We further experiment with automated models for morpheme segmentation, where we find allomorphy as the major reason for segmentation errors. Finally, we show that subword tokenization algorithms are not viable for discovering morphemes in low-resource scenarios.

Title: A General Purpose Spectral Foundational Model for Both Proximal and Remote Sensing Spectral Imaging

• Authors: William Michael Laprade, Jesper Cairo Westergaard, Svend Christensen, Mads Nielsen, Anders Bjorholm Dahl
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01628
• Pdf URL: https://arxiv.org/pdf/2503.01628
• Copy Paste: [[2503.01628]] A General Purpose Spectral Foundational Model for Both Proximal and Remote Sensing Spectral Imaging(https://arxiv.org/abs/2503.01628)
• Keywords: robust
• Abstract: Spectral imaging data acquired via multispectral and hyperspectral cameras can have hundreds of channels, where each channel records the reflectance at a specific wavelength and bandwidth. Time and resource constraints limit our ability to collect large spectral datasets, making it difficult to build and train predictive models from scratch. In the RGB domain, we can often alleviate some of the limitations of smaller datasets by using pretrained foundational models as a starting point. However, most existing foundation models are pretrained on large datasets of 3-channel RGB images, severely limiting their effectiveness when used with spectral imaging data. The few spectral foundation models that do exist usually have one of two limitations: (1) they are built and trained only on remote sensing data limiting their application in proximal spectral imaging, (2) they utilize the more widely available multispectral imaging datasets with less than 15 channels restricting their use with hundred-channel hyperspectral images. To alleviate these issues, we propose a large-scale foundational model and dataset built upon the masked autoencoder architecture that takes advantage of spectral channel encoding, spatial-spectral masking and ImageNet pretraining for an adaptable and robust model for downstream spectral imaging tasks.

Title: Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data

• Authors: Henrik Nolte, Michèle Finck, Kristof Meding
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.01630
• Pdf URL: https://arxiv.org/pdf/2503.01630
• Copy Paste: [[2503.01630]] Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data(https://arxiv.org/abs/2503.01630)
• Keywords: protect, large language model
• Abstract: Does GPT know you? The answer depends on your level of public recognition; however, if your information was available on a website, the answer is probably yes. All Large Language Models (LLMs) memorize training data to some extent. If an LLM training corpus includes personal data, it also memorizes personal data. Developing an LLM typically involves processing personal data, which falls directly within the scope of data protection laws. If a person is identified or identifiable, the implications are far-reaching: the AI system is subject to EU General Data Protection Regulation requirements even after the training phase is concluded. To back our arguments: (1.) We reiterate that LLMs output training data at inference time, be it verbatim or in generalized form. (2.) We show that some LLMs can thus be considered personal data on their own. This triggers a cascade of data protection implications such as data subject rights, including rights to access, rectification, or erasure. These rights extend to the information embedded with-in the AI model. (3.) This paper argues that machine learning researchers must acknowledge the legal implications of LLMs as personal data throughout the full ML development lifecycle, from data collection and curation to model provision on, e.g., GitHub or Hugging Face. (4.) We propose different ways for the ML research community to deal with these legal implications. Our paper serves as a starting point for improving the alignment between data protection law and the technical capabilities of LLMs. Our findings underscore the need for more interaction between the legal domain and the ML community.

Title: SparseMamba-PCL: Scribble-Supervised Medical Image Segmentation via SAM-Guided Progressive Collaborative Learning

• Authors: Luyi Qiu, Tristan Till, Xiaobao Guo, Adams Wai-Kin Kong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01633
• Pdf URL: https://arxiv.org/pdf/2503.01633
• Copy Paste: [[2503.01633]] SparseMamba-PCL: Scribble-Supervised Medical Image Segmentation via SAM-Guided Progressive Collaborative Learning(https://arxiv.org/abs/2503.01633)
• Keywords: segmentation
• Abstract: Scribble annotations significantly reduce the cost and labor required for dense labeling in large medical datasets with complex anatomical structures. However, current scribble-supervised learning methods are limited in their ability to effectively propagate sparse annotation labels to dense segmentation masks and accurately segment object boundaries. To address these issues, we propose a Progressive Collaborative Learning framework that leverages novel algorithms and the Med-SAM foundation model to enhance information quality during training. (1) We enrich ground truth scribble segmentation labels through a new algorithm, propagating scribbles to estimate object boundaries. (2) We enhance feature representation by optimizing Med-SAM-guided training through the fusion of feature embeddings from Med-SAM and our proposed Sparse Mamba network. This enriched representation also facilitates the fine-tuning of the Med-SAM decoder with enriched scribbles. (3) For inference, we introduce a Sparse Mamba network, which is highly capable of capturing local and global dependencies by replacing the traditional sequential patch processing method with a skip-sampling procedure. Experiments on the ACDC, CHAOS, and MSCMRSeg datasets validate the effectiveness of our framework, outperforming nine state-of-the-art methods. Our code is available at \href{this https URL}{this http URL}.

Title: DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models

• Authors: Zhendong Wang, Jianmin Bao, Shuyang Gu, Dong Chen, Wengang Zhou, Houqiang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01645
• Pdf URL: https://arxiv.org/pdf/2503.01645
• Copy Paste: [[2503.01645]] DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models(https://arxiv.org/abs/2503.01645)
• Keywords: diffusion
• Abstract: In this paper, we present DesignDiffusion, a simple yet effective framework for the novel task of synthesizing design images from textual descriptions. A primary challenge lies in generating accurate and style-consistent textual and visual content. Existing works in a related task of visual text generation often focus on generating text within given specific regions, which limits the creativity of generation models, resulting in style or color inconsistencies between textual and visual elements if applied to design image generation. To address this issue, we propose an end-to-end, one-stage diffusion-based framework that avoids intricate components like position and layout modeling. Specifically, the proposed framework directly synthesizes textual and visual design elements from user prompts. It utilizes a distinctive character embedding derived from the visual text to enhance the input prompt, along with a character localization loss for enhanced supervision during text generation. Furthermore, we employ a self-play Direct Preference Optimization fine-tuning strategy to improve the quality and accuracy of the synthesized visual text. Extensive experiments demonstrate that DesignDiffusion achieves state-of-the-art performance in design image generation.

Title: OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understanding

• Authors: Dianyi Yang, Yu Gao, Xihan Wang, Yufeng Yue, Yi Yang, Mengyin Fu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01646
• Pdf URL: https://arxiv.org/pdf/2503.01646
• Copy Paste: [[2503.01646]] OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understanding(https://arxiv.org/abs/2503.01646)
• Keywords: robust, segmentation
• Abstract: Recent advancements in 3D Gaussian Splatting have significantly improved the efficiency and quality of dense semantic SLAM. However, previous methods are generally constrained by limited-category pre-trained classifiers and implicit semantic representation, which hinder their performance in open-set scenarios and restrict 3D object-level scene understanding. To address these issues, we propose OpenGS-SLAM, an innovative framework that utilizes 3D Gaussian representation to perform dense semantic SLAM in open-set environments. Our system integrates explicit semantic labels derived from 2D foundational models into the 3D Gaussian framework, facilitating robust 3D object-level scene understanding. We introduce Gaussian Voting Splatting to enable fast 2D label map rendering and scene updating. Additionally, we propose a Confidence-based 2D Label Consensus method to ensure consistent labeling across multiple views. Furthermore, we employ a Segmentation Counter Pruning strategy to improve the accuracy of semantic scene representation. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our method in scene understanding, tracking, and mapping, achieving 10 times faster semantic rendering and 2 times lower storage costs compared to existing methods. Project page: this https URL.

Title: Distilled Prompt Learning for Incomplete Multimodal Survival Prediction

• Authors: Yingxue Xu, Fengtao Zhou, Chenyu Zhao, Yihui Wang, Can Yang, Hao Chen
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01653
• Pdf URL: https://arxiv.org/pdf/2503.01653
• Copy Paste: [[2503.01653]] Distilled Prompt Learning for Incomplete Multimodal Survival Prediction(https://arxiv.org/abs/2503.01653)
• Keywords: robust, large language model
• Abstract: The integration of multimodal data including pathology images and gene profiles is widely applied in precise survival prediction. Despite recent advances in multimodal survival models, collecting complete modalities for multimodal fusion still poses a significant challenge, hindering their application in clinical settings. Current approaches tackling incomplete modalities often fall short, as they typically compensate for only a limited part of the knowledge of missing modalities. To address this issue, we propose a Distilled Prompt Learning framework (DisPro) to utilize the strong robustness of Large Language Models (LLMs) to missing modalities, which employs two-stage prompting for compensation of comprehensive information for missing modalities. In the first stage, Unimodal Prompting (UniPro) distills the knowledge distribution of each modality, preparing for supplementing modality-specific knowledge of the missing modality in the subsequent stage. In the second stage, Multimodal Prompting (MultiPro) leverages available modalities as prompts for LLMs to infer the missing modality, which provides modality-common information. Simultaneously, the unimodal knowledge acquired in the first stage is injected into multimodal inference to compensate for the modality-specific knowledge of the missing modality. Extensive experiments covering various missing scenarios demonstrated the superiority of the proposed method. The code is available at this https URL.

Title: CoPL: Collaborative Preference Learning for Personalizing LLMs

• Authors: Youngbin Choi, Seunghyuk Cho, Minjong Lee, MoonJeong Park, Yesong Ko, Jungseul Ok, Dongwoo Kim
• Subjects: cs.LG, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01658
• Pdf URL: https://arxiv.org/pdf/2503.01658
• Copy Paste: [[2503.01658]] CoPL: Collaborative Preference Learning for Personalizing LLMs(https://arxiv.org/abs/2503.01658)
• Keywords: large language model
• Abstract: Personalizing large language models (LLMs) is important for aligning outputs with diverse user preferences, yet existing methods struggle with flexibility and generalization. We propose CoPL (Collaborative Preference Learning), a graph-based collaborative filtering framework that models user-response relationships to enhance preference estimation, particularly in sparse annotation settings. By integrating a mixture of LoRA experts, CoPL efficiently fine-tunes LLMs while dynamically balancing shared and user-specific preferences. Additionally, an optimization-free adaptation strategy enables generalization to unseen users without fine-tuning. Experiments on UltraFeedback-P demonstrate that CoPL outperforms existing personalized reward models, effectively capturing both common and controversial preferences, making it a scalable solution for personalized LLM alignment.

Title: Detecting Stylistic Fingerprints of Large Language Models

• Authors: Yehonatan Bitton, Elad Bitton, Shai Nisan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01659
• Pdf URL: https://arxiv.org/pdf/2503.01659
• Copy Paste: [[2503.01659]] Detecting Stylistic Fingerprints of Large Language Models(https://arxiv.org/abs/2503.01659)
• Keywords: protect, large language model
• Abstract: Large language models (LLMs) have distinct and consistent stylistic fingerprints, even when prompted to write in different writing styles. Detecting these fingerprints is important for many reasons, among them protecting intellectual property, ensuring transparency regarding AI-generated content, and preventing the misuse of AI technologies. In this paper, we present a novel method to classify texts based on the stylistic fingerprints of the models that generated them. We introduce an LLM-detection ensemble that is composed of three classifiers with varied architectures and training data. This ensemble is trained to classify texts generated by four well-known LLM families: Claude, Gemini, Llama, and OpenAI. As this task is highly cost-sensitive and might have severe implications, we want to minimize false-positives and increase confidence. We consider a prediction as valid when all three classifiers in the ensemble unanimously agree on the output classification. Our ensemble is validated on a test set of texts generated by Claude, Gemini, Llama, and OpenAI models, and achieves extremely high precision (0.9988) and a very low false-positive rate (0.0004). Furthermore, we demonstrate the ensemble's ability to distinguish between texts generated by seen and unseen models. This reveals interesting stylistic relationships between models. This approach to stylistic analysis has implications for verifying the originality of AI-generated texts and tracking the origins of model training techniques.

Title: MUSt3R: Multi-view Network for Stereo 3D Reconstruction

• Authors: Yohann Cabon, Lucas Stoffl, Leonid Antsfeld, Gabriela Csurka, Boris Chidlovskii, Jerome Revaud, Vincent Leroy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01661
• Pdf URL: https://arxiv.org/pdf/2503.01661
• Copy Paste: [[2503.01661]] MUSt3R: Multi-view Network for Stereo 3D Reconstruction(https://arxiv.org/abs/2503.01661)
• Keywords: robust
• Abstract: DUSt3R introduced a novel paradigm in geometric computer vision by proposing a model that can provide dense and unconstrained Stereo 3D Reconstruction of arbitrary image collections with no prior information about camera calibration nor viewpoint poses. Under the hood, however, DUSt3R processes image pairs, regressing local 3D reconstructions that need to be aligned in a global coordinate system. The number of pairs, growing quadratically, is an inherent limitation that becomes especially concerning for robust and fast optimization in the case of large image collections. In this paper, we propose an extension of DUSt3R from pairs to multiple views, that addresses all aforementioned concerns. Indeed, we propose a Multi-view Network for Stereo 3D Reconstruction, or MUSt3R, that modifies the DUSt3R architecture by making it symmetric and extending it to directly predict 3D structure for all views in a common coordinate frame. Second, we entail the model with a multi-layer memory mechanism which allows to reduce the computational complexity and to scale the reconstruction to large collections, inferring thousands of 3D pointmaps at high frame-rates with limited added complexity. The framework is designed to perform 3D reconstruction both offline and online, and hence can be seamlessly applied to SfM and visual SLAM scenarios showing state-of-the-art performance on various 3D downstream tasks, including uncalibrated Visual Odometry, relative camera pose, scale and focal estimation, 3D reconstruction and multi-view depth estimation.

Title: ToLo: A Two-Stage, Training-Free Layout-To-Image Generation Framework For High-Overlap Layouts

• Authors: Linhao Huang, Jing Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01667
• Pdf URL: https://arxiv.org/pdf/2503.01667
• Copy Paste: [[2503.01667]] ToLo: A Two-Stage, Training-Free Layout-To-Image Generation Framework For High-Overlap Layouts(https://arxiv.org/abs/2503.01667)
• Keywords: diffusion
• Abstract: Recent training-free layout-to-image diffusion models have demonstrated remarkable performance in generating high-quality images with controllable layouts. These models follow a one-stage framework: Encouraging the model to focus the attention map of each concept on its corresponding region by defining attention map-based losses. However, these models still struggle to accurately follow layouts with significant overlap, often leading to issues like attribute leakage and missing entities. In this paper, we propose ToLo, a two-stage, training-free layout-to-image generation framework for high-overlap layouts. Our framework consists of two stages: the aggregation stage and the separation stage, each with its own loss function based on the attention map. To provide a more effective evaluation, we partition the HRS dataset based on the Intersection over Union (IoU) of the input layouts, creating a new dataset for layout-to-image generation with varying levels of overlap. Through extensive experiments on this dataset, we demonstrate that ToLo significantly enhances the performance of existing methods when dealing with high-overlap layouts. Our code and dataset are available here: this https URL.

Title: An Efficient Continual Learning Framework for Multivariate Time Series Prediction Tasks with Application to Vehicle State Estimation

• Authors: Arvin Hosseinzadeh, Ladan Khoshnevisan, Mohammad Pirani, Shojaeddin Chenouri, Amir Khajepour
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01669
• Pdf URL: https://arxiv.org/pdf/2503.01669
• Copy Paste: [[2503.01669]] An Efficient Continual Learning Framework for Multivariate Time Series Prediction Tasks with Application to Vehicle State Estimation(https://arxiv.org/abs/2503.01669)
• Keywords: robust, transformer
• Abstract: In continual time series analysis using neural networks, catastrophic forgetting (CF) of previously learned models when training on new data domains has always been a significant challenge. This problem is especially challenging in vehicle estimation and control, where new information is sequentially introduced to the model. Unfortunately, existing work on continual learning has not sufficiently addressed the adverse effects of catastrophic forgetting in time series analysis, particularly in multivariate output environments. In this paper, we present EM-ReSeleCT (Efficient Multivariate Representative Selection for Continual Learning in Time Series Tasks), an enhanced approach designed to handle continual learning in multivariate environments. Our approach strategically selects representative subsets from old and historical data and incorporates memory-based continual learning techniques with an improved optimization algorithm to adapt the pre-trained model on new information while preserving previously acquired information. Additionally, we develop a sequence-to-sequence transformer model (autoregressive model) specifically designed for vehicle state estimation. Moreover, we propose an uncertainty quantification framework using conformal prediction to assess the sensitivity of the memory size and to showcase the robustness of the proposed method. Experimental results from tests on an electric Equinox vehicle highlight the superiority of our method in continually learning new information while retaining prior knowledge, outperforming state-of-the-art continual learning methods. Furthermore, EM-ReSeleCT significantly reduces training time, a critical advantage in continual learning applications.

Title: Evaluating LLMs' Assessment of Mixed-Context Hallucination Through the Lens of Summarization

• Authors: Siya Qi, Rui Cao, Yulan He, Zheng Yuan
• Subjects: cs.CL, cs.AI, cs.CY, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01670
• Pdf URL: https://arxiv.org/pdf/2503.01670
• Copy Paste: [[2503.01670]] Evaluating LLMs' Assessment of Mixed-Context Hallucination Through the Lens of Summarization(https://arxiv.org/abs/2503.01670)
• Keywords: large language model
• Abstract: With the rapid development of large language models (LLMs), LLM-as-a-judge has emerged as a widely adopted approach for text quality evaluation, including hallucination evaluation. While previous studies have focused exclusively on single-context evaluation (e.g., discourse faithfulness or world factuality), real-world hallucinations typically involve mixed contexts, which remains inadequately evaluated. In this study, we use summarization as a representative task to comprehensively evaluate LLMs' capability in detecting mixed-context hallucinations, specifically distinguishing between factual and non-factual hallucinations. Through extensive experiments across direct generation and retrieval-based models of varying scales, our main observations are: (1) LLMs' intrinsic knowledge introduces inherent biases in hallucination evaluation; (2) These biases particularly impact the detection of factual hallucinations, yielding a significant performance bottleneck; (3) The fundamental challenge lies in effective knowledge utilization, balancing between LLMs' intrinsic knowledge and external context for accurate mixed-context hallucination evaluation.

Title: Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledge

• Authors: Xiao Liu, Zirui Wu, Jiayi Li, Zhicheng Shao, Xun Pang, Yansong Feng
• Subjects: cs.CL, cs.SI
• Abstract URL: https://arxiv.org/abs/2503.01672
• Pdf URL: https://arxiv.org/pdf/2503.01672
• Copy Paste: [[2503.01672]] Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledge(https://arxiv.org/abs/2503.01672)
• Keywords: large language model
• Abstract: Longitudinal network data are essential for analyzing political, economic, and social systems and processes. In political science, these datasets are often generated through human annotation or supervised machine learning applied to evolving corpora. However, as semantic contexts shift over time, inferring dynamic interaction types on emerging issues among a diverse set of entities poses significant challenges, particularly in maintaining timely and consistent annotations. This paper presents the Expert-Augmented LLM Annotation (EALA) approach, which leverages Large Language Models (LLMs) in combination with historically annotated data and expert-constructed codebooks to extrapolate and extend datasets into future periods. We evaluate the performance and reliability of EALA using a dataset of climate negotiations. Our findings demonstrate that EALA effectively predicts nuanced interactions between negotiation parties and captures the evolution of topics over time. At the same time, we identify several limitations inherent to LLM-based annotation, highlighting areas for further improvement. Given the wide availability of codebooks and annotated datasets, EALA holds substantial promise for advancing research in political science and beyond.

Title: Using (Not so) Large Language Models for Generating Simulation Models in a Formal DSL -- A Study on Reaction Networks

• Authors: Justin N. Kreikemeyer, Miłosz Jankowski, Pia Wilsdorf, Adelinde M. Uhrmacher
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01675
• Pdf URL: https://arxiv.org/pdf/2503.01675
• Copy Paste: [[2503.01675]] Using (Not so) Large Language Models for Generating Simulation Models in a Formal DSL -- A Study on Reaction Networks(https://arxiv.org/abs/2503.01675)
• Keywords: large language model
• Abstract: Formal languages are an integral part of modeling and simulation. They allow the distillation of knowledge into concise simulation models amenable to automatic execution, interpretation, and analysis. However, the arguably most humanly accessible means of expressing models is through natural language, which is not easily interpretable by computers. Here, we evaluate how a Large Language Model (LLM) might be used for formalizing natural language into simulation models. Existing studies only explored using very large LLMs, like the commercial GPT models, without fine-tuning model weights. To close this gap, we show how an open-weights, 7B-parameter Mistral model can be fine-tuned to translate natural language descriptions to reaction network models in a domain-specific language, offering a self-hostable, compute-, and memory efficient alternative. To this end, we develop a synthetic data generator to serve as the basis for fine-tuning and evaluation. Our quantitative evaluation shows that our fine-tuned Mistral model can recover the ground truth simulation model in up to 84.5% of cases. In addition, our small-scale user study demonstrates the model's practical potential for one-time generation as well as interactive modeling in various domains. While promising, in its current form, the fine-tuned small LLM cannot catch up with large LLMs. We conclude that higher-quality training data are required, and expect future small and open-source LLMs to offer new opportunities.

Title: When an LLM is apprehensive about its answers -- and when its uncertainty is justified

• Authors: Petr Sychev, Andrey Goncharov, Daniil Vyazhev, Edvard Khalafyan, Alexey Zaytsev
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01688
• Pdf URL: https://arxiv.org/pdf/2503.01688
• Copy Paste: [[2503.01688]] When an LLM is apprehensive about its answers -- and when its uncertainty is justified(https://arxiv.org/abs/2503.01688)
• Keywords: fair, large language model
• Abstract: Uncertainty estimation is crucial for evaluating Large Language Models (LLMs), particularly in high-stakes domains where incorrect answers result in significant consequences. Numerous approaches consider this problem, while focusing on a specific type of uncertainty, ignoring others. We investigate what estimates, specifically token-wise entropy and model-as-judge (MASJ), would work for multiple-choice question-answering tasks for different question topics. Our experiments consider three LLMs: Phi-4, Mistral, and Qwen of different sizes from 1.5B to 72B and $14$ topics. While MASJ performs similarly to a random error predictor, the response entropy predicts model error in knowledge-dependent domains and serves as an effective indicator of question difficulty: for biology ROC AUC is $0.73$. This correlation vanishes for the reasoning-dependent domain: for math questions ROC-AUC is $0.55$. More principally, we found out that the entropy measure required a reasoning amount. Thus, data-uncertainty related entropy should be integrated within uncertainty estimates frameworks, while MASJ requires refinement. Moreover, existing MMLU-Pro samples are biased, and should balance required amount of reasoning for different subdomains to provide a more fair assessment of LLMs performance.

Title: Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution

• Authors: Kun Li, Tianhua Zhang, Yunxiang Li, Hongyin Luo, Abdalla Moustafa, Xixin Wu, James Glass, Helen Meng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01695
• Pdf URL: https://arxiv.org/pdf/2503.01695
• Copy Paste: [[2503.01695]] Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution(https://arxiv.org/abs/2503.01695)
• Keywords: robust, generative, large language model
• Abstract: Improving context faithfulness in large language models is essential for developing trustworthy retrieval augmented generation systems and mitigating hallucinations, especially in long-form question answering (LFQA) tasks or scenarios involving knowledge conflicts. Existing methods either intervene LLMs only at inference without addressing their inherent limitations or overlook the potential for self-improvement. In this paper, we introduce GenDiE (Generate, Discriminate, Evolve), a novel self-evolving framework that enhances context faithfulness through fine-grained sentence-level optimization. GenDiE combines both generative and discriminative training, equipping LLMs with self-generation and self-scoring capabilities to facilitate iterative self-evolution. This supports both data construction for model alignment and score-guided search during inference. Furthermore, by treating each sentence in a response as an independent optimization unit, GenDiE effectively addresses the limitations of previous approaches that optimize at the holistic answer level, which may miss unfaithful details. Experiments on ASQA (in-domain LFQA) and ConFiQA (out-of-domain counterfactual QA) datasets demonstrate that GenDiE surpasses various baselines in both faithfulness and correctness, and exhibits robust performance for domain adaptation.

Title: Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks

• Authors: Nandi Schoots, Mattia Jacopo Villani, Niels uit de Bos
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01702
• Pdf URL: https://arxiv.org/pdf/2503.01702
• Copy Paste: [[2503.01702]] Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks(https://arxiv.org/abs/2503.01702)
• Keywords: interpretability
• Abstract: Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (arXiv:2404.19756). In this paper, we explore the connection between Kolmogorov Arnold Networks (KANs) with piecewise linear (univariate real) functions and ReLU networks. We provide completely explicit constructions to convert a piecewise linear KAN into a ReLU network and vice versa.

Title: DILEMMA: Joint LLM Quantization and Distributed LLM Inference Over Edge Computing Systems

• Authors: Minoo Hosseinzadeh, Hana Khamfroush
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01704
• Pdf URL: https://arxiv.org/pdf/2503.01704
• Copy Paste: [[2503.01704]] DILEMMA: Joint LLM Quantization and Distributed LLM Inference Over Edge Computing Systems(https://arxiv.org/abs/2503.01704)
• Keywords: large language model
• Abstract: With a recent trend of using Large Language Models (LLMs) for different applications within smart cities, there is a need for pushing these models toward the edge of network while still preserving their performance. Edge Computing (EC) as a physically closer computing resource to the end users can help to reduce the communication delay for serving end users' tasks for LLM-dependent services. However, EC servers have limited capacity in terms of communication, computation, and storage capacity. This paper introduces DILEMMA, a novel framework addressing the challenges of deploying LLMs in EC systems by jointly optimizing layer placement and layer quantization in EC systems. DILEMMA formulates an Integer Linear Programming problem to minimize total inference delay while ensuring acceptable LLM performance levels, leveraging layer-wise quantization and knowledge distillation for LLM performance control. Experimental evaluations on OPT-350 model using the SQuAD dataset demonstrate that DILEMMA achieves a quantization ratio of up to 12.75% while preserving model loss, highlighting its effectiveness in resource-constrained environments.

Title: SAGE: A Framework of Precise Retrieval for RAG

• Authors: Jintao Zhang, Guoliang Li, Jinyang Su
• Subjects: cs.LG, cs.AI, cs.DB, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01713
• Pdf URL: https://arxiv.org/pdf/2503.01713
• Copy Paste: [[2503.01713]] SAGE: A Framework of Precise Retrieval for RAG(https://arxiv.org/abs/2503.01713)
• Keywords: large language model, segmentation
• Abstract: Retrieval-augmented generation (RAG) has demonstrated significant proficiency in conducting question-answering (QA) tasks within a specified corpus. Nonetheless, numerous failure instances of RAG in QA still exist. These failures are not solely attributable to the limitations of Large Language Models (LLMs); instead, they predominantly arise from the retrieval of inaccurate information for LLMs due to two limitations: (1) Current RAG methods segment the corpus without considering semantics, making it difficult to find relevant context due to impaired correlation between questions and the segments. (2) There is a trade-off between missing essential context with fewer context retrieved and getting irrelevant context with more context retrieved. In this paper, we introduce a RAG framework (SAGE), to overcome these limitations. First, to address the segmentation issue without considering semantics, we propose to train a semantic segmentation model. This model is trained to segment the corpus into semantically complete chunks. Second, to ensure that only the most relevant chunks are retrieved while the irrelevant ones are ignored, we design a chunk selection algorithm to dynamically select chunks based on the decreasing speed of the relevance score, leading to a more relevant selection. Third, to further ensure the precision of the retrieved chunks, we propose letting LLMs assess whether retrieved chunks are excessive or lacking and then adjust the amount of context accordingly. Experiments show that SAGE outperforms baselines by 61.25% in the quality of QA on average. Moreover, by avoiding retrieving noisy context, SAGE lowers the cost of the tokens consumed in LLM inference and achieves a 49.41% enhancement in cost efficiency on average. Additionally, our work offers valuable insights for boosting RAG.

Title: Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia

• Authors: Chenxi Wang, Tianle Gu, Zhongyu Wei, Lang Gao, Zirui Song, Xiuying Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01714
• Pdf URL: https://arxiv.org/pdf/2503.01714
• Copy Paste: [[2503.01714]] Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia(https://arxiv.org/abs/2503.01714)
• Keywords: large language model
• Abstract: Human readers can efficiently comprehend scrambled words, a phenomenon known as Typoglycemia, primarily by relying on word form; if word form alone is insufficient, they further utilize contextual cues for interpretation. While advanced large language models (LLMs) exhibit similar abilities, the underlying mechanisms remain unclear. To investigate this, we conduct controlled experiments to analyze the roles of word form and contextual information in semantic reconstruction and examine LLM attention patterns. Specifically, we first propose SemRecScore, a reliable metric to quantify the degree of semantic reconstruction, and validate its effectiveness. Using this metric, we study how word form and contextual information influence LLMs' semantic reconstruction ability, identifying word form as the core factor in this process. Furthermore, we analyze how LLMs utilize word form and find that they rely on specialized attention heads to extract and process word form information, with this mechanism remaining stable across varying levels of word scrambling. This distinction between LLMs' fixed attention patterns primarily focused on word form and human readers' adaptive strategy in balancing word form and contextual information provides insights into enhancing LLM performance by incorporating human-like, context-aware mechanisms.

Title: KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation

• Authors: Antoni Bigata, Michał Stypułkowski, Rodrigo Mira, Stella Bounareli, Konstantinos Vougioukas, Zoe Landgraf, Nikita Drobyshev, Maciej Zieba, Stavros Petridis, Maja Pantic
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01715
• Pdf URL: https://arxiv.org/pdf/2503.01715
• Copy Paste: [[2503.01715]] KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation(https://arxiv.org/abs/2503.01715)
• Keywords: diffusion
• Abstract: Current audio-driven facial animation methods achieve impressive results for short videos but suffer from error accumulation and identity drift when extended to longer durations. Existing methods attempt to mitigate this through external spatial control, increasing long-term consistency but compromising the naturalness of motion. We propose KeyFace, a novel two-stage diffusion-based framework, to address these issues. In the first stage, keyframes are generated at a low frame rate, conditioned on audio input and an identity frame, to capture essential facial expressions and movements over extended periods of time. In the second stage, an interpolation model fills in the gaps between keyframes, ensuring smooth transitions and temporal coherence. To further enhance realism, we incorporate continuous emotion representations and handle a wide range of non-speech vocalizations (NSVs), such as laughter and sighs. We also introduce two new evaluation metrics for assessing lip synchronization and NSV generation. Experimental results show that KeyFace outperforms state-of-the-art methods in generating natural, coherent facial animations over extended durations, successfully encompassing NSVs and continuous emotions.

Title: Quality Measures for Dynamic Graph Generative Models

• Authors: Ryien Hosseini, Filippo Simini, Venkatram Vishwanath, Rebecca Willett, Henry Hoffmann
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01720
• Pdf URL: https://arxiv.org/pdf/2503.01720
• Copy Paste: [[2503.01720]] Quality Measures for Dynamic Graph Generative Models(https://arxiv.org/abs/2503.01720)
• Keywords: generative
• Abstract: Deep generative models have recently achieved significant success in modeling graph data, including dynamic graphs, where topology and features evolve over time. However, unlike in vision and natural language domains, evaluating generative models for dynamic graphs is challenging due to the difficulty of visualizing their output, making quantitative metrics essential. In this work, we develop a new quality metric for evaluating generative models of dynamic graphs. Current metrics for dynamic graphs typically involve discretizing the continuous-evolution of graphs into static snapshots and then applying conventional graph similarity measures. This approach has several limitations: (a) it models temporally related events as i.i.d. samples, failing to capture the non-uniform evolution of dynamic graphs; (b) it lacks a unified measure that is sensitive to both features and topology; (c) it fails to provide a scalar metric, requiring multiple metrics without clear superiority; and (d) it requires explicitly instantiating each static snapshot, leading to impractical runtime demands that hinder evaluation at scale. We propose a novel metric based on the \textit{Johnson-Lindenstrauss} lemma, applying random projections directly to dynamic graph data. This results in an expressive, scalar, and application-agnostic measure of dynamic graph similarity that overcomes the limitations of traditional methods. We also provide a comprehensive empirical evaluation of metrics for continuous-time dynamic graphs, demonstrating the effectiveness of our approach compared to existing methods. Our implementation is available at this https URL.

Title: Syntactic Learnability of Echo State Neural Language Models at Scale

• Authors: Ryo Ueda, Tatsuki Kuribayashi, Shunsuke Kando, Kentaro Inui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01724
• Pdf URL: https://arxiv.org/pdf/2503.01724
• Copy Paste: [[2503.01724]] Syntactic Learnability of Echo State Neural Language Models at Scale(https://arxiv.org/abs/2503.01724)
• Keywords: transformer
• Abstract: What is a neural model with minimum architectural complexity that exhibits reasonable language learning capability? To explore such a simple but sufficient neural language model, we revisit a basic reservoir computing (RC) model, Echo State Network (ESN), a restricted class of simple Recurrent Neural Networks. Our experiments showed that ESN with a large hidden state is comparable or superior to Transformer in grammaticality judgment tasks when trained with about 100M words, suggesting that architectures as complex as that of Transformer may not always be necessary for syntactic learning.

Title: DeepSuM: Deep Sufficient Modality Learning Framework

• Authors: Zhe Gao, Jian Huang, Ting Li, Xueqin Wang
• Subjects: cs.LG, stat.ME
• Abstract URL: https://arxiv.org/abs/2503.01728
• Pdf URL: https://arxiv.org/pdf/2503.01728
• Copy Paste: [[2503.01728]] DeepSuM: Deep Sufficient Modality Learning Framework(https://arxiv.org/abs/2503.01728)
• Keywords: robust, large language model
• Abstract: Multimodal learning has become a pivotal approach in developing robust learning models with applications spanning multimedia, robotics, large language models, and healthcare. The efficiency of multimodal systems is a critical concern, given the varying costs and resource demands of different modalities. This underscores the necessity for effective modality selection to balance performance gains against resource expenditures. In this study, we propose a novel framework for modality selection that independently learns the representation of each modality. This approach allows for the assessment of each modality's significance within its unique representation space, enabling the development of tailored encoders and facilitating the joint analysis of modalities with distinct characteristics. Our framework aims to enhance the efficiency and effectiveness of multimodal learning by optimizing modality integration and selection.

Title: Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning

• Authors: Kyle Domico, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Eric Pauley, Josiah Hanna, Patrick McDaniel
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01734
• Pdf URL: https://arxiv.org/pdf/2503.01734
• Copy Paste: [[2503.01734]] Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning(https://arxiv.org/abs/2503.01734)
• Keywords: security, attack
• Abstract: Reinforcement learning (RL) offers powerful techniques for solving complex sequential decision-making tasks from experience. In this paper, we demonstrate how RL can be applied to adversarial machine learning (AML) to develop a new class of attacks that learn to generate adversarial examples: inputs designed to fool machine learning models. Unlike traditional AML methods that craft adversarial examples independently, our RL-based approach retains and exploits past attack experience to improve future attacks. We formulate adversarial example generation as a Markov Decision Process and evaluate RL's ability to (a) learn effective and efficient attack strategies and (b) compete with state-of-the-art AML. On CIFAR-10, our agent increases the success rate of adversarial examples by 19.4% and decreases the median number of victim model queries per adversarial example by 53.2% from the start to the end of training. In a head-to-head comparison with a state-of-the-art image attack, SquareAttack, our approach enables an adversary to generate adversarial examples with 13.1% more success after 5000 episodes of training. From a security perspective, this work demonstrates a powerful new attack vector that uses RL to attack ML models efficiently and at scale.

Title: Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarios

• Authors: Mohammad Rafid Ul Islam, Prasad Tadepalli, Alan Fern
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.01737
• Pdf URL: https://arxiv.org/pdf/2503.01737
• Copy Paste: [[2503.01737]] Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarios(https://arxiv.org/abs/2503.01737)
• Keywords: diffusion
• Abstract: Missing values in multivariate time series data can harm machine learning performance and introduce bias. These gaps arise from sensor malfunctions, blackouts, and human error and are typically addressed by data imputation. Previous work has tackled the imputation of missing data in random, complete blackouts and forecasting scenarios. The current paper addresses a more general missing pattern, which we call "partial blackout," where a subset of features is missing for consecutive time steps. We introduce a two-stage imputation process using self-attention and diffusion processes to model feature and temporal correlations. Notably, our model effectively handles missing data during training, enhancing adaptability and ensuring reliable imputation and performance, even with incomplete datasets. Our experiments on benchmark and two real-world time series datasets demonstrate that our model outperforms the state-of-the-art in partial blackout scenarios and shows better scalability.

Title: VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation

• Authors: Wenhao Wang, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01739
• Pdf URL: https://arxiv.org/pdf/2503.01739
• Copy Paste: [[2503.01739]] VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation(https://arxiv.org/abs/2503.01739)
• Keywords: generative
• Abstract: Text-to-video generative models convert textual prompts into dynamic visual content, offering wide-ranging applications in film production, gaming, and education. However, their real-world performance often falls short of user expectations. One key reason is that these models have not been trained on videos related to some topics users want to create. In this paper, we propose VideoUFO, the first Video dataset specifically curated to align with Users' FOcus in real-world scenarios. Beyond this, our VideoUFO also features: (1) minimal ($0.29\%$) overlap with existing video datasets, and (2) videos searched exclusively via YouTube's official API under the Creative Commons license. These two attributes provide future researchers with greater freedom to broaden their training sources. The VideoUFO comprises over $1.09$ million video clips, each paired with both a brief and a detailed caption (description). Specifically, through clustering, we first identify $1,291$ user-focused topics from the million-scale real text-to-video prompt dataset, VidProM. Then, we use these topics to retrieve videos from YouTube, split the retrieved videos into clips, and generate both brief and detailed captions for each clip. After verifying the clips with specified topics, we are left with about $1.09$ million video clips. Our experiments reveal that (1) current $16$ text-to-video models do not achieve consistent performance across all user-focused topics; and (2) a simple model trained on VideoUFO outperforms others on worst-performing topics. The dataset is publicly available at this https URL under the CC BY 4.0 License.

Title: Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models

• Authors: Alberto Purpura, Sahil Wadhwa, Jesse Zymet, Akshay Gupta, Andy Luo, Melissa Kazemi Rad, Swapnil Shinde, Mohammad Shahed Sorower
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01742
• Pdf URL: https://arxiv.org/pdf/2503.01742
• Copy Paste: [[2503.01742]] Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models(https://arxiv.org/abs/2503.01742)
• Keywords: security, privacy, attack, large language model
• Abstract: The rapid growth of Large Language Models (LLMs) presents significant privacy, security, and ethical concerns. While much research has proposed methods for defending LLM systems against misuse by malicious actors, researchers have recently complemented these efforts with an offensive approach that involves red teaming, i.e., proactively attacking LLMs with the purpose of identifying their vulnerabilities. This paper provides a concise and practical overview of the LLM red teaming literature, structured so as to describe a multi-component system end-to-end. To motivate red teaming we survey the initial safety needs of some high-profile LLMs, and then dive into the different components of a red teaming system as well as software packages for implementing them. We cover various attack methods, strategies for attack-success evaluation, metrics for assessing experiment outcomes, as well as a host of other considerations. Our survey will be useful for any reader who wants to rapidly obtain a grasp of the major red teaming concepts for their own use in practical applications.

Title: ECG-EmotionNet: Nested Mixture of Expert (NMoE) Adaptation of ECG-Foundation Model for Driver Emotion Recognition

• Authors: Nastaran Mansourian, Arash Mohammadi, M. Omair Ahmad, M.N.S. Swamy
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2503.01750
• Pdf URL: https://arxiv.org/pdf/2503.01750
• Copy Paste: [[2503.01750]] ECG-EmotionNet: Nested Mixture of Expert (NMoE) Adaptation of ECG-Foundation Model for Driver Emotion Recognition(https://arxiv.org/abs/2503.01750)
• Keywords: robust, transformer
• Abstract: Driver emotion recognition plays a crucial role in driver monitoring systems, enhancing human-autonomy interactions and the trustworthiness of Autonomous Driving (AD). Various physiological and behavioural modalities have been explored for this purpose, with Electrocardiogram (ECG) emerging as a standout choice for real-time emotion monitoring, particularly in dynamic and unpredictable driving conditions. Existing methods, however, often rely on multi-channel ECG signals recorded under static conditions, limiting their applicability in real-world dynamic driving scenarios. To address this limitation, the paper introduces ECG-EmotionNet, a novel architecture designed specifically for emotion recognition in dynamic driving environments. ECG-EmotionNet is constructed by adapting a recently introduced ECG Foundation Model (FM) and uniquely employs single-channel ECG signals, ensuring both robust generalizability and computational efficiency. Unlike conventional adaptation methods such as full fine-tuning, linear probing, or low-rank adaptation, we propose an intuitively pleasing alternative, referred to as the nested Mixture of Experts (MoE) adaptation. More precisely, each transformer layer of the underlying FM is treated as a separate expert, with embeddings extracted from these experts fused using trainable weights within a gating mechanism. This approach enhances the representation of both global and local ECG features, leading to a 6% improvement in accuracy and a 7% increase in the F1 score, all while maintaining computational efficiency. The effectiveness of the proposed ECG-EmotionNet architecture is evaluated using a recently introduced and challenging driver emotion monitoring dataset.

Title: Enhancing Multi-hop Reasoning in Vision-Language Models via Self-Distillation with Multi-Prompt Ensembling

• Authors: Guande Wu, Huan Song, Yawei Wang, Qiaojing Yan, Yijun Tian, Lin Lee Cheong, Panpan Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01754
• Pdf URL: https://arxiv.org/pdf/2503.01754
• Copy Paste: [[2503.01754]] Enhancing Multi-hop Reasoning in Vision-Language Models via Self-Distillation with Multi-Prompt Ensembling(https://arxiv.org/abs/2503.01754)
• Keywords: large language model
• Abstract: Multi-modal large language models have seen rapid advancement alongside large language models. However, while language models can effectively leverage chain-of-thought prompting for zero or few-shot learning, similar prompting strategies are less effective for multi-modal LLMs due to modality gaps and task complexity. To address this challenge, we explore two prompting approaches: a dual-query method that separates multi-modal input analysis and answer generation into two prompting steps, and an ensemble prompting method that combines multiple prompt variations to arrive at the final answer. Although these approaches enhance the model's reasoning capabilities without fine-tuning, they introduce significant inference overhead. Therefore, building on top of these two prompting techniques, we propose a self-distillation framework such that the model can improve itself without any annotated data. Our self-distillation framework learns representation intervention modules from the reasoning traces collected from ensembled dual-query prompts, in the form of hidden representations. The lightweight intervention modules operate in parallel with the frozen original model, which makes it possible to maintain computational efficiency while significantly improving model capability. We evaluate our method on five widely-used VQA benchmarks, demonstrating its effectiveness in performing multi-hop reasoning for complex tasks.

Title: Zero-Trust Artificial Intelligence Model Security Based on Moving Target Defense and Content Disarm and Reconstruction

• Authors: Daniel Gilkarov, Ran Dubin
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01758
• Pdf URL: https://arxiv.org/pdf/2503.01758
• Copy Paste: [[2503.01758]] Zero-Trust Artificial Intelligence Model Security Based on Moving Target Defense and Content Disarm and Reconstruction(https://arxiv.org/abs/2503.01758)
• Keywords: security, protect, defense, attack
• Abstract: This paper examines the challenges in distributing AI models through model zoos and file transfer mechanisms. Despite advancements in security measures, vulnerabilities persist, necessitating a multi-layered approach to mitigate risks effectively. The physical security of model files is critical, requiring stringent access controls and attack prevention solutions. This paper proposes a novel solution architecture composed of two prevention approaches. The first is Content Disarm and Reconstruction (CDR), which focuses on disarming serialization attacks that enable attackers to run malicious code as soon as the model is loaded. The second is protecting the model architecture and weights from attacks by using Moving Target Defense (MTD), alerting the model structure, and providing verification steps to detect such attacks. The paper focuses on the highly exploitable Pickle and PyTorch file formats. It demonstrates a 100% disarm rate while validated against known AI model repositories and actual malware attacks from the HuggingFace model zoo.

Title: Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models

• Authors: Zhengliang Shi, Yuhan Wang, Lingyong Yan, Pengjie Ren, Shuaiqiang Wang, Dawei Yin, Zhaochun Ren
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01763
• Pdf URL: https://arxiv.org/pdf/2503.01763
• Copy Paste: [[2503.01763]] Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models(https://arxiv.org/abs/2503.01763)
• Keywords: large language model
• Abstract: Tool learning aims to augment large language models (LLMs) with diverse tools, enabling them to act as agents for solving practical tasks. Due to the limited context length of tool-using LLMs, adopting information retrieval (IR) models to select useful tools from large toolsets is a critical initial step. However, the performance of IR models in tool retrieval tasks remains underexplored and unclear. Most tool-use benchmarks simplify this step by manually pre-annotating a small set of relevant tools for each task, which is far from the real-world scenarios. In this paper, we propose ToolRet, a heterogeneous tool retrieval benchmark comprising 7.6k diverse retrieval tasks, and a corpus of 43k tools, collected from existing datasets. We benchmark six types of models on ToolRet. Surprisingly, even the models with strong performance in conventional IR benchmarks, exhibit poor performance on ToolRet. This low retrieval quality degrades the task pass rate of tool-use LLMs. As a further step, we contribute a large-scale training dataset with over 200k instances, which substantially optimizes the tool retrieval ability of IR models.

Title: SHADE-AD: An LLM-Based Framework for Synthesizing Activity Data of Alzheimer's Patients

• Authors: Heming Fu, Hongkai Chen, Shan Lin, Guoliang Xing
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01768
• Pdf URL: https://arxiv.org/pdf/2503.01768
• Copy Paste: [[2503.01768]] SHADE-AD: An LLM-Based Framework for Synthesizing Activity Data of Alzheimer's Patients(https://arxiv.org/abs/2503.01768)
• Keywords: privacy, large language model
• Abstract: Alzheimer's Disease (AD) has become an increasingly critical global health concern, which necessitates effective monitoring solutions in smart health applications. However, the development of such solutions is significantly hindered by the scarcity of AD-specific activity datasets. To address this challenge, we propose SHADE-AD, a Large Language Model (LLM) framework for Synthesizing Human Activity Datasets Embedded with AD features. Leveraging both public datasets and our own collected data from 99 AD patients, SHADE-AD synthesizes human activity videos that specifically represent AD-related behaviors. By employing a three-stage training mechanism, it broadens the range of activities beyond those collected from limited deployment settings. We conducted comprehensive evaluations of the generated dataset, demonstrating significant improvements in downstream tasks such as Human Activity Recognition (HAR) detection, with enhancements of up to 79.69%. Detailed motion metrics between real and synthetic data show strong alignment, validating the realism and utility of the synthesized dataset. These results underscore SHADE-AD's potential to advance smart health applications by providing a cost-effective, privacy-preserving solution for AD monitoring.

Title: Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas

• Authors: Shiqi Chen, Tongyao Zhu, Ruochen Zhou, Jinghan Zhang, Siyang Gao, Juan Carlos Niebles, Mor Geva, Junxian He, Jiajun Wu, Manling Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01773
• Pdf URL: https://arxiv.org/pdf/2503.01773
• Copy Paste: [[2503.01773]] Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas(https://arxiv.org/abs/2503.01773)
• Keywords: interpretability
• Abstract: Large Vision Language Models (VLMs) have long struggled with spatial reasoning tasks. Surprisingly, even simple spatial reasoning tasks, such as recognizing "under" or "behind" relationships between only two objects, pose significant challenges for current VLMs. In this work, we study the spatial reasoning challenge from the lens of mechanistic interpretability, diving into the model's internal states to examine the interactions between image and text tokens. By tracing attention distribution over the image through out intermediate layers, we observe that successful spatial reasoning correlates strongly with the model's ability to align its attention distribution with actual object locations, particularly differing between familiar and unfamiliar spatial relationships. Motivated by these findings, we propose ADAPTVIS based on inference-time confidence scores to sharpen the attention on highly relevant regions when confident, while smoothing and broadening the attention window to consider a wider context when confidence is lower. This training-free decoding method shows significant improvement (e.g., up to a 50 absolute point improvement) on spatial reasoning benchmarks such as WhatsUp and VSR with negligible cost. We make code and data publicly available for research purposes at this https URL.

Title: Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models

• Authors: Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki, Xuanchi Ren, Jun Gao, Mike Zheng Shou, Sanja Fidler, Zan Gojcic, Huan Ling
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01774
• Pdf URL: https://arxiv.org/pdf/2503.01774
• Copy Paste: [[2503.01774]] Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models(https://arxiv.org/abs/2503.01774)
• Keywords: diffusion
• Abstract: Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D reconstruction and novel-view synthesis task. However, achieving photorealistic rendering from extreme novel viewpoints remains challenging, as artifacts persist across representations. In this work, we introduce Difix3D+, a novel pipeline designed to enhance 3D reconstruction and novel-view synthesis through single-step diffusion models. At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views caused by underconstrained regions of the 3D representation. Difix serves two critical roles in our pipeline. First, it is used during the reconstruction phase to clean up pseudo-training views that are rendered from the reconstruction and then distilled back into 3D. This greatly enhances underconstrained regions and improves the overall 3D representation quality. More importantly, Difix also acts as a neural enhancer during inference, effectively removing residual artifacts arising from imperfect 3D supervision and the limited capacity of current reconstruction models. Difix3D+ is a general solution, a single model compatible with both NeRF and 3DGS representations, and it achieves an average 2$\times$ improvement in FID score over baselines while maintaining 3D consistency.

Title: Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation

• Authors: Tiansheng Wen, Yifei Wang, Zequn Zeng, Zhong Peng, Yudi Su, Xinyang Liu, Bo Chen, Hongwei Liu, Stefanie Jegelka, Chenyu You
• Subjects: cs.LG, cs.AI, cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.01776
• Pdf URL: https://arxiv.org/pdf/2503.01776
• Copy Paste: [[2503.01776]] Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation(https://arxiv.org/abs/2503.01776)
• Keywords: generative
• Abstract: Many large-scale systems rely on high-quality deep representations (embeddings) to facilitate tasks like retrieval, search, and generative modeling. Matryoshka Representation Learning (MRL) recently emerged as a solution for adaptive embedding lengths, but it requires full model retraining and suffers from noticeable performance degradations at short lengths. In this paper, we show that sparse coding offers a compelling alternative for achieving adaptive representation with minimal overhead and higher fidelity. We propose Contrastive Sparse Representation (CSR), a method that sparsifies pre-trained embeddings into a high-dimensional but selectively activated feature space. By leveraging lightweight autoencoding and task-aware contrastive objectives, CSR preserves semantic quality while allowing flexible, cost-effective inference at different sparsity levels. Extensive experiments on image, text, and multimodal benchmarks demonstrate that CSR consistently outperforms MRL in terms of both accuracy and retrieval speed-often by large margins-while also cutting training time to a fraction of that required by MRL. Our results establish sparse coding as a powerful paradigm for adaptive representation learning in real-world applications where efficiency and fidelity are both paramount. Code is available at this https URL

Title: Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models

• Authors: Meghana Rajeev, Rajkumar Ramamurthy, Prapti Trivedi, Vikas Yadav, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudan, James Zou, Nazneen Rajani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01781
• Pdf URL: https://arxiv.org/pdf/2503.01781
• Copy Paste: [[2503.01781]] Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models(https://arxiv.org/abs/2503.01781)
• Keywords: security, attack, robust
• Abstract: We investigate the robustness of reasoning models trained for step-by-step problem solving by introducing query-agnostic adversarial triggers - short, irrelevant text that, when appended to math problems, systematically mislead models to output incorrect answers without altering the problem's semantics. We propose CatAttack, an automated iterative attack pipeline for generating triggers on a weaker, less expensive proxy model (DeepSeek V3) and successfully transfer them to more advanced reasoning target models like DeepSeek R1 and DeepSeek R1-distilled-Qwen-32B, resulting in greater than 300% increase in the likelihood of the target model generating an incorrect answer. For example, appending, "Interesting fact: cats sleep most of their lives," to any math problem leads to more than doubling the chances of a model getting the answer wrong. Our findings highlight critical vulnerabilities in reasoning models, revealing that even state-of-the-art models remain susceptible to subtle adversarial inputs, raising security and reliability concerns. The CatAttack triggers dataset with model responses is available at this https URL.

Title: OFF-CLIP: Improving Normal Detection Confidence in Radiology CLIP with Simple Off-Diagonal Term Auto-Adjustment

• Authors: Junhyun Park, Chanyu Moon, Donghwan Lee, Kyungsu Kim, Minho Hwang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.01794
• Pdf URL: https://arxiv.org/pdf/2503.01794
• Copy Paste: [[2503.01794]] OFF-CLIP: Improving Normal Detection Confidence in Radiology CLIP with Simple Off-Diagonal Term Auto-Adjustment(https://arxiv.org/abs/2503.01794)
• Keywords: robust
• Abstract: Contrastive Language-Image Pre-Training (CLIP) has enabled zero-shot classification in radiology, reducing reliance on manual annotations. However, conventional contrastive learning struggles with normal case detection due to its strict intra-sample alignment, which disrupts normal sample clustering and leads to high false positives (FPs) and false negatives (FNs). To address these issues, we propose OFF-CLIP, a contrastive learning refinement that improves normal detection by introducing an off-diagonal term loss to enhance normal sample clustering and applying sentence-level text filtering to mitigate FNs by removing misaligned normal statements from abnormal reports. OFF-CLIP can be applied to radiology CLIP models without requiring any architectural modifications. Experimental results show that OFF-CLIP significantly improves normal classification, achieving a 0.61 Area under the curve (AUC) increase on VinDr-CXR over CARZero, the state-of-the-art zero-shot classification baseline, while maintaining or improving abnormal classification performance. Additionally, OFF-CLIP enhances zero-shot grounding by improving pointing game accuracy, confirming better anomaly localization. These results demonstrate OFF-CLIP's effectiveness as a robust and efficient enhancement for medical vision-language models.

Title: PhishVQC: Optimizing Phishing URL Detection with Correlation Based Feature Selection and Variational Quantum Classifier

• Authors: Md. Farhan Shahriyar, Gazi Tanbhir, Abdullah Md Raihan Chy, Mohammed Abdul Al Arafat Tanzin, Md. Jisan Mashrafi
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01799
• Pdf URL: https://arxiv.org/pdf/2503.01799
• Copy Paste: [[2503.01799]] PhishVQC: Optimizing Phishing URL Detection with Correlation Based Feature Selection and Variational Quantum Classifier(https://arxiv.org/abs/2503.01799)
• Keywords: security, steal
• Abstract: Phishing URL detection is crucial in cybersecurity as malicious websites disguise themselves to steal sensitive infor mation. Traditional machine learning techniques struggle to per form well in complex real-world scenarios due to large datasets and intricate patterns. Motivated by quantum computing, this paper proposes using Variational Quantum Classifiers (VQC) to enhance phishing URL detection. We present PhishVQC, a quantum model that combines quantum feature maps and vari ational ansatzes such as RealAmplitude and EfficientSU2. The model is evaluated across two experimental setups with varying dataset sizes and feature map repetitions. PhishVQC achieves a maximum macro average F1-score of 0.89, showing a 22% improvement over prior studies. This highlights the potential of quantum machine learning to improve phishing detection accuracy. The study also notes computational challenges, with execution wall times increasing as dataset size grows.

Title: Deep Reinforcement Learning-Based User Association in Hybrid LiFi/WiFi Indoor Networks

• Authors: Peijun Hou, Nan Cen
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2503.01803
• Pdf URL: https://arxiv.org/pdf/2503.01803
• Copy Paste: [[2503.01803]] Deep Reinforcement Learning-Based User Association in Hybrid LiFi/WiFi Indoor Networks(https://arxiv.org/abs/2503.01803)
• Keywords: fair
• Abstract: Hybrid light fidelity (LiFi) and wireless fidelity (WiFi) indoor networks has been envisioned as a promising technology to alleviate radio frequency spectrum crunch to accommodate the ever-increasing data rate demand in indoor scenarios. The hybrid LiFi/WiFi indoor networks can leverage the advantages of fast data transmission from LiFi and wider coverage of WiFi, thus complementing well with each other and further improving the network performance compared with the standalone networks. However, to leverage the co-existence, several challenges should be addressed, including but not limited to user association, mobility support, and efficient resource allocation. Therefore, the objective of the paper is to design a new user-access point association algorithm to maximize the sum throughput of the hybrid networks. We first mathematically formulate the sum data rate maximization problem by determining the AP selection for each user in indoor networks with consideration of user mobility and practical capacity limitations, which is a nonconvex binary integer programming problem. To solve this problem, we then propose a sequential-proximal policy optimization (S-PPO) based deep reinforcement learning method. Extensive simulations are conducted to evaluate the proposed method by comparing it with exhaustive search (ES), signal strength strategy (SSS), and trust region policy optimization (TRPO) methods. Comprehensive simulation results demonstrate that our solution algorithm can outperform SSS by about 32.25% of the sum throughput and 19.09% of the fairness on average, and outperform TRPO by about 10.34% and 10.23%, respectively.

Title: $\texttt{SEM-CTRL}$: Semantically Controlled Decoding

• Authors: Mohammad Albinhassan, Pranava Madhyastha, Alessandra Russo
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01804
• Pdf URL: https://arxiv.org/pdf/2503.01804
• Copy Paste: [[2503.01804]] $\texttt{SEM-CTRL}$: Semantically Controlled Decoding(https://arxiv.org/abs/2503.01804)
• Keywords: large language model
• Abstract: Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce $\texttt{SEM-CTRL}$, a unified approach that enforces rich context-sensitive constraints and task- and instance-specific semantics directly on an LLM decoder. Our approach integrates token-level MCTS, which is guided by specific syntactic and semantic constraints. The constraints over the desired outputs are expressed using Answer Set Grammars -- a logic-based formalism that generalizes context-sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach guarantees correct completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate $\texttt{SEM-CTRL}$ on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, and planning. Our results demonstrate that $\texttt{SEM-CTRL}$ allows small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models (e.g., o1-preview) while simultaneously guaranteeing solution correctness.

Title: Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers

• Authors: Gilad Yehudai, Clayton Sanford, Maya Bechler-Speicher, Orr Fischer, Ran Gilad-Bachrach, Amir Globerson
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01805
• Pdf URL: https://arxiv.org/pdf/2503.01805
• Copy Paste: [[2503.01805]] Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers(https://arxiv.org/abs/2503.01805)
• Keywords: transformer
• Abstract: Transformers have revolutionized the field of machine learning. In particular, they can be used to solve complex algorithmic problems, including graph-based tasks. In such algorithmic tasks a key question is what is the minimal size of a transformer that can implement a task. Recent work has begun to explore this problem for graph-based tasks, showing that for sub-linear embedding dimension (i.e., model width) logarithmic depth suffices. However, an open question, which we address here, is what happens if width is allowed to grow linearly. Here we analyze this setting, and provide the surprising result that with linear width, constant depth suffices for solving a host of graph-based problems. This suggests that a moderate increase in width can allow much shallower models, which are advantageous in terms of inference time. For other problems, we show that quadratic width is required. Our results demonstrate the complex and intriguing landscape of transformer implementations of graph-based algorithms. We support our theoretical results with empirical evaluations.

Title: AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses

• Authors: Nicholas Carlini, Javier Rando, Edoardo Debenedetti, Milad Nasr, Florian Tramèr
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01811
• Pdf URL: https://arxiv.org/pdf/2503.01811
• Copy Paste: [[2503.01811]] AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses(https://arxiv.org/abs/2503.01811)
• Keywords: security, defense, attack, large language model
• Abstract: We introduce AutoAdvExBench, a benchmark to evaluate if large language models (LLMs) can autonomously exploit defenses to adversarial examples. Unlike existing security benchmarks that often serve as proxies for real-world tasks, bench directly measures LLMs' success on tasks regularly performed by machine learning security experts. This approach offers a significant advantage: if a LLM could solve the challenges presented in bench, it would immediately present practical utility for adversarial machine learning researchers. We then design a strong agent that is capable of breaking 75% of CTF-like ("homework exercise") adversarial example defenses. However, we show that this agent is only able to succeed on 13% of the real-world defenses in our benchmark, indicating the large gap between difficulty in attacking "real" code, and CTF-like code. In contrast, a stronger LLM that can attack 21% of real defenses only succeeds on 54% of CTF-like defenses. We make this benchmark available at this https URL.

Title: Noise to the Rescue: Escaping Local Minima in Neurosymbolic Local Search

• Authors: Alessandro Daniele, Emile van Krieken
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01817
• Pdf URL: https://arxiv.org/pdf/2503.01817
• Copy Paste: [[2503.01817]] Noise to the Rescue: Escaping Local Minima in Neurosymbolic Local Search(https://arxiv.org/abs/2503.01817)
• Keywords: robust
• Abstract: Deep learning has achieved remarkable success across various domains, largely thanks to the efficiency of backpropagation (BP). However, BP's reliance on differentiability poses challenges in neurosymbolic learning, where discrete computation is combined with neural models. We show that applying BP to Godel logic, which represents conjunction and disjunction as min and max, is equivalent to a local search algorithm for SAT solving, enabling the optimisation of discrete Boolean formulas without sacrificing differentiability. However, deterministic local search algorithms get stuck in local optima. Therefore, we propose the Godel Trick, which adds noise to the model's logits to escape local optima. We evaluate the Godel Trick on SATLIB, and demonstrate its ability to solve a broad range of SAT problems. Additionally, we apply it to neurosymbolic models and achieve state-of-the-art performance on Visual Sudoku, all while avoiding expensive probabilistic reasoning. These results highlight the Godel Trick's potential as a robust, scalable approach for integrating symbolic reasoning with neural architectures.

Title: On the Power of Context-Enhanced Learning in LLMs

• Authors: Xingyu Zhu, Abhishek Panigrahi, Sanjeev Arora
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01821
• Pdf URL: https://arxiv.org/pdf/2503.01821
• Copy Paste: [[2503.01821]] On the Power of Context-Enhanced Learning in LLMs(https://arxiv.org/abs/2503.01821)
• Keywords: security
• Abstract: We formalize a new concept for LLMs, context-enhanced learning. It involves standard gradient-based learning on text except that the context is enhanced with additional data on which no auto-regressive gradients are computed. This setting is a gradient-based analog of usual in-context learning (ICL) and appears in some recent works. Using a multi-step reasoning task, we prove in a simplified setting that context-enhanced learning can be exponentially more sample-efficient than standard learning when the model is capable of ICL. At a mechanistic level, we find that the benefit of context-enhancement arises from a more accurate gradient learning signal. We also experimentally demonstrate that it appears hard to detect or recover learning materials that were used in the context during training. This may have implications for data security as well as copyright.

Title: Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry

• Authors: Sai Sumedh R. Hindupur, Ekdeep Singh Lubana, Thomas Fel, Demba Ba
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.01822
• Pdf URL: https://arxiv.org/pdf/2503.01822
• Copy Paste: [[2503.01822]] Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry(https://arxiv.org/abs/2503.01822)
• Keywords: interpretability
• Abstract: Sparse Autoencoders (SAEs) are widely used to interpret neural networks by identifying meaningful concepts from their representations. However, do SAEs truly uncover all concepts a model relies on, or are they inherently biased toward certain kinds of concepts? We introduce a unified framework that recasts SAEs as solutions to a bilevel optimization problem, revealing a fundamental challenge: each SAE imposes structural assumptions about how concepts are encoded in model representations, which in turn shapes what it can and cannot detect. This means different SAEs are not interchangeable -- switching architectures can expose entirely new concepts or obscure existing ones. To systematically probe this effect, we evaluate SAEs across a spectrum of settings: from controlled toy models that isolate key variables, to semi-synthetic experiments on real model activations and finally to large-scale, naturalistic datasets. Across this progression, we examine two fundamental properties that real-world concepts often exhibit: heterogeneity in intrinsic dimensionality (some concepts are inherently low-dimensional, others are not) and nonlinear separability. We show that SAEs fail to recover concepts when these properties are ignored, and we design a new SAE that explicitly incorporates both, enabling the discovery of previously hidden concepts and reinforcing our theoretical insights. Our findings challenge the idea of a universal SAE and underscores the need for architecture-specific choices in model interpretability. Overall, we argue an SAE does not just reveal concepts -- it determines what can be seen at all.

Title: From superposition to sparse codes: interpretable representations in neural networks

• Authors: David Klindt, Charles O'Neill, Patrik Reizinger, Harald Maurer, Nina Miolane
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.01824
• Pdf URL: https://arxiv.org/pdf/2503.01824
• Copy Paste: [[2503.01824]] From superposition to sparse codes: interpretable representations in neural networks(https://arxiv.org/abs/2503.01824)
• Keywords: interpretability
• Abstract: Understanding how information is represented in neural networks is a fundamental challenge in both neuroscience and artificial intelligence. Despite their nonlinear architectures, recent evidence suggests that neural networks encode features in superposition, meaning that input concepts are linearly overlaid within the network's representations. We present a perspective that explains this phenomenon and provides a foundation for extracting interpretable representations from neural activations. Our theoretical framework consists of three steps: (1) Identifiability theory shows that neural networks trained for classification recover latent features up to a linear transformation. (2) Sparse coding methods can extract disentangled features from these representations by leveraging principles from compressed sensing. (3) Quantitative interpretability metrics provide a means to assess the success of these methods, ensuring that extracted features align with human-interpretable concepts. By bridging insights from theoretical neuroscience, representation learning, and interpretability research, we propose an emerging perspective on understanding neural representations in both artificial and biological systems. Our arguments have implications for neural coding theories, AI transparency, and the broader goal of making deep learning models more interpretable.

Title: Open-source framework for detecting bias and overfitting for large pathology images

• Authors: Anders Sildnes, Nikita Shvetsov, Masoud Tafavvoghi, Vi Ngoc-Nha Tran, Kajsa Møllersen, Lill-Tove Rasmussen Busund, Thomas K. Kilvær, Lars Ailo Bongo
• Subjects: cs.LG, cs.SE, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.01827
• Pdf URL: https://arxiv.org/pdf/2503.01827
• Copy Paste: [[2503.01827]] Open-source framework for detecting bias and overfitting for large pathology images(https://arxiv.org/abs/2503.01827)
• Keywords: robust
• Abstract: Even foundational models that are trained on datasets with billions of data samples may develop shortcuts that lead to overfitting and bias. Shortcuts are non-relevant patterns in data, such as the background color or color intensity. So, to ensure the robustness of deep learning applications, there is a need for methods to detect and remove such shortcuts. Today's model debugging methods are time consuming since they often require customization to fit for a given model architecture in a specific domain. We propose a generalized, model-agnostic framework to debug deep learning models. We focus on the domain of histopathology, which has very large images that require large models - and therefore large computation resources. It can be run on a workstation with a commodity GPU. We demonstrate that our framework can replicate non-image shortcuts that have been found in previous work for self-supervised learning models, and we also identify possible shortcuts in a foundation model. Our easy to use tests contribute to the development of more reliable, accurate, and generalizable models for WSI analysis. Our framework is available as an open-source tool available on github.

Title: Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models

• Authors: Nimet Beyza Bozdag, Shuhaib Mehri, Gokhan Tur, Dilek Hakkani-Tür
• Subjects: cs.CL, cs.AI, cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.01829
• Pdf URL: https://arxiv.org/pdf/2503.01829
• Copy Paste: [[2503.01829]] Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models(https://arxiv.org/abs/2503.01829)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) demonstrate persuasive capabilities that rival human-level persuasion. While these capabilities can be used for social good, they also present risks of potential misuse. Moreover, LLMs' susceptibility to persuasion raises concerns about alignment with ethical principles. To study these dynamics, we introduce Persuade Me If You Can (PMIYC), an automated framework for evaluating persuasion through multi-agent interactions. Here, Persuader agents engage in multi-turn conversations with the Persuadee agents, allowing us to measure LLMs' persuasive effectiveness and their susceptibility to persuasion. We conduct comprehensive evaluations across diverse LLMs, ensuring each model is assessed against others in both subjective and misinformation contexts. We validate the efficacy of our framework through human evaluations and show alignment with prior work. PMIYC offers a scalable alternative to human annotation for studying persuasion in LLMs. Through PMIYC, we find that Llama-3.3-70B and GPT-4o exhibit similar persuasive effectiveness, outperforming Claude 3 Haiku by 30%. However, GPT-4o demonstrates over 50% greater resistance to persuasion for misinformation compared to Llama-3.3-70B. These findings provide empirical insights into the persuasive dynamics of LLMs and contribute to the development of safer AI systems.

Title: From Language to Cognition: How LLMs Outgrow the Human Language Network

• Authors: Badr AlKhamissi, Greta Tuckute, Yingtian Tang, Taha Binhuraib, Antoine Bosselut, Martin Schrimpf
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.01830
• Pdf URL: https://arxiv.org/pdf/2503.01830
• Copy Paste: [[2503.01830]] From Language to Cognition: How LLMs Outgrow the Human Language Network(https://arxiv.org/abs/2503.01830)
• Keywords: large language model
• Abstract: Large language models (LLMs) exhibit remarkable similarity to neural activity in the human language network. However, the key properties of language shaping brain-like representations, and their evolution during training as a function of different tasks remain unclear. We here benchmark 34 training checkpoints spanning 300B tokens across 8 different model sizes to analyze how brain alignment relates to linguistic competence. Specifically, we find that brain alignment tracks the development of formal linguistic competence -- i.e., knowledge of linguistic rules -- more closely than functional linguistic competence. While functional competence, which involves world knowledge and reasoning, continues to develop throughout training, its relationship with brain alignment is weaker, suggesting that the human language network primarily encodes formal linguistic structure rather than broader cognitive functions. We further show that model size is not a reliable predictor of brain alignment when controlling for feature size and find that the correlation between next-word prediction, behavioral alignment and brain alignment fades once models surpass human language proficiency. Finally, using the largest set of rigorous neural language benchmarks to date, we show that language brain alignment benchmarks remain unsaturated, highlighting opportunities for improving future models. Taken together, our findings suggest that the human language network is best modeled by formal, rather than functional, aspects of language.