2025-05-12

Title: Data extraction and processing methods to aid the study of driving behaviors at intersections in naturalistic driving

• Authors: Shrinivas Pundlik, Seonggyu Choe, Patrick Baker, Chen-Yuan Lee, Naser Al-Madi, Alex R. Bowers, Gang Luo
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.05487
• Pdf URL: https://arxiv.org/pdf/2505.05487
• Copy Paste: [[2505.05487]] Data extraction and processing methods to aid the study of driving behaviors at intersections in naturalistic driving(https://arxiv.org/abs/2505.05487)
• Keywords: extraction
• Abstract: Naturalistic driving studies use devices in participants' own vehicles to record daily driving over many months. Due to diverse and extensive amounts of data recorded, automated processing is necessary. This report describes methods to extract and characterize driver head scans at intersections from data collected from an in-car recording system that logged vehicle speed, GPS location, scene videos, and cabin videos. Custom tools were developed to mark the intersections, synchronize location and video data, and clip the cabin and scene videos for +/-100 meters from the intersection location. A custom-developed head pose detection AI model for wide angle head turns was run on the cabin videos to estimate the driver head pose, from which head scans >20 deg were computed in the horizontal direction. The scene videos were processed using a YOLO object detection model to detect traffic lights, stop signs, pedestrians, and other vehicles on the road. Turning maneuvers were independently detected using vehicle self-motion patterns. Stop lines on the road surface were detected using changing intensity patterns over time as the vehicle moved. The information obtained from processing the scene videos, along with the speed data was used in a rule-based algorithm to infer the intersection type, maneuver, and bounds. We processed 190 intersections from 3 vehicles driven in cities and suburban areas from Massachusetts and California. The automated video processing algorithm correctly detected intersection signage and maneuvers in 100% and 94% of instances, respectively. The median [IQR] error in detecting vehicle entry into the intersection was 1.1[0.4-4.9] meters and 0.2[0.1-0.54] seconds. The median overlap between ground truth and estimated intersection bounds was 0.88[0.82-0.93].

Title: MDDFNet: Mamba-based Dynamic Dual Fusion Network for Traffic Sign Detection

• Authors: TianYi Yu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05491
• Pdf URL: https://arxiv.org/pdf/2505.05491
• Copy Paste: [[2505.05491]] MDDFNet: Mamba-based Dynamic Dual Fusion Network for Traffic Sign Detection(https://arxiv.org/abs/2505.05491)
• Keywords: extraction
• Abstract: The Detection of small objects, especially traffic signs, is a critical sub-task in object detection and autonomous driving. Despite signficant progress in previous research, two main challenges remain. First, the issue of feature extraction being too singular. Second, the detection process struggles to efectively handle objects of varying sizes or scales. These problems are also prevalent in general object detection tasks. To address these challenges, we propose a novel object detection network, Mamba-based Dynamic Dual Fusion Network (MDDFNet), for traffic sign detection. The network integrates a dynamic dual fusion module and a Mamba-based backbone to simultaneously tackle the aforementioned issues. Specifically, the dynamic dual fusion module utilizes multiple branches to consolidate various spatial and semantic information, thus enhancing feature diversity. The Mamba-based backbone leverages global feature fusion and local feature interaction, combining features in an adaptive manner to generate unique classification characteristics. Extensive experiments conducted on the TT100K (Tsinghua-Tencent 100K) datasets demonstrate that MDDFNet outperforms other state-of-the-art detectors, maintaining real-time processing capabilities of single-stage models while achieving superior performance. This confirms the efectiveness of MDDFNet in detecting small traffic signs.

Title: DetoxAI: a Python Toolkit for Debiasing Deep Learning Models in Computer Vision

• Authors: Ignacy Stępka, Lukasz Sztukiewicz, Michał Wiliński, Jerzy Stefanowski
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05492
• Pdf URL: https://arxiv.org/pdf/2505.05492
• Copy Paste: [[2505.05492]] DetoxAI: a Python Toolkit for Debiasing Deep Learning Models in Computer Vision(https://arxiv.org/abs/2505.05492)
• Keywords: fair
• Abstract: While machine learning fairness has made significant progress in recent years, most existing solutions focus on tabular data and are poorly suited for vision-based classification tasks, which rely heavily on deep learning. To bridge this gap, we introduce DetoxAI, an open-source Python library for improving fairness in deep learning vision classifiers through post-hoc debiasing. DetoxAI implements state-of-the-art debiasing algorithms, fairness metrics, and visualization tools. It supports debiasing via interventions in internal representations and includes attribution-based visualization tools and quantitative algorithmic fairness metrics to show how bias is mitigated. This paper presents the motivation, design, and use cases of DetoxAI, demonstrating its tangible value to engineers and researchers.

Title: Learning 3D Persistent Embodied World Models

• Authors: Siyuan Zhou, Yilun Du, Yuncong Yang, Lei Han, Peihao Chen, Dit-Yan Yeung, Chuang Gan
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.05495
• Pdf URL: https://arxiv.org/pdf/2505.05495
• Copy Paste: [[2505.05495]] Learning 3D Persistent Embodied World Models(https://arxiv.org/abs/2505.05495)
• Keywords: diffusion
• Abstract: The ability to simulate the effects of future actions on the world is a crucial ability of intelligent embodied agents, enabling agents to anticipate the effects of their actions and make plans accordingly. While a large body of existing work has explored how to construct such world models using video models, they are often myopic in nature, without any memory of a scene not captured by currently observed images, preventing agents from making consistent long-horizon plans in complex environments where many parts of the scene are partially observed. We introduce a new persistent embodied world model with an explicit memory of previously generated content, enabling much more consistent long-horizon simulation. During generation time, our video diffusion model predicts RGB-D video of the future observations of the agent. This generation is then aggregated into a persistent 3D map of the environment. By conditioning the video model on this 3D spatial map, we illustrate how this enables video world models to faithfully simulate both seen and unseen parts of the world. Finally, we illustrate the efficacy of such a world model in downstream embodied applications, enabling effective planning and policy learning.

Title: Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation

• Authors: Yiming Qin, Zhu Xu, Yang Liu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05505
• Pdf URL: https://arxiv.org/pdf/2505.05505
• Copy Paste: [[2505.05505]] Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation(https://arxiv.org/abs/2505.05505)
• Keywords: large language model
• Abstract: Recent text-to-3D models can render high-quality assets, yet they still stumble on objects with complex attributes. The key obstacles are: (1) existing text-to-3D approaches typically lift text-to-image models to extract semantics via text encoders, while the text encoder exhibits limited comprehension ability for long descriptions, leading to deviated cross-attention focus, subsequently wrong attribute binding in generated results. (2) Occluded object parts demand a disciplined generation order and explicit part disentanglement. Though some works introduce manual efforts to alleviate the above issues, their quality is unstable and highly reliant on manual information. To tackle above problems, we propose a automated method Hierarchical-Chain-of-Generation (HCoG). It leverages a large language model to decompose the long description into blocks representing different object parts, and orders them from inside out according to occlusions, forming a hierarchical chain. Within each block we first coarsely create components, then precisely bind attributes via target-region localization and corresponding 3D Gaussian kernel optimization. Between blocks, we introduce Gaussian Extension and Label Elimination to seamlessly generate new parts by extending new Gaussian kernels, re-assigning semantic labels, and eliminating unnecessary kernels, ensuring that only relevant parts are added without disrupting previously optimized parts. Experiments confirm that HCoG yields structurally coherent, attribute-faithful 3D objects with complex attributes. The code is available at this https URL .

Title: Occupancy World Model for Robots

• Authors: Zhang Zhang, Qiang Zhang, Wei Cui, Shuai Shi, Yijie Guo, Gang Han, Wen Zhao, Jingkai Sun, Jiahang Cao, Jiaxu Wang, Hao Cheng, Xiaozhu Ju, Zhengping Che, Renjing Xu, Jian Tang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2505.05512
• Pdf URL: https://arxiv.org/pdf/2505.05512
• Copy Paste: [[2505.05512]] Occupancy World Model for Robots(https://arxiv.org/abs/2505.05512)
• Keywords: transformer, generative
• Abstract: Understanding and forecasting the scene evolutions deeply affect the exploration and decision of embodied agents. While traditional methods simulate scene evolutions through trajectory prediction of potential instances, current works use the occupancy world model as a generative framework for describing fine-grained overall scene dynamics. However, existing methods cluster on the outdoor structured road scenes, while ignoring the exploration of forecasting 3D occupancy scene evolutions for robots in indoor scenes. In this work, we explore a new framework for learning the scene evolutions of observed fine-grained occupancy and propose an occupancy world model based on the combined spatio-temporal receptive field and guided autoregressive transformer to forecast the scene evolutions, called RoboOccWorld. We propose the Conditional Causal State Attention (CCSA), which utilizes camera poses of next state as conditions to guide the autoregressive transformer to adapt and understand the indoor robotics scenarios. In order to effectively exploit the spatio-temporal cues from historical observations, Hybrid Spatio-Temporal Aggregation (HSTA) is proposed to obtain the combined spatio-temporal receptive field based on multi-scale spatio-temporal windows. In addition, we restructure the OccWorld-ScanNet benchmark based on local annotations to facilitate the evaluation of the indoor 3D occupancy scene evolution prediction task. Experimental results demonstrate that our RoboOccWorld outperforms state-of-the-art methods in indoor 3D occupancy scene evolution prediction task. The code will be released soon.

Title: Exploring Convolutional Neural Networks for Rice Grain Classification: An Explainable AI Approach

• Authors: Muhammad Junaid Asif, Hamza Khan, Rabia Tehseen, Syed Tahir Hussain Rizvi, Mujtaba Asad, Shazia Saqib, Rana Fayyaz Ahmad
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05513
• Pdf URL: https://arxiv.org/pdf/2505.05513
• Copy Paste: [[2505.05513]] Exploring Convolutional Neural Networks for Rice Grain Classification: An Explainable AI Approach(https://arxiv.org/abs/2505.05513)
• Keywords: explainability
• Abstract: Rice is an essential staple food worldwide that is important in promoting international trade, economic growth, and nutrition. Asian countries such as China, India, Pakistan, Thailand, Vietnam, and Indonesia are notable for their significant contribution to the cultivation and utilization of rice. These nations are also known for cultivating different rice grains, including short and long grains. These sizes are further classified as basmati, jasmine, kainat saila, ipsala, arborio, etc., catering to diverse culinary preferences and cultural traditions. For both local and international trade, inspecting and maintaining the quality of rice grains to satisfy customers and preserve a country's reputation is necessary. Manual quality check and classification is quite a laborious and time-consuming process. It is also highly prone to mistakes. Therefore, an automatic solution must be proposed for the effective and efficient classification of different varieties of rice grains. This research paper presents an automatic framework based on a convolutional neural network (CNN) for classifying different varieties of rice grains. We evaluated the proposed model based on performance metrics such as accuracy, recall, precision, and F1-Score. The CNN model underwent rigorous training and validation, achieving a remarkable accuracy rate and a perfect area under each class's Receiver Operating Characteristic (ROC) curve. The confusion matrix analysis confirmed the model's effectiveness in distinguishing between the different rice varieties, indicating minimal misclassifications. Additionally, the integration of explainability techniques such as LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) provided valuable insights into the model's decision-making process, revealing how specific features of the rice grains influenced classification outcomes.

Title: Real-Time Privacy Preservation for Robot Visual Perception

• Authors: Minkyu Choi, Yunhao Yang, Neel P. Bhatt, Kushagra Gupta, Sahil Shah, Aditya Rai, David Fridovich-Keil, Ufuk Topcu, Sandeep P. Chinchali
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05519
• Pdf URL: https://arxiv.org/pdf/2505.05519
• Copy Paste: [[2505.05519]] Real-Time Privacy Preservation for Robot Visual Perception(https://arxiv.org/abs/2505.05519)
• Keywords: privacy
• Abstract: Many robots (e.g., iRobot's Roomba) operate based on visual observations from live video streams, and such observations may inadvertently include privacy-sensitive objects, such as personal identifiers. Existing approaches for preserving privacy rely on deep learning models, differential privacy, or cryptography. They lack guarantees for the complete concealment of all sensitive objects. Guaranteeing concealment requires post-processing techniques and thus is inadequate for real-time video streams. We develop a method for privacy-constrained video streaming, PCVS, that conceals sensitive objects within real-time video streams. PCVS takes a logical specification constraining the existence of privacy-sensitive objects, e.g., never show faces when a person exists. It uses a detection model to evaluate the existence of these objects in each incoming frame. Then, it blurs out a subset of objects such that the existence of the remaining objects satisfies the specification. We then propose a conformal prediction approach to (i) establish a theoretical lower bound on the probability of the existence of these objects in a sequence of frames satisfying the specification and (ii) update the bound with the arrival of each subsequent frame. Quantitative evaluations show that PCVS achieves over 95 percent specification satisfaction rate in multiple datasets, significantly outperforming other methods. The satisfaction rate is consistently above the theoretical bounds across all datasets, indicating that the established bounds hold. Additionally, we deploy PCVS on robots in real-time operation and show that the robots operate normally without being compromised when PCVS conceals objects.

Title: GaMNet: A Hybrid Network with Gabor Fusion and NMamba for Efficient 3D Glioma Segmentation

• Authors: Chengwei Ye, Huanzhen Zhang, Yufei Lin, Kangsheng Wang, Linuo Xu, Shuyan Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05520
• Pdf URL: https://arxiv.org/pdf/2505.05520
• Copy Paste: [[2505.05520]] GaMNet: A Hybrid Network with Gabor Fusion and NMamba for Efficient 3D Glioma Segmentation(https://arxiv.org/abs/2505.05520)
• Keywords: extraction, interpretability, transformer, segmentation
• Abstract: Gliomas are aggressive brain tumors that pose serious health risks. Deep learning aids in lesion segmentation, but CNN and Transformer-based models often lack context modeling or demand heavy computation, limiting real-time use on mobile medical devices. We propose GaMNet, integrating the NMamba module for global modeling and a multi-scale CNN for efficient local feature extraction. To improve interpretability and mimic the human visual system, we apply Gabor filters at multiple scales. Our method achieves high segmentation accuracy with fewer parameters and faster computation. Extensive experiments show GaMNet outperforms existing methods, notably reducing false positives and negatives, which enhances the reliability of clinical diagnosis.

Title: A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows

• Authors: Selim Mecanna, Aurore Loisy, Christophe Eloy
• Subjects: cs.LG, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2505.05525
• Pdf URL: https://arxiv.org/pdf/2505.05525
• Copy Paste: [[2505.05525]] A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows(https://arxiv.org/abs/2505.05525)
• Keywords: robust
• Abstract: Navigating in a fluid flow while being carried by it, using only information accessible from on-board sensors, is a problem commonly faced by small planktonic organisms. It is also directly relevant to autonomous robots deployed in the oceans. In the last ten years, the fluid mechanics community has widely adopted reinforcement learning, often in the form of its simplest implementations, to address this challenge. But it is unclear how good are the strategies learned by these algorithms. In this paper, we perform a quantitative assessment of reinforcement learning methods applied to navigation in partially observable flows. We first introduce a well-posed problem of directional navigation for which a quasi-optimal policy is known analytically. We then report on the poor performance and robustness of commonly used algorithms (Q-Learning, Advantage Actor Critic) in flows regularly encountered in the literature: Taylor-Green vortices, Arnold-Beltrami-Childress flow, and two-dimensional turbulence. We show that they are vastly surpassed by PPO (Proximal Policy Optimization), a more advanced algorithm that has established dominance across a wide range of benchmarks in the reinforcement learning community. In particular, our custom implementation of PPO matches the theoretical quasi-optimal performance in turbulent flow and does so in a robust manner. Reaching this result required the use of several additional techniques, such as vectorized environments and generalized advantage estimation, as well as hyperparameter optimization. This study demonstrates the importance of algorithm selection, implementation details, and fine-tuning for discovering truly smart autonomous navigation strategies in complex flows.

Title: X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP

• Authors: Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma, James Bailey
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05528
• Pdf URL: https://arxiv.org/pdf/2505.05528
• Copy Paste: [[2505.05528]] X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP(https://arxiv.org/abs/2505.05528)
• Keywords: attack
• Abstract: As Contrastive Language-Image Pre-training (CLIP) models are increasingly adopted for diverse downstream tasks and integrated into large vision-language models (VLMs), their susceptibility to adversarial perturbations has emerged as a critical concern. In this work, we introduce \textbf{X-Transfer}, a novel attack method that exposes a universal adversarial vulnerability in CLIP. X-Transfer generates a Universal Adversarial Perturbation (UAP) capable of deceiving various CLIP encoders and downstream VLMs across different samples, tasks, and domains. We refer to this property as \textbf{super transferability}--a single perturbation achieving cross-data, cross-domain, cross-model, and cross-task adversarial transferability simultaneously. This is achieved through \textbf{surrogate scaling}, a key innovation of our approach. Unlike existing methods that rely on fixed surrogate models, which are computationally intensive to scale, X-Transfer employs an efficient surrogate scaling strategy that dynamically selects a small subset of suitable surrogates from a large search space. Extensive evaluations demonstrate that X-Transfer significantly outperforms previous state-of-the-art UAP methods, establishing a new benchmark for adversarial transferability across CLIP models. The code is publicly available in our \href{this https URL}{GitHub repository}.

Title: OXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contours

• Authors: Hanie Moghaddasi, Christina Chambers, Sarah N. Mattson, Jeffrey R. Wozniak, Claire D. Coles, Raja Mukherjee, Michael Suttie
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05531
• Pdf URL: https://arxiv.org/pdf/2505.05531
• Copy Paste: [[2505.05531]] OXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contours(https://arxiv.org/abs/2505.05531)
• Keywords: generative, segmentation
• Abstract: Lip segmentation plays a crucial role in various domains, such as lip synchronization, lipreading, and diagnostics. However, the effectiveness of supervised lip segmentation is constrained by the availability of lip contour in the training phase. A further challenge with lip segmentation is its reliance on image quality , lighting, and skin tone, leading to inaccuracies in the detected boundaries. To address these challenges, we propose a sequential lip segmentation method that integrates attention UNet and multidimensional input. We unravel the micro-patterns in facial images using local binary patterns to build multidimensional inputs. Subsequently, the multidimensional inputs are fed into sequential attention UNets, where the lip contour is reconstructed. We introduce a mask generation method that uses a few anatomical landmarks and estimates the complete lip contour to improve segmentation accuracy. This mask has been utilized in the training phase for lip segmentation. To evaluate the proposed method, we use facial images to segment the upper lips and subsequently assess lip-related facial anomalies in subjects with fetal alcohol syndrome (FAS). Using the proposed lip segmentation method, we achieved a mean dice score of 84.75%, and a mean pixel accuracy of 99.77% in upper lip segmentation. To further evaluate the method, we implemented classifiers to identify those with FAS. Using a generative adversarial network (GAN), we reached an accuracy of 98.55% in identifying FAS in one of the study populations. This method could be used to improve lip segmentation accuracy, especially around Cupid's bow, and shed light on distinct lip-related characteristics of FAS.

Title: Cardioformer: Advancing AI in ECG Analysis with Multi-Granularity Patching and ResNet

• Authors: Md Kamrujjaman Mobin, Md Saiful Islam, Sadik Al Barid, Md Masum
• Subjects: cs.LG, cs.AI, eess.SP
• Abstract URL: https://arxiv.org/abs/2505.05538
• Pdf URL: https://arxiv.org/pdf/2505.05538
• Copy Paste: [[2505.05538]] Cardioformer: Advancing AI in ECG Analysis with Multi-Granularity Patching and ResNet(https://arxiv.org/abs/2505.05538)
• Keywords: robust, transformer
• Abstract: Electrocardiogram (ECG) classification is crucial for automated cardiac disease diagnosis, yet existing methods often struggle to capture local morphological details and long-range temporal dependencies simultaneously. To address these challenges, we propose Cardioformer, a novel multi-granularity hybrid model that integrates cross-channel patching, hierarchical residual learning, and a two-stage self-attention mechanism. Cardioformer first encodes multi-scale token embeddings to capture fine-grained local features and global contextual information and then selectively fuses these representations through intra- and inter-granularity self-attention. Extensive evaluations on three benchmark ECG datasets under subject-independent settings demonstrate that model consistently outperforms four state-of-the-art baselines. Our Cardioformer model achieves the AUROC of 96.34$\pm$0.11, 89.99$\pm$0.12, and 95.59$\pm$1.66 in MIMIC-IV, PTB-XL and PTB dataset respectively outperforming PatchTST, Reformer, Transformer, and Medformer models. It also demonstrates strong cross-dataset generalization, achieving 49.18% AUROC on PTB and 68.41% on PTB-XL when trained on MIMIC-IV. These findings underscore the potential of Cardioformer to advance automated ECG analysis, paving the way for more accurate and robust cardiovascular disease diagnosis. We release the source code at this https URL.

Title: Benchmarking Vision, Language, & Action Models in Procedurally Generated, Open Ended Action Environments

• Authors: Pranav Guruprasad, Yangyue Wang, Sudipta Chowdhury, Harshvardhan Sikka
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05540
• Pdf URL: https://arxiv.org/pdf/2505.05540
• Copy Paste: [[2505.05540]] Benchmarking Vision, Language, & Action Models in Procedurally Generated, Open Ended Action Environments(https://arxiv.org/abs/2505.05540)
• Keywords: robust
• Abstract: Vision-language-action (VLA) models represent an important step toward general-purpose robotic systems by integrating visual perception, language understanding, and action execution. However, systematic evaluation of these models, particularly their zero-shot generalization capabilities in out-of-distribution (OOD) environments, remains limited. In this paper, we introduce MultiNet v0.2, a comprehensive benchmark designed to evaluate and analyze the generalization performance of state-of-the-art VLM and VLA models-including GPT-4o, GPT-4.1, OpenVLA,Pi0 Base, and Pi0 FAST-on diverse procedural tasks from the Procgen benchmark. Our analysis reveals several critical insights: (1) all evaluated models exhibit significant limitations in zero-shot generalization to OOD tasks, with performance heavily influenced by factors such as action representation and task complexit; (2) VLAs generally outperform other models due to their robust architectural design; and (3) VLM variants demonstrate substantial improvements when constrained appropriately, highlighting the sensitivity of model performance to precise prompt engineering.

Title: Prompt to Polyp: Clinically-Aware Medical Image Synthesis with Diffusion Models

• Authors: Mikhail Chaichuk, Sushant Gautam, Steven Hicks, Elena Tutubalina
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05573
• Pdf URL: https://arxiv.org/pdf/2505.05573
• Copy Paste: [[2505.05573]] Prompt to Polyp: Clinically-Aware Medical Image Synthesis with Diffusion Models(https://arxiv.org/abs/2505.05573)
• Keywords: privacy, diffusion
• Abstract: The generation of realistic medical images from text descriptions has significant potential to address data scarcity challenges in healthcare AI while preserving patient privacy. This paper presents a comprehensive study of text-to-image synthesis in the medical domain, comparing two distinct approaches: (1) fine-tuning large pre-trained latent diffusion models and (2) training small, domain-specific models. We introduce a novel model named MSDM, an optimized architecture based on Stable Diffusion that integrates a clinical text encoder, variational autoencoder, and cross-attention mechanisms to better align medical text prompts with generated images. Our study compares two approaches: fine-tuning large pre-trained models (FLUX, Kandinsky) versus training compact domain-specific models (MSDM). Evaluation across colonoscopy (MedVQA-GI) and radiology (ROCOv2) datasets reveals that while large models achieve higher fidelity, our optimized MSDM delivers comparable quality with lower computational costs. Quantitative metrics and qualitative evaluations by medical experts reveal strengths and limitations of each approach.

Title: KG-HTC: Integrating Knowledge Graphs into LLMs for Effective Zero-shot Hierarchical Text Classification

• Authors: Qianbo Zang, Christophe Zgrzendek, Igor Tchappi, Afshin Khadangi, Johannes Sedlmeir
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05583
• Pdf URL: https://arxiv.org/pdf/2505.05583
• Copy Paste: [[2505.05583]] KG-HTC: Integrating Knowledge Graphs into LLMs for Effective Zero-shot Hierarchical Text Classification(https://arxiv.org/abs/2505.05583)
• Keywords: large language model
• Abstract: Hierarchical Text Classification (HTC) involves assigning documents to labels organized within a taxonomy. Most previous research on HTC has focused on supervised methods. However, in real-world scenarios, employing supervised HTC can be challenging due to a lack of annotated data. Moreover, HTC often faces issues with large label spaces and long-tail distributions. In this work, we present Knowledge Graphs for zero-shot Hierarchical Text Classification (KG-HTC), which aims to address these challenges of HTC in applications by integrating knowledge graphs with Large Language Models (LLMs) to provide structured semantic context during classification. Our method retrieves relevant subgraphs from knowledge graphs related to the input text using a Retrieval-Augmented Generation (RAG) approach. Our KG-HTC can enhance LLMs to understand label semantics at various hierarchy levels. We evaluate KG-HTC on three open-source HTC datasets: WoS, DBpedia, and Amazon. Our experimental results show that KG-HTC significantly outperforms three baselines in the strict zero-shot setting, particularly achieving substantial improvements at deeper levels of the hierarchy. This evaluation demonstrates the effectiveness of incorporating structured knowledge into LLMs to address HTC's challenges in large label spaces and long-tailed label distributions. Our code is available at: this https URL.

Title: ReactDance: Progressive-Granular Representation for Long-Term Coherent Reactive Dance Generation

• Authors: Jingzhong Lin, Yuanyuan Qi, Xinru Li, Wenxuan Huang, Xiangfeng Xu, Bangyan Li, Xuejiao Wang, Gaoqi He
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05589
• Pdf URL: https://arxiv.org/pdf/2505.05589
• Copy Paste: [[2505.05589]] ReactDance: Progressive-Granular Representation for Long-Term Coherent Reactive Dance Generation(https://arxiv.org/abs/2505.05589)
• Keywords: diffusion
• Abstract: Reactive dance generation (RDG) produces follower movements conditioned on guiding dancer and music while ensuring spatial coordination and temporal coherence. However, existing methods overemphasize global constraints and optimization, overlooking local information, such as fine-grained spatial interactions and localized temporal context. Therefore, we present ReactDance, a novel diffusion-based framework for high-fidelity RDG with long-term coherence and multi-scale controllability. Unlike existing methods that struggle with interaction fidelity, synchronization, and temporal consistency in duet synthesis, our approach introduces two key innovations: 1)Group Residual Finite Scalar Quantization (GRFSQ), a multi-scale disentangled motion representation that captures interaction semantics from coarse body rhythms to fine-grained joint dynamics, and 2)Blockwise Local Context (BLC), a sampling strategy eliminating error accumulation in long sequence generation via local block causal masking and periodic positional encoding. Built on the decoupled multi-scale GRFSQ representation, we implement a diffusion model withLayer-Decoupled Classifier-free Guidance (LDCFG), allowing granular control over motion semantics across scales. Extensive experiments on standard benchmarks demonstrate that ReactDance surpasses existing methods, achieving state-of-the-art performance.

Title: Anticipating Gaming to Incentivize Improvement: Guiding Agents in (Fair) Strategic Classification

• Authors: Sura Alhanouti, Parinaz Naghizadeh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05594
• Pdf URL: https://arxiv.org/pdf/2505.05594
• Copy Paste: [[2505.05594]] Anticipating Gaming to Incentivize Improvement: Guiding Agents in (Fair) Strategic Classification(https://arxiv.org/abs/2505.05594)
• Keywords: fair
• Abstract: As machine learning algorithms increasingly influence critical decision making in different application areas, understanding human strategic behavior in response to these systems becomes vital. We explore individuals' choice between genuinely improving their qualifications (``improvement'') vs. attempting to deceive the algorithm by manipulating their features (``manipulation'') in response to an algorithmic decision system. We further investigate an algorithm designer's ability to shape these strategic responses, and its fairness implications. Specifically, we formulate these interactions as a Stackelberg game, where a firm deploys a (fair) classifier, and individuals strategically respond. Our model incorporates both different costs and stochastic efficacy for manipulation and improvement. The analysis reveals different potential classes of agent responses, and characterizes optimal classifiers accordingly. Based on these, we highlight the impact of the firm's anticipation of strategic behavior, identifying when and why a (fair) strategic policy can not only prevent manipulation, but also incentivize agents to opt for improvement.

Title: Enhancing Satellite Object Localization with Dilated Convolutions and Attention-aided Spatial Pooling

• Authors: Seraj Al Mahmud Mostafa, Chenxi Wang, Jia Yue, Yuta Hozumi, Jianwu Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05599
• Pdf URL: https://arxiv.org/pdf/2505.05599
• Copy Paste: [[2505.05599]] Enhancing Satellite Object Localization with Dilated Convolutions and Attention-aided Spatial Pooling(https://arxiv.org/abs/2505.05599)
• Keywords: robust
• Abstract: Object localization in satellite imagery is particularly challenging due to the high variability of objects, low spatial resolution, and interference from noise and dominant features such as clouds and city lights. In this research, we focus on three satellite datasets: upper atmospheric Gravity Waves (GW), mesospheric Bores (Bore), and Ocean Eddies (OE), each presenting its own unique challenges. These challenges include the variability in the scale and appearance of the main object patterns, where the size, shape, and feature extent of objects of interest can differ significantly. To address these challenges, we introduce YOLO-DCAP, a novel enhanced version of YOLOv5 designed to improve object localization in these complex scenarios. YOLO-DCAP incorporates a Multi-scale Dilated Residual Convolution (MDRC) block to capture multi-scale features at scale with varying dilation rates, and an Attention-aided Spatial Pooling (AaSP) module to focus on the global relevant spatial regions, enhancing feature selection. These structural improvements help to better localize objects in satellite imagery. Experimental results demonstrate that YOLO-DCAP significantly outperforms both the YOLO base model and state-of-the-art approaches, achieving an average improvement of 20.95% in mAP50 and 32.23% in IoU over the base model, and 7.35% and 9.84% respectively over state-of-the-art alternatives, consistently across all three satellite datasets. These consistent gains across all three satellite datasets highlight the robustness and generalizability of the proposed approach. Our code is open sourced at this https URL.

Title: On Corruption-Robustness in Performative Reinforcement Learning

• Authors: Vasilis Pollatos, Debmalya Mandal, Goran Radanovic
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05609
• Pdf URL: https://arxiv.org/pdf/2505.05609
• Copy Paste: [[2505.05609]] On Corruption-Robustness in Performative Reinforcement Learning(https://arxiv.org/abs/2505.05609)
• Keywords: robust
• Abstract: In performative Reinforcement Learning (RL), an agent faces a policy-dependent environment: the reward and transition functions depend on the agent's policy. Prior work on performative RL has studied the convergence of repeated retraining approaches to a performatively stable policy. In the finite sample regime, these approaches repeatedly solve for a saddle point of a convex-concave objective, which estimates the Lagrangian of a regularized version of the reinforcement learning problem. In this paper, we aim to extend such repeated retraining approaches, enabling them to operate under corrupted data. More specifically, we consider Huber's $\epsilon$-contamination model, where an $\epsilon$ fraction of data points is corrupted by arbitrary adversarial noise. We propose a repeated retraining approach based on convex-concave optimization under corrupted gradients and a novel problem-specific robust mean estimator for the gradients. We prove that our approach exhibits last-iterate convergence to an approximately stable policy, with the approximation error linear in $\sqrt{\epsilon}$. We experimentally demonstrate the importance of accounting for corruption in performative RL.

Title: LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities

• Authors: Kalyan Nakka, Jimmy Dani, Ausmit Mondal, Nitesh Saxena
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05619
• Pdf URL: https://arxiv.org/pdf/2505.05619
• Copy Paste: [[2505.05619]] LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities(https://arxiv.org/abs/2505.05619)
• Keywords: privacy, defense, attack, fair, large language model
• Abstract: The growing adoption of Large Language Models (LLMs) has influenced the development of their lighter counterparts-Small Language Models (SLMs)-to enable on-device deployment across smartphones and edge devices. These SLMs offer enhanced privacy, reduced latency, server-free functionality, and improved user experience. However, due to resource constraints of on-device environment, SLMs undergo size optimization through compression techniques like quantization, which can inadvertently introduce fairness, ethical and privacy risks. Critically, quantized SLMs may respond to harmful queries directly, without requiring adversarial manipulation, raising significant safety and trust concerns. To address this, we propose LiteLMGuard (LLMG), an on-device prompt guard that provides real-time, prompt-level defense for quantized SLMs. Additionally, our prompt guard is designed to be model-agnostic such that it can be seamlessly integrated with any SLM, operating independently of underlying architectures. Our LLMG formalizes prompt filtering as a deep learning (DL)-based prompt answerability classification task, leveraging semantic understanding to determine whether a query should be answered by any SLM. Using our curated dataset, Answerable-or-Not, we trained and fine-tuned several DL models and selected ELECTRA as the candidate, with 97.75% answerability classification accuracy. Our safety effectiveness evaluations demonstrate that LLMG defends against over 87% of harmful prompts, including both direct instruction and jailbreak attack strategies. We further showcase its ability to mitigate the Open Knowledge Attacks, where compromised SLMs provide unsafe responses without adversarial prompting. In terms of prompt filtering effectiveness, LLMG achieves near state-of-the-art filtering accuracy of 94%, with an average latency of 135 ms, incurring negligible overhead for users.

Title: SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimation

• Authors: Wenqing Peng, Zhi-Song Liu, Michael Boy
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05625
• Pdf URL: https://arxiv.org/pdf/2505.05625
• Copy Paste: [[2505.05625]] SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimation(https://arxiv.org/abs/2505.05625)
• Keywords: robust
• Abstract: Estimating rate constants from complex chemical reactions is essential for advancing detailed chemistry. However, the stiffness inherent in real-world atmospheric chemistry systems poses severe challenges, leading to training instability and poor convergence that hinder effective rate constant estimation using learning-based approaches. To address this, we propose a Stiff Physics-Informed Neural ODE framework (SPIN-ODE) for chemical reaction modelling. Our method introduces a three-stage optimisation process: first, a latent neural ODE learns the continuous and differentiable trajectory between chemical concentrations and their time derivatives; second, an explicit Chemical Reaction Neural Network (CRNN) extracts the underlying rate coefficients based on the learned dynamics; and third, fine-tune CRNN using a neural ODE solver to further improve rate coefficient estimation. Extensive experiments on both synthetic and newly proposed real-world datasets validate the effectiveness and robustness of our approach. As the first work on stiff Neural ODEs for chemical rate coefficient discovery, our study opens promising directions for integrating neural networks with detailed chemistry.

Title: Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models

• Authors: Aarti Ghatkesar, Uddeshya Upadhyay, Ganesh Venkatesh
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05626
• Pdf URL: https://arxiv.org/pdf/2505.05626
• Copy Paste: [[2505.05626]] Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models(https://arxiv.org/abs/2505.05626)
• Keywords: large language model
• Abstract: Achieving deep alignment between vision and language remains a central challenge for Multimodal Large Language Models (MLLMs). These models often fail to fully leverage visual input, defaulting to strong language priors. Our approach first provides insights into how MLLMs internally build visual understanding of image regions and then introduces techniques to amplify this capability. Specifically, we explore techniques designed both to deepen the model's understanding of visual content and to ensure that these visual insights actively guide language generation. We demonstrate the superior multimodal understanding of our resultant model through a detailed upstream analysis quantifying its ability to predict visually-dependent tokens as well as 10 pt boost on visually challenging tasks.

Title: Semantic Style Transfer for Enhancing Animal Facial Landmark Detection

• Authors: Anadil Hussein, Anna Zamansky, George Martvel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05640
• Pdf URL: https://arxiv.org/pdf/2505.05640
• Copy Paste: [[2505.05640]] Semantic Style Transfer for Enhancing Animal Facial Landmark Detection(https://arxiv.org/abs/2505.05640)
• Keywords: robust
• Abstract: Neural Style Transfer (NST) is a technique for applying the visual characteristics of one image onto another while preserving structural content. Traditionally used for artistic transformations, NST has recently been adapted, e.g., for domain adaptation and data augmentation. This study investigates the use of this technique for enhancing animal facial landmark detectors training. As a case study, we use a recently introduced Ensemble Landmark Detector for 48 anatomical cat facial landmarks and the CatFLW dataset it was trained on, making three main contributions. First, we demonstrate that applying style transfer to cropped facial images rather than full-body images enhances structural consistency, improving the quality of generated images. Secondly, replacing training images with style-transferred versions raised challenges of annotation misalignment, but Supervised Style Transfer (SST) - which selects style sources based on landmark accuracy - retained up to 98% of baseline accuracy. Finally, augmenting the dataset with style-transferred images further improved robustness, outperforming traditional augmentation methods. These findings establish semantic style transfer as an effective augmentation strategy for enhancing the performance of facial landmark detection models for animals and beyond. While this study focuses on cat facial landmarks, the proposed method can be generalized to other species and landmark detection models.

Title: The Moon's Many Faces: A Single Unified Transformer for Multimodal Lunar Reconstruction

• Authors: Tom Sander, Moritz Tenthoff, Kay Wohlfarth, Christian Wöhler
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05644
• Pdf URL: https://arxiv.org/pdf/2505.05644
• Copy Paste: [[2505.05644]] The Moon's Many Faces: A Single Unified Transformer for Multimodal Lunar Reconstruction(https://arxiv.org/abs/2505.05644)
• Keywords: transformer
• Abstract: Multimodal learning is an emerging research topic across multiple disciplines but has rarely been applied to planetary science. In this contribution, we identify that reflectance parameter estimation and image-based 3D reconstruction of lunar images can be formulated as a multimodal learning problem. We propose a single, unified transformer architecture trained to learn shared representations between multiple sources like grayscale images, digital elevation models, surface normals, and albedo maps. The architecture supports flexible translation from any input modality to any target modality. Predicting DEMs and albedo maps from grayscale images simultaneously solves the task of 3D reconstruction of planetary surfaces and disentangles photometric parameters and height information. Our results demonstrate that our foundation model learns physically plausible relations across these four modalities. Adding more input modalities in the future will enable tasks such as photometric normalization and co-registration.

Title: Privacy-Preserving Transformers: SwiftKey's Differential Privacy Implementation

• Authors: Abdelrahman Abouelenin, Mohamed Abdelrehim, Raffy Fahim, Amr Hendy, Mohamed Afify
• Subjects: cs.CL, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05648
• Pdf URL: https://arxiv.org/pdf/2505.05648
• Copy Paste: [[2505.05648]] Privacy-Preserving Transformers: SwiftKey's Differential Privacy Implementation(https://arxiv.org/abs/2505.05648)
• Keywords: privacy, transformer
• Abstract: In this paper we train a transformer using differential privacy (DP) for language modeling in SwiftKey. We run multiple experiments to balance the trade-off between the model size, run-time speed and accuracy. We show that we get small and consistent gains in the next-word-prediction and accuracy with graceful increase in memory and speed compared to the production GRU. This is obtained by scaling down a GPT2 architecture to fit the required size and a two stage training process that builds a seed model on general data and DP finetunes it on typing data. The transformer is integrated using ONNX offering both flexibility and efficiency.

Title: EquiHGNN: Scalable Rotationally Equivariant Hypergraph Neural Networks

• Authors: Tien Dang, Truong-Son Hy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05650
• Pdf URL: https://arxiv.org/pdf/2505.05650
• Copy Paste: [[2505.05650]] EquiHGNN: Scalable Rotationally Equivariant Hypergraph Neural Networks(https://arxiv.org/abs/2505.05650)
• Keywords: robust
• Abstract: Molecular interactions often involve high-order relationships that cannot be fully captured by traditional graph-based models limited to pairwise connections. Hypergraphs naturally extend graphs by enabling multi-way interactions, making them well-suited for modeling complex molecular systems. In this work, we introduce EquiHGNN, an Equivariant HyperGraph Neural Network framework that integrates symmetry-aware representations to improve molecular modeling. By enforcing the equivariance under relevant transformation groups, our approach preserves geometric and topological properties, leading to more robust and physically meaningful representations. We examine a range of equivariant architectures and demonstrate that integrating symmetry constraints leads to notable performance gains on large-scale molecular datasets. Experiments on both small and large molecules show that high-order interactions offer limited benefits for small molecules but consistently outperform 2D graphs on larger ones. Adding geometric features to these high-order structures further improves the performance, emphasizing the value of spatial information in molecular learning. Our source code is available at this https URL

Title: Invariant-Based Cryptography

• Authors: Stanislav Semenov
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05653
• Pdf URL: https://arxiv.org/pdf/2505.05653
• Copy Paste: [[2505.05653]] Invariant-Based Cryptography(https://arxiv.org/abs/2505.05653)
• Keywords: secure, security
• Abstract: We propose a new symmetric cryptographic scheme based on functional invariants defined over discrete oscillatory functions with hidden parameters. The scheme encodes a secret integer through a four-point algebraic identity preserved under controlled parameterization. Security arises not from algebraic inversion but from structural coherence: the transmitted values satisfy an invariant that is computationally hard to forge or invert without knowledge of the shared secret. We develop the full analytic and modular framework, prove exact identities, define index-recovery procedures, and analyze security assumptions, including oscillator construction, hash binding, and invertibility conditions. The result is a compact, self-verifying mechanism suitable for secure authentication, parameter exchange, and lightweight communication protocols.

Title: Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

• Authors: Alexander Most, Joseph Winjum, Ayan Biswas, Shawn Jones, Nishath Rajiv Ranasinghe, Dan O'Malley, Manish Bhattarai
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05666
• Pdf URL: https://arxiv.org/pdf/2505.05666
• Copy Paste: [[2505.05666]] Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval(https://arxiv.org/abs/2505.05666)
• Keywords: robust, large language model
• Abstract: Retrieval-Augmented Generation (RAG) has become a popular technique for enhancing the reliability and utility of Large Language Models (LLMs) by grounding responses in external documents. Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text. However, even state-of-the-art OCRs can introduce errors, especially in degraded or complex documents. Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR. This study presents a systematic comparison between a vision-based RAG system (ColPali) and more traditional OCR-based pipelines utilizing Llama 3.2 (90B) and Nougat OCR across varying document qualities. Beyond conventional retrieval accuracy metrics, we introduce a semantic answer evaluation benchmark to assess end-to-end question-answering performance. Our findings indicate that while vision-based RAG performs well on documents it has been fine-tuned on, OCR-based RAG is better able to generalize to unseen documents of varying quality. We highlight the key trade-offs between computational efficiency and semantic accuracy, offering practical guidance for RAG practitioners in selecting between OCR-dependent and vision-based document retrieval systems in production environments.

Title: InstanceGen: Image Generation with Instance-level Instructions

• Authors: Etai Sella, Yanir Kleiman, Hadar Averbuch-Elor
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05678
• Pdf URL: https://arxiv.org/pdf/2505.05678
• Copy Paste: [[2505.05678]] InstanceGen: Image Generation with Instance-level Instructions(https://arxiv.org/abs/2505.05678)
• Keywords: generative
• Abstract: Despite rapid advancements in the capabilities of generative models, pretrained text-to-image models still struggle in capturing the semantics conveyed by complex prompts that compound multiple objects and instance-level attributes. Consequently, we are witnessing growing interests in integrating additional structural constraints, %leveraging additional structural inputs typically in the form of coarse bounding boxes, to better guide the generation process in such challenging cases. In this work, we take the idea of structural guidance a step further by making the observation that contemporary image generation models can directly provide a plausible \emph{fine-grained} structural initialization. We propose a technique that couples this image-based structural guidance with LLM-based instance-level instructions, yielding output images that adhere to all parts of the text prompt, including object counts, instance-level attributes, and spatial relations between instances.

Title: Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos

• Authors: Giulio Cesare Mastrocinque Santo, Patrícia Izar, Irene Delval, Victor de Napole Gregolin, Nina S. T. Hirata
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05681
• Pdf URL: https://arxiv.org/pdf/2505.05681
• Copy Paste: [[2505.05681]] Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos(https://arxiv.org/abs/2505.05681)
• Keywords: large language model
• Abstract: Video recordings of nonhuman primates in their natural habitat are a common source for studying their behavior in the wild. We fine-tune pre-trained video-text foundational models for the specific domain of capuchin monkeys, with the goal of developing useful computational models to help researchers to retrieve useful clips from videos. We focus on the challenging problem of training a model based solely on raw, unlabeled video footage, using weak audio descriptions sometimes provided by field collaborators. We leverage recent advances in Multimodal Large Language Models (MLLMs) and Vision-Language Models (VLMs) to address the extremely noisy nature of both video and audio content. Specifically, we propose a two-folded approach: an agentic data treatment pipeline and a fine-tuning process. The data processing pipeline automatically extracts clean and semantically aligned video-text pairs from the raw videos, which are subsequently used to fine-tune a pre-trained Microsoft's X-CLIP model through Low-Rank Adaptation (LoRA). We obtained an uplift in $Hits@5$ of $167\%$ for the 16 frames model and an uplift of $114\%$ for the 8 frame model on our domain data. Moreover, based on $NDCG@K$ results, our model is able to rank well most of the considered behaviors, while the tested raw pre-trained models are not able to rank them at all. The code will be made available upon acceptance.

Title: Bringing Forensic Readiness to Modern Computer Firmware

• Authors: Tobias Latzo, Florian Hantke, Lukas Kotschi, Felix Freiling
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05697
• Pdf URL: https://arxiv.org/pdf/2505.05697
• Copy Paste: [[2505.05697]] Bringing Forensic Readiness to Modern Computer Firmware(https://arxiv.org/abs/2505.05697)
• Keywords: attack
• Abstract: Today's computer systems come with a pre-installed tiny operating system, which is also known as UEFI. UEFI has slowly displaced the former legacy PC-BIOS while the main task has not changed: It is responsible for booting the actual operating system. However, features like the network stack make it also useful for other applications. This paper introduces UEberForensIcs, a UEFI application that makes it easy to acquire memory from the firmware, similar to the well-known cold boot attacks. There is even UEFI code called by the operating system during runtime, and we demonstrate how to utilize this for forensic purposes.

Title: Hypergraph Neural Sheaf Diffusion: A Symmetric Simplicial Set Framework for Higher-Order Learning

• Authors: Seongjin Choi, Gahee Kim, Yong-Geun Oh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05702
• Pdf URL: https://arxiv.org/pdf/2505.05702
• Copy Paste: [[2505.05702]] Hypergraph Neural Sheaf Diffusion: A Symmetric Simplicial Set Framework for Higher-Order Learning(https://arxiv.org/abs/2505.05702)
• Keywords: diffusion
• Abstract: The absence of intrinsic adjacency relations and orientation systems in hypergraphs creates fundamental challenges for constructing sheaf Laplacians of arbitrary degrees. We resolve these limitations through symmetric simplicial sets derived directly from hypergraphs, which encode all possible oriented subrelations within each hyperedge as ordered tuples. This construction canonically defines adjacency via facet maps while inherently preserving hyperedge provenance. We establish that the normalized degree zero sheaf Laplacian on our induced symmetric simplicial set reduces exactly to the traditional graph normalized sheaf Laplacian when restricted to graphs, validating its mathematical consistency with prior graph-based sheaf theory. Furthermore, the induced structure preserves all structural information from the original hypergraph, ensuring that every multi-way relational detail is faithfully retained. Leveraging this framework, we introduce Hypergraph Neural Sheaf Diffusion (HNSD), the first principled extension of Neural Sheaf Diffusion (NSD) to hypergraphs. HNSD operates via normalized degree zero sheaf Laplacians over symmetric simplicial sets, resolving orientation ambiguity and adjacency sparsity inherent to hypergraph learning. Experimental evaluations demonstrate HNSD's competitive performance across established benchmarks.

Title: Assessing Robustness to Spurious Correlations in Post-Training Language Models

• Authors: Julia Shuieh, Prasann Singhal, Apaar Shanker, John Heyer, George Pu, Samuel Denton
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05704
• Pdf URL: https://arxiv.org/pdf/2505.05704
• Copy Paste: [[2505.05704]] Assessing Robustness to Spurious Correlations in Post-Training Language Models(https://arxiv.org/abs/2505.05704)
• Keywords: robust, large language model
• Abstract: Supervised and preference-based fine-tuning techniques have become popular for aligning large language models (LLMs) with user intent and correctness criteria. However, real-world training data often exhibits spurious correlations -- arising from biases, dataset artifacts, or other "shortcut" features -- that can compromise a model's performance or generalization. In this paper, we systematically evaluate three post-training algorithms -- Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and KTO (Kahneman-Tversky Optimization) -- across a diverse set of synthetic tasks and spuriousness conditions. Our tasks span mathematical reasoning, constrained instruction-following, and document-grounded question answering. We vary the degree of spurious correlation (10% vs. 90%) and investigate two forms of artifacts: "Feature Ambiguity" and "Distributional Narrowness." Our results show that the models often but not always degrade under higher spuriousness. The preference-based methods (DPO/KTO) can demonstrate relative robustness in mathematical reasoning tasks. By contrast, SFT maintains stronger performance in complex, context-intensive tasks. These findings highlight that no single post-training strategy universally outperforms in all scenarios; the best choice depends on the type of target task and the nature of spurious correlations.

Title: Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy

• Authors: Rushabh Solanki, Meghana Bhange, Ulrich Aïvodji, Elliot Creager
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05707
• Pdf URL: https://arxiv.org/pdf/2505.05707
• Copy Paste: [[2505.05707]] Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy(https://arxiv.org/abs/2505.05707)
• Keywords: privacy, protect
• Abstract: The integration of AI into daily life has generated considerable attention and excitement, while also raising concerns about automating algorithmic harms and re-entrenching existing social inequities. While the responsible deployment of trustworthy AI systems is a worthy goal, there are many possible ways to realize it, from policy and regulation to improved algorithm design and evaluation. In fact, since AI trains on social data, there is even a possibility for everyday users, citizens, or workers to directly steer its behavior through Algorithmic Collective Action, by deliberately modifying the data they share with a platform to drive its learning process in their favor. This paper considers how these grassroots efforts to influence AI interact with methods already used by AI firms and governments to improve model trustworthiness. In particular, we focus on the setting where the AI firm deploys a differentially private model, motivated by the growing regulatory focus on privacy and data protection. We investigate how the use of Differentially Private Stochastic Gradient Descent (DPSGD) affects the collective's ability to influence the learning process. Our findings show that while differential privacy contributes to the protection of individual data, it introduces challenges for effective algorithmic collective action. We characterize lower bounds on the success of algorithmic collective action under differential privacy as a function of the collective's size and the firm's privacy parameters, and verify these trends experimentally by simulating collective action during the training of deep neural network classifiers across several datasets.

Title: HyperspectralMAE: The Hyperspectral Imagery Classification Model using Fourier-Encoded Dual-Branch Masked Autoencoder

• Authors: Wooyoung Jeong, Hyun Jae Park, Seonghun Jeong, Jong Wook Jang, Tae Hoon Lim, Dae Seoung Kim
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05710
• Pdf URL: https://arxiv.org/pdf/2505.05710
• Copy Paste: [[2505.05710]] HyperspectralMAE: The Hyperspectral Imagery Classification Model using Fourier-Encoded Dual-Branch Masked Autoencoder(https://arxiv.org/abs/2505.05710)
• Keywords: robust, transformer
• Abstract: Hyperspectral imagery provides rich spectral detail but poses unique challenges because of its high dimensionality in both spatial and spectral domains. We propose \textit{HyperspectralMAE}, a Transformer-based foundation model for hyperspectral data that employs a \textit{dual masking} strategy: during pre-training we randomly occlude 50\% of spatial patches and 50\% of spectral bands. This forces the model to learn representations capable of reconstructing missing information across both dimensions. To encode spectral order, we introduce learnable harmonic Fourier positional embeddings based on wavelength. The reconstruction objective combines mean-squared error (MSE) with the spectral angle mapper (SAM) to balance pixel-level accuracy and spectral-shape fidelity. The resulting model contains about $1.8\times10^{8}$ parameters and produces 768-dimensional embeddings, giving it sufficient capacity for transfer learning. We pre-trained HyperspectralMAE on two large hyperspectral corpora -- NASA EO-1 Hyperion ($\sim$1\,600 scenes, $\sim$$3\times10^{11}$ pixel spectra) and DLR EnMAP Level-0 ($\sim$1\,300 scenes, $\sim$$3\times10^{11}$ pixel spectra) -- and fine-tuned it for land-cover classification on the Indian Pines benchmark. HyperspectralMAE achieves state-of-the-art transfer-learning accuracy on Indian Pines, confirming that masked dual-dimensional pre-training yields robust spectral-spatial representations. These results demonstrate that dual masking and wavelength-aware embeddings advance hyperspectral image reconstruction and downstream analysis.

Title: DiGIT: Multi-Dilated Gated Encoder and Central-Adjacent Region Integrated Decoder for Temporal Action Detection Transformer

• Authors: Ho-Joong Kim, Yearang Lee, Jung-Ho Hong, Seong-Whan Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05711
• Pdf URL: https://arxiv.org/pdf/2505.05711
• Copy Paste: [[2505.05711]] DiGIT: Multi-Dilated Gated Encoder and Central-Adjacent Region Integrated Decoder for Temporal Action Detection Transformer(https://arxiv.org/abs/2505.05711)
• Keywords: transformer
• Abstract: In this paper, we examine a key limitation in query-based detectors for temporal action detection (TAD), which arises from their direct adaptation of originally designed architectures for object detection. Despite the effectiveness of the existing models, they struggle to fully address the unique challenges of TAD, such as the redundancy in multi-scale features and the limited ability to capture sufficient temporal context. To address these issues, we propose a multi-dilated gated encoder and central-adjacent region integrated decoder for temporal action detection transformer (DiGIT). Our approach replaces the existing encoder that consists of multi-scale deformable attention and feedforward network with our multi-dilated gated encoder. Our proposed encoder reduces the redundant information caused by multi-level features while maintaining the ability to capture fine-grained and long-range temporal information. Furthermore, we introduce a central-adjacent region integrated decoder that leverages a more comprehensive sampling strategy for deformable cross-attention to capture the essential information. Extensive experiments demonstrate that DiGIT achieves state-of-the-art performance on THUMOS14, ActivityNet v1.3, and HACS-Segment. Code is available at: this https URL

Title: LLM-Text Watermarking based on Lagrange Interpolation

• Authors: Jarosław Janas, Paweł Morawiecki, Josef Pieprzyk
• Subjects: cs.CR, cs.IT
• Abstract URL: https://arxiv.org/abs/2505.05712
• Pdf URL: https://arxiv.org/pdf/2505.05712
• Copy Paste: [[2505.05712]] LLM-Text Watermarking based on Lagrange Interpolation(https://arxiv.org/abs/2505.05712)
• Keywords: secure, security, watermark, large language model
• Abstract: The rapid advancement of LLMs (Large Language Models) has established them as a foundational technology for many AI and ML powered human computer interactions. A critical challenge in this context is the attribution of LLM-generated text, either to the specific language model used or to the individual user who generated it. This is essential for combating misinformation, fake news, misinterpretation, and plagiarism. One of the key techniques for addressing this issue is watermarking. This work presents a watermarking scheme for LLM-generated text based on Lagrange interpolation, which enables the recovery of a secret author identity even when the text has been heavily redacted by an adversary. The core idea is to embed a continuous sequence of points (x, f(x)) that lie on a single straight line. The x-coordinates are generated pseudorandomly using either an LFSR (when security is not a priority) or a cryptographically secure NFSR for high-security applications. The scheme efficiency and resilience to adversarial modifications are analysed. Experimental results show that the proposed method is highly effective, allowing the recovery of the author identity when as few as three points survive adversarial manipulation.

Title: Semantic-Space-Intervened Diffusive Alignment for Visual Classification

• Authors: Zixuan Li, Lei Meng, Guoqing Chao, Wei Wu, Xiaoshuo Yan, Yimeng Yang, Zhuang Qi, Xiangxu Meng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05721
• Pdf URL: https://arxiv.org/pdf/2505.05721
• Copy Paste: [[2505.05721]] Semantic-Space-Intervened Diffusive Alignment for Visual Classification(https://arxiv.org/abs/2505.05721)
• Keywords: diffusion
• Abstract: Cross-modal alignment is an effective approach to improving visual classification. Existing studies typically enforce a one-step mapping that uses deep neural networks to project the visual features to mimic the distribution of textual features. However, they typically face difficulties in finding such a projection due to the two modalities in both the distribution of class-wise samples and the range of their feature values. To address this issue, this paper proposes a novel Semantic-Space-Intervened Diffusive Alignment method, termed SeDA, models a semantic space as a bridge in the visual-to-textual projection, considering both types of features share the same class-level information in classification. More importantly, a bi-stage diffusion framework is developed to enable the progressive alignment between the two modalities. Specifically, SeDA first employs a Diffusion-Controlled Semantic Learner to model the semantic features space of visual features by constraining the interactive features of the diffusion model and the category centers of visual features. In the later stage of SeDA, the Diffusion-Controlled Semantic Translator focuses on learning the distribution of textual features from the semantic space. Meanwhile, the Progressive Feature Interaction Network introduces stepwise feature interactions at each alignment step, progressively integrating textual information into mapped features. Experimental results show that SeDA achieves stronger cross-modal feature alignment, leading to superior performance over existing methods across multiple scenarios.

Title: You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation

• Authors: Valay Bundele, Mehran Hosseinzadeh, Hendrik Lensch
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05722
• Pdf URL: https://arxiv.org/pdf/2505.05722
• Copy Paste: [[2505.05722]] You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation(https://arxiv.org/abs/2505.05722)
• Keywords: robust
• Abstract: Synthetic datasets have enabled significant progress in point tracking by providing large-scale, densely annotated supervision. However, deploying these models in real-world domains remains challenging due to domain shift and lack of labeled data-issues that are especially severe in surgical videos, where scenes exhibit complex tissue deformation, occlusion, and lighting variation. While recent approaches adapt synthetic-trained trackers to natural videos using teacher ensembles or augmentation-heavy pseudo-labeling pipelines, their effectiveness in high-shift domains like surgery remains unexplored. This work presents SurgTracker, a semi-supervised framework for adapting synthetic-trained point trackers to surgical video using filtered self-distillation. Pseudo-labels are generated online by a fixed teacher-identical in architecture and initialization to the student-and are filtered using a cycle consistency constraint to discard temporally inconsistent trajectories. This simple yet effective design enforces geometric consistency and provides stable supervision throughout training, without the computational overhead of maintaining multiple teachers. Experiments on the STIR benchmark show that SurgTracker improves tracking performance using only 80 unlabeled videos, demonstrating its potential for robust adaptation in high-shift, data-scarce domains.

Title: Automated Learning of Semantic Embedding Representations for Diffusion Models

• Authors: Limai Jiang, Yunpeng Cai
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05732
• Pdf URL: https://arxiv.org/pdf/2505.05732
• Copy Paste: [[2505.05732]] Automated Learning of Semantic Embedding Representations for Diffusion Models(https://arxiv.org/abs/2505.05732)
• Keywords: diffusion, transformer, generative
• Abstract: Generative models capture the true distribution of data, yielding semantically rich representations. Denoising diffusion models (DDMs) exhibit superior generative capabilities, though efficient representation learning for them are lacking. In this work, we employ a multi-level denoising autoencoder framework to expand the representation capacity of DDMs, which introduces sequentially consistent Diffusion Transformers and an additional timestep-dependent encoder to acquire embedding representations on the denoising Markov chain through self-conditional diffusion learning. Intuitively, the encoder, conditioned on the entire diffusion process, compresses high-dimensional data into directional vectors in latent under different noise levels, facilitating the learning of image embeddings across all timesteps. To verify the semantic adequacy of embeddings generated through this approach, extensive experiments are conducted on various datasets, demonstrating that optimally learned embeddings by DDMs surpass state-of-the-art self-supervised representation learning methods in most cases, achieving remarkable discriminative semantic representation quality. Our work justifies that DDMs are not only suitable for generative tasks, but also potentially advantageous for general-purpose deep learning applications.

Title: Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering

• Authors: Yiming Niu, Jinliang Deng, Lulu Zhang, Zimu Zhou, Yongxin Tong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05738
• Pdf URL: https://arxiv.org/pdf/2505.05738
• Copy Paste: [[2505.05738]] Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering(https://arxiv.org/abs/2505.05738)
• Keywords: transformer
• Abstract: Accurate and efficient multivariate time series (MTS) forecasting is essential for applications such as traffic management and weather prediction, which depend on capturing long-range temporal dependencies and interactions between entities. Existing methods, particularly those based on Transformer architectures, compute pairwise dependencies across all time steps, leading to a computational complexity that scales quadratically with the length of the input. To overcome these challenges, we introduce the Forecaster with Offline Clustering Using Segments (FOCUS), a novel approach to MTS forecasting that simplifies long-range dependency modeling through the use of prototypes extracted via offline clustering. These prototypes encapsulate high-level events in the real-world system underlying the data, summarizing the key characteristics of similar time segments. In the online phase, FOCUS dynamically adapts these patterns to the current input and captures dependencies between the input segment and high-level events, enabling both accurate and efficient forecasting. By identifying prototypes during the offline clustering phase, FOCUS reduces the computational complexity of modeling long-range dependencies in the online phase to linear scaling. Extensive experiments across diverse benchmarks demonstrate that FOCUS achieves state-of-the-art accuracy while significantly reducing computational costs.

Title: Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification

• Authors: Ruxue Shi, Hengrui Gu, Xu Shen, Xin Wang
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05744
• Pdf URL: https://arxiv.org/pdf/2505.05744
• Copy Paste: [[2505.05744]] Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification(https://arxiv.org/abs/2505.05744)
• Keywords: interpretability, large language model
• Abstract: Large Language Models (LLMs) have shown remarkable ability in solving complex tasks, making them a promising tool for enhancing tabular learning. However, existing LLM-based methods suffer from high resource requirements, suboptimal demonstration selection, and limited interpretability, which largely hinder their prediction performance and application in the real world. To overcome these problems, we propose a novel in-context learning framework for tabular prediction. The core idea is to leverage the explanations generated by LLMs to guide a smaller, locally deployable Surrogate Language Model (SLM) to make interpretable tabular predictions. Specifically, our framework mainly involves three stages: (i) Post Hoc Explanation Generation, where LLMs are utilized to generate explanations for question-answer pairs in candidate demonstrations, providing insights into the reasoning behind the answer. (ii) Post Hoc Explanation-Guided Demonstrations Selection, which utilizes explanations generated by LLMs to guide the process of demonstration selection from candidate demonstrations. (iii) Post Hoc Explanation-Guided Interpretable SLM Prediction, which utilizes the demonstrations obtained in step (ii) as in-context and merges corresponding explanations as rationales to improve the performance of SLM and guide the model to generate interpretable outputs. Experimental results highlight the framework's effectiveness, with an average accuracy improvement of 5.31% across various tabular datasets in diverse domains.

Title: Efficient Full-Stack Private Federated Deep Learning with Post-Quantum Security

• Authors: Yiwei Zhang, Rouzbeh Behnia, Attila A. Yavuz, Reza Ebrahimi, Elisa Bertino
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05751
• Pdf URL: https://arxiv.org/pdf/2505.05751
• Copy Paste: [[2505.05751]] Efficient Full-Stack Private Federated Deep Learning with Post-Quantum Security(https://arxiv.org/abs/2505.05751)
• Keywords: secure, security, privacy, protect, attack, robust, federate
• Abstract: Federated learning (FL) enables collaborative model training while preserving user data privacy by keeping data local. Despite these advantages, FL remains vulnerable to privacy attacks on user updates and model parameters during training and deployment. Secure aggregation protocols have been proposed to protect user updates by encrypting them, but these methods often incur high computational costs and are not resistant to quantum computers. Additionally, differential privacy (DP) has been used to mitigate privacy leakages, but existing methods focus on secure aggregation or DP, neglecting their potential synergies. To address these gaps, we introduce Beskar, a novel framework that provides post-quantum secure aggregation, optimizes computational overhead for FL settings, and defines a comprehensive threat model that accounts for a wide spectrum of adversaries. We also integrate DP into different stages of FL training to enhance privacy protection in diverse scenarios. Our framework provides a detailed analysis of the trade-offs between security, performance, and model accuracy, representing the first thorough examination of secure aggregation protocols combined with various DP approaches for post-quantum secure FL. Beskar aims to address the pressing privacy and security issues FL while ensuring quantum-safety and robust performance.

Title: Automating Infrastructure Surveying: A Framework for Geometric Measurements and Compliance Assessment Using Point Cloud Data

• Authors: Amin Ghafourian, Andrew Lee, Dechen Gao, Tyler Beer, Kin Yen, Iman Soltani
• Subjects: cs.CV, cs.CY, cs.LG, cs.RO, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05752
• Pdf URL: https://arxiv.org/pdf/2505.05752
• Copy Paste: [[2505.05752]] Automating Infrastructure Surveying: A Framework for Geometric Measurements and Compliance Assessment Using Point Cloud Data(https://arxiv.org/abs/2505.05752)
• Keywords: robust, segmentation
• Abstract: Automation can play a prominent role in improving efficiency, accuracy, and scalability in infrastructure surveying and assessing construction and compliance standards. This paper presents a framework for automation of geometric measurements and compliance assessment using point cloud data. The proposed approach integrates deep learning-based detection and segmentation, in conjunction with geometric and signal processing techniques, to automate surveying tasks. As a proof of concept, we apply this framework to automatically evaluate the compliance of curb ramps with the Americans with Disabilities Act (ADA), demonstrating the utility of point cloud data in survey automation. The method leverages a newly collected, large annotated dataset of curb ramps, made publicly available as part of this work, to facilitate robust model training and evaluation. Experimental results, including comparison with manual field measurements of several ramps, validate the accuracy and reliability of the proposed method, highlighting its potential to significantly reduce manual effort and improve consistency in infrastructure assessment. Beyond ADA compliance, the proposed framework lays the groundwork for broader applications in infrastructure surveying and automated construction evaluation, promoting wider adoption of point cloud data in these domains. The annotated database, manual ramp survey data, and developed algorithms are publicly available on the project's GitHub page: this https URL.

Title: Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions

• Authors: Dhruvesh Patel, Aishwarya Sahoo, Avinash Amballa, Tahira Naseem, Tim G. J. Rudner, Andrew McCallum
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05755
• Pdf URL: https://arxiv.org/pdf/2505.05755
• Copy Paste: [[2505.05755]] Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions(https://arxiv.org/abs/2505.05755)
• Keywords: diffusion
• Abstract: Autoregressive models (ARMs), which predict subsequent tokens one-by-one ``from left to right,'' have achieved significant success across a wide range of sequence generation tasks. However, they struggle to accurately represent sequences that require satisfying sophisticated constraints or whose sequential dependencies are better addressed by out-of-order generation. Masked Diffusion Models (MDMs) address some of these limitations, but the process of unmasking multiple tokens simultaneously in MDMs can introduce incoherences, and MDMs cannot handle arbitrary infilling constraints when the number of tokens to be filled in is not known in advance. In this work, we introduce Insertion Language Models (ILMs), which learn to insert tokens at arbitrary positions in a sequence -- that is, they select jointly both the position and the vocabulary element to be inserted. By inserting tokens one at a time, ILMs can represent strong dependencies between tokens, and their ability to generate sequences in arbitrary order allows them to accurately model sequences where token dependencies do not follow a left-to-right sequential structure. To train ILMs, we propose a tailored network parameterization and use a simple denoising objective. Our empirical evaluation demonstrates that ILMs outperform both ARMs and MDMs on common planning tasks. Furthermore, we show that ILMs outperform MDMs and perform on par with ARMs in an unconditional text generation task while offering greater flexibility than MDMs in arbitrary-length text infilling.

Title: A review of advancements in low-light image enhancement using deep learning

• Authors: Fangxue Liu, Lei Fan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05759
• Pdf URL: https://arxiv.org/pdf/2505.05759
• Copy Paste: [[2505.05759]] A review of advancements in low-light image enhancement using deep learning(https://arxiv.org/abs/2505.05759)
• Keywords: segmentation
• Abstract: In low-light environments, the performance of computer vision algorithms often deteriorates significantly, adversely affecting key vision tasks such as segmentation, detection, and classification. With the rapid advancement of deep learning, its application to low-light image processing has attracted widespread attention and seen significant progress in recent years. However, there remains a lack of comprehensive surveys that systematically examine how recent deep-learning-based low-light image enhancement methods function and evaluate their effectiveness in enhancing downstream vison tasks. To address this gap, this review provides a detailed elaboration on how various recent approaches (from 2020) operate and their enhancement mechanisms, supplemented with clear illustrations. It also investigates the impact of different enhancement techniques on subsequent vision tasks, critically analyzing their strengths and limitations. Additionally, it proposes future research directions. This review serves as a useful reference for determining low-light image enhancement techniques and optimizing vision task performance in low-light conditions.

Title: Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM

• Authors: Zehao Fan, Garrett Gagnon, Zhenyu Liu, Liu Liu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05772
• Pdf URL: https://arxiv.org/pdf/2505.05772
• Copy Paste: [[2505.05772]] Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM(https://arxiv.org/abs/2505.05772)
• Keywords: transformer, large language model
• Abstract: Transformer-based models are the foundation of modern machine learning, but their execution, particularly during autoregressive decoding in large language models (LLMs), places significant pressure on memory systems due to frequent memory accesses and growing key-value (KV) caches. This creates a bottleneck in memory bandwidth, especially as context lengths increase. Processing-in-memory (PIM) architectures are a promising solution, offering high internal bandwidth and compute parallelism near memory. However, current PIM designs are primarily optimized for dense attention and struggle with the dynamic, irregular access patterns introduced by modern KV cache sparsity techniques. Consequently, they suffer from workload imbalance, reducing throughput and resource utilization. In this work, we propose STARC, a novel sparsity-optimized data mapping scheme tailored specifically for efficient LLM decoding on PIM architectures. STARC clusters KV pairs by semantic similarity and maps them to contiguous memory regions aligned with PIM bank structures. During decoding, queries retrieve relevant tokens at cluster granularity by matching against precomputed centroids, enabling selective attention and parallel processing without frequent reclustering or data movement overhead. Experiments on the HBM-PIM system show that, compared to common token-wise sparsity methods, STARC reduces attention-layer latency by 19%--31% and energy consumption by 19%--27%. Under a KV cache budget of 1024, it achieves up to 54%--74% latency reduction and 45%--67% energy reduction compared to full KV cache retrieval. Meanwhile, STARC maintains model accuracy comparable to state-of-the-art sparse attention methods, demonstrating its effectiveness in enabling efficient and hardware-friendly long-context LLM inference on PIM architectures.

Title: Improving Generalizability of Kolmogorov-Arnold Networks via Error-Correcting Output Codes

• Authors: Youngjoon Lee, Jinu Gong, Joonhyuk Kang
• Subjects: cs.LG, cs.CV, eess.IV, eess.SP
• Abstract URL: https://arxiv.org/abs/2505.05798
• Pdf URL: https://arxiv.org/pdf/2505.05798
• Copy Paste: [[2505.05798]] Improving Generalizability of Kolmogorov-Arnold Networks via Error-Correcting Output Codes(https://arxiv.org/abs/2505.05798)
• Keywords: robust
• Abstract: Kolmogorov-Arnold Networks (KAN) offer universal function approximation using univariate spline compositions without nonlinear activations. In this work, we integrate Error-Correcting Output Codes (ECOC) into the KAN framework to transform multi-class classification into multiple binary tasks, improving robustness via Hamming-distance decoding. Our proposed KAN with ECOC method outperforms vanilla KAN on a challenging blood cell classification dataset, achieving higher accuracy under diverse hyperparameter settings. Ablation studies further confirm that ECOC consistently enhances performance across FastKAN and FasterKAN variants. These results demonstrate that ECOC integration significantly boosts KAN generalizability in critical healthcare AI applications. To the best of our knowledge, this is the first integration of ECOC with KAN for enhancing multi-class medical image classification performance.

Title: Describe Anything in Medical Images

• Authors: Xi Xiao, Yunbei Zhang, Thanh-Huy Nguyen, Ba-Thinh Lam, Janet Wang, Jihun Hamm, Tianyang Wang, Xingjian Li, Xiao Wang, Hao Xu, Tianming Liu, Min Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05804
• Pdf URL: https://arxiv.org/pdf/2505.05804
• Copy Paste: [[2505.05804]] Describe Anything in Medical Images(https://arxiv.org/abs/2505.05804)
• Keywords: robust
• Abstract: Localized image captioning has made significant progress with models like the Describe Anything Model (DAM), which can generate detailed region-specific descriptions without explicit region-text supervision. However, such capabilities have yet to be widely applied to specialized domains like medical imaging, where diagnostic interpretation relies on subtle regional findings rather than global understanding. To mitigate this gap, we propose MedDAM, the first comprehensive framework leveraging large vision-language models for region-specific captioning in medical images. MedDAM employs medical expert-designed prompts tailored to specific imaging modalities and establishes a robust evaluation benchmark comprising a customized assessment protocol, data pre-processing pipeline, and specialized QA template library. This benchmark evaluates both MedDAM and other adaptable large vision-language models, focusing on clinical factuality through attribute-level verification tasks, thereby circumventing the absence of ground-truth region-caption pairs in medical datasets. Extensive experiments on the VinDr-CXR, LIDC-IDRI, and SkinCon datasets demonstrate MedDAM's superiority over leading peers (including GPT-4o, Claude 3.7 Sonnet, LLaMA-3.2 Vision, Qwen2.5-VL, GPT-4Rol, and OMG-LLaVA) in the task, revealing the importance of region-level semantic alignment in medical image understanding and establishing MedDAM as a promising foundation for clinical vision-language integration.

Title: Image Segmentation via Variational Model Based Tailored UNet: A Deep Variational Framework

• Authors: Kaili Qi, Wenli Yang, Ye Li, Zhongyi Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05806
• Pdf URL: https://arxiv.org/pdf/2505.05806
• Copy Paste: [[2505.05806]] Image Segmentation via Variational Model Based Tailored UNet: A Deep Variational Framework(https://arxiv.org/abs/2505.05806)
• Keywords: extraction, interpretability, segmentation
• Abstract: Traditional image segmentation methods, such as variational models based on partial differential equations (PDEs), offer strong mathematical interpretability and precise boundary modeling, but often suffer from sensitivity to parameter settings and high computational costs. In contrast, deep learning models such as UNet, which are relatively lightweight in parameters, excel in automatic feature extraction but lack theoretical interpretability and require extensive labeled data. To harness the complementary strengths of both paradigms, we propose Variational Model Based Tailored UNet (VM_TUNet), a novel hybrid framework that integrates the fourth-order modified Cahn-Hilliard equation with the deep learning backbone of UNet, which combines the interpretability and edge-preserving properties of variational methods with the adaptive feature learning of neural networks. Specifically, a data-driven operator is introduced to replace manual parameter tuning, and we incorporate the tailored finite point method (TFPM) to enforce high-precision boundary preservation. Experimental results on benchmark datasets demonstrate that VM_TUNet achieves superior segmentation performance compared to existing approaches, especially for fine boundary delineation.

Title: Intrusion Detection System Using Deep Learning for Network Security

• Authors: Soham Chatterjee, Satvik Chaudhary, Aswani Kumar Cherukuri
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05810
• Pdf URL: https://arxiv.org/pdf/2505.05810
• Copy Paste: [[2505.05810]] Intrusion Detection System Using Deep Learning for Network Security(https://arxiv.org/abs/2505.05810)
• Keywords: security, attack
• Abstract: As the number of cyberattacks and their particualr nature escalate, the need for effective intrusion detection systems (IDS) has become indispensable for ensuring the security of contemporary networks. Adaptive and more sophisticated threats are often beyond the reach of traditional approaches to intrusion detection and access control. This paper proposes an experimental evaluation of IDS models based on deep learning techniques, focusing on the classification of network traffic into malicious and benign categories. We analyze and retrain an assortment of architectures, such as Convolutional Neural Networks (CNN), Artificial Neural Networks (ANN), and LSTM models. Each model was tested based on a real dataset simulated in a multi-faceted and everchanging network traffic environment. Among the tested models, the best achieved an accuracy of 96 percent, underscoring the potential of deep learning models in improving efficiency and rapid response in IDS systems. The goal of the research is to demonstrate the effectiveness of distinct architectures and their corresponding trade-offs to enhance framework development for adaptive IDS solutions and improve overall network security.

Title: Tell Me Who Your Students Are: GPT Can Generate Valid Multiple-Choice Questions When Students' (Mis)Understanding Is Hinted

• Authors: Machi Shimmei, Masaki Uto, Yuichiroh Matsubayashi, Kentaro Inui, Aditi Mallavarapu, Noboru Matsuda
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05815
• Pdf URL: https://arxiv.org/pdf/2505.05815
• Copy Paste: [[2505.05815]] Tell Me Who Your Students Are: GPT Can Generate Valid Multiple-Choice Questions When Students' (Mis)Understanding Is Hinted(https://arxiv.org/abs/2505.05815)
• Keywords: large language model
• Abstract: The primary goal of this study is to develop and evaluate an innovative prompting technique, AnaQuest, for generating multiple-choice questions (MCQs) using a pre-trained large language model. In AnaQuest, the choice items are sentence-level assertions about complex concepts. The technique integrates formative and summative assessments. In the formative phase, students answer open-ended questions for target concepts in free text. For summative assessment, AnaQuest analyzes these responses to generate both correct and incorrect assertions. To evaluate the validity of the generated MCQs, Item Response Theory (IRT) was applied to compare item characteristics between MCQs generated by AnaQuest, a baseline ChatGPT prompt, and human-crafted items. An empirical study found that expert instructors rated MCQs generated by both AI models to be as valid as those created by human instructors. However, IRT-based analysis revealed that AnaQuest-generated questions - particularly those with incorrect assertions (foils) - more closely resembled human-crafted items in terms of difficulty and discrimination than those produced by ChatGPT.

Title: Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition

• Authors: Zhiyuan Chen, Keyi Li, Yifan Jia, Le Ye, Yufei Ma
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05829
• Pdf URL: https://arxiv.org/pdf/2505.05829
• Copy Paste: [[2505.05829]] Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition(https://arxiv.org/abs/2505.05829)
• Keywords: diffusion, transformer, generative
• Abstract: Diffusion transformer (DiT) models have achieved remarkable success in image generation, thanks for their exceptional generative capabilities and scalability. Nonetheless, the iterative nature of diffusion models (DMs) results in high computation complexity, posing challenges for deployment. Although existing cache-based acceleration methods try to utilize the inherent temporal similarity to skip redundant computations of DiT, the lack of correction may induce potential quality degradation. In this paper, we propose increment-calibrated caching, a training-free method for DiT acceleration, where the calibration parameters are generated from the pre-trained model itself with low-rank approximation. To deal with the possible correction failure arising from outlier activations, we introduce channel-aware Singular Value Decomposition (SVD), which further strengthens the calibration effect. Experimental results show that our method always achieve better performance than existing naive caching methods with a similar computation resource budget. When compared with 35-step DDIM, our method eliminates more than 45% computation and improves IS by 12 at the cost of less than 0.06 FID increase. Code is available at this https URL.

Title: Enhancing Noisy Functional Encryption for Privacy-Preserving Machine Learning

• Authors: Linda Scheu-Hachtel, Jasmin Zalonis
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05843
• Pdf URL: https://arxiv.org/pdf/2505.05843
• Copy Paste: [[2505.05843]] Enhancing Noisy Functional Encryption for Privacy-Preserving Machine Learning(https://arxiv.org/abs/2505.05843)
• Keywords: security, privacy, protect
• Abstract: Functional encryption (FE) has recently attracted interest in privacy-preserving machine learning (PPML) for its unique ability to compute specific functions on encrypted data. A related line of work focuses on noisy FE, which ensures differential privacy in the output while keeping the data encrypted. We extend the notion of noisy multi-input functional encryption (NMIFE) to (dynamic) noisy multi-client functional encryption ((Dy)NMCFE), which allows for more flexibility in the number of data holders and analyses, while protecting the privacy of the data holder with fine-grained access through the usage of labels. Following our new definition of DyNMCFE, we present DyNo, a concrete inner-product DyNMCFE scheme. Our scheme captures all the functionalities previously introduced in noisy FE schemes, while being significantly more efficient in terms of space and runtime and fulfilling a stronger security notion by allowing the corruption of clients. To further prove the applicability of DyNMCFE, we present a protocol for PPML based on DyNo. According to this protocol, we train a privacy-preserving logistic regression.

Title: Automated Knot Detection and Pairing for Wood Analysis in the Timber Industry

• Authors: Guohao Lin, Shidong Pan, Rasul Khanbayov, Changxi Yang, Ani Khaloian-Sarnaghi, Andriy Kovryga
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05845
• Pdf URL: https://arxiv.org/pdf/2505.05845
• Copy Paste: [[2505.05845]] Automated Knot Detection and Pairing for Wood Analysis in the Timber Industry(https://arxiv.org/abs/2505.05845)
• Keywords: extraction
• Abstract: Knots in wood are critical to both aesthetics and structural integrity, making their detection and pairing essential in timber processing. However, traditional manual annotation was labor-intensive and inefficient, necessitating automation. This paper proposes a lightweight and fully automated pipeline for knot detection and pairing based on machine learning techniques. In the detection stage, high-resolution surface images of wooden boards were collected using industrial-grade cameras, and a large-scale dataset was manually annotated and preprocessed. After the transfer learning, the YOLOv8l achieves an mAP@0.5 of 0.887. In the pairing stage, detected knots were analyzed and paired based on multidimensional feature extraction. A triplet neural network was used to map the features into a latent space, enabling clustering algorithms to identify and pair corresponding knots. The triplet network with learnable weights achieved a pairing accuracy of 0.85. Further analysis revealed that he distances from the knot's start and end points to the bottom of the wooden board, and the longitudinal coordinates play crucial roles in achieving high pairing accuracy. Our experiments validate the effectiveness of the proposed solution, demonstrating the potential of AI in advancing wood science and industry.

Title: AgentXploit: End-to-End Redteaming of Black-Box AI Agents

• Authors: Zhun Wang, Vincent Siu, Zhe Ye, Tianneng Shi, Yuzhou Nie, Xuandong Zhao, Chenguang Wang, Wenbo Guo, Dawn Song
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05849
• Pdf URL: https://arxiv.org/pdf/2505.05849
• Copy Paste: [[2505.05849]] AgentXploit: End-to-End Redteaming of Black-Box AI Agents(https://arxiv.org/abs/2505.05849)
• Keywords: security, defense, attack, large language model
• Abstract: The strong planning and reasoning capabilities of Large Language Models (LLMs) have fostered the development of agent-based systems capable of leveraging external tools and interacting with increasingly complex environments. However, these powerful features also introduce a critical security risk: indirect prompt injection, a sophisticated attack vector that compromises the core of these agents, the LLM, by manipulating contextual information rather than direct user prompts. In this work, we propose a generic black-box fuzzing framework, AgentXploit, designed to automatically discover and exploit indirect prompt injection vulnerabilities across diverse LLM agents. Our approach starts by constructing a high-quality initial seed corpus, then employs a seed selection algorithm based on Monte Carlo Tree Search (MCTS) to iteratively refine inputs, thereby maximizing the likelihood of uncovering agent weaknesses. We evaluate AgentXploit on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o, respectively, nearly doubling the performance of baseline attacks. Moreover, AgentXploit exhibits strong transferability across unseen tasks and internal LLMs, as well as promising results against defenses. Beyond benchmark evaluations, we apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.

Title: PICD: Versatile Perceptual Image Compression with Diffusion Rendering

• Authors: Tongda Xu, Jiahao Li, Bin Li, Yan Wang, Ya-Qin Zhang, Yan Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05853
• Pdf URL: https://arxiv.org/pdf/2505.05853
• Copy Paste: [[2505.05853]] PICD: Versatile Perceptual Image Compression with Diffusion Rendering(https://arxiv.org/abs/2505.05853)
• Keywords: diffusion
• Abstract: Recently, perceptual image compression has achieved significant advancements, delivering high visual quality at low bitrates for natural images. However, for screen content, existing methods often produce noticeable artifacts when compressing text. To tackle this challenge, we propose versatile perceptual screen image compression with diffusion rendering (PICD), a codec that works well for both screen and natural images. More specifically, we propose a compression framework that encodes the text and image separately, and renders them into one image using diffusion model. For this diffusion rendering, we integrate conditional information into diffusion models at three distinct levels: 1). Domain level: We fine-tune the base diffusion model using text content prompts with screen content. 2). Adaptor level: We develop an efficient adaptor to control the diffusion model using compressed image and text as input. 3). Instance level: We apply instance-wise guidance to further enhance the decoding process. Empirically, our PICD surpasses existing perceptual codecs in terms of both text accuracy and perceptual quality. Additionally, without text conditions, our approach serves effectively as a perceptual codec for natural images.

Title: Decoupling Multi-Contrast Super-Resolution: Pairing Unpaired Synthesis with Implicit Representations

• Authors: Hongyu Rui, Yinzhe Wu, Fanwen Wang, Jiahao Huang, Liutao Yang, Zi Wang, Guang Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05855
• Pdf URL: https://arxiv.org/pdf/2505.05855
• Copy Paste: [[2505.05855]] Decoupling Multi-Contrast Super-Resolution: Pairing Unpaired Synthesis with Implicit Representations(https://arxiv.org/abs/2505.05855)
• Keywords: extraction, diffusion
• Abstract: Magnetic Resonance Imaging (MRI) is critical for clinical diagnostics but is often limited by long acquisition times and low signal-to-noise ratios, especially in modalities like diffusion and functional MRI. The multi-contrast nature of MRI presents a valuable opportunity for cross-modal enhancement, where high-resolution (HR) modalities can serve as references to boost the quality of their low-resolution (LR) counterparts-motivating the development of Multi-Contrast Super-Resolution (MCSR) techniques. Prior work has shown that leveraging complementary contrasts can improve SR performance; however, effective feature extraction and fusion across modalities with varying resolutions remains a major challenge. Moreover, existing MCSR methods often assume fixed resolution settings and all require large, perfectly paired training datasets-conditions rarely met in real-world clinical environments. To address these challenges, we propose a novel Modular Multi-Contrast Super-Resolution (MCSR) framework that eliminates the need for paired training data and supports arbitrary upscaling. Our method decouples the MCSR task into two stages: (1) Unpaired Cross-Modal Synthesis (U-CMS), which translates a high-resolution reference modality into a synthesized version of the target contrast, and (2) Unsupervised Super-Resolution (U-SR), which reconstructs the final output using implicit neural representations (INRs) conditioned on spatial coordinates. This design enables scale-agnostic and anatomically faithful reconstruction by bridging un-paired cross-modal synthesis with unsupervised resolution enhancement. Experiments show that our method achieves superior performance at 4x and 8x upscaling, with improved fidelity and anatomical consistency over existing baselines. Our framework demonstrates strong potential for scalable, subject-specific, and data-efficient MCSR in real-world clinical settings.

Title: Mixed-Integer Optimization for Responsible Machine Learning

• Authors: Nathan Justin, Qingshi Sun, Andrés Gómez, Phebe Vayanos
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.05857
• Pdf URL: https://arxiv.org/pdf/2505.05857
• Copy Paste: [[2505.05857]] Mixed-Integer Optimization for Responsible Machine Learning(https://arxiv.org/abs/2505.05857)
• Keywords: privacy, robust, fair
• Abstract: In the last few decades, Machine Learning (ML) has achieved significant success across domains ranging from healthcare, sustainability, and the social sciences, to criminal justice and finance. But its deployment in increasingly sophisticated, critical, and sensitive areas affecting individuals, the groups they belong to, and society as a whole raises critical concerns around fairness, transparency, robustness, and privacy, among others. As the complexity and scale of ML systems and of the settings in which they are deployed grow, so does the need for responsible ML methods that address these challenges while providing guaranteed performance in deployment. Mixed-integer optimization (MIO) offers a powerful framework for embedding responsible ML considerations directly into the learning process while maintaining performance. For example, it enables learning of inherently transparent models that can conveniently incorporate fairness or other domain specific constraints. This tutorial paper provides an accessible and comprehensive introduction to this topic discussing both theoretical and practical aspects. It outlines some of the core principles of responsible ML, their importance in applications, and the practical utility of MIO for building ML models that align with these principles. Through examples and mathematical formulations, it illustrates practical strategies and available tools for efficiently solving MIO problems for responsible ML. It concludes with a discussion on current limitations and open research questions, providing suggestions for future work.

Title: Symbol-based entity marker highlighting for enhanced text mining in materials science with generative AI

• Authors: Junhyeong Lee, Jong Min Yuk, Chan-Woo Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05864
• Pdf URL: https://arxiv.org/pdf/2505.05864
• Copy Paste: [[2505.05864]] Symbol-based entity marker highlighting for enhanced text mining in materials science with generative AI(https://arxiv.org/abs/2505.05864)
• Keywords: extraction, generative
• Abstract: The construction of experimental datasets is essential for expanding the scope of data-driven scientific discovery. Recent advances in natural language processing (NLP) have facilitated automatic extraction of structured data from unstructured scientific literature. While existing approaches-multi-step and direct methods-offer valuable capabilities, they also come with limitations when applied independently. Here, we propose a novel hybrid text-mining framework that integrates the advantages of both methods to convert unstructured scientific text into structured data. Our approach first transforms raw text into entity-recognized text, and subsequently into structured form. Furthermore, beyond the overall data structuring framework, we also enhance entity recognition performance by introducing an entity marker-a simple yet effective technique that uses symbolic annotations to highlight target entities. Specifically, our entity marker-based hybrid approach not only consistently outperforms previous entity recognition approaches across three benchmark datasets (MatScholar, SOFC, and SOFC slot NER) but also improve the quality of final structured data-yielding up to a 58% improvement in entity-level F1 score and up to 83% improvement in relation-level F1 score compared to direct approach.

Title: Generative Discovery of Partial Differential Equations by Learning from Math Handbooks

• Authors: Hao Xu, Yuntian Chen, Rui Cao, Tianning Tang, Mengge Du, Jian Li, Adrian H. Callaghan, Dongxiao Zhang
• Subjects: cs.LG, cs.AI, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2505.05869
• Pdf URL: https://arxiv.org/pdf/2505.05869
• Copy Paste: [[2505.05869]] Generative Discovery of Partial Differential Equations by Learning from Math Handbooks(https://arxiv.org/abs/2505.05869)
• Keywords: generative
• Abstract: Data driven discovery of partial differential equations (PDEs) is a promising approach for uncovering the underlying laws governing complex systems. However, purely data driven techniques face the dilemma of balancing search space with optimization efficiency. This study introduces a knowledge guided approach that incorporates existing PDEs documented in a mathematical handbook to facilitate the discovery process. These PDEs are encoded as sentence like structures composed of operators and basic terms, and used to train a generative model, called EqGPT, which enables the generation of free form PDEs. A loop of generation evaluation optimization is constructed to autonomously identify the most suitable PDE. Experimental results demonstrate that this framework can recover a variety of PDE forms with high accuracy and computational efficiency, particularly in cases involving complex temporal derivatives or intricate spatial terms, which are often beyond the reach of conventional methods. The approach also exhibits generalizability to irregular spatial domains and higher dimensional settings. Notably, it succeeds in discovering a previously unreported PDE governing strongly nonlinear surface gravity waves propagating toward breaking, based on real world experimental data, highlighting its applicability to practical scenarios and its potential to support scientific discovery.

Title: Towards Facial Image Compression with Consistency Preserving Diffusion Prior

• Authors: Yimin Zhou, Yichong Xia, Bin Chen, Baoyi An, Haoqian Wang, Zhi Wang, Yaowei Wang, Zikun Zhou
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.05870
• Pdf URL: https://arxiv.org/pdf/2505.05870
• Copy Paste: [[2505.05870]] Towards Facial Image Compression with Consistency Preserving Diffusion Prior(https://arxiv.org/abs/2505.05870)
• Keywords: robust, diffusion
• Abstract: With the widespread application of facial image data across various domains, the efficient storage and transmission of facial images has garnered significant attention. However, the existing learned face image compression methods often produce unsatisfactory reconstructed image quality at low bit rates. Simply adapting diffusion-based compression methods to facial compression tasks results in reconstructed images that perform poorly in downstream applications due to insufficient preservation of high-frequency information. To further explore the diffusion prior in facial image compression, we propose Facial Image Compression with a Stable Diffusion Prior (FaSDiff), a method that preserves consistency through frequency enhancement. FaSDiff employs a high-frequency-sensitive compressor in an end-to-end framework to capture fine image details and produce robust visual prompts. Additionally, we introduce a hybrid low-frequency enhancement module that disentangles low-frequency facial semantics and stably modulates the diffusion prior alongside visual prompts. The proposed modules allow FaSDiff to leverage diffusion priors for superior human visual perception while minimizing performance loss in machine vision due to semantic inconsistency. Extensive experiments show that FaSDiff outperforms state-of-the-art methods in balancing human visual quality and machine vision accuracy. The code will be released after the paper is accepted.

Title: A Taxonomy of Attacks and Defenses in Split Learning

• Authors: Aqsa Shabbir, Halil İbrahim Kanpak, Alptekin Küpçü, Sinem Sav
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05872
• Pdf URL: https://arxiv.org/pdf/2505.05872
• Copy Paste: [[2505.05872]] A Taxonomy of Attacks and Defenses in Split Learning(https://arxiv.org/abs/2505.05872)
• Keywords: security, privacy, defense, attack
• Abstract: Split Learning (SL) has emerged as a promising paradigm for distributed deep learning, allowing resource-constrained clients to offload portions of their model computation to servers while maintaining collaborative learning. However, recent research has demonstrated that SL remains vulnerable to a range of privacy and security threats, including information leakage, model inversion, and adversarial attacks. While various defense mechanisms have been proposed, a systematic understanding of the attack landscape and corresponding countermeasures is still lacking. In this study, we present a comprehensive taxonomy of attacks and defenses in SL, categorizing them along three key dimensions: employed strategies, constraints, and effectiveness. Furthermore, we identify key open challenges and research gaps in SL based on our systematization, highlighting potential future directions.

Title: A 3D pocket-aware and evolutionary conserved interaction guided diffusion model for molecular optimization

• Authors: Anjie Qiao, Hao Zhang, Qianmu Yuan, Qirui Deng, Jingtian Su, Weifeng Huang, Huihao Zhou, Guo-Bo Li, Zhen Wang, Jinping Lei
• Subjects: cs.LG, physics.chem-ph, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2505.05874
• Pdf URL: https://arxiv.org/pdf/2505.05874
• Copy Paste: [[2505.05874]] A 3D pocket-aware and evolutionary conserved interaction guided diffusion model for molecular optimization(https://arxiv.org/abs/2505.05874)
• Keywords: diffusion
• Abstract: Generating molecules that bind to specific protein targets via diffusion models has shown good promise for structure-based drug design and molecule optimization. Especially, the diffusion models with binding interaction guidance enables molecule generation with high affinity through forming favorable interaction within protein pocket. However, the generated molecules may not form interactions with the highly conserved residues, which are important for protein functions and bioactivities of the ligands. Herein, we developed a new 3D target-aware diffusion model DiffDecip, which explicitly incorporates the protein-ligand binding interactions and evolutionary conservation information of protein residues into both diffusion and sampling process, for molecule optimization through scaffold decoration. The model performance revealed that DiffDecip outperforms baseline model DiffDec on molecule optimization towards higher affinity through forming more non-covalent interactions with highly conserved residues in the protein pocket.

Title: Multi-Modal Molecular Representation Learning via Structure Awareness

• Authors: Rong Yin, Ruyue Liu, Xiaoshuai Hao, Xingrui Zhou, Yong Liu, Can Ma, Weiping Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.05877
• Pdf URL: https://arxiv.org/pdf/2505.05877
• Copy Paste: [[2505.05877]] Multi-Modal Molecular Representation Learning via Structure Awareness(https://arxiv.org/abs/2505.05877)
• Keywords: extraction
• Abstract: Accurate extraction of molecular representations is a critical step in the drug discovery process. In recent years, significant progress has been made in molecular representation learning methods, among which multi-modal molecular representation methods based on images, and 2D/3D topologies have become increasingly mainstream. However, existing these multi-modal approaches often directly fuse information from different modalities, overlooking the potential of intermodal interactions and failing to adequately capture the complex higher-order relationships and invariant features between molecules. To overcome these challenges, we propose a structure-awareness-based multi-modal self-supervised molecular representation pre-training framework (MMSA) designed to enhance molecular graph representations by leveraging invariant knowledge between molecules. The framework consists of two main modules: the multi-modal molecular representation learning module and the structure-awareness module. The multi-modal molecular representation learning module collaboratively processes information from different modalities of the same molecule to overcome intermodal differences and generate a unified molecular embedding. Subsequently, the structure-awareness module enhances the molecular representation by constructing a hypergraph structure to model higher-order correlations between molecules. This module also introduces a memory mechanism for storing typical molecular representations, aligning them with memory anchors in the memory bank to integrate invariant knowledge, thereby improving the model generalization ability. Extensive experiments have demonstrated the effectiveness of MMSA, which achieves state-of-the-art performance on the MoleculeNet benchmark, with average ROC-AUC improvements ranging from 1.8% to 9.6% over baseline methods.

Title: Register and CLS tokens yield a decoupling of local and global features in large ViTs

• Authors: Alexander Lappe, Martin A. Giese
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05892
• Pdf URL: https://arxiv.org/pdf/2505.05892
• Copy Paste: [[2505.05892]] Register and CLS tokens yield a decoupling of local and global features in large ViTs(https://arxiv.org/abs/2505.05892)
• Keywords: interpretability
• Abstract: Recent work has shown that the attention maps of the widely popular DINOv2 model exhibit artifacts, which hurt both model interpretability and performance on dense image tasks. These artifacts emerge due to the model repurposing patch tokens with redundant local information for the storage of global image information. To address this problem, additional register tokens have been incorporated in which the model can store such information instead. We carefully examine the influence of these register tokens on the relationship between global and local image features, showing that while register tokens yield cleaner attention maps, these maps do not accurately reflect the integration of local image information in large models. Instead, global information is dominated by information extracted from register tokens, leading to a disconnect between local and global features. Inspired by these findings, we show that the CLS token itself, which can be interpreted as a register, leads to a very similar phenomenon in models without explicit register tokens. Our work shows that care must be taken when interpreting attention maps of large ViTs. Further, by clearly attributing the faulty behaviour to register and CLS tokens, we show a path towards more interpretable vision models.

Title: Exploring the Susceptibility to Fraud of Monetary Incentive Mechanisms for Strengthening FOSS Projects

• Authors: Ben Swierzy, Timo Pohl, Marc Ohm, Michael Meier
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2505.05897
• Pdf URL: https://arxiv.org/pdf/2505.05897
• Copy Paste: [[2505.05897]] Exploring the Susceptibility to Fraud of Monetary Incentive Mechanisms for Strengthening FOSS Projects(https://arxiv.org/abs/2505.05897)
• Keywords: security, attack
• Abstract: Free and open source software (FOSS) is ubiquitous on modern IT systems, accelerating the speed of software engineering over the past decades. With its increasing importance and historical reliance on uncompensated contributions, questions have been raised regarding the continuous maintenance of FOSS and its implications from a security perspective. In recent years, different funding programs have emerged to provide external incentives to reinforce community FOSS' sustainability. Past research primarily focused on analyses what type of projects have been funded and for what reasons. However, it has neither been considered whether there is a need for such external incentives, nor whether the incentive mechanisms, especially with the development of decentralized approaches, are susceptible to fraud. In this study, we explore the need for funding through a literature review and compare the susceptibility to fraud of centralized and decentralized incentive programs by performing case studies on the Sovereign Tech Fund (STF) and the tea project. We find non-commercial incentives to fill an important gap, ensuring longevity and sustainability of projects. Furthermore, we find the STF to be able to achieve a high resilience against fraud attempts, while tea is highly susceptible to fraud, as evidenced by revelation of an associated sybil attack on npm. Our results imply that special considerations must be taken into account when utilizing quantitative repository metrics regardless whether spoofing is expected.

Title: DFEN: Dual Feature Equalization Network for Medical Image Segmentation

• Authors: Jianjian Yin, Yi Chen, Chengyu Li, Zhichao Zheng, Yanhui Gu, Junsheng Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.05913
• Pdf URL: https://arxiv.org/pdf/2505.05913
• Copy Paste: [[2505.05913]] DFEN: Dual Feature Equalization Network for Medical Image Segmentation(https://arxiv.org/abs/2505.05913)
• Keywords: transformer, segmentation
• Abstract: Current methods for medical image segmentation primarily focus on extracting contextual feature information from the perspective of the whole image. While these methods have shown effective performance, none of them take into account the fact that pixels at the boundary and regions with a low number of class pixels capture more contextual feature information from other classes, leading to misclassification of pixels by unequal contextual feature information. In this paper, we propose a dual feature equalization network based on the hybrid architecture of Swin Transformer and Convolutional Neural Network, aiming to augment the pixel feature representations by image-level equalization feature information and class-level equalization feature information. Firstly, the image-level feature equalization module is designed to equalize the contextual information of pixels within the image. Secondly, we aggregate regions of the same class to equalize the pixel feature representations of the corresponding class by class-level feature equalization module. Finally, the pixel feature representations are enhanced by learning weights for image-level equalization feature information and class-level equalization feature information. In addition, Swin Transformer is utilized as both the encoder and decoder, thereby bolstering the ability of the model to capture long-range dependencies and spatial correlations. We conducted extensive experiments on Breast Ultrasound Images (BUSI), International Skin Imaging Collaboration (ISIC2017), Automated Cardiac Diagnosis Challenge (ACDC) and PH$^2$ datasets. The experimental results demonstrate that our method have achieved state-of-the-art performance. Our code is publicly available at this https URL.

Title: Privacy-Preserving Credit Card Approval Using Homomorphic SVM: Toward Secure Inference in FinTech Applications

• Authors: Faneela, Baraq Ghaleb, Jawad Ahmad, William J. Buchanan, Sana Ullah Jan
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05920
• Pdf URL: https://arxiv.org/pdf/2505.05920
• Copy Paste: [[2505.05920]] Privacy-Preserving Credit Card Approval Using Homomorphic SVM: Toward Secure Inference in FinTech Applications(https://arxiv.org/abs/2505.05920)
• Keywords: secure, security, privacy, robust
• Abstract: The growing use of machine learning in cloud environments raises critical concerns about data security and privacy, especially in finance. Fully Homomorphic Encryption (FHE) offers a solution by enabling computations on encrypted data, but its high computational cost limits practicality. In this paper, we propose PP-FinTech, a privacy-preserving scheme for financial applications that employs a CKKS-based encrypted soft-margin SVM, enhanced with a hybrid kernel for modeling non-linear patterns and an adaptive thresholding mechanism for robust encrypted classification. Experiments on the Credit Card Approval dataset demonstrate comparable performance to the plaintext models, highlighting PP-FinTech's ability to balance privacy, and efficiency in secure financial ML systems.

Title: CAPE: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

• Authors: Haoqi Wu, Wei Dai, Li Wang, Qiang Yan
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05922
• Pdf URL: https://arxiv.org/pdf/2505.05922
• Copy Paste: [[2505.05922]] CAPE: Context-Aware Prompt Perturbation Mechanism with Differential Privacy(https://arxiv.org/abs/2505.05922)
• Keywords: privacy, large language model
• Abstract: Large Language Models (LLMs) have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen. Existing solutions primarily rely on privacy-enhancing technologies to mitigate such risks, facing the trade-off among efficiency, privacy, and utility. To narrow this gap, we propose Cape, a context-aware prompt perturbation mechanism based on differential privacy, to enable efficient inference with an improved privacy-utility trade-off. Concretely, we introduce a hybrid utility function that better captures the token similarity. Additionally, we propose a bucketized sampling mechanism to handle large sampling space, which might lead to long-tail phenomenons. Extensive experiments across multiple datasets, along with ablation studies, demonstrate that Cape achieves a better privacy-utility trade-off compared to prior state-of-the-art works.

Title: Autoencoder-Based Hybrid Replay for Class-Incremental Learning

• Authors: Milad Khademi Nori, Il-Min Kim, Guanghui Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05926
• Pdf URL: https://arxiv.org/pdf/2505.05926
• Copy Paste: [[2505.05926]] Autoencoder-Based Hybrid Replay for Class-Incremental Learning(https://arxiv.org/abs/2505.05926)
• Keywords: generative
• Abstract: In class-incremental learning (CIL), effective incremental learning strategies are essential to mitigate task confusion and catastrophic forgetting, especially as the number of tasks $t$ increases. Current exemplar replay strategies impose $\mathcal{O}(t)$ memory/compute complexities. We propose an autoencoder-based hybrid replay (AHR) strategy that leverages our new hybrid autoencoder (HAE) to function as a compressor to alleviate the requirement for large memory, achieving $\mathcal{O}(0.1 t)$ at the worst case with the computing complexity of $\mathcal{O}(t)$ while accomplishing state-of-the-art performance. The decoder later recovers the exemplar data stored in the latent space, rather than in raw format. Additionally, HAE is designed for both discriminative and generative modeling, enabling classification and replay capabilities, respectively. HAE adopts the charged particle system energy minimization equations and repulsive force algorithm for the incremental embedding and distribution of new class centroids in its latent space. Our results demonstrate that AHR consistently outperforms recent baselines across multiple benchmarks while operating with the same memory/compute budgets. The source code is included in the supplementary material and will be open-sourced upon publication.

Title: Cryptanalysis of a Lattice-Based PIR Scheme for Arbitrary Database Sizes

• Authors: Svenja Lage
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05934
• Pdf URL: https://arxiv.org/pdf/2505.05934
• Copy Paste: [[2505.05934]] Cryptanalysis of a Lattice-Based PIR Scheme for Arbitrary Database Sizes(https://arxiv.org/abs/2505.05934)
• Keywords: secure, security, privacy, attack
• Abstract: Private Information Retrieval (PIR) schemes enable users to securely retrieve files from a server without disclosing the content of their queries, thereby preserving their privacy. In 2008, Melchor and Gaborit proposed a PIR scheme that achieves a balance between communication overhead and server-side computational cost. However, for particularly small databases, Liu and Bi identified a vulnerability in the scheme using lattice-based methods. Nevertheless, the rapid increase in computational cost associated with the attack limited its practical applicability, leaving the scheme's overall security largely intact. In this paper, we present a novel two-stage attack that extends the work of Liu and Bi to databases of arbitrary sizes. To this end, we employ a binary-search-like preprocessing technique, which enables a significant reduction in the number of lattice problems that need to be considered. Specifically, we demonstrate how to compromise the scheme in a matter of minutes using an ordinary laptop. Our findings are substantiated through both rigorous analytical proofs and comprehensive numerical experiments.

Title: Elastic Weight Consolidation for Full-Parameter Continual Pre-Training of Gemma2

• Authors: Vytenis Šliogeris, Povilas Daniušis, Artūras Nakvosas
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.05946
• Pdf URL: https://arxiv.org/pdf/2505.05946
• Copy Paste: [[2505.05946]] Elastic Weight Consolidation for Full-Parameter Continual Pre-Training of Gemma2(https://arxiv.org/abs/2505.05946)
• Keywords: large language model
• Abstract: This technical report describes an experiment on autoregressive pre-training of Gemma2 2 billion parameter large language model (LLM) with 10\% on the Lithuanian language component of CulturaX from the point of view of continual learning. We apply elastic weight consolidation (EWC) to the full set of the model's parameters and investigate language understanding benchmarks, consisting of Arc, Belebele, Gsm8K, Hellaswag, MMLU, TruthfulQA, and Winogrande sets (both in English and Lithuanian versions), and perplexity benchmarks. We empirically demonstrate that EWC regularisation allows us not only to mitigate catastrophic forgetting effects but also that it is potentially beneficial for learning of the new task with LLMs.

Title: Summarisation of German Judgments in conjunction with a Class-based Evaluation

• Authors: Bianca Steffes, Nils Torben Wiedemann, Alexander Gratz, Pamela Hochreither, Jana Elina Meyer, Katharina Luise Schilke
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05947
• Pdf URL: https://arxiv.org/pdf/2505.05947
• Copy Paste: [[2505.05947]] Summarisation of German Judgments in conjunction with a Class-based Evaluation(https://arxiv.org/abs/2505.05947)
• Keywords: generative, large language model
• Abstract: The automated summarisation of long legal documents can be a great aid for legal experts in their daily work. We automatically create summaries (guiding principles) of German judgments by fine-tuning a decoder-based large language model. We enrich the judgments with information about legal entities before the training. For the evaluation of the created summaries, we define a set of evaluation classes which allows us to measure their language, pertinence, completeness and correctness. Our results show that employing legal entities helps the generative model to find the relevant content, but the quality of the created summaries is not yet sufficient for a use in practice.

Title: NeoQA: Evidence-based Question Answering with Generated News Events

• Authors: Max Glockner, Xiang Jiang, Leonardo F. R. Ribeiro, Iryna Gurevych, Markus Dreyer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.05949
• Pdf URL: https://arxiv.org/pdf/2505.05949
• Copy Paste: [[2505.05949]] NeoQA: Evidence-based Question Answering with Generated News Events(https://arxiv.org/abs/2505.05949)
• Keywords: large language model
• Abstract: Evaluating Retrieval-Augmented Generation (RAG) in large language models (LLMs) is challenging because benchmarks can quickly become stale. Questions initially requiring retrieval may become answerable from pretraining knowledge as newer models incorporate more recent information during pretraining, making it difficult to distinguish evidence-based reasoning from recall. We introduce NeoQA (News Events for Out-of-training Question Answering), a benchmark designed to address this issue. To construct NeoQA, we generated timelines and knowledge bases of fictional news events and entities along with news articles and Q\&A pairs to prevent LLMs from leveraging pretraining knowledge, ensuring that no prior evidence exists in their training data. We propose our dataset as a new platform for evaluating evidence-based question answering, as it requires LLMs to generate responses exclusively from retrieved evidence and only when sufficient evidence is available. NeoQA enables controlled evaluation across various evidence scenarios, including cases with missing or misleading details. Our findings indicate that LLMs struggle to distinguish subtle mismatches between questions and evidence, and suffer from short-cut reasoning when key information required to answer a question is missing from the evidence, underscoring key limitations in evidence-based reasoning.

Title: Towards Quantum Resilience: Data-Driven Migration Strategy Design

• Authors: Nahid Aliyev, Ozan Cetin, Emil Huseynov
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.05959
• Pdf URL: https://arxiv.org/pdf/2505.05959
• Copy Paste: [[2505.05959]] Towards Quantum Resilience: Data-Driven Migration Strategy Design(https://arxiv.org/abs/2505.05959)
• Keywords: attack
• Abstract: The advancements in quantum computing are a threat to classical cryptographic systems. The traditional cryptographic methods that utilize factorization-based or discrete-logarithm-based algorithms, such as RSA and ECC, are some of these. This paper thoroughly investigates the vulnerabilities of traditional cryptographic methods against quantum attacks and provides a decision-support framework to help organizations in recommending mitigation plans and determining appropriate transition strategies to post-quantum cryptography. A semi-synthetic dataset, consisting of key features such as key size, network complexity, and sensitivity levels, is crafted, with each configuration labeled according to its recommended mitigation plan. Using decision tree and random forest models, a classifier is trained to recommend appropriate mitigation/transition plans such as continuous monitoring, scheduled transitions, and immediate hybrid implementation. The proposed approach introduces a data-driven and dynamic solution for organizations to assess the scale of the migration, specifying a structured roadmap toward quantum resilience. The results highlight important features that influence strategy decisions and support actionable recommendations for cryptographic modernization based on system context.

Title: Offline Multi-agent Reinforcement Learning via Score Decomposition

• Authors: Dan Qiao, Wenhao Li, Shanchao Yang, Hongyuan Zha, Baoxiang Wang
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2505.05968
• Pdf URL: https://arxiv.org/pdf/2505.05968
• Copy Paste: [[2505.05968]] Offline Multi-agent Reinforcement Learning via Score Decomposition(https://arxiv.org/abs/2505.05968)
• Keywords: diffusion, generative
• Abstract: Offline multi-agent reinforcement learning (MARL) faces critical challenges due to distributional shifts, further exacerbated by the high dimensionality of joint action spaces and the diversity in coordination strategies and quality among agents. Conventional approaches, including independent learning frameworks and value decomposition methods based on pessimistic principles, remain susceptible to out-of-distribution (OOD) joint actions and often yield suboptimal performance. Through systematic analysis of prevalent offline MARL benchmarks, we identify that this limitation primarily stems from the inherently multimodal nature of joint collaborative policies induced by offline data collection. To address these challenges, we propose a novel two-stage framework: First, we employ a diffusion-based generative model to explicitly capture the complex behavior policy, enabling accurate modeling of diverse multi-agent coordination patterns. Second, we introduce a sequential score function decomposition mechanism to regularize individual policies and enable decentralized execution. Extensive experiments on continuous control tasks demonstrate state-of-the-art performance across multiple standard offline MARL benchmarks, outperforming existing methods by 26.3\% in normalized returns. Our approach provides new insights into offline coordination and equilibrium selection in cooperative multi-agent systems.

Title: Task-Adapter++: Task-specific Adaptation with Order-aware Alignment for Few-shot Action Recognition

• Authors: Congqi Cao, Peiheng Han, Yueran zhang, Yating Yu, Qinyi Lv, Lingtong Min, Yanning zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06002
• Pdf URL: https://arxiv.org/pdf/2505.06002
• Copy Paste: [[2505.06002]] Task-Adapter++: Task-specific Adaptation with Order-aware Alignment for Few-shot Action Recognition(https://arxiv.org/abs/2505.06002)
• Keywords: extraction, large language model
• Abstract: Large-scale pre-trained models have achieved remarkable success in language and image tasks, leading an increasing number of studies to explore the application of pre-trained image models, such as CLIP, in the domain of few-shot action recognition (FSAR). However, current methods generally suffer from several problems: 1) Direct fine-tuning often undermines the generalization capability of the pre-trained model; 2) The exploration of task-specific information is insufficient in the visual tasks; 3) The semantic order information is typically overlooked during text modeling; 4) Existing cross-modal alignment techniques ignore the temporal coupling of multimodal information. To address these, we propose Task-Adapter++, a parameter-efficient dual adaptation method for both image and text encoders. Specifically, to make full use of the variations across different few-shot learning tasks, we design a task-specific adaptation for the image encoder so that the most discriminative information can be well noticed during feature extraction. Furthermore, we leverage large language models (LLMs) to generate detailed sequential sub-action descriptions for each action class, and introduce semantic order adapters into the text encoder to effectively model the sequential relationships between these sub-actions. Finally, we develop an innovative fine-grained cross-modal alignment strategy that actively maps visual features to reside in the same temporal stage as semantic descriptions. Extensive experiments fully demonstrate the effectiveness and superiority of the proposed method, which achieves state-of-the-art performance on 5 benchmarks consistently. The code is open-sourced at this https URL.

Title: Fuzzy-UCS Revisited: Self-Adaptation of Rule Representations in Michigan-Style Learning Fuzzy-Classifier Systems

• Authors: Hiroki Shiraishi, Yohei Hayamizu, Tomonori Hashiyama
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06017
• Pdf URL: https://arxiv.org/pdf/2505.06017
• Copy Paste: [[2505.06017]] Fuzzy-UCS Revisited: Self-Adaptation of Rule Representations in Michigan-Style Learning Fuzzy-Classifier Systems(https://arxiv.org/abs/2505.06017)
• Keywords: robust
• Abstract: This paper focuses on the impact of rule representation in Michigan-style Learning Fuzzy-Classifier Systems (LFCSs) on its classification performance. A well-representation of the rules in an LFCS is crucial for improving its performance. However, conventional rule representations frequently need help addressing problems with unknown data characteristics. To address this issue, this paper proposes a supervised LFCS (i.e., Fuzzy-UCS) with a self-adaptive rule representation mechanism, entitled Adaptive-UCS. Adaptive-UCS incorporates a fuzzy indicator as a new rule parameter that sets the membership function of a rule as either rectangular (i.e., crisp) or triangular (i.e., fuzzy) shapes. The fuzzy indicator is optimized with evolutionary operators, allowing the system to search for an optimal rule representation. Results from extensive experiments conducted on continuous space problems demonstrate that Adaptive-UCS outperforms other UCSs with conventional crisp-hyperrectangular and fuzzy-hypertrapezoidal rule representations in classification accuracy. Additionally, Adaptive-UCS exhibits robustness in the case of noisy inputs and real-world problems with inherent uncertainty, such as missing values, leading to stable classification performance.

Title: Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation

• Authors: Stefan Vasilev, Christian Herold, Baohao Liao, Seyyed Hadi Hashemi, Shahram Khadivi, Christof Monz
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06027
• Pdf URL: https://arxiv.org/pdf/2505.06027
• Copy Paste: [[2505.06027]] Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation(https://arxiv.org/abs/2505.06027)
• Keywords: privacy, robust, large language model
• Abstract: This paper introduces Unilogit, a novel self-distillation method for machine unlearning in Large Language Models. Unilogit addresses the challenge of selectively forgetting specific information while maintaining overall model utility, a critical task in compliance with data privacy regulations like GDPR. Unlike prior methods that rely on static hyperparameters or starting model outputs, Unilogit dynamically adjusts target logits to achieve a uniform probability for the target token, leveraging the current model's outputs for more accurate self-distillation targets. This approach not only eliminates the need for additional hyperparameters but also enhances the model's ability to approximate the golden targets. Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit's superior performance in balancing forget and retain objectives, outperforming state-of-the-art methods such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness across various scenarios, highlighting its practical applicability and effectiveness in achieving efficacious machine unlearning.

Title: Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification

• Authors: Leon Eshuijs, Shihan Wang, Antske Fokkens
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.06032
• Pdf URL: https://arxiv.org/pdf/2505.06032
• Copy Paste: [[2505.06032]] Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification(https://arxiv.org/abs/2505.06032)
• Keywords: interpretability
• Abstract: Reliance on spurious correlations (shortcuts) has been shown to underlie many of the successes of language models. Previous work focused on identifying the input elements that impact prediction. We investigate how shortcuts are actually processed within the model's decision-making mechanism. We use actor names in movie reviews as controllable shortcuts with known impact on the outcome. We use mechanistic interpretability methods and identify specific attention heads that focus on shortcuts. These heads gear the model towards a label before processing the complete input, effectively making premature decisions that bypass contextual analysis. Based on these findings, we introduce Head-based Token Attribution (HTA), which traces intermediate decisions back to input tokens. We show that HTA is effective in detecting shortcuts in LLMs and enables targeted mitigation by selectively deactivating shortcut-related attention heads.

Title: Document Image Rectification Bases on Self-Adaptive Multitask Fusion

• Authors: Heng Li, Xiangping Wu, Qingcai Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06038
• Pdf URL: https://arxiv.org/pdf/2505.06038
• Copy Paste: [[2505.06038]] Document Image Rectification Bases on Self-Adaptive Multitask Fusion(https://arxiv.org/abs/2505.06038)
• Keywords: segmentation
• Abstract: Deformed document image rectification is essential for real-world document understanding tasks, such as layout analysis and text recognition. However, current multi-task methods -- such as background removal, 3D coordinate prediction, and text line segmentation -- often overlook the complementary features between tasks and their interactions. To address this gap, we propose a self-adaptive learnable multi-task fusion rectification network named SalmRec. This network incorporates an inter-task feature aggregation module that adaptively improves the perception of geometric distortions, enhances feature complementarity, and reduces negative interference. We also introduce a gating mechanism to balance features both within global tasks and between local tasks effectively. Experimental results on two English benchmarks (DIR300 and DocUNet) and one Chinese benchmark (DocReal) demonstrate that our method significantly improves rectification performance. Ablation studies further highlight the positive impact of different tasks on dewarping and the effectiveness of our proposed module.

Title: Healthy LLMs? Benchmarking LLM Knowledge of UK Government Public Health Information

• Authors: Joshua Harris, Fan Grayson, Felix Feldman, Timothy Laurence, Toby Nonnenmacher, Oliver Higgins, Leo Loman, Selina Patel, Thomas Finnie, Samuel Collins, Michael Borowitz
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06046
• Pdf URL: https://arxiv.org/pdf/2505.06046
• Copy Paste: [[2505.06046]] Healthy LLMs? Benchmarking LLM Knowledge of UK Government Public Health Information(https://arxiv.org/abs/2505.06046)
• Keywords: large language model
• Abstract: As Large Language Models (LLMs) become widely accessible, a detailed understanding of their knowledge within specific domains becomes necessary for successful real world use. This is particularly critical in public health, where failure to retrieve relevant, accurate, and current information could significantly impact UK residents. However, currently little is known about LLM knowledge of UK Government public health information. To address this issue, this paper introduces a new benchmark, PubHealthBench, with over 8000 questions for evaluating LLMs' Multiple Choice Question Answering (MCQA) and free form responses to public health queries, created via an automated pipeline. We also release a new dataset of the extracted UK Government public health guidance documents used as source text for PubHealthBench. Assessing 24 LLMs on PubHealthBench we find the latest private LLMs (GPT-4.5, GPT-4.1 and o1) have a high degree of knowledge, achieving >90% in the MCQA setup, and outperform humans with cursory search engine use. However, in the free form setup we see lower performance with no model scoring >75%. Therefore, whilst there are promising signs that state of the art (SOTA) LLMs are an increasingly accurate source of public health information, additional safeguards or tools may still be needed when providing free form responses on public health topics.

Title: PYRREGULAR: A Unified Framework for Irregular Time Series, with Classification Benchmarks

• Authors: Francesco Spinnato, Cristiano Landi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.06047
• Pdf URL: https://arxiv.org/pdf/2505.06047
• Copy Paste: [[2505.06047]] PYRREGULAR: A Unified Framework for Irregular Time Series, with Classification Benchmarks(https://arxiv.org/abs/2505.06047)
• Keywords: robust
• Abstract: Irregular temporal data, characterized by varying recording frequencies, differing observation durations, and missing values, presents significant challenges across fields like mobility, healthcare, and environmental science. Existing research communities often overlook or address these challenges in isolation, leading to fragmented tools and methods. To bridge this gap, we introduce a unified framework, and the first standardized dataset repository for irregular time series classification, built on a common array format to enhance interoperability. This repository comprises 34 datasets on which we benchmark 12 classifier models from diverse domains and communities. This work aims to centralize research efforts and enable a more robust evaluation of irregular temporal data analysis methods.

Title: Safe-EF: Error Feedback for Nonsmooth Constrained Optimization

• Authors: Rustem Islamov, Yarden As, Ilyas Fatkhullin
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.06053
• Pdf URL: https://arxiv.org/pdf/2505.06053
• Copy Paste: [[2505.06053]] Safe-EF: Error Feedback for Nonsmooth Constrained Optimization(https://arxiv.org/abs/2505.06053)
• Keywords: federate
• Abstract: Federated learning faces severe communication bottlenecks due to the high dimensionality of model updates. Communication compression with contractive compressors (e.g., Top-K) is often preferable in practice but can degrade performance without proper handling. Error feedback (EF) mitigates such issues but has been largely restricted for smooth, unconstrained problems, limiting its real-world applicability where non-smooth objectives and safety constraints are critical. We advance our understanding of EF in the canonical non-smooth convex setting by establishing new lower complexity bounds for first-order algorithms with contractive compression. Next, we propose Safe-EF, a novel algorithm that matches our lower bound (up to a constant) while enforcing safety constraints essential for practical applications. Extending our approach to the stochastic setting, we bridge the gap between theory and practical implementation. Extensive experiments in a reinforcement learning setup, simulating distributed humanoid robot training, validate the effectiveness of Safe-EF in ensuring safety and reducing communication complexity.

Title: Towards Better Cephalometric Landmark Detection with Diffusion Data Generation

• Authors: Dongqian Guo, Wencheng Han, Pang Lyu, Yuxi Zhou, Jianbing Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06055
• Pdf URL: https://arxiv.org/pdf/2505.06055
• Copy Paste: [[2505.06055]] Towards Better Cephalometric Landmark Detection with Diffusion Data Generation(https://arxiv.org/abs/2505.06055)
• Keywords: diffusion
• Abstract: Cephalometric landmark detection is essential for orthodontic diagnostics and treatment planning. Nevertheless, the scarcity of samples in data collection and the extensive effort required for manual annotation have significantly impeded the availability of diverse datasets. This limitation has restricted the effectiveness of deep learning-based detection methods, particularly those based on large-scale vision models. To address these challenges, we have developed an innovative data generation method capable of producing diverse cephalometric X-ray images along with corresponding annotations without human intervention. To achieve this, our approach initiates by constructing new cephalometric landmark annotations using anatomical priors. Then, we employ a diffusion-based generator to create realistic X-ray images that correspond closely with these annotations. To achieve precise control in producing samples with different attributes, we introduce a novel prompt cephalometric X-ray image dataset. This dataset includes real cephalometric X-ray images and detailed medical text prompts describing the images. By leveraging these detailed prompts, our method improves the generation process to control different styles and attributes. Facilitated by the large, diverse generated data, we introduce large-scale vision detection models into the cephalometric landmark detection task to improve accuracy. Experimental results demonstrate that training with the generated data substantially enhances the performance. Compared to methods without using the generated data, our approach improves the Success Detection Rate (SDR) by 6.5%, attaining a notable 82.2%. All code and data are available at: this https URL

Title: Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation

• Authors: Kunpeng Qiu, Zhiqiang Gao, Zhiying Zhou, Mingjie Sun, Yongxin Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06068
• Pdf URL: https://arxiv.org/pdf/2505.06068
• Copy Paste: [[2505.06068]] Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation(https://arxiv.org/abs/2505.06068)
• Keywords: robust, diffusion, segmentation
• Abstract: Deep learning has revolutionized medical image segmentation, yet its full potential remains constrained by the paucity of annotated datasets. While diffusion models have emerged as a promising approach for generating synthetic image-mask pairs to augment these datasets, they paradoxically suffer from the same data scarcity challenges they aim to mitigate. Traditional mask-only models frequently yield low-fidelity images due to their inability to adequately capture morphological intricacies, which can critically compromise the robustness and reliability of segmentation models. To alleviate this limitation, we introduce Siamese-Diffusion, a novel dual-component model comprising Mask-Diffusion and Image-Diffusion. During training, a Noise Consistency Loss is introduced between these components to enhance the morphological fidelity of Mask-Diffusion in the parameter space. During sampling, only Mask-Diffusion is used, ensuring diversity and scalability. Comprehensive experiments demonstrate the superiority of our method. Siamese-Diffusion boosts SANet's mDice and mIoU by 3.6% and 4.4% on the Polyps, while UNet improves by 1.52% and 1.64% on the ISIC2018. Code is available at GitHub.

Title: HashKitty: Distributed Password Analysis

• Authors: Pedro Antunes, Tomás Santos, Daniel Fuentes, Luís Frazão
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2505.06084
• Pdf URL: https://arxiv.org/pdf/2505.06084
• Copy Paste: [[2505.06084]] HashKitty: Distributed Password Analysis(https://arxiv.org/abs/2505.06084)
• Keywords: security
• Abstract: This article documents the HashKitty platform, a distributed solution for password analysis based on the hashcat tool, designed to improve efficiency in both offensive and defensive security operations. The main objectives of this work are to utilise and characterise the hashcat tool, to develop a central platform that connects various computational nodes, to allow the use of nodes with different equipment and manufacturers, to distribute tasks among the nodes through a web platform, and to perform distributed password analysis. The results show that the presented solution achieves the proposed objectives, demonstrating effectiveness in workload distribution and password analysis using different types of nodes based on various operating systems and architectures. The architecture of HashKitty is based on a scalable and modular distributed architecture, composed of several components such as computational nodes, integration and control software, a web platform that implements our API, and database servers. In order to achieve a fast and organised development process for our application we used multiple frameworks, runtimes and libraries. For the communication between the computational nodes and the other software we made use of websockets so that we have real-time updates between them.

Title: Deep Diffusion Maps

• Authors: Sergio García-Heredia, Ángela Fernández, Carlos M. Alaíz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06087
• Pdf URL: https://arxiv.org/pdf/2505.06087
• Copy Paste: [[2505.06087]] Deep Diffusion Maps(https://arxiv.org/abs/2505.06087)
• Keywords: diffusion
• Abstract: One of the fundamental problems within the field of machine learning is dimensionality reduction. Dimensionality reduction methods make it possible to combat the so-called curse of dimensionality, visualize high-dimensional data and, in general, improve the efficiency of storing and processing large data sets. One of the best-known nonlinear dimensionality reduction methods is Diffusion Maps. However, despite their virtues, both Diffusion Maps and many other manifold learning methods based on the spectral decomposition of kernel matrices have drawbacks such as the inability to apply them to data outside the initial set, their computational complexity, and high memory costs for large data sets. In this work, we propose to alleviate these problems by resorting to deep learning. Specifically, a new formulation of Diffusion Maps embedding is offered as a solution to a certain unconstrained minimization problem and, based on it, a cost function to train a neural network which computes Diffusion Maps embedding -- both inside and outside the training sample -- without the need to perform any spectral decomposition. The capabilities of this approach are compared on different data sets, both real and synthetic, with those of Diffusion Maps and the Nystrom method.

Title: UniSymNet: A Unified Symbolic Network Guided by Transformer

• Authors: Xinxin Li, Juan Zhang, Da Li, Xingyu Liu, Jin Xu, Junping Yin
• Subjects: cs.LG, cs.AI, cs.SC
• Abstract URL: https://arxiv.org/abs/2505.06091
• Pdf URL: https://arxiv.org/pdf/2505.06091
• Copy Paste: [[2505.06091]] UniSymNet: A Unified Symbolic Network Guided by Transformer(https://arxiv.org/abs/2505.06091)
• Keywords: transformer
• Abstract: Symbolic Regression (SR) is a powerful technique for automatically discovering mathematical expressions from input data. Mainstream SR algorithms search for the optimal symbolic tree in a vast function space, but the increasing complexity of the tree structure limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, most existing symbolic networks still face certain challenges: binary nonlinear operators $\{\times, ÷\}$ cannot be naturally extended to multivariate operators, and training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a Unified Symbolic Network that unifies nonlinear binary operators into nested unary operators and define the conditions under which UniSymNet can reduce complexity. Moreover, we pre-train a Transformer model with a novel label encoding method to guide structural selection, and adopt objective-specific optimization strategies to learn the parameters of the symbolic network. UniSymNet shows high fitting accuracy, excellent symbolic solution rate, and relatively low expression complexity, achieving competitive performance on low-dimensional Standard Benchmarks and high-dimensional SRBench.

Title: LLMs Outperform Experts on Challenging Biology Benchmarks

• Authors: Lennart Justen
• Subjects: cs.LG, cs.AI, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2505.06108
• Pdf URL: https://arxiv.org/pdf/2505.06108
• Copy Paste: [[2505.06108]] LLMs Outperform Experts on Challenging Biology Benchmarks(https://arxiv.org/abs/2505.06108)
• Keywords: security, large language model
• Abstract: This study systematically evaluates 27 frontier Large Language Models on eight diverse biology benchmarks spanning molecular biology, genetics, cloning, virology, and biosecurity. Models from major AI developers released between November 2022 and April 2025 were assessed through ten independent runs per benchmark. The findings reveal dramatic improvements in biological capabilities. Top model performance increased more than 4-fold on the challenging text-only subset of the Virology Capabilities Test over the study period, with the top model now performing twice as well as expert virologists. Several models now match or exceed expert-level performance on other challenging benchmarks, including LAB-Bench CloningScenarios and the biology subsets of GPQA and WMDP. Contrary to expectations, chain-of-thought did not substantially improve performance over zero-shot evaluation, while extended reasoning features in o3-mini and Claude 3.7 Sonnet typically improved performance as predicted by inference scaling. Benchmarks such as PubMedQA and the MMLU and WMDP biology subsets exhibited performance plateaus well below 100%, suggesting benchmark saturation and errors in the underlying benchmark data. The analysis highlights the need for more sophisticated evaluation methodologies as AI systems continue to advance.

Title: Multimodal Sentiment Analysis on CMU-MOSEI Dataset using Transformer-based Models

• Authors: Jugal Gajjar, Kaustik Ranaware
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.06110
• Pdf URL: https://arxiv.org/pdf/2505.06110
• Copy Paste: [[2505.06110]] Multimodal Sentiment Analysis on CMU-MOSEI Dataset using Transformer-based Models(https://arxiv.org/abs/2505.06110)
• Keywords: robust, interpretability, transformer
• Abstract: This project performs multimodal sentiment analysis using the CMU-MOSEI dataset, using transformer-based models with early fusion to integrate text, audio, and visual modalities. We employ BERT-based encoders for each modality, extracting embeddings that are concatenated before classification. The model achieves strong performance, with 97.87\% 7-class accuracy and a 0.9682 F1-score on the test set, demonstrating the effectiveness of early fusion in capturing cross-modal interactions. The training utilized Adam optimization (lr=1e-4), dropout (0.3), and early stopping to ensure generalization and robustness. Results highlight the superiority of transformer architectures in modeling multimodal sentiment, with a low MAE (0.1060) indicating precise sentiment intensity prediction. Future work may compare fusion strategies or enhance interpretability. This approach utilizes multimodal learning by effectively combining linguistic, acoustic, and visual cues for sentiment analysis.

Title: Camera-Only Bird's Eye View Perception: A Neural Approach to LiDAR-Free Environmental Mapping for Autonomous Vehicles

• Authors: Anupkumar Bochare
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06113
• Pdf URL: https://arxiv.org/pdf/2505.06113
• Copy Paste: [[2505.06113]] Camera-Only Bird's Eye View Perception: A Neural Approach to LiDAR-Free Environmental Mapping for Autonomous Vehicles(https://arxiv.org/abs/2505.06113)
• Keywords: segmentation
• Abstract: Autonomous vehicle perception systems have traditionally relied on costly LiDAR sensors to generate precise environmental representations. In this paper, we propose a camera-only perception framework that produces Bird's Eye View (BEV) maps by extending the Lift-Splat-Shoot architecture. Our method combines YOLOv11-based object detection with DepthAnythingV2 monocular depth estimation across multi-camera inputs to achieve comprehensive 360-degree scene understanding. We evaluate our approach on the OpenLane-V2 and NuScenes datasets, achieving up to 85% road segmentation accuracy and 85-90% vehicle detection rates when compared against LiDAR ground truth, with average positional errors limited to 1.2 meters. These results highlight the potential of deep learning to extract rich spatial information using only camera inputs, enabling cost-efficient autonomous navigation without sacrificing accuracy.

Title: Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigation

• Authors: Dongying Li, Binyi Su, Hua Zhang, Yong Li, Haiyong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06117
• Pdf URL: https://arxiv.org/pdf/2505.06117
• Copy Paste: [[2505.06117]] Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigation(https://arxiv.org/abs/2505.06117)
• Keywords: diffusion, generative
• Abstract: Accurate defect detection of photovoltaic (PV) cells is critical for ensuring quality and efficiency in intelligent PV manufacturing systems. However, the scarcity of rich defect data poses substantial challenges for effective model training. While existing methods have explored generative models to augment datasets, they often suffer from instability, limited diversity, and domain shifts. To address these issues, we propose PDIG, a Photovoltaic Defect Image Generator based on Stable Diffusion (SD). PDIG leverages the strong priors learned from large-scale datasets to enhance generation quality under limited data. Specifically, we introduce a Semantic Concept Embedding (SCE) module that incorporates text-conditioned priors to capture the relational concepts between defect types and their appearances. To further enrich the domain distribution, we design a Lightweight Industrial Style Adaptor (LISA), which injects industrial defect characteristics into the SD model through cross-disentangled attention. At inference, we propose a Text-Image Dual-Space Constraints (TIDSC) module, enforcing the quality of generated images via positional consistency and spatial smoothing alignment. Extensive experiments demonstrate that PDIG achieves superior realism and diversity compared to state-of-the-art methods. Specifically, our approach improves Frechet Inception Distance (FID) by 19.16 points over the second-best method and significantly enhances the performance of downstream defect detection tasks.

Title: LLMs Get Lost In Multi-Turn Conversation

• Authors: Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer Neville
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2505.06120
• Pdf URL: https://arxiv.org/pdf/2505.06120
• Copy Paste: [[2505.06120]] LLMs Get Lost In Multi-Turn Conversation(https://arxiv.org/abs/2505.06120)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are conversational interfaces. As such, LLMs have the potential to assist their users not only when they can fully specify the task at hand, but also to help them define, explore, and refine what they need through multi-turn conversational exchange. Although analysis of LLM conversation logs has confirmed that underspecification occurs frequently in user instructions, LLM evaluation has predominantly focused on the single-turn, fully-specified instruction setting. In this work, we perform large-scale simulation experiments to compare LLM performance in single- and multi-turn settings. Our experiments confirm that all the top open- and closed-weight LLMs we test exhibit significantly lower performance in multi-turn conversations than single-turn, with an average drop of 39% across six generation tasks. Analysis of 200,000+ simulated conversations decomposes the performance degradation into two components: a minor loss in aptitude and a significant increase in unreliability. We find that LLMs often make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely. In simpler terms, we discover that *when LLMs take a wrong turn in a conversation, they get lost and do not recover*.

Title: BrainSegDMlF: A Dynamic Fusion-enhanced SAM for Brain Lesion Segmentation

• Authors: Hongming Wang, Yifeng Wu, Huimin Huang, Hongtao Wu, Jia-Xuan Jiang, Xiaodong Zhang, Hao Zheng, Xian Wu, Yefeng Zheng, Jinping Xu, Jing Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06133
• Pdf URL: https://arxiv.org/pdf/2505.06133
• Copy Paste: [[2505.06133]] BrainSegDMlF: A Dynamic Fusion-enhanced SAM for Brain Lesion Segmentation(https://arxiv.org/abs/2505.06133)
• Keywords: segmentation
• Abstract: The segmentation of substantial brain lesions is a significant and challenging task in the field of medical image segmentation. Substantial brain lesions in brain imaging exhibit high heterogeneity, with indistinct boundaries between lesion regions and normal brain tissue. Small lesions in single slices are difficult to identify, making the accurate and reproducible segmentation of abnormal regions, as well as their feature description, highly complex. Existing methods have the following limitations: 1) They rely solely on single-modal information for learning, neglecting the multi-modal information commonly used in diagnosis. This hampers the ability to comprehensively acquire brain lesion information from multiple perspectives and prevents the effective integration and utilization of multi-modal data inputs, thereby limiting a holistic understanding of lesions. 2) They are constrained by the amount of data available, leading to low sensitivity to small lesions and difficulty in detecting subtle pathological changes. 3) Current SAM-based models rely on external prompts, which cannot achieve automatic segmentation and, to some extent, affect diagnostic this http URL address these issues, we have developed a large-scale fully automated segmentation model specifically designed for brain lesion segmentation, named BrainSegDMLF. This model has the following features: 1) Dynamic Modal Interactive Fusion (DMIF) module that processes and integrates multi-modal data during the encoding process, providing the SAM encoder with more comprehensive modal information. 2) Layer-by-Layer Upsampling Decoder, enabling the model to extract rich low-level and high-level features even with limited data, thereby detecting the presence of small lesions. 3) Automatic segmentation masks, allowing the model to generate lesion masks automatically without requiring manual prompts.

Title: Realistic Adversarial Attacks for Robustness Evaluation of Trajectory Prediction Models via Future State Perturbation

• Authors: Julian F. Schumann, Jeroen Hagenus, Frederik Baymler Mathiesen, Arkady Zgonnikov
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2505.06134
• Pdf URL: https://arxiv.org/pdf/2505.06134
• Copy Paste: [[2505.06134]] Realistic Adversarial Attacks for Robustness Evaluation of Trajectory Prediction Models via Future State Perturbation(https://arxiv.org/abs/2505.06134)
• Keywords: attack, robust
• Abstract: Trajectory prediction is a key element of autonomous vehicle systems, enabling them to anticipate and react to the movements of other road users. Evaluating the robustness of prediction models against adversarial attacks is essential to ensure their reliability in real-world traffic. However, current approaches tend to focus on perturbing the past positions of surrounding agents, which can generate unrealistic scenarios and overlook critical vulnerabilities. This limitation may result in overly optimistic assessments of model performance in real-world conditions. In this work, we demonstrate that perturbing not just past but also future states of adversarial agents can uncover previously undetected weaknesses and thereby provide a more rigorous evaluation of model robustness. Our novel approach incorporates dynamic constraints and preserves tactical behaviors, enabling more effective and realistic adversarial attacks. We introduce new performance measures to assess the realism and impact of these adversarial trajectories. Testing our method on a state-of-the-art prediction model revealed significant increases in prediction errors and collision rates under adversarial conditions. Qualitative analysis further showed that our attacks can expose critical weaknesses, such as the inability of the model to detect potential collisions in what appear to be safe predictions. These results underscore the need for more comprehensive adversarial testing to better evaluate and improve the reliability of trajectory prediction models for autonomous vehicles.

Title: Towards Robust Few-Shot Text Classification Using Transformer Architectures and Dual Loss Strategies

• Authors: Xu Han, Yumeng Sun, Weiqiang Huang, Hongye Zheng, Junliang Du
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06145
• Pdf URL: https://arxiv.org/pdf/2505.06145
• Copy Paste: [[2505.06145]] Towards Robust Few-Shot Text Classification Using Transformer Architectures and Dual Loss Strategies(https://arxiv.org/abs/2505.06145)
• Keywords: robust, transformer, generative
• Abstract: Few-shot text classification has important application value in low-resource environments. This paper proposes a strategy that combines adaptive fine-tuning, contrastive learning, and regularization optimization to improve the classification performance of Transformer-based models. Experiments on the FewRel 2.0 dataset show that T5-small, DeBERTa-v3, and RoBERTa-base perform well in few-shot tasks, especially in the 5-shot setting, which can more effectively capture text features and improve classification accuracy. The experiment also found that there are significant differences in the classification difficulty of different relationship categories. Some categories have fuzzy semantic boundaries or complex feature distributions, making it difficult for the standard cross entropy loss to learn the discriminative information required to distinguish categories. By introducing contrastive loss and regularization loss, the generalization ability of the model is enhanced, effectively alleviating the overfitting problem in few-shot environments. In addition, the research results show that the use of Transformer models or generative architectures with stronger self-attention mechanisms can help improve the stability and accuracy of few-shot classification.

Title: Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study

• Authors: Faeze Ghorbanpour, Daryna Dementieva, Alexander Fraser
• Subjects: cs.CL, cs.CY, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.06149
• Pdf URL: https://arxiv.org/pdf/2505.06149
• Copy Paste: [[2505.06149]] Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study(https://arxiv.org/abs/2505.06149)
• Keywords: large language model
• Abstract: Despite growing interest in automated hate speech detection, most existing approaches overlook the linguistic diversity of online content. Multilingual instruction-tuned large language models such as LLaMA, Aya, Qwen, and BloomZ offer promising capabilities across languages, but their effectiveness in identifying hate speech through zero-shot and few-shot prompting remains underexplored. This work evaluates LLM prompting-based detection across eight non-English languages, utilizing several prompting techniques and comparing them to fine-tuned encoder models. We show that while zero-shot and few-shot prompting lag behind fine-tuned encoder models on most of the real-world evaluation sets, they achieve better generalization on functional tests for hate speech detection. Our study also reveals that prompt design plays a critical role, with each language often requiring customized prompting techniques to maximize performance.

Title: A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets

• Authors: Ryan Lagasse, Aidan Kiernans, Avijit Ghosh, Shiri Dori-Hacohen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.06150
• Pdf URL: https://arxiv.org/pdf/2505.06150
• Copy Paste: [[2505.06150]] A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets(https://arxiv.org/abs/2505.06150)
• Keywords: large language model
• Abstract: We introduce a scaling law for fine-tuning large language models (LLMs) under fixed compute budgets that explicitly accounts for data composition. Conventional approaches measure training data solely by total tokens, yet the number of examples and their average token length -- what we term \emph{dataset volume} -- play a decisive role in model performance. Our formulation is tuned following established procedures. Experiments on the BRICC dataset \cite{salavati2024reducing} and subsets of the MMLU dataset \cite{hendrycks2021measuringmassivemultitasklanguage}, evaluated under multiple subsampling strategies, reveal that data composition significantly affects token efficiency. These results motivate refined scaling laws for practical LLM fine-tuning in resource-constrained settings.

Title: Estimating Quality in Therapeutic Conversations: A Multi-Dimensional Natural Language Processing Framework

• Authors: Alice Rueda, Argyrios Perivolaris, Niloy Roy, Dylan Weston, Sarmed Shaya, Zachary Cote, Martin Ivanov, Bazen G. Teferra, Yuqi Wu, Sirisha Rambhatla, Divya Sharma, Andrew Greenshaw, Rakesh Jetly, Yanbo Zhang, Bo Cao, Reza Samavi, Sridhar Krishnan, Venkat Bhat
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.06151
• Pdf URL: https://arxiv.org/pdf/2505.06151
• Copy Paste: [[2505.06151]] Estimating Quality in Therapeutic Conversations: A Multi-Dimensional Natural Language Processing Framework(https://arxiv.org/abs/2505.06151)
• Keywords: robust
• Abstract: Engagement between client and therapist is a critical determinant of therapeutic success. We propose a multi-dimensional natural language processing (NLP) framework that objectively classifies engagement quality in counseling sessions based on textual transcripts. Using 253 motivational interviewing transcripts (150 high-quality, 103 low-quality), we extracted 42 features across four domains: conversational dynamics, semantic similarity as topic alignment, sentiment classification, and question detection. Classifiers, including Random Forest (RF), Cat-Boost, and Support Vector Machines (SVM), were hyperparameter tuned and trained using a stratified 5-fold cross-validation and evaluated on a holdout test set. On balanced (non-augmented) data, RF achieved the highest classification accuracy (76.7%), and SVM achieved the highest AUC (85.4%). After SMOTE-Tomek augmentation, performance improved significantly: RF achieved up to 88.9% accuracy, 90.0% F1-score, and 94.6% AUC, while SVM reached 81.1% accuracy, 83.1% F1-score, and 93.6% AUC. The augmented data results reflect the potential of the framework in future larger-scale applications. Feature contribution revealed conversational dynamics and semantic similarity between clients and therapists were among the top contributors, led by words uttered by the client (mean and standard deviation). The framework was robust across the original and augmented datasets and demonstrated consistent improvements in F1 scores and recall. While currently text-based, the framework supports future multimodal extensions (e.g., vocal tone, facial affect) for more holistic assessments. This work introduces a scalable, data-driven method for evaluating engagement quality of the therapy session, offering clinicians real-time feedback to enhance the quality of both virtual and in-person therapeutic interactions.

Title: DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models

• Authors: Radu Alexandru Rosu, Keyu Wu, Yao Feng, Youyi Zheng, Michael J. Black
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06166
• Pdf URL: https://arxiv.org/pdf/2505.06166
• Copy Paste: [[2505.06166]] DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models(https://arxiv.org/abs/2505.06166)
• Keywords: diffusion, transformer
• Abstract: We address the task of generating 3D hair geometry from a single image, which is challenging due to the diversity of hairstyles and the lack of paired image-to-3D hair data. Previous methods are primarily trained on synthetic data and cope with the limited amount of such data by using low-dimensional intermediate representations, such as guide strands and scalp-level embeddings, that require post-processing to decode, upsample, and add realism. These approaches fail to reconstruct detailed hair, struggle with curly hair, or are limited to handling only a few hairstyles. To overcome these limitations, we propose DiffLocks, a novel framework that enables detailed reconstruction of a wide variety of hairstyles directly from a single image. First, we address the lack of 3D hair data by automating the creation of the largest synthetic hair dataset to date, containing 40K hairstyles. Second, we leverage the synthetic hair dataset to learn an image-conditioned diffusion-transfomer model that generates accurate 3D strands from a single frontal image. By using a pretrained image backbone, our method generalizes to in-the-wild images despite being trained only on synthetic data. Our diffusion model predicts a scalp texture map in which any point in the map contains the latent code for an individual hair strand. These codes are directly decoded to 3D strands without post-processing techniques. Representing individual strands, instead of guide strands, enables the transformer to model the detailed spatial structure of complex hairstyles. With this, DiffLocks can recover highly curled hair, like afro hairstyles, from a single image for the first time. Data and code is available at this https URL

Title: Self-Supervised Federated GNSS Spoofing Detection with Opportunistic Data

• Authors: Wenjie Liu, Panos Papadimitratos
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.06171
• Pdf URL: https://arxiv.org/pdf/2505.06171
• Copy Paste: [[2505.06171]] Self-Supervised Federated GNSS Spoofing Detection with Opportunistic Data(https://arxiv.org/abs/2505.06171)
• Keywords: privacy, attack, federate
• Abstract: Global navigation satellite systems (GNSS) are vulnerable to spoofing attacks, with adversarial signals manipulating the location or time information of receivers, potentially causing severe disruptions. The task of discerning the spoofing signals from benign ones is naturally relevant for machine learning, thus recent interest in applying it for detection. While deep learning-based methods are promising, they require extensive labeled datasets, consume significant computational resources, and raise privacy concerns due to the sensitive nature of position data. This is why this paper proposes a self-supervised federated learning framework for GNSS spoofing detection. It consists of a cloud server and local mobile platforms. Each mobile platform employs a self-supervised anomaly detector using long short-term memory (LSTM) networks. Labels for training are generated locally through a spoofing-deviation prediction algorithm, ensuring privacy. Local models are trained independently, and only their parameters are uploaded to the cloud server, which aggregates them into a global model using FedAvg. The updated global model is then distributed back to the mobile platforms and trained iteratively. The evaluation shows that our self-supervised federated learning framework outperforms position-based and deep learning-based methods in detecting spoofing attacks while preserving data privacy.

Title: Leakage-resilient Algebraic Manipulation Detection Codes with Optimal Parameters

• Authors: Divesh Aggarwal, Tomasz Kazana, Maciej Obremski
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2505.06174
• Pdf URL: https://arxiv.org/pdf/2505.06174
• Copy Paste: [[2505.06174]] Leakage-resilient Algebraic Manipulation Detection Codes with Optimal Parameters(https://arxiv.org/abs/2505.06174)
• Keywords: secure, security, protect
• Abstract: Algebraic Manipulation Detection (AMD) codes is a cryptographic primitive that was introduced by Cramer, Dodis, Fehr, Padro and Wichs. They are keyless message authentication codes that protect messages against additive tampering by the adversary assuming that the adversary cannot "see" the codeword. For certain applications, it is unreasonable to assume that the adversary computes the added offset without any knowledge of the codeword c. Recently, Ahmadi and Safavi-Naini, and then Lin, Safavi-Naini, and Wang gave a construction of leakage-resilient AMD codes where the adversary has some partial information about the codeword before choosing added offset, and the scheme is secure even conditioned on this partial information. In this paper we establish bounds on the leakage rate r and the code rate k for leakage-resilient AMD codes. In particular we prove that 2r + k < 1 and for the weak case (security is averaged over a uniformly random message) r + k < 1. These bounds hold even if adversary is polynomial-time bounded, as long as we allow leakage function to be arbitrary. We present constructions of AMD codes that (asymptotically) fulfill the above bounds for almost full range of parameters r and k. This shows that the above bounds and constructions are in-fact optimal. In the last section we show that if a leakage function is computationally bounded (we use the Ideal Cipher Model) then it is possible to break these bounds.

Title: A Large Language Model-Enhanced Q-learning for Capacitated Vehicle Routing Problem with Time Windows

• Authors: Linjiang Cao, Maonan Wang, Xi Xiong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06178
• Pdf URL: https://arxiv.org/pdf/2505.06178
• Copy Paste: [[2505.06178]] A Large Language Model-Enhanced Q-learning for Capacitated Vehicle Routing Problem with Time Windows(https://arxiv.org/abs/2505.06178)
• Keywords: large language model
• Abstract: The Capacitated Vehicle Routing Problem with Time Windows (CVRPTW) is a classic NP-hard combinatorial optimization problem widely applied in logistics distribution and transportation management. Its complexity stems from the constraints of vehicle capacity and time windows, which pose significant challenges to traditional approaches. Advances in Large Language Models (LLMs) provide new possibilities for finding approximate solutions to CVRPTW. This paper proposes a novel LLM-enhanced Q-learning framework to address the CVRPTW with real-time emergency constraints. Our solution introduces an adaptive two-phase training mechanism that transitions from the LLM-guided exploration phase to the autonomous optimization phase of Q-network. To ensure reliability, we design a three-tier self-correction mechanism based on the Chain-of-Thought (CoT) for LLMs: syntactic validation, semantic verification, and physical constraint enforcement. In addition, we also prioritized replay of the experience generated by LLMs to amplify the regulatory role of LLMs in the architecture. Experimental results demonstrate that our framework achieves a 7.3\% average reduction in cost compared to traditional Q-learning, with fewer training steps required for convergence.

Title: Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet

• Authors: Kodai Hirata, Tsuyoshi Okita
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06185
• Pdf URL: https://arxiv.org/pdf/2505.06185
• Copy Paste: [[2505.06185]] Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet(https://arxiv.org/abs/2505.06185)
• Keywords: transformer, segmentation
• Abstract: This paper proposes a method MTL-Swin-Unet which is multi-task learning using transformers for classification and semantic segmentation. For spurious-correlation problems, this method allows us to enhance the image representation with two other image representations: representation obtained by semantic segmentation and representation obtained by image reconstruction. In our experiments, the proposed method outperformed in F-value measure than other classifiers when the test data included slices from the same patient (no covariate shift). Similarly, when the test data did not include slices from the same patient (covariate shift setting), the proposed method outperformed in AUC measure.

Title: Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies

• Authors: Massimiliano Pronesti, Joao Bettencourt-Silva, Paul Flanagan, Alessandra Pascale, Oisin Redmond, Anya Belz, Yufang Hou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.06186
• Pdf URL: https://arxiv.org/pdf/2505.06186
• Copy Paste: [[2505.06186]] Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies(https://arxiv.org/abs/2505.06186)
• Keywords: extraction
• Abstract: Extracting scientific evidence from biomedical studies for clinical research questions (e.g., Does stem cell transplantation improve quality of life in patients with medically refractory Crohn's disease compared to placebo?) is a crucial step in synthesising biomedical evidence. In this paper, we focus on the task of document-level scientific evidence extraction for clinical questions with conflicting evidence. To support this task, we create a dataset called CochraneForest, leveraging forest plots from Cochrane systematic reviews. It comprises 202 annotated forest plots, associated clinical research questions, full texts of studies, and study-specific conclusions. Building on CochraneForest, we propose URCA (Uniform Retrieval Clustered Augmentation), a retrieval-augmented generation framework designed to tackle the unique challenges of evidence extraction. Our experiments show that URCA outperforms the best existing methods by up to 10.3% in F1 score on this task. However, the results also underscore the complexity of CochraneForest, establishing it as a challenging testbed for advancing automated evidence synthesis systems.

Title: Auto Tensor Singular Value Thresholding: A Non-Iterative and Rank-Free Framework for Tensor Denoising

• Authors: Hiroki Hasegawa, Yukihiko Okada
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06203
• Pdf URL: https://arxiv.org/pdf/2505.06203
• Copy Paste: [[2505.06203]] Auto Tensor Singular Value Thresholding: A Non-Iterative and Rank-Free Framework for Tensor Denoising(https://arxiv.org/abs/2505.06203)
• Keywords: extraction
• Abstract: In modern data-driven tasks such as classification, optimization, and forecasting, mitigating the effects of intrinsic noise is crucial for improving predictive accuracy. While numerous denoising techniques have been developed, the rising dimensionality of real-world datasets limits conventional matrix-based methods in preserving data structure and accuracy. This challenge has led to increasing interest in tensor-based approaches, which naturally capture multi-way data relationships. However, classical tensor decomposition methods (e.g., HOSVD, HOOI) typically require pre-specified ranks and iterative optimization, making them computationally expensive and less practical. In this work, we propose a novel low-rank approximation method for tensor data that avoids these limitations. Our approach applies statistically grounded singular value thresholding to mode-wise matricizations, enabling automatic extraction of significant components without requiring prior rank specification or iterative refinement. Experiments on synthetic and real-world tensors show that our method consistently outperforms existing techniques in terms of estimation accuracy and computational efficiency, especially in noisy high-dimensional settings.

Title: Adapting a Segmentation Foundation Model for Medical Image Classification

• Authors: Pengfei Gu, Haoteng Tang, Islam A. Ebeid, Jose A. Nunez, Fabian Vazquez, Diego Adame, Marcus Zhan, Huimin Li, Bin Fu, Danny Z. Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.06217
• Pdf URL: https://arxiv.org/pdf/2505.06217
• Copy Paste: [[2505.06217]] Adapting a Segmentation Foundation Model for Medical Image Classification(https://arxiv.org/abs/2505.06217)
• Keywords: segmentation
• Abstract: Recent advancements in foundation models, such as the Segment Anything Model (SAM), have shown strong performance in various vision tasks, particularly image segmentation, due to their impressive zero-shot segmentation capabilities. However, effectively adapting such models for medical image classification is still a less explored topic. In this paper, we introduce a new framework to adapt SAM for medical image classification. First, we utilize the SAM image encoder as a feature extractor to capture segmentation-based features that convey important spatial and contextual details of the image, while freezing its weights to avoid unnecessary overhead during training. Next, we propose a novel Spatially Localized Channel Attention (SLCA) mechanism to compute spatially localized attention weights for the feature maps. The features extracted from SAM's image encoder are processed through SLCA to compute attention weights, which are then integrated into deep learning classification models to enhance their focus on spatially relevant or meaningful regions of the image, thus improving classification performance. Experimental results on three public medical image classification datasets demonstrate the effectiveness and data-efficiency of our approach.

Title: Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks

• Authors: Christos Plachouras, Julien Guinot, George Fazekas, Elio Quinton, Emmanouil Benetos, Johan Pauwels
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.06224
• Pdf URL: https://arxiv.org/pdf/2505.06224
• Copy Paste: [[2505.06224]] Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks(https://arxiv.org/abs/2505.06224)
• Keywords: interpretability
• Abstract: Downstream probing has been the dominant method for evaluating model representations, an important process given the increasing prominence of self-supervised learning and foundation models. However, downstream probing primarily assesses the availability of task-relevant information in the model's latent space, overlooking attributes such as equivariance, invariance, and disentanglement, which contribute to the interpretability, adaptability, and utility of representations in real-world applications. While some attempts have been made to measure these qualities in representations, no unified evaluation framework with modular, generalizable, and interpretable metrics exists. In this paper, we argue for the importance of representation evaluation beyond downstream probing. We introduce a standardized protocol to quantify informativeness, equivariance, invariance, and disentanglement of factors of variation in model representations. We use it to evaluate representations from a variety of models in the image and speech domains using different architectures and pretraining approaches on identified controllable factors of variation. We find that representations from models with similar downstream performance can behave substantially differently with regard to these attributes. This hints that the respective mechanisms underlying their downstream performance are functionally different, prompting new research directions to understand and improve representations.