2025-03-06

Title: Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies

• Authors: Shaibal Saha, Lanyu Xu
• Subjects: cs.CV, cs.AR
• Abstract URL: https://arxiv.org/abs/2503.02891
• Pdf URL: https://arxiv.org/pdf/2503.02891
• Copy Paste: [[2503.02891]] Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies(https://arxiv.org/abs/2503.02891)
• Keywords: extraction, transformer, segmentation
• Abstract: In recent years, vision transformers (ViTs) have emerged as powerful and promising techniques for computer vision tasks such as image classification, object detection, and segmentation. Unlike convolutional neural networks (CNNs), which rely on hierarchical feature extraction, ViTs treat images as sequences of patches and leverage self-attention mechanisms. However, their high computational complexity and memory demands pose significant challenges for deployment on resource-constrained edge devices. To address these limitations, extensive research has focused on model compression techniques and hardware-aware acceleration strategies. Nonetheless, a comprehensive review that systematically categorizes these techniques and their trade-offs in accuracy, efficiency, and hardware adaptability for edge deployment remains lacking. This survey bridges this gap by providing a structured analysis of model compression techniques, software tools for inference on edge, and hardware acceleration strategies for ViTs. We discuss their impact on accuracy, efficiency, and hardware adaptability, highlighting key challenges and emerging research directions to advance ViT deployment on edge platforms, including graphics processing units (GPUs), tensor processing units (TPUs), and field-programmable gate arrays (FPGAs). The goal is to inspire further research with a contemporary guide on optimizing ViTs for efficient deployment on edge devices.

Title: ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection

• Authors: Hong Lu, Yali Bian, Rahul C. Shah
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02897
• Pdf URL: https://arxiv.org/pdf/2503.02897
• Copy Paste: [[2503.02897]] ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection(https://arxiv.org/abs/2503.02897)
• Keywords: robust
• Abstract: High-quality annotations are essential for object detection models, but ensuring label accuracy - especially for bounding boxes - remains both challenging and costly. This paper introduces ClipGrader, a novel approach that leverages vision-language models to automatically assess the accuracy of bounding box annotations. By adapting CLIP (Contrastive Language-Image Pre-training) to evaluate both class label correctness and spatial precision of bounding box, ClipGrader offers an effective solution for grading object detection labels. Tested on modified object detection datasets with artificially disturbed bounding boxes, ClipGrader achieves 91% accuracy on COCO with a 1.8% false positive rate. Moreover, it maintains 87% accuracy with a 2.1% false positive rate when trained on just 10% of the COCO data. ClipGrader also scales effectively to larger datasets such as LVIS, achieving 79% accuracy across 1,203 classes. Our experiments demonstrate ClipGrader's ability to identify errors in existing COCO annotations, highlighting its potential for dataset refinement. When integrated into a semi-supervised object detection (SSOD) model, ClipGrader readily improves the pseudo label quality, helping achieve higher mAP (mean Average Precision) throughout the training process. ClipGrader thus provides a scalable AI-assisted tool for enhancing annotation quality control and verifying annotations in large-scale object detection datasets.

Title: LangGas: Introducing Language in Selective Zero-Shot Background Subtraction for Semi-Transparent Gas Leak Detection with a New Dataset

• Authors: Wenqi Guo, Yiyang Du, Shan Du
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.02910
• Pdf URL: https://arxiv.org/pdf/2503.02910
• Copy Paste: [[2503.02910]] LangGas: Introducing Language in Selective Zero-Shot Background Subtraction for Semi-Transparent Gas Leak Detection with a New Dataset(https://arxiv.org/abs/2503.02910)
• Keywords: segmentation
• Abstract: Gas leakage poses a significant hazard that requires prevention. Traditionally, human inspection has been used for detection, a slow and labour-intensive process. Recent research has applied machine learning techniques to this problem, yet there remains a shortage of high-quality, publicly available datasets. This paper introduces a synthetic dataset featuring diverse backgrounds, interfering foreground objects, diverse leak locations, and precise segmentation ground truth. We propose a zero-shot method that combines background subtraction, zero-shot object detection, filtering, and segmentation to leverage this dataset. Experimental results indicate that our approach significantly outperforms baseline methods based solely on background subtraction and zero-shot object detection with segmentation, reaching an IoU of 69\% overall. We also present an analysis of various prompt configurations and threshold settings to provide deeper insights into the performance of our method. The code and dataset will be released after publication.

Title: Straight-Line Diffusion Model for Efficient 3D Molecular Generation

• Authors: Yuyan Ni, Shikun Feng, Haohan Chi, Bowen Zheng, Huan-ang Gao, Wei-Ying Ma, Zhi-Ming Ma, Yanyan Lan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02918
• Pdf URL: https://arxiv.org/pdf/2503.02918
• Copy Paste: [[2503.02918]] Straight-Line Diffusion Model for Efficient 3D Molecular Generation(https://arxiv.org/abs/2503.02918)
• Keywords: robust, diffusion, generative
• Abstract: Diffusion-based models have shown great promise in molecular generation but often require a large number of sampling steps to generate valid samples. In this paper, we introduce a novel Straight-Line Diffusion Model (SLDM) to tackle this problem, by formulating the diffusion process to follow a linear trajectory. The proposed process aligns well with the noise sensitivity characteristic of molecular structures and uniformly distributes reconstruction effort across the generative process, thus enhancing learning efficiency and efficacy. Consequently, SLDM achieves state-of-the-art performance on 3D molecule generation benchmarks, delivering a 100-fold improvement in sampling efficiency. Furthermore, experiments on toy data and image generation tasks validate the generality and robustness of SLDM, showcasing its potential across diverse generative modeling domains.

Title: Robust time series generation via Schrödinger Bridge: a comprehensive evaluation

• Authors: Alexandre Alouadi, Baptiste Barreau, Laurent Carlier, Huyên Pham
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2503.02943
• Pdf URL: https://arxiv.org/pdf/2503.02943
• Copy Paste: [[2503.02943]] Robust time series generation via Schrödinger Bridge: a comprehensive evaluation(https://arxiv.org/abs/2503.02943)
• Keywords: robust, generative
• Abstract: We investigate the generative capabilities of the Schrödinger Bridge (SB) approach for time series. The SB framework formulates time series synthesis as an entropic optimal interpolation transport problem between a reference probability measure on path space and a target joint distribution. This results in a stochastic differential equation over a finite horizon that accurately captures the temporal dynamics of the target time series. While the SB approach has been largely explored in fields like image generation, there is a scarcity of studies for its application to time series. In this work, we bridge this gap by conducting a comprehensive evaluation of the SB method's robustness and generative performance. We benchmark it against state-of-the-art (SOTA) time series generation methods across diverse datasets, assessing its strengths, limitations, and capacity to model complex temporal dependencies. Our results offer valuable insights into the SB framework's potential as a versatile and robust tool for time series generation.

Title: KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

• Authors: Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, Radha Poovendran
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.02951
• Pdf URL: https://arxiv.org/pdf/2503.02951
• Copy Paste: [[2503.02951]] KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding(https://arxiv.org/abs/2503.02951)
• Keywords: robust, large language model
• Abstract: We introduce KodCode, a synthetic dataset that addresses the persistent challenge of acquiring high-quality, verifiable training data across diverse difficulties and domains for training Large Language Models for coding. Existing code-focused resources typically fail to ensure either the breadth of coverage (e.g., spanning simple coding tasks to advanced algorithmic problems) or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises question-solution-test triplets that are systematically validated via a self-verification procedure. Our pipeline begins by synthesizing a broad range of coding questions, then generates solutions and test cases with additional attempts allocated to challenging problems. Finally, post-training data synthesis is done by rewriting questions into diverse formats and generating responses under a test-based reject sampling procedure from a reasoning model (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding dataset. KodCode is suitable for supervised fine-tuning and the paired unit tests also provide great potential for RL tuning. Fine-tuning experiments on coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench) demonstrate that KodCode-tuned models achieve state-of-the-art performance, surpassing models like Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B.

Title: Koopman-Based Generalization of Deep Reinforcement Learning With Application to Wireless Communications

• Authors: Atefeh Termehchi, Ekram Hossain, Isaac Woungang
• Subjects: cs.LG, cs.IT
• Abstract URL: https://arxiv.org/abs/2503.02961
• Pdf URL: https://arxiv.org/pdf/2503.02961
• Copy Paste: [[2503.02961]] Koopman-Based Generalization of Deep Reinforcement Learning With Application to Wireless Communications(https://arxiv.org/abs/2503.02961)
• Keywords: robust, interpretability
• Abstract: Deep Reinforcement Learning (DRL) is a key machine learning technology driving progress across various scientific and engineering fields, including wireless communication. However, its limited interpretability and generalizability remain major challenges. In supervised learning, generalizability is commonly evaluated through the generalization error using information-theoretic methods. In DRL, the training data is sequential and not independent and identically distributed (i.i.d.), rendering traditional information-theoretic methods unsuitable for generalizability analysis. To address this challenge, this paper proposes a novel analytical method for evaluating the generalizability of DRL. Specifically, we first model the evolution of states and actions in trained DRL algorithms as unknown discrete, stochastic, and nonlinear dynamical functions. Then, we employ a data-driven identification method, the Koopman operator, to approximate these functions, and propose two interpretable representations. Based on these interpretable representations, we develop a rigorous mathematical approach to evaluate the generalizability of DRL algorithms. This approach is formulated using the spectral feature analysis of the Koopman operator, leveraging the H_\infty norm. Finally, we apply this generalization analysis to compare the soft actor-critic method, widely recognized as a robust DRL approach, against the proximal policy optimization algorithm for an unmanned aerial vehicle-assisted mmWave wireless communication scenario.

Title: Revolutionizing Traffic Management with AI-Powered Machine Vision: A Step Toward Smart Cities

• Authors: Seyed Hossein Hosseini DolatAbadi, Sayyed Mohammad Hossein Hashemi, Mohammad Hosseini, Moein-Aldin AliHosseini
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.02967
• Pdf URL: https://arxiv.org/pdf/2503.02967
• Copy Paste: [[2503.02967]] Revolutionizing Traffic Management with AI-Powered Machine Vision: A Step Toward Smart Cities(https://arxiv.org/abs/2503.02967)
• Keywords: robust
• Abstract: The rapid urbanization of cities and increasing vehicular congestion have posed significant challenges to traffic management and safety. This study explores the transformative potential of artificial intelligence (AI) and machine vision technologies in revolutionizing traffic systems. By leveraging advanced surveillance cameras and deep learning algorithms, this research proposes a system for real-time detection of vehicles, traffic anomalies, and driver behaviors. The system integrates geospatial and weather data to adapt dynamically to environmental conditions, ensuring robust performance in diverse scenarios. Using YOLOv8 and YOLOv11 models, the study achieves high accuracy in vehicle detection and anomaly recognition, optimizing traffic flow and enhancing road safety. These findings contribute to the development of intelligent traffic management solutions and align with the vision of creating smart cities with sustainable and efficient urban infrastructure.

Title: Privacy-Preserving Fair Synthetic Tabular Data

• Authors: Fatima J. Sarmin, Atiquer R. Rahman, Christopher J. Henry, Noman Mohammed
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.02968
• Pdf URL: https://arxiv.org/pdf/2503.02968
• Copy Paste: [[2503.02968]] Privacy-Preserving Fair Synthetic Tabular Data(https://arxiv.org/abs/2503.02968)
• Keywords: privacy, protect, fair
• Abstract: Sharing of tabular data containing valuable but private information is limited due to legal and ethical issues. Synthetic data could be an alternative solution to this sharing problem, as it is artificially generated by machine learning algorithms and tries to capture the underlying data distribution. However, machine learning models are not free from memorization and may introduce biases, as they rely on training data. Producing synthetic data that preserves privacy and fairness while maintaining utility close to the real data is a challenging task. This research simultaneously addresses both the privacy and fairness aspects of synthetic data, an area not explored by other studies. In this work, we present PF-WGAN, a privacy-preserving, fair synthetic tabular data generator based on the WGAN-GP model. We have modified the original WGAN-GP by adding privacy and fairness constraints forcing it to produce privacy-preserving fair data. This approach will enable the publication of datasets that protect individual's privacy and remain unbiased toward any particular group. We compared the results with three state-of-the-art synthetic data generator models in terms of utility, privacy, and fairness across four different datasets. We found that the proposed model exhibits a more balanced trade-off among utility, privacy, and fairness.

Title: InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model

• Authors: Siqi Ouyang, Xi Xu, Lei Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02969
• Pdf URL: https://arxiv.org/pdf/2503.02969
• Copy Paste: [[2503.02969]] InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model(https://arxiv.org/abs/2503.02969)
• Keywords: robust, large language model
• Abstract: Simultaneous translation of unbounded streaming speech remains a challenging problem due to the need for effectively processing the history speech context and past translations so that quality and latency, including computation overhead, can be balanced. Most prior works assume pre-segmented speech, limiting their real-world applicability. In this paper, we propose InfiniSST, a novel approach that formulates SST as a multi-turn dialogue task, enabling seamless translation of unbounded speech. We construct translation trajectories and robust segments from MuST-C with multi-latency augmentation during training and develop a key-value (KV) cache management strategy to facilitate efficient inference. Experiments on MuST-C En-Es, En-De, and En-Zh demonstrate that InfiniSST reduces computation-aware latency by 0.5 to 1 second while maintaining the same translation quality compared to baselines. Ablation studies further validate the contributions of our data construction and cache management strategy. We release the code at this https URL

Title: Multilingual Relative Clause Attachment Ambiguity Resolution in Large Language Models

• Authors: So Young Lee, Russell Scheinberg, Amber Shore, Ameeta Agrawal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.02971
• Pdf URL: https://arxiv.org/pdf/2503.02971
• Copy Paste: [[2503.02971]] Multilingual Relative Clause Attachment Ambiguity Resolution in Large Language Models(https://arxiv.org/abs/2503.02971)
• Keywords: large language model
• Abstract: This study examines how large language models (LLMs) resolve relative clause (RC) attachment ambiguities and compares their performance to human sentence processing. Focusing on two linguistic factors, namely the length of RCs and the syntactic position of complex determiner phrases (DPs), we assess whether LLMs can achieve human-like interpretations amid the complexities of language. In this study, we evaluated several LLMs, including Claude, Gemini and Llama, in multiple languages: English, Spanish, French, German, Japanese, and Korean. While these models performed well in Indo-European languages (English, Spanish, French, and German), they encountered difficulties in Asian languages (Japanese and Korean), often defaulting to incorrect English translations. The findings underscore the variability in LLMs' handling of linguistic ambiguities and highlight the need for model improvements, particularly for non-European languages. This research informs future enhancements in LLM design to improve accuracy and human-like processing in diverse linguistic environments.

Title: LINGOLY-TOO: Disentangling Memorisation from Reasoning with Linguistic Templatisation and Orthographic Obfuscation

• Authors: Jude Khouja, Karolina Korgul, Simi Hellsten, Lingyi Yang, Vlad Neacs, Harry Mayne, Ryan Kearns, Andrew Bean, Adam Mahdi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.02972
• Pdf URL: https://arxiv.org/pdf/2503.02972
• Copy Paste: [[2503.02972]] LINGOLY-TOO: Disentangling Memorisation from Reasoning with Linguistic Templatisation and Orthographic Obfuscation(https://arxiv.org/abs/2503.02972)
• Keywords: large language model
• Abstract: Effective evaluation of the reasoning capabilities of large language models (LLMs) are susceptible to overestimation due to data exposure of evaluation benchmarks. We introduce a framework for producing linguistic reasoning problems that reduces the effect of memorisation in model performance estimates and apply this framework to develop LINGOLY-TOO, a challenging evaluation benchmark for linguistic reasoning. By developing orthographic templates, we dynamically obfuscate the writing systems of real languages to generate numerous question variations. These variations preserve the reasoning steps required for each solution while reducing the likelihood of specific problem instances appearing in model training data. Our experiments demonstrate that frontier models, including OpenAI o1-preview and DeepSeem R1, struggle with advanced reasoning. Our analysis also shows that LLMs exhibit noticeable variance in accuracy across permutations of the same problem, and on average perform better on questions appearing in their original orthography. Our findings highlight the opaque nature of response generation in LLMs and provide evidence that prior data exposure contributes to overestimating the reasoning capabilities of frontier models.

Title: Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model

• Authors: Boris N. Slautin, Utkarsh Pratiush, Doru C. Lupascu, Maxim A. Ziatdinov, Sergei V. Kalinin
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2503.02978
• Pdf URL: https://arxiv.org/pdf/2503.02978
• Copy Paste: [[2503.02978]] Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model(https://arxiv.org/abs/2503.02978)
• Keywords: generative
• Abstract: We introduce a Deep Kernel Learning Variational Autoencoder (VAE-DKL) framework that integrates the generative power of a Variational Autoencoder (VAE) with the predictive nature of Deep Kernel Learning (DKL). The VAE learns a latent representation of high-dimensional data, enabling the generation of novel structures, while DKL refines this latent space by structuring it in alignment with target properties through Gaussian Process (GP) regression. This approach preserves the generative capabilities of the VAE while enhancing its latent space for GP-based property prediction. We evaluate the framework on two datasets: a structured card dataset with predefined variational factors and the QM9 molecular dataset, where enthalpy serves as the target function for optimization. The model demonstrates high-precision property prediction and enables the generation of novel out-of-training subset structures with desired characteristics. The VAE-DKL framework offers a promising approach for high-throughput material discovery and molecular design, balancing structured latent space organization with generative flexibility.

Title: Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis

• Authors: Jeonghwan Park, Niall McLaughlin, Ihsen Alouani
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.02986
• Pdf URL: https://arxiv.org/pdf/2503.02986
• Copy Paste: [[2503.02986]] Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis(https://arxiv.org/abs/2503.02986)
• Keywords: defense, attack, robust
• Abstract: Adversarial attacks remain a significant threat that can jeopardize the integrity of Machine Learning (ML) models. In particular, query-based black-box attacks can generate malicious noise without having access to the victim model's architecture, making them practical in real-world contexts. The community has proposed several defenses against adversarial attacks, only to be broken by more advanced and adaptive attack strategies. In this paper, we propose a framework that detects if an adversarial noise instance is being generated. Unlike existing stateful defenses that detect adversarial noise generation by monitoring the input space, our approach learns adversarial patterns in the input update similarity space. In fact, we propose to observe a new metric called Delta Similarity (DS), which we show it captures more efficiently the adversarial behavior. We evaluate our approach against 8 state-of-the-art attacks, including adaptive attacks, where the adversary is aware of the defense and tries to evade detection. We find that our approach is significantly more robust than existing defenses both in terms of specificity and sensitivity.

Title: Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability

• Authors: Hayden McAlister, Anthony Robins, Lech Szymanski
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2503.03018
• Pdf URL: https://arxiv.org/pdf/2503.03018
• Copy Paste: [[2503.03018]] Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability(https://arxiv.org/abs/2503.03018)
• Keywords: interpretability
• Abstract: We extend the existing work on Hopfield network state classification, employing more complex models that remain interpretable, such as densely-connected feed-forward deep neural networks and support vector machines. The states of the Hopfield network can be grouped into several classes, including learned (those presented during training), spurious (stable states that were not learned), and prototype (stable states that were not learned but are representative for a subset of learned states). It is often useful to determine to what class a given state belongs to; for example to ignore spurious states when retrieving from the network. Previous research has approached the state classification task with simple linear methods, most notably the stability ratio. We deepen the research on classifying states from prototype-regime Hopfield networks, investigating how varying the factors strengthening prototypes influences the state classification task. We study the generalizability of different classification models when trained on states derived from different prototype tasks -- for example, can a network trained on a Hopfield network with 10 prototypes classify states from a network with 20 prototypes? We find that simple models often outperform the stability ratio while remaining interpretable. These models require surprisingly little training data and generalize exceptionally well to states generated by a range of Hopfield networks, even those that were trained on exceedingly different datasets.

Title: Adopt a PET! An Exploration of PETs, Policy, and Practicalities for Industry in Canada

• Authors: Masoumeh Shafieinejad, Xi He, Bailey Kacsmar
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03027
• Pdf URL: https://arxiv.org/pdf/2503.03027
• Copy Paste: [[2503.03027]] Adopt a PET! An Exploration of PETs, Policy, and Practicalities for Industry in Canada(https://arxiv.org/abs/2503.03027)
• Keywords: privacy, protect
• Abstract: Privacy enhancing technologies (PETs) are technical solutions for privacy issues that exist in our digital society. Despite increased privacy challenges and a corresponding increase in new regulations being proposed by governments across the globe, a low adoption rate of PETs persists. In this work, we investigate the relationship that new privacy regulations have on industry's decision-making processes as well as the extent to which privacy regulations inspire the adoption of PETs. We conducted a qualitative survey study with 22 industry participants from across Canada to investigate how businesses in Canada make decisions to adopt novel technologies and how new privacy regulations impact their business processes. Through this study, we identify the breadth of approaches employed by organizations considering PETs and the challenges they face in their efforts to ensure compliance with all pertinent laws and regulations. We further identify a gap between how companies think of privacy technologies and how researchers think of privacy technologies that can contribute to low adoption of the increasingly sophisticated privacy technologies produced by researchers, such as applications of differential privacy, multiparty computation, and trusted execution environments. Informed by the results of our analysis, we make recommendations for industry, researchers, and policymakers on how to support what each of them seeks from the other when attempting to improve digital privacy protections. By advancing our understanding of what challenges industry faces in ensuring compliance with novel and existing privacy regulations, we increase the effectiveness of future privacy research that aims to help overcome these issues.

Title: Network Anomaly Detection for IoT Using Hyperdimensional Computing on NSL-KDD

• Authors: Ghazal Ghajari, Ashutosh Ghimire, Elaheh Ghajari, Fathi Amsaad
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03031
• Pdf URL: https://arxiv.org/pdf/2503.03031
• Copy Paste: [[2503.03031]] Network Anomaly Detection for IoT Using Hyperdimensional Computing on NSL-KDD(https://arxiv.org/abs/2503.03031)
• Keywords: secure, security, attack, robust
• Abstract: With the rapid growth of IoT devices, ensuring robust network security has become a critical challenge. Traditional intrusion detection systems (IDSs) often face limitations in detecting sophisticated attacks within high-dimensional and complex data environments. This paper presents a novel approach to network anomaly detection using hyperdimensional computing (HDC) techniques, specifically applied to the NSL-KDD dataset. The proposed method leverages the efficiency of HDC in processing large-scale data to identify both known and unknown attack patterns. The model achieved an accuracy of 91.55% on the KDDTrain+ subset, outperforming traditional approaches. These comparative evaluations underscore the model's superior performance, highlighting its potential in advancing anomaly detection for IoT networks and contributing to more secure and intelligent cybersecurity solutions.

Title: SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMs

• Authors: Samir Abdaljalil, Filippo Pallucchini, Andrea Seveso, Hasan Kurban, Fabio Mercorio, Erchin Serpedin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03032
• Pdf URL: https://arxiv.org/pdf/2503.03032
• Copy Paste: [[2503.03032]] SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMs(https://arxiv.org/abs/2503.03032)
• Keywords: robust, large language model
• Abstract: Despite the state-of-the-art performance of Large Language Models (LLMs), these models often suffer from hallucinations, which can undermine their performance in critical applications. In this work, we propose SAFE, a novel method for detecting and mitigating hallucinations by leveraging Sparse Autoencoders (SAEs). While hallucination detection techniques and SAEs have been explored independently, their synergistic application in a comprehensive system, particularly for hallucination-aware query enrichment, has not been fully investigated. To validate the effectiveness of SAFE, we evaluate it on two models with available SAEs across three diverse cross-domain datasets designed to assess hallucination problems. Empirical results demonstrate that SAFE consistently improves query generation accuracy and mitigates hallucinations across all datasets, achieving accuracy improvements of up to 29.45%.

Title: Intrusion Detection in IoT Networks Using Hyperdimensional Computing: A Case Study on the NSL-KDD Dataset

• Authors: Ghazal Ghajari, Elaheh Ghajari, Hossein Mohammadi, Fathi Amsaad
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03037
• Pdf URL: https://arxiv.org/pdf/2503.03037
• Copy Paste: [[2503.03037]] Intrusion Detection in IoT Networks Using Hyperdimensional Computing: A Case Study on the NSL-KDD Dataset(https://arxiv.org/abs/2503.03037)
• Keywords: security, attack, robust
• Abstract: The rapid expansion of Internet of Things (IoT) networks has introduced new security challenges, necessitating efficient and reliable methods for intrusion detection. In this study, a detection framework based on hyperdimensional computing (HDC) is proposed to identify and classify network intrusions using the NSL-KDD dataset, a standard benchmark for intrusion detection systems. By leveraging the capabilities of HDC, including high-dimensional representation and efficient computation, the proposed approach effectively distinguishes various attack categories such as DoS, probe, R2L, and U2R, while accurately identifying normal traffic patterns. Comprehensive evaluations demonstrate that the proposed method achieves an accuracy of 99.54%, significantly outperforming conventional intrusion detection techniques, making it a promising solution for IoT network security. This work emphasizes the critical role of robust and precise intrusion detection in safeguarding IoT systems against evolving cyber threats.

Title: Generative assimilation and prediction for weather and climate

• Authors: Shangshang Yang, Congyi Nai, Xinyan Liu, Weidong Li, Jie Chao, Jingnan Wang, Leyi Wang, Xichen Li, Xi Chen, Bo Lu, Ziniu Xiao, Niklas Boers, Huiling Yuan, Baoxiang Pan
• Subjects: cs.LG, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2503.03038
• Pdf URL: https://arxiv.org/pdf/2503.03038
• Copy Paste: [[2503.03038]] Generative assimilation and prediction for weather and climate(https://arxiv.org/abs/2503.03038)
• Keywords: generative
• Abstract: Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on the prediction task, and do not incorporate the necessary data assimilation. Moreover, these models suffer from error accumulation in long roll-outs, limiting their applicability to seasonal predictions or climate projections. Here, we introduce Generative Assimilation and Prediction (GAP), a unified deep generative framework for assimilation and prediction of both weather and climate. By learning to quantify the probabilistic distribution of atmospheric states under observational, predictive, and external forcing constraints, GAP excels in a broad range of weather-climate related tasks, including data assimilation, seamless prediction, and climate simulation. In particular, GAP is competitive with state-of-the-art ensemble assimilation, probabilistic weather forecast and seasonal prediction, yields stable millennial simulations, and reproduces climate variability from daily to decadal time scales.

Title: LLM Misalignment via Adversarial RLHF Platforms

• Authors: Erfan Entezami, Ali Naseh
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03039
• Pdf URL: https://arxiv.org/pdf/2503.03039
• Copy Paste: [[2503.03039]] LLM Misalignment via Adversarial RLHF Platforms(https://arxiv.org/abs/2503.03039)
• Keywords: security, attack
• Abstract: Reinforcement learning has shown remarkable performance in aligning language models with human preferences, leading to the rise of attention towards developing RLHF platforms. These platforms enable users to fine-tune models without requiring any expertise in developing complex machine learning algorithms. While these platforms offer useful features such as reward modeling and RLHF fine-tuning, their security and reliability remain largely unexplored. Given the growing adoption of RLHF and open-source RLHF frameworks, we investigate the trustworthiness of these systems and their potential impact on behavior of LLMs. In this paper, we present an attack targeting publicly available RLHF tools. In our proposed attack, an adversarial RLHF platform corrupts the LLM alignment process by selectively manipulating data samples in the preference dataset. In this scenario, when a user's task aligns with the attacker's objective, the platform manipulates a subset of the preference dataset that contains samples related to the attacker's target. This manipulation results in a corrupted reward model, which ultimately leads to the misalignment of the language model. Our results demonstrate that such an attack can effectively steer LLMs toward undesirable behaviors within the targeted domains. Our work highlights the critical need to explore the vulnerabilities of RLHF platforms and their potential to cause misalignment in LLMs during the RLHF fine-tuning process.

Title: SAGE: Steering and Refining Dialog Generation with State-Action Augmentation

• Authors: Yizhe Zhang, Navdeep Jaitly
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03040
• Pdf URL: https://arxiv.org/pdf/2503.03040
• Copy Paste: [[2503.03040]] SAGE: Steering and Refining Dialog Generation with State-Action Augmentation(https://arxiv.org/abs/2503.03040)
• Keywords: large language model
• Abstract: Recent advances in large language models have demonstrated impressive capabilities in task-oriented applications, yet building emotionally intelligent chatbots that can engage in natural, strategic conversations remains a challenge. We present a novel approach called SAGE that uses latent variables to control long-horizon behavior in dialogue generation. At the core of our method is the State-Action Chain (SAC), which augments standard language model fine-tuning by introducing latent variables that encapsulate emotional states and conversational strategies between dialogue turns. During inference, these variables are generated before each response, enabling coarse-grained control over dialogue progression while maintaining natural interaction patterns. We also introduce a self-improvement pipeline that leverages dialogue tree search, LLM-based reward modeling, and targeted fine-tuning to optimize conversational trajectories. Our experimental results show that models trained with this approach demonstrate improved performance in emotional intelligence metrics while maintaining strong capabilities on LLM benchmarks. The discrete nature of our latent variables facilitates search-based strategies and provides a foundation for future applications of reinforcement learning to dialogue systems, where learning can occur at the state level rather than the token level.

Title: Learning from Noisy Labels with Contrastive Co-Transformer

• Authors: Yan Han, Soumava Kumar Roy, Mehrtash Harandi, Lars Petersson
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03042
• Pdf URL: https://arxiv.org/pdf/2503.03042
• Copy Paste: [[2503.03042]] Learning from Noisy Labels with Contrastive Co-Transformer(https://arxiv.org/abs/2503.03042)
• Keywords: robust, transformer
• Abstract: Deep learning with noisy labels is an interesting challenge in weakly supervised learning. Despite their significant learning capacity, CNNs have a tendency to overfit in the presence of samples with noisy labels. Alleviating this issue, the well known Co-Training framework is used as a fundamental basis for our work. In this paper, we introduce a Contrastive Co-Transformer framework, which is simple and fast, yet able to improve the performance by a large margin compared to the state-of-the-art approaches. We argue the robustness of transformers when dealing with label noise. Our Contrastive Co-Transformer approach is able to utilize all samples in the dataset, irrespective of whether they are clean or noisy. Transformers are trained by a combination of contrastive loss and classification loss. Extensive experimental results on corrupted data from six standard benchmark datasets including Clothing1M, demonstrate that our Contrastive Co-Transformer is superior to existing state-of-the-art methods.

Title: Leveraging Randomness in Model and Data Partitioning for Privacy Amplification

• Authors: Andy Dong, Wei-Ning Chen, Ayfer Ozgur
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03043
• Pdf URL: https://arxiv.org/pdf/2503.03043
• Copy Paste: [[2503.03043]] Leveraging Randomness in Model and Data Partitioning for Privacy Amplification(https://arxiv.org/abs/2503.03043)
• Keywords: privacy, federate
• Abstract: We study how inherent randomness in the training process -- where each sample (or client in federated learning) contributes only to a randomly selected portion of training -- can be leveraged for privacy amplification. This includes (1) data partitioning, where a sample participates in only a subset of training iterations, and (2) model partitioning, where a sample updates only a subset of the model parameters. We apply our framework to model parallelism in federated learning, where each client updates a randomly selected subnetwork to reduce memory and computational overhead, and show that existing methods, e.g. model splitting or dropout, provide a significant privacy amplification gain not captured by previous privacy analysis techniques. Additionally, we introduce Balanced Iteration Subsampling, a new data partitioning method where each sample (or client) participates in a fixed number of training iterations. We show that this method yields stronger privacy amplification than Poisson (i.i.d.) sampling of data (or clients). Our results demonstrate that randomness in the training process, which is structured rather than i.i.d. and interacts with data in complex ways, can be systematically leveraged for significant privacy amplification.

Title: Graph Transformer with Disease Subgraph Positional Encoding for Improved Comorbidity Prediction

• Authors: Xihan Qin, Li Liao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03046
• Pdf URL: https://arxiv.org/pdf/2503.03046
• Copy Paste: [[2503.03046]] Graph Transformer with Disease Subgraph Positional Encoding for Improved Comorbidity Prediction(https://arxiv.org/abs/2503.03046)
• Keywords: transformer
• Abstract: Comorbidity, the co-occurrence of multiple medical conditions in a single patient, profoundly impacts disease management and outcomes. Understanding these complex interconnections is crucial, especially in contexts where comorbidities exacerbate outcomes. Leveraging insights from the human interactome (HI) and advancements in graph-based methodologies, this study introduces Transformer with Subgraph Positional Encoding (TSPE) for disease comorbidity prediction. Inspired by Biologically Supervised Embedding (BSE), TSPE employs Transformer's attention mechanisms and Subgraph Positional Encoding (SPE) to capture interactions between nodes and disease associations. Our proposed SPE proves more effective than LPE, as used in Dwivedi et al.'s Graph Transformer, underscoring the importance of integrating clustering and disease-specific information for improved predictive accuracy. Evaluated on real clinical benchmark datasets (RR0 and RR1), TSPE demonstrates substantial performance enhancements over the state-of-the-art method, achieving up to 28.24% higher ROC AUC and 4.93% higher accuracy. This method shows promise for adaptation to other complex graph-based tasks and applications. The source code is available in the GitHub repository at: this https URL.

Title: AHCPTQ: Accurate and Hardware-Compatible Post-Training Quantization for Segment Anything Model

• Authors: Wenlun Zhang, Shimpei Ando, Kentaro Yoshioka
• Subjects: cs.CV, cs.AR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03088
• Pdf URL: https://arxiv.org/pdf/2503.03088
• Copy Paste: [[2503.03088]] AHCPTQ: Accurate and Hardware-Compatible Post-Training Quantization for Segment Anything Model(https://arxiv.org/abs/2503.03088)
• Keywords: segmentation
• Abstract: The Segment Anything Model (SAM) has demonstrated strong versatility across various visual tasks. However, its large storage requirements and high computational cost pose challenges for practical deployment. Post-training quantization (PTQ) has emerged as an effective strategy for efficient deployment, but we identify two key challenges in SAM that hinder the effectiveness of existing PTQ methods: the heavy-tailed and skewed distribution of post-GELU activations, and significant inter-channel variation in linear projection activations. To address these challenges, we propose AHCPTQ, an accurate and hardware-efficient PTQ method for SAM. AHCPTQ introduces hardware-compatible Hybrid Log-Uniform Quantization (HLUQ) to manage post-GELU activations, employing log2 quantization for dense small values and uniform quantization for sparse large values to enhance quantization resolution. Additionally, AHCPTQ incorporates Channel-Aware Grouping (CAG) to mitigate inter-channel variation by progressively clustering activation channels with similar distributions, enabling them to share quantization parameters and improving hardware efficiency. The combination of HLUQ and CAG not only enhances quantization effectiveness but also ensures compatibility with efficient hardware execution. For instance, under the W4A4 configuration on the SAM-L model, AHCPTQ achieves 36.6% mAP on instance segmentation with the DINO detector, while achieving a 7.89x speedup and 8.64x energy efficiency over its floating-point counterpart in FPGA implementation.

Title: Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation

• Authors: Yurui Chang, Bochuan Cao, Lu Lin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03106
• Pdf URL: https://arxiv.org/pdf/2503.03106
• Copy Paste: [[2503.03106]] Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation(https://arxiv.org/abs/2503.03106)
• Keywords: large language model
• Abstract: While large language models have demonstrated exceptional performance across a wide range of tasks, they remain susceptible to hallucinations -- generating plausible yet factually incorrect contents. Existing methods to mitigating such risk often rely on sampling multiple full-length generations, which introduces significant response latency and becomes ineffective when the model consistently produces hallucinated outputs with high confidence. To address these limitations, we introduce Monitoring Decoding (MD), a novel framework that dynamically monitors the generation process and selectively applies in-process interventions, focusing on revising crucial tokens responsible for hallucinations. Instead of waiting until completion of multiple full-length generations, we identify hallucination-prone tokens during generation using a monitor function, and further refine these tokens through a tree-based decoding strategy. This approach ensures an enhanced factual accuracy and coherence in the generated output while maintaining efficiency. Experimental results demonstrate that MD consistently outperforms self-consistency-based approaches in both effectiveness and efficiency, achieving higher factual accuracy while significantly reducing computational overhead.

Title: SoK: Knowledge is All You Need: Last Mile Delivery for Automated Provenance-based Intrusion Detection with LLMs

• Authors: Wenrui Cheng, Tiantian Zhu, Chunlin Xiong, Haofei Sun, Zijun Wang, Shunan Jing, Mingqi Lv, Yan Chen
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03108
• Pdf URL: https://arxiv.org/pdf/2503.03108
• Copy Paste: [[2503.03108]] SoK: Knowledge is All You Need: Last Mile Delivery for Automated Provenance-based Intrusion Detection with LLMs(https://arxiv.org/abs/2503.03108)
• Keywords: attack, large language model
• Abstract: Recently, provenance-based intrusion detection systems (PIDSes) have been widely proposed for endpoint threat analysis. However, due to the lack of systematic integration and utilization of knowledge, existing PIDSes still require significant manual intervention for practical deployment, making full automation challenging. This paper presents a disruptive innovation by categorizing PIDSes according to the types of knowledge they utilize. In response to the prevalent issue of ``knowledge silos problem'' in existing research, we introduce a novel knowledge-driven provenance-based intrusion detection framework, powered by large language models (LLMs). We also present OmniSec, a best practice system built upon this framework. By integrating attack representation knowledge, threat intelligence knowledge, and benign behavior knowledge, OmniSec outperforms the state-of-the-art approaches on public benchmark datasets. OmniSec is available online at this https URL.

Title: WarmFed: Federated Learning with Warm-Start for Globalization and Personalization Via Personalized Diffusion Models

• Authors: Tao Feng, Jie Zhang, Xiangjian Li, Rong Huang, Huashan Liu, Zhijie Wang
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03110
• Pdf URL: https://arxiv.org/pdf/2503.03110
• Copy Paste: [[2503.03110]] WarmFed: Federated Learning with Warm-Start for Globalization and Personalization Via Personalized Diffusion Models(https://arxiv.org/abs/2503.03110)
• Keywords: privacy, federate, diffusion
• Abstract: Federated Learning (FL) stands as a prominent distributed learning paradigm among multiple clients to achieve a unified global model without privacy leakage. In contrast to FL, Personalized federated learning aims at serving for each client in achieving persoanlized model. However, previous FL frameworks have grappled with a dilemma: the choice between developing a singular global model at the server to bolster globalization or nurturing personalized model at the client to accommodate personalization. Instead of making trade-offs, this paper commences its discourse from the pre-trained initialization, obtaining resilient global information and facilitating the development of both global and personalized models. Specifically, we propose a novel method called WarmFed to achieve this. WarmFed customizes Warm-start through personalized diffusion models, which are generated by local efficient fine-tunining (LoRA). Building upon the Warm-Start, we advance a server-side fine-tuning strategy to derive the global model, and propose a dynamic self-distillation (DSD) to procure more resilient personalized models simultaneously. Comprehensive experiments underscore the substantial gains of our approach across both global and personalized models, achieved within just one-shot and five communication(s).

Title: The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models

• Authors: Zichao Li, Xueru Wen, Jie Lou, Yuqiu Ji, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03122
• Pdf URL: https://arxiv.org/pdf/2503.03122
• Copy Paste: [[2503.03122]] The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models(https://arxiv.org/abs/2503.03122)
• Keywords: robust, large language model
• Abstract: Multimodal Reward Models (MM-RMs) are crucial for aligning Large Language Models (LLMs) with human preferences, particularly as LLMs increasingly interact with multimodal data. However, we find that MM-RMs trained on existing datasets often struggle to generalize to out-of-distribution data due to their reliance on unimodal spurious correlations, primarily text-only shortcuts within the training distribution, which prevents them from leveraging true multimodal reward functions. To address this, we introduce a Shortcut-aware MM-RM learning algorithm that mitigates this issue by dynamically reweighting training samples, shifting the distribution toward better multimodal understanding, and reducing dependence on unimodal spurious correlations. Our experiments demonstrate significant improvements in generalization, downstream task performance, and scalability, establishing a more robust framework for multimodal reward modeling.

Title: Bridging Molecular Graphs and Large Language Models

• Authors: Runze Wang, Mingqi Yang, Yanming Shen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03135
• Pdf URL: https://arxiv.org/pdf/2503.03135
• Copy Paste: [[2503.03135]] Bridging Molecular Graphs and Large Language Models(https://arxiv.org/abs/2503.03135)
• Keywords: large language model
• Abstract: While Large Language Models (LLMs) have shown exceptional generalization capabilities, their ability to process graph data, such as molecular structures, remains limited. To bridge this gap, this paper proposes Graph2Token, an efficient solution that aligns graph tokens to LLM tokens. The key idea is to represent a graph token with the LLM token vocabulary, without fine-tuning the LLM backbone. To achieve this goal, we first construct a molecule-text paired dataset from multisources, including CHEBI and HMDB, to train a graph structure encoder, which reduces the distance between graphs and texts representations in the feature space. Then, we propose a novel alignment strategy that associates a graph token with LLM tokens. To further unleash the potential of LLMs, we collect molecular IUPAC name identifiers, which are incorporated into the LLM prompts. By aligning molecular graphs as special tokens, we can activate LLM generalization ability to molecular few-shot learning. Extensive experiments on molecular classification and regression tasks demonstrate the effectiveness of our proposed Graph2Token.

Title: Convergence Analysis of Federated Learning Methods Using Backward Error Analysis

• Authors: Jinwoo Lim, Suhyun Kim, Soo-Mook Moon
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.03139
• Pdf URL: https://arxiv.org/pdf/2503.03139
• Copy Paste: [[2503.03139]] Convergence Analysis of Federated Learning Methods Using Backward Error Analysis(https://arxiv.org/abs/2503.03139)
• Keywords: federate
• Abstract: Backward error analysis allows finding a modified loss function, which the parameter updates really follow under the influence of an optimization method. The additional loss terms included in this modified function is called implicit regularizer. In this paper, we attempt to find the implicit regularizer for various federated learning algorithms on non-IID data distribution, and explain why each method shows different convergence behavior. We first show that the implicit regularizer of FedAvg disperses the gradient of each client from the average gradient, thus increasing the gradient variance. We also empirically show that the implicit regularizer hampers its convergence. Similarly, we compute the implicit regularizers of FedSAM and SCAFFOLD, and explain why they converge better. While existing convergence analyses focus on pointing out the advantages of FedSAM and SCAFFOLD, our approach can explain their limitations in complex non-convex settings. In specific, we demonstrate that FedSAM can partially remove the bias in the first-order term of the implicit regularizer in FedAvg, whereas SCAFFOLD can fully eliminate the bias in the first-order term, but not in the second-order term. Consequently, the implicit regularizer can provide a useful insight on the convergence behavior of federated learning from a different theoretical perspective.

Title: PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Function Secret Sharing

• Authors: Zhichao You, Xuewen Dong, Ke Cheng, Xutong Mu, Jiaxuan Fu, Shiyang Ma, Qiang Qu, Yulong Shen
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03146
• Pdf URL: https://arxiv.org/pdf/2503.03146
• Copy Paste: [[2503.03146]] PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Function Secret Sharing(https://arxiv.org/abs/2503.03146)
• Keywords: secure, security, privacy, protect, federate, large language model
• Abstract: Fine-tuning large language models (LLMs) raises privacy concerns due to the risk of exposing sensitive training data. Federated learning (FL) mitigates this risk by keeping training samples on local devices, but recent studies show that adversaries can still infer private information from model updates in FL. Additionally, LLM parameters are typically shared publicly during federated fine-tuning, while developers are often reluctant to disclose these parameters, posing further security challenges. Inspired by the above problems, we propose PriFFT, a privacy-preserving federated fine-tuning mechanism, to protect both the model updates and parameters. In PriFFT, clients and the server share model inputs and parameters by secret sharing, performing secure fine-tuning on shared values without accessing plaintext data. Due to considerable LLM parameters, privacy-preserving federated fine-tuning invokes complex secure calculations and requires substantial communication and computation resources. To optimize the efficiency of privacy-preserving federated fine-tuning of LLMs, we introduce function secret-sharing protocols for various operations, including reciprocal calculation, tensor products, natural exponentiation, softmax, hyperbolic tangent, and dropout. The proposed protocols achieve up to 4.02X speed improvement and reduce 7.19X communication overhead compared to the implementation based on existing secret sharing methods. Besides, PriFFT achieves a 2.23X speed improvement and reduces 4.08X communication overhead in privacy-preserving fine-tuning without accuracy drop compared to the existing secret sharing methods.

Title: Partial Convolution Meets Visual Attention

• Authors: Haiduo Huang, Fuwei Yang, Dong Li, Ji Liu, Lu Tian, Jinzhang Peng, Pengju Ren, Emad Barsoum
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03148
• Pdf URL: https://arxiv.org/pdf/2503.03148
• Copy Paste: [[2503.03148]] Partial Convolution Meets Visual Attention(https://arxiv.org/abs/2503.03148)
• Keywords: segmentation
• Abstract: Designing an efficient and effective neural network has remained a prominent topic in computer vision research. Depthwise onvolution (DWConv) is widely used in efficient CNNs or ViTs, but it needs frequent memory access during inference, which leads to low throughput. FasterNet attempts to introduce partial convolution (PConv) as an alternative to DWConv but compromises the accuracy due to underutilized channels. To remedy this shortcoming and consider the redundancy between feature map channels, we introduce a novel Partial visual ATtention mechanism (PAT) that can efficiently combine PConv with visual attention. Our exploration indicates that the partial attention mechanism can completely replace the full attention mechanism and reduce model parameters and FLOPs. Our PAT can derive three types of blocks: Partial Channel-Attention block (PAT_ch), Partial Spatial-Attention block (PAT_sp) and Partial Self-Attention block (PAT_sf). First, PAT_ch integrates the enhanced Gaussian channel attention mechanism to infuse global distribution information into the untouched channels of PConv. Second, we introduce the spatial-wise attention to the MLP layer to further improve model accuracy. Finally, we replace PAT_ch in the last stage with the self-attention mechanism to extend the global receptive field. Building upon PAT, we propose a novel hybrid network family, named PATNet, which achieves superior top-1 accuracy and inference speed compared to FasterNet on ImageNet-1K classification and excel in both detection and segmentation on the COCO dataset. Particularly, our PATNet-T2 achieves 1.3% higher accuracy than FasterNet-T2, while exhibiting 25% higher GPU throughput and 24% lower CPU latency.

Title: DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models

• Authors: YiQiu Guo, Yuchen Yang, Zhe Chen, Pingjie Wang, Yusheng Liao, Ya Zhang, Yanfeng Wang, Yu Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03149
• Pdf URL: https://arxiv.org/pdf/2503.03149
• Copy Paste: [[2503.03149]] DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models(https://arxiv.org/abs/2503.03149)
• Keywords: large language model
• Abstract: The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models' self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD's effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability.

Title: Position: Model Collapse Does Not Mean What You Think

• Authors: Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, Sanmi Koyejo
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.03150
• Pdf URL: https://arxiv.org/pdf/2503.03150
• Copy Paste: [[2503.03150]] Position: Model Collapse Does Not Mean What You Think(https://arxiv.org/abs/2503.03150)
• Keywords: generative
• Abstract: The proliferation of AI-generated content online has fueled concerns over \emph{model collapse}, a degradation in future generative models' performance when trained on synthetic data generated by earlier models. Industry leaders, premier research journals and popular science publications alike have prophesied catastrophic societal consequences stemming from model collapse. In this position piece, we contend this widespread narrative fundamentally misunderstands the scientific evidence. We highlight that research on model collapse actually encompasses eight distinct and at times conflicting definitions of model collapse, and argue that inconsistent terminology within and between papers has hindered building a comprehensive understanding of model collapse. To assess how significantly different interpretations of model collapse threaten future generative models, we posit what we believe are realistic conditions for studying model collapse and then conduct a rigorous assessment of the literature's methodologies through this lens. While we leave room for reasonable disagreement, our analysis of research studies, weighted by how faithfully each study matches real-world conditions, leads us to conclude that certain predicted claims of model collapse rely on assumptions and conditions that poorly match real-world conditions, and in fact several prominent collapse scenarios are readily avoidable. Altogether, this position paper argues that model collapse has been warped from a nuanced multifaceted consideration into an oversimplified threat, and that the evidence suggests specific harms more likely under society's current trajectory have received disproportionately less attention.

Title: SpinML: Customized Synthetic Data Generation for Private Training of Specialized ML Models

• Authors: Jiang Zhang, Rohan Xavier Sequeira, Konstantinos Psounis
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03160
• Pdf URL: https://arxiv.org/pdf/2503.03160
• Copy Paste: [[2503.03160]] SpinML: Customized Synthetic Data Generation for Private Training of Specialized ML Models(https://arxiv.org/abs/2503.03160)
• Keywords: privacy
• Abstract: Specialized machine learning (ML) models tailored to users needs and requests are increasingly being deployed on smart devices with cameras, to provide personalized intelligent services taking advantage of camera data. However, two primary challenges hinder the training of such models: the lack of publicly available labeled data suitable for specialized tasks and the inaccessibility of labeled private data due to concerns about user privacy. To address these challenges, we propose a novel system SpinML, where the server generates customized Synthetic image data to Privately traIN a specialized ML model tailored to the user request, with the usage of only a few sanitized reference images from the user. SpinML offers users fine-grained, object-level control over the reference images, which allows user to trade between the privacy and utility of the generated synthetic data according to their privacy preferences. Through experiments on three specialized model training tasks, we demonstrate that our proposed system can enhance the performance of specialized models without compromising users privacy preferences.

Title: AttackSeqBench: Benchmarking Large Language Models' Understanding of Sequential Patterns in Cyber Attacks

• Authors: Javier Yong, Haokai Ma, Yunshan Ma, Anis Yusof, Zhenkai Liang, Ee-Chien Chang
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03170
• Pdf URL: https://arxiv.org/pdf/2503.03170
• Copy Paste: [[2503.03170]] AttackSeqBench: Benchmarking Large Language Models' Understanding of Sequential Patterns in Cyber Attacks(https://arxiv.org/abs/2503.03170)
• Keywords: security, attack, extraction, large language model
• Abstract: The observations documented in Cyber Threat Intelligence (CTI) reports play a critical role in describing adversarial behaviors, providing valuable insights for security practitioners to respond to evolving threats. Recent advancements of Large Language Models (LLMs) have demonstrated significant potential in various cybersecurity applications, including CTI report understanding and attack knowledge graph construction. While previous works have proposed benchmarks that focus on the CTI extraction ability of LLMs, the sequential characteristic of adversarial behaviors within CTI reports remains largely unexplored, which holds considerable significance in developing a comprehensive understanding of how adversaries operate. To address this gap, we introduce AttackSeqBench, a benchmark tailored to systematically evaluate LLMs' capability to understand and reason attack sequences in CTI reports. Our benchmark encompasses three distinct Question Answering (QA) tasks, each task focuses on the varying granularity in adversarial behavior. To alleviate the laborious effort of QA construction, we carefully design an automated dataset construction pipeline to create scalable and well-formulated QA datasets based on real-world CTI reports. To ensure the quality of our dataset, we adopt a hybrid approach of combining human evaluation and systematic evaluation metrics. We conduct extensive experiments and analysis with both fast-thinking and slow-thinking LLMs, while highlighting their strengths and limitations in analyzing the sequential patterns in cyber attacks. The overarching goal of this work is to provide a benchmark that advances LLM-driven CTI report understanding and fosters its application in real-world cybersecurity operations. Our dataset and code are available at this https URL .

Title: Enhancing Cybersecurity in Critical Infrastructure with LLM-Assisted Explainable IoT Systems

• Authors: Ashutosh Ghimire, Ghazal Ghajari, Karma Gurung, Love K. Sah, Fathi Amsaad
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03180
• Pdf URL: https://arxiv.org/pdf/2503.03180
• Copy Paste: [[2503.03180]] Enhancing Cybersecurity in Critical Infrastructure with LLM-Assisted Explainable IoT Systems(https://arxiv.org/abs/2503.03180)
• Keywords: security, interpretability, large language model
• Abstract: Ensuring the security of critical infrastructure has become increasingly vital with the proliferation of Internet of Things (IoT) systems. However, the heterogeneous nature of IoT data and the lack of human-comprehensible insights from anomaly detection models remain significant challenges. This paper presents a hybrid framework that combines numerical anomaly detection using Autoencoders with Large Language Models (LLMs) for enhanced preprocessing and interpretability. Two preprocessing approaches are implemented: a traditional method utilizing Principal Component Analysis (PCA) to reduce dimensionality and an LLM-assisted method where GPT-4 dynamically recommends feature selection, transformation, and encoding strategies. Experimental results on the KDDCup99 10% corrected dataset demonstrate that the LLM-assisted preprocessing pipeline significantly improves anomaly detection performance. The macro-average F1 score increased from 0.49 in the traditional PCA-based approach to 0.98 with LLM-driven insights. Additionally, the LLM generates natural language explanations for detected anomalies, providing contextual insights into their causes and implications. This framework highlights the synergy between numerical AI models and LLMs, delivering an accurate, interpretable, and efficient solution for IoT cybersecurity in critical infrastructure.

Title: DSPNet: Dual-vision Scene Perception for Robust 3D Question Answering

• Authors: Jingzhou Luo, Yang Liu, Weixing Chen, Zhen Li, Yaowei Wang, Guanbin Li, Liang Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03190
• Pdf URL: https://arxiv.org/pdf/2503.03190
• Copy Paste: [[2503.03190]] DSPNet: Dual-vision Scene Perception for Robust 3D Question Answering(https://arxiv.org/abs/2503.03190)
• Keywords: robust
• Abstract: 3D Question Answering (3D QA) requires the model to comprehensively understand its situated 3D scene described by the text, then reason about its surrounding environment and answer a question under that situation. However, existing methods usually rely on global scene perception from pure 3D point clouds and overlook the importance of rich local texture details from multi-view images. Moreover, due to the inherent noise in camera poses and complex occlusions, there exists significant feature degradation and reduced feature robustness problems when aligning 3D point cloud with multi-view images. In this paper, we propose a Dual-vision Scene Perception Network (DSPNet), to comprehensively integrate multi-view and point cloud features to improve robustness in 3D QA. Our Text-guided Multi-view Fusion (TGMF) module prioritizes image views that closely match the semantic content of the text. To adaptively fuse back-projected multi-view images with point cloud features, we design the Adaptive Dual-vision Perception (ADVP) module, enhancing 3D scene comprehension. Additionally, our Multimodal Context-guided Reasoning (MCGR) module facilitates robust reasoning by integrating contextual information across visual and linguistic modalities. Experimental results on SQA3D and ScanQA datasets demonstrate the superiority of our DSPNet. Codes will be available at this https URL.

Title: Structured Outputs Enable General-Purpose LLMs to be Medical Experts

• Authors: Guangfu Guo, Kai Zhang, Bryan Hoo, Yujun Cai, Xiaoqian Lu, Nanyun Peng, Yiwei Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03194
• Pdf URL: https://arxiv.org/pdf/2503.03194
• Copy Paste: [[2503.03194]] Structured Outputs Enable General-Purpose LLMs to be Medical Experts(https://arxiv.org/abs/2503.03194)
• Keywords: large language model
• Abstract: Medical question-answering (QA) is a critical task for evaluating how effectively large language models (LLMs) encode clinical knowledge and assessing their potential applications in medicine. Despite showing promise on multiple-choice tests, LLMs frequently struggle with open-ended medical questions, producing responses with dangerous hallucinations or lacking comprehensive coverage of critical aspects. Existing approaches attempt to address these challenges through domain-specific fine-tuning, but this proves resource-intensive and difficult to scale across models. To improve the comprehensiveness and factuality of medical responses, we propose a novel approach utilizing structured medical reasoning. Our method guides LLMs through an seven-step cognitive process inspired by clinical diagnosis, enabling more accurate and complete answers without additional training. Experiments on the MedLFQA benchmark demonstrate that our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models. Notably, this improvement transfers to smaller models, highlighting the method's efficiency and scalability. Our code and datasets are available.

Title: SpiritSight Agent: Advanced GUI Agent with One Look

• Authors: Zhiyuan Huang, Ziming Cheng, Junting Pan, Zhaohui Hou, Mingjie Zhan
• Subjects: cs.CV, cs.HC, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03196
• Pdf URL: https://arxiv.org/pdf/2503.03196
• Copy Paste: [[2503.03196]] SpiritSight Agent: Advanced GUI Agent with One Look(https://arxiv.org/abs/2503.03196)
• Keywords: robust
• Abstract: Graphical User Interface (GUI) agents show amazing abilities in assisting human-computer interaction, automating human user's navigation on digital devices. An ideal GUI agent is expected to achieve high accuracy, low latency, and compatibility for different GUI platforms. Recent vision-based approaches have shown promise by leveraging advanced Vision Language Models (VLMs). While they generally meet the requirements of compatibility and low latency, these vision-based GUI agents tend to have low accuracy due to their limitations in element grounding. To address this issue, we propose $\textbf{SpiritSight}$, a vision-based, end-to-end GUI agent that excels in GUI navigation tasks across various GUI platforms. First, we create a multi-level, large-scale, high-quality GUI dataset called $\textbf{GUI-Lasagne}$ using scalable methods, empowering SpiritSight with robust GUI understanding and grounding capabilities. Second, we introduce the $\textbf{Universal Block Parsing (UBP)}$ method to resolve the ambiguity problem in dynamic high-resolution of visual inputs, further enhancing SpiritSight's ability to ground GUI objects. Through these efforts, SpiritSight agent outperforms other advanced methods on diverse GUI benchmarks, demonstrating its superior capability and compatibility in GUI navigation tasks. Models are available at $\href{this https URL}{this\ URL}$.

Title: Directly Follows Graphs Go Predictive Process Monitoring With Graph Neural Networks

• Authors: Attila Lischka, Simon Rauch, Oliver Stritzel
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03197
• Pdf URL: https://arxiv.org/pdf/2503.03197
• Copy Paste: [[2503.03197]] Directly Follows Graphs Go Predictive Process Monitoring With Graph Neural Networks(https://arxiv.org/abs/2503.03197)
• Keywords: transformer
• Abstract: In the past years, predictive process monitoring (PPM) techniques based on artificial neural networks have evolved as a method to monitor the future behavior of business processes. Existing approaches mostly focus on interpreting the processes as sequences, so-called traces, and feeding them to neural architectures designed to operate on sequential data such as recurrent neural networks (RNNs) or transformers. In this study, we investigate an alternative way to perform PPM: by transforming each process in its directly-follows-graph (DFG) representation we are able to apply graph neural networks (GNNs) for the prediction tasks. By this, we aim to develop models that are more suitable for complex processes that are long and contain an abundance of loops. In particular, we present different ways to create DFG representations depending on the particular GNN we use. The tested GNNs range from classical node-based to novel edge-based architectures. Further, we investigate the possibility of using multi-graphs. By these steps, we aim to design graph representations that minimize the information loss when transforming traces into graphs.

Title: Transformer-Based Spatio-Temporal Association of Apple Fruitlets

• Authors: Harry Freeman, George Kantor
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03200
• Pdf URL: https://arxiv.org/pdf/2503.03200
• Copy Paste: [[2503.03200]] Transformer-Based Spatio-Temporal Association of Apple Fruitlets(https://arxiv.org/abs/2503.03200)
• Keywords: transformer
• Abstract: In this paper, we present a transformer-based method to spatio-temporally associate apple fruitlets in stereo-images collected on different days and from different camera poses. State-of-the-art association methods in agriculture are dedicated towards matching larger crops using either high-resolution point clouds or temporally stable features, which are both difficult to obtain for smaller fruit in the field. To address these challenges, we propose a transformer-based architecture that encodes the shape and position of each fruitlet, and propagates and refines these features through a series of transformer encoder layers with alternating self and cross-attention. We demonstrate that our method is able to achieve an F1-score of 92.4% on data collected in a commercial apple orchard and outperforms all baselines and ablations.

Title: Towards Robust Universal Information Extraction: Benchmark, Evaluation, and Solution

• Authors: Jizhao Zhu, Akang Shi, Zixuan Li, Long Bai, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03201
• Pdf URL: https://arxiv.org/pdf/2503.03201
• Copy Paste: [[2503.03201]] Towards Robust Universal Information Extraction: Benchmark, Evaluation, and Solution(https://arxiv.org/abs/2503.03201)
• Keywords: robust, extraction, large language model
• Abstract: In this paper, we aim to enhance the robustness of Universal Information Extraction (UIE) by introducing a new benchmark dataset, a comprehensive evaluation, and a feasible solution. Existing robust benchmark datasets have two key limitations: 1) They generate only a limited range of perturbations for a single Information Extraction (IE) task, which fails to evaluate the robustness of UIE models effectively; 2) They rely on small models or handcrafted rules to generate perturbations, often resulting in unnatural adversarial examples. Considering the powerful generation capabilities of Large Language Models (LLMs), we introduce a new benchmark dataset for Robust UIE, called RUIE-Bench, which utilizes LLMs to generate more diverse and realistic perturbations across different IE tasks. Based on this dataset, we comprehensively evaluate existing UIE models and reveal that both LLM-based models and other models suffer from significant performance drops. To improve robustness and reduce training costs, we propose a data-augmentation solution that dynamically selects hard samples for iterative training based on the model's inference loss. Experimental results show that training with only \textbf{15\%} of the data leads to an average \textbf{7.5\%} relative performance improvement across three IE tasks.

Title: Variance-Aware Loss Scheduling for Multimodal Alignment in Low-Data Settings

• Authors: Sneh Pillai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03202
• Pdf URL: https://arxiv.org/pdf/2503.03202
• Copy Paste: [[2503.03202]] Variance-Aware Loss Scheduling for Multimodal Alignment in Low-Data Settings(https://arxiv.org/abs/2503.03202)
• Keywords: robust
• Abstract: Training vision-language models for image-text alignment typically requires large datasets to achieve robust performance. In low-data scenarios, standard contrastive learning can struggle to align modalities effectively due to overfitting and unstable training dynamics. In this paper, we propose a variance-aware loss scheduling approach that dynamically adjusts the weighting of the contrastive loss based on the statistical variability (uncertainty) in the model's alignment predictions. Using a subset of the Flickr8k image-caption dataset to simulate limited data conditions, we demonstrate that our approach improves image-text retrieval accuracy compared to a fixed-weight baseline. We also compare against other adaptive weighting strategies (using output entropy and cosine similarity spread) and find that variance-aware scheduling provides the best overall trade-off. Qualitatively, our method yields more distinct multimodal embeddings as shown by t-SNE visualizations. Moreover, in a stress test with noise-injected captions and images, the variance-guided loss proves more robust, maintaining higher recall when random perturbations are introduced. These results highlight the benefit of adaptive loss weighting for multimodal alignment in low-data regimes.

Title: Find Matching Faces Based On Face Parameters

• Authors: Setu A. Bhatt, Harshadkumar B. Prajapati, Vipul K. Dabhi, Ankush Tyagi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03204
• Pdf URL: https://arxiv.org/pdf/2503.03204
• Copy Paste: [[2503.03204]] Find Matching Faces Based On Face Parameters(https://arxiv.org/abs/2503.03204)
• Keywords: extraction
• Abstract: This paper presents an innovative approach that enables the user to find matching faces based on the user-selected face parameters. Through gradio-based user interface, the users can interactively select the face parameters they want in their desired partner. These user-selected face parameters are transformed into a text prompt which is used by the Text-To-Image generation model to generate a realistic face image. Further, the generated image along with the images downloaded from the this http URL are processed through face detection and feature extraction model, which results in high dimensional vector embedding of 512 dimensions. The vector embeddings generated from the downloaded images are stored into vector database. Now, the similarity search is carried out between the vector embedding of generated image and the stored vector embeddings. As a result, it displays the top five similar faces based on the user-selected face parameters. This contribution holds a significant potential to turn into a high-quality personalized face matching tool.

Title: MA-LoT: Multi-Agent Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving

• Authors: Ruida Wang, Rui Pan, Yuxin Li, Jipeng Zhang, Yizhen Jia, Shizhe Diao, Renjie Pi, Junjie Hu, Tong Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03205
• Pdf URL: https://arxiv.org/pdf/2503.03205
• Copy Paste: [[2503.03205]] MA-LoT: Multi-Agent Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving(https://arxiv.org/abs/2503.03205)
• Keywords: large language model
• Abstract: Solving mathematical problems using computer-verifiable languages like Lean has significantly impacted mathematical and computer science communities. State-of-the-art methods utilize single Large Language Models (LLMs) as agents or provers to either generate complete proof or perform tree searches. However, single-agent methods inherently lack a structured way to combine high-level reasoning in Natural Language (NL) with Formal Language (FL) verification feedback. To solve these issues, we propose MA-LoT: Multi-Agent Lean-based Long Chain-of-Thought framework, (to the best of our knowledge), the first multi-agent framework for Lean4 theorem proving that balance high-level NL reasoning and FL verification in Long CoT. Using this structured interaction, our approach enables deeper insights and long-term coherence in proof generation, with which past methods struggle. We do this by leveraging emergent formal reasoning ability in Long CoT using our novel LoT-Transfer Learning training-inference pipeline. Extensive experiments show that our framework achieves 54.51% accuracy rate on the Lean4 version of MiniF2F-Test dataset, largely outperforming GPT-4 (22.95%), single-agent tree search (InternLM-Step-Prover, 50.70%), and whole-proof generation (DeepSeek-Prover-v1.5, 48.36%) baselines. Furthermore, our findings highlight the potential of combining Long CoT with formal verification for a more insightful generation in a broader perspective.

Title: An Analytical Theory of Power Law Spectral Bias in the Learning Dynamics of Diffusion Models

• Authors: Binxu Wang
• Subjects: cs.LG, cs.CV, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.03206
• Pdf URL: https://arxiv.org/pdf/2503.03206
• Copy Paste: [[2503.03206]] An Analytical Theory of Power Law Spectral Bias in the Learning Dynamics of Diffusion Models(https://arxiv.org/abs/2503.03206)
• Keywords: robust, diffusion, generative
• Abstract: We developed an analytical framework for understanding how the learned distribution evolves during diffusion model training. Leveraging the Gaussian equivalence principle, we derived exact solutions for the gradient-flow dynamics of weights in one- or two-layer linear denoiser settings with arbitrary data. Remarkably, these solutions allowed us to derive the generated distribution in closed form and its KL divergence through training. These analytical results expose a pronounced power-law spectral bias, i.e., for weights and distributions, the convergence time of a mode follows an inverse power law of its variance. Empirical experiments on both Gaussian and image datasets demonstrate that the power-law spectral bias remains robust even when using deeper or convolutional architectures. Our results underscore the importance of the data covariance in dictating the order and rate at which diffusion models learn different modes of the data, providing potential explanations for why earlier stopping could lead to incorrect details in image generative models.

Title: Mocap-2-to-3: Lifting 2D Diffusion-Based Pretrained Models for 3D Motion Capture

• Authors: Zhumei Wang, Zechen Hu, Ruoxi Guo, Huaijin Pi, Ziyong Feng, Sida Peng, Xiaowei Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03222
• Pdf URL: https://arxiv.org/pdf/2503.03222
• Copy Paste: [[2503.03222]] Mocap-2-to-3: Lifting 2D Diffusion-Based Pretrained Models for 3D Motion Capture(https://arxiv.org/abs/2503.03222)
• Keywords: diffusion, generative
• Abstract: Recovering absolute poses in the world coordinate system from monocular views presents significant challenges. Two primary issues arise in this context. Firstly, existing methods rely on 3D motion data for training, which requires collection in limited environments. Acquiring such 3D labels for new actions in a timely manner is impractical, severely restricting the model's generalization capabilities. In contrast, 2D poses are far more accessible and easier to obtain. Secondly, estimating a person's absolute position in metric space from a single viewpoint is inherently more complex. To address these challenges, we introduce Mocap-2-to-3, a novel framework that decomposes intricate 3D motions into 2D poses, leveraging 2D data to enhance 3D motion reconstruction in diverse scenarios and accurately predict absolute positions in the world coordinate system. We initially pretrain a single-view diffusion model with extensive 2D data, followed by fine-tuning a multi-view diffusion model for view consistency using publicly available 3D data. This strategy facilitates the effective use of large-scale 2D data. Additionally, we propose an innovative human motion representation that decouples local actions from global movements and encodes geometric priors of the ground, ensuring the generative model learns accurate motion priors from 2D data. During inference, this allows for the gradual recovery of global movements, resulting in more plausible positioning. We evaluate our model's performance on real-world datasets, demonstrating superior accuracy in motion and absolute human positioning compared to state-of-the-art methods, along with enhanced generalization and scalability. Our code will be made publicly available.

Title: Targeted Distillation for Sentiment Analysis

• Authors: Yice Zhang, Guangyu Xie, Jingjie Lin, Jianzhu Bao, Qianlong Wang, Xi Zeng, Ruifeng Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03225
• Pdf URL: https://arxiv.org/pdf/2503.03225
• Copy Paste: [[2503.03225]] Targeted Distillation for Sentiment Analysis(https://arxiv.org/abs/2503.03225)
• Keywords: large language model
• Abstract: This paper presents a compact model that achieves strong sentiment analysis capabilities through targeted distillation from advanced large language models (LLMs). Our methodology decouples the distillation target into two key components: sentiment-related knowledge and task alignment. To transfer these components, we propose a two-stage distillation framework. The first stage, knowledge-driven distillation (\textsc{KnowDist}), transfers sentiment-related knowledge to enhance fundamental sentiment analysis capabilities. The second stage, in-context learning distillation (\textsc{ICLDist}), transfers task-specific prompt-following abilities to optimize task alignment. For evaluation, we introduce \textsc{SentiBench}, a comprehensive sentiment analysis benchmark comprising 3 task categories across 12 datasets. Experiments on this benchmark demonstrate that our model effectively balances model size and performance, showing strong competitiveness compared to existing small-scale LLMs.

Title: Path-Adaptive Matting for Efficient Inference Under Various Computational Cost Constraints

• Authors: Qinglin Liu, Zonglin Li, Xiaoqian Lv, Xin Sun, Ru Li, Shengping Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03228
• Pdf URL: https://arxiv.org/pdf/2503.03228
• Copy Paste: [[2503.03228]] Path-Adaptive Matting for Efficient Inference Under Various Computational Cost Constraints(https://arxiv.org/abs/2503.03228)
• Keywords: robust
• Abstract: In this paper, we explore a novel image matting task aimed at achieving efficient inference under various computational cost constraints, specifically FLOP limitations, using a single matting network. Existing matting methods which have not explored scalable architectures or path-learning strategies, fail to tackle this challenge. To overcome these limitations, we introduce Path-Adaptive Matting (PAM), a framework that dynamically adjusts network paths based on image contexts and computational cost constraints. We formulate the training of the computational cost-constrained matting network as a bilevel optimization problem, jointly optimizing the matting network and the path estimator. Building on this formalization, we design a path-adaptive matting architecture by incorporating path selection layers and learnable connect layers to estimate optimal paths and perform efficient inference within a unified network. Furthermore, we propose a performance-aware path-learning strategy to generate path labels online by evaluating a few paths sampled from the prior distribution of optimal paths and network estimations, enabling robust and efficient online path learning. Experiments on five image matting datasets demonstrate that the proposed PAM framework achieves competitive performance across a range of computational cost constraints.

Title: FANS -- Formal Answer Selection for Natural Language Math Reasoning Using Lean4

• Authors: Jiarui Yao, Ruida Wang, Tong Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03238
• Pdf URL: https://arxiv.org/pdf/2503.03238
• Copy Paste: [[2503.03238]] FANS -- Formal Answer Selection for Natural Language Math Reasoning Using Lean4(https://arxiv.org/abs/2503.03238)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have displayed astonishing abilities in various tasks, especially in text generation, classification, question answering, etc. However, the reasoning ability of LLMs still faces many debates. The inherent ambiguity of Natural Language (NL) limits LLMs' ability to perform verifiable reasoning, making its answers lack coherence and trustworthy support. To tackle the above problems, we propose a novel framework named FANS: Formal ANswer Selection for Natural Language Math Reasoning Using Lean4. To the best of our knowledge, it is the first framework that utilizes Lean4 to enhance LLMs' NL math reasoning ability. In particular, given an NL math question and LLM-generated answers, FANS first translates it into Lean4 theorem statements. Then it tries to prove it using a Lean4 prover and verify it by Lean4. Finally, it uses the FL result to assist in answer selection. It enhances LLMs' NL math ability in providing a computer-verifiable solution for its correct answer and proposes an alternative method for answer selection beyond the reward model. Extensive experiments indicate the effectiveness of our framework. It can improve the accuracy rate of reward model enhanced LLMs in the MATH-500 dataset by at most 1.91% and AMC-23 by at most 8.33% on strong reward-model baselines. In some particular fields like number theory that Lean4 experts in, we can even select all correct solutions. The qualitative analysis also shows our framework can make NL results formally backed by Lean4 proofs. As a pioneering work in the corresponding field, we will open-source all our models and datasets to further boost the development of the field.

Title: Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care

• Authors: Jorge García-Torres, Øyvind Meinich-Bache, Sara Brunner, Siren Rettedal, Vilde Kolstad, Kjersti Engan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03244
• Pdf URL: https://arxiv.org/pdf/2503.03244
• Copy Paste: [[2503.03244]] Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care(https://arxiv.org/abs/2503.03244)
• Keywords: robust
• Abstract: Around 10% of newborns require some help to initiate breathing, and 5\% need ventilation assistance. Accurate Time of Birth (ToB) documentation is essential for optimizing neonatal care, as timely interventions are vital for proper resuscitation. However, current clinical methods for recording ToB often rely on manual processes, which can be prone to inaccuracies. In this study, we present a novel two-stream fusion system that combines the power of image and video analysis to accurately detect the ToB from thermal recordings in the delivery room and operating theater. By integrating static and dynamic streams, our approach captures richer birth-related spatiotemporal features, leading to more robust and precise ToB estimation. We demonstrate that this synergy between data modalities enhances performance over single-stream approaches. Our system achieves 95.7% precision and 84.8% recall in detecting birth within short video clips. Additionally, with the help of a score aggregation module, it successfully identifies ToB in 100% of test cases, with a median absolute error of 2 seconds and an absolute mean deviation of 4.5 seconds compared to manual annotations.

Title: Less is more? Rewards in RL for Cyber Defence

• Authors: Elizabeth Bates, Chris Hicks, Vasilios Mavroudis
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03245
• Pdf URL: https://arxiv.org/pdf/2503.03245
• Copy Paste: [[2503.03245]] Less is more? Rewards in RL for Cyber Defence(https://arxiv.org/abs/2503.03245)
• Keywords: robust
• Abstract: The last few years has seen an explosion of interest in autonomous cyber defence agents based on deep reinforcement learning. Such agents are typically trained in a cyber gym environment, also known as a cyber simulator, at least 32 of which have already been built. Most, if not all cyber gyms provide dense "scaffolded" reward functions which combine many penalties or incentives for a range of (un)desirable states and costly actions. Whilst dense rewards help alleviate the challenge of exploring complex environments, yielding seemingly effective strategies from relatively few environment steps; they are also known to bias the solutions an agent can find, potentially towards suboptimal solutions. Sparse rewards could offer preferable or more effective solutions and have been overlooked by cyber gyms to date. In this work we set out to evaluate whether sparse reward functions might enable training more effective cyber defence agents. Towards this goal we first break down several evaluation limitations in existing work by proposing a ground truth evaluation score that goes beyond the standard RL paradigm used to train and evaluate agents. By adapting a well-established cyber gym to accommodate our methodology and ground truth score, we propose and evaluate two sparse reward mechanisms and compare them with a typical dense reward. Our evaluation considers a range of network sizes, from 2 to 50 nodes, and both reactive and proactive defensive actions. Our results show that sparse rewards, particularly positive reinforcement for an uncompromised network state, enable the training of more effective cyber defence agents. Furthermore, we show that sparse rewards provide more stable training than dense rewards, and that both effectiveness and training stability are robust to a variety of cyber environment considerations.

Title: Exploring the Potential of Large Language Models as Predictors in Dynamic Text-Attributed Graphs

• Authors: Runlin Lei, Jiarui Ji, Haipeng Ding, Lu Yi, Zhewei Wei, Yongchao Liu, Chuntao Hong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03258
• Pdf URL: https://arxiv.org/pdf/2503.03258
• Copy Paste: [[2503.03258]] Exploring the Potential of Large Language Models as Predictors in Dynamic Text-Attributed Graphs(https://arxiv.org/abs/2503.03258)
• Keywords: large language model
• Abstract: With the rise of large language models (LLMs), there has been growing interest in Graph Foundation Models (GFMs) for graph-based tasks. By leveraging LLMs as predictors, GFMs have demonstrated impressive generalizability across various tasks and datasets. However, existing research on LLMs as predictors has predominantly focused on static graphs, leaving their potential in dynamic graph prediction unexplored. In this work, we pioneer using LLMs for predictive tasks on dynamic graphs. We identify two key challenges: the constraints imposed by context length when processing large-scale historical data and the significant variability in domain characteristics, both of which complicate the development of a unified predictor. To address these challenges, we propose the GraphAgent-Dynamic (GAD) Framework, a multi-agent system that leverages collaborative LLMs. In contrast to using a single LLM as the predictor, GAD incorporates global and local summary agents to generate domain-specific knowledge, enhancing its transferability across domains. Additionally, knowledge reflection agents enable adaptive updates to GAD's knowledge, maintaining a unified and self-consistent architecture. In experiments, GAD demonstrates performance comparable to or even exceeds that of full-supervised graph neural networks without dataset-specific training. Finally, to enhance the task-specific performance of LLM-based predictors, we discuss potential improvements, such as dataset-specific fine-tuning to LLMs. By developing tailored strategies for different tasks, we provide new insights for the future design of LLM-based predictors.

Title: Can Frontier LLMs Replace Annotators in Biomedical Text Mining? Analyzing Challenges and Exploring Solutions

• Authors: Yichong Zhao, Susumu Goto
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03261
• Pdf URL: https://arxiv.org/pdf/2503.03261
• Copy Paste: [[2503.03261]] Can Frontier LLMs Replace Annotators in Biomedical Text Mining? Analyzing Challenges and Exploring Solutions(https://arxiv.org/abs/2503.03261)
• Keywords: large language model
• Abstract: Large language models (LLMs) can perform various natural language processing (NLP) tasks through in-context learning without relying on supervised data. However, multiple previous studies have reported suboptimal performance of LLMs in biological text mining. By analyzing failure patterns in these evaluations, we identified three primary challenges for LLMs in biomedical corpora: (1) LLMs fail to learn implicit dataset-specific nuances from supervised data, (2) The common formatting requirements of discriminative tasks limit the reasoning capabilities of LLMs particularly for LLMs that lack test-time compute, and (3) LLMs struggle to adhere to annotation guidelines and match exact schemas, which hinders their ability to understand detailed annotation requirements which is essential in biomedical annotation workflow. To address these challenges, we experimented with prompt engineering techniques targeted to the above issues, and developed a pipeline that dynamically extracts instructions from annotation guidelines. Our findings show that frontier LLMs can approach or surpass the performance of state-of-the-art (SOTA) BERT-based models with minimal reliance on manually annotated data and without fine-tuning. Furthermore, we performed model distillation on a closed-source LLM, demonstrating that a BERT model trained exclusively on synthetic data annotated by LLMs can also achieve a practical performance. Based on these results, we explored the feasibility of partially replacing manual annotation with LLMs in production scenarios for biomedical text mining.

Title: Optimizing for the Shortest Path in Denoising Diffusion Model

• Authors: Ping Chen, Xingpeng Zhang, Zhaoxiang Liu, Huan Hu, Xiang Liu, Kai Wang, Min Wang, Yanlin Qian, Shiguo Lian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03265
• Pdf URL: https://arxiv.org/pdf/2503.03265
• Copy Paste: [[2503.03265]] Optimizing for the Shortest Path in Denoising Diffusion Model(https://arxiv.org/abs/2503.03265)
• Keywords: diffusion
• Abstract: In this research, we propose a novel denoising diffusion model based on shortest-path modeling that optimizes residual propagation to enhance both denoising efficiency and this http URL on Denoising Diffusion Implicit Models (DDIM) and insights from graph theory, our model, termed the Shortest Path Diffusion Model (ShortDF), treats the denoising process as a shortest-path problem aimed at minimizing reconstruction error. By optimizing the initial residuals, we improve the efficiency of the reverse diffusion process and the quality of the generated this http URL experiments on multiple standard benchmarks demonstrate that ShortDF significantly reduces diffusion time (or steps) while enhancing the visual fidelity of generated samples compared to prior this http URL work, we suppose, paves the way for interactive diffusion-based applications and establishes a foundation for rapid data generation. Code is available at this https URL.

Title: Quantum-Inspired Privacy-Preserving Federated Learning Framework for Secure Dementia Classification

• Authors: Gazi Tanbhir, Md. Farhan Shahriyar
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.03267
• Pdf URL: https://arxiv.org/pdf/2503.03267
• Copy Paste: [[2503.03267]] Quantum-Inspired Privacy-Preserving Federated Learning Framework for Secure Dementia Classification(https://arxiv.org/abs/2503.03267)
• Keywords: secure, security, privacy, protect, attack, robust, federate
• Abstract: Dementia, a neurological disorder impacting millions globally, presents significant challenges in diagnosis and patient care. With the rise of privacy concerns and security threats in healthcare, federated learning (FL) has emerged as a promising approach to enable collaborative model training across decentralized datasets without exposing sensitive patient information. However, FL remains vulnerable to advanced security breaches such as gradient inversion and eavesdropping attacks. This paper introduces a novel framework that integrates federated learning with quantum-inspired encryption techniques for dementia classification, emphasizing privacy preservation and security. Leveraging quantum key distribution (QKD), the framework ensures secure transmission of model weights, protecting against unauthorized access and interception during training. The methodology utilizes a convolutional neural network (CNN) for dementia classification, with federated training conducted across distributed healthcare nodes, incorporating QKD-encrypted weight sharing to secure the aggregation process. Experimental evaluations conducted on MRI data from the OASIS dataset demonstrate that the proposed framework achieves identical accuracy levels to a baseline model while enhancing data security and reducing loss by almost 1% compared to the classical baseline model. The framework offers significant implications for democratizing access to AI-driven dementia diagnostics in low- and middle-income countries, addressing critical resource and privacy constraints. This work contributes a robust, scalable, and secure federated learning solution for healthcare applications, paving the way for broader adoption of quantum-inspired techniques in AI-driven medical research.

Title: Conformal Transformations for Symmetric Power Transformers

• Authors: Saurabh Kumar, Jacob Buckman, Carles Gelada, Sean Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03269
• Pdf URL: https://arxiv.org/pdf/2503.03269
• Copy Paste: [[2503.03269]] Conformal Transformations for Symmetric Power Transformers(https://arxiv.org/abs/2503.03269)
• Keywords: robust, transformer
• Abstract: Transformers with linear attention offer significant computational advantages over softmax-based transformers but often suffer from degraded performance. The symmetric power (sympow) transformer, a particular type of linear transformer, addresses some of this performance gap by leveraging symmetric tensor embeddings, achieving comparable performance to softmax transformers. However, the finite capacity of the recurrent state in sympow transformers limits their ability to retain information, leading to performance degradation when scaling the training or evaluation context length. To address this issue, we propose the conformal-sympow transformer, which dynamically frees up capacity using data-dependent multiplicative gating and adaptively stores information using data-dependent rotary embeddings. Preliminary experiments on the LongCrawl64 dataset demonstrate that conformal-sympow overcomes the limitations of sympow transformers, achieving robust performance across scaled training and evaluation contexts.

Title: Reduced Spatial Dependency for More General Video-level Deepfake Detection

• Authors: Beilin Chu, Xuan Xu, Yufei Zhang, Weike You, Linna Zhou
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03270
• Pdf URL: https://arxiv.org/pdf/2503.03270
• Copy Paste: [[2503.03270]] Reduced Spatial Dependency for More General Video-level Deepfake Detection(https://arxiv.org/abs/2503.03270)
• Keywords: extraction, transformer
• Abstract: As one of the prominent AI-generated content, Deepfake has raised significant safety concerns. Although it has been demonstrated that temporal consistency cues offer better generalization capability, existing methods based on CNNs inevitably introduce spatial bias, which hinders the extraction of intrinsic temporal features. To address this issue, we propose a novel method called Spatial Dependency Reduction (SDR), which integrates common temporal consistency features from multiple spatially-perturbed clusters, to reduce the dependency of the model on spatial information. Specifically, we design multiple Spatial Perturbation Branch (SPB) to construct spatially-perturbed feature clusters. Subsequently, we utilize the theory of mutual information and propose a Task-Relevant Feature Integration (TRFI) module to capture temporal features residing in similar latent space from these clusters. Finally, the integrated feature is fed into a temporal transformer to capture long-range dependencies. Extensive benchmarks and ablation studies demonstrate the effectiveness and rationale of our approach.

Title: Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients

• Authors: Li Lun, Kunyu Feng, Qinglong Ni, Ling Liang, Yuan Wang, Ying Li, Dunshan Yu, Xiaoxin Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03272
• Pdf URL: https://arxiv.org/pdf/2503.03272
• Copy Paste: [[2503.03272]] Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients(https://arxiv.org/abs/2503.03272)
• Keywords: attack, robust
• Abstract: Spiking neural networks (SNNs) have shown their competence in handling spatial-temporal event-based data with low energy consumption. Similar to conventional artificial neural networks (ANNs), SNNs are also vulnerable to gradient-based adversarial attacks, wherein gradients are calculated by spatial-temporal back-propagation (STBP) and surrogate gradients (SGs). However, the SGs may be invisible for an inference-only model as they do not influence the inference results, and current gradient-based attacks are ineffective for binary dynamic images captured by the dynamic vision sensor (DVS). While some approaches addressed the issue of invisible SGs through universal SGs, their SGs lack a correlation with the victim model, resulting in sub-optimal performance. Moreover, the imperceptibility of existing SNN-based binary attacks is still insufficient. In this paper, we introduce an innovative potential-dependent surrogate gradient (PDSG) method to establish a robust connection between the SG and the model, thereby enhancing the adaptability of adversarial attacks across various models with invisible SGs. Additionally, we propose the sparse dynamic attack (SDA) to effectively attack binary dynamic images. Utilizing a generation-reduction paradigm, SDA can fully optimize the sparsity of adversarial perturbations. Experimental results demonstrate that our PDSG and SDA outperform state-of-the-art SNN-based attacks across various models and datasets. Specifically, our PDSG achieves 100% attack success rate on ImageNet, and our SDA obtains 82% attack success rate by modifying only 0.24% of the pixels on CIFAR10DVS. The code is available at this https URL .

Title: TrafficKAN-GCN: Graph Convolutional-based Kolmogorov-Arnold Network for Traffic Flow Optimization

• Authors: Jiayi Zhang, Yiming Zhang, Yuan Zheng, Yuchen Wang, Jinjiang You, Yuchen Xu, Wenxing Jiang, Soumyabrata Dev
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03276
• Pdf URL: https://arxiv.org/pdf/2503.03276
• Copy Paste: [[2503.03276]] TrafficKAN-GCN: Graph Convolutional-based Kolmogorov-Arnold Network for Traffic Flow Optimization(https://arxiv.org/abs/2503.03276)
• Keywords: robust, transformer
• Abstract: Urban traffic optimization is critical for improving transportation efficiency and alleviating congestion, particularly in large-scale dynamic networks. Traditional methods, such as Dijkstra's and Floyd's algorithms, provide effective solutions in static settings, but they struggle with the spatial-temporal complexity of real-world traffic flows. In this work, we propose TrafficKAN-GCN, a hybrid deep learning framework combining Kolmogorov-Arnold Networks (KAN) with Graph Convolutional Networks (GCN), designed to enhance urban traffic flow optimization. By integrating KAN's adaptive nonlinear function approximation with GCN's spatial graph learning capabilities, TrafficKAN-GCN captures both complex traffic patterns and topological dependencies. We evaluate the proposed framework using real-world traffic data from the Baltimore Metropolitan area. Compared with baseline models such as MLP-GCN, standard GCN, and Transformer-based approaches, TrafficKAN-GCN achieves competitive prediction accuracy while demonstrating improved robustness in handling noisy and irregular traffic data. Our experiments further highlight the framework's ability to redistribute traffic flow, mitigate congestion, and adapt to disruptive events, such as the Francis Scott Key Bridge collapse. This study contributes to the growing body of work on hybrid graph learning for intelligent transportation systems, highlighting the potential of combining KAN and GCN for real-time traffic optimization. Future work will focus on reducing computational overhead and integrating Transformer-based temporal modeling for enhanced long-term traffic prediction. The proposed TrafficKAN-GCN framework offers a promising direction for data-driven urban mobility management, balancing predictive accuracy, robustness, and computational efficiency.

Title: BEVMOSNet: Multimodal Fusion for BEV Moving Object Segmentation

• Authors: Hiep Truong Cong, Ajay Kumar Sigatapu, Arindam Das, Yashwanth Sharma, Venkatesh Satagopan, Ganesh Sistu, Ciaran Eising
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03280
• Pdf URL: https://arxiv.org/pdf/2503.03280
• Copy Paste: [[2503.03280]] BEVMOSNet: Multimodal Fusion for BEV Moving Object Segmentation(https://arxiv.org/abs/2503.03280)
• Keywords: segmentation
• Abstract: Accurate motion understanding of the dynamic objects within the scene in bird's-eye-view (BEV) is critical to ensure a reliable obstacle avoidance system and smooth path planning for autonomous vehicles. However, this task has received relatively limited exploration when compared to object detection and segmentation with only a few recent vision-based approaches presenting preliminary findings that significantly deteriorate in low-light, nighttime, and adverse weather conditions such as rain. Conversely, LiDAR and radar sensors remain almost unaffected in these scenarios, and radar provides key velocity information of the objects. Therefore, we introduce BEVMOSNet, to our knowledge, the first end-to-end multimodal fusion leveraging cameras, LiDAR, and radar to precisely predict the moving objects in BEV. In addition, we perform a deeper analysis to find out the optimal strategy for deformable cross-attention-guided sensor fusion for cross-sensor knowledge sharing in BEV. While evaluating BEVMOSNet on the nuScenes dataset, we show an overall improvement in IoU score of 36.59% compared to the vision-based unimodal baseline BEV-MoSeg (Sigatapu et al., 2023), and 2.35% compared to the multimodel SimpleBEV (Harley et al., 2022), extended for the motion segmentation task, establishing this method as the state-of-the-art in BEV motion segmentation.

Title: Enhancing Visual Forced Alignment with Local Context-Aware Feature Extraction and Multi-Task Learning

• Authors: Yi He, Lei Yang, Shilin Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03286
• Pdf URL: https://arxiv.org/pdf/2503.03286
• Copy Paste: [[2503.03286]] Enhancing Visual Forced Alignment with Local Context-Aware Feature Extraction and Multi-Task Learning(https://arxiv.org/abs/2503.03286)
• Keywords: extraction
• Abstract: This paper introduces a novel approach to Visual Forced Alignment (VFA), aiming to accurately synchronize utterances with corresponding lip movements, without relying on audio cues. We propose a novel VFA approach that integrates a local context-aware feature extractor and employs multi-task learning to refine both global and local context features, enhancing sensitivity to subtle lip movements for precise word-level and phoneme-level alignment. Incorporating the improved Viterbi algorithm for post-processing, our method significantly reduces misalignments. Experimental results show our approach outperforms existing methods, achieving a 6% accuracy improvement at the word-level and 27% improvement at the phoneme-level in LRS2 dataset. These improvements offer new potential for applications in automatically subtitling TV shows or user-generated content platforms like TikTok and YouTube Shorts.

Title: Label-Efficient LiDAR Semantic Segmentation with 2D-3D Vision Transformer Adapters

• Authors: Julia Hindel, Rohit Mohan, Jelena Bratulic, Daniele Cattaneo, Thomas Brox, Abhinav Valada
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03299
• Pdf URL: https://arxiv.org/pdf/2503.03299
• Copy Paste: [[2503.03299]] Label-Efficient LiDAR Semantic Segmentation with 2D-3D Vision Transformer Adapters(https://arxiv.org/abs/2503.03299)
• Keywords: transformer, segmentation
• Abstract: LiDAR semantic segmentation models are typically trained from random initialization as universal pre-training is hindered by the lack of large, diverse datasets. Moreover, most point cloud segmentation architectures incorporate custom network layers, limiting the transferability of advances from vision-based architectures. Inspired by recent advances in universal foundation models, we propose BALViT, a novel approach that leverages frozen vision models as amodal feature encoders for learning strong LiDAR encoders. Specifically, BALViT incorporates both range-view and bird's-eye-view LiDAR encoding mechanisms, which we combine through a novel 2D-3D adapter. While the range-view features are processed through a frozen image backbone, our bird's-eye-view branch enhances them through multiple cross-attention interactions. Thereby, we continuously improve the vision network with domain-dependent knowledge, resulting in a strong label-efficient LiDAR encoding mechanism. Extensive evaluations of BALViT on the SemanticKITTI and nuScenes benchmarks demonstrate that it outperforms state-of-the-art methods on small data regimes. We make the code and models publicly available at: this http URL.

Title: SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection

• Authors: Yi-Fan Lu, Xian-Ling Mao, Tian Lan, Tong Zhang, Yu-Shi Zhu, Heyan Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03303
• Pdf URL: https://arxiv.org/pdf/2503.03303
• Copy Paste: [[2503.03303]] SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection(https://arxiv.org/abs/2503.03303)
• Keywords: large language model
• Abstract: Automatic evaluation for Open Domain Event Detection (ODED) is a highly challenging task, because ODED is characterized by a vast diversity of un-constrained output labels from various domains. Nearly all existing evaluation methods for ODED usually first construct evaluation benchmarks with limited labels and domain coverage, and then evaluate ODED methods using metrics based on token-level label matching rules. However, this kind of evaluation framework faces two issues: (1) The limited evaluation benchmarks lack representatives of the real world, making it difficult to accurately reflect the performance of various ODED methods in real-world scenarios; (2) Evaluation metrics based on token-level matching rules fail to capture semantic similarity between predictions and golden labels. To address these two problems above, we propose a scalable and reliable Semantic-level Evaluation framework for Open domain Event detection (SEOE) by constructing a more representative evaluation benchmark and introducing a semantic evaluation metric. Specifically, our proposed framework first constructs a scalable evaluation benchmark that currently includes 564 event types covering 7 major domains, with a cost-effective supplementary annotation strategy to ensure the benchmark's representativeness. The strategy also allows for the supplement of new event types and domains in the future. Then, the proposed SEOE leverages large language models (LLMs) as automatic evaluation agents to compute a semantic F1-score, incorporating fine-grained definitions of semantically similar labels to enhance the reliability of the evaluation. Extensive experiments validate the representatives of the benchmark and the reliability of the semantic evaluation metric. Existing ODED methods are thoroughly evaluated, and the error patterns of predictions are analyzed, revealing several insightful findings.

Title: LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models

• Authors: Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, Yongfeng Zhang
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03313
• Pdf URL: https://arxiv.org/pdf/2503.03313
• Copy Paste: [[2503.03313]] LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models(https://arxiv.org/abs/2503.03313)
• Keywords: large language model
• Abstract: Text-Attributed Graphs (TAGs), where each node is associated with text descriptions, are ubiquitous in real-world scenarios. They typically exhibit distinctive structure and domain-specific knowledge, motivating the development of a Graph Foundation Model (GFM) that generalizes across diverse graphs and tasks. Despite large efforts to integrate Large Language Models (LLMs) and Graph Neural Networks (GNNs) for TAGs, existing approaches suffer from decoupled architectures with two-stage alignment, limiting their synergistic potential. Even worse, existing methods assign out-of-vocabulary (OOV) tokens to graph nodes, leading to graph-specific semantics, token explosion, and incompatibility with task-oriented prompt templates, which hinders cross-graph and cross-task transferability. To address these challenges, we propose PromptGFM, a versatile GFM for TAGs grounded in graph vocabulary learning. PromptGFM comprises two key components: (1) Graph Understanding Module, which explicitly prompts LLMs to replicate the finest GNN workflow within the text space, facilitating seamless GNN-LLM integration and elegant graph-text alignment; (2) Graph Inference Module, which establishes a language-based graph vocabulary ensuring expressiveness, transferability, and scalability, enabling readable instructions for LLM fine-tuning. Extensive experiments demonstrate our superiority and transferability across diverse graphs and tasks. The code is available at this: this https URL.

Title: See What You Are Told: Visual Attention Sink in Large Multimodal Models

• Authors: Seil Kang, Jinyeong Kim, Junhyeok Kim, Seong Jae Hwang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03321
• Pdf URL: https://arxiv.org/pdf/2503.03321
• Copy Paste: [[2503.03321]] See What You Are Told: Visual Attention Sink in Large Multimodal Models(https://arxiv.org/abs/2503.03321)
• Keywords: transformer
• Abstract: Large multimodal models (LMMs) "see" images by leveraging the attention mechanism between text and visual tokens in the transformer decoder. Ideally, these models should focus on key visual information relevant to the text token. However, recent findings indicate that LMMs have an extraordinary tendency to consistently allocate high attention weights to specific visual tokens, even when these tokens are irrelevant to the corresponding text. In this study, we investigate the property behind the appearance of these irrelevant visual tokens and examine their characteristics. Our findings show that this behavior arises due to the massive activation of certain hidden state dimensions, which resembles the attention sink found in language models. Hence, we refer to this phenomenon as the visual attention sink. In particular, our analysis reveals that removing the irrelevant visual sink tokens does not impact model performance, despite receiving high attention weights. Consequently, we recycle the attention to these tokens as surplus resources, redistributing the attention budget to enhance focus on the image. To achieve this, we introduce Visual Attention Redistribution (VAR), a method that redistributes attention in image-centric heads, which we identify as innately focusing on visual information. VAR can be seamlessly applied across different LMMs to improve performance on a wide range of tasks, including general vision-language tasks, visual hallucination tasks, and vision-centric tasks, all without the need for additional training, models, or inference steps. Experimental results demonstrate that VAR enables LMMs to process visual information more effectively by adjusting their internal attention mechanisms, offering a new direction to enhancing the multimodal capabilities of LMMs.

Title: Golden Cudgel Network for Real-Time Semantic Segmentation

• Authors: Guoyu Yang, Yuan Wang, Daming Shi, Yanzhong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03325
• Pdf URL: https://arxiv.org/pdf/2503.03325
• Copy Paste: [[2503.03325]] Golden Cudgel Network for Real-Time Semantic Segmentation(https://arxiv.org/abs/2503.03325)
• Keywords: segmentation
• Abstract: Recent real-time semantic segmentation models, whether single-branch or multi-branch, achieve good performance and speed. However, their speed is limited by multi-path blocks, and some depend on high-performance teacher models for training. To overcome these issues, we propose Golden Cudgel Network (GCNet). Specifically, GCNet uses vertical multi-convolutions and horizontal multi-paths for training, which are reparameterized into a single convolution for inference, optimizing both performance and speed. This design allows GCNet to self-enlarge during training and self-contract during inference, effectively becoming a "teacher model" without needing external ones. Experimental results show that GCNet outperforms existing state-of-the-art models in terms of performance and speed on the Cityscapes, CamVid, and Pascal VOC 2012 datasets. The code is available at this https URL.

Title: Deep Learning-Based Diffusion MRI Tractography: Integrating Spatial and Anatomical Information

• Authors: Yiqiong Yang, Yitian Yuan, Baoxing Ren, Ye Wu, Yanqiu Feng, Xinyuan Zhang
• Subjects: cs.CV, physics.med-ph
• Abstract URL: https://arxiv.org/abs/2503.03329
• Pdf URL: https://arxiv.org/pdf/2503.03329
• Copy Paste: [[2503.03329]] Deep Learning-Based Diffusion MRI Tractography: Integrating Spatial and Anatomical Information(https://arxiv.org/abs/2503.03329)
• Keywords: diffusion, transformer
• Abstract: Diffusion MRI tractography technique enables non-invasive visualization of the white matter pathways in the brain. It plays a crucial role in neuroscience and clinical fields by facilitating the study of brain connectivity and neurological disorders. However, the accuracy of reconstructed tractograms has been a longstanding challenge. Recently, deep learning methods have been applied to improve tractograms for better white matter coverage, but often comes at the expense of generating excessive false-positive connections. This is largely due to their reliance on local information to predict long range streamlines. To improve the accuracy of streamline propagation predictions, we introduce a novel deep learning framework that integrates image-domain spatial information and anatomical information along tracts, with the former extracted through convolutional layers and the later modeled via a Transformer-decoder. Additionally, we employ a weighted loss function to address fiber class imbalance encountered during training. We evaluate the proposed method on the simulated ISMRM 2015 Tractography Challenge dataset, achieving a valid streamline rate of 66.2%, white matter coverage of 63.8%, and successfully reconstructing 24 out of 25 bundles. Furthermore, on the multi-site Tractoinferno dataset, the proposed method demonstrates its ability to handle various diffusion MRI acquisition schemes, achieving a 5.7% increase in white matter coverage and a 4.1% decrease in overreach compared to RNN-based methods.

Title: Automated Attendee Recognition System for Large-Scale Social Events or Conference Gathering

• Authors: Dhruv Motwani, Ankush Tyagi, Vipul Dabhi, Harshadkumar Prajapati
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03330
• Pdf URL: https://arxiv.org/pdf/2503.03330
• Copy Paste: [[2503.03330]] Automated Attendee Recognition System for Large-Scale Social Events or Conference Gathering(https://arxiv.org/abs/2503.03330)
• Keywords: security, robust
• Abstract: Manual attendance tracking at large-scale events, such as marriage functions or conferences, is often inefficient and prone to human error. To address this challenge, we propose an automated, cloud-based attendance tracking system that uses cameras mounted at the entrance and exit gates. The mounted cameras continuously capture video and send the video data to cloud services to perform real-time face detection and recognition. Unlike existing solutions, our system accurately identifies attendees even when they are not looking directly at the camera, allowing natural movements, such as looking around or talking while walking. To the best of our knowledge, this is the first system to achieve high recognition rates under such dynamic conditions. Our system demonstrates overall 90% accuracy, with each video frame processed in 5 seconds, ensuring real time operation without frame loss. In addition, notifications are sent promptly to security personnel within the same latency. This system achieves 100% accuracy for individuals without facial obstructions and successfully recognizes all attendees appearing within the camera's field of view, providing a robust solution for attendee recognition in large-scale social events.

Title: EnigmaToM: Improve LLMs' Theory-of-Mind Reasoning Capabilities with Neural Knowledge Base of Entity States

• Authors: Hainiu Xu, Siya Qi, Jiazheng Li, Yuxiang Zhou, Jinhua Du, Caroline Catmur, Yulan He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03340
• Pdf URL: https://arxiv.org/pdf/2503.03340
• Copy Paste: [[2503.03340]] EnigmaToM: Improve LLMs' Theory-of-Mind Reasoning Capabilities with Neural Knowledge Base of Entity States(https://arxiv.org/abs/2503.03340)
• Keywords: large language model
• Abstract: Theory-of-Mind (ToM), the ability to infer others' perceptions and mental states, is fundamental to human interaction but remains a challenging task for Large Language Models (LLMs). While existing ToM reasoning methods show promise with reasoning via perceptual perspective-taking, they often rely excessively on LLMs, reducing their efficiency and limiting their applicability to high-order ToM reasoning, which requires multi-hop reasoning about characters' beliefs. To address these issues, we present EnigmaToM, a novel neuro-symbolic framework that enhances ToM reasoning by integrating a Neural Knowledge Base of entity states (Enigma) for (1) a psychology-inspired iterative masking mechanism that facilitates accurate perspective-taking and (2) knowledge injection that elicits key entity information. Enigma generates structured representations of entity states, which construct spatial scene graphs -- leveraging spatial information as an inductive bias -- for belief tracking of various ToM orders and enhancing events with fine-grained entity state details. Experimental results on multiple benchmarks, including ToMi, HiToM, and FANToM, show that EnigmaToM significantly improves ToM reasoning across LLMs of varying sizes, particularly excelling in high-order reasoning scenarios.

Title: Video Super-Resolution: All You Need is a Video Diffusion Model

• Authors: Zhihao Zhan, Wang Pang, Xiang Zhu, Yechao Bai
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.03355
• Pdf URL: https://arxiv.org/pdf/2503.03355
• Copy Paste: [[2503.03355]] Video Super-Resolution: All You Need is a Video Diffusion Model(https://arxiv.org/abs/2503.03355)
• Keywords: diffusion, transformer
• Abstract: We present a generic video super-resolution algorithm in this paper, based on the Diffusion Posterior Sampling framework with an unconditional video generation model in latent space. The video generation model, a diffusion transformer, functions as a space-time model. We argue that a powerful model, which learns the physics of the real world, can easily handle various kinds of motion patterns as prior knowledge, thus eliminating the need for explicit estimation of optical flows or motion parameters for pixel alignment. Furthermore, a single instance of the proposed video diffusion transformer model can adapt to different sampling conditions without re-training. Due to limited computational resources and training data, our experiments provide empirical evidence of the algorithm's strong super-resolution capabilities using synthetic data.

Title: Transformers for molecular property prediction: Domain adaptation efficiently improves performance

• Authors: Afnan Sultan, Max Rausch-Dupont, Shahrukh Khan, Olga Kalinina, Andrea Volkamer, Dietrich Klakow
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03360
• Pdf URL: https://arxiv.org/pdf/2503.03360
• Copy Paste: [[2503.03360]] Transformers for molecular property prediction: Domain adaptation efficiently improves performance(https://arxiv.org/abs/2503.03360)
• Keywords: transformer
• Abstract: Most of the current transformer-based chemical language models are pre-trained on millions to billions of molecules. However, the improvement from such scaling in dataset size is not confidently linked to improved molecular property prediction. The aim of this study is to investigate and overcome some of the limitations of transformer models in predicting molecular properties. Specifically, we examine the impact of pre-training dataset size and diversity on the performance of transformer models and investigate the use of domain adaptation as a technique for improving model performance. First, our findings indicate that increasing pretraining dataset size beyond 400K molecules from the GuacaMol dataset does not result in a significant improvement on four ADME endpoints, namely, solubility, permeability, microsomal stability, and plasma protein binding. Second, our results demonstrate that using domain adaptation by further training the transformer model on a small set of domain-relevant molecules, i.e., a few hundred to a few thousand, using multi-task regression of physicochemical properties was sufficient to significantly improve performance for three out of the four investigated ADME endpoints (P-value < 0.001). Finally, we observe that a model pre-trained on 400K molecules and domain adopted on a few hundred/thousand molecules performs similarly (P-value > 0.05) to more complicated transformer models like MolBERT(pre-trained on 1.3M molecules) and MolFormer (pre-trained on 100M molecules). A comparison to a random forest model trained on basic physicochemical properties showed similar performance to the examined transformer models. We believe that current transformer models can be improved through further systematic analysis of pre-training and downstream data, pre-training objectives, and scaling laws, ultimately leading to better and more helpful models.

Title: TopoMortar: A dataset to evaluate image segmentation methods focused on topology accuracy

• Authors: Juan Miguel Valverde, Motoya Koga, Nijihiko Otsuka, Anders Bjorholm Dahl
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03365
• Pdf URL: https://arxiv.org/pdf/2503.03365
• Copy Paste: [[2503.03365]] TopoMortar: A dataset to evaluate image segmentation methods focused on topology accuracy(https://arxiv.org/abs/2503.03365)
• Keywords: segmentation
• Abstract: We present TopoMortar, a brick wall dataset that is the first dataset specifically designed to evaluate topology-focused image segmentation methods, such as topology loss functions. TopoMortar enables to investigate in two ways whether methods incorporate prior topological knowledge. First, by eliminating challenges seen in real-world data, such as small training set, noisy labels, and out-of-distribution test-set images, that, as we show, impact the effectiveness of topology losses. Second, by allowing to assess in the same dataset topology accuracy across dataset challenges, isolating dataset-related effects from the effect of incorporating prior topological knowledge. In these two experiments, it is deliberately difficult to improve topology accuracy without actually using topology information, thus, permitting to attribute an improvement in topology accuracy to the incorporation of prior topological knowledge. To this end, TopoMortar includes three types of labels (accurate, noisy, pseudo-labels), two fixed training sets (large and small), and in-distribution and out-of-distribution test-set images. We compared eight loss functions on TopoMortar, and we found that clDice achieved the most topologically accurate segmentations, Skeleton Recall loss performed best particularly with noisy labels, and the relative advantageousness of the other loss functions depended on the experimental setting. Additionally, we show that simple methods, such as data augmentation and self-distillation, can elevate Cross entropy Dice loss to surpass most topology loss functions, and that those simple methods can enhance topology loss functions as well. clDice and Skeleton Recall loss, both skeletonization-based loss functions, were also the fastest to train, making this type of loss function a promising research direction. TopoMortar and our code can be found at this https URL

Title: Top-K Maximum Intensity Projection Priors for 3D Liver Vessel Segmentation

• Authors: Xiaotong Zhang, Alexander Broersen, Gonnie CM van Erp, Silvia L. Pintea, Jouke Dijkstra
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03367
• Pdf URL: https://arxiv.org/pdf/2503.03367
• Copy Paste: [[2503.03367]] Top-K Maximum Intensity Projection Priors for 3D Liver Vessel Segmentation(https://arxiv.org/abs/2503.03367)
• Keywords: diffusion, segmentation
• Abstract: Liver-vessel segmentation is an essential task in the pre-operative planning of liver resection. State-of-the-art 2D or 3D convolution-based methods focusing on liver vessel segmentation on 2D CT cross-sectional views, which do not take into account the global liver-vessel topology. To maintain this global vessel topology, we rely on the underlying physics used in the CT reconstruction process, and apply this to liver-vessel segmentation. Concretely, we introduce the concept of top-k maximum intensity projections, which mimics the CT reconstruction by replacing the integral along each projection direction, with keeping the top-k maxima along each projection direction. We use these top-k maximum projections to condition a diffusion model and generate 3D liver-vessel trees. We evaluate our 3D liver-vessel segmentation on the 3D-ircadb-01 dataset, and achieve the highest Dice coefficient, intersection-over-union (IoU), and Sensitivity scores compared to prior work.

Title: MIAdapt: Source-free Few-shot Domain Adaptive Object Detection for Microscopic Images

• Authors: Nimra Dilawar, Sara Nadeem, Javed Iqbal, Waqas Sultani, Mohsen Ali
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03370
• Pdf URL: https://arxiv.org/pdf/2503.03370
• Copy Paste: [[2503.03370]] MIAdapt: Source-free Few-shot Domain Adaptive Object Detection for Microscopic Images(https://arxiv.org/abs/2503.03370)
• Keywords: privacy
• Abstract: Existing generic unsupervised domain adaptation approaches require access to both a large labeled source dataset and a sufficient unlabeled target dataset during adaptation. However, collecting a large dataset, even if unlabeled, is a challenging and expensive endeavor, especially in medical imaging. In addition, constraints such as privacy issues can result in cases where source data is unavailable. Taking in consideration these challenges, we propose MIAdapt, an adaptive approach for Microscopic Imagery Adaptation as a solution for Source-free Few-shot Domain Adaptive Object detection (SF-FSDA). We also define two competitive baselines (1) Faster-FreeShot and (2) MT-FreeShot. Extensive experiments on the challenging M5-Malaria and Raabin-WBC datasets validate the effectiveness of MIAdapt. Without using any image from the source domain MIAdapt surpasses state-of-the-art source-free UDA (SF-UDA) methods by +21.3% mAP and few-shot domain adaptation (FSDA) approaches by +4.7% mAP on Raabin-WBC. Our code and models will be publicly available.

Title: The Serendipity of Claude AI: Case of the 13 Low-Resource National Languages of Mali

• Authors: Alou Dembele, Nouhoum Souleymane Coulibaly, Michael Leventhal (RobotsMali AI4D Lab, Bamako, Mali)
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03380
• Pdf URL: https://arxiv.org/pdf/2503.03380
• Copy Paste: [[2503.03380]] The Serendipity of Claude AI: Case of the 13 Low-Resource National Languages of Mali(https://arxiv.org/abs/2503.03380)
• Keywords: robust, generative
• Abstract: Recent advances in artificial intelligence (AI) and natural language processing (NLP) have improved the representation of underrepresented languages. However, most languages, including Mali's 13 official national languages, continue to be poorly supported or unsupported by automatic translation and generative AI. This situation appears to have slightly improved with certain recent LLM releases. The study evaluated Claude AI's translation performance on each of the 13 national languages of Mali. In addition to ChrF2 and BLEU scores, human evaluators assessed translation accuracy, contextual consistency, robustness to dialect variations, management of linguistic bias, adaptation to a limited corpus, and ease of understanding. The study found that Claude AI performs robustly for languages with very modest language resources and, while unable to produce understandable and coherent texts for Malian languages with minimal resources, still manages to produce results which demonstrate the ability to mimic some elements of the language.

Title: Predicting Practically? Domain Generalization for Predictive Analytics in Real-world Environments

• Authors: Hanyu Duan, Yi Yang, Ahmed Abbasi, Kar Yan Tam
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03399
• Pdf URL: https://arxiv.org/pdf/2503.03399
• Copy Paste: [[2503.03399]] Predicting Practically? Domain Generalization for Predictive Analytics in Real-world Environments(https://arxiv.org/abs/2503.03399)
• Keywords: robust
• Abstract: Predictive machine learning models are widely used in customer relationship management (CRM) to forecast customer behaviors and support decision-making. However, the dynamic nature of customer behaviors often results in significant distribution shifts between training data and serving data, leading to performance degradation in predictive models. Domain generalization, which aims to train models that can generalize to unseen environments without prior knowledge of their distributions, has become a critical area of research. In this work, we propose a novel domain generalization method tailored to handle complex distribution shifts, encompassing both covariate and concept shifts. Our method builds upon the Distributionally Robust Optimization framework, optimizing model performance over a set of hypothetical worst-case distributions rather than relying solely on the training data. Through simulation experiments, we demonstrate the working mechanism of the proposed method. We also conduct experiments on a real-world customer churn dataset, and validate its effectiveness in both temporal and spatial generalization settings. Finally, we discuss the broader implications of our method for advancing Information Systems (IS) design research, particularly in building robust predictive models for dynamic managerial environments.

Title: When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

• Authors: Jabez Magomere, Emanuele La Malfa, Manuel Tonneau, Ashkan Kazemi, Scott Hale
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03417
• Pdf URL: https://arxiv.org/pdf/2503.03417
• Copy Paste: [[2503.03417]] When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits(https://arxiv.org/abs/2503.03417)
• Keywords: robust
• Abstract: Online misinformation remains a critical challenge, and fact-checkers increasingly rely on embedding-based methods to retrieve relevant fact-checks. Yet, when debunked claims reappear in edited forms, the performance of these methods is unclear. In this work, we introduce a taxonomy of six common real-world misinformation edits and propose a perturbation framework that generates valid, natural claim variations. Our multi-stage retrieval evaluation reveals that standard embedding models struggle with user-introduced edits, while LLM-distilled embeddings offer improved robustness at a higher computational cost. Although a strong reranker helps mitigate some issues, it cannot fully compensate for first-stage retrieval gaps. Addressing these retrieval gaps, our train- and inference-time mitigation approaches enhance in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points over baseline models. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation.

Title: Automatic Drywall Analysis for Progress Tracking and Quality Control in Construction

• Authors: Mariusz Trzeciakiewicz, Aleixo Cambeiro Barreiro, Niklas Gard, Anna Hilsmann, Peter Eisert
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03422
• Pdf URL: https://arxiv.org/pdf/2503.03422
• Copy Paste: [[2503.03422]] Automatic Drywall Analysis for Progress Tracking and Quality Control in Construction(https://arxiv.org/abs/2503.03422)
• Keywords: segmentation
• Abstract: Digitalization in the construction industry has become essential, enabling centralized, easy access to all relevant information of a building. Automated systems can facilitate the timely and resource-efficient documentation of changes, which is crucial for key processes such as progress tracking and quality control. This paper presents a method for image-based automated drywall analysis enabling construction progress and quality assessment through on-site camera systems. Our proposed solution integrates a deep learning-based instance segmentation model to detect and classify various drywall elements with an analysis module to cluster individual wall segments, estimate camera perspective distortions, and apply the corresponding corrections. This system extracts valuable information from images, enabling more accurate progress tracking and quality assessment on construction sites. Our main contributions include a fully automated pipeline for drywall analysis, improving instance segmentation accuracy through architecture modifications and targeted data augmentation, and a novel algorithm to extract important information from the segmentation results. Our modified model, enhanced with data augmentation, achieves significantly higher accuracy compared to other architectures, offering more detailed and precise information than existing approaches. Combined with the proposed drywall analysis steps, it enables the reliable automation of construction progress and quality assessment.

Title: Privacy is All You Need: Revolutionizing Wearable Health Data with Advanced PETs

• Authors: Karthik Barma, Seshu Babu Barma
• Subjects: cs.CR, cs.AI, cs.ET, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.03428
• Pdf URL: https://arxiv.org/pdf/2503.03428
• Copy Paste: [[2503.03428]] Privacy is All You Need: Revolutionizing Wearable Health Data with Advanced PETs(https://arxiv.org/abs/2503.03428)
• Keywords: secure, privacy, federate
• Abstract: In a world where data is the new currency, wearable health devices offer unprecedented insights into daily life, continuously monitoring vital signs and metrics. However, this convenience raises privacy concerns, as these devices collect sensitive data that can be misused or breached. Traditional measures often fail due to real-time data processing needs and limited device power. Users also lack awareness and control over data sharing and usage. We propose a Privacy-Enhancing Technology (PET) framework for wearable devices, integrating federated learning, lightweight cryptographic methods, and selectively deployed blockchain technology. The blockchain acts as a secure ledger triggered only upon data transfer requests, granting users real-time notifications and control. By dismantling data monopolies, this approach returns data sovereignty to individuals. Through real-world applications like secure medical data sharing, privacy-preserving fitness tracking, and continuous health monitoring, our framework reduces privacy risks by up to 70 percent while preserving data utility and performance. This innovation sets a new benchmark for wearable privacy and can scale to broader IoT ecosystems, including smart homes and industry. As data continues to shape our digital landscape, our research underscores the critical need to maintain privacy and user control at the forefront of technological progress.

Title: RASD: Retrieval-Augmented Speculative Decoding

• Authors: Guofeng Quan, Wenfeng Feng, Chuzhan Hao, Guochao Jiang, Yuewei Zhang, Hao Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03434
• Pdf URL: https://arxiv.org/pdf/2503.03434
• Copy Paste: [[2503.03434]] RASD: Retrieval-Augmented Speculative Decoding(https://arxiv.org/abs/2503.03434)
• Keywords: large language model
• Abstract: Speculative decoding accelerates inference in large language models (LLMs) by generating draft tokens for target model verification. Current approaches for obtaining draft tokens rely on lightweight draft models or additional model structures to generate draft tokens and retrieve context from databases. Due to the draft model's small size and limited training data, model-based speculative decoding frequently becomes less effective in out-of-domain scenarios. Additionally, the time cost of the drafting phase results in a low upper limit on acceptance length during the verification step, limiting overall efficiency. This paper proposes RASD (Retrieval-Augmented Speculative Decoding), which adopts retrieval methods to enhance model-based speculative decoding. We introduce tree pruning and tree fusion to achieve this. Specifically, we develop a pruning method based on the draft model's probability distribution to construct the optimal retrieval tree. Second, we employ the longest prefix matching algorithm to merge the tree generated by the draft model with the retrieval tree, resulting in a unified tree for verification. Experimental results demonstrate that RASD achieves state-of-the-art inference acceleration across tasks such as DocQA, Summary, Code, and In-Domain QA. Moreover, RASD exhibits strong scalability, seamlessly integrating with various speculative decoding approaches, including both generation-based and retrieval-based methods.

Title: JamMa: Ultra-lightweight Local Feature Matching with Joint Mamba

• Authors: Xiaoyong Lu, Songlin Du
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03437
• Pdf URL: https://arxiv.org/pdf/2503.03437
• Copy Paste: [[2503.03437]] JamMa: Ultra-lightweight Local Feature Matching with Joint Mamba(https://arxiv.org/abs/2503.03437)
• Keywords: transformer
• Abstract: Existing state-of-the-art feature matchers capture long-range dependencies with Transformers but are hindered by high spatial complexity, leading to demanding training and highlatency inference. Striking a better balance between performance and efficiency remains a challenge in feature matching. Inspired by the linear complexity O(N) of Mamba, we propose an ultra-lightweight Mamba-based matcher, named JamMa, which converges on a single GPU and achieves an impressive performance-efficiency balance in inference. To unlock the potential of Mamba for feature matching, we propose Joint Mamba with a scan-merge strategy named JEGO, which enables: (1) Joint scan of two images to achieve high-frequency mutual interaction, (2) Efficient scan with skip steps to reduce sequence length, (3) Global receptive field, and (4) Omnidirectional feature representation. With the above properties, the JEGO strategy significantly outperforms the scan-merge strategies proposed in VMamba and EVMamba in the feature matching task. Compared to attention-based sparse and semi-dense matchers, JamMa demonstrates a superior balance between performance and efficiency, delivering better performance with less than 50% of the parameters and FLOPs.

Title: Conceptualizing Uncertainty

• Authors: Isaac Roberts, Alexander Schulz, Sarah Schroeder, Fabian Hinder, Barbara Hammer
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03443
• Pdf URL: https://arxiv.org/pdf/2503.03443
• Copy Paste: [[2503.03443]] Conceptualizing Uncertainty(https://arxiv.org/abs/2503.03443)
• Keywords: interpretability
• Abstract: Uncertainty in machine learning refers to the degree of confidence or lack thereof in a model's predictions. While uncertainty quantification methods exist, explanations of uncertainty, especially in high-dimensional settings, remain an open challenge. Existing work focuses on feature attribution approaches which are restricted to local explanations. Understanding uncertainty, its origins, and characteristics on a global scale is crucial for enhancing interpretability and trust in a model's predictions. In this work, we propose to explain the uncertainty in high-dimensional data classification settings by means of concept activation vectors which give rise to local and global explanations of uncertainty. We demonstrate the utility of the generated explanations by leveraging them to refine and improve our model.

Title: Taxation Perspectives from Large Language Models: A Case Study on Additional Tax Penalties

• Authors: Eunkyung Choi, Young Jin Suh, Hun Park, Wonseok Hwang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03444
• Pdf URL: https://arxiv.org/pdf/2503.03444
• Copy Paste: [[2503.03444]] Taxation Perspectives from Large Language Models: A Case Study on Additional Tax Penalties(https://arxiv.org/abs/2503.03444)
• Keywords: large language model
• Abstract: How capable are large language models (LLMs) in the domain of taxation? Although numerous studies have explored the legal domain in general, research dedicated to taxation remain scarce. Moreover, the datasets used in these studies are either simplified, failing to reflect the real-world complexities, or unavailable as open source. To address this gap, we introduce PLAT, a new benchmark designed to assess the ability of LLMs to predict the legitimacy of additional tax penalties. PLAT is constructed to evaluate LLMs' understanding of tax law, particularly in cases where resolving the issue requires more than just applying related statutes. Our experiments with six LLMs reveal that their baseline capabilities are limited, especially when dealing with conflicting issues that demand a comprehensive understanding. However, we found that enabling retrieval, self-reasoning, and discussion among multiple agents with specific role assignments, this limitation can be mitigated.

Title: Biased Heritage: How Datasets Shape Models in Facial Expression Recognition

• Authors: Iris Dominguez-Catena, Daniel Paternain, Mikel Galar, MaryBeth Defrance, Maarten Buyl, Tijl De Bie
• Subjects: cs.CV, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.03446
• Pdf URL: https://arxiv.org/pdf/2503.03446
• Copy Paste: [[2503.03446]] Biased Heritage: How Datasets Shape Models in Facial Expression Recognition(https://arxiv.org/abs/2503.03446)
• Keywords: protect, fair
• Abstract: In recent years, the rapid development of artificial intelligence (AI) systems has raised concerns about our ability to ensure their fairness, that is, how to avoid discrimination based on protected characteristics such as gender, race, or age. While algorithmic fairness is well-studied in simple binary classification tasks on tabular data, its application to complex, real-world scenarios-such as Facial Expression Recognition (FER)-remains underexplored. FER presents unique challenges: it is inherently multiclass, and biases emerge across intersecting demographic variables, each potentially comprising multiple protected groups. We present a comprehensive framework to analyze bias propagation from datasets to trained models in image-based FER systems, while introducing new bias metrics specifically designed for multiclass problems with multiple demographic groups. Our methodology studies bias propagation by (1) inducing controlled biases in FER datasets, (2) training models on these biased datasets, and (3) analyzing the correlation between dataset bias metrics and model fairness notions. Our findings reveal that stereotypical biases propagate more strongly to model predictions than representational biases, suggesting that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets. Additionally, we observe that biased datasets lead to reduced model accuracy, challenging the assumed fairness-accuracy trade-off.

Title: Active Learning for Deep Learning-Based Hemodynamic Parameter Estimation

• Authors: Patryk Rygiel, Julian Suk, Kak Khee Yeung, Christoph Brune, Jelmer M. Wolterink
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03453
• Pdf URL: https://arxiv.org/pdf/2503.03453
• Copy Paste: [[2503.03453]] Active Learning for Deep Learning-Based Hemodynamic Parameter Estimation(https://arxiv.org/abs/2503.03453)
• Keywords: robust
• Abstract: Hemodynamic parameters such as pressure and wall shear stress play an important role in diagnosis, prognosis, and treatment planning in cardiovascular diseases. These parameters can be accurately computed using computational fluid dynamics (CFD), but CFD is computationally intensive. Hence, deep learning methods have been adopted as a surrogate to rapidly estimate CFD outcomes. A drawback of such data-driven models is the need for time-consuming reference CFD simulations for training. In this work, we introduce an active learning framework to reduce the number of CFD simulations required for the training of surrogate models, lowering the barriers to their deployment in new applications. We propose three distinct querying strategies to determine for which unlabeled samples CFD simulations should be obtained. These querying strategies are based on geometrical variance, ensemble uncertainty, and adherence to the physics governing fluid dynamics. We benchmark these methods on velocity field estimation in synthetic coronary artery bifurcations and find that they allow for substantial reductions in annotation cost. Notably, we find that our strategies reduce the number of samples required by up to 50% and make the trained models more robust to difficult cases. Our results show that active learning is a feasible strategy to increase the potential of deep learning-based CFD surrogates.

Title: Data Poisoning Attacks to Locally Differentially Private Range Query Protocols

• Authors: I-Jung Hsu, Chih-Hsun Lin, Chia-Mu Yu, Sy-Yen Kuo, Chun-Ying Huang
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03454
• Pdf URL: https://arxiv.org/pdf/2503.03454
• Copy Paste: [[2503.03454]] Data Poisoning Attacks to Locally Differentially Private Range Query Protocols(https://arxiv.org/abs/2503.03454)
• Keywords: privacy, defense, attack, robust
• Abstract: Trajectory data, which tracks movements through geographic locations, is crucial for improving real-world applications. However, collecting such sensitive data raises considerable privacy concerns. Local differential privacy (LDP) offers a solution by allowing individuals to locally perturb their trajectory data before sharing it. Despite its privacy benefits, LDP protocols are vulnerable to data poisoning attacks, where attackers inject fake data to manipulate aggregated results. In this work, we make the first attempt to analyze vulnerabilities in several representative LDP trajectory protocols. We propose \textsc{TraP}, a heuristic algorithm for data \underline{P}oisoning attacks using a prefix-suffix method to optimize fake \underline{Tra}jectory selection, significantly reducing computational complexity. Our experimental results demonstrate that our attack can substantially increase target pattern occurrences in the perturbed trajectory dataset with few fake users. This study underscores the urgent need for robust defenses and better protocol designs to safeguard LDP trajectory data against malicious manipulation.

Title: Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models

• Authors: Alessio Galatolo, Zhenbang Dai, Katie Winkle, Meriem Beloucif
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03460
• Pdf URL: https://arxiv.org/pdf/2503.03460
• Copy Paste: [[2503.03460]] Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models(https://arxiv.org/abs/2503.03460)
• Keywords: generative, large language model
• Abstract: Fine-tuning LLMs with first-order methods like back-propagation is computationally intensive. Zeroth-Order (ZO) optimisation, using function evaluations instead of gradients, reduces memory usage but suffers from slow convergence in high-dimensional models. As a result, ZO research in LLMs has mostly focused on classification, overlooking more complex generative tasks. In this paper, we introduce ZOPrO, a novel ZO algorithm designed for \textit{Preference Optimisation} in LLMs. We begin by analysing the interplay between policy and reward models during traditional (first-order) Preference Optimisation, uncovering patterns in their relative updates. Guided by these insights, we adapt Simultaneous Perturbation Stochastic Approximation (SPSA) with a targeted sampling strategy to accelerate convergence. Through experiments on summarisation, machine translation, and conversational assistants, we demonstrate that our method consistently enhances reward signals while achieving convergence times comparable to first-order methods. While it falls short of some state-of-the-art methods, our work is the first to apply Zeroth-Order methods to Preference Optimisation in LLMs, going beyond classification tasks and paving the way for a largely unexplored research direction. Code and visualisations are available at this https URL

Title: Open-Source Large Language Models as Multilingual Crowdworkers: Synthesizing Open-Domain Dialogues in Several Languages With No Examples in Targets and No Machine Translation

• Authors: Ahmed Njifenjou, Virgile Sucal, Bassam Jabaian, Fabrice Lefèvre
• Subjects: cs.CL, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03462
• Pdf URL: https://arxiv.org/pdf/2503.03462
• Copy Paste: [[2503.03462]] Open-Source Large Language Models as Multilingual Crowdworkers: Synthesizing Open-Domain Dialogues in Several Languages With No Examples in Targets and No Machine Translation(https://arxiv.org/abs/2503.03462)
• Keywords: large language model
• Abstract: The prevailing paradigm in the domain of Open-Domain Dialogue agents predominantly focuses on the English language, encompassing both models and datasets. Furthermore, the financial and temporal investments required for crowdsourcing such datasets for finetuning are substantial, particularly when multiple languages are involved. Fortunately, advancements in Large Language Models (LLMs) have unveiled a plethora of possibilities across diverse tasks. Specifically, instruction-tuning has enabled LLMs to execute tasks based on natural language instructions, occasionally surpassing the performance of human crowdworkers. Additionally, these models possess the capability to function in various languages within a single thread. Consequently, to generate new samples in different languages, we propose leveraging these capabilities to replicate the data collection process. We introduce a pipeline for generating Open-Domain Dialogue data in multiple Target Languages using LLMs, with demonstrations provided in a unique Source Language. By eschewing explicit Machine Translation in this approach, we enhance the adherence to language-specific nuances. We apply this methodology to the PersonaChat dataset. To enhance the openness of generated dialogues and mimic real life scenarii, we added the notion of speech events corresponding to the type of conversation the speakers are involved in and also that of common ground which represents the premises of a conversation.

Title: DTU-Net: A Multi-Scale Dilated Transformer Network for Nonlinear Hyperspectral Unmixing

• Authors: ChenTong Wang, Jincheng Gao, Fei Zhu, Abderrahim Halimi, C'edric Richard
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.03465
• Pdf URL: https://arxiv.org/pdf/2503.03465
• Copy Paste: [[2503.03465]] DTU-Net: A Multi-Scale Dilated Transformer Network for Nonlinear Hyperspectral Unmixing(https://arxiv.org/abs/2503.03465)
• Keywords: extraction, interpretability, transformer
• Abstract: Transformers have shown significant success in hyperspectral unmixing (HU). However, challenges remain. While multi-scale and long-range spatial correlations are essential in unmixing tasks, current Transformer-based unmixing networks, built on Vision Transformer (ViT) or Swin-Transformer, struggle to capture them effectively. Additionally, current Transformer-based unmixing networks rely on the linear mixing model, which lacks the flexibility to accommodate scenarios where nonlinear effects are significant. To address these limitations, we propose a multi-scale Dilated Transformer-based unmixing network for nonlinear HU (DTU-Net). The encoder employs two branches. The first one performs multi-scale spatial feature extraction using Multi-Scale Dilated Attention (MSDA) in the Dilated Transformer, which varies dilation rates across attention heads to capture long-range and multi-scale spatial correlations. The second one performs spectral feature extraction utilizing 3D-CNNs with channel attention. The outputs from both branches are then fused to integrate multi-scale spatial and spectral information, which is subsequently transformed to estimate the abundances. The decoder is designed to accommodate both linear and nonlinear mixing scenarios. Its interpretability is enhanced by explicitly modeling the relationships between endmembers, abundances, and nonlinear coefficients in accordance with the polynomial post-nonlinear mixing model (PPNMM). Experiments on synthetic and real datasets validate the effectiveness of the proposed DTU-Net compared to PPNMM-derived methods and several advanced unmixing networks.

Title: Feature Point Extraction for Extra-Affine Image

• Authors: Tao Wang, Yinghui Wang, Yanxing Liang, Liangyi Huang, Jinlong Yang, Wei Li, Xiaojuan Ning
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03479
• Pdf URL: https://arxiv.org/pdf/2503.03479
• Copy Paste: [[2503.03479]] Feature Point Extraction for Extra-Affine Image(https://arxiv.org/abs/2503.03479)
• Keywords: extraction
• Abstract: The issue concerning the significant decline in the stability of feature extraction for images subjected to large-angle affine transformations, where the angle exceeds 50 degrees, still awaits a satisfactory solution. Even ASIFT, which is built upon SIFT and entails a considerable number of image comparisons simulated by affine transformations, inevitably exhibits the drawbacks of being time-consuming and imposing high demands on memory usage. And the stability of feature extraction drops rapidly under large-view affine transformations. Consequently, we propose a method that represents an improvement over ASIFT. On the premise of improving the precision and maintaining the affine invariance, it currently ranks as the fastest feature extraction method for extra-affine images that we know of at present. Simultaneously, the stability of feature extraction regarding affine transformation images has been approximated to the maximum limits. Both the angle between the shooting direction and the normal direction of the photographed object (absolute tilt angle), and the shooting transformation angle between two images (transition tilt angle) are close to 90 degrees. The central idea of the method lies in obtaining the optimal parameter set by simulating affine transformation with the reference image. And the simulated affine transformation is reproduced by combining it with the Lanczos interpolation based on the optimal parameter set. Subsequently, it is combined with ORB, which exhibits excellent real-time performance for rapid orientation binary description. Moreover, a scale parameter simulation is introduced to further augment the operational efficiency.

Title: TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology

• Authors: Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Julien Delile, Christoph Meier, Leonid Zhukov, Iya Khalil, Srayanta Mukherjee, Judith Mueller
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2503.03485
• Pdf URL: https://arxiv.org/pdf/2503.03485
• Copy Paste: [[2503.03485]] TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology(https://arxiv.org/abs/2503.03485)
• Keywords: transformer
• Abstract: Understanding the biological mechanism of disease is critical for medicine, and in particular drug discovery. AI-powered analysis of genome-scale biological data hold great potential in this regard. The increasing availability of single-cell RNA sequencing data has enabled the development of large foundation models for disease biology. However, existing foundation models either do not improve or only modestly improve over task-specific models in downstream applications. Here, we explored two avenues for improving the state-of-the-art. First, we scaled the pre-training dataset to 116 million cells, which is larger than those used by previous models. Second, we leveraged the availability of large-scale biological annotations as a form of supervision during pre-training. We trained the TEDDY family of models comprising six transformer-based state-of-the-art single-cell foundation models with 70 million, 160 million, and 400 million parameters. We vetted our models on two downstream evaluation tasks -- identifying the underlying disease state of held-out donors not seen during training and distinguishing healthy cells from diseased ones for disease conditions and donors not seen during training. Scaling experiments showed that performance improved predictably with both data volume and parameter count. Our models showed substantial improvement over existing work on the first task and more muted improvements on the second.

Title: Differentially Private Learners for Heterogeneous Treatment Effects

• Authors: Maresa Schröder, Valentyn Melnychuk, Stefan Feuerriegel
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03486
• Pdf URL: https://arxiv.org/pdf/2503.03486
• Copy Paste: [[2503.03486]] Differentially Private Learners for Heterogeneous Treatment Effects(https://arxiv.org/abs/2503.03486)
• Keywords: privacy
• Abstract: Patient data is widely used to estimate heterogeneous treatment effects and thus understand the effectiveness and safety of drugs. Yet, patient data includes highly sensitive information that must be kept private. In this work, we aim to estimate the conditional average treatment effect (CATE) from observational data under differential privacy. Specifically, we present DP-CATE, a novel framework for CATE estimation that is Neyman-orthogonal and further ensures differential privacy of the estimates. Our framework is highly general: it applies to any two-stage CATE meta-learner with a Neyman-orthogonal loss function, and any machine learning model can be used for nuisance estimation. We further provide an extension of our DP-CATE, where we employ RKHS regression to release the complete CATE function while ensuring differential privacy. We demonstrate our DP-CATE across various experiments using synthetic and real-world datasets. To the best of our knowledge, we are the first to provide a framework for CATE estimation that is Neyman-orthogonal and differentially private.

Title: Federated Learning for Predicting Mild Cognitive Impairment to Dementia Conversion

• Authors: Gaurang Sharma, Elaheh Moradi, Juha Pajula, Mika Hilvo, Jussi Tohka (for the Alzheimerś Disease Neuroimaging Initiative)
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03489
• Pdf URL: https://arxiv.org/pdf/2503.03489
• Copy Paste: [[2503.03489]] Federated Learning for Predicting Mild Cognitive Impairment to Dementia Conversion(https://arxiv.org/abs/2503.03489)
• Keywords: privacy, federate
• Abstract: Dementia is a progressive condition that impairs an individual's cognitive health and daily functioning, with mild cognitive impairment (MCI) often serving as its precursor. The prediction of MCI to dementia conversion has been well studied, but previous studies have almost always focused on traditional Machine Learning (ML) based methods that require sharing sensitive clinical information to train predictive models. This study proposes a privacy-enhancing solution using Federated Learning (FL) to train predictive models for MCI to dementia conversion without sharing sensitive data, leveraging socio demographic and cognitive measures. We simulated and compared two network architectures, Peer to Peer (P2P) and client-server, to enable collaborative learning. Our results demonstrated that FL had comparable predictive performance to centralized ML, and each clinical site showed similar performance without sharing local data. Moreover, the predictive performance of FL models was superior to site specific models trained without collaboration. This work highlights that FL can eliminate the need for data sharing without compromising model efficacy.

Title: Find First, Track Next: Decoupling Identification and Propagation in Referring Video Object Segmentation

• Authors: Suhwan Cho, Seunghoon Lee, Minhyeok Lee, Jungho Lee, Sangyoun Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03492
• Pdf URL: https://arxiv.org/pdf/2503.03492
• Copy Paste: [[2503.03492]] Find First, Track Next: Decoupling Identification and Propagation in Referring Video Object Segmentation(https://arxiv.org/abs/2503.03492)
• Keywords: robust, segmentation
• Abstract: Referring video object segmentation aims to segment and track a target object in a video using a natural language prompt. Existing methods typically fuse visual and textual features in a highly entangled manner, processing multi-modal information together to generate per-frame masks. However, this approach often struggles with ambiguous target identification, particularly in scenes with multiple similar objects, and fails to ensure consistent mask propagation across frames. To address these limitations, we introduce FindTrack, a novel decoupled framework that separates target identification from mask propagation. FindTrack first adaptively selects a key frame by balancing segmentation confidence and vision-text alignment, establishing a robust reference for the target object. This reference is then utilized by a dedicated propagation module to track and segment the object across the entire video. By decoupling these processes, FindTrack effectively reduces ambiguities in target association and enhances segmentation consistency. We demonstrate that FindTrack outperforms existing methods on public benchmarks.

Title: Oblivious Digital Tokens

• Authors: Mihael Liskij, Xuhua Ding, Gene Tsudik, David Basin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03494
• Pdf URL: https://arxiv.org/pdf/2503.03494
• Copy Paste: [[2503.03494]] Oblivious Digital Tokens(https://arxiv.org/abs/2503.03494)
• Keywords: security, protect
• Abstract: A computing device typically identifies itself by exhibiting unique measurable behavior or by proving its knowledge of a secret. In both cases, the identifying device must reveal information to a verifier. Considerable research has focused on protecting identifying entities (provers) and reducing the amount of leaked data. However, little has been done to conceal the fact that the verification occurred. We show how this problem naturally arises in the context of digital emblems, which were recently proposed by the International Committee of the Red Cross to protect digital resources during cyber-conflicts. To address this new and important open problem, we define a new primitive, called an Oblivious Digital Token (ODT) that can be verified obliviously. Verifiers can use this procedure to check whether a device has an ODT without revealing to any other parties (including the device itself) that this check occurred. We demonstrate the feasibility of ODTs and present a concrete construction that provably meets the ODT security requirements, even if the prover device's software is fully compromised. We also implement a prototype of the proposed construction and evaluate its performance, thereby confirming its practicality.

Title: State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for State Space Models

• Authors: Wonjun Kang, Kevin Galim, Yuchen Zeng, Minjae Lee, Hyung Il Koo, Nam Ik Cho
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03499
• Pdf URL: https://arxiv.org/pdf/2503.03499
• Copy Paste: [[2503.03499]] State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for State Space Models(https://arxiv.org/abs/2503.03499)
• Keywords: transformer
• Abstract: State Space Models (SSMs) have emerged as efficient alternatives to Transformers, mitigating their quadratic computational cost. However, the application of Parameter-Efficient Fine-Tuning (PEFT) methods to SSMs remains largely unexplored. In particular, prompt-based methods like Prompt Tuning and Prefix-Tuning, which are widely used in Transformers, do not perform well on SSMs. To address this, we propose state-based methods as a superior alternative to prompt-based methods. This new family of methods naturally stems from the architectural characteristics of SSMs. State-based methods adjust state-related features directly instead of depending on external prompts. Furthermore, we introduce a novel state-based PEFT method: State-offset Tuning. At every timestep, our method directly affects the state at the current step, leading to more effective adaptation. Through extensive experiments across diverse datasets, we demonstrate the effectiveness of our method. Code is available at this https URL.

Title: CURVALID: Geometrically-guided Adversarial Prompt Detection

• Authors: Canaan Yung, Hanxun Huang, Sarah Monazam Erfani, Christopher Leckie
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03502
• Pdf URL: https://arxiv.org/pdf/2503.03502
• Copy Paste: [[2503.03502]] CURVALID: Geometrically-guided Adversarial Prompt Detection(https://arxiv.org/abs/2503.03502)
• Keywords: defense, large language model
• Abstract: Adversarial prompts capable of jailbreaking large language models (LLMs) and inducing undesirable behaviours pose a significant obstacle to their safe deployment. Current mitigation strategies rely on activating built-in defence mechanisms or fine-tuning the LLMs, but the fundamental distinctions between adversarial and benign prompts are yet to be understood. In this work, we introduce CurvaLID, a novel defense framework that efficiently detects adversarial prompts by leveraging their geometric properties. It is agnostic to the type of LLM, offering a unified detection framework across diverse adversarial prompts and LLM architectures. CurvaLID builds on the geometric analysis of text prompts to uncover their underlying differences. We theoretically extend the concept of curvature via the Whewell equation into an $n$-dimensional word embedding space, enabling us to quantify local geometric properties, including semantic shifts and curvature in the underlying manifolds. Additionally, we employ Local Intrinsic Dimensionality (LID) to capture geometric features of text prompts within adversarial subspaces. Our findings reveal that adversarial prompts differ fundamentally from benign prompts in terms of their geometric characteristics. Our results demonstrate that CurvaLID delivers superior detection and rejection of adversarial queries, paving the way for safer LLM deployment. The source code can be found at this https URL

Title: Rethinking Synthetic Data definitions: A privacy driven approach

• Authors: Vibeke Binz Vallevik, Serena Elizabeth Marshall, Aleksandar Babic, Jan Franz Nygaard
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03506
• Pdf URL: https://arxiv.org/pdf/2503.03506
• Copy Paste: [[2503.03506]] Rethinking Synthetic Data definitions: A privacy driven approach(https://arxiv.org/abs/2503.03506)
• Keywords: privacy, generative
• Abstract: Synthetic data is gaining traction as a cost-effective solution for the increasing data demands of AI development and can be generated either from existing knowledge or derived data captured from real-world events. The source of the synthetic data generation and the technique used significantly impacts its residual privacy risk and therefore its opportunity for sharing. Traditional classification of synthetic data types no longer fit the newer generation techniques and there is a need to better align the classification with practical needs. We suggest a new way of grouping synthetic data types that better supports privacy evaluations to aid regulatory policymaking. Our novel classification provides flexibility to new advancements like deep generative methods and offers a more practical framework for future applications.

Title: Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks

• Authors: Samuel Repka, Bořek Reich, Fedor Zolotarev, Tuomas Eerola, Pavel Zemčík
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03507
• Pdf URL: https://arxiv.org/pdf/2503.03507
• Copy Paste: [[2503.03507]] Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks(https://arxiv.org/abs/2503.03507)
• Keywords: segmentation
• Abstract: We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images. In most cases, Backscattered Electron (BSE) images obtained using SEM do not contain sufficient information for mineral segmentation. Therefore, imaging is often complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral measurements that provide highly accurate information about the chemical composition but that are time-consuming to acquire. This motivates the use of sparse spectral data in conjunction with BSE images for mineral segmentation. The unstructured nature of the spectral data makes most traditional image fusion techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks to fuse the two modalities and segment the mineral phases simultaneously. Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation, enabling rapid analysis of mineral samples. The proposed data fusion pipeline is versatile and can be adapted to other domains that involve image data and point-wise measurements.

Title: An Aspect Extraction Framework using Different Embedding Types, Learning Models, and Dependency Structure

• Authors: Ali Erkan, Tunga Güngör
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03512
• Pdf URL: https://arxiv.org/pdf/2503.03512
• Copy Paste: [[2503.03512]] An Aspect Extraction Framework using Different Embedding Types, Learning Models, and Dependency Structure(https://arxiv.org/abs/2503.03512)
• Keywords: extraction
• Abstract: Aspect-based sentiment analysis has gained significant attention in recent years due to its ability to provide fine-grained insights for sentiment expressions related to specific features of entities. An important component of aspect-based sentiment analysis is aspect extraction, which involves identifying and extracting aspect terms from text. Effective aspect extraction serves as the foundation for accurate sentiment analysis at the aspect level. In this paper, we propose aspect extraction models that use different types of embeddings for words and part-of-speech tags and that combine several learning models. We also propose tree positional encoding that is based on dependency parsing output to capture better the aspect positions in sentences. In addition, a new aspect extraction dataset is built for Turkish by machine translating an English dataset in a controlled setting. The experiments conducted on two Turkish datasets showed that the proposed models mostly outperform the studies that use the same datasets, and incorporating tree positional encoding increases the performance of the models.

Title: Do ImageNet-trained models learn shortcuts? The impact of frequency shortcuts on generalization

• Authors: Shunxin Wang, Raymond Veldhuis, Nicola Strisciuglio
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03519
• Pdf URL: https://arxiv.org/pdf/2503.03519
• Copy Paste: [[2503.03519]] Do ImageNet-trained models learn shortcuts? The impact of frequency shortcuts on generalization(https://arxiv.org/abs/2503.03519)
• Keywords: transformer
• Abstract: Frequency shortcuts refer to specific frequency patterns that models heavily rely on for correct classification. Previous studies have shown that models trained on small image datasets often exploit such shortcuts, potentially impairing their generalization performance. However, existing methods for identifying frequency shortcuts require expensive computations and become impractical for analyzing models trained on large datasets. In this work, we propose the first approach to more efficiently analyze frequency shortcuts at a larger scale. We show that both CNN and transformer models learn frequency shortcuts on ImageNet. We also expose that frequency shortcut solutions can yield good performance on out-of-distribution (OOD) test sets which largely retain texture information. However, these shortcuts, mostly aligned with texture patterns, hinder model generalization on rendition-based OOD test sets. These observations suggest that current OOD evaluations often overlook the impact of frequency shortcuts on model generalization. Future benchmarks could thus benefit from explicitly assessing and accounting for these shortcuts to build models that generalize across a broader range of OOD scenarios.

Title: AdaSin: Enhancing Hard Sample Metrics with Dual Adaptive Penalty for Face Recognition

• Authors: Qiqi Guo, Zhuowen Zheng, Guanghua Yang, Zhiquan Liu, Xiaofan Li, Jianqing Li, Jinyu Tian, Xueyuan Gong
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03528
• Pdf URL: https://arxiv.org/pdf/2503.03528
• Copy Paste: [[2503.03528]] AdaSin: Enhancing Hard Sample Metrics with Dual Adaptive Penalty for Face Recognition(https://arxiv.org/abs/2503.03528)
• Keywords: extraction
• Abstract: In recent years, the emergence of deep convolutional neural networks has positioned face recognition as a prominent research focus in computer vision. Traditional loss functions, such as margin-based, hard-sample mining-based, and hybrid approaches, have achieved notable performance improvements, with some leveraging curriculum learning to optimize training. However, these methods often fall short in effectively quantifying the difficulty of hard samples. To address this, we propose Adaptive Sine (AdaSin) loss function, which introduces the sine of the angle between a sample's embedding feature and its ground-truth class center as a novel difficulty metric. This metric enables precise and effective penalization of hard samples. By incorporating curriculum learning, the model dynamically adjusts classification boundaries across different training stages. Unlike previous adaptive-margin loss functions, AdaSin introduce a dual adaptive penalty, applied to both the positive and negative cosine similarities of hard samples. This design imposes stronger constraints, enhancing intra-class compactness and inter-class separability. The combination of the dual adaptive penalty and curriculum learning is guided by a well-designed difficulty metric. It enables the model to focus more effectively on hard samples in later training stages, and lead to the extraction of highly discriminative face features. Extensive experiments across eight benchmarks demonstrate that AdaSin achieves superior accuracy compared to other state-of-the-art methods.

Title: Unified Human Localization and Trajectory Prediction with Monocular Vision

• Authors: Po-Chien Luan, Yang Gao, Celine Demonsant, Alexandre Alahi
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03535
• Pdf URL: https://arxiv.org/pdf/2503.03535
• Copy Paste: [[2503.03535]] Unified Human Localization and Trajectory Prediction with Monocular Vision(https://arxiv.org/abs/2503.03535)
• Keywords: robust, transformer
• Abstract: Conventional human trajectory prediction models rely on clean curated data, requiring specialized equipment or manual labeling, which is often impractical for robotic applications. The existing predictors tend to overfit to clean observation affecting their robustness when used with noisy inputs. In this work, we propose MonoTransmotion (MT), a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks. Our framework has two main modules: Bird's Eye View (BEV) localization and trajectory prediction. The BEV localization module estimates the position of a person using 2D human poses, enhanced by a novel directional loss for smoother sequential localizations. The trajectory prediction module predicts future motion from these estimates. We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs. We validate our MT network on both curated and non-curated datasets. On the curated dataset, MT achieves around 12% improvement over baseline models on BEV localization and trajectory prediction. On real-world non-curated dataset, experimental results indicate that MT maintains similar performance levels, highlighting its robustness and generalization capability. The code is available at this https URL.

Title: Data Sharing, Privacy and Security Considerations in the Energy Sector: A Review from Technical Landscape to Regulatory Specifications

• Authors: Shiliang Zhang, Sabita Maharjan, Lee Andrew Bygrave, Shui Yu
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03539
• Pdf URL: https://arxiv.org/pdf/2503.03539
• Copy Paste: [[2503.03539]] Data Sharing, Privacy and Security Considerations in the Energy Sector: A Review from Technical Landscape to Regulatory Specifications(https://arxiv.org/abs/2503.03539)
• Keywords: security, privacy
• Abstract: Decarbonization, decentralization and digitalization are the three key elements driving the twin energy transition. The energy system is evolving to a more data driven ecosystem, leading to the need of communication and storage of large amount of data of different resolution from the prosumers and other stakeholders in the energy ecosystem. While the energy system is certainly advancing, this paradigm shift is bringing in new privacy and security issues related to collection, processing and storage of data - not only from the technical dimension, but also from the regulatory perspective. Understanding data privacy and security in the evolving energy system, regarding regulatory compliance, is an immature field of research. Contextualized knowledge of how related issues are regulated is still in its infancy, and the practical and technical basis for the regulatory framework for data privacy and security is not clear. To fill this gap, this paper conducts a comprehensive review of the data-related issues for the energy system by integrating both technical and regulatory dimensions. We start by reviewing open-access data, data communication and data-processing techniques for the energy system, and use it as the basis to connect the analysis of data-related issues from the integrated perspective. We classify the issues into three categories: (i) data-sharing among energy end users and stakeholders (ii) privacy of end users, and (iii) cyber security, and then explore these issues from a regulatory perspective. We analyze the evolution of related regulations, and introduce the relevant regulatory initiatives for the categorized issues in terms of regulatory definitions, concepts, principles, rights and obligations in the context of energy systems. Finally, we provide reflections on the gaps that still exist, and guidelines for regulatory frameworks for a truly participatory energy system.

Title: A self-supervised cyclic neural-analytic approach for novel view synthesis and 3D reconstruction

• Authors: Dragos Costea, Alina Marcu, Marius Leordeanu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03543
• Pdf URL: https://arxiv.org/pdf/2503.03543
• Copy Paste: [[2503.03543]] A self-supervised cyclic neural-analytic approach for novel view synthesis and 3D reconstruction(https://arxiv.org/abs/2503.03543)
• Keywords: transformer
• Abstract: Generating novel views from recorded videos is crucial for enabling autonomous UAV navigation. Recent advancements in neural rendering have facilitated the rapid development of methods capable of rendering new trajectories. However, these methods often fail to generalize well to regions far from the training data without an optimized flight path, leading to suboptimal reconstructions. We propose a self-supervised cyclic neural-analytic pipeline that combines high-quality neural rendering outputs with precise geometric insights from analytical methods. Our solution improves RGB and mesh reconstructions for novel view synthesis, especially in undersampled areas and regions that are completely different from the training dataset. We use an effective transformer-based architecture for image reconstruction to refine and adapt the synthesis process, enabling effective handling of novel, unseen poses without relying on extensive labeled datasets. Our findings demonstrate substantial improvements in rendering views of novel and also 3D reconstruction, which to the best of our knowledge is a first, setting a new standard for autonomous navigation in complex outdoor environments.

Title: Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulation

• Authors: Xiaomeng Zhu, Yuyang Li, Leiyao Cui, Pengfei Li, Huan-ang Gao, Yixin Zhu, Hao Zhao
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03556
• Pdf URL: https://arxiv.org/pdf/2503.03556
• Copy Paste: [[2503.03556]] Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulation(https://arxiv.org/abs/2503.03556)
• Keywords: large language model
• Abstract: Object affordance reasoning, the ability to infer object functionalities based on physical properties, is fundamental for task-oriented planning and activities in both humans and Artificial Intelligence (AI). This capability, required for planning and executing daily activities in a task-oriented manner, relies on commonsense knowledge of object physics and functionalities, extending beyond simple object recognition. Current computational models for affordance reasoning from perception lack generalizability, limiting their applicability in novel scenarios. Meanwhile, comprehensive Large Language Models (LLMs) with emerging reasoning capabilities are challenging to deploy on local devices for task-oriented manipulations. Here, we introduce LVIS-Aff, a large-scale dataset comprising 1,496 tasks and 119k images, designed to enhance the generalizability of affordance reasoning from perception. Utilizing this dataset, we develop Afford-X, an end-to-end trainable affordance reasoning model that incorporates Verb Attention and Bi-Fusion modules to improve multi-modal understanding. This model achieves up to a 12.1% performance improvement over the best-reported results from non-LLM methods, while also demonstrating a 1.2% enhancement compared to our previous conference paper. Additionally, it maintains a compact 187M parameter size and infers nearly 50 times faster than the GPT-4V API. Our work demonstrates the potential for efficient, generalizable affordance reasoning models that can be deployed on local devices for task-oriented manipulations. We showcase Afford-X's effectiveness in enabling task-oriented manipulations for robots across various tasks and environments, underscoring its efficiency and broad implications for advancing robotics and AI systems in real-world applications.

Title: Scaling Crowdsourced Election Monitoring: Construction and Evaluation of Classification Models for Multilingual and Cross-Domain Classification Settings

• Authors: Jabez Magomere, Scott Hale
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03582
• Pdf URL: https://arxiv.org/pdf/2503.03582
• Copy Paste: [[2503.03582]] Scaling Crowdsourced Election Monitoring: Construction and Evaluation of Classification Models for Multilingual and Cross-Domain Classification Settings(https://arxiv.org/abs/2503.03582)
• Keywords: transformer
• Abstract: The adoption of crowdsourced election monitoring as a complementary alternative to traditional election monitoring is on the rise. Yet, its reliance on digital response volunteers to manually process incoming election reports poses a significant scaling bottleneck. In this paper, we address the challenge of scaling crowdsourced election monitoring by advancing the task of automated classification of crowdsourced election reports to multilingual and cross-domain classification settings. We propose a two-step classification approach of first identifying informative reports and then categorising them into distinct information types. We conduct classification experiments using multilingual transformer models such as XLM-RoBERTa and multilingual embeddings such as SBERT, augmented with linguistically motivated features. Our approach achieves F1-Scores of 77\% for informativeness detection and 75\% for information type classification. We conduct cross-domain experiments, applying models trained in a source electoral domain to a new target electoral domain in zero-shot and few-shot classification settings. Our results show promising potential for model transfer across electoral domains, with F1-Scores of 59\% in zero-shot and 63\% in few-shot settings. However, our analysis also reveals a performance bias in detecting informative English reports over Swahili, likely due to imbalances in the training data, indicating a need for caution when deploying classification models in real-world election scenarios.

Title: Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories

• Authors: Alperen Yildiz, Sin G. Teo, Yiling Lou, Yebo Feng, Chong Wang, Dinil M. Divakaran
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03586
• Pdf URL: https://arxiv.org/pdf/2503.03586
• Copy Paste: [[2503.03586]] Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories(https://arxiv.org/abs/2503.03586)
• Keywords: security, large language model
• Abstract: Large Language Models (LLMs) have shown promise in software vulnerability detection, particularly on function-level benchmarks like Devign and BigVul. However, real-world detection requires interprocedural analysis, as vulnerabilities often emerge through multi-hop function calls rather than isolated functions. While repository-level benchmarks like ReposVul and VulEval introduce interprocedural context, they remain computationally expensive, lack pairwise evaluation of vulnerability fixes, and explore limited context retrieval, limiting their practicality. We introduce JitVul, a JIT vulnerability detection benchmark linking each function to its vulnerability-introducing and fixing commits. Built from 879 CVEs spanning 91 vulnerability types, JitVul enables comprehensive evaluation of detection capabilities. Our results show that ReAct Agents, leveraging thought-action-observation and interprocedural context, perform better than LLMs in distinguishing vulnerable from benign code. While prompting strategies like Chain-of-Thought help LLMs, ReAct Agents require further refinement. Both methods show inconsistencies, either misidentifying vulnerabilities or over-analyzing security guards, indicating significant room for improvement.

Title: PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention

• Authors: Lida Chen, Dong Xu, Chenxin An, Xintao Wang, Yikai Zhang, Jiangjie Chen, Zujie Liang, Feng Wei, Jiaqing Liang, Yanghua Xiao, Wei Wang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03588
• Pdf URL: https://arxiv.org/pdf/2503.03588
• Copy Paste: [[2503.03588]] PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention(https://arxiv.org/abs/2503.03588)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) face efficiency bottlenecks due to the quadratic complexity of the attention mechanism when processing long contexts. Sparse attention methods offer a promising solution, but existing approaches often suffer from incomplete effective context and/or require complex implementation of pipeline. We present a comprehensive analysis of sparse attention for autoregressive LLMs from the respective of receptive field, recognize the suboptimal nature of existing methods for expanding the receptive field, and introduce PowerAttention, a novel sparse attention design that facilitates effective and complete context extension through the theoretical analysis. PowerAttention achieves exponential receptive field growth in $d$-layer LLMs, allowing each output token to attend to $2^d$ tokens, ensuring completeness and continuity of the receptive field. Experiments demonstrate that PowerAttention outperforms existing static sparse attention methods by $5\sim 40\%$, especially on tasks demanding long-range dependencies like Passkey Retrieval and RULER, while maintaining a comparable time complexity to sliding window attention. Efficiency evaluations further highlight PowerAttention's superior speedup in both prefilling and decoding phases compared with dynamic sparse attentions and full attention ($3.0\times$ faster on 128K context), making it a highly effective and user-friendly solution for processing long sequences in LLMs.

Title: Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias

• Authors: Rui Lu, Runzhe Wang, Kaifeng Lyu, Xitai Jiang, Gao Huang, Mengdi Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03595
• Pdf URL: https://arxiv.org/pdf/2503.03595
• Copy Paste: [[2503.03595]] Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias(https://arxiv.org/abs/2503.03595)
• Keywords: diffusion, transformer
• Abstract: Score-based diffusion models have achieved incredible performance in generating realistic images, audio, and video data. While these models produce high-quality samples with impressive details, they often introduce unrealistic artifacts, such as distorted fingers or hallucinated texts with no meaning. This paper focuses on textual hallucinations, where diffusion models correctly generate individual symbols but assemble them in a nonsensical manner. Through experimental probing, we consistently observe that such phenomenon is attributed it to the network's local generation bias. Denoising networks tend to produce outputs that rely heavily on highly correlated local regions, particularly when different dimensions of the data distribution are nearly pairwise independent. This behavior leads to a generation process that decomposes the global distribution into separate, independent distributions for each symbol, ultimately failing to capture the global structure, including underlying grammar. Intriguingly, this bias persists across various denoising network architectures including MLP and transformers which have the structure to model global dependency. These findings also provide insights into understanding other types of hallucinations, extending beyond text, as a result of implicit biases in the denoising models. Additionally, we theoretically analyze the training dynamics for a specific case involving a two-layer MLP learning parity points on a hypercube, offering an explanation of its underlying mechanism.

Title: REGRACE: A Robust and Efficient Graph-based Re-localization Algorithm using Consistency Evaluation

• Authors: Débora N.P. Oliveira, Joshua Knights, Sebastián Barbas Laina, Simon Boche, Wolfram Burgard, Stefan Leutenegger
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03599
• Pdf URL: https://arxiv.org/pdf/2503.03599
• Copy Paste: [[2503.03599]] REGRACE: A Robust and Efficient Graph-based Re-localization Algorithm using Consistency Evaluation(https://arxiv.org/abs/2503.03599)
• Keywords: robust
• Abstract: Loop closures are essential for correcting odometry drift and creating consistent maps, especially in the context of large-scale navigation. Current methods using dense point clouds for accurate place recognition do not scale well due to computationally expensive scan-to-scan comparisons. Alternative object-centric approaches are more efficient but often struggle with sensitivity to viewpoint variation. In this work, we introduce REGRACE, a novel approach that addresses these challenges of scalability and perspective difference in re-localization by using LiDAR-based submaps. We introduce rotation-invariant features for each labeled object and enhance them with neighborhood context through a graph neural network. To identify potential revisits, we employ a scalable bag-of-words approach, pooling one learned global feature per submap. Additionally, we define a revisit with geometrical consistency cues rather than embedding distance, allowing us to recognize far-away loop closures. Our evaluations demonstrate that REGRACE achieves similar results compared to state-of-the-art place recognition and registration baselines while being twice as fast.

Title: Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders

• Authors: Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina, Anton Razzhigaev, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03601
• Pdf URL: https://arxiv.org/pdf/2503.03601
• Copy Paste: [[2503.03601]] Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders(https://arxiv.org/abs/2503.03601)
• Keywords: interpretability, large language model
• Abstract: Artificial Text Detection (ATD) is becoming increasingly important with the rise of advanced Large Language Models (LLMs). Despite numerous efforts, no single algorithm performs consistently well across different types of unseen text or guarantees effective generalization to new LLMs. Interpretability plays a crucial role in achieving this goal. In this study, we enhance ATD interpretability by using Sparse Autoencoders (SAE) to extract features from Gemma-2-2b residual stream. We identify both interpretable and efficient features, analyzing their semantics and relevance through domain- and model-specific statistics, a steering approach, and manual or LLM-based interpretation. Our methods offer valuable insights into how texts from various models differ from human-written content. We show that modern LLMs have a distinct writing style, especially in information-dense domains, even though they can produce human-like outputs with personalized prompts.

Title: Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling

• Authors: Keqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03607
• Pdf URL: https://arxiv.org/pdf/2503.03607
• Copy Paste: [[2503.03607]] Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling(https://arxiv.org/abs/2503.03607)
• Keywords: large language model
• Abstract: The in-context learning capabilities of large language models (LLMs) show great potential in mental health support. However, the lack of counseling datasets, particularly in Chinese corpora, restricts their application in this field. To address this, we constructed Psy-Insight, the first mental health-oriented explainable multi-task bilingual dataset. We collected face-to-face multi-turn counseling dialogues, which are annotated with multi-task labels and conversation process explanations. Our annotations include psychotherapy, emotion, strategy, and topic labels, as well as turn-level reasoning and session-level guidance. Psy-Insight is not only suitable for tasks such as label recognition but also meets the need for training LLMs to act as empathetic counselors through logical reasoning. Experiments show that training LLMs on Psy-Insight enables the models to not only mimic the conversation style but also understand the underlying strategies and reasoning of counseling.

Title: CLIP is Strong Enough to Fight Back: Test-time Counterattacks towards Zero-shot Adversarial Robustness of CLIP

• Authors: Songlong Xing, Zhengyu Zhao, Nicu Sebe
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03613
• Pdf URL: https://arxiv.org/pdf/2503.03613
• Copy Paste: [[2503.03613]] CLIP is Strong Enough to Fight Back: Test-time Counterattacks towards Zero-shot Adversarial Robustness of CLIP(https://arxiv.org/abs/2503.03613)
• Keywords: attack, robust
• Abstract: Despite its prevalent use in image-text matching tasks in a zero-shot manner, CLIP has been shown to be highly vulnerable to adversarial perturbations added onto images. Recent studies propose to finetune the vision encoder of CLIP with adversarial samples generated on the fly, and show improved robustness against adversarial attacks on a spectrum of downstream datasets, a property termed as zero-shot robustness. In this paper, we show that malicious perturbations that seek to maximise the classification loss lead to `falsely stable' images, and propose to leverage the pre-trained vision encoder of CLIP to counterattack such adversarial images during inference to achieve robustness. Our paradigm is simple and training-free, providing the first method to defend CLIP from adversarial attacks at test time, which is orthogonal to existing methods aiming to boost zero-shot adversarial robustness of CLIP. We conduct experiments across 16 classification datasets, and demonstrate stable and consistent gains compared to test-time defence methods adapted from existing adversarial robustness studies that do not rely on external networks, without noticeably impairing performance on clean images. We also show that our paradigm can be employed on CLIP models that have been adversarially finetuned to further enhance their robustness at test time. Our code is available \href{this https URL}{here}.

Title: It's My Data Too: Private ML for Datasets with Multi-User Training Examples

• Authors: Arun Ganesh, Ryan McKenna, Brendan McMahan, Adam Smith, Fan Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03622
• Pdf URL: https://arxiv.org/pdf/2503.03622
• Copy Paste: [[2503.03622]] It's My Data Too: Private ML for Datasets with Multi-User Training Examples(https://arxiv.org/abs/2503.03622)
• Keywords: privacy, transformer
• Abstract: We initiate a study of algorithms for model training with user-level differential privacy (DP), where each example may be attributed to multiple users, which we call the multi-attribution model. We first provide a carefully chosen definition of user-level DP under the multi-attribution model. Training in the multi-attribution model is facilitated by solving the contribution bounding problem, i.e. the problem of selecting a subset of the dataset for which each user is associated with a limited number of examples. We propose a greedy baseline algorithm for the contribution bounding problem. We then empirically study this algorithm for a synthetic logistic regression task and a transformer training task, including studying variants of this baseline algorithm that optimize the subset chosen using different techniques and criteria. We find that the baseline algorithm remains competitive with its variants in most settings, and build a better understanding of the practical importance of a bias-variance tradeoff inherent in solutions to the contribution bounding problem.

Title: An Adaptive Underwater Image Enhancement Framework via Multi-Domain Fusion and Color Compensation

• Authors: Yuezhe Tian, Kangchen Yao, Xiaoyang Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03640
• Pdf URL: https://arxiv.org/pdf/2503.03640
• Copy Paste: [[2503.03640]] An Adaptive Underwater Image Enhancement Framework via Multi-Domain Fusion and Color Compensation(https://arxiv.org/abs/2503.03640)
• Keywords: robust
• Abstract: Underwater optical imaging is severely degraded by light absorption, scattering, and color distortion, hindering visibility and accurate image analysis. This paper presents an adaptive enhancement framework integrating illumination compensation, multi-domain filtering, and dynamic color correction. A hybrid illumination compensation strategy combining CLAHE, Gamma correction, and Retinex enhances visibility. A two-stage filtering process, including spatial-domain (Gaussian, Bilateral, Guided) and frequency-domain (Fourier, Wavelet) methods, effectively reduces noise while preserving details. To correct color distortion, an adaptive color compensation (ACC) model estimates spectral attenuation and water type to combine RCP, DCP, and MUDCP dynamically. Finally, a perceptually guided color balance mechanism ensures natural color restoration. Experimental results on benchmark datasets demonstrate superior performance over state-of-the-art methods in contrast enhancement, color correction, and structural preservation, making the framework robust for underwater imaging applications.

Title: DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms

• Authors: Xiaojun Bi, Shuo Li, Ziyue Wang, Fuwen Luo, Weizheng Qiao, Lu Han, Ziwei Sun, Peng Li, Yang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03644
• Pdf URL: https://arxiv.org/pdf/2503.03644
• Copy Paste: [[2503.03644]] DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms(https://arxiv.org/abs/2503.03644)
• Keywords: extraction
• Abstract: Dongba pictographs are the only pictographs still in use in the world. They have pictorial ideographic features, and their symbols carry rich cultural and contextual information. Due to the lack of relevant datasets, existing research has difficulty in advancing the study of semantic understanding of Dongba pictographs. To this end, we propose DongbaMIE, the first multimodal dataset for semantic understanding and extraction of Dongba pictographs. The dataset consists of Dongba pictograph images and their corresponding Chinese semantic annotations. It contains 23,530 sentence-level and 2,539 paragraph-level images, covering four semantic dimensions: objects, actions, relations, and attributes. We systematically evaluate the GPT-4o, Gemini-2.0, and Qwen2-VL models. Experimental results show that the F1 scores of GPT-4o and Gemini in the best object extraction are only 3.16 and 3.11 respectively. The F1 score of Qwen2-VL after supervised fine-tuning is only 11.49. These results suggest that current large multimodal models still face significant challenges in accurately recognizing the diverse semantic information in Dongba pictographs. The dataset can be obtained from this URL.

Title: Psy-Copilot: Visual Chain of Thought for Counseling

• Authors: Keqi Chen, Zekai Sun, Huijun Lian, Yingming Gao, Ya Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03645
• Pdf URL: https://arxiv.org/pdf/2503.03645
• Copy Paste: [[2503.03645]] Psy-Copilot: Visual Chain of Thought for Counseling(https://arxiv.org/abs/2503.03645)
• Keywords: large language model
• Abstract: Large language models (LLMs) are becoming increasingly popular in the field of psychological counseling. However, when human therapists work with LLMs in therapy sessions, it is hard to understand how the model gives the answers. To address this, we have constructed Psy-COT, a graph designed to visualize the thought processes of LLMs during therapy sessions. The Psy-COT graph presents semi-structured counseling conversations alongside step-by-step annotations that capture the reasoning and insights of therapists. Moreover, we have developed Psy-Copilot, which is a conversational AI assistant designed to assist human psychological therapists in their consultations. It can offer traceable psycho-information based on retrieval, including response candidates, similar dialogue sessions, related strategies, and visual traces of results. We have also built an interactive platform for AI-assisted counseling. It has an interface that displays the relevant parts of the retrieval sub-graph. The Psy-Copilot is designed not to replace psychotherapists but to foster collaboration between AI and human therapists, thereby promoting mental health development. Our code and demo are both open-sourced and available for use.

Title: DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles

• Authors: Rui Zhao, Weijia Mao, Mike Zheng Shou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03651
• Pdf URL: https://arxiv.org/pdf/2503.03651
• Copy Paste: [[2503.03651]] DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles(https://arxiv.org/abs/2503.03651)
• Keywords: generative
• Abstract: Adapting generative models to specific domains presents an effective solution for satisfying specialized requirements. However, adapting to some complex domains remains challenging, especially when these domains require substantial paired data to capture the targeted distributions. Since unpaired data from a single modality, such as vision or language, is more readily available, we utilize the bidirectional mappings between vision and language learned by the unified generative model to enable training on unpaired data for domain adaptation. Specifically, we propose DoraCycle, which integrates two multimodal cycles: text-to-image-to-text and image-to-text-to-image. The model is optimized through cross-entropy loss computed at the cycle endpoints, where both endpoints share the same modality. This facilitates self-evolution of the model without reliance on annotated text-image pairs. Experimental results demonstrate that for tasks independent of paired knowledge, such as stylization, DoraCycle can effectively adapt the unified model using only unpaired data. For tasks involving new paired knowledge, such as specific identities, a combination of a small set of paired image-text examples and larger-scale unpaired data is sufficient for effective domain-oriented adaptation. The code will be released at this https URL.

Title: Token-Level Privacy in Large Language Models

• Authors: Re'em Harel, Niv Gilboa, Yuval Pinter
• Subjects: cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03652
• Pdf URL: https://arxiv.org/pdf/2503.03652
• Copy Paste: [[2503.03652]] Token-Level Privacy in Large Language Models(https://arxiv.org/abs/2503.03652)
• Keywords: privacy, robust, large language model
• Abstract: The use of language models as remote services requires transmitting private information to external providers, raising significant privacy concerns. This process not only risks exposing sensitive data to untrusted service providers but also leaves it vulnerable to interception by eavesdroppers. Existing privacy-preserving methods for natural language processing (NLP) interactions primarily rely on semantic similarity, overlooking the role of contextual information. In this work, we introduce dchi-stencil, a novel token-level privacy-preserving mechanism that integrates contextual and semantic information while ensuring strong privacy guarantees under the dchi differential privacy framework, achieving 2epsilon-dchi-privacy. By incorporating both semantic and contextual nuances, dchi-stencil achieves a robust balance between privacy and utility. We evaluate dchi-stencil using state-of-the-art language models and diverse datasets, achieving comparable and even better trade-off between utility and privacy compared to existing methods. This work highlights the potential of dchi-stencil to set a new standard for privacy-preserving NLP in modern, high-risk applications.

Title: Improving Neutral Point of View Text Generation through Parameter-Efficient Reinforcement Learning and a Small-Scale High-Quality Dataset

• Authors: Jessica Hoffmann, Christiane Ahlheim, Zac Yu, Aria Walfrand, Jarvis Jin, Marie Tano, Ahmad Beirami, Erin van Liemt, Nithum Thain, Hakim Sidahmed, Lucas Dixon
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03654
• Pdf URL: https://arxiv.org/pdf/2503.03654
• Copy Paste: [[2503.03654]] Improving Neutral Point of View Text Generation through Parameter-Efficient Reinforcement Learning and a Small-Scale High-Quality Dataset(https://arxiv.org/abs/2503.03654)
• Keywords: generative, large language model
• Abstract: This paper describes the construction of a dataset and the evaluation of training methods to improve generative large language models' (LLMs) ability to answer queries on sensitive topics with a Neutral Point of View (NPOV), i.e., to provide significantly more informative, diverse and impartial answers. The dataset, the SHQ-NPOV dataset, comprises 300 high-quality, human-written quadruplets: a query on a sensitive topic, an answer, an NPOV rating, and a set of links to source texts elaborating the various points of view. The first key contribution of this paper is a new methodology to create such datasets through iterative rounds of human peer-critique and annotator training, which we release alongside the dataset. The second key contribution is the identification of a highly effective training regime for parameter-efficient reinforcement learning (PE-RL) to improve NPOV generation. We compare and extensively evaluate PE-RL and multiple baselines-including LoRA finetuning (a strong baseline), SFT and RLHF. PE-RL not only improves on overall NPOV quality compared to the strongest baseline ($97.06\%\rightarrow 99.08\%$), but also scores much higher on features linguists identify as key to separating good answers from the best answers ($60.25\%\rightarrow 85.21\%$ for presence of supportive details, $68.74\%\rightarrow 91.43\%$ for absence of oversimplification). A qualitative analysis corroborates this. Finally, our evaluation finds no statistical differences between results on topics that appear in the training dataset and those on separated evaluation topics, which provides strong evidence that our approach to training PE-RL exhibits very effective out of topic generalization.

Title: Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns

• Authors: Dong Tian, Ge Li, Hongyi Zhou, Onur Celik, Gerhard Neumann
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03660
• Pdf URL: https://arxiv.org/pdf/2503.03660
• Copy Paste: [[2503.03660]] Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns(https://arxiv.org/abs/2503.03660)
• Keywords: robust, transformer
• Abstract: Soft Actor-Critic (SAC) critically depends on its critic network, which typically evaluates a single state-action pair to guide policy updates. Using N-step returns is a common practice to reduce the bias in the target values of the critic. However, using N-step returns can again introduce high variance and necessitates importance sampling, often destabilizing training. Recent algorithms have also explored action chunking-such as direct action repetition and movement primitives-to enhance exploration. In this paper, we propose a Transformer-based Critic Network for SAC that integrates the N-returns framework in a stable and efficient manner. Unlike approaches that perform chunking in the actor network, we feed chunked actions into the critic network to explore potential performance gains. Our architecture leverages the Transformer's ability to process sequential information, facilitating more robust value estimation. Empirical results show that this method not only achieves efficient, stable training but also excels in sparse reward/multi-phase environments-traditionally a challenge for step-based methods. These findings underscore the promise of combining Transformer-based critics with N-returns to advance reinforcement learning performance

Title: A Generative Approach to High Fidelity 3D Reconstruction from Text Data

• Authors: Venkat Kumar R, Deepak Saravanan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03664
• Pdf URL: https://arxiv.org/pdf/2503.03664
• Copy Paste: [[2503.03664]] A Generative Approach to High Fidelity 3D Reconstruction from Text Data(https://arxiv.org/abs/2503.03664)
• Keywords: diffusion, generative
• Abstract: The convergence of generative artificial intelligence and advanced computer vision technologies introduces a groundbreaking approach to transforming textual descriptions into three-dimensional representations. This research proposes a fully automated pipeline that seamlessly integrates text-to-image generation, various image processing techniques, and deep learning methods for reflection removal and 3D reconstruction. By leveraging state-of-the-art generative models like Stable Diffusion, the methodology translates natural language inputs into detailed 3D models through a multi-stage workflow. The reconstruction process begins with the generation of high-quality images from textual prompts, followed by enhancement by a reinforcement learning agent and reflection removal using the Stable Delight model. Advanced image upscaling and background removal techniques are then applied to further enhance visual fidelity. These refined two-dimensional representations are subsequently transformed into volumetric 3D models using sophisticated machine learning algorithms, capturing intricate spatial relationships and geometric characteristics. This process achieves a highly structured and detailed output, ensuring that the final 3D models reflect both semantic accuracy and geometric precision. This approach addresses key challenges in generative reconstruction, such as maintaining semantic coherence, managing geometric complexity, and preserving detailed visual information. Comprehensive experimental evaluations will assess reconstruction quality, semantic accuracy, and geometric fidelity across diverse domains and varying levels of complexity. By demonstrating the potential of AI-driven 3D reconstruction techniques, this research offers significant implications for fields such as augmented reality (AR), virtual reality (VR), and digital content creation.

Title: Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction

• Authors: Gustaw Opiełka, Hannes Rosenbusch, Claire E. Stevenson
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03666
• Pdf URL: https://arxiv.org/pdf/2503.03666
• Copy Paste: [[2503.03666]] Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction(https://arxiv.org/abs/2503.03666)
• Keywords: large language model
• Abstract: Analogical reasoning relies on conceptual abstractions, but it is unclear whether Large Language Models (LLMs) harbor such internal representations. We explore distilled representations from LLM activations and find that function vectors (FVs; Todd et al., 2024) - compact representations for in-context learning (ICL) tasks - are not invariant to simple input changes (e.g., open-ended vs. multiple-choice), suggesting they capture more than pure concepts. Using representational similarity analysis (RSA), we localize a small set of attention heads that encode invariant concept vectors (CVs) for verbal concepts like "antonym". These CVs function as feature detectors that operate independently of the final output - meaning that a model may form a correct internal representation yet still produce an incorrect output. Furthermore, CVs can be used to causally guide model behaviour. However, for more abstract concepts like "previous" and "next", we do not observe invariant linear representations, a finding we link to generalizability issues LLMs display within these domains.

Title: Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models

• Authors: Bar Karov, Dor Zohar, Yam Marcovitz
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03669
• Pdf URL: https://arxiv.org/pdf/2503.03669
• Copy Paste: [[2503.03669]] Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models(https://arxiv.org/abs/2503.03669)
• Keywords: large language model
• Abstract: We present Attentive Reasoning Queries (ARQs), a novel structured reasoning approach that significantly improves instruction-following in Large Language Models through domain-specialized reasoning blueprints. While LLMs demonstrate remarkable capabilities across diverse tasks, they often fail to maintain adherence to complex, use-case-specific instructions during multi-turn conversations, presenting challenges for business-critical applications. ARQs address this limitation by guiding LLMs through systematic reasoning steps with targeted queries that reinstate critical instructions and facilitate intermediate reasoning throughout the completion process. In extensive testing within Parlant, our framework for reliable customer-facing agents in which ARQs were born out of necessity, they achieved a 90.2% success rate across 87 test scenarios, outperforming both Chain-of-Thought reasoning (86.1%) and direct response generation (81.5%). ARQs showed particular strength in addressing persistent failure modes like guideline re-application and hallucination prevention. Our analysis also revealed that ARQs can potentially be more computationally efficient than free-form reasoning when carefully designed. These findings demonstrate that structured reasoning approaches provide effective mechanisms for controlling how LLMs process information and make decisions in complex scenarios.

Title: Towards Trustworthy Federated Learning

• Authors: Alina Basharat, Yijun Bian, Ping Xu, Zhi Tian
• Subjects: cs.LG, cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.03684
• Pdf URL: https://arxiv.org/pdf/2503.03684
• Copy Paste: [[2503.03684]] Towards Trustworthy Federated Learning(https://arxiv.org/abs/2503.03684)
• Keywords: privacy, defense, attack, robust, federate, fair
• Abstract: This paper develops a comprehensive framework to address three critical trustworthy challenges in federated learning (FL): robustness against Byzantine attacks, fairness, and privacy preservation. To improve the system's defense against Byzantine attacks that send malicious information to bias the system's performance, we develop a Two-sided Norm Based Screening (TNBS) mechanism, which allows the central server to crop the gradients that have the l lowest norms and h highest norms. TNBS functions as a screening tool to filter out potential malicious participants whose gradients are far from the honest ones. To promote egalitarian fairness, we adopt the q-fair federated learning (q-FFL). Furthermore, we adopt a differential privacy-based scheme to prevent raw data at local clients from being inferred by curious parties. Convergence guarantees are provided for the proposed framework under different scenarios. Experimental results on real datasets demonstrate that the proposed framework effectively improves robustness and fairness while managing the trade-off between privacy and accuracy. This work appears to be the first study that experimentally and theoretically addresses fairness, privacy, and robustness in trustworthy FL.

Title: MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems

• Authors: Rui Ye, Shuo Tang, Rui Ge, Yaxin Du, Zhenfei Yin, Siheng Chen, Jing Shao
• Subjects: cs.CL, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.03686
• Pdf URL: https://arxiv.org/pdf/2503.03686
• Copy Paste: [[2503.03686]] MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems(https://arxiv.org/abs/2503.03686)
• Keywords: generative
• Abstract: LLM-based multi-agent systems (MAS) have shown significant potential in tackling diverse tasks. However, to design effective MAS, existing approaches heavily rely on manual configurations or multiple calls of advanced LLMs, resulting in inadaptability and high inference costs. In this paper, we simplify the process of building an MAS by reframing it as a generative language task, where the input is a user query and the output is a corresponding MAS. To address this novel task, we unify the representation of MAS as executable code and propose a consistency-oriented data construction pipeline to create a high-quality dataset comprising coherent and consistent query-MAS pairs. Using this dataset, we train MAS-GPT, an open-source medium-sized LLM that is capable of generating query-adaptive MAS within a single LLM inference. The generated MAS can be seamlessly applied to process user queries and deliver high-quality responses. Extensive experiments on 9 benchmarks and 5 LLMs show that the proposed MAS-GPT consistently outperforms 10+ baseline MAS methods on diverse settings, indicating MAS-GPT's high effectiveness, efficiency and strong generalization ability. Code will be available at this https URL.

Title: DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance

• Authors: Zhao Yang, Zezhong Qian, Xiaofan Li, Weixiang Xu, Gongpeng Zhao, Ruohong Yu, Lingsi Zhu, Longjun Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03689
• Pdf URL: https://arxiv.org/pdf/2503.03689
• Copy Paste: [[2503.03689]] DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance(https://arxiv.org/abs/2503.03689)
• Keywords: diffusion, segmentation
• Abstract: Accurate and high-fidelity driving scene reconstruction demands the effective utilization of comprehensive scene information as conditional inputs. Existing methods predominantly rely on 3D bounding boxes and BEV road maps for foreground and background control, which fail to capture the full complexity of driving scenes and adequately integrate multimodal information. In this work, we present DualDiff, a dual-branch conditional diffusion model designed to enhance driving scene generation across multiple views and video sequences. Specifically, we introduce Occupancy Ray-shape Sampling (ORS) as a conditional input, offering rich foreground and background semantics alongside 3D spatial geometry to precisely control the generation of both elements. To improve the synthesis of fine-grained foreground objects, particularly complex and distant ones, we propose a Foreground-Aware Mask (FGM) denoising loss function. Additionally, we develop the Semantic Fusion Attention (SFA) mechanism to dynamically prioritize relevant information and suppress noise, enabling more effective multimodal fusion. Finally, to ensure high-quality image-to-video generation, we introduce the Reward-Guided Diffusion (RGD) framework, which maintains global consistency and semantic coherence in generated videos. Extensive experiments demonstrate that DualDiff achieves state-of-the-art (SOTA) performance across multiple datasets. On the NuScenes dataset, DualDiff reduces the FID score by 4.09% compared to the best baseline. In downstream tasks, such as BEV segmentation, our method improves vehicle mIoU by 4.50% and road mIoU by 1.70%, while in BEV 3D object detection, the foreground mAP increases by 1.46%. Code will be made available at this https URL.

Title: Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models

• Authors: Jiyue Jiang, Alfred Kar Yin Truong, Yanyu Chen, Qinghang Bao, Sheng Wang, Pengan Chen, Jiuming Wang, Lingpeng Kong, Yu Li, Chuan Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03702
• Pdf URL: https://arxiv.org/pdf/2503.03702
• Copy Paste: [[2503.03702]] Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models(https://arxiv.org/abs/2503.03702)
• Keywords: large language model
• Abstract: High-quality data resources play a crucial role in learning large language models (LLMs), particularly for low-resource languages like Cantonese. Despite having more than 85 million native speakers, Cantonese is still considered a low-resource language in the field of natural language processing (NLP) due to factors such as the dominance of Mandarin, lack of cohesion within the Cantonese-speaking community, diversity in character encoding and input methods, and the tendency of overseas Cantonese speakers to prefer using English. In addition, rich colloquial vocabulary of Cantonese, English loanwords, and code-switching characteristics add to the complexity of corpus collection and processing. To address these challenges, we collect Cantonese texts from a variety of sources, including open source corpora, Hong Kong-specific forums, Wikipedia, and Common Crawl data. We conduct rigorous data processing through language filtering, quality filtering, content filtering, and de-duplication steps, successfully constructing a high-quality Cantonese corpus of over 2 billion tokens for training large language models. We further refined the model through supervised fine-tuning (SFT) on curated Cantonese tasks, enhancing its ability to handle specific applications. Upon completion of the training, the model achieves state-of-the-art (SOTA) performance on four Cantonese benchmarks. After training on our dataset, the model also exhibits improved performance on other mainstream language tasks.

Title: A Practical Memory Injection Attack against LLM Agents

• Authors: Shen Dong, Shaocheng Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, Zhen Xiang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03704
• Pdf URL: https://arxiv.org/pdf/2503.03704
• Copy Paste: [[2503.03704]] A Practical Memory Injection Attack against LLM Agents(https://arxiv.org/abs/2503.03704)
• Keywords: attack, large language model
• Abstract: Agents based on large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious. In this paper, we propose a novel Memory INJection Attack, MINJA, that enables the injection of malicious records into the memory bank by only interacting with the agent via queries and output observations. These malicious records are designed to elicit a sequence of malicious reasoning steps leading to undesirable agent actions when executing the victim user's query. Specifically, we introduce a sequence of bridging steps to link the victim query to the malicious reasoning steps. During the injection of the malicious record, we propose an indication prompt to guide the agent to autonomously generate our designed bridging steps. We also propose a progressive shortening strategy that gradually removes the indication prompt, such that the malicious record will be easily retrieved when processing the victim query comes after. Our extensive experiments across diverse agents demonstrate the effectiveness of MINJA in compromising agent memory. With minimal requirements for execution, MINJA enables any user to influence agent memory, highlighting practical risks of LLM agents.

Title: Effective LLM Knowledge Learning via Model Generalization

• Authors: Mingkang Zhu, Xi Chen, Zhongdao Wang, Bei Yu, Hengshuang Zhao, Jiaya Jia
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03705
• Pdf URL: https://arxiv.org/pdf/2503.03705
• Copy Paste: [[2503.03705]] Effective LLM Knowledge Learning via Model Generalization(https://arxiv.org/abs/2503.03705)
• Keywords: large language model
• Abstract: Large language models (LLMs) are trained on enormous documents that contain extensive world knowledge. However, it is still not well-understood how knowledge is acquired via autoregressive pre-training. This lack of understanding greatly hinders effective knowledge learning, especially for continued pretraining on up-to-date information, as this evolving information often lacks diverse repetitions like foundational knowledge. In this paper, we focus on understanding and improving LLM knowledge learning. We found and verified that knowledge learning for LLMs can be deemed as an implicit supervised task hidden in the autoregressive pre-training objective. Our findings suggest that knowledge learning for LLMs would benefit from methods designed to improve generalization ability for supervised tasks. Based on our analysis, we propose the formatting-based data augmentation to grow in-distribution samples, which does not present the risk of altering the facts embedded in documents as text paraphrasing. We also introduce sharpness-aware minimization as an effective optimization algorithm to better improve generalization. Moreover, our analysis and method can be readily extended to instruction tuning. Extensive experiment results validate our findings and demonstrate our methods' effectiveness in both continued pre-training and instruction tuning. This paper offers new perspectives and insights to interpret and design effective strategies for LLM knowledge learning.

Title: Rethinking Video Tokenization: A Conditioned Diffusion-based Approach

• Authors: Nianzu Yang, Pandeng Li, Liming Zhao, Yang Li, Chen-Wei Xie, Yehui Tang, Xudong Lu, Zhihang Liu, Yun Zheng, Yu Liu, Junchi Yan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03708
• Pdf URL: https://arxiv.org/pdf/2503.03708
• Copy Paste: [[2503.03708]] Rethinking Video Tokenization: A Conditioned Diffusion-based Approach(https://arxiv.org/abs/2503.03708)
• Keywords: diffusion, generative
• Abstract: Video tokenizers, which transform videos into compact latent representations, are key to video generation. Existing video tokenizers are based on the VAE architecture and follow a paradigm where an encoder compresses videos into compact latents, and a deterministic decoder reconstructs the original videos from these latents. In this paper, we propose a novel \underline{\textbf{C}}onditioned \underline{\textbf{D}}iffusion-based video \underline{\textbf{T}}okenizer entitled \textbf{\ourmethod}, which departs from previous methods by replacing the deterministic decoder with a 3D causal diffusion model. The reverse diffusion generative process of the decoder is conditioned on the latent representations derived via the encoder. With a feature caching and sampling acceleration, the framework efficiently reconstructs high-fidelity videos of arbitrary lengths. Results show that {\ourmethod} achieves state-of-the-art performance in video reconstruction tasks using just a single-step sampling. Even a smaller version of {\ourmethod} still achieves reconstruction results on par with the top two baselines. Furthermore, the latent video generation model trained using {\ourmethod} also shows superior performance.

Title: Improving LLM Safety Alignment with Dual-Objective Optimization

• Authors: Xuandong Zhao, Will Cai, Tianneng Shi, David Huang, Licong Lin, Song Mei, Dawn Song
• Subjects: cs.CL, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03710
• Pdf URL: https://arxiv.org/pdf/2503.03710
• Copy Paste: [[2503.03710]] Improving LLM Safety Alignment with Dual-Objective Optimization(https://arxiv.org/abs/2503.03710)
• Keywords: attack, robust, large language model
• Abstract: Existing training-time safety alignment techniques for large language models (LLMs) remain vulnerable to jailbreak attacks. Direct preference optimization (DPO), a widely deployed alignment method, exhibits limitations in both experimental and theoretical contexts as its loss function proves suboptimal for refusal learning. Through gradient-based analysis, we identify these shortcomings and propose an improved safety alignment that disentangles DPO objectives into two components: (1) robust refusal training, which encourages refusal even when partial unsafe generations are produced, and (2) targeted unlearning of harmful knowledge. This approach significantly increases LLM robustness against a wide range of jailbreak attacks, including prefilling, suffix, and multi-turn attacks across both in-distribution and out-of-distribution scenarios. Furthermore, we introduce a method to emphasize critical refusal tokens by incorporating a reward-based token-level weighting mechanism for refusal learning, which further improves the robustness against adversarial exploits. Our research also suggests that robustness to jailbreak attacks is correlated with token distribution shifts in the training process and internal representations of refusal and harmful tokens, offering valuable directions for future research in LLM safety alignment. The code is available at this https URL

Title: Handling Uncertainty in Health Data using Generative Algorithms

• Authors: Mahdi Arab Loodaricheh, Neh Majmudar, Anita Raja, Ansaf Salleb-Aouissi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03715
• Pdf URL: https://arxiv.org/pdf/2503.03715
• Copy Paste: [[2503.03715]] Handling Uncertainty in Health Data using Generative Algorithms(https://arxiv.org/abs/2503.03715)
• Keywords: robust, generative
• Abstract: Understanding and managing uncertainty is crucial in machine learning, especially in high-stakes domains like healthcare, where class imbalance can impact predictions. This paper introduces RIGA, a novel pipeline that mitigates class imbalance using generative AI. By converting tabular healthcare data into images, RIGA leverages models like cGAN, VQVAE, and VQGAN to generate balanced samples, improving classification performance. These representations are processed by CNNs and later transformed back into tabular format for seamless integration. This approach enhances traditional classifiers like XGBoost, improves Bayesian structure learning, and strengthens ML model robustness by generating realistic synthetic data for underrepresented classes.

Title: Towards Understanding Distilled Reasoning Models: A Representational Approach

• Authors: David D. Baek, Max Tegmark
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03730
• Pdf URL: https://arxiv.org/pdf/2503.03730
• Copy Paste: [[2503.03730]] Towards Understanding Distilled Reasoning Models: A Representational Approach(https://arxiv.org/abs/2503.03730)
• Keywords: large language model
• Abstract: In this paper, we investigate how model distillation impacts the development of reasoning features in large language models (LLMs). To explore this, we train a crosscoder on Qwen-series models and their fine-tuned variants. Our results suggest that the crosscoder learns features corresponding to various types of reasoning, including self-reflection and computation verification. Moreover, we observe that distilled models contain unique reasoning feature directions, which could be used to steer the model into over-thinking or incisive-thinking mode. In particular, we perform analysis on four specific reasoning categories: (a) self-reflection, (b) deductive reasoning, (c) alternative reasoning, and (d) contrastive reasoning. Finally, we examine the changes in feature geometry resulting from the distillation process and find indications that larger distilled models may develop more structured representations, which correlate with enhanced distillation performance. By providing insights into how distillation modifies the model, our study contributes to enhancing the transparency and reliability of AI systems.

Title: Process-based Self-Rewarding Language Models

• Authors: Shimao Zhang, Xiao Liu, Xin Zhang, Junxiao Liu, Zheheng Luo, Shujian Huang, Yeyun Gong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03746
• Pdf URL: https://arxiv.org/pdf/2503.03746
• Copy Paste: [[2503.03746]] Process-based Self-Rewarding Language Models(https://arxiv.org/abs/2503.03746)
• Keywords: large language model
• Abstract: Large Language Models have demonstrated outstanding performance across various downstream tasks and have been widely applied in multiple scenarios. Human-annotated preference data is used for training to further improve LLMs' performance, which is constrained by the upper limit of human performance. Therefore, Self-Rewarding method has been proposed, where LLMs generate training data by rewarding their own outputs. However, the existing self-rewarding paradigm is not effective in mathematical reasoning scenarios and may even lead to a decline in performance. In this work, we propose the Process-based Self-Rewarding pipeline for language models, which introduces long-thought reasoning, step-wise LLM-as-a-Judge, and step-wise preference optimization within the self-rewarding paradigm. Our new paradigm successfully enhances the performance of LLMs on multiple mathematical reasoning benchmarks through iterative Process-based Self-Rewarding, demonstrating the immense potential of self-rewarding to achieve LLM reasoning that may surpass human capabilities.

Title: PacketCLIP: Multi-Modal Embedding of Network Traffic and Language for Cybersecurity Reasoning

• Authors: Ryozo Masukawa, Sanggeon Yun, Sungheon Jeong, Wenjun Huang, Yang Ni, Ian Bryant, Nathaniel D. Bastian, Mohsen Imani
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03747
• Pdf URL: https://arxiv.org/pdf/2503.03747
• Copy Paste: [[2503.03747]] PacketCLIP: Multi-Modal Embedding of Network Traffic and Language for Cybersecurity Reasoning(https://arxiv.org/abs/2503.03747)
• Keywords: security, robust, interpretability
• Abstract: Traffic classification is vital for cybersecurity, yet encrypted traffic poses significant challenges. We present PacketCLIP, a multi-modal framework combining packet data with natural language semantics through contrastive pretraining and hierarchical Graph Neural Network (GNN) reasoning. PacketCLIP integrates semantic reasoning with efficient classification, enabling robust detection of anomalies in encrypted network flows. By aligning textual descriptions with packet behaviors, it offers enhanced interpretability, scalability, and practical applicability across diverse security scenarios. PacketCLIP achieves a 95% mean AUC, outperforms baselines by 11.6%, and reduces model size by 92%, making it ideal for real-time anomaly detection. By bridging advanced machine learning techniques and practical cybersecurity needs, PacketCLIP provides a foundation for scalable, efficient, and interpretable solutions to tackle encrypted traffic classification and network intrusion detection challenges in resource-constrained environments.

Title: The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems

• Authors: Richard Ren, Arunim Agarwal, Mantas Mazeika, Cristina Menghini, Robert Vacareanu, Brad Kenstler, Mick Yang, Isabelle Barrass, Alice Gatti, Xuwang Yin, Eduardo Trevino, Matias Geralnik, Adam Khoja, Dean Lee, Summer Yue, Dan Hendrycks
• Subjects: cs.LG, cs.AI, cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.03750
• Pdf URL: https://arxiv.org/pdf/2503.03750
• Copy Paste: [[2503.03750]] The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems(https://arxiv.org/abs/2503.03750)
• Keywords: robust, large language model
• Abstract: As large language models (LLMs) become more capable and agentic, the requirement for trust in their outputs grows significantly, yet at the same time concerns have been mounting that models may learn to lie in pursuit of their goals. To address these concerns, a body of work has emerged around the notion of "honesty" in LLMs, along with interventions aimed at mitigating deceptive behaviors. However, evaluations of honesty are currently highly limited, with no benchmark combining large scale and applicability to all models. Moreover, many benchmarks claiming to measure honesty in fact simply measure accuracy--the correctness of a model's beliefs--in disguise. In this work, we introduce a large-scale human-collected dataset for measuring honesty directly, allowing us to disentangle accuracy from honesty for the first time. Across a diverse set of LLMs, we find that while larger models obtain higher accuracy on our benchmark, they do not become more honest. Surprisingly, while most frontier LLMs obtain high scores on truthfulness benchmarks, we find a substantial propensity in frontier LLMs to lie when pressured to do so, resulting in low honesty scores on our benchmark. We find that simple methods, such as representation engineering interventions, can improve honesty. These results underscore the growing need for robust evaluations and effective interventions to ensure LLMs remain trustworthy.

Title: GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control

• Authors: Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, Jun Gao
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2503.03751
• Pdf URL: https://arxiv.org/pdf/2503.03751
• Copy Paste: [[2503.03751]] GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control(https://arxiv.org/abs/2503.03751)
• Keywords: generative
• Abstract: We present GEN3C, a generative video model with precise Camera Control and temporal 3D Consistency. Prior video models already generate realistic videos, but they tend to leverage little 3D information, leading to inconsistencies, such as objects popping in and out of existence. Camera control, if implemented at all, is imprecise, because camera parameters are mere inputs to the neural network which must then infer how the video depends on the camera. In contrast, GEN3C is guided by a 3D cache: point clouds obtained by predicting the pixel-wise depth of seed images or previously generated frames. When generating the next frames, GEN3C is conditioned on the 2D renderings of the 3D cache with the new camera trajectory provided by the user. Crucially, this means that GEN3C neither has to remember what it previously generated nor does it have to infer the image structure from the camera pose. The model, instead, can focus all its generative power on previously unobserved regions, as well as advancing the scene state to the next frame. Our results demonstrate more precise camera control than prior work, as well as state-of-the-art results in sparse-view novel view synthesis, even in challenging settings such as driving scenes and monocular dynamic video. Results are best viewed in videos. Check out our webpage! this https URL