2025-03-07

Title: Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation

• Authors: Hiroshi Takahashi, Tomoharu Iwata, Atsutoshi Kumagai, Yuuki Yamanaka, Tomoya Yamashita
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.03789
• Pdf URL: https://arxiv.org/pdf/2503.03789
• Copy Paste: [[2503.03789]] Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation(https://arxiv.org/abs/2503.03789)
• Keywords: diffusion, generative
• Abstract: Diffusion models are powerful generative models but often generate sensitive data that are unwanted by users, mainly because the unlabeled training data frequently contain such sensitive data. Since labeling all sensitive data in the large-scale unlabeled training data is impractical, we address this problem by using a small amount of labeled sensitive data. In this paper, we propose positive-unlabeled diffusion models, which prevent the generation of sensitive data using unlabeled and sensitive data. Our approach can approximate the evidence lower bound (ELBO) for normal (negative) data using only unlabeled and sensitive (positive) data. Therefore, even without labeled normal data, we can maximize the ELBO for normal data and minimize it for labeled sensitive data, ensuring the generation of only normal data. Through experiments across various datasets and settings, we demonstrated that our approach can prevent the generation of sensitive images without compromising image quality.

Title: DeepGrav: Anomalous Gravitational-Wave Detection Through Deep Latent Features

• Authors: Jianqi Yan (1), Alex P. Leung (1), Zhiyuan Pei (2), David C. Y. Hui (3), Sangin Kim (3) ((1) The University of Hong Kong, (2) Macau University of Science and Technology, (3) Chungnam National University)
• Subjects: cs.LG, astro-ph.HE, gr-qc
• Abstract URL: https://arxiv.org/abs/2503.03799
• Pdf URL: https://arxiv.org/pdf/2503.03799
• Copy Paste: [[2503.03799]] DeepGrav: Anomalous Gravitational-Wave Detection Through Deep Latent Features(https://arxiv.org/abs/2503.03799)
• Keywords: robust
• Abstract: This work introduces a novel deep learning-based approach for gravitational wave anomaly detection, aiming to overcome the limitations of traditional matched filtering techniques in identifying unknown waveform gravitational wave signals. We introduce a modified convolutional neural network architecture inspired by ResNet that leverages residual blocks to extract high-dimensional features, effectively capturing subtle differences between background noise and gravitational wave signals. This network architecture learns a high-dimensional projection while preserving discrepancies with the original input, facilitating precise identification of gravitational wave signals. In our experiments, we implement an innovative data augmentation strategy that generates new data by computing the arithmetic mean of multiple signal samples while retaining the key features of the original signals. In the NSF HDR A3D3: Detecting Anomalous Gravitational Wave Signals competition, it is honorable for us (group name: easonyan123) to get to the first place at the end with our model achieving a true negative rate (TNR) of 0.9708 during development/validation phase and 0.9832 on an unseen challenge dataset during final/testing phase, the highest among all competitors. These results demonstrate that our method not only achieves excellent generalization performance but also maintains robust adaptability in addressing the complex uncertainties inherent in gravitational wave anomaly detection.

Title: RiskAgent: Autonomous Medical AI Copilot for Generalist Risk Prediction

• Authors: Fenglin Liu, Jinge Wu, Hongjian Zhou, Xiao Gu, Soheila Molaei, Anshul Thakur, Lei Clifton, Honghan Wu, David A. Clifton
• Subjects: cs.LG, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.03802
• Pdf URL: https://arxiv.org/pdf/2503.03802
• Copy Paste: [[2503.03802]] RiskAgent: Autonomous Medical AI Copilot for Generalist Risk Prediction(https://arxiv.org/abs/2503.03802)
• Keywords: large language model
• Abstract: The application of Large Language Models (LLMs) to various clinical applications has attracted growing research attention. However, real-world clinical decision-making differs significantly from the standardized, exam-style scenarios commonly used in current efforts. In this paper, we present the RiskAgent system to perform a broad range of medical risk predictions, covering over 387 risk scenarios across diverse complex diseases, e.g., cardiovascular disease and cancer. RiskAgent is designed to collaborate with hundreds of clinical decision tools, i.e., risk calculators and scoring systems that are supported by evidence-based medicine. To evaluate our method, we have built the first benchmark MedRisk specialized for risk prediction, including 12,352 questions spanning 154 diseases, 86 symptoms, 50 specialties, and 24 organ systems. The results show that our RiskAgent, with 8 billion model parameters, achieves 76.33% accuracy, outperforming the most recent commercial LLMs, o1, o3-mini, and GPT-4.5, and doubling the 38.39% accuracy of GPT-4o. On rare diseases, e.g., Idiopathic Pulmonary Fibrosis (IPF), RiskAgent outperforms o1 and GPT-4.5 by 27.27% and 45.46% accuracy, respectively. Finally, we further conduct a generalization evaluation on an external evidence-based diagnosis benchmark and show that our RiskAgent achieves the best results. These encouraging results demonstrate the great potential of our solution for diverse diagnosis domains. To improve the adaptability of our model in different scenarios, we have built and open-sourced a family of models ranging from 1 billion to 70 billion parameters. Our code, data, and models are all available at this https URL.

Title: EgoLife: Towards Egocentric Life Assistant

• Authors: Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin, Marco Cominelli, Zhongang Cai, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Widmer, Francesco Gringoli, Lei Yang, Bo Li, Ziwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03803
• Pdf URL: https://arxiv.org/pdf/2503.03803
• Copy Paste: [[2503.03803]] EgoLife: Towards Egocentric Life Assistant(https://arxiv.org/abs/2503.03803)
• Keywords: robust
• Abstract: We introduce EgoLife, a project to develop an egocentric life assistant that accompanies and enhances personal efficiency through AI-powered wearable glasses. To lay the foundation for this assistant, we conducted a comprehensive data collection study where six participants lived together for one week, continuously recording their daily activities - including discussions, shopping, cooking, socializing, and entertainment - using AI glasses for multimodal egocentric video capture, along with synchronized third-person-view video references. This effort resulted in the EgoLife Dataset, a comprehensive 300-hour egocentric, interpersonal, multiview, and multimodal daily life dataset with intensive annotation. Leveraging this dataset, we introduce EgoLifeQA, a suite of long-context, life-oriented question-answering tasks designed to provide meaningful assistance in daily life by addressing practical questions such as recalling past relevant events, monitoring health habits, and offering personalized recommendations. To address the key technical challenges of (1) developing robust visual-audio models for egocentric data, (2) enabling identity recognition, and (3) facilitating long-context question answering over extensive temporal information, we introduce EgoButler, an integrated system comprising EgoGPT and EgoRAG. EgoGPT is an omni-modal model trained on egocentric datasets, achieving state-of-the-art performance on egocentric video understanding. EgoRAG is a retrieval-based component that supports answering ultra-long-context questions. Our experimental studies verify their working mechanisms and reveal critical factors and bottlenecks, guiding future improvements. By releasing our datasets, models, and benchmarks, we aim to stimulate further research in egocentric AI assistants.

Title: Decoupling the components of geometric understanding in Vision Language Models

• Authors: Eliza Kosoy, Annya Dahmani, Andrew K. Lampinen, Iulia M. Comsa, Soojin Jeong, Ishita Dasgupta, Kelsey Allen
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03840
• Pdf URL: https://arxiv.org/pdf/2503.03840
• Copy Paste: [[2503.03840]] Decoupling the components of geometric understanding in Vision Language Models(https://arxiv.org/abs/2503.03840)
• Keywords: robust
• Abstract: Understanding geometry relies heavily on vision. In this work, we evaluate whether state-of-the-art vision language models (VLMs) can understand simple geometric concepts. We use a paradigm from cognitive science that isolates visual understanding of simple geometry from the many other capabilities it is often conflated with such as reasoning and world knowledge. We compare model performance with human adults from the USA, as well as with prior research on human adults without formal education from an Amazonian indigenous group. We find that VLMs consistently underperform both groups of human adults, although they succeed with some concepts more than others. We also find that VLM geometric understanding is more brittle than human understanding, and is not robust when tasks require mental rotation. This work highlights interesting differences in the origin of geometric understanding in humans and machines -- e.g. from printed materials used in formal education vs. interactions with the physical world or a combination of the two -- and a small step toward understanding these differences.

Title: Task-Agnostic Attacks Against Vision Foundation Models

• Authors: Brian Pulfer, Yury Belousov, Vitaliy Kinakh, Teddy Furon, Slava Voloshynovskiy
• Subjects: cs.CV, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03842
• Pdf URL: https://arxiv.org/pdf/2503.03842
• Copy Paste: [[2503.03842]] Task-Agnostic Attacks Against Vision Foundation Models(https://arxiv.org/abs/2503.03842)
• Keywords: security, attack, segmentation
• Abstract: The study of security in machine learning mainly focuses on downstream task-specific attacks, where the adversarial example is obtained by optimizing a loss function specific to the downstream task. At the same time, it has become standard practice for machine learning practitioners to adopt publicly available pre-trained vision foundation models, effectively sharing a common backbone architecture across a multitude of applications such as classification, segmentation, depth estimation, retrieval, question-answering and more. The study of attacks on such foundation models and their impact to multiple downstream tasks remains vastly unexplored. This work proposes a general framework that forges task-agnostic adversarial examples by maximally disrupting the feature representation obtained with foundation models. We extensively evaluate the security of the feature representations obtained by popular vision foundation models by measuring the impact of this attack on multiple downstream tasks and its transferability between models.

Title: LEWIS (LayEr WIse Sparsity) -- A Training Free Guided Model Merging Approach

• Authors: Hetarth Chopra, Vidhi Rambhia, Vikram Adve
• Subjects: cs.LG, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.03874
• Pdf URL: https://arxiv.org/pdf/2503.03874
• Copy Paste: [[2503.03874]] LEWIS (LayEr WIse Sparsity) -- A Training Free Guided Model Merging Approach(https://arxiv.org/abs/2503.03874)
• Keywords: large language model
• Abstract: As specialized large language models (LLMs) become increasingly prevalent, model merging methods are being used to combine them to create a single multi-task model without requiring any additional data or training. However, these approaches fall short when the objective of merging is to increase the downstream model's performance on a particular task-specific benchmark. In this work, we propose LEWIS (Layer Wise Sparsity), a guided model-merging framework that uses activation-based layer importance to dynamically adjust layer-wise task-vector sparsity required for the merge process. LEWIS uses a calibration dataset to prioritize critical layers during the task-vector pruning process required for model merging. This approach guides existing merging methods by preserving essential layer-wise task-specific knowledge while ensuring the merged model performs the best at benchmarks resembling the calibration dataset. Our experiments demonstrate the effectiveness of LEWIS with performance improvements of code instruction-following and math-solving models created through model merging up to 4 percent and 11.3 percent, respectively, outperforming unguided data-less model merging approaches that use uniform-sparsity.

Title: CRAFT: Characterizing and Root-Causing Fault Injection Threats at Pre-Silicon

• Authors: Arsalan Ali Malik, Harshvadan Mihir, Aydin Aysu
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2503.03877
• Pdf URL: https://arxiv.org/pdf/2503.03877
• Copy Paste: [[2503.03877]] CRAFT: Characterizing and Root-Causing Fault Injection Threats at Pre-Silicon(https://arxiv.org/abs/2503.03877)
• Keywords: secure, security, attack
• Abstract: Fault injection attacks represent a class of threats that can compromise embedded systems across multiple layers of abstraction, such as system software, instruction set architecture (ISA), microarchitecture, and physical implementation. Early detection of these vulnerabilities and understanding their root causes, along with their propagation from the physical layer to the system software, is critical to secure the cyberinfrastructure. This work presents a comprehensive methodology for conducting controlled fault injection attacks at the pre-silicon level and an analysis of the underlying system for root-causing behavior. As the driving application, we use the clock glitch attacks in AI/ML applications for critical misclassification. Our study aims to characterize and diagnose the impact of faults within the RISC-V instruction set and pipeline stages, while tracing fault propagation from the circuit level to the AI/ML application software. This analysis resulted in discovering two new vulnerabilities through controlled clock glitch parameters. First, we reveal a novel method for causing instruction skips, thereby preventing the loading of critical values from memory. This can cause disruption and affect program continuity and correctness. Second, we demonstrate an attack that converts legal instructions into illegal ones, thereby diverting control flow in a manner exploitable by attackers. Our work underscores the complexity of fault injection attack exploits and emphasizes the importance of preemptive security analysis.

Title: A Quantum Good Authentication Protocol

• Authors: Shuangbao Wang
• Subjects: cs.CR, quant-ph
• Abstract URL: https://arxiv.org/abs/2503.03884
• Pdf URL: https://arxiv.org/pdf/2503.03884
• Copy Paste: [[2503.03884]] A Quantum Good Authentication Protocol(https://arxiv.org/abs/2503.03884)
• Keywords: secure, security, privacy, protect, attack
• Abstract: This article presents a novel network protocol that incorporates a quantum photonic channel for symmetric key distribution, a Dilithium signature to replace factor-based public key cryptography for enhanced authentication, security, and privacy. The protocol uses strong hash functions to hash original messages and verify heightened data integrity at the destination. This Quantum Good Authentication Protocol (QGP) provides high-level security provided by the theory of quantum mechanics. QGP also has the advantage of quantum-resistant data protection that prevents current digital computer and future quantum computer attacks. QGP transforms the Transmission Control Protocol/Internet Protocol (TCP/IP) by adding a quantum layer at the bottom of Open Systems Interconnection (OSI) model (layer 0) and modifying the top layer (layer 7) with Dilithium signatures, thus improving the security of the original OSI model. In addition, QGP incorporates strong encryption, hardware-based quantum channels, post-quantum signatures, and secure hash algorithms over a platform of decryptors, switches, routers, and network controllers to form a testbed of the next-generation, secure quantum internet. The experiments presented here show that QGP provides secure authentication and improved security and privacy and can be adopted as a new protocol for the next-generation quantum Internet.

Title: AI for Scaling Legal Reform: Mapping and Redacting Racial Covenants in Santa Clara County

• Authors: Faiz Surani, Mirac Suzgun, Vyoma Raman, Christopher D. Manning, Peter Henderson, Daniel E. Ho
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03888
• Pdf URL: https://arxiv.org/pdf/2503.03888
• Copy Paste: [[2503.03888]] AI for Scaling Legal Reform: Mapping and Redacting Racial Covenants in Santa Clara County(https://arxiv.org/abs/2503.03888)
• Keywords: large language model
• Abstract: Many jurisdictions have moved to identify and strike these provisions, including California, which mandated in 2021 that all counties implement such a process. Yet the scale can be overwhelming, with Santa Clara County (SCC) alone having over 24 million property deed documents, making purely manual review infeasible. We present a novel approach to addressing this pressing issue, developed through a partnership with the SCC Clerk-Recorder's Office. First, we leverage an open large language model, fine-tuned to detect racial covenants with high precision and recall. We estimate that this system reduces manual efforts by 86,500 person hours and costs less than 2% of the cost for a comparable off-the-shelf closed model. Second, we illustrate the County's integration of this model into responsible operational practice, including legal review and the creation of a historical registry, and release our model to assist the hundreds of jurisdictions engaged in similar efforts. Finally, our results reveal distinct periods of utilization of racial covenants, sharp geographic clustering, and the disproportionate role of a small number of developers in maintaining housing discrimination. We estimate that by 1950, one in four properties across the County were subject to racial covenants.

Title: The Signed Two-Space Proximity Model for Learning Representations in Protein-Protein Interaction Networks

• Authors: Nikolaos Nakis, Chrysoula Kosma, Anastasia Brativnyk, Michail Chatzianastasis, Iakovos Evdaimon, Michalis Vazirgiannis
• Subjects: cs.LG, q-bio.MN
• Abstract URL: https://arxiv.org/abs/2503.03904
• Pdf URL: https://arxiv.org/pdf/2503.03904
• Copy Paste: [[2503.03904]] The Signed Two-Space Proximity Model for Learning Representations in Protein-Protein Interaction Networks(https://arxiv.org/abs/2503.03904)
• Keywords: robust
• Abstract: Accurately predicting complex protein-protein interactions (PPIs) is crucial for decoding biological processes, from cellular functioning to disease mechanisms. However, experimental methods for determining PPIs are computationally expensive. Thus, attention has been recently drawn to machine learning approaches. Furthermore, insufficient effort has been made toward analyzing signed PPI networks, which capture both activating (positive) and inhibitory (negative) interactions. To accurately represent biological relationships, we present the Signed Two-Space Proximity Model (S2-SPM) for signed PPI networks, which explicitly incorporates both types of interactions, reflecting the complex regulatory mechanisms within biological systems. This is achieved by leveraging two independent latent spaces to differentiate between positive and negative interactions while representing protein similarity through proximity in these spaces. Our approach also enables the identification of archetypes representing extreme protein profiles. S2-SPM's superior performance in predicting the presence and sign of interactions in SPPI networks is demonstrated in link prediction tasks against relevant baseline methods. Additionally, the biological prevalence of the identified archetypes is confirmed by an enrichment analysis of Gene Ontology (GO) terms, which reveals that distinct biological tasks are associated with archetypal groups formed by both interactions. This study is also validated regarding statistical significance and sensitivity analysis, providing insights into the functional roles of different interaction types. Finally, the robustness and consistency of the extracted archetype structures are confirmed using the Bayesian Normalized Mutual Information (BNMI) metric, proving the model's reliability in capturing meaningful SPPI patterns.

Title: Neural Descriptors: Self-Supervised Learning of Robust Local Surface Descriptors Using Polynomial Patches

• Authors: Gal Yona, Roy Velich, Ron Kimmel, Ehud Rivlin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03907
• Pdf URL: https://arxiv.org/pdf/2503.03907
• Copy Paste: [[2503.03907]] Neural Descriptors: Self-Supervised Learning of Robust Local Surface Descriptors Using Polynomial Patches(https://arxiv.org/abs/2503.03907)
• Keywords: robust
• Abstract: Classical shape descriptors such as Heat Kernel Signature (HKS), Wave Kernel Signature (WKS), and Signature of Histograms of OrienTations (SHOT), while widely used in shape analysis, exhibit sensitivity to mesh connectivity, sampling patterns, and topological noise. While differential geometry offers a promising alternative through its theory of differential invariants, which are theoretically guaranteed to be robust shape descriptors, the computation of these invariants on discrete meshes often leads to unstable numerical approximations, limiting their practical utility. We present a self-supervised learning approach for extracting geometric features from 3D surfaces. Our method combines synthetic data generation with a neural architecture designed to learn sampling-invariant features. By integrating our features into existing shape correspondence frameworks, we demonstrate improved performance on standard benchmarks including FAUST, SCAPE, TOPKIDS, and SHREC'16, showing particular robustness to topological noise and partial shapes.

Title: On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness

• Authors: Xiaochuan Gong, Jie Hao, Mingrui Liu
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2503.03908
• Pdf URL: https://arxiv.org/pdf/2503.03908
• Copy Paste: [[2503.03908]] On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness(https://arxiv.org/abs/2503.03908)
• Keywords: transformer
• Abstract: Adam has become one of the most popular optimizers for training modern deep neural networks, such as transformers. However, its applicability is largely restricted to single-level optimization problems. In this paper, we aim to extend vanilla Adam to tackle bilevel optimization problems, which have important applications in machine learning, such as meta-learning. In particular, we study stochastic bilevel optimization problems where the lower-level function is strongly convex and the upper-level objective is nonconvex with potentially unbounded smoothness. This unbounded smooth objective function covers a broad class of neural networks, including transformers, which may exhibit non-Lipschitz gradients. In this work, we introduce AdamBO, a single-loop Adam-type method that achieves $\widetilde{O}(\epsilon^{-4})$ oracle complexity to find $\epsilon$-stationary points, where the oracle calls involve stochastic gradient or Hessian/Jacobian-vector product evaluations. The key to our analysis is a novel randomness decoupling lemma that provides refined control over the lower-level variable. We conduct extensive experiments on various machine learning tasks involving bilevel formulations with recurrent neural networks (RNNs) and transformers, demonstrating the effectiveness of our proposed Adam-type algorithm.

Title: Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA

• Authors: Jie Hao, Yuman Wu, Ali Payani, Myungjin Lee, Mingrui Liu
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03920
• Pdf URL: https://arxiv.org/pdf/2503.03920
• Copy Paste: [[2503.03920]] Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA(https://arxiv.org/abs/2503.03920)
• Keywords: federate
• Abstract: We study the task of personalized federated fine-tuning with heterogeneous data in the context of language models, where clients collaboratively fine-tune a language model (e.g., BERT, GPT) without sharing their local data, achieving personalization simultaneously. While recent efforts have applied parameter-efficient fine-tuning techniques like low-rank adaptation (LoRA) in federated settings, they typically use single or multiple independent low-rank adapters with predefined maximal and minimal ranks, which may not be optimal for diverse data sources over clients. To address this issue, we propose PF2LoRA, a new personalized federated fine-tuning algorithm built on a novel \emph{automatic rank learning approach via two-level LoRA}. Given the pretrained language model whose weight is frozen, our algorithm aims to learn two levels of adaptation simultaneously: the first level aims to learn a common adapter for all clients, while the second level fosters individual client personalization. A key advantage of PF2LoRA is its ability to adaptively determine a suitable rank based on an individual client's data, rather than relying on a predefined rank that is agnostic to data heterogeneity. We present a synthetic example that highlights how PF2LoRA automatically learns the ground-truth rank for each client, tailoring the adaptation to match the properties of their individual data. Notably, this approach introduces minimal additional memory overhead, as the second-level adaptation comprises a small number of parameters compared to the first level. Our experiments on natural language understanding and generation tasks demonstrate that PF2LoRA significantly outperforms existing federated fine-tuning methods.

Title: Tec-Habilidad: Skill Classification for Bridging Education and Employment

• Authors: Sabur Butt, Hector G. Ceballos, Diana P. Madera
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03932
• Pdf URL: https://arxiv.org/pdf/2503.03932
• Copy Paste: [[2503.03932]] Tec-Habilidad: Skill Classification for Bridging Education and Employment(https://arxiv.org/abs/2503.03932)
• Keywords: robust, extraction
• Abstract: Job application and assessment processes have evolved significantly in recent years, largely due to advancements in technology and changes in the way companies operate. Skill extraction and classification remain an important component of the modern hiring process as it provides a more objective way to evaluate candidates and automatically align their skills with the job requirements. However, to effectively evaluate the skills, the skill extraction tools must recognize varied mentions of skills on resumes, including direct mentions, implications, synonyms, acronyms, phrases, and proficiency levels, and differentiate between hard and soft skills. While tools like LLMs (Large Model Models) help extract and categorize skills from job applications, there's a lack of comprehensive datasets for evaluating the effectiveness of these models in accurately identifying and classifying skills in Spanish-language job applications. This gap hinders our ability to assess the reliability and precision of the models, which is crucial for ensuring that the selected candidates truly possess the required skills for the job. In this paper, we develop a Spanish language dataset for skill extraction and classification, provide annotation methodology to distinguish between knowledge, skill, and abilities, and provide deep learning baselines to advance robust solutions for skill classification.

Title: SurgiSAM2: Fine-tuning a foundational model for surgical video anatomy segmentation and detection

• Authors: Devanish N. Kamtam, Joseph B. Shrager, Satya Deepya Malla, Xiaohan Wang, Nicole Lin, Juan J. Cardona, Serena Yeung-Levy, Clarence Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03942
• Pdf URL: https://arxiv.org/pdf/2503.03942
• Copy Paste: [[2503.03942]] SurgiSAM2: Fine-tuning a foundational model for surgical video anatomy segmentation and detection(https://arxiv.org/abs/2503.03942)
• Keywords: segmentation
• Abstract: Background: We evaluate SAM 2 for surgical scene understanding by examining its semantic segmentation capabilities for organs/tissues both in zero-shot scenarios and after fine-tuning. Methods: We utilized five public datasets to evaluate and fine-tune SAM 2 for segmenting anatomical tissues in surgical videos/images. Fine-tuning was applied to the image encoder and mask decoder. We limited training subsets from 50 to 400 samples per class to better model real-world constraints with data acquisition. The impact of dataset size on fine-tuning performance was evaluated with weighted mean Dice coefficient (WMDC), and the results were also compared against previously reported state-of-the-art (SOTA) results. Results: SurgiSAM 2, a fine-tuned SAM 2 model, demonstrated significant improvements in segmentation performance, achieving a 17.9% relative WMDC gain compared to the baseline SAM 2. Increasing prompt points from 1 to 10 and training data scale from 50/class to 400/class enhanced performance; the best WMDC of 0.92 on the validation subset was achieved with 10 prompt points and 400 samples per class. On the test subset, this model outperformed prior SOTA methods in 24/30 (80%) of the classes with a WMDC of 0.91 using 10-point prompts. Notably, SurgiSAM 2 generalized effectively to unseen organ classes, achieving SOTA on 7/9 (77.8%) of them. Conclusion: SAM 2 achieves remarkable zero-shot and fine-tuned performance for surgical scene segmentation, surpassing prior SOTA models across several organ classes of diverse datasets. This suggests immense potential for enabling automated/semi-automated annotation pipelines, thereby decreasing the burden of annotations facilitating several surgical applications.

Title: GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoors

• Authors: Yaopei Zeng, Yuanpu Cao, Lu Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.03944
• Pdf URL: https://arxiv.org/pdf/2503.03944
• Copy Paste: [[2503.03944]] GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoors(https://arxiv.org/abs/2503.03944)
• Keywords: protect, robust, diffusion, generative
• Abstract: The growing accessibility of diffusion models has revolutionized image editing but also raised significant concerns about unauthorized modifications, such as misinformation and plagiarism. Existing countermeasures largely rely on adversarial perturbations designed to disrupt diffusion model outputs. However, these approaches are found to be easily neutralized by simple image preprocessing techniques, such as compression and noise addition. To address this limitation, we propose GuardDoor, a novel and robust protection mechanism that fosters collaboration between image owners and model providers. Specifically, the model provider participating in the mechanism fine-tunes the image encoder to embed a protective backdoor, allowing image owners to request the attachment of imperceptible triggers to their images. When unauthorized users attempt to edit these protected images with this diffusion model, the model produces meaningless outputs, reducing the risk of malicious image editing. Our method demonstrates enhanced robustness against image preprocessing operations and is scalable for large-scale deployment. This work underscores the potential of cooperative frameworks between model providers and image owners to safeguard digital content in the era of generative AI.

Title: COARSE: Collaborative Pseudo-Labeling with Coarse Real Labels for Off-Road Semantic Segmentation

• Authors: Aurelio Noca, Xianmei Lei, Jonathan Becktor, Jeffrey Edlund, Anna Sabel, Patrick Spieler, Curtis Padgett, Alexandre Alahi, Deegan Atha
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.03947
• Pdf URL: https://arxiv.org/pdf/2503.03947
• Copy Paste: [[2503.03947]] COARSE: Collaborative Pseudo-Labeling with Coarse Real Labels for Off-Road Semantic Segmentation(https://arxiv.org/abs/2503.03947)
• Keywords: robust, transformer, segmentation
• Abstract: Autonomous off-road navigation faces challenges due to diverse, unstructured environments, requiring robust perception with both geometric and semantic understanding. However, scarce densely labeled semantic data limits generalization across domains. Simulated data helps, but introduces domain adaptation issues. We propose COARSE, a semi-supervised domain adaptation framework for off-road semantic segmentation, leveraging sparse, coarse in-domain labels and densely labeled out-of-domain data. Using pretrained vision transformers, we bridge domain gaps with complementary pixel-level and patch-level decoders, enhanced by a collaborative pseudo-labeling strategy on unlabeled data. Evaluations on RUGD and Rellis-3D datasets show significant improvements of 9.7\% and 8.4\% respectively, versus only using coarse data. Tests on real-world off-road vehicle data in a multi-biome setting further demonstrate COARSE's applicability.

Title: Performance Comparison of Large Language Models on Advanced Calculus Problems

• Authors: In Hak Moon
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03960
• Pdf URL: https://arxiv.org/pdf/2503.03960
• Copy Paste: [[2503.03960]] Performance Comparison of Large Language Models on Advanced Calculus Problems(https://arxiv.org/abs/2503.03960)
• Keywords: robust, large language model
• Abstract: This paper presents an in-depth analysis of the performance of seven different Large Language Models (LLMs) in solving a diverse set of math advanced calculus problems. The study aims to evaluate these models' accuracy, reliability, and problem-solving capabilities, including ChatGPT 4o, Gemini Advanced with 1.5 Pro, Copilot Pro, Claude 3.5 Sonnet, Meta AI, Mistral AI, and Perplexity. The assessment was conducted through a series of thirty-two test problems, encompassing a total of 320 points. The problems covered various topics, from vector calculations and geometric interpretations to integral evaluations and optimization tasks. The results highlight significant trends and patterns in the models' performance, revealing both their strengths and weaknesses - for instance, models like ChatGPT 4o and Mistral AI demonstrated consistent accuracy across various problem types, indicating their robustness and reliability in mathematical problem-solving, while models such as Gemini Advanced with 1.5 Pro and Meta AI exhibited specific weaknesses, particularly in complex problems involving integrals and optimization, suggesting areas for targeted improvements. The study also underscores the importance of re-prompting in achieving accurate solutions, as seen in several instances where models initially provided incorrect answers but corrected them upon re-prompting. Overall, this research provides valuable insights into the current capabilities and limitations of LLMs in the domain of math calculus, with the detailed analysis of each model's performance on specific problems offering a comprehensive understanding of their strengths and areas for improvement, contributing to the ongoing development and refinement of LLM technology. The findings are particularly relevant for educators, researchers, and developers seeking to leverage LLMs for educational and practical applications in mathematics.

Title: A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers

• Authors: William Merrill, Ashish Sabharwal
• Subjects: cs.LG, cs.CC
• Abstract URL: https://arxiv.org/abs/2503.03961
• Pdf URL: https://arxiv.org/pdf/2503.03961
• Copy Paste: [[2503.03961]] A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers(https://arxiv.org/abs/2503.03961)
• Keywords: transformer
• Abstract: Recent theoretical results show transformers cannot express sequential reasoning problems over long input lengths, intuitively because their computational depth is bounded. However, prior work treats the depth as a constant, leaving it unclear to what degree bounded depth may suffice for solving problems over short inputs, or how increasing the transformer's depth affects its expressive power. We address these questions by analyzing the expressive power of transformers whose depth can grow minimally with context length $n$. We show even highly uniform transformers with depth $\Theta(\log n)$ can express two important problems: recognizing regular languages, which captures state tracking abilities, and graph connectivity, which underlies multi-step reasoning. Notably, both of these problems cannot be expressed by fixed-depth transformers under standard complexity conjectures, demonstrating the expressivity benefit of growing depth. Moreover, our theory quantitatively predicts how depth must grow with input length to express these problems, showing that depth scaling is more efficient than scaling width or chain-of-thought steps. Empirically, we find our theoretical depth requirements for regular language recognition match the practical depth requirements of transformers remarkably well. Thus, our results clarify precisely how depth affects transformers' reasoning capabilities, providing potential practical insights for designing models that are better at sequential reasoning.

Title: On the Acquisition of Shared Grammatical Representations in Bilingual Language Models

• Authors: Catherine Arnett, Tyler A. Chang, James A. Michaelov, Benjamin K. Bergen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.03962
• Pdf URL: https://arxiv.org/pdf/2503.03962
• Copy Paste: [[2503.03962]] On the Acquisition of Shared Grammatical Representations in Bilingual Language Models(https://arxiv.org/abs/2503.03962)
• Keywords: robust
• Abstract: While crosslingual transfer is crucial to contemporary language models' multilingual capabilities, how it occurs is not well understood. In this paper, we ask what happens to a monolingual language model when it begins to be trained on a second language. Specifically, we train small bilingual models for which we control the amount of data for each language and the order of language exposure. To find evidence of shared multilingual representations, we turn to structural priming, a method used to study grammatical representations in humans. We first replicate previous crosslingual structural priming results and find that after controlling for training data quantity and language exposure, there are asymmetrical effects across language pairs and directions. We argue that this asymmetry may shape hypotheses about human structural priming effects. We also find that structural priming effects are less robust for less similar language pairs, highlighting potential limitations of crosslingual transfer learning and shared representations for typologically diverse languages.

Title: Generative Learning of Densities on Manifolds

• Authors: Dimitris G. Giovanis, Ellis Crabtree, Roger G. Ghanem, Ioannis G. kevrekidis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03963
• Pdf URL: https://arxiv.org/pdf/2503.03963
• Copy Paste: [[2503.03963]] Generative Learning of Densities on Manifolds(https://arxiv.org/abs/2503.03963)
• Keywords: diffusion, generative
• Abstract: A generative modeling framework is proposed that combines diffusion models and manifold learning to efficiently sample data densities on manifolds. The approach utilizes Diffusion Maps to uncover possible low-dimensional underlying (latent) spaces in the high-dimensional data (ambient) space. Two approaches for sampling from the latent data density are described. The first is a score-based diffusion model, which is trained to map a standard normal distribution to the latent data distribution using a neural network. The second one involves solving an Itô stochastic differential equation in the latent space. Additional realizations of the data are generated by lifting the samples back to the ambient space using Double Diffusion Maps, a recently introduced technique typically employed in studying dynamical system reduction; here the focus lies in sampling densities rather than system dynamics. The proposed approaches enable sampling high dimensional data densities restricted to low-dimensional, a priori unknown manifolds. The efficacy of the proposed framework is demonstrated through a benchmark problem and a material with multiscale structure.

Title: All-atom Diffusion Transformers: Unified generative modelling of molecules and materials

• Authors: Chaitanya K. Joshi, Xiang Fu, Yi-Lun Liao, Vahe Gharakhanyan, Benjamin Kurt Miller, Anuroop Sriram, Zachary W. Ulissi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03965
• Pdf URL: https://arxiv.org/pdf/2503.03965
• Copy Paste: [[2503.03965]] All-atom Diffusion Transformers: Unified generative modelling of molecules and materials(https://arxiv.org/abs/2503.03965)
• Keywords: diffusion, transformer, generative
• Abstract: Diffusion models are the standard toolkit for generative modelling of 3D atomic systems. However, for different types of atomic systems - such as molecules and materials - the generative processes are usually highly specific to the target system despite the underlying physics being the same. We introduce the All-atom Diffusion Transformer (ADiT), a unified latent diffusion framework for jointly generating both periodic materials and non-periodic molecular systems using the same model: (1) An autoencoder maps a unified, all-atom representations of molecules and materials to a shared latent embedding space; and (2) A diffusion model is trained to generate new latent embeddings that the autoencoder can decode to sample new molecules or materials. Experiments on QM9 and MP20 datasets demonstrate that jointly trained ADiT generates realistic and valid molecules as well as materials, exceeding state-of-the-art results from molecule and crystal-specific models. ADiT uses standard Transformers for both the autoencoder and diffusion model, resulting in significant speedups during training and inference compared to equivariant diffusion models. Scaling ADiT up to half a billion parameters predictably improves performance, representing a step towards broadly generalizable foundation models for generative chemistry. Open source code: this https URL

Title: Cryptographic Verifiability for Voter Registration Systems

• Authors: Andrés Fábrega, Jack Cable, Michael A. Specter, Sunoo Park
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03974
• Pdf URL: https://arxiv.org/pdf/2503.03974
• Copy Paste: [[2503.03974]] Cryptographic Verifiability for Voter Registration Systems(https://arxiv.org/abs/2503.03974)
• Keywords: secure, privacy
• Abstract: Voter registration systems are a critical - and surprisingly understudied - element of most high-stakes elections. Despite a history of targeting by adversaries, relatively little academic work has been done to increase visibility into how voter registration systems keep voters' data secure, accurate, and up to date. Enhancing transparency and verifiability could help election officials and the public detect and mitigate risks to this essential component of electoral processes worldwide. This work introduces cryptographic verifiability for voter registration systems. Based on consultation with diverse expert stakeholders that support elections systems, we precisely define the requirements for cryptographic verifiability in voter registration and systematize the practical challenges that must be overcome for near-term deployment. We then introduce VRLog, the first system to bring strong verifiability to voter registration. VRLog enables election officials to provide a transparent log that (1) allows voters to verify that their registration data has not been tampered with and (2) allows the public to monitor update patterns and database consistency. We also introduce VRLog$^x$, an enhancement to VRLog that offers cryptographic privacy to voter deduplication between jurisdictions - a common maintenance task currently performed in plaintext or using trusted third parties. Our designs rely on standard, efficient cryptographic primitives, and are backward compatible with existing voter registration systems. Finally, we provide an open-source implementation of VRLog and benchmarks to demonstrate that the system is practical - capable of running on low-cost commodity hardware and scaling to support databases the size of the largest U.S. state voter registration systems.

Title: ReasonGraph: Visualisation of Reasoning Paths

• Authors: Zongqian Li, Ehsan Shareghi, Nigel Collier
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.03979
• Pdf URL: https://arxiv.org/pdf/2503.03979
• Copy Paste: [[2503.03979]] ReasonGraph: Visualisation of Reasoning Paths(https://arxiv.org/abs/2503.03979)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) reasoning processes are challenging to analyze due to their complexity and the lack of organized visualization tools. We present ReasonGraph, a web-based platform for visualizing and analyzing LLM reasoning processes. It supports both sequential and tree-based reasoning methods while integrating with major LLM providers and over fifty state-of-the-art models. ReasonGraph incorporates an intuitive UI with meta reasoning method selection, configurable visualization parameters, and a modular framework that facilitates efficient extension. Our evaluation shows high parsing reliability, efficient processing, and strong usability across various downstream applications. By providing a unified visualization framework, ReasonGraph reduces cognitive load in analyzing complex reasoning paths, improves error detection in logical processes, and enables more effective development of LLM-based applications. The platform is open-source, promoting accessibility and reproducibility in LLM reasoning analysis.

Title: USBSnoop -- Revealing Device Activities via USB Congestions

• Authors: Davis Ranney, Yufei Wang, A. Adam Ding, Yunsi Fei
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.03980
• Pdf URL: https://arxiv.org/pdf/2503.03980
• Copy Paste: [[2503.03980]] USBSnoop -- Revealing Device Activities via USB Congestions(https://arxiv.org/abs/2503.03980)
• Keywords: security, privacy, protect, attack
• Abstract: The USB protocol has become a ubiquitous standard for connecting peripherals to computers, making its security a critical concern. A recent research study demonstrated the potential to exploit weaknesses in well-established protocols, such as PCIe, and created a side-channel for leaking sensitive information by leveraging congestion within shared interfaces. Drawing inspiration from that, this project introduces an innovative approach to USB side-channel attacks via congestion. We evaluated the susceptibility of USB devices and hubs to remote profiling and side-channel attacks, identified potential weaknesses within the USB standard, and highlighted the critical need for heightened security and privacy in USB technology. Our findings discover vulnerabilities within the USB standard, which are difficult to effectively mitigate and underscore the need for enhanced security measures to protect user privacy in an era increasingly dependent on USB-connected devices.

Title: RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models

• Authors: Wenhui Zhu, Xin Li, Xiwen Chen, Peijie Qiu, Vamsi Krishna Vasa, Xuanzhao Dong, Yanxi Chen, Natasha Lepore, Oana Dumitrascu, Yi Su, Yalin Wang
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.03987
• Pdf URL: https://arxiv.org/pdf/2503.03987
• Copy Paste: [[2503.03987]] RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models(https://arxiv.org/abs/2503.03987)
• Keywords: large language model
• Abstract: Recently, Multimodal Large Language Models (MLLMs) have gained significant attention for their remarkable ability to process and analyze non-textual data, such as images, videos, and audio. Notably, several adaptations of general-domain MLLMs to the medical field have been explored, including LLaVA-Med. However, these medical adaptations remain insufficiently advanced in understanding and interpreting retinal images. In contrast, medical experts emphasize the importance of quantitative analyses for disease detection and interpretation. This underscores a gap between general-domain and medical-domain MLLMs: while general-domain MLLMs excel in broad applications, they lack the specialized knowledge necessary for precise diagnostic and interpretative tasks in the medical field. To address these challenges, we introduce \textit{RetinalGPT}, a multimodal conversational assistant for clinically preferred quantitative analysis of retinal images. Specifically, we achieve this by compiling a large retinal image dataset, developing a novel data pipeline, and employing customized visual instruction tuning to enhance both retinal analysis and enrich medical knowledge. In particular, RetinalGPT outperforms MLLM in the generic domain by a large margin in the diagnosis of retinal diseases in 8 benchmark retinal datasets. Beyond disease diagnosis, RetinalGPT features quantitative analyses and lesion localization, representing a pioneering step in leveraging LLMs for an interpretable and end-to-end clinical research framework. The code is available at this https URL

Title: Subgraph Federated Learning for Local Generalization

• Authors: Sungwon Kim, Yoonho Lee, Yunhak Oh, Namkyeong Lee, Sukwon Yun, Junseok Lee, Sein Kim, Carl Yang, Chanyoung Park
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.03995
• Pdf URL: https://arxiv.org/pdf/2503.03995
• Copy Paste: [[2503.03995]] Subgraph Federated Learning for Local Generalization(https://arxiv.org/abs/2503.03995)
• Keywords: privacy, federate
• Abstract: Federated Learning (FL) on graphs enables collaborative model training to enhance performance without compromising the privacy of each client. However, existing methods often overlook the mutable nature of graph data, which frequently introduces new nodes and leads to shifts in label distribution. Since they focus solely on performing well on each client's local data, they are prone to overfitting to their local distributions (i.e., local overfitting), which hinders their ability to generalize to unseen data with diverse label distributions. In contrast, our proposed method, FedLoG, effectively tackles this issue by mitigating local overfitting. Our model generates global synthetic data by condensing the reliable information from each class representation and its structural information across clients. Using these synthetic data as a training set, we alleviate the local overfitting problem by adaptively generalizing the absent knowledge within each local dataset. This enhances the generalization capabilities of local models, enabling them to handle unseen data effectively. Our model outperforms baselines in our proposed experimental settings, which are designed to measure generalization power to unseen data in practical scenarios. Our code is available at this https URL

Title: DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation

• Authors: Amin Karimi, Charalambos Poullis
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04006
• Pdf URL: https://arxiv.org/pdf/2503.04006
• Copy Paste: [[2503.04006]] DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation(https://arxiv.org/abs/2503.04006)
• Keywords: robust, large language model, segmentation
• Abstract: Few-shot semantic segmentation (FSS) aims to enable models to segment novel/unseen object classes using only a limited number of labeled examples. However, current FSS methods frequently struggle with generalization due to incomplete and biased feature representations, especially when support images do not capture the full appearance variability of the target class. To improve the FSS pipeline, we propose a novel framework that utilizes large language models (LLMs) to adapt general class semantic information to the query image. Furthermore, the framework employs dense pixel-wise matching to identify similarities between query and support images, resulting in enhanced FSS performance. Inspired by reasoning-based segmentation frameworks, our method, named DSV-LFS, introduces an additional token into the LLM vocabulary, allowing a multimodal LLM to generate a "semantic prompt" from class descriptions. In parallel, a dense matching module identifies visual similarities between the query and support images, generating a "visual prompt". These prompts are then jointly employed to guide the prompt-based decoder for accurate segmentation of the query image. Comprehensive experiments on the benchmark datasets Pascal-$5^{i}$ and COCO-$20^{i}$ demonstrate that our framework achieves state-of-the-art performance-by a significant margin-demonstrating superior generalization to novel classes and robustness across diverse scenarios. The source code is available at \href{this https URL}{this https URL}

Title: Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting

• Authors: Jiyue Jiang, Pengan Chen, Jiuming Wang, Dongchen He, Ziqin Wei, Liang Hong, Licheng Zong, Sheng Wang, Qinze Yu, Zixian Ma, Yanyu Chen, Yimin Fan, Xiangyu Shi, Jiawei Sun, Chuan Wu, Yu Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04013
• Pdf URL: https://arxiv.org/pdf/2503.04013
• Copy Paste: [[2503.04013]] Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting(https://arxiv.org/abs/2503.04013)
• Keywords: robust, extraction, large language model
• Abstract: Large language models (LLMs) have become important tools in solving biological problems, offering improvements in accuracy and adaptability over conventional methods. Several benchmarks have been proposed to evaluate the performance of these LLMs. However, current benchmarks can hardly evaluate the performance of these models across diverse tasks effectively. In this paper, we introduce a comprehensive prompting-based benchmarking framework, termed Bio-benchmark, which includes 30 key bioinformatics tasks covering areas such as proteins, RNA, drugs, electronic health records, and traditional Chinese medicine. Using this benchmark, we evaluate six mainstream LLMs, including GPT-4o and Llama-3.1-70b, etc., using 0-shot and few-shot Chain-of-Thought (CoT) settings without fine-tuning to reveal their intrinsic capabilities. To improve the efficiency of our evaluations, we demonstrate BioFinder, a new tool for extracting answers from LLM responses, which increases extraction accuracy by round 30% compared to existing methods. Our benchmark results show the biological tasks suitable for current LLMs and identify specific areas requiring enhancement. Furthermore, we propose targeted prompt engineering strategies for optimizing LLM performance in these contexts. Based on these findings, we provide recommendations for the development of more robust LLMs tailored for various biological applications. This work offers a comprehensive evaluation framework and robust tools to support the application of LLMs in bioinformatics.

Title: NsBM-GAT: A Non-stationary Block Maximum and Graph Attention Framework for General Traffic Crash Risk Prediction

• Authors: Kequan Chen, Pan Liu, Yuxuan Wang, David Z. W. Wang, Yifan Dai, Zhibin Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04018
• Pdf URL: https://arxiv.org/pdf/2503.04018
• Copy Paste: [[2503.04018]] NsBM-GAT: A Non-stationary Block Maximum and Graph Attention Framework for General Traffic Crash Risk Prediction(https://arxiv.org/abs/2503.04018)
• Keywords: interpretability
• Abstract: Accurate prediction of traffic crash risks for individual vehicles is essential for enhancing vehicle safety. While significant attention has been given to traffic crash risk prediction, existing studies face two main challenges: First, due to the scarcity of individual vehicle data before crashes, most models rely on hypothetical scenarios deemed dangerous by researchers. This raises doubts about their applicability to actual pre-crash conditions. Second, some crash risk prediction frameworks were learned from dashcam videos. Although such videos capture the pre-crash behavior of individual vehicles, they often lack critical information about the movements of surrounding vehicles. However, the interaction between a vehicle and its surrounding vehicles is highly influential in crash occurrences. To overcome these challenges, we propose a novel non-stationary extreme value theory (EVT), where the covariate function is optimized in a nonlinear fashion using a graph attention network. The EVT component incorporates the stochastic nature of crashes through probability distribution, which enhances model interpretability. Notably, the nonlinear covariate function enables the model to capture the interactive behavior between the target vehicle and its multiple surrounding vehicles, facilitating crash risk prediction across different driving tasks. We train and test our model using 100 sets of vehicle trajectory data before real crashes, collected via drones over three years from merging and weaving segments. We demonstrate that our model successfully learns micro-level precursors of crashes and fits a more accurate distribution with the aid of the nonlinear covariate function. Our experiments on the testing dataset show that the proposed model outperforms existing models by providing more accurate predictions for both rear-end and sideswipe crashes simultaneously.

Title: TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models

• Authors: Wanglong Lu, Lingming Su, Jingjing Zheng, Vinícius Veloso de Melo, Farzaneh Shoeleh, John Hawkin, Terrence Tricco, Hanli Zhao, Xianta Jiang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04021
• Pdf URL: https://arxiv.org/pdf/2503.04021
• Copy Paste: [[2503.04021]] TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models(https://arxiv.org/abs/2503.04021)
• Keywords: diffusion
• Abstract: Digital versions of real-world text documents often suffer from issues like environmental corrosion of the original document, low-quality scanning, or human interference. Existing document restoration and inpainting methods typically struggle with generalizing to unseen document styles and handling high-resolution images. To address these challenges, we introduce TextDoctor, a novel unified document image inpainting method. Inspired by human reading behavior, TextDoctor restores fundamental text elements from patches and then applies diffusion models to entire document images instead of training models on specific document types. To handle varying text sizes and avoid out-of-memory issues, common in high-resolution documents, we propose using structure pyramid prediction and patch pyramid diffusion models. These techniques leverage multiscale inputs and pyramid patches to enhance the quality of inpainting both globally and locally. Extensive qualitative and quantitative experiments on seven public datasets validated that TextDoctor outperforms state-of-the-art methods in restoring various types of high-resolution document images.

Title: GaussianGraph: 3D Gaussian-based Scene Graph Generation for Open-world Scene Understanding

• Authors: Xihan Wang, Dianyi Yang, Yu Gao, Yufeng Yue, Yi Yang, Mengyin Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04034
• Pdf URL: https://arxiv.org/pdf/2503.04034
• Copy Paste: [[2503.04034]] GaussianGraph: 3D Gaussian-based Scene Graph Generation for Open-world Scene Understanding(https://arxiv.org/abs/2503.04034)
• Keywords: robust, segmentation
• Abstract: Recent advancements in 3D Gaussian Splatting(3DGS) have significantly improved semantic scene understanding, enabling natural language queries to localize objects within a scene. However, existing methods primarily focus on embedding compressed CLIP features to 3D Gaussians, suffering from low object segmentation accuracy and lack spatial reasoning capabilities. To address these limitations, we propose GaussianGraph, a novel framework that enhances 3DGS-based scene understanding by integrating adaptive semantic clustering and scene graph generation. We introduce a "Control-Follow" clustering strategy, which dynamically adapts to scene scale and feature distribution, avoiding feature compression and significantly improving segmentation accuracy. Additionally, we enrich scene representation by integrating object attributes and spatial relations extracted from 2D foundation models. To address inaccuracies in spatial relationships, we propose 3D correction modules that filter implausible relations through spatial consistency verification, ensuring reliable scene graph construction. Extensive experiments on three datasets demonstrate that GaussianGraph outperforms state-of-the-art methods in both semantic segmentation and object grounding tasks, providing a robust solution for complex scene understanding and interaction.

Title: Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge

• Authors: Xinyue Cui, Johnny Tian-Zheng Wei, Swabha Swayamdipta, Robin Jia
• Subjects: cs.CR, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04036
• Pdf URL: https://arxiv.org/pdf/2503.04036
• Copy Paste: [[2503.04036]] Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge(https://arxiv.org/abs/2503.04036)
• Keywords: robust, watermark
• Abstract: Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership. Previous data watermarking techniques primarily focus on effective memorization after pretraining, while overlooking challenges that arise in other stages of the LLM pipeline, such as the risk of watermark filtering during data preprocessing, or potential forgetting through post-training, or verification difficulties due to API-only access. We propose a novel data watermarking approach that injects coherent and plausible yet fictitious knowledge into training data using generated passages describing a fictitious entity and its associated attributes. Our watermarks are designed to be memorized by the LLM through seamlessly integrating in its training data, making them harder to detect lexically during this http URL demonstrate that our watermarks can be effectively memorized by LLMs, and that increasing our watermarks' density, length, and diversity of attributes strengthens their memorization. We further show that our watermarks remain robust throughout LLM development, maintaining their effectiveness after continual pretraining and supervised finetuning. Finally, we show that our data watermarks can be evaluated even under API-only access via question answering.

Title: Underlying Semantic Diffusion for Effective and Efficient In-Context Learning

• Authors: Zhong Ji, Weilong Cao, Yan Zhang, Yanwei Pang, Jungong Han, Xuelong Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04050
• Pdf URL: https://arxiv.org/pdf/2503.04050
• Copy Paste: [[2503.04050]] Underlying Semantic Diffusion for Effective and Efficient In-Context Learning(https://arxiv.org/abs/2503.04050)
• Keywords: robust, diffusion
• Abstract: Diffusion models has emerged as a powerful framework for tasks like image controllable generation and dense prediction. However, existing models often struggle to capture underlying semantics (e.g., edges, textures, shapes) and effectively utilize in-context learning, limiting their contextual understanding and image generation quality. Additionally, high computational costs and slow inference speeds hinder their real-time applicability. To address these challenges, we propose Underlying Semantic Diffusion (US-Diffusion), an enhanced diffusion model that boosts underlying semantics learning, computational efficiency, and in-context learning capabilities on multi-task scenarios. We introduce Separate & Gather Adapter (SGA), which decouples input conditions for different tasks while sharing the architecture, enabling better in-context learning and generalization across diverse visual domains. We also present a Feedback-Aided Learning (FAL) framework, which leverages feedback signals to guide the model in capturing semantic details and dynamically adapting to task-specific contextual cues. Furthermore, we propose a plug-and-play Efficient Sampling Strategy (ESS) for dense sampling at time steps with high-noise levels, which aims at optimizing training and inference efficiency while maintaining strong in-context learning performance. Experimental results demonstrate that US-Diffusion outperforms the state-of-the-art method, achieving an average reduction of 7.47 in FID on Map2Image tasks and an average reduction of 0.026 in RMSE on Image2Map tasks, while achieving approximately 9.45 times faster inference speed. Our method also demonstrates superior training efficiency and in-context learning capabilities, excelling in new datasets and tasks, highlighting its robustness and adaptability across diverse visual domains.

Title: The Impact Analysis of Delays in Asynchronous Federated Learning with Data Heterogeneity for Edge Intelligence

• Authors: Ziruo Hao, Zhenhua Cui, Tao Yang, Bo Hu, Xiaofeng Wu, Hui Feng
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04052
• Pdf URL: https://arxiv.org/pdf/2503.04052
• Copy Paste: [[2503.04052]] The Impact Analysis of Delays in Asynchronous Federated Learning with Data Heterogeneity for Edge Intelligence(https://arxiv.org/abs/2503.04052)
• Keywords: federate
• Abstract: Federated learning (FL) has provided a new methodology for coordinating a group of clients to train a machine learning model collaboratively, bringing an efficient paradigm in edge intelligence. Despite its promise, FL faces several critical challenges in practical applications involving edge devices, such as data heterogeneity and delays stemming from communication and computation constraints. This paper examines the impact of unknown causes of delay on training performance in an Asynchronous Federated Learning (AFL) system with data heterogeneity. Initially, an asynchronous error definition is proposed, based on which the solely adverse impact of data heterogeneity is theoretically analyzed within the traditional Synchronous Federated Learning (SFL) framework. Furthermore, Asynchronous Updates with Delayed Gradients (AUDG), a conventional AFL scheme, is discussed. Investigation into AUDG reveals that the negative influence of data heterogeneity is correlated with delays, while a shorter average delay from a specific client does not consistently enhance training performance. In order to compensate for the scenarios where AUDG are not adapted, Pseudo-synchronous Updates by Reusing Delayed Gradients (PSURDG) is proposed, and its theoretical convergence is analyzed. In both AUDG and PSURDG, only a random set of clients successfully transmits their updated results to the central server in each iteration. The critical difference between them lies in whether the delayed information is reused. Finally, both schemes are validated and compared through theoretical analysis and simulations, demonstrating more intuitively that discarding outdated information due to time delays is not always the best approach.

Title: Controlled privacy leakage propagation throughout overlapping grouped learning

• Authors: Shahrzad Kiani, Franziska Boenisch, Stark C. Draper
• Subjects: cs.LG, cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.04054
• Pdf URL: https://arxiv.org/pdf/2503.04054
• Copy Paste: [[2503.04054]] Controlled privacy leakage propagation throughout overlapping grouped learning(https://arxiv.org/abs/2503.04054)
• Keywords: privacy, federate
• Abstract: Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially overlapping groups. In this setup where workers can belong and contribute to more than one group at a time, complexities arise in understanding privacy leakage and in adhering to privacy policies. To address the challenges, we propose differential private overlapping grouped learning (DPOGL), a novel method to implement privacy guarantees within overlapping groups. Under the honest-but-curious threat model, we derive novel privacy guarantees between arbitrary pairs of workers. These privacy guarantees describe and quantify two key effects of privacy leakage in DP-OGL: propagation delay, i.e., the fact that information from one group will leak to other groups only with temporal offset through the common workers and information degradation, i.e., the fact that noise addition over model updates limits information leakage between workers. Our experiments show that applying DP-OGL enhances utility while maintaining strong privacy compared to standard FL setups.

Title: EVE: Towards End-to-End Video Subtitle Extraction with Vision-Language Models

• Authors: Haiyang Yu, Jinghui Lu, Yanjie Wang, Yang Li, Han Wang, Can Huang, Bin Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04058
• Pdf URL: https://arxiv.org/pdf/2503.04058
• Copy Paste: [[2503.04058]] EVE: Towards End-to-End Video Subtitle Extraction with Vision-Language Models(https://arxiv.org/abs/2503.04058)
• Keywords: robust, extraction, large language model
• Abstract: The advent of Large Vision-Language Models (LVLMs) has advanced the video-based tasks, such as video captioning and video understanding. Some previous research indicates that taking texts in videos as input can further improve the performance of video understanding. As a type of indispensable information in short videos or movies, subtitles can assist LVLMs to better understand videos. Most existing methods for video subtitle extraction are based on a multi-stage framework, handling each frame independently. They can hardly exploit the temporal information of videos. Although some LVLMs exhibit the robust OCR capability, predicting accurate timestamps for subtitle texts is still challenging. In this paper, we propose an End-to-end Video Subtitle Extraction method, called EVE, which consists of three modules: a vision encoder, an adapter module, and a large language model. To effectively compress the visual tokens from the vision encoder, we propose a novel adapter InterleavedVT to interleave two modalities. It contains a visual compressor and a textual region compressor. The proposed InterleavedVT exploits both the merits of average pooling and Q-Former in token compression. Taking the temporal information of videos into account, we introduce a sliding-window mechanism in the textual region compressor. To benchmark the video subtitle extraction task, we propose a large dataset ViSa including 2.5M videos. Extensive experiments on ViSa demonstrate that the proposed EVE can outperform existing open-sourced tools and LVLMs.

Title: H3O: Hyper-Efficient 3D Occupancy Prediction with Heterogeneous Supervision

• Authors: Yunxiao Shi, Hong Cai, Amin Ansari, Fatih Porikli
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04059
• Pdf URL: https://arxiv.org/pdf/2503.04059
• Copy Paste: [[2503.04059]] H3O: Hyper-Efficient 3D Occupancy Prediction with Heterogeneous Supervision(https://arxiv.org/abs/2503.04059)
• Keywords: segmentation
• Abstract: 3D occupancy prediction has recently emerged as a new paradigm for holistic 3D scene understanding and provides valuable information for downstream planning in autonomous driving. Most existing methods, however, are computationally expensive, requiring costly attention-based 2D-3D transformation and 3D feature processing. In this paper, we present a novel 3D occupancy prediction approach, H3O, which features highly efficient architecture designs that incur a significantly lower computational cost as compared to the current state-of-the-art methods. In addition, to compensate for the ambiguity in ground-truth 3D occupancy labels, we advocate leveraging auxiliary tasks to complement the direct 3D supervision. In particular, we integrate multi-camera depth estimation, semantic segmentation, and surface normal estimation via differentiable volume rendering, supervised by corresponding 2D labels that introduces rich and heterogeneous supervision signals. We conduct extensive experiments on the Occ3D-nuScenes and SemanticKITTI benchmarks that demonstrate the superiority of our proposed H3O.

Title: Uncovering inequalities in new knowledge learning by large language models across different languages

• Authors: Chenglong Wang, Haoyu Tang, Xiyuan Yang, Yueqi Xie, Jina Suh, Sunayana Sitaram, Junming Huang, Yu Xie, Zhaoya Gong, Xing Xie, Fangzhao Wu
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2503.04064
• Pdf URL: https://arxiv.org/pdf/2503.04064
• Copy Paste: [[2503.04064]] Uncovering inequalities in new knowledge learning by large language models across different languages(https://arxiv.org/abs/2503.04064)
• Keywords: robust, large language model
• Abstract: As large language models (LLMs) gradually become integral tools for problem solving in daily life worldwide, understanding linguistic inequality is becoming increasingly important. Existing research has primarily focused on static analyses that assess the disparities in the existing knowledge and capabilities of LLMs across languages. However, LLMs are continuously evolving, acquiring new knowledge to generate up-to-date, domain-specific responses. Investigating linguistic inequalities within this dynamic process is, therefore, also essential. In this paper, we explore inequalities in new knowledge learning by LLMs across different languages and four key dimensions: effectiveness, transferability, prioritization, and robustness. Through extensive experiments under two settings (in-context learning and fine-tuning) using both proprietary and open-source models, we demonstrate that low-resource languages consistently face disadvantages across all four dimensions. By shedding light on these disparities, we aim to raise awareness of linguistic inequalities in LLMs' new knowledge learning, fostering the development of more inclusive and equitable future LLMs.

Title: PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks

• Authors: Feng Ni, Kui Huang, Yao Lu, Wenyu Lv, Guanzhong Wang, Zeyu Chen, Yi Liu
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04065
• Pdf URL: https://arxiv.org/pdf/2503.04065
• Copy Paste: [[2503.04065]] PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks(https://arxiv.org/abs/2503.04065)
• Keywords: large language model
• Abstract: With the rapid advancement of digitalization, various document images are being applied more extensively in production and daily life, and there is an increasingly urgent need for fast and accurate parsing of the content in document images. Therefore, this report presents PP-DocBee, a novel multimodal large language model designed for end-to-end document image understanding. First, we develop a data synthesis strategy tailored to document scenarios in which we build a diverse dataset to improve the model generalization. Then, we apply a few training techniques, including dynamic proportional sampling, data preprocessing, and OCR postprocessing strategies. Extensive evaluations demonstrate the superior performance of PP-DocBee, achieving state-of-the-art results on English document understanding benchmarks and even outperforming existing open source and commercial models in Chinese document understanding. The source code and pre-trained models are publicly available at \href{this https URL}{this https URL}.

Title: FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis

• Authors: Ziqi Ni, Ao Fu, Yi Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04067
• Pdf URL: https://arxiv.org/pdf/2503.04067
• Copy Paste: [[2503.04067]] FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis(https://arxiv.org/abs/2503.04067)
• Keywords: diffusion
• Abstract: Achieving high-fidelity lip-speech synchronization in audio-driven talking portrait synthesis remains challenging. While multi-stage pipelines or diffusion models yield high-quality results, they suffer from high computational costs. Some approaches perform well on specific individuals with low resources, yet still exhibit mismatched lip movements. The aforementioned methods are modeled in the pixel domain. We observed that there are noticeable discrepancies in the frequency domain between the synthesized talking videos and natural videos. Currently, no research on talking portrait synthesis has considered this aspect. To address this, we propose a FREquency-modulated, high-fidelity, and real-time Audio-driven talKing portrait synthesis framework, named FREAK, which models talking portraits from the frequency domain perspective, enhancing the fidelity and naturalness of the synthesized portraits. FREAK introduces two novel frequency-based modules: 1) the Visual Encoding Frequency Modulator (VEFM) to couple multi-scale visual features in the frequency domain, better preserving visual frequency information and reducing the gap in the frequency spectrum between synthesized and natural frames. and 2) the Audio Visual Frequency Modulator (AVFM) to help the model learn the talking pattern in the frequency domain and improve audio-visual synchronization. Additionally, we optimize the model in both pixel domain and frequency domain jointly. Furthermore, FREAK supports seamless switching between one-shot and video dubbing settings, offering enhanced flexibility. Due to its superior performance, it can simultaneously support high-resolution video results and real-time inference. Extensive experiments demonstrate that our method synthesizes high-fidelity talking portraits with detailed facial textures and precise lip synchronization in real-time, outperforming state-of-the-art methods.

Title: Can We Optimize Deep RL Policy Weights as Trajectory Modeling?

• Authors: Hongyao Tang
• Subjects: cs.LG, cs.AI, cs.NE
• Abstract URL: https://arxiv.org/abs/2503.04074
• Pdf URL: https://arxiv.org/pdf/2503.04074
• Copy Paste: [[2503.04074]] Can We Optimize Deep RL Policy Weights as Trajectory Modeling?(https://arxiv.org/abs/2503.04074)
• Keywords: transformer
• Abstract: Learning the optimal policy from a random network initialization is the theme of deep Reinforcement Learning (RL). As the scale of DRL training increases, treating DRL policy network weights as a new data modality and exploring the potential becomes appealing and possible. In this work, we focus on the policy learning path in deep RL, represented by the trajectory of network weights of historical policies, which reflects the evolvement of the policy learning process. Taking the idea of trajectory modeling with Transformer, we propose Transformer as Implicit Policy Learner (TIPL), which processes policy network weights in an autoregressive manner. We collect the policy learning path data by running independent RL training trials, with which we then train our TIPL model. In the experiments, we demonstrate that TIPL is able to fit the implicit dynamics of policy learning and perform the optimization of policy network by inference.

Title: Instrument-Splatting: Controllable Photorealistic Reconstruction of Surgical Instruments Using Gaussian Splatting

• Authors: Shuojue Yang, Zijian Wu, Mingxuan Hong, Qian Li, Daiyun Shen, Septimiu E. Salcudean, Yueming Jin
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.04082
• Pdf URL: https://arxiv.org/pdf/2503.04082
• Copy Paste: [[2503.04082]] Instrument-Splatting: Controllable Photorealistic Reconstruction of Surgical Instruments Using Gaussian Splatting(https://arxiv.org/abs/2503.04082)
• Keywords: robust
• Abstract: Real2Sim is becoming increasingly important with the rapid development of surgical artificial intelligence (AI) and autonomy. In this work, we propose a novel Real2Sim methodology, \textit{Instrument-Splatting}, that leverages 3D Gaussian Splatting to provide fully controllable 3D reconstruction of surgical instruments from monocular surgical videos. To maintain both high visual fidelity and manipulability, we introduce a geometry pre-training to bind Gaussian point clouds on part mesh with accurate geometric priors and define a forward kinematics to control the Gaussians as flexible as real instruments. Afterward, to handle unposed videos, we design a novel instrument pose tracking method leveraging semantics-embedded Gaussians to robustly refine per-frame instrument poses and joint states in a render-and-compare manner, which allows our instrument Gaussian to accurately learn textures and reach photorealistic rendering. We validated our method on 2 publicly released surgical videos and 4 videos collected on ex vivo tissues and green screens. Quantitative and qualitative evaluations demonstrate the effectiveness and superiority of the proposed method.

Title: Brain Tumor Detection in MRI Based on Federated Learning with YOLOv11

• Authors: Sheikh Moonwara Anjum Monisha, Ratun Rahman
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04087
• Pdf URL: https://arxiv.org/pdf/2503.04087
• Copy Paste: [[2503.04087]] Brain Tumor Detection in MRI Based on Federated Learning with YOLOv11(https://arxiv.org/abs/2503.04087)
• Keywords: privacy, protect, robust, federate
• Abstract: One of the primary challenges in medical diagnostics is the accurate and efficient use of magnetic resonance imaging (MRI) for the detection of brain tumors. But the current machine learning (ML) approaches have two major limitations, data privacy and high latency. To solve the problem, in this work we propose a federated learning architecture for a better accurate brain tumor detection incorporating the YOLOv11 algorithm. In contrast to earlier methods of centralized learning, our federated learning approach protects the underlying medical data while supporting cooperative deep learning model training across multiple institutions. To allow the YOLOv11 model to locate and identify tumor areas, we adjust it to handle MRI data. To ensure robustness and generalizability, the model is trained and tested on a wide range of MRI data collected from several anonymous medical facilities. The results indicate that our method significantly maintains higher accuracy than conventional approaches.

Title: PokéChamp: an Expert-level Minimax Language Agent

• Authors: Seth Karten, Andy Luu Nguyen, Chi Jin
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.04094
• Pdf URL: https://arxiv.org/pdf/2503.04094
• Copy Paste: [[2503.04094]] PokéChamp: an Expert-level Minimax Language Agent(https://arxiv.org/abs/2503.04094)
• Keywords: large language model
• Abstract: We introduce PokéChamp, a minimax agent powered by Large Language Models (LLMs) for Pokémon battles. Built on a general framework for two-player competitive games, PokéChamp leverages the generalist capabilities of LLMs to enhance minimax tree search. Specifically, LLMs replace three key modules: (1) player action sampling, (2) opponent modeling, and (3) value function estimation, enabling the agent to effectively utilize gameplay history and human knowledge to reduce the search space and address partial observability. Notably, our framework requires no additional LLM training. We evaluate PokéChamp in the popular Gen 9 OU format. When powered by GPT-4o, it achieves a win rate of 76% against the best existing LLM-based bot and 84% against the strongest rule-based bot, demonstrating its superior performance. Even with an open-source 8-billion-parameter Llama 3.1 model, PokéChamp consistently outperforms the previous best LLM-based bot, Pokéllmon powered by GPT-4o, with a 64% win rate. PokéChamp attains a projected Elo of 1300-1500 on the Pokémon Showdown online ladder, placing it among the top 30%-10% of human players. In addition, this work compiles the largest real-player Pokémon battle dataset, featuring over 3 million games, including more than 500k high-Elo matches. Based on this dataset, we establish a series of battle benchmarks and puzzles to evaluate specific battling skills. We further provide key updates to the local game engine. We hope this work fosters further research that leverage Pokémon battle as benchmark to integrate LLM technologies with game-theoretic algorithms addressing general multiagent problems. Videos, code, and dataset available at this https URL.

Title: Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts

• Authors: Xiangnan Chen, Yuancheng Fang, Qian Xiao, Juncheng Li, Jun Lin, Siliang Tang, Yi Yang, Yueting Zhuang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04095
• Pdf URL: https://arxiv.org/pdf/2503.04095
• Copy Paste: [[2503.04095]] Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts(https://arxiv.org/abs/2503.04095)
• Keywords: large language model
• Abstract: Multimodal Large Language Models (MLLMs) have garnered significant attention for their strong visual-semantic understanding. Most existing chart benchmarks evaluate MLLMs' ability to parse information from charts to answer this http URL, they overlook the inherent output biases of MLLMs, where models rely on their parametric memory to answer questions rather than genuinely understanding the chart content. To address this limitation, we introduce a novel Chart Hypothetical Question Answering (HQA) task, which imposes assumptions on the same question to compel models to engage in counterfactual reasoning based on the chart content. Furthermore, we introduce HAI, a human-AI interactive data synthesis approach that leverages the efficient text-editing capabilities of LLMs alongside human expert knowledge to generate diverse and high-quality HQA data at a low cost. Using HAI, we construct Chart-HQA, a challenging benchmark synthesized from publicly available data sources. Evaluation results on 18 MLLMs of varying model sizes reveal that current models face significant generalization challenges and exhibit imbalanced reasoning performance on the HQA task.

Title: Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English

• Authors: Runtao Zhou, Guangya Wan, Saadia Gabriel, Sheng Li, Alexander J Gates, Maarten Sap, Thomas Hartvigsen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04099
• Pdf URL: https://arxiv.org/pdf/2503.04099
• Copy Paste: [[2503.04099]] Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English(https://arxiv.org/abs/2503.04099)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning tasks, leading to their widespread deployment. However, recent studies have highlighted concerning biases in these models, particularly in their handling of dialectal variations like African American English (AAE). In this work, we systematically investigate dialectal disparities in LLM reasoning tasks. We develop an experimental framework comparing LLM performance given Standard American English (SAE) and AAE prompts, combining LLM-based dialect conversion with established linguistic analyses. We find that LLMs consistently produce less accurate responses and simpler reasoning chains and explanations for AAE inputs compared to equivalent SAE questions, with disparities most pronounced in social science and humanities domains. These findings highlight systematic differences in how LLMs process and reason about different language varieties, raising important questions about the development and deployment of these systems in our multilingual and multidialectal world. Our code repository is publicly available at this https URL.

Title: LLMs Can Generate a Better Answer by Aggregating Their Own Responses

• Authors: Zichong Li, Xinyu Feng, Yuheng Cai, Zixuan Zhang, Tianyi Liu, Chen Liang, Weizhu Chen, Haoyu Wang, Tuo Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04104
• Pdf URL: https://arxiv.org/pdf/2503.04104
• Copy Paste: [[2503.04104]] LLMs Can Generate a Better Answer by Aggregating Their Own Responses(https://arxiv.org/abs/2503.04104)
• Keywords: generative, large language model
• Abstract: Large Language Models (LLMs) have shown remarkable capabilities across tasks, yet they often require additional prompting techniques when facing complex problems. While approaches like self-correction and response selection have emerged as popular solutions, recent studies have shown these methods perform poorly when relying on the LLM itself to provide feedback or selection criteria. We argue this limitation stems from the fact that common LLM post-training procedures lack explicit supervision for discriminative judgment tasks. In this paper, we propose Generative Self-Aggregation (GSA), a novel prompting method that improves answer quality without requiring the model's discriminative capabilities. GSA first samples multiple diverse responses from the LLM, then aggregates them to obtain an improved solution. Unlike previous approaches, our method does not require the LLM to correct errors or compare response quality; instead, it leverages the model's generative abilities to synthesize a new response based on the context of multiple samples. While GSA shares similarities with the self-consistency (SC) approach for response aggregation, SC requires specific verifiable tokens to enable majority voting. In contrast, our approach is more general and can be applied to open-ended tasks. Empirical evaluation demonstrates that GSA effectively improves response quality across various tasks, including mathematical reasoning, knowledge-based problems, and open-ended generation tasks such as code synthesis and conversational responses.

Title: WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Mining

• Authors: Haoran Wang, Lian Huai, Wenbin Li, Lei Qi, Xingqun Jiang, Yinghuan Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04106
• Pdf URL: https://arxiv.org/pdf/2503.04106
• Copy Paste: [[2503.04106]] WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Mining(https://arxiv.org/abs/2503.04106)
• Keywords: segmentation
• Abstract: We have witnessed remarkable progress in foundation models in vision tasks. Currently, several recent works have utilized the segmenting anything model (SAM) to boost the segmentation performance in medical images, where most of them focus on training an adaptor for fine-tuning a large amount of pixel-wise annotated medical images following a fully supervised manner. In this paper, to reduce the labeling cost, we investigate a novel weakly-supervised SAM-based segmentation model, namely WeakMedSAM. Specifically, our proposed WeakMedSAM contains two modules: 1) to mitigate severe co-occurrence in medical images, a sub-class exploration module is introduced to learn accurate feature representations. 2) to improve the quality of the class activation maps, our prompt affinity mining module utilizes the prompt capability of SAM to obtain an affinity map for random-walk refinement. Our method can be applied to any SAM-like backbone, and we conduct experiments with SAMUS and EfficientSAM. The experimental results on three popularly-used benchmark datasets, i.e., BraTS 2019, AbdomenCT-1K, and MSD Cardiac dataset, show the promising results of our proposed WeakMedSAM. Our code is available at this https URL.

Title: Fractional Correspondence Framework in Detection Transformer

• Authors: Masoumeh Zareapoor, Pourya Shamsolmoali, Huiyu Zhou, Yue Lu, Salvador García
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04107
• Pdf URL: https://arxiv.org/pdf/2503.04107
• Copy Paste: [[2503.04107]] Fractional Correspondence Framework in Detection Transformer(https://arxiv.org/abs/2503.04107)
• Keywords: transformer
• Abstract: The Detection Transformer (DETR), by incorporating the Hungarian algorithm, has significantly simplified the matching process in object detection tasks. This algorithm facilitates optimal one-to-one matching of predicted bounding boxes to ground-truth annotations during training. While effective, this strict matching process does not inherently account for the varying densities and distributions of objects, leading to suboptimal correspondences such as failing to handle multiple detections of the same object or missing small objects. To address this, we propose the Regularized Transport Plan (RTP). RTP introduces a flexible matching strategy that captures the cost of aligning predictions with ground truths to find the most accurate correspondences between these sets. By utilizing the differentiable Sinkhorn algorithm, RTP allows for soft, fractional matching rather than strict one-to-one assignments. This approach enhances the model's capability to manage varying object densities and distributions effectively. Our extensive evaluations on the MS-COCO and VOC benchmarks demonstrate the effectiveness of our approach. RTP-DETR, surpassing the performance of the Deform-DETR and the recently introduced DINO-DETR, achieving absolute gains in mAP of +3.8% and +1.7%, respectively.

Title: Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Ability

• Authors: Lijia Yu, Yibo Miao, Yifan Zhu, Xiao-Shan Gao, Lijun Zhang
• Subjects: cs.LG, cs.AI, math.ST
• Abstract URL: https://arxiv.org/abs/2503.04111
• Pdf URL: https://arxiv.org/pdf/2503.04111
• Copy Paste: [[2503.04111]] Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Ability(https://arxiv.org/abs/2503.04111)
• Keywords: robust
• Abstract: The primary objective of learning methods is generalization. Classic uniform generalization bounds, which rely on VC-dimension or Rademacher complexity, fail to explain the significant attribute that over-parameterized models in deep learning exhibit nice generalizability. On the other hand, algorithm-dependent generalization bounds, like stability bounds, often rely on strict assumptions. To establish generalizability under less stringent assumptions, this paper investigates the generalizability of neural networks that minimize or approximately minimize empirical risk. We establish a lower bound for population accuracy based on the expressiveness of these networks, which indicates that with an adequate large number of training samples and network sizes, these networks, including over-parameterized ones, can generalize effectively. Additionally, we provide a necessary condition for generalization, demonstrating that, for certain data distributions, the quantity of training data required to ensure generalization exceeds the network size needed to represent the corresponding data distribution. Finally, we provide theoretical insights into several phenomena in deep learning, including robust generalization, importance of over-parameterization, and effect of loss function on generalization.

Title: TimeFound: A Foundation Model for Time Series Forecasting

• Authors: Congxi Xiao, Jingbo Zhou, Yixiong Xiao, Xinjiang Lu, Le Zhang, Hui Xiong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04118
• Pdf URL: https://arxiv.org/pdf/2503.04118
• Copy Paste: [[2503.04118]] TimeFound: A Foundation Model for Time Series Forecasting(https://arxiv.org/abs/2503.04118)
• Keywords: transformer
• Abstract: We present TimeFound, an encoder-decoder transformer-based time series foundation model for out-of-the-box zero-shot forecasting. To handle time series data from various domains, TimeFound employs a multi-resolution patching strategy to capture complex temporal patterns at multiple scales. We pre-train our model with two sizes (200M and 710M parameters) on a large time-series corpus comprising both real-world and synthetic datasets. Over a collection of unseen datasets across diverse domains and forecasting horizons, our empirical evaluations suggest that TimeFound can achieve superior or competitive zero-shot forecasting performance, compared to state-of-the-art time series foundation models.

Title: SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfer

• Authors: Chunnan Shang, Zhizhong Wang, Hongwei Wang, Xiangming Meng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04119
• Pdf URL: https://arxiv.org/pdf/2503.04119
• Copy Paste: [[2503.04119]] SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfer(https://arxiv.org/abs/2503.04119)
• Keywords: diffusion, transformer
• Abstract: Attention-based arbitrary style transfer methods, including CNN-based, Transformer-based, and Diffusion-based, have flourished and produced high-quality stylized images. However, they perform poorly on the content and style images with the same semantics, i.e., the style of the corresponding semantic region of the generated stylized image is inconsistent with that of the style image. We argue that the root cause lies in their failure to consider the relationship between local regions and semantic regions. To address this issue, we propose a plug-and-play semantic continuous-sparse attention, dubbed SCSA, for arbitrary semantic style transfer -- each query point considers certain key points in the corresponding semantic region. Specifically, semantic continuous attention ensures each query point fully attends to all the continuous key points in the same semantic region that reflect the overall style characteristics of that region; Semantic sparse attention allows each query point to focus on the most similar sparse key point in the same semantic region that exhibits the specific stylistic texture of that region. By combining the two modules, the resulting SCSA aligns the overall style of the corresponding semantic regions while transferring the vivid textures of these regions. Qualitative and quantitative results prove that SCSA enables attention-based arbitrary style transfer methods to produce high-quality semantic stylized images.

Title: Simple Self Organizing Map with Visual Transformer

• Authors: Alan Luo, Kaiwen Yuan
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04121
• Pdf URL: https://arxiv.org/pdf/2503.04121
• Copy Paste: [[2503.04121]] Simple Self Organizing Map with Visual Transformer(https://arxiv.org/abs/2503.04121)
• Keywords: transformer
• Abstract: Vision Transformers (ViTs) have demonstrated exceptional performance in various vision tasks. However, they tend to underperform on smaller datasets due to their inherent lack of inductive biases. Current approaches address this limitation implicitly-often by pairing ViTs with pretext tasks or by distilling knowledge from convolutional neural networks (CNNs) to strengthen the prior. In contrast, Self-Organizing Maps (SOMs), a widely adopted self-supervised framework, are inherently structured to preserve topology and spatial organization, making them a promising candidate to directly address the limitations of ViTs in limited or small training datasets. Despite this potential, equipping SOMs with modern deep learning architectures remains largely unexplored. In this study, we conduct a novel exploration on how Vision Transformers (ViTs) and Self-Organizing Maps (SOMs) can empower each other, aiming to bridge this critical research gap. Our findings demonstrate that these architectures can synergistically enhance each other, leading to significantly improved performance in both unsupervised and supervised tasks. Code will be publicly available.

Title: Diff-Reg v2: Diffusion-Based Matching Matrix Estimation for Image Matching and 3D Registration

• Authors: Qianliang Wu, Haobo Jiang, Yaqing Ding, Lei Luo, Jin Xie, Jian Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04127
• Pdf URL: https://arxiv.org/pdf/2503.04127
• Copy Paste: [[2503.04127]] Diff-Reg v2: Diffusion-Based Matching Matrix Estimation for Image Matching and 3D Registration(https://arxiv.org/abs/2503.04127)
• Keywords: robust, diffusion
• Abstract: Establishing reliable correspondences is crucial for all registration tasks, including 2D image registration, 3D point cloud registration, and 2D-3D image-to-point cloud registration. However, these tasks are often complicated by challenges such as scale inconsistencies, symmetry, and large deformations, which can lead to ambiguous matches. Previous feature-based and correspondence-based methods typically rely on geometric or semantic features to generate or polish initial potential correspondences. Some methods typically leverage specific geometric priors, such as topological preservation, to devise diverse and innovative strategies tailored to a given enhancement goal, which cannot be exhaustively enumerated. Additionally, many previous approaches rely on a single-step prediction head, which can struggle with local minima in complex matching scenarios. To address these challenges, we introduce an innovative paradigm that leverages a diffusion model in matrix space for robust matching matrix estimation. Our model treats correspondence estimation as a denoising diffusion process in the matching matrix space, gradually refining the intermediate matching matrix to the optimal one. Specifically, we apply the diffusion model in the doubly stochastic matrix space for 3D-3D and 2D-3D registration tasks. In the 2D image registration task, we deploy the diffusion model in a matrix subspace where dual-softmax projection regularization is applied. For all three registration tasks, we provide adaptive matching matrix embedding implementations tailored to the specific characteristics of each task while maintaining a consistent "match-to-warp" encoding pattern. Furthermore, we adopt a lightweight design for the denoising module. In inference, once points or image features are extracted and fixed, this module performs multi-step denoising predictions through reverse sampling.

Title: Token-Efficient Long Video Understanding for Multimodal LLMs

• Authors: Jindong Jiang, Xiuyu Li, Zhijian Liu, Muyang Li, Guo Chen, Zhiqi Li, De-An Huang, Guilin Liu, Zhiding Yu, Kurt Keutzer, Sungjin Ahn, Jan Kautz, Hongxu Yin, Yao Lu, Song Han, Wonmin Byeon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04130
• Pdf URL: https://arxiv.org/pdf/2503.04130
• Copy Paste: [[2503.04130]] Token-Efficient Long Video Understanding for Multimodal LLMs(https://arxiv.org/abs/2503.04130)
• Keywords: robust, large language model
• Abstract: Recent advances in video-based multimodal large language models (Video-LLMs) have significantly improved video understanding by processing videos as sequences of image frames. However, many existing methods treat frames independently in the vision backbone, lacking explicit temporal modeling, which limits their ability to capture dynamic patterns and efficiently handle long videos. To address these limitations, we introduce STORM (\textbf{S}patiotemporal \textbf{TO}ken \textbf{R}eduction for \textbf{M}ultimodal LLMs), a novel architecture incorporating a dedicated temporal encoder between the image encoder and the LLM. Our temporal encoder leverages the Mamba State Space Model to integrate temporal information into image tokens, generating enriched representations that preserve inter-frame dynamics across the entire video sequence. This enriched encoding not only enhances video reasoning capabilities but also enables effective token reduction strategies, including test-time sampling and training-based temporal and spatial pooling, substantially reducing computational demands on the LLM without sacrificing key temporal information. By integrating these techniques, our approach simultaneously reduces training and inference latency while improving performance, enabling efficient and robust video understanding over extended temporal contexts. Extensive evaluations show that STORM achieves state-of-the-art results across various long video understanding benchmarks (more than 5\% improvement on MLVU and LongVideoBench) while reducing the computation costs by up to $8\times$ and the decoding latency by 2.4-2.9$\times$ for the fixed numbers of input frames. Project page is available at this https URL

Title: Q-PART: Quasi-Periodic Adaptive Regression with Test-time Training for Pediatric Left Ventricular Ejection Fraction Regression

• Authors: Jie Liu, Tiexin Qin, Hui Liu, Yilei Shi, Lichao Mou, Xiao Xiang Zhu, Shiqi Wang, Haoliang Li
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04131
• Pdf URL: https://arxiv.org/pdf/2503.04131
• Copy Paste: [[2503.04131]] Q-PART: Quasi-Periodic Adaptive Regression with Test-time Training for Pediatric Left Ventricular Ejection Fraction Regression(https://arxiv.org/abs/2503.04131)
• Keywords: robust, fair
• Abstract: In this work, we address the challenge of adaptive pediatric Left Ventricular Ejection Fraction (LVEF) assessment. While Test-time Training (TTT) approaches show promise for this task, they suffer from two significant limitations. Existing TTT works are primarily designed for classification tasks rather than continuous value regression, and they lack mechanisms to handle the quasi-periodic nature of cardiac signals. To tackle these issues, we propose a novel \textbf{Q}uasi-\textbf{P}eriodic \textbf{A}daptive \textbf{R}egression with \textbf{T}est-time Training (Q-PART) framework. In the training stage, the proposed Quasi-Period Network decomposes the echocardiogram into periodic and aperiodic components within latent space by combining parameterized helix trajectories with Neural Controlled Differential Equations. During inference, our framework further employs a variance minimization strategy across image augmentations that simulate common quality issues in echocardiogram acquisition, along with differential adaptation rates for periodic and aperiodic components. Theoretical analysis is provided to demonstrate that our variance minimization objective effectively bounds the regression error under mild conditions. Furthermore, extensive experiments across three pediatric age groups demonstrate that Q-PART not only significantly outperforms existing approaches in pediatric LVEF prediction, but also exhibits strong clinical screening capability with high mAUROC scores (up to 0.9747) and maintains gender-fair performance across all metrics, validating its robustness and practical utility in pediatric echocardiography analysis.

Title: Biological Sequence with Language Model Prompting: A Survey

• Authors: Jiyue Jiang, Zikang Wang, Yuheng Shan, Heyan Chai, Jiayi Li, Zixian Ma, Xinrui Zhang, Yu Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04135
• Pdf URL: https://arxiv.org/pdf/2503.04135
• Copy Paste: [[2503.04135]] Biological Sequence with Language Model Prompting: A Survey(https://arxiv.org/abs/2503.04135)
• Keywords: large language model
• Abstract: Large Language models (LLMs) have emerged as powerful tools for addressing challenges across diverse domains. Notably, recent studies have demonstrated that large language models significantly enhance the efficiency of biomolecular analysis and synthesis, attracting widespread attention from academics and medicine. In this paper, we systematically investigate the application of prompt-based methods with LLMs to biological sequences, including DNA, RNA, proteins, and drug discovery tasks. Specifically, we focus on how prompt engineering enables LLMs to tackle domain-specific problems, such as promoter sequence prediction, protein structure modeling, and drug-target binding affinity prediction, often with limited labeled data. Furthermore, our discussion highlights the transformative potential of prompting in bioinformatics while addressing key challenges such as data scarcity, multimodal fusion, and computational resource limitations. Our aim is for this paper to function both as a foundational primer for newcomers and a catalyst for continued innovation within this dynamic field of study.

Title: Robust Computer-Vision based Construction Site Detection for Assistive-Technology Applications

• Authors: Junchi Feng, Giles Hamilton-Fletcher, Nikhil Ballem, Michael Batavia, Yifei Wang, Jiuling Zhong, Maurizio Porfiri, John-Ross Rizzo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04139
• Pdf URL: https://arxiv.org/pdf/2503.04139
• Copy Paste: [[2503.04139]] Robust Computer-Vision based Construction Site Detection for Assistive-Technology Applications(https://arxiv.org/abs/2503.04139)
• Keywords: robust
• Abstract: Navigating urban environments poses significant challenges for people with disabilities, particularly those with blindness and low vision. Environments with dynamic and unpredictable elements like construction sites are especially challenging. Construction sites introduce hazards like uneven surfaces, obstructive barriers, hazardous materials, and excessive noise, and they can alter routing, complicating safe mobility. Existing assistive technologies are limited, as navigation apps do not account for construction sites during trip planning, and detection tools that attempt hazard recognition struggle to address the extreme variability of construction paraphernalia. This study introduces a novel computer vision-based system that integrates open-vocabulary object detection, a YOLO-based scaffolding-pole detection model, and an optical character recognition (OCR) module to comprehensively identify and interpret construction site elements for assistive navigation. In static testing across seven construction sites, the system achieved an overall accuracy of 88.56\%, reliably detecting objects from 2m to 10m within a 0$^\circ$ -- 75$^\circ$ angular offset. At closer distances (2--4m), the detection rate was 100\% at all tested angles. At

Title: LiteChain: A Lightweight Blockchain for Verifiable and Scalable Federated Learning in Massive Edge Networks

• Authors: Handi Chen, Rui Zhou, Yun-Hin Chan, Zhihan Jiang, Xianhao Chen, Edith C.H. Ngai
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.04140
• Pdf URL: https://arxiv.org/pdf/2503.04140
• Copy Paste: [[2503.04140]] LiteChain: A Lightweight Blockchain for Verifiable and Scalable Federated Learning in Massive Edge Networks(https://arxiv.org/abs/2503.04140)
• Keywords: secure, security, attack, robust, federate
• Abstract: Leveraging blockchain in Federated Learning (FL) emerges as a new paradigm for secure collaborative learning on Massive Edge Networks (MENs). As the scale of MENs increases, it becomes more difficult to implement and manage a blockchain among edge devices due to complex communication topologies, heterogeneous computation capabilities, and limited storage capacities. Moreover, the lack of a standard metric for blockchain security becomes a significant issue. To address these challenges, we propose a lightweight blockchain for verifiable and scalable FL, namely LiteChain, to provide efficient and secure services in MENs. Specifically, we develop a distributed clustering algorithm to reorganize MENs into a two-level structure to improve communication and computing efficiency under security requirements. Moreover, we introduce a Comprehensive Byzantine Fault Tolerance (CBFT) consensus mechanism and a secure update mechanism to ensure the security of model transactions through LiteChain. Our experiments based on Hyperledger Fabric demonstrate that LiteChain presents the lowest end-to-end latency and on-chain storage overheads across various network scales, outperforming the other two benchmarks. In addition, LiteChain exhibits a high level of robustness against replay and data poisoning attacks.

Title: MTS: A Deep Reinforcement Learning Portfolio Management Framework with Time-Awareness and Short-Selling

• Authors: Fengchen Gu, Zhengyong Jiang, Ángel F. García-Fernández, Angelos Stefanidis, Jionglong Su, Huakang Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04143
• Pdf URL: https://arxiv.org/pdf/2503.04143
• Copy Paste: [[2503.04143]] MTS: A Deep Reinforcement Learning Portfolio Management Framework with Time-Awareness and Short-Selling(https://arxiv.org/abs/2503.04143)
• Keywords: robust
• Abstract: Portfolio management remains a crucial challenge in finance, with traditional methods often falling short in complex and volatile market environments. While deep reinforcement approaches have shown promise, they still face limitations in dynamic risk management, exploitation of temporal markets, and incorporation of complex trading strategies such as short-selling. These limitations can lead to suboptimal portfolio performance, increased vulnerability to market volatility, and missed opportunities in capturing potential returns from diverse market conditions. This paper introduces a Deep Reinforcement Learning Portfolio Management Framework with Time-Awareness and Short-Selling (MTS), offering a robust and adaptive strategy for sustainable investment performance. This framework utilizes a novel encoder-attention mechanism to address the limitations by incorporating temporal market characteristics, a parallel strategy for automated short-selling based on market trends, and risk management through innovative Incremental Conditional Value at Risk, enhancing adaptability and performance. Experimental validation on five diverse datasets from 2019 to 2023 demonstrates MTS's superiority over traditional algorithms and advanced machine learning techniques. MTS consistently achieves higher cumulative returns, Sharpe, Omega, and Sortino ratios, underscoring its effectiveness in balancing risk and return while adapting to market dynamics. MTS demonstrates an average relative increase of 30.67% in cumulative returns and 29.33% in Sharpe ratio compared to the next best-performing strategies across various datasets.

Title: DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval

• Authors: Yating Liu, Zimo Liu, Xiangyuan Lan, Wenming Yang, Yaowei Li, Qingmin Liao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04144
• Pdf URL: https://arxiv.org/pdf/2503.04144
• Copy Paste: [[2503.04144]] DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval(https://arxiv.org/abs/2503.04144)
• Keywords: extraction
• Abstract: Text-based person retrieval (TPR) has gained significant attention as a fine-grained and challenging task that closely aligns with practical applications. Tailoring CLIP to person domain is now a emerging research topic due to the abundant knowledge of vision-language pretraining, but challenges still remain during fine-tuning: (i) Previous full-model fine-tuning in TPR is computationally expensive and prone to overfitting.(ii) Existing parameter-efficient transfer learning (PETL) for TPR lacks of fine-grained feature extraction. To address these issues, we propose Domain-Aware Mixture-of-Adapters (DM-Adapter), which unifies Mixture-of-Experts (MOE) and PETL to enhance fine-grained feature representations while maintaining efficiency. Specifically, Sparse Mixture-of-Adapters is designed in parallel to MLP layers in both vision and language branches, where different experts specialize in distinct aspects of person knowledge to handle features more finely. To promote the router to exploit domain information effectively and alleviate the routing imbalance, Domain-Aware Router is then developed by building a novel gating function and injecting learnable domain-aware prompts. Extensive experiments show that our DM-Adapter achieves state-of-the-art performance, outperforming previous methods by a significant margin.

Title: Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Servers

• Authors: Varatheepan Paramanayakam, Andreas Karatzas, Dimitrios Stamoulis, Iraklis Anagnostopoulos
• Subjects: cs.LG, cs.DC, cs.PF
• Abstract URL: https://arxiv.org/abs/2503.04148
• Pdf URL: https://arxiv.org/pdf/2503.04148
• Copy Paste: [[2503.04148]] Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Servers(https://arxiv.org/abs/2503.04148)
• Keywords: transformer
• Abstract: Edge computing systems struggle to efficiently manage multiple concurrent deep neural network (DNN) workloads while meeting strict latency requirements, minimizing power consumption, and maintaining environmental sustainability. This paper introduces Ecomap, a sustainability-driven framework that dynamically adjusts the maximum power threshold of edge devices based on real-time carbon intensity. Ecomap incorporates the innovative use of mixed-quality models, allowing it to dynamically replace computationally heavy DNNs with lighter alternatives when latency constraints are violated, ensuring service responsiveness with minimal accuracy loss. Additionally, it employs a transformer-based estimator to guide efficient workload mappings. Experimental results using NVIDIA Jetson AGX Xavier demonstrate that Ecomap reduces carbon emissions by an average of 30% and achieves a 25% lower carbon delay product (CDP) compared to state-of-the-art methods, while maintaining comparable or better latency and power efficiency.

Title: Ticktack : Long Span Temporal Alignment of Large Language Models Leveraging Sexagenary Cycle Time Expression

• Authors: Xue Han, Qian Hu, Yitong Wang, Wenchun Gao, Lianlian Zhang, Qing Wang, Lijun Mei, Chao Deng, Junlan Feng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04150
• Pdf URL: https://arxiv.org/pdf/2503.04150
• Copy Paste: [[2503.04150]] Ticktack : Long Span Temporal Alignment of Large Language Models Leveraging Sexagenary Cycle Time Expression(https://arxiv.org/abs/2503.04150)
• Keywords: large language model
• Abstract: Large language models (LLMs) suffer from temporal misalignment issues especially across long span of time. The issue arises from knowing that LLMs are trained on large amounts of data where temporal information is rather sparse over long times, such as thousands of years, resulting in insufficient learning or catastrophic forgetting by the LLMs. This paper proposes a methodology named "Ticktack" for addressing the LLM's long-time span misalignment in a yearly setting. Specifically, we first propose to utilize the sexagenary year expression instead of the Gregorian year expression employed by LLMs, achieving a more uniform distribution in yearly granularity. Then, we employ polar coordinates to model the sexagenary cycle of 60 terms and the year order within each term, with additional temporal encoding to ensure LLMs understand them. Finally, we present a temporal representational alignment approach for post-training LLMs that effectively distinguishes time points with relevant knowledge, hence improving performance on time-related tasks, particularly over a long period. We also create a long time span benchmark for evaluation. Experimental results prove the effectiveness of our proposal.

Title: Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation

• Authors: Jie Xu, Na Zhao, Gang Niu, Masashi Sugiyama, Xiaofeng Zhu
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04151
• Pdf URL: https://arxiv.org/pdf/2503.04151
• Copy Paste: [[2503.04151]] Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation(https://arxiv.org/abs/2503.04151)
• Keywords: robust, transformer
• Abstract: Recently, multi-view learning (MVL) has garnered significant attention due to its ability to fuse discriminative information from multiple views. However, real-world multi-view datasets are often heterogeneous and imperfect, which usually makes MVL methods designed for specific combinations of views lack application potential and limits their effectiveness. To address this issue, we propose a novel robust MVL method (namely RML) with simultaneous representation fusion and alignment. Specifically, we introduce a simple yet effective multi-view transformer fusion network where we transform heterogeneous multi-view data into homogeneous word embeddings, and then integrate multiple views by the sample-level attention mechanism to obtain a fused representation. Furthermore, we propose a simulated perturbation based multi-view contrastive learning framework that dynamically generates the noise and unusable perturbations for simulating imperfect data conditions. The simulated noisy and unusable data obtain two distinct fused representations, and we utilize contrastive learning to align them for learning discriminative and robust representations. Our RML is self-supervised and can also be applied for downstream tasks as a regularization. In experiments, we employ it in unsupervised multi-view clustering, noise-label classification, and as a plug-and-play module for cross-modal hashing retrieval. Extensive comparison experiments and ablation studies validate the effectiveness of RML.

Title: BPQA Dataset: Evaluating How Well Language Models Leverage Blood Pressures to Answer Biomedical Questions

• Authors: Chi Hang, Ruiqi Deng, Lavender Yao Jiang, Zihao Yang, Anton Alyakin, Daniel Alber, Eric Karl Oermann
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04155
• Pdf URL: https://arxiv.org/pdf/2503.04155
• Copy Paste: [[2503.04155]] BPQA Dataset: Evaluating How Well Language Models Leverage Blood Pressures to Answer Biomedical Questions(https://arxiv.org/abs/2503.04155)
• Keywords: transformer
• Abstract: Clinical measurements such as blood pressures and respiration rates are critical in diagnosing and monitoring patient outcomes. It is an important component of biomedical data, which can be used to train transformer-based language models (LMs) for improving healthcare delivery. It is, however, unclear whether LMs can effectively interpret and use clinical measurements. We investigate two questions: First, can LMs effectively leverage clinical measurements to answer related medical questions? Second, how to enhance an LM's performance on medical question-answering (QA) tasks that involve measurements? We performed a case study on blood pressure readings (BPs), a vital sign routinely monitored by medical professionals. We evaluated the performance of four LMs: BERT, BioBERT, MedAlpaca, and GPT-3.5, on our newly developed dataset, BPQA (Blood Pressure Question Answering). BPQA contains $100$ medical QA pairs that were verified by medical students and designed to rely on BPs . We found that GPT-3.5 and MedAlpaca (larger and medium sized LMs) benefit more from the inclusion of BPs than BERT and BioBERT (small sized LMs). Further, augmenting measurements with labels improves the performance of BioBERT and Medalpaca (domain specific LMs), suggesting that retrieval may be useful for improving domain-specific LMs.

Title: DuCos: Duality Constrained Depth Super-Resolution via Foundation Model

• Authors: Zhiqiang Yan, Zhengxue Wang, Haoye Dong, Jun Li, Jian Yang, Gim Hee Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04171
• Pdf URL: https://arxiv.org/pdf/2503.04171
• Copy Paste: [[2503.04171]] DuCos: Duality Constrained Depth Super-Resolution via Foundation Model(https://arxiv.org/abs/2503.04171)
• Keywords: robust
• Abstract: We introduce DuCos, a novel depth super-resolution framework grounded in Lagrangian duality theory, offering a flexible integration of multiple constraints and reconstruction objectives to enhance accuracy and robustness. Our DuCos is the first to significantly improve generalization across diverse scenarios with foundation models as prompts. The prompt design consists of two key components: Correlative Fusion (CF) and Gradient Regulation (GR). CF facilitates precise geometric alignment and effective fusion between prompt and depth features, while GR refines depth predictions by enforcing consistency with sharp-edged depth maps derived from foundation models. Crucially, these prompts are seamlessly embedded into the Lagrangian constraint term, forming a synergistic and principled framework. Extensive experiments demonstrate that DuCos outperforms existing state-of-the-art methods, achieving superior accuracy, robustness, and generalization. The source codes and pre-trained models will be publicly available.

Title: UniNet: A Unified Multi-granular Traffic Modeling Framework for Network Security

• Authors: Binghui Wu, Dinil Mon Divakaran, Mohan Gurusamy
• Subjects: cs.CR, cs.LG, cs.NI
• Abstract URL: https://arxiv.org/abs/2503.04174
• Pdf URL: https://arxiv.org/pdf/2503.04174
• Copy Paste: [[2503.04174]] UniNet: A Unified Multi-granular Traffic Modeling Framework for Network Security(https://arxiv.org/abs/2503.04174)
• Keywords: security, privacy, attack, robust
• Abstract: As modern networks grow increasingly complex--driven by diverse devices, encrypted protocols, and evolving threats--network traffic analysis has become critically important. Existing machine learning models often rely only on a single representation of packets or flows, limiting their ability to capture the contextual relationships essential for robust analysis. Furthermore, task-specific architectures for supervised, semi-supervised, and unsupervised learning lead to inefficiencies in adapting to varying data formats and security tasks. To address these gaps, we propose UniNet, a unified framework that introduces a novel multi-granular traffic representation (T-Matrix), integrating session, flow, and packet-level features to provide comprehensive contextual information. Combined with T-Attent, a lightweight attention-based model, UniNet efficiently learns latent embeddings for diverse security tasks. Extensive evaluations across four key network security and privacy problems--anomaly detection, attack classification, IoT device identification, and encrypted website fingerprinting--demonstrate UniNet's significant performance gain over state-of-the-art methods, achieving higher accuracy, lower false positive rates, and improved scalability. By addressing the limitations of single-level models and unifying traffic analysis paradigms, UniNet sets a new benchmark for modern network security.

Title: Unsupervised anomaly detection on cybersecurity data streams: a case with BETH dataset

• Authors: Evgeniy Eremin
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04178
• Pdf URL: https://arxiv.org/pdf/2503.04178
• Copy Paste: [[2503.04178]] Unsupervised anomaly detection on cybersecurity data streams: a case with BETH dataset(https://arxiv.org/abs/2503.04178)
• Keywords: security, attack
• Abstract: In modern world the importance of cybersecurity of various systems is increasing from year to year. The number of information security events generated by information security tools grows up with the development of the IT infrastructure. At the same time, the cyber threat landscape does not remain constant, and monitoring should take into account both already known attack indicators and those for which there are no signature rules in information security products of various classes yet. Detecting anomalies in large cybersecurity data streams is a complex task that, if properly addressed, can allow for timely response to atypical and previously unknown cyber threats. The possibilities of using of offline algorithms may be limited for a number of reasons related to the time of training and the frequency of retraining. Using stream learning algorithms for solving this task is capable of providing near-real-time data processing. This article examines the results of ten algorithms from three Python stream machine-learning libraries on BETH dataset with cybersecurity events, which contains information about the creation, cloning, and destruction of operating system processes collected using extended eBPF. ROC-AUC metric and total processing time of processing with these algorithms are presented. Several combinations of features and the order of events are considered. In conclusion, some mentions are given about the most promising algorithms and possible directions for further research are outlined.

Title: CrowdHMTware: A Cross-level Co-adaptation Middleware for Context-aware Mobile DL Deployment

• Authors: Sicong Liu, Bin Guo, Shiyan Luo, Yuzhan Wang, Hao Luo, Cheng Fang, Yuan Xu, Ke Ma, Yao Li, Zhiwen Yu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04183
• Pdf URL: https://arxiv.org/pdf/2503.04183
• Copy Paste: [[2503.04183]] CrowdHMTware: A Cross-level Co-adaptation Middleware for Context-aware Mobile DL Deployment(https://arxiv.org/abs/2503.04183)
• Keywords: robust
• Abstract: There are many deep learning (DL) powered mobile and wearable applications today continuously and unobtrusively sensing the ambient surroundings to enhance all aspects of human this http URL enable robust and private mobile sensing, DL models are often deployed locally on resource-constrained mobile devices using techniques such as model compression or this http URL, existing methods, either front-end algorithm level (i.e. DL model compression/partitioning) or back-end scheduling level (i.e. operator/resource scheduling), cannot be locally online because they require offline retraining to ensure accuracy or rely on manually pre-defined strategies, struggle with dynamic this http URL primary challenge lies in feeding back runtime performance from the back-end level to the front-end level optimization decision. Moreover, the adaptive mobile DL model porting middleware with cross-level co-adaptation is less explored, particularly in mobile environments with diversity and dynamics. In response, we introduce CrowdHMTware, a dynamic context-adaptive DL model deployment middleware for heterogeneous mobile devices. It establishes an automated adaptation loop between cross-level functional components, i.e. elastic inference, scalable offloading, and model-adaptive engine, enhancing scalability and adaptability. Experiments with four typical tasks across 15 platforms and a real-world case study demonstrate that CrowdHMTware can effectively scale DL model, offloading, and engine actions across diverse platforms and tasks. It hides run-time system issues from developers, reducing the required developer expertise.

Title: Measuring temporal effects of agent knowledge by date-controlled tool use

• Authors: R. Patrick Xian, Qiming Cui, Stefan Bauer, Reza Abbasi-Asl
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2503.04188
• Pdf URL: https://arxiv.org/pdf/2503.04188
• Copy Paste: [[2503.04188]] Measuring temporal effects of agent knowledge by date-controlled tool use(https://arxiv.org/abs/2503.04188)
• Keywords: large language model
• Abstract: Temporal progression is an integral part of knowledge accumulation and update. Web search is frequently adopted as grounding for agent knowledge, yet its inappropriate configuration affects the quality of agent responses. Here, we construct a tool-based out-of-sample testing framework to measure the knowledge variability of large language model (LLM) agents from distinct date-controlled tools (DCTs). We demonstrate the temporal effects of an LLM agent as a writing assistant, which can use web search to help complete scientific publication abstracts. We show that temporal effects of the search engine translates into tool-dependent agent performance but can be alleviated with base model choice and explicit reasoning instructions such as chain-of-thought prompting. Our results indicate that agent evaluation should take a dynamical view and account for the temporal influence of tools and the updates of external resources.

Title: MASTER: Multimodal Segmentation with Text Prompts

• Authors: Fuyang Liu, Shun Lu, Jilin Mei, Yu Hu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04199
• Pdf URL: https://arxiv.org/pdf/2503.04199
• Copy Paste: [[2503.04199]] MASTER: Multimodal Segmentation with Text Prompts(https://arxiv.org/abs/2503.04199)
• Keywords: large language model, segmentation
• Abstract: RGB-Thermal fusion is a potential solution for various weather and light conditions in challenging scenarios. However, plenty of studies focus on designing complex modules to fuse different modalities. With the widespread application of large language models (LLMs), valuable information can be more effectively extracted from natural language. Therefore, we aim to leverage the advantages of large language models to design a structurally simple and highly adaptable multimodal fusion model architecture. We proposed MultimodAl Segmentation with TExt PRompts (MASTER) architecture, which integrates LLM into the fusion of RGB-Thermal multimodal data and allows complex query text to participate in the fusion process. Our model utilizes a dual-path structure to extract information from different modalities of images. Additionally, we employ LLM as the core module for multimodal fusion, enabling the model to generate learnable codebook tokens from RGB, thermal images, and textual information. A lightweight image decoder is used to obtain semantic segmentation results. The proposed MASTER performs exceptionally well in benchmark tests across various automated driving scenarios, yielding promising results.

Title: Knowledge-Decoupled Synergetic Learning: An MLLM based Collaborative Approach to Few-shot Multimodal Dialogue Intention Recognition

• Authors: Bin Chen, Yu Zhang, Hongfei Ye, Ziyi Huang, Hongyang Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04201
• Pdf URL: https://arxiv.org/pdf/2503.04201
• Copy Paste: [[2503.04201]] Knowledge-Decoupled Synergetic Learning: An MLLM based Collaborative Approach to Few-shot Multimodal Dialogue Intention Recognition(https://arxiv.org/abs/2503.04201)
• Keywords: large language model
• Abstract: Few-shot multimodal dialogue intention recognition is a critical challenge in the e-commerce domainn. Previous methods have primarily enhanced model classification capabilities through post-training techniques. However, our analysis reveals that training for few-shot multimodal dialogue intention recognition involves two interconnected tasks, leading to a seesaw effect in multi-task learning. This phenomenon is attributed to knowledge interference stemming from the superposition of weight matrix updates during the training process. To address these challenges, we propose Knowledge-Decoupled Synergetic Learning (KDSL), which mitigates these issues by utilizing smaller models to transform knowledge into interpretable rules, while applying the post-training of larger models. By facilitating collaboration between the large and small multimodal large language models for prediction, our approach demonstrates significant improvements. Notably, we achieve outstanding results on two real Taobao datasets, with enhancements of 6.37\% and 6.28\% in online weighted F1 scores compared to the state-of-the-art method, thereby validating the efficacy of our framework.

Title: Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models

• Authors: Rui Jiang, Xinghe Fu, Guangcong Zheng, Teng Li, Taiping Yao, Xi Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04215
• Pdf URL: https://arxiv.org/pdf/2503.04215
• Copy Paste: [[2503.04215]] Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models(https://arxiv.org/abs/2503.04215)
• Keywords: diffusion
• Abstract: The rapid advancement of pretrained text-driven diffusion models has significantly enriched applications in image generation and editing. However, as the demand for personalized content editing increases, new challenges emerge especially when dealing with arbitrary objects and complex scenes. Existing methods usually mistakes mask as the object shape prior, which struggle to achieve a seamless integration result. The mostly used inversion noise initialization also hinders the identity consistency towards the target object. To address these challenges, we propose a novel training-free framework that formulates personalized content editing as the optimization of edited images in the latent space, using diffusion models as the energy function guidance conditioned by reference text-image pairs. A coarse-to-fine strategy is proposed that employs text energy guidance at the early stage to achieve a natural transition toward the target class and uses point-to-point feature-level image energy guidance to perform fine-grained appearance alignment with the target object. Additionally, we introduce the latent space content composition to enhance overall identity consistency with the target. Extensive experiments demonstrate that our method excels in object replacement even with a large domain gap, highlighting its potential for high-quality, personalized image editing.

Title: FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion

• Authors: Ziyi Yang, Fanqi Wan, Longguang Zhong, Canbin Huang, Guosheng Liang, Xiaojun Quan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04222
• Pdf URL: https://arxiv.org/pdf/2503.04222
• Copy Paste: [[2503.04222]] FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion(https://arxiv.org/abs/2503.04222)
• Keywords: large language model
• Abstract: We introduce FuseChat-3.0, a suite of large language models (LLMs) developed by integrating the strengths of heterogeneous source LLMs into more compact target LLMs. Our source models include the powerful Gemma-2-27B-it, Mistral-Large-Instruct-2407, Qwen-2.5-72B-Instruct, and Llama-3.1-70B-Instruct. For target models, we focus on three widely-used smaller variants-Llama-3.1-8B-Instruct, Gemma-2-9B-it, and Qwen-2.5-7B-Instruct-along with two ultra-compact options, Llama-3.2-3B-Instruct and Llama-3.2-1B-Instruct. To leverage the diverse capabilities of these source models, we develop a specialized data construction protocol tailored to various tasks and domains. The FuseChat-3.0 training pipeline consists of two key stages: (1) supervised fine-tuning (SFT) to align the target and source model distributions, and (2) Direct Preference Optimization (DPO) to apply preferences from multiple source LLMs to fine-tune the target model. The resulting FuseChat-3.0 models exhibit significant performance gains across tasks such as instruction following, general knowledge, mathematics, and coding. As illustrated in Figure 1, using Llama-3.1-8B-Instruct as the target model, our fusion approach achieves an average improvement of 6.8 points across 14 benchmarks. Moreover, it demonstrates remarkable gains of 37.1 points and 30.1 points on the instruction-following benchmarks AlpacaEval-2 and Arena-Hard, respectively. Our code, models, and datasets are available at this https URL.

Title: Synthetic Data is an Elegant GIFT for Continual Vision-Language Models

• Authors: Bin Wu, Wuxuan Shi, Jinqiao Wang, Mang Ye
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04229
• Pdf URL: https://arxiv.org/pdf/2503.04229
• Copy Paste: [[2503.04229]] Synthetic Data is an Elegant GIFT for Continual Vision-Language Models(https://arxiv.org/abs/2503.04229)
• Keywords: diffusion
• Abstract: Pre-trained Vision-Language Models (VLMs) require Continual Learning (CL) to efficiently update their knowledge and adapt to various downstream tasks without retraining from scratch. However, for VLMs, in addition to the loss of knowledge previously learned from downstream tasks, pre-training knowledge is also corrupted during continual fine-tuning. This issue is exacerbated by the unavailability of original pre-training data, leaving VLM's generalization ability degrading. In this paper, we propose GIFT, a novel continual fine-tuning approach that utilizes synthetic data to overcome catastrophic forgetting in VLMs. Taking advantage of recent advances in text-to-image synthesis, we employ a pre-trained diffusion model to recreate both pre-training and learned downstream task data. In this way, the VLM can revisit previous knowledge through distillation on matching diffusion-generated images and corresponding text prompts. Leveraging the broad distribution and high alignment between synthetic image-text pairs in VLM's feature space, we propose a contrastive distillation loss along with an image-text alignment constraint. To further combat in-distribution overfitting and enhance distillation performance with limited amount of generated data, we incorporate adaptive weight consolidation, utilizing Fisher information from these synthetic image-text pairs and achieving a better stability-plasticity balance. Extensive experiments demonstrate that our method consistently outperforms previous state-of-the-art approaches across various settings.

Title: One-Shot Clustering for Federated Learning

• Authors: Maciej Krzysztof Zuziak, Roberto Pellungrini, Salvatore Rinzivillo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04231
• Pdf URL: https://arxiv.org/pdf/2503.04231
• Copy Paste: [[2503.04231]] One-Shot Clustering for Federated Learning(https://arxiv.org/abs/2503.04231)
• Keywords: federate
• Abstract: Federated Learning (FL) is a widespread and well adopted paradigm of decentralized learning that allows training one model from multiple sources without the need to directly transfer data between participating clients. Since its inception in 2015, it has been divided into numerous sub-fields that deal with application-specific issues, be it data heterogeneity or resource allocation. One such sub-field, Clustered Federated Learning (CFL), is dealing with the problem of clustering the population of clients into separate cohorts to deliver personalized models. Although few remarkable works have been published in this domain, the problem is still largely unexplored, as its basic assumption and settings are slightly different from standard FL. In this work, we present One-Shot Clustered Federated Learning (OCFL), a clustering-agnostic algorithm that can automatically detect the earliest suitable moment for clustering. Our algorithm is based on the computation of cosine similarity between gradients of the clients and a temperature measure that detects when the federated model starts to converge. We empirically evaluate our methodology by testing various one-shot clustering algorithms for over thirty different tasks on three benchmark datasets. Our experiments showcase the good performance of our approach when used to perform CFL in an automated manner without the need to adjust hyperparameters.

Title: Geometry-Constrained Monocular Scale Estimation Using Semantic Segmentation for Dynamic Scenes

• Authors: Hui Zhang, Zhiyang Wu, Qianqian Shangguan, Kang An
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04235
• Pdf URL: https://arxiv.org/pdf/2503.04235
• Copy Paste: [[2503.04235]] Geometry-Constrained Monocular Scale Estimation Using Semantic Segmentation for Dynamic Scenes(https://arxiv.org/abs/2503.04235)
• Keywords: segmentation
• Abstract: Monocular visual localization plays a pivotal role in advanced driver assistance systems and autonomous driving by estimating a vehicle's ego-motion from a single pinhole camera. Nevertheless, conventional monocular visual odometry encoun-ters challenges in scale estimation due to the absence of depth information during projection. Previous methodologies, whether rooted in physical constraints or deep learning paradigms, con-tend with issues related to computational complexity and the management of dynamic objects. This study extends our prior research, presenting innovative strategies for ego-motion estima-tion and the selection of ground points. Striving for a nuanced equilibrium between computational efficiency and precision, we propose a hybrid method that leverages the SegNeXt model for real-time applications, encompassing both ego-motion estimation and ground point selection. Our methodology incorporates dy-namic object masks to eliminate unstable features and employs ground plane masks for meticulous triangulation. Furthermore, we exploit Geometry-constraint to delineate road regions for scale recovery. The integration of this approach with the mo-nocular version of ORB-SLAM3 culminates in the accurate esti-mation of a road model, a pivotal component in our scale recov-ery process. Rigorous experiments, conducted on the KITTI da-taset, systematically compare our method with existing monocu-lar visual odometry algorithms and contemporary scale recovery methodologies. The results undeniably confirm the superior ef-fectiveness of our approach, surpassing state-of-the-art visual odometry algorithms. Our source code is available at https://git this http URL.

Title: DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models

• Authors: Ruizhe Chen, Wenhao Chai, Zhifei Yang, Xiaotian Zhang, Joey Tianyi Zhou, Tony Quek, Soujanya Poria, Zuozhu Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04240
• Pdf URL: https://arxiv.org/pdf/2503.04240
• Copy Paste: [[2503.04240]] DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models(https://arxiv.org/abs/2503.04240)
• Keywords: diffusion, large language model
• Abstract: Inference-time alignment provides an efficient alternative for aligning LLMs with humans. However, these approaches still face challenges, such as limited scalability due to policy-specific value functions and latency during the inference phase. In this paper, we propose a novel approach, Diffusion-styled Preference Optimization (\model), which provides an efficient and policy-agnostic solution for aligning LLMs with humans. By directly performing alignment at sentence level, \model~avoids the time latency associated with token-level generation. Designed as a plug-and-play module, \model~can be seamlessly integrated with various base models to enhance their alignment. Extensive experiments on AlpacaEval 2, MT-bench, and HH-RLHF demonstrate that \model~achieves superior alignment performance across various settings, achieving a favorable trade-off between alignment quality and inference-time latency. Furthermore, \model~demonstrates model-agnostic scalability, significantly improving the performance of large models such as Llama-3-70B.

Title: How to Mitigate Overfitting in Weak-to-strong Generalization?

• Authors: Junhao Shi, Qinyuan Cheng, Zhaoye Fei, Yining Zheng, Qipeng Guo, Xipeng Qiu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04249
• Pdf URL: https://arxiv.org/pdf/2503.04249
• Copy Paste: [[2503.04249]] How to Mitigate Overfitting in Weak-to-strong Generalization?(https://arxiv.org/abs/2503.04249)
• Keywords: large language model
• Abstract: Aligning powerful AI models on tasks that surpass human evaluation capabilities is the central problem of \textbf{superalignment}. To address this problem, weak-to-strong generalization aims to elicit the capabilities of strong models through weak supervisors and ensure that the behavior of strong models aligns with the intentions of weak supervisors without unsafe behaviors such as deception. Although weak-to-strong generalization exhibiting certain generalization capabilities, strong models exhibit significant overfitting in weak-to-strong generalization: Due to the strong fit ability of strong models, erroneous labels from weak supervisors may lead to overfitting in strong models. In addition, simply filtering out incorrect labels may lead to a degeneration in question quality, resulting in a weak generalization ability of strong models on hard questions. To mitigate overfitting in weak-to-strong generalization, we propose a two-stage framework that simultaneously improves the quality of supervision signals and the quality of input questions. Experimental results in three series of large language models and two mathematical benchmarks demonstrate that our framework significantly improves PGR compared to naive weak-to-strong generalization, even achieving up to 100\% PGR on some models.

Title: An Egocentric Vision-Language Model based Portable Real-time Smart Assistant

• Authors: Yifei Huang, Jilan Xu, Baoqi Pei, Yuping He, Guo Chen, Mingfang Zhang, Lijin Yang, Zheng Nie, Jinyao Liu, Guoshun Fan, Dechen Lin, Fang Fang, Kunpeng Li, Chang Yuan, Xinyuan Chen, Yaohui Wang, Yali Wang, Yu Qiao, Limin Wang
• Subjects: cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.04250
• Pdf URL: https://arxiv.org/pdf/2503.04250
• Copy Paste: [[2503.04250]] An Egocentric Vision-Language Model based Portable Real-time Smart Assistant(https://arxiv.org/abs/2503.04250)
• Keywords: large language model
• Abstract: We present Vinci, a vision-language system designed to provide real-time, comprehensive AI assistance on portable devices. At its core, Vinci leverages EgoVideo-VL, a novel model that integrates an egocentric vision foundation model with a large language model (LLM), enabling advanced functionalities such as scene understanding, temporal grounding, video summarization, and future planning. To enhance its utility, Vinci incorporates a memory module for processing long video streams in real time while retaining contextual history, a generation module for producing visual action demonstrations, and a retrieval module that bridges egocentric and third-person perspectives to provide relevant how-to videos for skill acquisition. Unlike existing systems that often depend on specialized hardware, Vinci is hardware-agnostic, supporting deployment across a wide range of devices, including smartphones and wearable cameras. In our experiments, we first demonstrate the superior performance of EgoVideo-VL on multiple public benchmarks, showcasing its vision-language reasoning and contextual understanding capabilities. We then conduct a series of user studies to evaluate the real-world effectiveness of Vinci, highlighting its adaptability and usability in diverse scenarios. We hope Vinci can establish a new framework for portable, real-time egocentric AI systems, empowering users with contextual and actionable insights. Including the frontend, backend, and models, all codes of Vinci are available at this https URL.

Title: Knowledge Retention for Continual Model-Based Reinforcement Learning

• Authors: Yixiang Sun, Haotian Fu, Michael Littman, George Konidaris
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04256
• Pdf URL: https://arxiv.org/pdf/2503.04256
• Copy Paste: [[2503.04256]] Knowledge Retention for Continual Model-Based Reinforcement Learning(https://arxiv.org/abs/2503.04256)
• Keywords: generative
• Abstract: We propose DRAGO, a novel approach for continual model-based reinforcement learning aimed at improving the incremental development of world models across a sequence of tasks that differ in their reward functions but not the state space or dynamics. DRAGO comprises two key components: Synthetic Experience Rehearsal, which leverages generative models to create synthetic experiences from past tasks, allowing the agent to reinforce previously learned dynamics without storing data, and Regaining Memories Through Exploration, which introduces an intrinsic reward mechanism to guide the agent toward revisiting relevant states from prior tasks. Together, these components enable the agent to maintain a comprehensive and continually developing world model, facilitating more effective learning and adaptation across diverse environments. Empirical evaluations demonstrate that DRAGO is able to preserve knowledge across tasks, achieving superior performance in various continual learning scenarios.

Title: How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects

• Authors: Wonkwang Lee, Jongwon Jeong, Taehong Moon, Hyeon-Jong Kim, Jaehyeon Kim, Gunhee Kim, Byeong-Uk Lee
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04257
• Pdf URL: https://arxiv.org/pdf/2503.04257
• Copy Paste: [[2503.04257]] How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects(https://arxiv.org/abs/2503.04257)
• Keywords: diffusion
• Abstract: Motion synthesis for diverse object categories holds great potential for 3D content creation but remains underexplored due to two key challenges: (1) the lack of comprehensive motion datasets that include a wide range of high-quality motions and annotations, and (2) the absence of methods capable of handling heterogeneous skeletal templates from diverse objects. To address these challenges, we contribute the following: First, we augment the Truebones Zoo dataset, a high-quality animal motion dataset covering over 70 species, by annotating it with detailed text descriptions, making it suitable for text-based motion synthesis. Second, we introduce rig augmentation techniques that generate diverse motion data while preserving consistent dynamics, enabling models to adapt to various skeletal configurations. Finally, we redesign existing motion diffusion models to dynamically adapt to arbitrary skeletal templates, enabling motion synthesis for a diverse range of objects with varying structures. Experiments show that our method learns to generate high-fidelity motions from textual descriptions for diverse and even unseen objects, setting a strong foundation for motion synthesis across diverse object categories and skeletal templates. Qualitative results are available on this link: this http URL

Title: Qualitative In-Depth Analysis of GDPR Data Subject Access Requests and Responses from Major Online Services

• Authors: Daniela Pöhn, Nils Gruschka
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04259
• Pdf URL: https://arxiv.org/pdf/2503.04259
• Copy Paste: [[2503.04259]] Qualitative In-Depth Analysis of GDPR Data Subject Access Requests and Responses from Major Online Services(https://arxiv.org/abs/2503.04259)
• Keywords: privacy, protect
• Abstract: The European General Data Protection Regulation (GDPR) grants European users the right to access their data processed and stored by organizations. Although the GDPR contains requirements for data processing organizations (e.g., understandable data provided within a month), it leaves much flexibility. In-depth research on how online services handle data subject access request is sparse. Specifically, it is unclear whether online services comply with the individual GDPR requirements, if the privacy policies and the data subject access responses are coherent, and how the responses change over time. To answer these questions, we perform a qualitative structured review of the processes and data exports of significant online services to (1) analyze the data received in 2023 in detail, (2) compare the data exports with the privacy policies, and (3) compare the data exports from November 2018 and November 2023. The study concludes that the quality of data subject access responses varies among the analyzed services, and none fulfills all requirements completely.

Title: DTL: Data Tumbling Layer. A Composable Unlinkability for Smart Contracts

• Authors: Mohsen Minaei, Pedro Moreno-Sanchez, Zhiyong Fang, Srinivasan Raghuraman, Navid Alamati, Panagiotis Chatzigiannis, Ranjit Kumaresan, Duc V. Le
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04260
• Pdf URL: https://arxiv.org/pdf/2503.04260
• Copy Paste: [[2503.04260]] DTL: Data Tumbling Layer. A Composable Unlinkability for Smart Contracts(https://arxiv.org/abs/2503.04260)
• Keywords: security, privacy
• Abstract: We propose Data Tumbling Layer (DTL), a cryptographic scheme for non-interactive data tumbling. The core concept is to enable users to commit to specific data and subsequently re-use to the encrypted version of these data across different applications while removing the link to the previous data commit action. We define the following security and privacy notions for DTL: (i) no one-more redemption: a malicious user cannot redeem and use the same data more than the number of times they have committed the data; (ii) theft prevention: a malicious user cannot use data that has not been committed by them; (iii) non-slanderabilty: a malicious user cannot prevent an honest user from using their previously committed data; and (iv) unlinkability: a malicious user cannot link tainted data from an honest user to the corresponding data after it has been tumbled. To showcase the practicality of DTL, we use DTL to realize applications for (a) unlinkable fixed-amount payments; (b) unlinkable and confidential payments for variable amounts; (c) unlinkable weighted voting protocol. Finally, we implemented and evaluated all the proposed applications. For the unlinkable and confidential payment application, a user can initiate such a transaction in less than $1.5$s on a personal laptop. In terms of on-chain verification, the gas cost is less than $1.8$ million.

Title: Frequency Hopping Synchronization by Reinforcement Learning for Satellite Communication System

• Authors: Inkyu Kim, Sangkeum Lee, Haechan Jeong, Sarvar Hussain Nengroo, Dongsoo Har
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04266
• Pdf URL: https://arxiv.org/pdf/2503.04266
• Copy Paste: [[2503.04266]] Frequency Hopping Synchronization by Reinforcement Learning for Satellite Communication System(https://arxiv.org/abs/2503.04266)
• Keywords: security, robust
• Abstract: Satellite communication systems (SCSs) used for tactical purposes require robust security and anti-jamming capabilities, making frequency hopping (FH) a powerful option. However, the current FH systems face challenges due to significant interference from other devices and the considerable path loss inherent in satellite communication. This misalignment leads to inefficient synchronization, crucial for maintaining reliable communication. Traditional methods, such as those employing long short-term memory (LSTM) networks, have made improvements, but they still struggle in dynamic conditions of satellite environments. This paper presents a novel method for synchronizing FH signals in tactical SCSs by combining serial search and reinforcement learning to achieve coarse and fine acquisition, respectively. The mathematical analysis and simulation results demonstrate that the proposed method reduces the average number of hops required for synchronization by 58.17% and mean squared error (MSE) of the uplink hop timing estimation by 76.95%, as compared to the conventional serial search method. Comparing with the early late gate synchronization method based on serial search and use of LSTM network, the average number of hops for synchronization is reduced by 12.24% and the MSE by 18.5%.

Title: ControlFill: Spatially Adjustable Image Inpainting from Prompt Learning

• Authors: Boseong Jeon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04268
• Pdf URL: https://arxiv.org/pdf/2503.04268
• Copy Paste: [[2503.04268]] ControlFill: Spatially Adjustable Image Inpainting from Prompt Learning(https://arxiv.org/abs/2503.04268)
• Keywords: diffusion
• Abstract: In this report, I present an inpainting framework named \textit{ControlFill}, which involves training two distinct prompts: one for generating plausible objects within a designated mask (\textit{creation}) and another for filling the region by extending the background (\textit{removal}). During the inference stage, these learned embeddings guide a diffusion network that operates without requiring heavy text encoders. By adjusting the relative significance of the two prompts and employing classifier-free guidance, users can control the intensity of removal or creation. Furthermore, I introduce a method to spatially vary the intensity of guidance by assigning different scales to individual pixels.

Title: Got Ya! -- Sensors for Identity Management Specific Security Situational Awareness

• Authors: Daniela Pöhn, Heiner Lüken
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04274
• Pdf URL: https://arxiv.org/pdf/2503.04274
• Copy Paste: [[2503.04274]] Got Ya! -- Sensors for Identity Management Specific Security Situational Awareness(https://arxiv.org/abs/2503.04274)
• Keywords: security, attack
• Abstract: Security situational awareness refers to identifying, mitigating, and preventing digital cyber threats by gathering information to understand the current situation. With awareness, the basis for decisions is present, particularly in complex situations. However, while logging can track the successful login into a system, it typically cannot determine if the login was performed by the user assigned to the account. An account takeover, for example, by a successful phishing attack, can be used as an entry into an organization's network. All identities within an organization are managed in an identity management system. Thereby, these systems are an interesting goal for malicious actors. Even within identity management systems, it is difficult to differentiate legitimate from malicious actions. We propose a security situational awareness approach specifically to identity management. We focus on protocol-specifics and identity-related sources in a general concept before providing the example of the protocol OAuth with a proof-of-concept implementation.

Title: A General Framework for Scalable UE-AP Association in User-Centric Cell-Free Massive MIMO based on Recurrent Neural Networks

• Authors: Giovanni Di Gennaro, Amedeo Buonanno, Gianmarco Romano, Stefano Buzzi, Francesco A. N. Palmieri
• Subjects: cs.LG, eess.SP, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.04278
• Pdf URL: https://arxiv.org/pdf/2503.04278
• Copy Paste: [[2503.04278]] A General Framework for Scalable UE-AP Association in User-Centric Cell-Free Massive MIMO based on Recurrent Neural Networks(https://arxiv.org/abs/2503.04278)
• Keywords: robust
• Abstract: This study addresses the challenge of access point (AP) and user equipment (UE) association in cell-free massive MIMO networks. It introduces a deep learning algorithm leveraging Bidirectional Long Short-Term Memory cells and a hybrid probabilistic methodology for weight updating. This approach enhances scalability by adapting to variations in the number of UEs without requiring retraining. Additionally, the study presents a training methodology that improves scalability not only with respect to the number of UEs but also to the number of APs. Furthermore, a variant of the proposed AP-UE algorithm ensures robustness against pilot contamination effects, a critical issue arising from pilot reuse in channel estimation. Extensive numerical results validate the effectiveness and adaptability of the proposed methods, demonstrating their superiority over widely used heuristic alternatives.

Title: Explainable AI in Time-Sensitive Scenarios: Prefetched Offline Explanation Model

• Authors: Fabio Michele Russo, Carlo Metta, Anna Monreale, Salvatore Rinzivillo, Fabio Pinelli
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04283
• Pdf URL: https://arxiv.org/pdf/2503.04283
• Copy Paste: [[2503.04283]] Explainable AI in Time-Sensitive Scenarios: Prefetched Offline Explanation Model(https://arxiv.org/abs/2503.04283)
• Keywords: explainability
• Abstract: As predictive machine learning models become increasingly adopted and advanced, their role has evolved from merely predicting outcomes to actively shaping them. This evolution has underscored the importance of Trustworthy AI, highlighting the necessity to extend our focus beyond mere accuracy and toward a comprehensive understanding of these models' behaviors within the specific contexts of their applications. To further progress in explainability, we introduce Poem, Prefetched Offline Explanation Model, a model-agnostic, local explainability algorithm for image data. The algorithm generates exemplars, counterexemplars and saliency maps to provide quick and effective explanations suitable for time-sensitive scenarios. Leveraging an existing local algorithm, \poem{} infers factual and counterfactual rules from data to create illustrative examples and opposite scenarios with an enhanced stability by design. A novel mechanism then matches incoming test points with an explanation base and produces diverse exemplars, informative saliency maps and believable counterexemplars. Experimental results indicate that Poem outperforms its predecessor Abele in speed and ability to generate more nuanced and varied exemplars alongside more insightful saliency maps and valuable counterexemplars.

Title: A Study on Malicious Browser Extensions in 2025

• Authors: Shreya Singh, Gaurav Varshney, Tarun Kumar Singh, Vidhi Mishra
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04292
• Pdf URL: https://arxiv.org/pdf/2503.04292
• Copy Paste: [[2503.04292]] A Study on Malicious Browser Extensions in 2025(https://arxiv.org/abs/2503.04292)
• Keywords: security, attack
• Abstract: Browser extensions are additional tools developed by third parties that integrate with web browsers to extend their functionality beyond standard capabilities. However, the browser extension platform is increasingly being exploited by hackers to launch sophisticated cyber threats. These threats encompass a wide range of malicious activities, including but not limited to phishing, spying, Distributed Denial of Service (DDoS) attacks, email spamming, affiliate fraud, malvertising, and payment fraud. This paper examines the evolving threat landscape of malicious browser extensions in 2025, focusing on Mozilla Firefox and Chrome. Our research successfully bypassed security mechanisms of Firefox and Chrome, demonstrating that malicious extensions can still be developed, published, and executed within the Mozilla Add-ons Store and Chrome Web Store. These findings highlight the persisting weaknesses in browser's vetting process and security framework. It provides insights into the risks associated with browser extensions, helping users understand these threats while aiding the industry in developing controls and countermeasures to defend against such attacks. All experiments discussed in this paper were conducted in a controlled laboratory environment by the researchers, adhering to proper ethical guidelines. The sole purpose of these experiments is to raise security awareness among the industry, research community, and the general public.

Title: Malware Detection at the Edge with Lightweight LLMs: A Performance Evaluation

• Authors: Christian Rondanini, Barbara Carminati, Elena Ferrari, Antonio Gaudiano, Ashish Kundu
• Subjects: cs.CR, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.04302
• Pdf URL: https://arxiv.org/pdf/2503.04302
• Copy Paste: [[2503.04302]] Malware Detection at the Edge with Lightweight LLMs: A Performance Evaluation(https://arxiv.org/abs/2503.04302)
• Keywords: attack, large language model
• Abstract: The rapid evolution of malware attacks calls for the development of innovative detection methods, especially in resource-constrained edge computing. Traditional detection techniques struggle to keep up with modern malware's sophistication and adaptability, prompting a shift towards advanced methodologies like those leveraging Large Language Models (LLMs) for enhanced malware detection. However, deploying LLMs for malware detection directly at edge devices raises several challenges, including ensuring accuracy in constrained environments and addressing edge devices' energy and computational limits. To tackle these challenges, this paper proposes an architecture leveraging lightweight LLMs' strengths while addressing limitations like reduced accuracy and insufficient computational power. To evaluate the effectiveness of the proposed lightweight LLM-based approach for edge computing, we perform an extensive experimental evaluation using several state-of-the-art lightweight LLMs. We test them with several publicly available datasets specifically designed for edge and IoT scenarios and different edge nodes with varying computational power and characteristics.

Title: Approaches to Quantum Remote Memory Attestation

• Authors: Jesse Laeuchli, Rolando Trujillo Rasua
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04311
• Pdf URL: https://arxiv.org/pdf/2503.04311
• Copy Paste: [[2503.04311]] Approaches to Quantum Remote Memory Attestation(https://arxiv.org/abs/2503.04311)
• Keywords: security
• Abstract: In this article we uncover flaws and pitfalls of a quantum-based remote memory attestation procedure for Internet-of-Things devices. We also show limitations of quantum memory that suggests the attestation problem for quantum memory is fundamentally different to the attestation problem for classical memory, even when the devices can perform quantum computation. The identified problems are of interest for quantum-based security protocol designers in general, particularly those dealing with corrupt devices. Finally, we make use of the lessons learned to design a quantum-based attestation system for classical memory with improved communication efficiency and security.

Title: S2Gaussian: Sparse-View Super-Resolution 3D Gaussian Splatting

• Authors: Yecong Wan, Mingwen Shao, Yuanshuo Cheng, Wangmeng Zuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04314
• Pdf URL: https://arxiv.org/pdf/2503.04314
• Copy Paste: [[2503.04314]] S2Gaussian: Sparse-View Super-Resolution 3D Gaussian Splatting(https://arxiv.org/abs/2503.04314)
• Keywords: robust
• Abstract: In this paper, we aim ambitiously for a realistic yet challenging problem, namely, how to reconstruct high-quality 3D scenes from sparse low-resolution views that simultaneously suffer from deficient perspectives and clarity. Whereas existing methods only deal with either sparse views or low-resolution observations, they fail to handle such hybrid and complicated scenarios. To this end, we propose a novel Sparse-view Super-resolution 3D Gaussian Splatting framework, dubbed S2Gaussian, that can reconstruct structure-accurate and detail-faithful 3D scenes with only sparse and low-resolution views. The S2Gaussian operates in a two-stage fashion. In the first stage, we initially optimize a low-resolution Gaussian representation with depth regularization and densify it to initialize the high-resolution Gaussians through a tailored Gaussian Shuffle Split operation. In the second stage, we refine the high-resolution Gaussians with the super-resolved images generated from both original sparse views and pseudo-views rendered by the low-resolution Gaussians. In which a customized blur-free inconsistency modeling scheme and a 3D robust optimization strategy are elaborately designed to mitigate multi-view inconsistency and eliminate erroneous updates caused by imperfect supervision. Extensive experiments demonstrate superior results and in particular establishing new state-of-the-art performances with more consistent geometry and finer details.

Title: Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization

• Authors: Shuang Liu, Yihan Wang, Yifan Zhu, Yibo Miao, Xiao-Shan Gao
• Subjects: cs.LG, cs.AI, math.ST
• Abstract URL: https://arxiv.org/abs/2503.04315
• Pdf URL: https://arxiv.org/pdf/2503.04315
• Copy Paste: [[2503.04315]] Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization(https://arxiv.org/abs/2503.04315)
• Keywords: robust
• Abstract: Wasserstein distributionally robust optimization (WDRO) optimizes against worst-case distributional shifts within a specified uncertainty set, leading to enhanced generalization on unseen adversarial examples, compared to standard adversarial training which focuses on pointwise adversarial perturbations. However, WDRO still suffers fundamentally from the robust overfitting problem, as it does not consider statistical error. We address this gap by proposing a novel robust optimization framework under a new uncertainty set for adversarial noise via Wasserstein distance and statistical error via Kullback-Leibler divergence, called the Statistically Robust WDRO. We establish a robust generalization bound for the new optimization framework, implying that out-of-distribution adversarial performance is at least as good as the statistically robust training loss with high probability. Furthermore, we derive conditions under which Stackelberg and Nash equilibria exist between the learner and the adversary, giving an optimal robust model in certain sense. Finally, through extensive experiments, we demonstrate that our method significantly mitigates robust overfitting and enhances robustness within the framework of WDRO.

Title: InFL-UX: A Toolkit for Web-Based Interactive Federated Learning

• Authors: Tim Maurer, Abdulrahman Mohamed Selim, Hasan Md Tusfiqur Alam, Matthias Eiletz, Michael Barz, Daniel Sonntag
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.04318
• Pdf URL: https://arxiv.org/pdf/2503.04318
• Copy Paste: [[2503.04318]] InFL-UX: A Toolkit for Web-Based Interactive Federated Learning(https://arxiv.org/abs/2503.04318)
• Keywords: federate
• Abstract: This paper presents InFL-UX, an interactive, proof-of-concept browser-based Federated Learning (FL) toolkit designed to integrate user contributions seamlessly into the machine learning (ML) workflow. InFL-UX enables users across multiple devices to upload datasets, define classes, and collaboratively train classification models directly in the browser using modern web technologies. Unlike traditional FL toolkits, which often focus on backend simulations, InFL-UX provides a simple user interface for researchers to explore how users interact with and contribute to FL systems in real-world, interactive settings. By prioritising usability and decentralised model training, InFL-UX bridges the gap between FL and Interactive Machine Learning (IML), empowering non-technical users to actively participate in ML classification tasks.

Title: A Modular Pipeline for 3D Object Tracking Using RGB Cameras

• Authors: Lars Bredereke, Yale Hartmann, Tanja Schultz
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04322
• Pdf URL: https://arxiv.org/pdf/2503.04322
• Copy Paste: [[2503.04322]] A Modular Pipeline for 3D Object Tracking Using RGB Cameras(https://arxiv.org/abs/2503.04322)
• Keywords: robust
• Abstract: Object tracking is a key challenge of computer vision with various applications that all require different architectures. Most tracking systems have limitations such as constraining all movement to a 2D plane and they often track only one object. In this paper, we present a new modular pipeline that calculates 3D trajectories of multiple objects. It is adaptable to various settings where multiple time-synced and stationary cameras record moving objects, using off the shelf webcams. Our pipeline was tested on the Table Setting Dataset, where participants are recorded with various sensors as they set a table with tableware objects. We need to track these manipulated objects, using 6 rgb webcams. Challenges include: Detecting small objects in 9.874.699 camera frames, determining camera poses, discriminating between nearby and overlapping objects, temporary occlusions, and finally calculating a 3D trajectory using the right subset of an average of 11.12.456 pixel coordinates per 3-minute trial. We implement a robust pipeline that results in accurate trajectories with covariance of x,y,z-position as a confidence metric. It deals dynamically with appearing and disappearing objects, instantiating new Extended Kalman Filters. It scales to hundreds of table-setting trials with very little human annotation input, even with the camera poses of each trial unknown. The code is available at this https URL

Title: Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples

• Authors: Tadej Škvorc, Marko Robnik-Šikonja
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04328
• Pdf URL: https://arxiv.org/pdf/2503.04328
• Copy Paste: [[2503.04328]] Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples(https://arxiv.org/abs/2503.04328)
• Keywords: transformer, large language model
• Abstract: Many less-resourced languages struggle with a lack of large, task-specific datasets that are required for solving relevant tasks with modern transformer-based large language models (LLMs). On the other hand, many linguistic resources, such as dictionaries, are rarely used in this context despite their large information contents. We show how LLMs can be used to extend existing language resources in less-resourced languages for two important tasks: word-sense disambiguation (WSD) and word-sense induction (WSI). We approach the two tasks through the related but much more accessible word-in-context (WiC) task where, given a pair of sentences and a target word, a classification model is tasked with predicting whether the sense of a given word differs between sentences. We demonstrate that a well-trained model for this task can distinguish between different word senses and can be adapted to solve the WSD and WSI tasks. The advantage of using the WiC task, instead of directly predicting senses, is that the WiC task does not need pre-constructed sense inventories with a sufficient number of examples for each sense, which are rarely available in less-resourced languages. We show that sentence pairs for the WiC task can be successfully generated from dictionary examples using LLMs. The resulting prediction models outperform existing models on WiC, WSD, and WSI tasks. We demonstrate our methodology on the Slovene language, where a monolingual dictionary is available, but word-sense resources are tiny.

Title: The Challenge of Identifying the Origin of Black-Box Large Language Models

• Authors: Ziqing Yang, Yixin Wu, Yun Shen, Wei Dai, Michael Backes, Yang Zhang
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04332
• Pdf URL: https://arxiv.org/pdf/2503.04332
• Copy Paste: [[2503.04332]] The Challenge of Identifying the Origin of Black-Box Large Language Models(https://arxiv.org/abs/2503.04332)
• Keywords: fair, large language model
• Abstract: The tremendous commercial potential of large language models (LLMs) has heightened concerns about their unauthorized use. Third parties can customize LLMs through fine-tuning and offer only black-box API access, effectively concealing unauthorized usage and complicating external auditing processes. This practice not only exacerbates unfair competition, but also violates licensing agreements. In response, identifying the origin of black-box LLMs is an intrinsic solution to this issue. In this paper, we first reveal the limitations of state-of-the-art passive and proactive identification methods with experiments on 30 LLMs and two real-world black-box APIs. Then, we propose the proactive technique, PlugAE, which optimizes adversarial token embeddings in a continuous space and proactively plugs them into the LLM for tracing and identification. The experiments show that PlugAE can achieve substantial improvement in identifying fine-tuned derivatives. We further advocate for legal frameworks and regulations to better address the challenges posed by the unauthorized use of LLMs.

Title: LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding

• Authors: Shen Zhang, Yaning Tan, Siyuan Liang, Linze Li, Ge Wu, Yuhao Chen, Shuheng Li, Zhenyu Zhao, Caihua Chen, Jiajun Liang, Yao Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04344
• Pdf URL: https://arxiv.org/pdf/2503.04344
• Copy Paste: [[2503.04344]] LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding(https://arxiv.org/abs/2503.04344)
• Keywords: diffusion, transformer
• Abstract: Diffusion transformers(DiTs) struggle to generate images at resolutions higher than their training resolutions. The primary obstacle is that the explicit positional encodings(PE), such as RoPE, need extrapolation which degrades performance when the inference resolution differs from training. In this paper, we propose a Length-Extrapolatable Diffusion Transformer(LEDiT), a simple yet powerful architecture to overcome this limitation. LEDiT needs no explicit PEs, thereby avoiding extrapolation. The key innovations of LEDiT are introducing causal attention to implicitly impart global positional information to tokens, while enhancing locality to precisely distinguish adjacent tokens. Experiments on 256x256 and 512x512 ImageNet show that LEDiT can scale the inference resolution to 512x512 and 1024x1024, respectively, while achieving better image quality compared to current state-of-the-art length extrapolation methods(NTK-aware, YaRN). Moreover, LEDiT achieves strong extrapolation performance with just 100K steps of fine-tuning on a pretrained DiT, demonstrating its potential for integration into existing text-to-image DiTs.

Title: Large Language Models for Zero-shot Inference of Causal Structures in Biology

• Authors: Izzy Newsham, Luka Kovačević, Richard Moulange, Nan Rosemary Ke, Sach Mukherjee
• Subjects: cs.LG, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2503.04347
• Pdf URL: https://arxiv.org/pdf/2503.04347
• Copy Paste: [[2503.04347]] Large Language Models for Zero-shot Inference of Causal Structures in Biology(https://arxiv.org/abs/2503.04347)
• Keywords: large language model
• Abstract: Genes, proteins and other biological entities influence one another via causal molecular networks. Causal relationships in such networks are mediated by complex and diverse mechanisms, through latent variables, and are often specific to cellular context. It remains challenging to characterise such networks in practice. Here, we present a novel framework to evaluate large language models (LLMs) for zero-shot inference of causal relationships in biology. In particular, we systematically evaluate causal claims obtained from an LLM using real-world interventional data. This is done over one hundred variables and thousands of causal hypotheses. Furthermore, we consider several prompting and retrieval-augmentation strategies, including large, and potentially conflicting, collections of scientific articles. Our results show that with tailored augmentation and prompting, even relatively small LLMs can capture meaningful aspects of causal structure in biological systems. This supports the notion that LLMs could act as orchestration tools in biological discovery, by helping to distil current knowledge in ways amenable to downstream analysis. Our approach to assessing LLMs with respect to experimental data is relevant for a broad range of problems at the intersection of causal learning, LLMs and scientific discovery.

Title: Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling

• Authors: Zhenghua Wang, Yiran Ding, Changze Lv, Zhibo Xu, Tianlong Li, Tianyuan Shi, Xiaoqing Zheng, Xuanjing Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04355
• Pdf URL: https://arxiv.org/pdf/2503.04355
• Copy Paste: [[2503.04355]] Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling(https://arxiv.org/abs/2503.04355)
• Keywords: large language model
• Abstract: Although large language models (LLMs) have achieved significant progress in handling long-context inputs, they still suffer from the ``lost-in-the-middle'' problem, where crucial information in the middle of the context is often underrepresented or lost. Our extensive experiments reveal that this issue may arise from the rapid long-term decay in Rotary Position Embedding (RoPE). To address this problem, we propose a layer-specific positional encoding scaling method that assigns distinct scaling factors to each layer, slowing down the decay rate caused by RoPE to make the model pay more attention to the middle context. A specially designed genetic algorithm is employed to efficiently select the optimal scaling factors for each layer by incorporating Bezier curves to reduce the search space. Through comprehensive experimentation, we demonstrate that our method significantly alleviates the ``lost-in-the-middle'' problem. Our approach results in an average accuracy improvement of up to 20% on the Key-Value Retrieval dataset. Furthermore, we show that layer-specific interpolation, as opposed to uniform interpolation across all layers, enhances the model's extrapolation capabilities when combined with PI and Dynamic-NTK positional encoding schemes.

Title: scDD: Latent Codes Based scRNA-seq Dataset Distillation with Foundation Model Knowledge

• Authors: Zhen Yu, Jianan Han, Yang Liu, Qingchao Chen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04357
• Pdf URL: https://arxiv.org/pdf/2503.04357
• Copy Paste: [[2503.04357]] scDD: Latent Codes Based scRNA-seq Dataset Distillation with Foundation Model Knowledge(https://arxiv.org/abs/2503.04357)
• Keywords: diffusion
• Abstract: Single-cell RNA sequencing (scRNA-seq) technology has profiled hundreds of millions of human cells across organs, diseases, development and perturbations to date. However, the high-dimensional sparsity, batch effect noise, category imbalance, and ever-increasing data scale of the original sequencing data pose significant challenges for multi-center knowledge transfer, data fusion, and cross-validation between scRNA-seq datasets. To address these barriers, (1) we first propose a latent codes-based scRNA-seq dataset distillation framework named scDD, which transfers and distills foundation model knowledge and original dataset information into a compact latent space and generates synthetic scRNA-seq dataset by a generator to replace the original dataset. Then, (2) we propose a single-step conditional diffusion generator named SCDG, which perform single-step gradient back-propagation to help scDD optimize distillation quality and avoid gradient decay caused by multi-step back-propagation. Meanwhile, SCDG ensures the scRNA-seq data characteristics and inter-class discriminability of the synthetic dataset through flexible conditional control and generation quality assurance. Finally, we propose a comprehensive benchmark to evaluate the performance of scRNA-seq dataset distillation in different data analysis tasks. It is validated that our proposed method can achieve 7.61% absolute and 15.70% relative improvement over previous state-of-the-art methods on average task.

Title: Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators

• Authors: Jiayi Chang, Mingqi Gao, Xinyu Hu, Xiaojun Wan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04360
• Pdf URL: https://arxiv.org/pdf/2503.04360
• Copy Paste: [[2503.04360]] Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators(https://arxiv.org/abs/2503.04360)
• Keywords: attack
• Abstract: Previous research has shown that LLMs have potential in multilingual NLG evaluation tasks. However, existing research has not fully explored the differences in the evaluation capabilities of LLMs across different languages. To this end, this study provides a comprehensive analysis of the multilingual evaluation performance of 10 recent LLMs, spanning high-resource and low-resource languages through correlation analysis, perturbation attacks, and fine-tuning. We found that 1) excluding the reference answer from the prompt and using large-parameter LLM-based evaluators leads to better performance across various languages; 2) most LLM-based evaluators show a higher correlation with human judgments in high-resource languages than in low-resource languages; 3) in the languages where they are most sensitive to such attacks, they also tend to exhibit the highest correlation with human judgments; and 4) fine-tuning with data from a particular language yields a broadly consistent enhancement in the model's evaluation performance across diverse languages. Our findings highlight the imbalance in LLMs'evaluation capabilities across different languages and suggest that low-resource language scenarios deserve more attention.

Title: A Generalist Cross-Domain Molecular Learning Framework for Structure-Based Drug Discovery

• Authors: Yiheng Zhu, Mingyang Li, Junlong Liu, Kun Fu, Jiansheng Wu, Qiuyi Li, Mingze Yin, Jieping Ye, Jian Wu, Zheng Wang
• Subjects: cs.LG, cs.AI, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2503.04362
• Pdf URL: https://arxiv.org/pdf/2503.04362
• Copy Paste: [[2503.04362]] A Generalist Cross-Domain Molecular Learning Framework for Structure-Based Drug Discovery(https://arxiv.org/abs/2503.04362)
• Keywords: transformer
• Abstract: Structure-based drug discovery (SBDD) is a systematic scientific process that develops new drugs by leveraging the detailed physical structure of the target protein. Recent advancements in pre-trained models for biomolecules have demonstrated remarkable success across various biochemical applications, including drug discovery and protein engineering. However, in most approaches, the pre-trained models primarily focus on the characteristics of either small molecules or proteins, without delving into their binding interactions which are essential cross-domain relationships pivotal to SBDD. To fill this gap, we propose a general-purpose foundation model named BIT (an abbreviation for Biomolecular Interaction Transformer), which is capable of encoding a range of biochemical entities, including small molecules, proteins, and protein-ligand complexes, as well as various data formats, encompassing both 2D and 3D structures. Specifically, we introduce Mixture-of-Domain-Experts (MoDE) to handle the biomolecules from diverse biochemical domains and Mixture-of-Structure-Experts (MoSE) to capture positional dependencies in the molecular structures. The proposed mixture-of-experts approach enables BIT to achieve both deep fusion and domain-specific encoding, effectively capturing fine-grained molecular interactions within protein-ligand complexes. Then, we perform cross-domain pre-training on the shared Transformer backbone via several unified self-supervised denoising tasks. Experimental results on various benchmarks demonstrate that BIT achieves exceptional performance in downstream tasks, including binding affinity prediction, structure-based virtual screening, and molecular property prediction.

Title: Causally Reliable Concept Bottleneck Models

• Authors: Giovanni De Felice, Arianna Casanova Flores, Francesco De Santis, Silvia Santini, Johannes Schneider, Pietro Barbiero, Alberto Termine
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04363
• Pdf URL: https://arxiv.org/pdf/2503.04363
• Copy Paste: [[2503.04363]] Causally Reliable Concept Bottleneck Models(https://arxiv.org/abs/2503.04363)
• Keywords: fair, explainability
• Abstract: Concept-based models are an emerging paradigm in deep learning that constrains the inference process to operate through human-interpretable concepts, facilitating explainability and human interaction. However, these architectures, on par with popular opaque neural models, fail to account for the true causal mechanisms underlying the target phenomena represented in the data. This hampers their ability to support causal reasoning tasks, limits out-of-distribution generalization, and hinders the implementation of fairness constraints. To overcome these issues, we propose \emph{Causally reliable Concept Bottleneck Models} (C$^2$BMs), a class of concept-based architectures that enforce reasoning through a bottleneck of concepts structured according to a model of the real-world causal mechanisms. We also introduce a pipeline to automatically learn this structure from observational data and \emph{unstructured} background knowledge (e.g., scientific literature). Experimental evidence suggest that C$^2$BM are more interpretable, causally reliable, and improve responsiveness to interventions w.r.t. standard opaque and concept-based models, while maintaining their accuracy.

Title: Lost in Literalism: How Supervised Training Shapes Translationese in LLMs

• Authors: Yafu Li, Ronghao Zhang, Zhilin Wang, Huajian Zhang, Leyang Cui, Yongjing Yin, Tong Xiao, Yue Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04369
• Pdf URL: https://arxiv.org/pdf/2503.04369
• Copy Paste: [[2503.04369]] Lost in Literalism: How Supervised Training Shapes Translationese in LLMs(https://arxiv.org/abs/2503.04369)
• Keywords: large language model
• Abstract: Large language models (LLMs) have achieved remarkable success in machine translation, demonstrating impressive performance across diverse languages. However, translationese, characterized by overly literal and unnatural translations, remains a persistent challenge in LLM-based translation systems. Despite their pre-training on vast corpora of natural utterances, LLMs exhibit translationese errors and generate unexpected unnatural translations, stemming from biases introduced during supervised fine-tuning (SFT). In this work, we systematically evaluate the prevalence of translationese in LLM-generated translations and investigate its roots during supervised training. We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances. Empirical evaluations demonstrate that these approaches significantly reduce translationese while improving translation naturalness, validated by human evaluations and automatic metrics. Our findings highlight the need for training-aware adjustments to optimize LLM translation outputs, paving the way for more fluent and target-language-consistent translations. We release the data and code at this https URL.

Title: FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique

• Authors: Antonio Guillén-Teruel (1), Marcos Caracena (1), Jose A. Pardo (1), Fernando de-la-Gándara (1), José Palma (1), Juan A. Botía (1,2) ((1) Departamento de Ingeniería de la Información y Las Comunicaciones, Universidad de Murcia, Murcia, 30100, Murcia, Spain, (2) Department of Neurodegenerative Disease, Institute of Neurology, University College London, London, WC1N 3BG, UK.)
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04370
• Pdf URL: https://arxiv.org/pdf/2503.04370
• Copy Paste: [[2503.04370]] FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique(https://arxiv.org/abs/2503.04370)
• Keywords: robust
• Abstract: This research addresses the challenges of handling unbalanced datasets for binary classification tasks. In such scenarios, standard evaluation metrics are often biased by the disproportionate representation of the minority class. Conducting experiments across seven datasets, we uncovered inconsistencies in evaluation metrics when determining the model that outperforms others for each binary classification problem. This justifies the need for a metric that provides a more consistent and unbiased evaluation across unbalanced datasets, thereby supporting robust model selection. To mitigate this problem, we propose a novel metric, the Unbiased Integration Coefficients (UIC), which exhibits significantly reduced bias ($p < 10^{-4}$) towards the minority class compared to conventional metrics. The UIC is constructed by aggregating existing metrics while penalising those more prone to imbalance. In addition, we introduce the Identical Partitions for Imbalance Problems (IPIP) algorithm for imbalanced ML problems, an ensemble-based approach. Our experimental results show that IPIP outperforms other baseline imbalance-aware approaches using Random Forest and Logistic Regression models in three out of seven datasets as assessed by the UIC metric, demonstrating its effectiveness in addressing imbalanced data challenges in binary classification tasks. This new framework for dealing with imbalanced datasets is materialized in the FILM (Framework for Imbalanced Learning Machines) R Package, accessible at this https URL.

Title: How can representation dimension dominate structurally pruned LLMs?

• Authors: Mingxue Xu, Lisa Alazraki, Danilo P. Mandic
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04377
• Pdf URL: https://arxiv.org/pdf/2503.04377
• Copy Paste: [[2503.04377]] How can representation dimension dominate structurally pruned LLMs?(https://arxiv.org/abs/2503.04377)
• Keywords: extraction, transformer
• Abstract: Pruning assumes a subnetwork exists in the original deep neural network, which can achieve comparative model performance with less computation than the original. However, it is unclear how the model performance varies with the different subnetwork extractions. In this paper, we choose the representation dimension (or embedding dimension, model dimension, the dimension of the residual stream in the relevant literature) as the entry point to this issue. We investigate the linear transformations in the LLM transformer blocks and consider a specific structured pruning approach, SliceGPT, to extract the subnetworks of different representation dimensions. We mechanistically analyse the activation flow during the model forward passes, and find the representation dimension dominates the linear transformations, model predictions, and, finally, the model performance. Explicit analytical relations are given to calculate the pruned model performance (perplexity and accuracy) without actual evaluation, and are empirically validated with Llama-3-8B-Instruct and Phi-3-mini-4k-Instruct.

Title: TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge

• Authors: Cheng-Han Chiang, Hung-yi Lee, Michal Lukasik
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04381
• Pdf URL: https://arxiv.org/pdf/2503.04381
• Copy Paste: [[2503.04381]] TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge(https://arxiv.org/abs/2503.04381)
• Keywords: large language model
• Abstract: The LLM-as-a-judge paradigm uses large language models (LLMs) for automated text evaluation, where a numerical assessment is assigned by an LLM to the input text following scoring rubrics. Existing methods for LLM-as-a-judge use cross-entropy (CE) loss for fine-tuning, which neglects the numeric nature of score prediction. Recent work addresses numerical prediction limitations of LLM fine-tuning through regression-aware fine-tuning, which, however, does not consider chain-of-thought (CoT) reasoning for score prediction. In this paper, we introduce TRACT (Two-stage Regression-Aware fine-tuning with CoT), a method combining CoT reasoning with regression-aware training. TRACT consists of two stages: first, seed LLM is fine-tuned to generate CoTs, which serve as supervision for the second stage fine-tuning. The training objective of TRACT combines the CE loss for learning the CoT reasoning capabilities, and the regression-aware loss for the score prediction. Experiments across four LLM-as-a-judge datasets and two LLMs show that TRACT significantly outperforms existing methods. Extensive ablation studies validate the importance of each component in TRACT.

Title: Scale-Invariant Adversarial Attack against Arbitrary-scale Super-resolution

• Authors: Yihao Huang, Xin Luo, Qing Guo, Felix Juefei-Xu, Xiaojun Jia, Weikai Miao, Geguang Pu, Yang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04385
• Pdf URL: https://arxiv.org/pdf/2503.04385
• Copy Paste: [[2503.04385]] Scale-Invariant Adversarial Attack against Arbitrary-scale Super-resolution(https://arxiv.org/abs/2503.04385)
• Keywords: attack, robust
• Abstract: The advent of local continuous image function (LIIF) has garnered significant attention for arbitrary-scale super-resolution (SR) techniques. However, while the vulnerabilities of fixed-scale SR have been assessed, the robustness of continuous representation-based arbitrary-scale SR against adversarial attacks remains an area warranting further exploration. The elaborately designed adversarial attacks for fixed-scale SR are scale-dependent, which will cause time-consuming and memory-consuming problems when applied to arbitrary-scale SR. To address this concern, we propose a simple yet effective ``scale-invariant'' SR adversarial attack method with good transferability, termed SIAGT. Specifically, we propose to construct resource-saving attacks by exploiting finite discrete points of continuous representation. In addition, we formulate a coordinate-dependent loss to enhance the cross-model transferability of the attack. The attack can significantly deteriorate the SR images while introducing imperceptible distortion to the targeted low-resolution (LR) images. Experiments carried out on three popular LIIF-based SR approaches and four classical SR datasets show remarkable attack performance and transferability of SIAGT.

Title: Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication

• Authors: Tom Kouwenhoven, Max Peeperkorn, Roy de Kleijn, Tessa Verhoef
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04395
• Pdf URL: https://arxiv.org/pdf/2503.04395
• Copy Paste: [[2503.04395]] Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication(https://arxiv.org/abs/2503.04395)
• Keywords: large language model
• Abstract: Languages are shaped by the inductive biases of their users. Using a classical referential game, we investigate how artificial languages evolve when optimised for inductive biases in humans and large language models (LLMs) via Human-Human, LLM-LLM and Human-LLM experiments. We show that referentially grounded vocabularies emerge that enable reliable communication in all conditions, even when humans and LLMs collaborate. Comparisons between conditions reveal that languages optimised for LLMs subtly differ from those optimised for humans. Interestingly, interactions between humans and LLMs alleviate these differences and result in vocabularies which are more human-like than LLM-like. These findings advance our understanding of how inductive biases in LLMs play a role in the dynamic nature of human language and contribute to maintaining alignment in human and machine communication. In particular, our work underscores the need to think of new methods that include human interaction in the training processes of LLMs, and shows that using communicative success as a reward signal can be a fruitful, novel direction.

Title: TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models

• Authors: Xinyi He, Yihao Liu, Mengyu Zhou, Yeye He, Haoyu Dong, Shi Han, Zejian Yuan, Dongmei Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04396
• Pdf URL: https://arxiv.org/pdf/2503.04396
• Copy Paste: [[2503.04396]] TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models(https://arxiv.org/abs/2503.04396)
• Keywords: robust, large language model
• Abstract: Tabular data are crucial in many fields and their understanding by large language models (LLMs) under high parameter efficiency paradigm is important. However, directly applying parameter-efficient fine-tuning (PEFT) techniques to tabular tasks presents significant challenges, particularly in terms of better table serialization and the representation of two-dimensional structured information within a one-dimensional sequence. To address this, we propose TableLoRA, a module designed to improve LLMs' understanding of table structure during PEFT. It incorporates special tokens for serializing tables with special token encoder and uses 2D LoRA to encode low-rank information on cell positions. Experiments on four tabular-related datasets demonstrate that TableLoRA consistently outperforms vanilla LoRA and surpasses various table encoding methods tested in control experiments. These findings reveal that TableLoRA, as a table-specific LoRA, enhances the ability of LLMs to process tabular data effectively, especially in low-parameter settings, demonstrating its potential as a robust solution for handling table-related tasks.

Title: Speculative MoE: Communication Efficient Parallel MoE Inference with Speculative Token and Expert Pre-scheduling

• Authors: Yan Li, Pengfei Zheng, Shuang Chen, Zewei Xu, Yunfei Du, Zhengang Wang
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2503.04398
• Pdf URL: https://arxiv.org/pdf/2503.04398
• Copy Paste: [[2503.04398]] Speculative MoE: Communication Efficient Parallel MoE Inference with Speculative Token and Expert Pre-scheduling(https://arxiv.org/abs/2503.04398)
• Keywords: transformer, large language model
• Abstract: MoE (Mixture of Experts) prevails as a neural architecture that can scale modern transformer-based LLMs (Large Language Models) to unprecedented scales. Nevertheless, large MoEs' great demands of computing power, memory capacity and memory bandwidth make scalable serving a fundamental challenge and efficient parallel inference has become a requisite to attain adequate throughput under latency constraints. DeepSpeed-MoE, one state-of-the-art MoE inference framework, adopts a 3D-parallel paradigm including EP (Expert Parallelism), TP (Tensor Parallel) and DP (Data Parallelism). However, our analysis shows DeepSpeed-MoE's inference efficiency is largely bottlenecked by EP, which is implemented with costly all-to-all collectives to route token activation. Our work aims to boost DeepSpeed-MoE by strategically reducing EP's communication overhead with a technique named Speculative MoE. Speculative MoE has two speculative parallelization schemes, speculative token shuffling and speculative expert grouping, which predict outstanding tokens' expert routing paths and pre-schedule tokens and experts across devices to losslessly trim EP's communication volume. Besides DeepSpeed-MoE, we also build Speculative MoE into a prevailing MoE inference engine SGLang. Experiments show Speculative MoE can significantly boost state-of-the-art MoE inference frameworks on fast homogeneous and slow heterogeneous interconnects.

Title: Temporal Analysis of NetFlow Datasets for Network Intrusion Detection Systems

• Authors: Majed Luay, Siamak Layeghy, Seyedehfaezeh Hosseininoorbin, Mohanad Sarhan, Nour Moustafa, Marius Portmann
• Subjects: cs.LG, cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2503.04404
• Pdf URL: https://arxiv.org/pdf/2503.04404
• Copy Paste: [[2503.04404]] Temporal Analysis of NetFlow Datasets for Network Intrusion Detection Systems(https://arxiv.org/abs/2503.04404)
• Keywords: attack
• Abstract: This paper investigates the temporal analysis of NetFlow datasets for machine learning (ML)-based network intrusion detection systems (NIDS). Although many previous studies have highlighted the critical role of temporal features, such as inter-packet arrival time and flow length/duration, in NIDS, the currently available NetFlow datasets for NIDS lack these temporal features. This study addresses this gap by creating and making publicly available a set of NetFlow datasets that incorporate these temporal features [1]. With these temporal features, we provide a comprehensive temporal analysis of NetFlow datasets by examining the distribution of various features over time and presenting time-series representations of NetFlow features. This temporal analysis has not been previously provided in the existing literature. We also borrowed an idea from signal processing, time frequency analysis, and tested it to see how different the time frequency signal presentations (TFSPs) are for various attacks. The results indicate that many attacks have unique patterns, which could help ML models to identify them more easily.

Title: Can Large Language Models Predict Antimicrobial Resistance Gene?

• Authors: Hyunwoo Yoo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04413
• Pdf URL: https://arxiv.org/pdf/2503.04413
• Copy Paste: [[2503.04413]] Can Large Language Models Predict Antimicrobial Resistance Gene?(https://arxiv.org/abs/2503.04413)
• Keywords: transformer, generative, large language model
• Abstract: This study demonstrates that generative large language models can be utilized in a more flexible manner for DNA sequence analysis and classification tasks compared to traditional transformer encoder-based models. While recent encoder-based models such as DNABERT and Nucleotide Transformer have shown significant performance in DNA sequence classification, transformer decoder-based generative models have not yet been extensively explored in this field. This study evaluates how effectively generative Large Language Models handle DNA sequences with various labels and analyzes performance changes when additional textual information is provided. Experiments were conducted on antimicrobial resistance genes, and the results show that generative Large Language Models can offer comparable or potentially better predictions, demonstrating flexibility and accuracy when incorporating both sequence and textual information. The code and data used in this work are available at the following GitHub repository: this https URL.

Title: Learning Transformer-based World Models with Contrastive Predictive Coding

• Authors: Maxime Burchi, Radu Timofte
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04416
• Pdf URL: https://arxiv.org/pdf/2503.04416
• Copy Paste: [[2503.04416]] Learning Transformer-based World Models with Contrastive Predictive Coding(https://arxiv.org/abs/2503.04416)
• Keywords: transformer
• Abstract: The DreamerV3 algorithm recently obtained remarkable performance across diverse environment domains by learning an accurate world model based on Recurrent Neural Networks (RNNs). Following the success of model-based reinforcement learning algorithms and the rapid adoption of the Transformer architecture for its superior training efficiency and favorable scaling properties, recent works such as STORM have proposed replacing RNN-based world models with Transformer-based world models using masked self-attention. However, despite the improved training efficiency of these methods, their impact on performance remains limited compared to the Dreamer algorithm, struggling to learn competitive Transformer-based world models. In this work, we show that the next state prediction objective adopted in previous approaches is insufficient to fully exploit the representation capabilities of Transformers. We propose to extend world model predictions to longer time horizons by introducing TWISTER (Transformer-based World model wIth contraSTivE Representations), a world model using action-conditioned Contrastive Predictive Coding to learn high-level temporal feature representations and improve the agent performance. TWISTER achieves a human-normalized mean score of 162% on the Atari 100k benchmark, setting a new record among state-of-the-art methods that do not employ look-ahead search.

Title: PointsToWood: A deep learning framework for complete canopy leaf-wood segmentation of TLS data across diverse European forests

• Authors: Harry J. F. Owen, Matthew J. A. Allen, Stuart W. D. Grieve, Phill Wilkes, Emily R. Lines
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04420
• Pdf URL: https://arxiv.org/pdf/2503.04420
• Copy Paste: [[2503.04420]] PointsToWood: A deep learning framework for complete canopy leaf-wood segmentation of TLS data across diverse European forests(https://arxiv.org/abs/2503.04420)
• Keywords: extraction, segmentation
• Abstract: Point clouds from Terrestrial Laser Scanning (TLS) are an increasingly popular source of data for studying plant structure and function but typically require extensive manual processing to extract ecologically important information. One key task is the accurate semantic segmentation of different plant material within point clouds, particularly wood and leaves, which is required to understand plant productivity, architecture and physiology. Existing automated semantic segmentation methods are primarily developed for single ecosystem types, and whilst they show good accuracy for biomass assessment from the trunk and large branches, often perform less well within the crown. In this study, we demonstrate a new framework that uses a deep learning architecture newly developed from PointNet and pointNEXT for processing 3D point clouds to provide a reliable semantic segmentation of wood and leaf in TLS point clouds from the tree base to branch tips, trained on data from diverse mature European forests. Our model uses meticulously labelled data combined with voxel-based sampling, neighbourhood rescaling, and a novel gated reflectance integration module embedded throughout the feature extraction layers. We evaluate its performance across open datasets from boreal, temperate, Mediterranean and tropical regions, encompassing diverse ecosystem types and sensor characteristics. Our results show consistent outperformance against the most widely used PointNet based approach for leaf/wood segmentation on our high-density TLS dataset collected across diverse mixed forest plots across all major biomes in Europe. We also find consistently strong performance tested on others open data from China, Eastern Cameroon, Germany and Finland, collected using both time-of-flight and phase-shift sensors, showcasing the transferability of our model to a wide range of ecosystems and sensors.

Title: A Dataset for Analysing News Framing in Chinese Media

• Authors: Owen Cook, Yida Mu, Xinye Yang, Xingyi Song, Kalina Bontcheva
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04439
• Pdf URL: https://arxiv.org/pdf/2503.04439
• Copy Paste: [[2503.04439]] A Dataset for Analysing News Framing in Chinese Media(https://arxiv.org/abs/2503.04439)
• Keywords: fair
• Abstract: Framing is an essential device in news reporting, allowing the writer to influence public perceptions of current affairs. While there are existing automatic news framing detection datasets in various languages, none of them focus on news framing in the Chinese language which has complex character meanings and unique linguistic features. This study introduces the first Chinese News Framing dataset, to be used as either a stand-alone dataset or a supplementary resource to the SemEval-2023 task 3 dataset. We detail its creation and we run baseline experiments to highlight the need for such a dataset and create benchmarks for future research, providing results obtained through fine-tuning XLM-RoBERTa-Base and using GPT-4o in the zero-shot setting. We find that GPT-4o performs significantly worse than fine-tuned XLM-RoBERTa across all languages. For the Chinese language, we obtain an F1-micro (the performance metric for SemEval task 3, subtask 2) score of 0.719 using only samples from our Chinese News Framing dataset and a score of 0.753 when we augment the SemEval dataset with Chinese news framing samples. With positive news frame detection results, this dataset is a valuable resource for detecting news frames in the Chinese language and is a valuable supplement to the SemEval-2023 task 3 dataset.

Title: Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settings

• Authors: Marco Arazzi, Mert Cihangiroglu, Antonino Nocera
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04451
• Pdf URL: https://arxiv.org/pdf/2503.04451
• Copy Paste: [[2503.04451]] Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settings(https://arxiv.org/abs/2503.04451)
• Keywords: secure, privacy, protect, attack, robust, federate
• Abstract: Federated Averaging remains the most widely used aggregation strategy in federated learning due to its simplicity and scalability. However, its performance degrades significantly in non-IID data settings, where client distributions are highly imbalanced or skewed. Additionally, it relies on clients transmitting metadata, specifically the number of training samples, which introduces privacy risks and may conflict with regulatory frameworks like the European GDPR. In this paper, we propose a novel aggregation strategy that addresses these challenges by introducing class-aware gradient masking. Unlike traditional approaches, our method relies solely on gradient updates, eliminating the need for any additional client metadata, thereby enhancing privacy protection. Furthermore, our approach validates and dynamically weights client contributions based on class-specific importance, ensuring robustness against non-IID distributions, convergence prevention, and backdoor attacks. Extensive experiments on benchmark datasets demonstrate that our method not only outperforms FedAvg and other widely accepted aggregation strategies in non-IID settings but also preserves model integrity in adversarial scenarios. Our results establish the effectiveness of gradient masking as a practical and secure solution for federated learning.

Title: TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction

• Authors: Chao Wang, Weiwei Fu, Yang Zhou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04457
• Pdf URL: https://arxiv.org/pdf/2503.04457
• Copy Paste: [[2503.04457]] TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction(https://arxiv.org/abs/2503.04457)
• Keywords: robust, large language model
• Abstract: Vision-language models (VLMs) have achieved remarkable advancements, capitalizing on the impressive capabilities of large language models (LLMs) across diverse tasks. Despite this, a critical challenge known as hallucination occurs when models overconfidently describe objects or attributes absent from the image, a problem exacerbated by the tendency of VLMs to rely on linguistic priors. This limitation reduces model reliability in high-stakes applications. In this work, we have observed the characteristic of logits' continuity consistency enhancement and introduced a straightforward and efficient method, Cross-Temporal Prediction Connection (TPC), designed to enhance the semantic consistency of logits by connecting them temporally across timesteps. TPC amplifies information flow and improves coherence, effectively reducing hallucination. Extensive experiments show that TPC surpasses existing representatives, delivering superior performance in both accuracy and efficiency while maintaining robustness in open-ended text generation tasks.

Title: Guiding LLMs to Generate High-Fidelity and High-Quality Counterfactual Explanations for Text Classification

• Authors: Van Bach Nguyen, Christin Seifert, Jörg Schlötterer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04463
• Pdf URL: https://arxiv.org/pdf/2503.04463
• Copy Paste: [[2503.04463]] Guiding LLMs to Generate High-Fidelity and High-Quality Counterfactual Explanations for Text Classification(https://arxiv.org/abs/2503.04463)
• Keywords: robust, interpretability, large language model
• Abstract: The need for interpretability in deep learning has driven interest in counterfactual explanations, which identify minimal changes to an instance that change a model's prediction. Current counterfactual (CF) generation methods require task-specific fine-tuning and produce low-quality text. Large Language Models (LLMs), though effective for high-quality text generation, struggle with label-flipping counterfactuals (i.e., counterfactuals that change the prediction) without fine-tuning. We introduce two simple classifier-guided approaches to support counterfactual generation by LLMs, eliminating the need for fine-tuning while preserving the strengths of LLMs. Despite their simplicity, our methods outperform state-of-the-art counterfactual generation methods and are effective across different LLMs, highlighting the benefits of guiding counterfactual generation by LLMs with classifier information. We further show that data augmentation by our generated CFs can improve a classifier's robustness. Our analysis reveals a critical issue in counterfactual generation by LLMs: LLMs rely on parametric knowledge rather than faithfully following the classifier.

Title: Runtime Backdoor Detection for Federated Learning via Representational Dissimilarity Analysis

• Authors: Xiyue Zhang, Xiaoyong Xue, Xiaoning Du, Xiaofei Xie, Yang Liu, Meng Sun
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04473
• Pdf URL: https://arxiv.org/pdf/2503.04473
• Copy Paste: [[2503.04473]] Runtime Backdoor Detection for Federated Learning via Representational Dissimilarity Analysis(https://arxiv.org/abs/2503.04473)
• Keywords: protect, defense, attack, federate
• Abstract: Federated learning (FL), as a powerful learning paradigm, trains a shared model by aggregating model updates from distributed clients. However, the decoupling of model learning from local data makes FL highly vulnerable to backdoor attacks, where a single compromised client can poison the shared model. While recent progress has been made in backdoor detection, existing methods face challenges with detection accuracy and runtime effectiveness, particularly when dealing with complex model architectures. In this work, we propose a novel approach to detecting malicious clients in an accurate, stable, and efficient manner. Our method utilizes a sampling-based network representation method to quantify dissimilarities between clients, identifying model deviations caused by backdoor injections. We also propose an iterative algorithm to progressively detect and exclude malicious clients as outliers based on these dissimilarity measurements. Evaluations across a range of benchmark tasks demonstrate that our approach outperforms state-of-the-art methods in detection accuracy and defense effectiveness. When deployed for runtime protection, our approach effectively eliminates backdoor injections with marginal overheads.

Title: Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges

• Authors: Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04474
• Pdf URL: https://arxiv.org/pdf/2503.04474
• Copy Paste: [[2503.04474]] Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges(https://arxiv.org/abs/2503.04474)
• Keywords: security, attack, robust, large language model
• Abstract: Large Language Model (LLM) based judges form the underpinnings of key safety evaluation processes such as offline benchmarking, automated red-teaming, and online guardrailing. This widespread requirement raises the crucial question: can we trust the evaluations of these evaluators? In this paper, we highlight two critical challenges that are typically overlooked: (i) evaluations in the wild where factors like prompt sensitivity and distribution shifts can affect performance and (ii) adversarial attacks that target the judge. We highlight the importance of these through a study of commonly used safety judges, showing that small changes such as the style of the model output can lead to jumps of up to 0.24 in the false negative rate on the same dataset, whereas adversarial attacks on the model generation can fool some judges into misclassifying 100% of harmful generations as safe ones. These findings reveal gaps in commonly used meta-evaluation benchmarks and weaknesses in the robustness of current LLM judges, indicating that low attack success under certain judges could create a false sense of security.

Title: ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images

• Authors: Yanqing Shen, Turcan Tuna, Marco Hutter, Cesar Cadena, Nanning Zheng
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.04475
• Pdf URL: https://arxiv.org/pdf/2503.04475
• Copy Paste: [[2503.04475]] ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images(https://arxiv.org/abs/2503.04475)
• Keywords: robust, transformer
• Abstract: Place recognition is essential to maintain global consistency in large-scale localization systems. While research in urban environments has progressed significantly using LiDARs or cameras, applications in natural forest-like environments remain largely under-explored. Furthermore, forests present particular challenges due to high self-similarity and substantial variations in vegetation growth over time. In this work, we propose a robust LiDAR-based place recognition method for natural forests, ForestLPR. We hypothesize that a set of cross-sectional images of the forest's geometry at different heights contains the information needed to recognize revisiting a place. The cross-sectional images are represented by \ac{bev} density images of horizontal slices of the point cloud at different heights. Our approach utilizes a visual transformer as the shared backbone to produce sets of local descriptors and introduces a multi-BEV interaction module to attend to information at different heights adaptively. It is followed by an aggregation layer that produces a rotation-invariant place descriptor. We evaluated the efficacy of our method extensively on real-world data from public benchmarks as well as robotic datasets and compared it against the state-of-the-art (SOTA) methods. The results indicate that ForestLPR has consistently good performance on all evaluations and achieves an average increase of 7.38\% and 9.11\% on Recall@1 over the closest competitor on intra-sequence loop closure detection and inter-sequence re-localization, respectively, validating our hypothesis

Title: Generalized Interpolating Discrete Diffusion

• Authors: Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04482
• Pdf URL: https://arxiv.org/pdf/2503.04482
• Copy Paste: [[2503.04482]] Generalized Interpolating Discrete Diffusion(https://arxiv.org/abs/2503.04482)
• Keywords: diffusion
• Abstract: While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated tokens. This has prompted exploration of alternative approaches such as discrete diffusion. However, masked diffusion, which has emerged as a popular choice due to its simplicity and effectiveness, reintroduces this inability to revise words. To overcome this, we generalize masked diffusion and derive the theoretical backbone of a family of general interpolating discrete diffusion (GIDD) processes offering greater flexibility in the design of the noising processes. Leveraging a novel diffusion ELBO, we achieve compute-matched state-of-the-art performance in diffusion language modeling. Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality and unlocking the ability for the model to correct its own mistakes, an area where autoregressive models notoriously have struggled. Our code and models are open-source: this https URL

Title: Large Language Models in Bioinformatics: A Survey

• Authors: Zhenyu Wang, Zikang Wang, Jiyue Jiang, Pengan Chen, Xiangyu Shi, Yu Li
• Subjects: cs.CL, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2503.04490
• Pdf URL: https://arxiv.org/pdf/2503.04490
• Copy Paste: [[2503.04490]] Large Language Models in Bioinformatics: A Survey(https://arxiv.org/abs/2503.04490)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are revolutionizing bioinformatics, enabling advanced analysis of DNA, RNA, proteins, and single-cell data. This survey provides a systematic review of recent advancements, focusing on genomic sequence modeling, RNA structure prediction, protein function inference, and single-cell transcriptomics. Meanwhile, we also discuss several key challenges, including data scarcity, computational complexity, and cross-omics integration, and explore future directions such as multimodal learning, hybrid AI models, and clinical applications. By offering a comprehensive perspective, this paper underscores the transformative potential of LLMs in driving innovations in bioinformatics and precision medicine.

Title: Spatial regularisation for improved accuracy and interpretability in keypoint-based registration

• Authors: Benjamin Billot, Ramya Muthukrishnan, Esra Abaci-Turk, Ellen P. Grant, Nicholas Ayache, Hervé Delingette, Polina Golland
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04499
• Pdf URL: https://arxiv.org/pdf/2503.04499
• Copy Paste: [[2503.04499]] Spatial regularisation for improved accuracy and interpretability in keypoint-based registration(https://arxiv.org/abs/2503.04499)
• Keywords: interpretability, segmentation
• Abstract: Unsupervised registration strategies bypass requirements in ground truth transforms or segmentations by optimising similarity metrics between fixed and moved volumes. Among these methods, a recent subclass of approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Specifically, these methods train a network to predict feature maps for fixed and moving images, from which explainable centres of mass are computed to obtain point clouds, that are then aligned in closed-form. However, the features returned by the network often yield spatially diffuse patterns that are hard to interpret, thus undermining the purpose of keypoint-based registration. Here, we propose a three-fold loss to regularise the spatial distribution of the features. First, we use the KL divergence to model features as point spread functions that we interpret as probabilistic keypoints. Then, we sharpen the spatial distributions of these features to increase the precision of the detected landmarks. Finally, we introduce a new repulsive loss across keypoints to encourage spatial diversity. Overall, our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks. We demonstrate our three-fold loss in foetal rigid motion tracking and brain MRI affine registration tasks, where it not only outperforms state-of-the-art unsupervised strategies, but also bridges the gap with state-of-the-art supervised methods. Our code is available at this https URL.

Title: ReynoldsFlow: Exquisite Flow Estimation via Reynolds Transport Theorem

• Authors: Yu-Hsi Chen, Chin-Tien Wu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04500
• Pdf URL: https://arxiv.org/pdf/2503.04500
• Copy Paste: [[2503.04500]] ReynoldsFlow: Exquisite Flow Estimation via Reynolds Transport Theorem(https://arxiv.org/abs/2503.04500)
• Keywords: robust
• Abstract: Optical flow is a fundamental technique for motion estimation, widely applied in video stabilization, interpolation, and object tracking. Recent advancements in artificial intelligence (AI) have enabled deep learning models to leverage optical flow as an important feature for motion analysis. However, traditional optical flow methods rely on restrictive assumptions, such as brightness constancy and slow motion constraints, limiting their effectiveness in complex scenes. Deep learning-based approaches require extensive training on large domain-specific datasets, making them computationally demanding. Furthermore, optical flow is typically visualized in the HSV color space, which introduces nonlinear distortions when converted to RGB and is highly sensitive to noise, degrading motion representation accuracy. These limitations inherently constrain the performance of downstream models, potentially hindering object tracking and motion analysis tasks. To address these challenges, we propose Reynolds flow, a novel training-free flow estimation inspired by the Reynolds transport theorem, offering a principled approach to modeling complex motion dynamics. Beyond the conventional HSV-based visualization, denoted ReynoldsFlow, we introduce an alternative representation, ReynoldsFlow+, designed to improve flow visualization. We evaluate ReynoldsFlow and ReynoldsFlow+ across three video-based benchmarks: tiny object detection on UAVDB, infrared object detection on Anti-UAV, and pose estimation on GolfDB. Experimental results demonstrate that networks trained with ReynoldsFlow+ achieve state-of-the-art (SOTA) performance, exhibiting improved robustness and efficiency across all tasks.

Title: IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement

• Authors: Zhihao Shi, Dong Huo, Yuhongze Zhou, Kejia Yin, Yan Min, Juwei Lu, Xinxin Zuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04501
• Pdf URL: https://arxiv.org/pdf/2503.04501
• Copy Paste: [[2503.04501]] IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement(https://arxiv.org/abs/2503.04501)
• Keywords: robust
• Abstract: Current 3D inpainting and object removal methods are largely limited to front-facing scenes, facing substantial challenges when applied to diverse, "unconstrained" scenes where the camera orientation and trajectory are unrestricted. To bridge this gap, we introduce a novel approach that produces inpainted 3D scenes with consistent visual quality and coherent underlying geometry across both front-facing and unconstrained scenes. Specifically, we propose a robust 3D inpainting pipeline that incorporates geometric priors and a multi-view refinement network trained via test-time adaptation, building on a pre-trained image inpainting model. Additionally, we develop a novel inpainting mask detection technique to derive targeted inpainting masks from object masks, boosting the performance in handling unconstrained scenes. To validate the efficacy of our approach, we create a challenging and diverse benchmark that spans a wide range of scenes. Comprehensive experiments demonstrate that our proposed method substantially outperforms existing state-of-the-art approaches.

Title: STX-Search: Explanation Search for Continuous Dynamic Spatio-Temporal Models

• Authors: Saif Anwar, Nathan Griffiths, Thomas Popham, Abhir Bhalerao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04509
• Pdf URL: https://arxiv.org/pdf/2503.04509
• Copy Paste: [[2503.04509]] STX-Search: Explanation Search for Continuous Dynamic Spatio-Temporal Models(https://arxiv.org/abs/2503.04509)
• Keywords: interpretability
• Abstract: Recent improvements in the expressive power of spatio-temporal models have led to performance gains in many real-world applications, such as traffic forecasting and social network modelling. However, understanding the predictions from a model is crucial to ensure reliability and trustworthiness, particularly for high-risk applications, such as healthcare and transport. Few existing methods are able to generate explanations for models trained on continuous-time dynamic graph data and, of these, the computational complexity and lack of suitable explanation objectives pose challenges. In this paper, we propose $\textbf{S}$patio-$\textbf{T}$emporal E$\textbf{X}$planation $\textbf{Search}$ (STX-Search), a novel method for generating instance-level explanations that is applicable to static and dynamic temporal graph structures. We introduce a novel search strategy and objective function, to find explanations that are highly faithful and interpretable. When compared with existing methods, STX-Search produces explanations of higher fidelity whilst optimising explanation size to maintain interpretability.

Title: In-Context Reverse Classification Accuracy: Efficient Estimation of Segmentation Quality without Ground-Truth

• Authors: Matias Cosarinsky, Ramiro Billot, Lucas Mansilla, Gabriel Gimenez, Nicolas Gaggión, Guanghui Fu, Enzo Ferrante
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04522
• Pdf URL: https://arxiv.org/pdf/2503.04522
• Copy Paste: [[2503.04522]] In-Context Reverse Classification Accuracy: Efficient Estimation of Segmentation Quality without Ground-Truth(https://arxiv.org/abs/2503.04522)
• Keywords: robust, segmentation
• Abstract: Assessing the quality of automatic image segmentation is crucial in clinical practice, but often very challenging due to the limited availability of ground truth annotations. In this paper, we introduce In-Context Reverse Classification Accuracy (In-Context RCA), a novel framework for automatically estimating segmentation quality in the absence of ground-truth annotations. By leveraging recent in-context learning segmentation models and incorporating retrieval-augmentation techniques to select the most relevant reference images, our approach enables efficient quality estimation with minimal reference data. Validated across diverse medical imaging modalities, our method demonstrates robust performance and computational efficiency, offering a promising solution for automated quality control in clinical workflows, where fast and reliable segmentation assessment is essential. The code is available at this https URL.

Title: Federated Dynamic Modeling and Learning for Spatiotemporal Data Forecasting

• Authors: Thien Pham, Angelo Furno, Faïcel Chamroukhi, Latifa Oukhellou
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04528
• Pdf URL: https://arxiv.org/pdf/2503.04528
• Copy Paste: [[2503.04528]] Federated Dynamic Modeling and Learning for Spatiotemporal Data Forecasting(https://arxiv.org/abs/2503.04528)
• Keywords: privacy, robust, federate
• Abstract: This paper presents an advanced Federated Learning (FL) framework for forecasting complex spatiotemporal data, improving upon recent state-of-the-art models. In the proposed approach, the original Gated Recurrent Unit (GRU) module within previous Dynamic Spatial--Temporal Graph Convolutional Recurrent Network (DSTGCRN) modeling is first replaced with a Long Short-Term Memory (LSTM) network, enabling the resulting model to more effectively capture long-term dependencies inherent to time series data. The resulting architecture significantly improves the model's capacity to handle complex temporal patterns in diverse forecasting applications. Furthermore, the proposed FL framework integrates a novel Client-Side Validation (CSV) mechanism, introducing a critical validation step at the client level before incorporating aggregated parameters from the central server into local models. This ensures that only the most effective updates are adopted, improving both the robustness and accuracy of the forecasting model across clients. The efficiency of our approach is demonstrated through extensive experiments on real-world applications, including public datasets for multimodal transport demand forecasting and private datasets for Origin-Destination (OD) matrix forecasting in urban areas. The results demonstrate substantial improvements over conventional methods, highlighting the framework's ability to capture complex spatiotemporal dependencies while preserving data privacy. This work not only provides a scalable and privacy-preserving solution for real-time, region-specific forecasting and management but also underscores the potential of leveraging distributed data sources in a FL context. We provide our algorithms as open-source on GitHub.

Title: Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model

• Authors: Wenke Huang, Jian Liang, Xianda Guo, Yiyang Fang, Guancheng Wan, Xuankun Rong, Chi Wen, Zekun Shi, Qingyun Li, Didi Zhu, Yanbiao Ma, Ke Liang, Bin Yang, He Li, Jiawei Shao, Mang Ye, Bo Du
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04543
• Pdf URL: https://arxiv.org/pdf/2503.04543
• Copy Paste: [[2503.04543]] Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model(https://arxiv.org/abs/2503.04543)
• Keywords: large language model
• Abstract: Multi-modal Large Language Models (MLLMs) integrate visual and linguistic reasoning to address complex tasks such as image captioning and visual question answering. While MLLMs demonstrate remarkable versatility, MLLMs appears limited performance on special applications. But tuning MLLMs for downstream tasks encounters two key challenges: Task-Expert Specialization, where distribution shifts between pre-training and target datasets constrain target performance, and Open-World Stabilization, where catastrophic forgetting erases the model general knowledge. In this work, we systematically review recent advancements in MLLM tuning methodologies, classifying them into three paradigms: (I) Selective Tuning, (II) Additive Tuning, and (III) Reparameterization Tuning. Furthermore, we benchmark these tuning strategies across popular MLLM architectures and diverse downstream tasks to establish standardized evaluation analysis and systematic tuning principles. Finally, we highlight several open challenges in this domain and propose future research directions. To facilitate ongoing progress in this rapidly evolving field, we provide a public repository that continuously tracks developments: this https URL.

Title: Lite-PoT: Practical Powers-of-Tau Setup Ceremony

• Authors: Lucien K. L. Ng, Pedro Moreno-Sanchez, Mohsen Minaei, Panagiotis Chatzigiannis, Adithya Bhat, Duc V. Le
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04549
• Pdf URL: https://arxiv.org/pdf/2503.04549
• Copy Paste: [[2503.04549]] Lite-PoT: Practical Powers-of-Tau Setup Ceremony(https://arxiv.org/abs/2503.04549)
• Keywords: privacy, attack
• Abstract: Zero-Knowledge Succinct Non-Interactive Argument of Knowledge (zk-SNARK) schemes have gained significant adoption in privacy-preserving applications, decentralized systems (e.g., blockchain), and verifiable computation due to their efficiency. However, the most efficient zk-SNARKs often rely on a one-time trusted setup to generate a public parameter, often known as the ``Powers of Tau" (PoT) string. The leakage of the secret parameter, $\tau$, in the string would allow attackers to generate false proofs, compromising the soundness of all zk-SNARK systems built on it. Prior proposals for decentralized setup ceremonies have utilized blockchain-based smart contracts to allow any party to contribute randomness to $\tau$ while also preventing censorship of contributions. For a PoT string of $d$-degree generated by the randomness of $m$ contributors, these solutions required a total of $O(md)$ on-chain operations (i.e., in terms of both storage and cryptographic operations). These operations primarily consisted of costly group operations, particularly scalar multiplication on pairing curves, which discouraged participation and limited the impact of decentralization In this work, we present Lite-PoT, which includes two key protocols designed to reduce participation costs: \emph{(i)} a fraud-proof protocol to reduce the number of expensive on-chain cryptographic group operations to $O(1)$ per contributor. Our experimental results show that (with one transaction per update) our protocol enables decentralized ceremonies for PoT strings up to a $2^{15}$ degree, an $\approx 16x$ improvement over existing on-chain solutions; \emph{(ii)} a proof aggregation technique that batches $m$ randomness contributions into one on-chain update with only $O(d)$ on-chain operations, independent of $m$. This significantly reduces the monetary cost of on-chain updates by $m$-fold via amortization.

Title: Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation

• Authors: Armel Zebaze, Benoît Sagot, Rachel Bawden
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04554
• Pdf URL: https://arxiv.org/pdf/2503.04554
• Copy Paste: [[2503.04554]] Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation(https://arxiv.org/abs/2503.04554)
• Keywords: generative, large language model
• Abstract: The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural language processing tasks. Machine Translation (MT) has been shown to benefit from in-context examples, in particular when they are semantically similar to the sentence to translate. In this paper, we propose a new LLM-based translation paradigm, compositional translation, to replace naive few-shot MT with similarity-based demonstrations. An LLM is used to decompose a sentence into simpler phrases, and then to translate each phrase with the help of retrieved demonstrations. Finally, the LLM is prompted to translate the initial sentence with the help of the self-generated phrase-translation pairs. Our intuition is that this approach should improve translation because these shorter phrases should be intrinsically easier to translate and easier to match with relevant examples. This is especially beneficial in low-resource scenarios, and more generally whenever the selection pool is small or out of domain. We show that compositional translation boosts LLM translation performance on a wide range of popular MT benchmarks, including FLORES 200, NTREX 128 and TICO-19. Code and outputs are available at this https URL

Title: Compositional Causal Reasoning Evaluation in Language Models

• Authors: Jacqueline R. M. A. Maasch, Alihan Hüyük, Xinnuo Xu, Aditya V. Nori, Javier Gonzalez
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04556
• Pdf URL: https://arxiv.org/pdf/2503.04556
• Copy Paste: [[2503.04556]] Compositional Causal Reasoning Evaluation in Language Models(https://arxiv.org/abs/2503.04556)
• Keywords: generative
• Abstract: Causal reasoning and compositional reasoning are two core aspirations in generative AI. Measuring the extent of these behaviors requires principled evaluation methods. We explore a unified perspective that considers both behaviors simultaneously, termed compositional causal reasoning (CCR): the ability to infer how causal measures compose and, equivalently, how causal quantities propagate through graphs. We instantiate a framework for the systematic evaluation of CCR for the average treatment effect and the probability of necessity and sufficiency. As proof of concept, we demonstrate the design of CCR tasks for language models in the LLama, Phi, and GPT families. On a math word problem, our framework revealed a range of taxonomically distinct error patterns. Additionally, CCR errors increased with the complexity of causal paths for all models except o1.

Title: Meta Learning not to Learn: Robustly Informing Meta-Learning under Nuisance-Varying Families

• Authors: Louis McConnell
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04570
• Pdf URL: https://arxiv.org/pdf/2503.04570
• Copy Paste: [[2503.04570]] Meta Learning not to Learn: Robustly Informing Meta-Learning under Nuisance-Varying Families(https://arxiv.org/abs/2503.04570)
• Keywords: robust
• Abstract: In settings where both spurious and causal predictors are available, standard neural networks trained under the objective of empirical risk minimization (ERM) with no additional inductive biases tend to have a dependence on a spurious feature. As a result, it is necessary to integrate additional inductive biases in order to guide the network toward generalizable hypotheses. Often these spurious features are shared across related tasks, such as estimating disease prognoses from image scans coming from different hospitals, making the challenge of generalization more difficult. In these settings, it is important that methods are able to integrate the proper inductive biases to generalize across both nuisance-varying families as well as task families. Motivated by this setting, we present RIME (Robustly Informed Meta lEarning), a new method for meta learning under the presence of both positive and negative inductive biases (what to learn and what not to learn). We first develop a theoretical causal framework showing why existing approaches at knowledge integration can lead to worse performance on distributionally robust objectives. We then show that RIME is able to simultaneously integrate both biases, reaching state of the art performance under distributionally robust objectives in informed meta-learning settings under nuisance-varying families.

Title: PSDNorm: Test-Time Temporal Normalization for Deep Learning on EEG Signals

• Authors: Théo Gnassounou, Antoine Collas, Rémi Flamary, Alexandre Gramfort
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04582
• Pdf URL: https://arxiv.org/pdf/2503.04582
• Copy Paste: [[2503.04582]] PSDNorm: Test-Time Temporal Normalization for Deep Learning on EEG Signals(https://arxiv.org/abs/2503.04582)
• Keywords: robust
• Abstract: Distribution shift poses a significant challenge in machine learning, particularly in biomedical applications such as EEG signals collected across different subjects, institutions, and recording devices. While existing normalization layers, Batch-Norm, LayerNorm and InstanceNorm, help address distribution shifts, they fail to capture the temporal dependencies inherent in temporal signals. In this paper, we propose PSDNorm, a layer that leverages Monge mapping and temporal context to normalize feature maps in deep learning models. Notably, the proposed method operates as a test-time domain adaptation technique, addressing distribution shifts without additional training. Evaluations on 10 sleep staging datasets using the U-Time model demonstrate that PSDNorm achieves state-of-the-art performance at test time on datasets not seen during training while being 4x more data-efficient than the best baseline. Additionally, PSDNorm provides a significant improvement in robustness, achieving markedly higher F1 scores for the 20% hardest subjects.

Title: HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

• Authors: Zhijian Zhuo, Yutao Zeng, Ya Wang, Sijun Zhang, Jian Yang, Xiaoqing Li, Xun Zhou, Jinwen Ma
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04598
• Pdf URL: https://arxiv.org/pdf/2503.04598
• Copy Paste: [[2503.04598]] HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization(https://arxiv.org/abs/2503.04598)
• Keywords: transformer, large language model
• Abstract: Transformers have become the de facto architecture for a wide range of machine learning tasks, particularly in large language models (LLMs). Despite their remarkable performance, challenges remain in training deep transformer networks, especially regarding the location of layer normalization. While Pre-Norm structures facilitate easier training due to their more prominent identity path, they often yield suboptimal performance compared to Post-Norm. In this paper, we propose $\textbf{HybridNorm}$, a straightforward yet effective hybrid normalization strategy that integrates the advantages of both Pre-Norm and Post-Norm approaches. Specifically, HybridNorm employs QKV normalization within the attention mechanism and Post-Norm in the feed-forward network (FFN) of each transformer block. This design not only stabilizes training but also enhances performance, particularly in the context of LLMs. Comprehensive experiments in both dense and sparse architectures show that HybridNorm consistently outperforms both Pre-Norm and Post-Norm approaches, achieving state-of-the-art results across various benchmarks. These findings highlight the potential of HybridNorm as a more stable and effective technique for improving the training and performance of deep transformer models. %Code will be made publicly available. Code is available at this https URL.

Title: The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation

• Authors: Aoxiong Yin, Kai Shen, Yichong Leng, Xu Tan, Xinyu Zhou, Juncheng Li, Siliang Tang
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04606
• Pdf URL: https://arxiv.org/pdf/2503.04606
• Copy Paste: [[2503.04606]] The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation(https://arxiv.org/abs/2503.04606)
• Keywords: diffusion
• Abstract: Recent advancements in text-to-video (T2V) generation have been driven by two competing paradigms: autoregressive language models and diffusion models. However, each paradigm has intrinsic limitations: language models struggle with visual quality and error accumulation, while diffusion models lack semantic understanding and causal modeling. In this work, we propose LanDiff, a hybrid framework that synergizes the strengths of both paradigms through coarse-to-fine generation. Our architecture introduces three key innovations: (1) a semantic tokenizer that compresses 3D visual features into compact 1D discrete representations through efficient semantic compression, achieving a $\sim$14,000$\times$ compression ratio; (2) a language model that generates semantic tokens with high-level semantic relationships; (3) a streaming diffusion model that refines coarse semantics into high-fidelity videos. Experiments show that LanDiff, a 5B model, achieves a score of 85.43 on the VBench T2V benchmark, surpassing the state-of-the-art open-source models Hunyuan Video (13B) and other commercial models such as Sora, Keling, and Hailuo. Furthermore, our model also achieves state-of-the-art performance in long video generation, surpassing other open-source models in this field. Our demo can be viewed at this https URL.

Title: Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning

• Authors: Mohammad Amin Ghanizadeh, Mohammad Javad Dousti
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04611
• Pdf URL: https://arxiv.org/pdf/2503.04611
• Copy Paste: [[2503.04611]] Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning(https://arxiv.org/abs/2503.04611)
• Keywords: large language model
• Abstract: In this work, we explain our approach employed in the BabyLM Challenge, which uses various methods of training language models (LMs) with significantly less data compared to traditional large language models (LLMs) and are inspired by how human children learn. While a human child is exposed to far less linguistic input than an LLM, they still achieve remarkable language understanding and generation abilities. To this end, we develop a model trained on a curated dataset consisting of 10 million words, primarily sourced from child-directed transcripts. The 2024 BabyLM Challenge initial dataset of 10M words is filtered to 8.5M. Next, it is supplemented with a randomly selected subset of TVR dataset consisting of 1.5M words of television dialogues. The latter dataset ensures that similar to children, the model is also exposed to language through media. Furthermore, we reduce the vocabulary size to 32,000 tokens, aligning it with the limited vocabulary of children in the early stages of language acquisition. We use curriculum learning and is able to match the baseline on certain benchmarks while surpassing the baseline on others. Additionally, incorporating common LLM training datasets, such as MADLAD-400, degrades performance. These findings underscore the importance of dataset selection, vocabulary scaling, and curriculum learning in creating more data-efficient language models that better mimic human learning processes.

Title: Better Process Supervision with Bi-directional Rewarding Signals

• Authors: Wenxiang Chen, Wei He, Zhiheng Xi, Honglin Guo, Boyang Hong, Jiazheng Zhang, Rui Zheng, Nijun Li, Tao Gui, Yun Li, Qi Zhang, Xuanjing Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04618
• Pdf URL: https://arxiv.org/pdf/2503.04618
• Copy Paste: [[2503.04618]] Better Process Supervision with Bi-directional Rewarding Signals(https://arxiv.org/abs/2503.04618)
• Keywords: large language model
• Abstract: Process supervision, i.e., evaluating each step, is critical for complex large language model (LLM) reasoning and test-time searching with increased inference compute. Existing approaches, represented by process reward models (PRMs), primarily focus on rewarding signals up to the current step, exhibiting a one-directional nature and lacking a mechanism to model the distance to the final target. To address this problem, we draw inspiration from the A* algorithm, which states that an effective supervisory signal should simultaneously consider the incurred cost and the estimated cost for reaching the target. Building on this key insight, we introduce BiRM, a novel process supervision model that not only evaluates the correctness of previous steps but also models the probability of future success. We conduct extensive experiments on mathematical reasoning tasks and demonstrate that BiRM provides more precise evaluations of LLM reasoning steps, achieving an improvement of 3.1% on Gaokao2023 over PRM under the Best-of-N sampling method. Besides, in search-based strategies, BiRM provides more comprehensive guidance and outperforms ORM by 5.0% and PRM by 3.8% respectively on MATH-500.

Title: PathoPainter: Augmenting Histopathology Segmentation via Tumor-aware Inpainting

• Authors: Hong Liu, Haosen Yang, Evi M.C. Huijben, Mark Schuiveling, Ruisheng Su, Josien P.W. Pluim, Mitko Veta
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04634
• Pdf URL: https://arxiv.org/pdf/2503.04634
• Copy Paste: [[2503.04634]] PathoPainter: Augmenting Histopathology Segmentation via Tumor-aware Inpainting(https://arxiv.org/abs/2503.04634)
• Keywords: segmentation
• Abstract: Tumor segmentation plays a critical role in histopathology, but it requires costly, fine-grained image-mask pairs annotated by pathologists. Thus, synthesizing histopathology data to expand the dataset is highly desirable. Previous works suffer from inaccuracies and limited diversity in image-mask pairs, both of which affect training segmentation, particularly in small-scale datasets and the inherently complex nature of histopathology images. To address this challenge, we propose PathoPainter, which reformulates image-mask pair generation as a tumor inpainting task. Specifically, our approach preserves the background while inpainting the tumor region, ensuring precise alignment between the generated image and its corresponding mask. To enhance dataset diversity while maintaining biological plausibility, we incorporate a sampling mechanism that conditions tumor inpainting on regional embeddings from a different image. Additionally, we introduce a filtering strategy to exclude uncertain synthetic regions, further improving the quality of the generated data. Our comprehensive evaluation spans multiple datasets featuring diverse tumor types and various training data scales. As a result, segmentation improved significantly with our synthetic data, surpassing existing segmentation data synthesis approaches, e.g., 75.69% -> 77.69% on CAMELYON16. The code is available at this https URL.

Title: Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking

• Authors: Yijie Xu, Aiwei Liu, Xuming Hu, Lijie Wen, Hui Xiong
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04636
• Pdf URL: https://arxiv.org/pdf/2503.04636
• Copy Paste: [[2503.04636]] Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking(https://arxiv.org/abs/2503.04636)
• Keywords: robust, watermark, generative, large language model
• Abstract: As open-source large language models (LLMs) like Llama3 become more capable, it is crucial to develop watermarking techniques to detect their potential misuse. Existing watermarking methods either add watermarks during LLM inference, which is unsuitable for open-source LLMs, or primarily target classification LLMs rather than recent generative LLMs. Adapting these watermarks to open-source LLMs for misuse detection remains an open challenge. This work defines two misuse scenarios for open-source LLMs: intellectual property (IP) violation and LLM Usage Violation. Then, we explore the application of inference-time watermark distillation and backdoor watermarking in these contexts. We propose comprehensive evaluation methods to assess the impact of various real-world further fine-tuning scenarios on watermarks and the effect of these watermarks on LLM performance. Our experiments reveal that backdoor watermarking could effectively detect IP Violation, while inference-time watermark distillation is applicable in both scenarios but less robust to further fine-tuning and has a more significant impact on LLM performance compared to backdoor watermarking. Exploring more advanced watermarking methods for open-source LLMs to detect their misuse should be an important future direction.

Title: Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation

• Authors: Aishik Konwer, Zhijian Yang, Erhan Bas, Cao Xiao, Prateek Prasanna, Parminder Bhatia, Taha Kass-Hout
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04639
• Pdf URL: https://arxiv.org/pdf/2503.04639
• Copy Paste: [[2503.04639]] Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation(https://arxiv.org/abs/2503.04639)
• Keywords: segmentation
• Abstract: Foundational models such as the Segment Anything Model (SAM) are gaining traction in medical imaging segmentation, supporting multiple downstream tasks. However, such models are supervised in nature, still relying on large annotated datasets or prompts supplied by experts. Conventional techniques such as active learning to alleviate such limitations are limited in scope and still necessitate continuous human involvement and complex domain knowledge for label refinement or establishing reward ground truth. To address these challenges, we propose an enhanced Segment Anything Model (SAM) framework that utilizes annotation-efficient prompts generated in a fully unsupervised fashion, while still capturing essential semantic, location, and shape information through contrastive language-image pretraining and visual question answering. We adopt the direct preference optimization technique to design an optimal policy that enables the model to generate high-fidelity segmentations with simple ratings or rankings provided by a virtual annotator simulating the human annotation process. State-of-the-art performance of our framework in tasks such as lung segmentation, breast tumor segmentation, and organ segmentation across various modalities, including X-ray, ultrasound, and abdominal CT, justifies its effectiveness in low-annotation data scenarios.

Title: Simulating the Real World: A Unified Survey of Multimodal Generative Models

• Authors: Yuqi Hu, Longguang Wang, Xian Liu, Ling-Hao Chen, Yuwei Guo, Yukai Shi, Ce Liu, Anyi Rao, Zeyu Wang, Hui Xiong
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04641
• Pdf URL: https://arxiv.org/pdf/2503.04641
• Copy Paste: [[2503.04641]] Simulating the Real World: A Unified Survey of Multimodal Generative Models(https://arxiv.org/abs/2503.04641)
• Keywords: generative
• Abstract: Understanding and replicating the real world is a critical challenge in Artificial General Intelligence (AGI) research. To achieve this, many existing approaches, such as world models, aim to capture the fundamental principles governing the physical world, enabling more accurate simulations and meaningful interactions. However, current methods often treat different modalities, including 2D (images), videos, 3D, and 4D representations, as independent domains, overlooking their interdependencies. Additionally, these methods typically focus on isolated dimensions of reality without systematically integrating their connections. In this survey, we present a unified survey for multimodal generative models that investigate the progression of data dimensionality in real-world simulation. Specifically, this survey starts from 2D generation (appearance), then moves to video (appearance+dynamics) and 3D generation (appearance+geometry), and finally culminates in 4D generation that integrate all dimensions. To the best of our knowledge, this is the first attempt to systematically unify the study of 2D, video, 3D and 4D generation within a single framework. To guide future research, we provide a comprehensive review of datasets, evaluation metrics and future directions, and fostering insights for newcomers. This survey serves as a bridge to advance the study of multimodal generative models and real-world simulation within a unified framework.

Title: Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment

• Authors: Wen Yang, Junhong Wu, Chen Wang, Chengqing Zong, Jiajun Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04647
• Pdf URL: https://arxiv.org/pdf/2503.04647
• Copy Paste: [[2503.04647]] Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment(https://arxiv.org/abs/2503.04647)
• Keywords: large language model
• Abstract: Direct Preference Optimization (DPO) has become a prominent method for aligning Large Language Models (LLMs) with human preferences. While DPO has enabled significant progress in aligning English LLMs, multilingual preference alignment is hampered by data scarcity. To address this, we propose a novel approach that $\textit{captures}$ learned preferences from well-aligned English models by implicit rewards and $\textit{transfers}$ them to other languages through iterative training. Specifically, we derive an implicit reward model from the logits of an English DPO-aligned model and its corresponding reference model. This reward model is then leveraged to annotate preference relations in cross-lingual instruction-following pairs, using English instructions to evaluate multilingual responses. The annotated data is subsequently used for multilingual DPO fine-tuning, facilitating preference knowledge transfer from English to other languages. Fine-tuning Llama3 for two iterations resulted in a 12.72% average improvement in Win Rate and a 5.97% increase in Length Control Win Rate across all training languages on the X-AlpacaEval leaderboard. Our findings demonstrate that leveraging existing English-aligned models can enable efficient and effective multilingual preference alignment, significantly reducing the need for extensive multilingual preference data. The code is available at this https URL

Title: Transferable Foundation Models for Geometric Tasks on Point Cloud Representations: Geometric Neural Operators

• Authors: Blaine Quackenbush, Paul J. Atzberger
• Subjects: cs.LG, cs.CV, math.NA, math.OC
• Abstract URL: https://arxiv.org/abs/2503.04649
• Pdf URL: https://arxiv.org/pdf/2503.04649
• Copy Paste: [[2503.04649]] Transferable Foundation Models for Geometric Tasks on Point Cloud Representations: Geometric Neural Operators(https://arxiv.org/abs/2503.04649)
• Keywords: robust
• Abstract: We introduce methods for obtaining pretrained Geometric Neural Operators (GNPs) that can serve as basal foundation models for use in obtaining geometric features. These can be used within data processing pipelines for machine learning tasks and numerical methods. We show how our GNPs can be trained to learn robust latent representations for the differential geometry of point-clouds to provide estimates of metric, curvature, and other shape-related features. We demonstrate how our pre-trained GNPs can be used (i) to estimate the geometric properties of surfaces of arbitrary shape and topologies with robustness in the presence of noise, (ii) to approximate solutions of geometric partial differential equations (PDEs) on manifolds, and (iii) to solve equations for shape deformations such as curvature driven flows. We also release a package of the codes and weights for using our pre-trained GNPs for processing point cloud representations. This allows for incorporating our pre-trained GNPs as components for reuse within existing and new data processing pipelines. The GNPs also can be used as part of numerical solvers involving geometry or as part of methods for performing inference and other geometric tasks.

Title: Joint Masked Reconstruction and Contrastive Learning for Mining Interactions Between Proteins

• Authors: Jiang Li, Xiaoping Wang
• Subjects: cs.LG, cs.ET
• Abstract URL: https://arxiv.org/abs/2503.04650
• Pdf URL: https://arxiv.org/pdf/2503.04650
• Copy Paste: [[2503.04650]] Joint Masked Reconstruction and Contrastive Learning for Mining Interactions Between Proteins(https://arxiv.org/abs/2503.04650)
• Keywords: extraction
• Abstract: Protein-protein interaction (PPI) prediction is an instrumental means in elucidating the mechanisms underlying cellular operations, holding significant practical implications for the realms of pharmaceutical development and clinical treatment. Presently, the majority of research methods primarily concentrate on the analysis of amino acid sequences, while investigations predicated on protein structures remain in the nascent stages of exploration. Despite the emergence of several structure-based algorithms in recent years, these are still confronted with inherent challenges: (1) the extraction of intrinsic structural information of proteins typically necessitates the expenditure of substantial computational resources; (2) these models are overly reliant on seen protein data, struggling to effectively unearth interaction cues between unknown proteins. To further propel advancements in this domain, this paper introduces a novel PPI prediction method jointing masked reconstruction and contrastive learning, termed JmcPPI. This methodology dissects the PPI prediction task into two distinct phases: during the residue structure encoding phase, JmcPPI devises two feature reconstruction tasks and employs graph attention mechanism to capture structural information between residues; during the protein interaction inference phase, JmcPPI perturbs the original PPI graph and employs a multi-graph contrastive learning strategy to thoroughly mine extrinsic interaction information of novel proteins. Extensive experiments conducted on three widely utilized PPI datasets demonstrate that JmcPPI surpasses existing optimal baseline models across various data partition schemes. The associated code can be accessed via this https URL.

Title: Evaluation of Privacy-aware Support Vector Machine (SVM) Learning using Homomorphic Encryption

• Authors: William J Buchanan, Hisham Ali
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2503.04652
• Pdf URL: https://arxiv.org/pdf/2503.04652
• Copy Paste: [[2503.04652]] Evaluation of Privacy-aware Support Vector Machine (SVM) Learning using Homomorphic Encryption(https://arxiv.org/abs/2503.04652)
• Keywords: security, privacy
• Abstract: The requirement for privacy-aware machine learning increases as we continue to use PII (Personally Identifiable Information) within machine training. To overcome these privacy issues, we can apply Fully Homomorphic Encryption (FHE) to encrypt data before it is fed into a machine learning model. This involves creating a homomorphic encryption key pair, and where the associated public key will be used to encrypt the input data, and the private key will decrypt the output. But, there is often a performance hit when we use homomorphic encryption, and so this paper evaluates the performance overhead of using the SVM machine learning technique with the OpenFHE homomorphic encryption library. This uses Python and the scikit-learn library for its implementation. The experiments include a range of variables such as multiplication depth, scale size, first modulus size, security level, batch size, and ring dimension, along with two different SVM models, SVM-Poly and SVM-Linear. Overall, the results show that the two main parameters which affect performance are the ring dimension and the modulus size, and that SVM-Poly and SVM-Linear show similar performance levels.

Title: An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding

• Authors: Dou Hu, Lingwei Wei, Wei Zhou, Songlin Hu
• Subjects: cs.CL, cs.IT, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04667
• Pdf URL: https://arxiv.org/pdf/2503.04667
• Copy Paste: [[2503.04667]] An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding(https://arxiv.org/abs/2503.04667)
• Keywords: robust
• Abstract: This paper proposes a new principled multi-task representation learning framework (InfoMTL) to extract noise-invariant sufficient representations for all tasks. It ensures sufficiency of shared representations for all tasks and mitigates the negative effect of redundant features, which can enhance language understanding of pre-trained language models (PLMs) under the multi-task paradigm. Firstly, a shared information maximization principle is proposed to learn more sufficient shared representations for all target tasks. It can avoid the insufficiency issue arising from representation compression in the multi-task paradigm. Secondly, a task-specific information minimization principle is designed to mitigate the negative effect of potential redundant features in the input for each task. It can compress task-irrelevant redundant information and preserve necessary information relevant to the target for multi-task prediction. Experiments on six classification benchmarks show that our method outperforms 12 comparative multi-task methods under the same multi-task settings, especially in data-constrained and noisy scenarios. Extensive experiments demonstrate that the learned representations are more sufficient, data-efficient, and robust.

Title: LLM-guided Plan and Retrieval: A Strategic Alignment for Interpretable User Satisfaction Estimation in Dialogue

• Authors: Sangyeop Kim, Sohhyung Park, Jaewon Jung, Jinseok Kim, Sungzoon Cho
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04675
• Pdf URL: https://arxiv.org/pdf/2503.04675
• Copy Paste: [[2503.04675]] LLM-guided Plan and Retrieval: A Strategic Alignment for Interpretable User Satisfaction Estimation in Dialogue(https://arxiv.org/abs/2503.04675)
• Keywords: interpretability, large language model
• Abstract: Understanding user satisfaction with conversational systems, known as User Satisfaction Estimation (USE), is essential for assessing dialogue quality and enhancing user experiences. However, existing methods for USE face challenges due to limited understanding of underlying reasons for user dissatisfaction and the high costs of annotating user intentions. To address these challenges, we propose PRAISE (Plan and Retrieval Alignment for Interpretable Satisfaction Estimation), an interpretable framework for effective user satisfaction prediction. PRAISE operates through three key modules. The Strategy Planner develops strategies, which are natural language criteria for classifying user satisfaction. The Feature Retriever then incorporates knowledge on user satisfaction from Large Language Models (LLMs) and retrieves relevance features from utterances. Finally, the Score Analyzer evaluates strategy predictions and classifies user satisfaction. Experimental results demonstrate that PRAISE achieves state-of-the-art performance on three benchmarks for the USE task. Beyond its superior performance, PRAISE offers additional benefits. It enhances interpretability by providing instance-level explanations through effective alignment of utterances with strategies. Moreover, PRAISE operates more efficiently than existing approaches by eliminating the need for LLMs during the inference phase.

Title: Matrix Factorization for Inferring Associations and Missing Links

• Authors: Ryan Barron, Maksim E. Eren, Duc P. Truong, Cynthia Matuszek, James Wendelberger, Mary F. Dorn, Boian Alexandrov
• Subjects: cs.LG, cs.AI, cs.LO
• Abstract URL: https://arxiv.org/abs/2503.04680
• Pdf URL: https://arxiv.org/pdf/2503.04680
• Copy Paste: [[2503.04680]] Matrix Factorization for Inferring Associations and Missing Links(https://arxiv.org/abs/2503.04680)
• Keywords: security
• Abstract: Missing link prediction is a method for network analysis, with applications in recommender systems, biology, social sciences, cybersecurity, information retrieval, and Artificial Intelligence (AI) reasoning in Knowledge Graphs. Missing link prediction identifies unseen but potentially existing connections in a network by analyzing the observed patterns and relationships. In proliferation detection, this supports efforts to identify and characterize attempts by state and non-state actors to acquire nuclear weapons or associated technology - a notoriously challenging but vital mission for global security. Dimensionality reduction techniques like Non-Negative Matrix Factorization (NMF) and Logistic Matrix Factorization (LMF) are effective but require selection of the matrix rank parameter, that is, of the number of hidden features, k, to avoid over/under-fitting. We introduce novel Weighted (WNMFk), Boolean (BNMFk), and Recommender (RNMFk) matrix factorization methods, along with ensemble variants incorporating logistic factorization, for link prediction. Our methods integrate automatic model determination for rank estimation by evaluating stability and accuracy using a modified bootstrap methodology and uncertainty quantification (UQ), assessing prediction reliability under random perturbations. We incorporate Otsu threshold selection and k-means clustering for Boolean matrix factorization, comparing them to coordinate descent-based Boolean thresholding. Our experiments highlight the impact of rank k selection, evaluate model performance under varying test-set sizes, and demonstrate the benefits of UQ for reliable predictions using abstention. We validate our methods on three synthetic datasets (Boolean and uniformly distributed) and benchmark them against LMF and symmetric LMF (symLMF) on five real-world protein-protein interaction networks, showcasing an improved prediction performance.

Title: Compositional World Knowledge leads to High Utility Synthetic data

• Authors: Sachit Gaudi, Gautam Sreekumar, Vishnu Boddeti
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04687
• Pdf URL: https://arxiv.org/pdf/2503.04687
• Copy Paste: [[2503.04687]] Compositional World Knowledge leads to High Utility Synthetic data(https://arxiv.org/abs/2503.04687)
• Keywords: robust, diffusion
• Abstract: Machine learning systems struggle with robustness, under subpopulation shifts. This problem becomes especially pronounced in scenarios where only a subset of attribute combinations is observed during training -a severe form of subpopulation shift, referred as compositional shift. To address this problem, we ask the following question: Can we improve the robustness by training on synthetic data, spanning all possible attribute combinations? We first show that training of conditional diffusion models on limited data lead to incorrect underlying distribution. Therefore, synthetic data sampled from such models will result in unfaithful samples and does not lead to improve performance of downstream machine learning systems. To address this problem, we propose CoInD to reflect the compositional nature of the world by enforcing conditional independence through minimizing Fisher's divergence between joint and marginal distributions. We demonstrate that synthetic data generated by CoInD is faithful and this translates to state-of-the-art worst-group accuracy on compositional shift tasks on CelebA.

Title: Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases

• Authors: Pengcheng Qiu, Chaoyi Wu, Shuyu Liu, Weike Zhao, Ya Zhang, Yanfeng Wang, Weidi Xie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04691
• Pdf URL: https://arxiv.org/pdf/2503.04691
• Copy Paste: [[2503.04691]] Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases(https://arxiv.org/abs/2503.04691)
• Keywords: large language model
• Abstract: The latest reasoning-enhanced large language models (reasoning LLMs), such as DeepSeek-R1 and OpenAI-o3, have demonstrated remarkable success. However, the application of such reasoning enhancements to the highly professional medical domain has not been clearly evaluated, particularly regarding with not only assessing the final generation but also examining the quality of their reasoning processes. In this study, we present MedR-Bench, a reasoning-focused medical evaluation benchmark comprising 1,453 structured patient cases with reasoning references mined from case reports. Our benchmark spans 13 body systems and 10 specialty disorders, encompassing both common and rare diseases. In our evaluation, we introduce a versatile framework consisting of three critical clinical stages: assessment recommendation, diagnostic decision-making, and treatment planning, comprehensively capturing the LLMs' performance across the entire patient journey in healthcare. For metrics, we propose a novel agentic system, Reasoning Evaluator, designed to automate and objectively quantify free-text reasoning responses in a scalable manner from the perspectives of efficiency, factuality, and completeness by dynamically searching and performing cross-referencing checks. As a result, we assess five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and others. Our results reveal that current LLMs can handle relatively simple diagnostic tasks with sufficient critical assessment results, achieving accuracy generally over 85%. However, they still struggle with more complex tasks, such as assessment recommendation and treatment planning. In reasoning, their reasoning processes are generally reliable, with factuality scores exceeding 90%, though they often omit critical reasoning steps. Our study clearly reveals further development directions for current clinical LLMs.

Title: UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets

• Authors: Wenyu Wang, Mengqi Zhang, Xiaotian Ye, Zhaochun Ren, Zhumin Chen, Pengjie Ren
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2503.04693
• Pdf URL: https://arxiv.org/pdf/2503.04693
• Copy Paste: [[2503.04693]] UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets(https://arxiv.org/abs/2503.04693)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) inevitably acquire harmful information during training on massive datasets. LLM unlearning aims to eliminate the influence of such harmful information while maintaining the model's overall performance. Existing unlearning methods, represented by gradient ascent-based approaches, primarily focus on forgetting target data while overlooking the crucial impact of logically related knowledge on the effectiveness of unlearning. In this paper, through both theoretical and experimental analyses, we first demonstrate that a key reason for the suboptimal unlearning performance is that models can reconstruct the target content through reasoning with logically related knowledge. To address this issue, we propose Unlearning Improvement via Parameter Extrapolation (UIPE), a method that removes knowledge highly correlated with the forgetting targets. Experimental results show that UIPE significantly enhances the performance of various mainstream LLM unlearning methods on the TOFU benchmark.

Title: DEAL-YOLO: Drone-based Efficient Animal Localization using YOLO

• Authors: Aditya Prashant Naidu, Hem Gosalia, Ishaan Gakhar, Shaurya Singh Rathore, Krish Didwania, Ujjwal Verma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04698
• Pdf URL: https://arxiv.org/pdf/2503.04698
• Copy Paste: [[2503.04698]] DEAL-YOLO: Drone-based Efficient Animal Localization using YOLO(https://arxiv.org/abs/2503.04698)
• Keywords: robust, extraction
• Abstract: Although advances in deep learning and aerial surveillance technology are improving wildlife conservation efforts, complex and erratic environmental conditions still pose a problem, requiring innovative solutions for cost-effective small animal detection. This work introduces DEAL-YOLO, a novel approach that improves small object detection in Unmanned Aerial Vehicle (UAV) images by using multi-objective loss functions like Wise IoU (WIoU) and Normalized Wasserstein Distance (NWD), which prioritize pixels near the centre of the bounding box, ensuring smoother localization and reducing abrupt deviations. Additionally, the model is optimized through efficient feature extraction with Linear Deformable (LD) convolutions, enhancing accuracy while maintaining computational efficiency. The Scaled Sequence Feature Fusion (SSFF) module enhances object detection by effectively capturing inter-scale relationships, improving feature representation, and boosting metrics through optimized multiscale fusion. Comparison with baseline models reveals high efficacy with up to 69.5\% fewer parameters compared to vanilla Yolov8-N, highlighting the robustness of the proposed modifications. Through this approach, our paper aims to facilitate the detection of endangered species, animal population analysis, habitat monitoring, biodiversity research, and various other applications that enrich wildlife conservation efforts. DEAL-YOLO employs a two-stage inference paradigm for object detection, refining selected regions to improve localization and confidence. This approach enhances performance, especially for small instances with low objectness scores.

Title: Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size

• Authors: Alireza Behtash, Marijan Fofonjka, Ethan Baird, Tyler Mauer, Hossein Moghimifam, David Stout, Joel Dennison
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04704
• Pdf URL: https://arxiv.org/pdf/2503.04704
• Copy Paste: [[2503.04704]] Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size(https://arxiv.org/abs/2503.04704)
• Keywords: transformer, large language model
• Abstract: We present a novel approach to selective model quantization that transcends the limitations of architecture-specific and size-dependent compression methods for Large Language Models (LLMs) using Entropy-Weighted Quantization (EWQ). By analyzing the entropy distribution across transformer blocks, EWQ determines which blocks can be safely quantized without causing significant performance degradation, independent of model architecture or size. Our method outperforms uniform quantization approaches, maintaining Massive Multitask Language Understanding (MMLU) accuracy scores within 0.5% of unquantized models while reducing memory usage by up to 18%. We demonstrate the effectiveness of EWQ across multiple architectures-from 1.6B to 70B parameters-showcasing consistent improvements in the quality-compression trade-off regardless of model scale or architectural design. A surprising finding of EWQ is its ability to reduce perplexity compared to unquantized models, suggesting the presence of beneficial regularization through selective precision reduction. This improvement holds across different model families, indicating a fundamental relationship between layer-level entropy and optimal precision requirements. Additionally, we introduce FastEWQ, a rapid method for entropy distribution analysis that eliminates the need for loading model weights. This technique leverages universal characteristics of entropy distribution that persist across various architectures and scales, enabling near-instantaneous quantization decisions while maintaining 80% classification accuracy with full entropy analysis. Our results demonstrate that effective quantization strategies can be developed independently of specific architectural choices or model sizes, opening new possibilities for efficient LLM deployment.

Title: Iris Style Transfer: Enhancing Iris Recognition with Style Features and Privacy Preservation through Neural Style Transfer

• Authors: Mengdi Wang, Efe Bozkir, Enkelejda Kasneci
• Subjects: cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.04707
• Pdf URL: https://arxiv.org/pdf/2503.04707
• Copy Paste: [[2503.04707]] Iris Style Transfer: Enhancing Iris Recognition with Style Features and Privacy Preservation through Neural Style Transfer(https://arxiv.org/abs/2503.04707)
• Keywords: secure, security, privacy, protect, attack, robust, biometric, segmentation
• Abstract: Iris texture is widely regarded as a gold standard biometric modality for authentication and identification. The demand for robust iris recognition methods, coupled with growing security and privacy concerns regarding iris attacks, has escalated recently. Inspired by neural style transfer, an advanced technique that leverages neural networks to separate content and style features, we hypothesize that iris texture's style features provide a reliable foundation for recognition and are more resilient to variations like rotation and perspective shifts than traditional approaches. Our experimental results support this hypothesis, showing a significantly higher classification accuracy compared to conventional features. Further, we propose using neural style transfer to mask identifiable iris style features, ensuring the protection of sensitive biometric information while maintaining the utility of eye images for tasks like eye segmentation and gaze estimation. This work opens new avenues for iris-oriented, secure, and privacy-aware biometric systems.

Title: Predictable Scale: Part I -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining

• Authors: Houyi Li, Wenzheng Zheng, Jingcheng Hu, Qiufeng Wang, Hanshan Zhang, Zili Wang, Yangshijie Xu, Shuigeng Zhou, Xiangyu Zhang, Daxin Jiang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04715
• Pdf URL: https://arxiv.org/pdf/2503.04715
• Copy Paste: [[2503.04715]] Predictable Scale: Part I -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining(https://arxiv.org/abs/2503.04715)
• Keywords: robust, transformer, large language model
• Abstract: The impressive capabilities of Large Language Models (LLMs) across diverse tasks are now well-established, yet their effective deployment necessitates careful hyperparameter optimization. Through extensive empirical studies involving grid searches across diverse configurations, we discover universal scaling laws governing these hyperparameters: optimal learning rate follows a power-law relationship with both model parameters and data sizes, while optimal batch size scales primarily with data sizes. Our analysis reveals a convex optimization landscape for hyperparameters under fixed models and data size conditions. This convexity implies an optimal hyperparameter plateau. We contribute a universal, plug-and-play optimal hyperparameter tool for the community. Its estimated values on the test set are merely 0.07\% away from the globally optimal LLM performance found via an exhaustive search. These laws demonstrate remarkable robustness across variations in model sparsity, training data distribution, and model shape. To our best known, this is the first work that unifies different model shapes and structures, such as Mixture-of-Experts models and dense transformers, as well as establishes optimal hyperparameter scaling laws across diverse data distributions. This exhaustive optimization process demands substantial computational resources, utilizing nearly one million NVIDIA H800 GPU hours to train 3,700 LLMs of varying sizes and hyperparameters from scratch and consuming approximately 100 trillion tokens in total. To facilitate reproducibility and further research, we will progressively release all loss measurements and model checkpoints through our designated repository this https URL

Title: Floxels: Fast Unsupervised Voxel Based Scene Flow Estimation

• Authors: David T. Hoffmann, Syed Haseeb Raza, Hanqiu Jiang, Denis Tananaev, Steffen Klingenhoefer, Martin Meinke
• Subjects: cs.CV, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.04718
• Pdf URL: https://arxiv.org/pdf/2503.04718
• Copy Paste: [[2503.04718]] Floxels: Fast Unsupervised Voxel Based Scene Flow Estimation(https://arxiv.org/abs/2503.04718)
• Keywords: robust
• Abstract: Scene flow estimation is a foundational task for many robotic applications, including robust dynamic object detection, automatic labeling, and sensor synchronization. Two types of approaches to the problem have evolved: 1) Supervised and 2) optimization-based methods. Supervised methods are fast during inference and achieve high-quality results, however, they are limited by the need for large amounts of labeled training data and are susceptible to domain gaps. In contrast, unsupervised test-time optimization methods do not face the problem of domain gaps but usually suffer from substantial runtime, exhibit artifacts, or fail to converge to the right solution. In this work, we mitigate several limitations of existing optimization-based methods. To this end, we 1) introduce a simple voxel grid-based model that improves over the standard MLP-based formulation in multiple dimensions and 2) introduce a new multiframe loss formulation. 3) We combine both contributions in our new method, termed Floxels. On the Argoverse 2 benchmark, Floxels is surpassed only by EulerFlow among unsupervised methods while achieving comparable performance at a fraction of the computational cost. Floxels achieves a massive speedup of more than ~60 - 140x over EulerFlow, reducing the runtime from a day to 10 minutes per sequence. Over the faster but low-quality baseline, NSFP, Floxels achieves a speedup of ~14x.

Title: FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video

• Authors: Yue Gao, Hong-Xing Yu, Bo Zhu, Jiajun Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.04720
• Pdf URL: https://arxiv.org/pdf/2503.04720
• Copy Paste: [[2503.04720]] FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video(https://arxiv.org/abs/2503.04720)
• Keywords: diffusion
• Abstract: We study reconstructing and predicting 3D fluid appearance and velocity from a single video. Current methods require multi-view videos for fluid reconstruction. We present FluidNexus, a novel framework that bridges video generation and physics simulation to tackle this task. Our key insight is to synthesize multiple novel-view videos as references for reconstruction. FluidNexus consists of two key components: (1) a novel-view video synthesizer that combines frame-wise view synthesis with video diffusion refinement for generating realistic videos, and (2) a physics-integrated particle representation coupling differentiable simulation and rendering to simultaneously facilitate 3D fluid reconstruction and prediction. To evaluate our approach, we collect two new real-world fluid datasets featuring textured backgrounds and object interactions. Our method enables dynamic novel view synthesis, future prediction, and interaction simulation from a single fluid video. Project website: this https URL.

Title: Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities

• Authors: Guan-Ting Lin, Jiachen Lian, Tingle Li, Qirui Wang, Gopala Anumanchipalli, Alexander H. Liu, Hung-yi Lee
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2503.04721
• Pdf URL: https://arxiv.org/pdf/2503.04721
• Copy Paste: [[2503.04721]] Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities(https://arxiv.org/abs/2503.04721)
• Keywords: robust
• Abstract: Spoken dialogue modeling introduces unique challenges beyond text-based language modeling, demanding robust turn-taking, backchanneling, and real-time interaction. Although most Spoken Dialogue Models (SDMs) rely on half-duplex processing (handling speech one turn at a time), emerging full-duplex SDMs can listen and speak simultaneously, enabling more natural and engaging conversations. However, current evaluations of such models remain limited, often focusing on turn-based metrics or high-level corpus analyses (e.g., turn gaps, pauses). To address this gap, we present Full-Duplex-Bench, a new benchmark that systematically evaluates key conversational behaviors: pause handling, backchanneling, turn-taking, and interruption management. Our framework uses automatic metrics for consistent and reproducible assessments of SDMs' interactive performance. By offering an open and standardized evaluation benchmark, we aim to advance spoken dialogue modeling and encourage the development of more interactive and natural dialogue systems.

Title: Enough Coin Flips Can Make LLMs Act Bayesian

• Authors: Ritwik Gupta, Rodolfo Corona, Jiaxin Ge, Eric Wang, Dan Klein, Trevor Darrell, David M. Chan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.04722
• Pdf URL: https://arxiv.org/pdf/2503.04722
• Copy Paste: [[2503.04722]] Enough Coin Flips Can Make LLMs Act Bayesian(https://arxiv.org/abs/2503.04722)
• Keywords: large language model
• Abstract: Large language models (LLMs) exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning (ICL). We investigate whether LLMs utilize ICL to perform structured reasoning in ways that are consistent with a Bayesian framework or rely on pattern matching. Using a controlled setting of biased coin flips, we find that: (1) LLMs often possess biased priors, causing initial divergence in zero-shot settings, (2) in-context evidence outweighs explicit bias instructions, (3) LLMs broadly follow Bayesian posterior updates, with deviations primarily due to miscalibrated priors rather than flawed updates, and (4) attention magnitude has negligible effect on Bayesian inference. With sufficient demonstrations of biased coin flips via ICL, LLMs update their priors in a Bayesian manner.

Title: Shifting Long-Context LLMs Research from Input to Output

• Authors: Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, Roy Ka-Wei Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.04723
• Pdf URL: https://arxiv.org/pdf/2503.04723
• Copy Paste: [[2503.04723]] Shifting Long-Context LLMs Research from Input to Output(https://arxiv.org/abs/2503.04723)
• Keywords: large language model
• Abstract: Recent advancements in long-context Large Language Models (LLMs) have primarily concentrated on processing extended input contexts, resulting in significant strides in long-context comprehension. However, the equally critical aspect of generating long-form outputs has received comparatively less attention. This paper advocates for a paradigm shift in NLP research toward addressing the challenges of long-output generation. Tasks such as novel writing, long-term planning, and complex reasoning require models to understand extensive contexts and produce coherent, contextually rich, and logically consistent extended text. These demands highlight a critical gap in current LLM capabilities. We underscore the importance of this under-explored domain and call for focused efforts to develop foundational LLMs tailored for generating high-quality, long-form outputs, which hold immense potential for real-world applications.

Title: L$^2$M: Mutual Information Scaling Law for Long-Context Language Modeling

• Authors: Zhuo Chen, Oriol Mayné i Comas, Zhuotao Jin, Di Luo, Marin Soljačić
• Subjects: cs.CL, cs.AI, cs.IT, cs.LG, physics.data-an
• Abstract URL: https://arxiv.org/abs/2503.04725
• Pdf URL: https://arxiv.org/pdf/2503.04725
• Copy Paste: [[2503.04725]] L$^2$M: Mutual Information Scaling Law for Long-Context Language Modeling(https://arxiv.org/abs/2503.04725)
• Keywords: transformer, large language model
• Abstract: We rigorously establish a bipartite mutual information scaling law in natural language that governs long-range dependencies. This scaling law, which we show is distinct from and scales independently of the conventional two-point mutual information, is the key to understanding long-context language modeling. Using this scaling law, we formulate the Long-context Language Modeling (L$^2$M) condition, which relates a model's capacity for effective long context length modeling to the scaling of its latent state size for storing past information. Our results are validated through experiments on both transformers and state space models. This work establishes a theoretical foundation that guides the development of large language models toward longer context lengths.