2025-01-14

Title: Enhancing AI Safety Through the Fusion of Low Rank Adapters

• Authors: Satya Swaroop Gudipudi, Sreeram Vipparla, Harpreet Singh, Shashwat Goel, Ponnurangam Kumaraguru
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06208
• Pdf URL: https://arxiv.org/pdf/2501.06208
• Copy Paste: [[2501.06208]] Enhancing AI Safety Through the Fusion of Low Rank Adapters(https://arxiv.org/abs/2501.06208)
• Keywords: large language model
• Abstract: Instruction fine-tuning of large language models (LLMs) is a powerful method for improving task-specific performance, but it can inadvertently lead to a phenomenon where models generate harmful responses when faced with malicious prompts. In this paper, we explore Low-Rank Adapter Fusion (LoRA) as a means to mitigate these risks while preserving the model's ability to handle diverse instructions effectively. Through an extensive comparative analysis against established baselines using recognized benchmark datasets, we demonstrate a 42\% reduction in the harmfulness rate by leveraging LoRA fusion between a task adapter and a safety adapter, the latter of which is specifically trained on our safety dataset. However, we also observe exaggerated safety behaviour, where the model rejects safe prompts that closely resemble unsafe ones

Title: Dissecting Bit-Level Scaling Laws in Quantizing Vision Generative Models

• Authors: Xin Ding, Shijie Cao, Ting Cao, Zhibo Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06218
• Pdf URL: https://arxiv.org/pdf/2501.06218
• Copy Paste: [[2501.06218]] Dissecting Bit-Level Scaling Laws in Quantizing Vision Generative Models(https://arxiv.org/abs/2501.06218)
• Keywords: diffusion, generative
• Abstract: Vision generative models have recently made significant advancements along two primary paradigms: diffusion-style and language-style, both of which have demonstrated excellent scaling laws. Quantization is crucial for efficiently deploying these models, as it reduces memory and computation costs. In this work, we systematically investigate the impact of quantization on these two paradigms. Surprisingly, despite achieving comparable performance in full precision, language-style models consistently outperform diffusion-style models across various quantization settings. This observation suggests that language-style models have superior bit-level scaling laws, offering a better tradeoff between model quality and total bits. To dissect this phenomenon, we conduct extensive experiments and find that the primary reason is the discrete representation space of language-style models, which is more tolerant of information loss during quantization. Furthermore, our analysis indicates that improving the bit-level scaling law of quantized vision generative models is challenging, with model distillation identified as a highly effective approach. Specifically, we propose TopKLD to optimize the transfer of distilled knowledge by balancing ``implicit knowledge'' and ``explicit knowledge'' during the distillation process. This approach elevates the bit-level scaling laws by one level across both integer and floating-point quantization settings.

Title: WhACC: Whisker Automatic Contact Classifier with Expert Human-Level Performance

• Authors: Phillip Maire, Samson G. King, Jonathan Andrew Cheung, Stefanie Walker, Samuel Andrew Hires
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06219
• Pdf URL: https://arxiv.org/pdf/2501.06219
• Copy Paste: [[2501.06219]] WhACC: Whisker Automatic Contact Classifier with Expert Human-Level Performance(https://arxiv.org/abs/2501.06219)
• Keywords: extraction
• Abstract: The rodent vibrissal system is pivotal in advancing neuroscience research, particularly for studies of cortical plasticity, learning, decision-making, sensory encoding, and sensorimotor integration. Despite the advantages, curating touch events is labor intensive and often requires >3 hours per million video frames, even after leveraging automated tools like the Janelia Whisker Tracker. We address this limitation by introducing Whisker Automatic Contact Classifier (WhACC), a python package designed to identify touch periods from high-speed videos of head-fixed behaving rodents with human-level performance. WhACC leverages ResNet50V2 for feature extraction, combined with LightGBM for Classification. Performance is assessed against three expert human curators on over one million frames. Pairwise touch classification agreement on 99.5% of video frames, equal to between-human agreement. Finally, we offer a custom retraining interface to allow model customization on a small subset of data, which was validated on four million frames across 16 single-unit electrophysiology recordings. Including this retraining step, we reduce human hours required to curate a 100 million frame dataset from ~333 hours to ~6 hours.

Title: Powerful Design of Small Vision Transformer on CIFAR10

• Authors: Gent Wu
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06220
• Pdf URL: https://arxiv.org/pdf/2501.06220
• Copy Paste: [[2501.06220]] Powerful Design of Small Vision Transformer on CIFAR10(https://arxiv.org/abs/2501.06220)
• Keywords: transformer
• Abstract: Vision Transformers (ViTs) have demonstrated remarkable success on large-scale datasets, but their performance on smaller datasets often falls short of convolutional neural networks (CNNs). This paper explores the design and optimization of Tiny ViTs for small datasets, using CIFAR-10 as a benchmark. We systematically evaluate the impact of data augmentation, patch token initialization, low-rank compression, and multi-class token strategies on model performance. Our experiments reveal that low-rank compression of queries in Multi-Head Latent Attention (MLA) incurs minimal performance loss, indicating redundancy in ViTs. Additionally, introducing multiple CLS tokens improves global representation capacity, boosting accuracy. These findings provide a comprehensive framework for optimizing Tiny ViTs, offering practical insights for efficient and effective designs. Code is available at this https URL.

Title: Optimizing Supply Chain Networks with the Power of Graph Neural Networks

• Authors: Chi-Sheng Chen, Ying-Jung Chen
• Subjects: cs.LG, econ.GN
• Abstract URL: https://arxiv.org/abs/2501.06221
• Pdf URL: https://arxiv.org/pdf/2501.06221
• Copy Paste: [[2501.06221]] Optimizing Supply Chain Networks with the Power of Graph Neural Networks(https://arxiv.org/abs/2501.06221)
• Keywords: robust
• Abstract: Graph Neural Networks (GNNs) have emerged as transformative tools for modeling complex relational data, offering unprecedented capabilities in tasks like forecasting and optimization. This study investigates the application of GNNs to demand forecasting within supply chain networks using the SupplyGraph dataset, a benchmark for graph-based supply chain analysis. By leveraging advanced GNN methodologies, we enhance the accuracy of forecasting models, uncover latent dependencies, and address temporal complexities inherent in supply chain operations. Comparative analyses demonstrate that GNN-based models significantly outperform traditional approaches, including Multilayer Perceptrons (MLPs) and Graph Convolutional Networks (GCNs), particularly in single-node demand forecasting tasks. The integration of graph representation learning with temporal data highlights GNNs' potential to revolutionize predictive capabilities for inventory management, production scheduling, and logistics optimization. This work underscores the pivotal role of forecasting in supply chain management and provides a robust framework for advancing research and applications in this domain.

Title: Can Explainable AI Assess Personalized Health Risks from Indoor Air Pollution?

• Authors: Pritisha Sarkar, Kushalava reddy Jala, Mousumi Saha
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06222
• Pdf URL: https://arxiv.org/pdf/2501.06222
• Copy Paste: [[2501.06222]] Can Explainable AI Assess Personalized Health Risks from Indoor Air Pollution?(https://arxiv.org/abs/2501.06222)
• Keywords: interpretability
• Abstract: Acknowledging the effects of outdoor air pollution, the literature inadequately addresses indoor air pollution's impacts. Despite daily health risks, existing research primarily focused on monitoring, lacking accuracy in pinpointing indoor pollution sources. In our research work, we thoroughly investigated the influence of indoor activities on pollution levels. A survey of 143 participants revealed limited awareness of indoor air pollution. Leveraging 65 days of diverse data encompassing activities like incense stick usage, indoor smoking, inadequately ventilated cooking, excessive AC usage, and accidental paper burning, we developed a comprehensive monitoring system. We identify pollutant sources and effects with high precision through clustering analysis and interpretability models (LIME and SHAP). Our method integrates Decision Trees, Random Forest, Naive Bayes, and SVM models, excelling at 99.8% accuracy with Decision Trees. Continuous 24-hour data allows personalized assessments for targeted pollution reduction strategies, achieving 91% accuracy in predicting activities and pollution exposure.

Title: Detection, Retrieval, and Explanation Unified: A Violence Detection System Based on Knowledge Graphs and GAT

• Authors: Wen-Dong Jiang, Chih-Yung Chang, Diptendu Sinha Roy
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06224
• Pdf URL: https://arxiv.org/pdf/2501.06224
• Copy Paste: [[2501.06224]] Detection, Retrieval, and Explanation Unified: A Violence Detection System Based on Knowledge Graphs and GAT(https://arxiv.org/abs/2501.06224)
• Keywords: interpretability, large language model
• Abstract: Recently, violence detection systems developed using unified multimodal models have achieved significant success and attracted widespread attention. However, most of these systems face two critical challenges: the lack of interpretability as black-box models and limited functionality, offering only classification or retrieval capabilities. To address these challenges, this paper proposes a novel interpretable violence detection system, termed the Three-in-One (TIO) System. The TIO system integrates knowledge graphs (KG) and graph attention networks (GAT) to provide three core functionalities: detection, retrieval, and explanation. Specifically, the system processes each video frame along with text descriptions generated by a large language model (LLM) for videos containing potential violent behavior. It employs ImageBind to generate high-dimensional embeddings for constructing a knowledge graph, uses GAT for reasoning, and applies lightweight time series modules to extract video embedding features. The final step connects a classifier and retriever for multi-functional outputs. The interpretability of KG enables the system to verify the reasoning process behind each output. Additionally, the paper introduces several lightweight methods to reduce the resource consumption of the TIO system and enhance its efficiency. Extensive experiments conducted on the XD-Violence and UCF-Crime datasets validate the effectiveness of the proposed system. A case study further reveals an intriguing phenomenon: as the number of bystanders increases, the occurrence of violent behavior tends to decrease.

Title: asanAI: In-Browser, No-Code, Offline-First Machine Learning Toolkit

• Authors: Norman Koch, Siavash Ghiasvand
• Subjects: cs.LG, cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.06226
• Pdf URL: https://arxiv.org/pdf/2501.06226
• Copy Paste: [[2501.06226]] asanAI: In-Browser, No-Code, Offline-First Machine Learning Toolkit(https://arxiv.org/abs/2501.06226)
• Keywords: privacy
• Abstract: Machine learning (ML) has become crucial in modern life, with growing interest from researchers and the public. Despite its potential, a significant entry barrier prevents widespread adoption, making it challenging for non-experts to understand and implement ML techniques. The increasing desire to leverage ML is counterbalanced by its technical complexity, creating a gap between potential and practical application. This work introduces asanAI, an offline-first, open-source, no-code machine learning toolkit designed for users of all skill levels. It allows individuals to design, debug, train, and test ML models directly in a web browser, eliminating the need for software installations and coding. The toolkit runs on any device with a modern web browser, including smartphones, and ensures user privacy through local computations while utilizing WebGL for enhanced GPU performance. Users can quickly experiment with neural networks and train custom models using various data sources, supported by intuitive visualizations of network structures and data flows. asanAI simplifies the teaching of ML concepts in educational settings and is released under an open-source MIT license, encouraging modifications. It also supports exporting models in industry-ready formats, empowering a diverse range of users to effectively learn and apply machine learning in their projects. The proposed toolkit is successfully utilized by researchers of this http URL to swiftly draft and test machine learning ideas, by trainers to effectively educate enthusiasts, and by teachers to introduce contemporary ML topics in classrooms with minimal effort and high clarity.

Title: Generating and Detecting Various Types of Fake Image and Audio Content: A Review of Modern Deep Learning Technologies and Tools

• Authors: Arash Dehghani, Hossein Saberi
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06227
• Pdf URL: https://arxiv.org/pdf/2501.06227
• Copy Paste: [[2501.06227]] Generating and Detecting Various Types of Fake Image and Audio Content: A Review of Modern Deep Learning Technologies and Tools(https://arxiv.org/abs/2501.06227)
• Keywords: security, privacy, robust, diffusion, generative
• Abstract: This paper reviews the state-of-the-art in deepfake generation and detection, focusing on modern deep learning technologies and tools based on the latest scientific advancements. The rise of deepfakes, leveraging techniques like Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Diffusion models and other generative models, presents significant threats to privacy, security, and democracy. This fake media can deceive individuals, discredit real people and organizations, facilitate blackmail, and even threaten the integrity of legal, political, and social systems. Therefore, finding appropriate solutions to counter the potential threats posed by this technology is essential. We explore various deepfake methods, including face swapping, voice conversion, reenactment and lip synchronization, highlighting their applications in both benign and malicious contexts. The review critically examines the ongoing "arms race" between deepfake generation and detection, analyzing the challenges in identifying manipulated contents. By examining current methods and highlighting future research directions, this paper contributes to a crucial understanding of this rapidly evolving field and the urgent need for robust detection strategies to counter the misuse of this powerful technology. While focusing primarily on audio, image, and video domains, this study allows the reader to easily grasp the latest advancements in deepfake generation and detection.

Title: Open-Source Manually Annotated Vocal Tract Database for Automatic Segmentation from 3D MRI Using Deep Learning: Benchmarking 2D and 3D Convolutional and Transformer Networks

• Authors: Subin Erattakulangara, Karthika Kelat, Katie Burnham, Rachel Balbi, Sarah E. Gerard, David Meyer, Sajan Goud Lingala
• Subjects: cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2501.06229
• Pdf URL: https://arxiv.org/pdf/2501.06229
• Copy Paste: [[2501.06229]] Open-Source Manually Annotated Vocal Tract Database for Automatic Segmentation from 3D MRI Using Deep Learning: Benchmarking 2D and 3D Convolutional and Transformer Networks(https://arxiv.org/abs/2501.06229)
• Keywords: transformer, segmentation
• Abstract: Accurate segmentation of the vocal tract from magnetic resonance imaging (MRI) data is essential for various voice and speech applications. Manual segmentation is time intensive and susceptible to errors. This study aimed to evaluate the efficacy of deep learning algorithms for automatic vocal tract segmentation from 3D MRI.

Title: BEN: Using Confidence-Guided Matting for Dichotomous Image Segmentation

• Authors: Maxwell Meyer, Jack Spruyt
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.06230
• Pdf URL: https://arxiv.org/pdf/2501.06230
• Copy Paste: [[2501.06230]] BEN: Using Confidence-Guided Matting for Dichotomous Image Segmentation(https://arxiv.org/abs/2501.06230)
• Keywords: segmentation
• Abstract: Current approaches to dichotomous image segmentation (DIS) treat image matting and object segmentation as fundamentally different tasks. As improvements in image segmentation become increasingly challenging to achieve, combining image matting and grayscale segmentation techniques offers promising new directions for architectural innovation. Inspired by the possibility of aligning these two model tasks, we propose a new architectural approach for DIS called Confidence-Guided Matting (CGM). We created the first CGM model called Background Erase Network (BEN). BEN is comprised of two components: BEN Base for initial segmentation and BEN Refiner for confidence refinement. Our approach achieves substantial improvements over current state-of-the-art methods on the DIS5K validation dataset, demonstrating that matting-based refinement can significantly enhance segmentation quality. This work opens new possibilities for cross-pollination between matting and segmentation techniques in computer vision.

Title: An Interpretable ML-based Model for Predicting p-y Curves of Monopile Foundations in Sand

• Authors: Biao Li, Qing-Kai Song, Wen-Gang Qi, Fu-Ping Gao
• Subjects: cs.LG, cond-mat.soft
• Abstract URL: https://arxiv.org/abs/2501.06232
• Pdf URL: https://arxiv.org/pdf/2501.06232
• Copy Paste: [[2501.06232]] An Interpretable ML-based Model for Predicting p-y Curves of Monopile Foundations in Sand(https://arxiv.org/abs/2501.06232)
• Keywords: interpretability
• Abstract: Predicting the lateral pile response is challenging due to the complexity of pile-soil interactions. Machine learning (ML) techniques have gained considerable attention for their effectiveness in non-linear analysis and prediction. This study develops an interpretable ML-based model for predicting p-y curves of monopile foundations. An XGBoost model was trained using a database compiled from existing research. The results demonstrate that the model achieves superior predictive accuracy. Shapley Additive Explanations (SHAP) was employed to enhance interpretability. The SHAP value distributions for each variable demonstrate strong alignment with established theoretical knowledge on factors affecting the lateral response of pile foundations.

Title: Mechanics and Design of Metastructured Auxetic Patches with Bio-inspired Materials

• Authors: Yingbin Chen, Milad Arzani, Xuan Mu, Sophia Jin, Shaoping Xiao
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2501.06233
• Pdf URL: https://arxiv.org/pdf/2501.06233
• Copy Paste: [[2501.06233]] Mechanics and Design of Metastructured Auxetic Patches with Bio-inspired Materials(https://arxiv.org/abs/2501.06233)
• Keywords: generative
• Abstract: Metastructured auxetic patches, characterized by negative Poisson's ratios, offer unique mechanical properties that closely resemble the behavior of human tissues and organs. As a result, these patches have gained significant attention for their potential applications in organ repair and tissue regeneration. This study focuses on neural networks-based computational modeling of auxetic patches with a sinusoidal metastructure fabricated from silk fibroin, a bio-inspired material known for its biocompatibility and strength. The primary objective of this research is to introduce a novel, data-driven framework for patch design. To achieve this, we conducted experimental fabrication and mechanical testing to determine material properties and validate the corresponding finite element models. Finite element simulations were then employed to generate the necessary data, while greedy sampling, an active learning technique, was utilized to reduce the computational cost associated with data labeling. Two neural networks were trained to accurately predict Poisson's ratios and stresses for strains up to 15\%, respectively. Both models achieved $R^2$ scores exceeding 0.995, which indicates highly reliable predictions. Building on this, we developed a neural network-based design model capable of tailoring patch designs to achieve specific mechanical properties. This model demonstrated superior performance when compared to traditional optimization methods, such as genetic algorithms, by providing more efficient and precise design solutions. The proposed framework represents a significant advancement in the design of bio-inspired metastructures for medical applications, paving the way for future innovations in tissue engineering and regenerative medicine.

Title: NextStop: An Improved Tracker For Panoptic LIDAR Segmentation Data

• Authors: Nirit Alkalay, Roy Orfaig, Ben-Zion Bobrovsky
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.06235
• Pdf URL: https://arxiv.org/pdf/2501.06235
• Copy Paste: [[2501.06235]] NextStop: An Improved Tracker For Panoptic LIDAR Segmentation Data(https://arxiv.org/abs/2501.06235)
• Keywords: segmentation
• Abstract: 4D panoptic LiDAR segmentation is essential for scene understanding in autonomous driving and robotics ,combining semantic and instance segmentation with temporal this http URL methods, like 4D-PLS and 4D-STOP, use a tracking-by-detection methodology, employing deep learning networks to perform semantic and instance segmentation on each frame. To maintain temporal consistency, large-size instances detected in the current frame are compared and associated with instances within a temporal window that includes the current and preceding frames. However, their reliance on short-term instance detection, lack of motion estimation, and exclusion of small-sized instances lead to frequent identity switches and reduced tracking performance. We address these issues with the NextStop1 tracker, which integrates Kalman filter-based motion estimation, data association, and lifespan management, along with a tracklet state concept to improve prioritization. Evaluated using the LiDAR Segmentation and Tracking Quality (LSTQ) metric on the SemanticKITTI validation set, NextStop demonstrated enhanced tracking performance, particularly for small-sized objects like people and bicyclists, with fewer ID switches, earlier tracking initiation, and improved reliability in complex environments. The source code is available at this https URL

Title: Data-Driven Radio Propagation Modeling using Graph Neural Networks

• Authors: Adrien Bufort, Laurent Lebocq, Stefan Cathabard
• Subjects: cs.LG, cs.AI, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.06236
• Pdf URL: https://arxiv.org/pdf/2501.06236
• Copy Paste: [[2501.06236]] Data-Driven Radio Propagation Modeling using Graph Neural Networks(https://arxiv.org/abs/2501.06236)
• Keywords: generative
• Abstract: Modeling radio propagation is essential for wireless network design and performance optimization. Traditional methods rely on physics models of radio propagation, which can be inaccurate or inflexible. In this work, we propose using graph neural networks to learn radio propagation behaviors directly from real-world network data. Our approach converts the radio propagation environment into a graph representation, with nodes corresponding to locations and edges representing spatial and ray-tracing relationships between locations. The graph is generated by converting images of the environment into a graph structure, with specific relationships between nodes. The model is trained on this graph representation, using sensor measurements as target data. We demonstrate that the graph neural network, which learns to predict radio propagation directly from data, achieves competitive performance compared to traditional heuristic models. This data-driven approach outperforms classic numerical solvers in terms of both speed and accuracy. To the best of our knowledge, we are the first to apply graph neural networks to real-world radio propagation data to generate coverage maps, enabling generative models of signal propagation with point measurements only.

Title: Forecasting Anonymized Electricity Load Profiles

• Authors: Joaquin Delgado Fernandez, Sergio Potenciano Menci, Alessio Magitteri
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06237
• Pdf URL: https://arxiv.org/pdf/2501.06237
• Copy Paste: [[2501.06237]] Forecasting Anonymized Electricity Load Profiles(https://arxiv.org/abs/2501.06237)
• Keywords: privacy, protect
• Abstract: In the evolving landscape of data privacy, the anonymization of electric load profiles has become a critical issue, especially with the enforcement of the General Data Protection Regulation (GDPR) in Europe. These electric load profiles, which are essential datasets in the energy industry, are classified as personal behavioral data, necessitating stringent protective measures. This article explores the implications of this classification, the importance of data anonymization, and the potential of forecasting using microaggregated data. The findings underscore that effective anonymization techniques, such as microaggregation, do not compromise the performance of forecasting models under certain conditions (i.e., forecasting aggregated). In such an aggregated level, microaggregated data maintains high levels of utility, with minimal impact on forecasting accuracy. The implications for the energy sector are profound, suggesting that privacy-preserving data practices can be integrated into smart metering technology applications without hindering their effectiveness.

Title: Towards a scalable AI-driven framework for data-independent Cyber Threat Intelligence Information Extraction

• Authors: Olga Sorokoletova, Emanuele Antonioni, Giordano Colò
• Subjects: cs.CR, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06239
• Pdf URL: https://arxiv.org/pdf/2501.06239
• Copy Paste: [[2501.06239]] Towards a scalable AI-driven framework for data-independent Cyber Threat Intelligence Information Extraction(https://arxiv.org/abs/2501.06239)
• Keywords: security, extraction, transformer
• Abstract: Cyber Threat Intelligence (CTI) is critical for mitigating threats to organizations, governments, and institutions, yet the necessary data are often dispersed across diverse formats. AI-driven solutions for CTI Information Extraction (IE) typically depend on high-quality, annotated data, which are not always available. This paper introduces 0-CTI, a scalable AI-based framework designed for efficient CTI Information Extraction. Leveraging advanced Natural Language Processing (NLP) techniques, particularly Transformer-based architectures, the proposed system processes complete text sequences of CTI reports to extract a cyber ontology of named entities and their relationships. Our contribution is the development of 0-CTI, the first modular framework for CTI Information Extraction that supports both supervised and zero-shot learning. Unlike existing state-of-the-art models that rely heavily on annotated datasets, our system enables fully dataless operation through zero-shot methods for both Entity and Relation Extraction, making it adaptable to various data availability scenarios. Additionally, our supervised Entity Extractor surpasses current state-of-the-art performance in cyber Entity Extraction, highlighting the dual strength of the framework in both low-resource and data-rich environments. By aligning the system's outputs with the Structured Threat Information Expression (STIX) format, a standard for information exchange in the cybersecurity domain, 0-CTI standardizes extracted knowledge, enhancing communication and collaboration in cybersecurity operations.

Title: Utility-inspired Reward Transformations Improve Reinforcement Learning Training of Language Models

• Authors: Roberto-Rafael Maura-Rivero, Chirag Nagpal, Roma Patel, Francesco Visin
• Subjects: cs.LG, cs.AI, cs.CL, econ.GN
• Abstract URL: https://arxiv.org/abs/2501.06248
• Pdf URL: https://arxiv.org/pdf/2501.06248
• Copy Paste: [[2501.06248]] Utility-inspired Reward Transformations Improve Reinforcement Learning Training of Language Models(https://arxiv.org/abs/2501.06248)
• Keywords: large language model
• Abstract: Current methods that train large language models (LLMs) with reinforcement learning feedback, often resort to averaging outputs of multiple rewards functions during training. This overlooks crucial aspects of individual reward dimensions and inter-reward dependencies that can lead to sub-optimal outcomes in generations. In this work, we show how linear aggregation of rewards exhibits some vulnerabilities that can lead to undesired properties of generated text. We then propose a transformation of reward functions inspired by economic theory of utility functions (specifically Inada conditions), that enhances sensitivity to low reward values while diminishing sensitivity to already high values. We compare our approach to the existing baseline methods that linearly aggregate rewards and show how the Inada-inspired reward feedback is superior to traditional weighted averaging. We quantitatively and qualitatively analyse the difference in the methods, and see that models trained with Inada-transformations score as more helpful while being less harmful.

Title: Generative AI for Cel-Animation: A Survey

• Authors: Yunlong Tang, Junjia Guo, Pinxin Liu, Zhiyuan Wang, Hang Hua, Jia-Xing Zhong, Yunzhong Xiao, Chao Huang, Luchuan Song, Susan Liang, Yizhi Song, Liu He, Jing Bi, Mingqian Feng, Xinyang Li, Zeliang Zhang, Chenliang Xu
• Subjects: cs.CV, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2501.06250
• Pdf URL: https://arxiv.org/pdf/2501.06250
• Copy Paste: [[2501.06250]] Generative AI for Cel-Animation: A Survey(https://arxiv.org/abs/2501.06250)
• Keywords: diffusion, generative, large language model
• Abstract: Traditional Celluloid (Cel) Animation production pipeline encompasses multiple essential steps, including storyboarding, layout design, keyframe animation, inbetweening, and colorization, which demand substantial manual effort, technical expertise, and significant time investment. These challenges have historically impeded the efficiency and scalability of Cel-Animation production. The rise of generative artificial intelligence (GenAI), encompassing large language models, multimodal models, and diffusion models, offers innovative solutions by automating tasks such as inbetween frame generation, colorization, and storyboard creation. This survey explores how GenAI integration is revolutionizing traditional animation workflows by lowering technical barriers, broadening accessibility for a wider range of creators through tools like AniDoc, ToonCrafter, and AniSora, and enabling artists to focus more on creative expression and artistic innovation. Despite its potential, issues such as maintaining visual consistency, ensuring stylistic coherence, and addressing ethical considerations continue to pose challenges. Furthermore, this paper discusses future directions and explores potential advancements in AI-assisted animation. For further exploration and resources, please visit our GitHub repository: this https URL

Title: $\text{Transformer}^2$: Self-adaptive LLMs

• Authors: Qi Sun, Edoardo Cetin, Yujin Tang
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06252
• Pdf URL: https://arxiv.org/pdf/2501.06252
• Copy Paste: [[2501.06252]] $\text{Transformer}^2$: Self-adaptive LLMs(https://arxiv.org/abs/2501.06252)
• Keywords: transformer, large language model
• Abstract: Self-adaptive large language models (LLMs) aim to solve the challenges posed by traditional fine-tuning methods, which are often computationally intensive and static in their ability to handle diverse tasks. We introduce \implname, a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. During inference, \implname employs a two-pass mechanism: first, a dispatch system identifies the task properties, and then task-specific "expert" vectors, trained using reinforcement learning, are dynamically mixed to obtain targeted behavior for the incoming prompt. Our method outperforms ubiquitous approaches such as LoRA, with fewer parameters and greater efficiency. \implname demonstrates versatility across different LLM architectures and modalities, including vision-language tasks. \implname represents a significant leap forward, offering a scalable, efficient solution for enhancing the adaptability and task-specific performance of LLMs, paving the way for truly dynamic, self-organizing AI systems.

Title: Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words

• Authors: Gouki Minegishi, Hiroki Furuta, Yusuke Iwasawa, Yutaka Matsuo
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06254
• Pdf URL: https://arxiv.org/pdf/2501.06254
• Copy Paste: [[2501.06254]] Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words(https://arxiv.org/abs/2501.06254)
• Keywords: extraction, interpretability, large language model
• Abstract: Sparse autoencoders (SAEs) have gained a lot of attention as a promising tool to improve the interpretability of large language models (LLMs) by mapping the complex superposition of polysemantic neurons into monosemantic features and composing a sparse dictionary of words. However, traditional performance metrics like Mean Squared Error and L0 sparsity ignore the evaluation of the semantic representational power of SAEs -- whether they can acquire interpretable monosemantic features while preserving the semantic relationship of words. For instance, it is not obvious whether a learned sparse feature could distinguish different meanings in one word. In this paper, we propose a suite of evaluations for SAEs to analyze the quality of monosemantic features by focusing on polysemous words. Our findings reveal that SAEs developed to improve the MSE-L0 Pareto frontier may confuse interpretability, which does not necessarily enhance the extraction of monosemantic features. The analysis of SAEs with polysemous words can also figure out the internal mechanism of LLMs; deeper layers and the Attention module contribute to distinguishing polysemy in a word. Our semantics focused evaluation offers new insights into the polysemy and the existing SAE objective and contributes to the development of more practical SAEs.

Title: What Matters for In-Context Learning: A Balancing Act of Look-up and In-Weight Learning

• Authors: Jelena Bratulić, Sudhanshu Mittal, Christian Rupprecht, Thomas Brox
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06256
• Pdf URL: https://arxiv.org/pdf/2501.06256
• Copy Paste: [[2501.06256]] What Matters for In-Context Learning: A Balancing Act of Look-up and In-Weight Learning(https://arxiv.org/abs/2501.06256)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated impressive performance in various tasks, including In-Context Learning (ICL), where the model performs new tasks by conditioning solely on the examples provided in the context, without updating the model's weights. While prior research has explored the roles of pretraining data and model architecture, the key mechanism behind ICL remains unclear. In this work, we systematically uncover properties present in LLMs that support the emergence of ICL. To disambiguate these factors, we conduct a study with a controlled dataset and data sequences using a deep autoregressive model. We show that conceptual repetitions in the data sequences are crucial for ICL, more so than previously indicated training data properties like burstiness or long-tail distribution. Conceptual repetitions could refer to $n$-gram repetitions in textual data or exact image copies in image sequence data. Such repetitions also offer other previously overlooked benefits such as reduced transiency in ICL performance. Furthermore, we show that the emergence of ICL depends on balancing the in-weight learning objective with the in-context solving ability during training.

Title: Quantum Down Sampling Filter for Variational Auto-encoder

• Authors: Farina Riaz, Fakhar Zaman, Hajime Suzuki, Sharif Abuadbba, David Nguyen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06259
• Pdf URL: https://arxiv.org/pdf/2501.06259
• Copy Paste: [[2501.06259]] Quantum Down Sampling Filter for Variational Auto-encoder(https://arxiv.org/abs/2501.06259)
• Keywords: robust, generative
• Abstract: Variational Autoencoders (VAEs) are essential tools in generative modeling and image reconstruction, with their performance heavily influenced by the encoder-decoder architecture. This study aims to improve the quality of reconstructed images by enhancing their resolution and preserving finer details, particularly when working with low-resolution inputs (16x16 pixels), where traditional VAEs often yield blurred or in-accurate results. To address this, we propose a hybrid model that combines quantum computing techniques in the VAE encoder with convolutional neural networks (CNNs) in the decoder. By upscaling the resolution from 16x16 to 32x32 during the encoding process, our approach evaluates how the model reconstructs images with enhanced resolution while maintaining key features and structures. This method tests the model's robustness in handling image reconstruction and its ability to preserve essential details despite training on lower-resolution data. We evaluate our proposed down sampling filter for Quantum VAE (Q-VAE) on the MNIST and USPS datasets and compare it with classical VAEs and a variant called Classical Direct Passing VAE (CDP-VAE), which uses windowing pooling filters in the encoding process. Performance is assessed using metrics such as the Fréchet Inception Distance (FID) and Mean Squared Error (MSE), which measure the fidelity of reconstructed images. Our results demonstrate that the Q-VAE consistently outperforms both the Classical VAE and CDP-VAE, achieving significantly lower FID and MSE scores. Additionally, CDP-VAE yields better performance than C-VAE. These findings highlight the potential of quantum-enhanced VAEs to improve image reconstruction quality by enhancing resolution and preserving essential features, offering a promising direction for future applications in computer vision and synthetic data generation.

Title: CAMs as Shapley Value-based Explainers

• Authors: Huaiguang Cai
• Subjects: cs.CV, cs.GT
• Abstract URL: https://arxiv.org/abs/2501.06261
• Pdf URL: https://arxiv.org/pdf/2501.06261
• Copy Paste: [[2501.06261]] CAMs as Shapley Value-based Explainers(https://arxiv.org/abs/2501.06261)
• Keywords: explainability
• Abstract: Class Activation Mapping (CAM) methods are widely used to visualize neural network decisions, yet their underlying mechanisms remain incompletely understood. To enhance the understanding of CAM methods and improve their explainability, we introduce the Content Reserved Game-theoretic (CRG) Explainer. This theoretical framework clarifies the theoretical foundations of GradCAM and HiResCAM by modeling the neural network prediction process as a cooperative game. Within this framework, we develop ShapleyCAM, a new method that leverages gradients and the Hessian matrix to provide more precise and theoretically grounded visual explanations. Due to the computational infeasibility of exact Shapley value calculation, ShapleyCAM employs a second-order Taylor expansion of the cooperative game's utility function to derive a closed-form expression. Additionally, we propose the Residual Softmax Target-Class (ReST) utility function to address the limitations of pre-softmax and post-softmax scores. Extensive experiments across 12 popular networks on the ImageNet validation set demonstrate the effectiveness of ShapleyCAM and its variants. Our findings not only advance CAM explainability but also bridge the gap between heuristic-driven CAM methods and compute-intensive Shapley value-based methods. The code is available at \url{this https URL}.

Title: HPAC-IDS: A Hierarchical Packet Attention Convolution for Intrusion Detection System

• Authors: Anass Grini, Btissam El Khamlichi, Abdellatif El Afia, Amal El Fallah-Seghrouchni
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06264
• Pdf URL: https://arxiv.org/pdf/2501.06264
• Copy Paste: [[2501.06264]] HPAC-IDS: A Hierarchical Packet Attention Convolution for Intrusion Detection System(https://arxiv.org/abs/2501.06264)
• Keywords: attack, robust
• Abstract: This research introduces a robust detection system against malicious network traffic, leveraging hierarchical structures and self-attention mechanisms. The proposed system includes a Packet Segmenter that divides a given raw network packet into fixed-size segments that are fed to the HPAC-IDS. The experiments performed on CIC-IDS2017 dataset show that the system exhibits high accuracy and low false positive rates while demonstrating resilience against diverse adversarial methods like Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and Wasserstein GAN (WGAN). The model's ability to withstand adversarial perturbations is attributed to the fusion of hierarchical attention mechanisms and convolutional neural networks, resulting in a 0% to 10% adversarial attack severity under tested adversarial attacks with different segment sizes, surpassing the state-of-the-art model in detection performance and adversarial attack robustness.

Title: AgoraSpeech: A multi-annotated comprehensive dataset of political discourse through the lens of humans and AI

• Authors: Pavlos Sermpezis, Stelios Karamanidis, Eva Paraschou, Ilias Dimitriadis, Sofia Yfantidou, Filitsa-Ioanna Kouskouveli, Thanasis Troboukis, Kelly Kiki, Antonis Galanopoulos, Athena Vakali
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06265
• Pdf URL: https://arxiv.org/pdf/2501.06265
• Copy Paste: [[2501.06265]] AgoraSpeech: A multi-annotated comprehensive dataset of political discourse through the lens of humans and AI(https://arxiv.org/abs/2501.06265)
• Keywords: large language model
• Abstract: Political discourse datasets are important for gaining political insights, analyzing communication strategies or social science phenomena. Although numerous political discourse corpora exist, comprehensive, high-quality, annotated datasets are scarce. This is largely due to the substantial manual effort, multidisciplinarity, and expertise required for the nuanced annotation of rhetorical strategies and ideological contexts. In this paper, we present AgoraSpeech, a meticulously curated, high-quality dataset of 171 political speeches from six parties during the Greek national elections in 2023. The dataset includes annotations (per paragraph) for six natural language processing (NLP) tasks: text classification, topic identification, sentiment analysis, named entity recognition, polarization and populism detection. A two-step annotation was employed, starting with ChatGPT-generated annotations and followed by exhaustive human-in-the-loop validation. The dataset was initially used in a case study to provide insights during the pre-election period. However, it has general applicability by serving as a rich source of information for political and social scientists, journalists, or data scientists, while it can be used for benchmarking and fine-tuning NLP and large language models (LLMs).

Title: Environmental large language model Evaluation (ELLE) dataset: A Benchmark for Evaluating Generative AI applications in Eco-environment Domain

• Authors: Jing Guo, Nan Li, Ming Xu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.06277
• Pdf URL: https://arxiv.org/pdf/2501.06277
• Copy Paste: [[2501.06277]] Environmental large language model Evaluation (ELLE) dataset: A Benchmark for Evaluating Generative AI applications in Eco-environment Domain(https://arxiv.org/abs/2501.06277)
• Keywords: generative, large language model
• Abstract: Generative AI holds significant potential for ecological and environmental applications such as monitoring, data analysis, education, and policy support. However, its effectiveness is limited by the lack of a unified evaluation framework. To address this, we present the Environmental Large Language model Evaluation (ELLE) question answer (QA) dataset, the first benchmark designed to assess large language models and their applications in ecological and environmental sciences. The ELLE dataset includes 1,130 question answer pairs across 16 environmental topics, categorized by domain, difficulty, and type. This comprehensive dataset standardizes performance assessments in these fields, enabling consistent and objective comparisons of generative AI performance. By providing a dedicated evaluation tool, ELLE dataset promotes the development and application of generative AI technologies for sustainable environmental outcomes. The dataset and code are available at this https URL and this https URL.

Title: Punctuation's Semantic Role between Brain and Transformers Models

• Authors: Zenon Lamprou, Frank Polick, Yashar Moshfeghi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06278
• Pdf URL: https://arxiv.org/pdf/2501.06278
• Copy Paste: [[2501.06278]] Punctuation's Semantic Role between Brain and Transformers Models(https://arxiv.org/abs/2501.06278)
• Keywords: transformer
• Abstract: Contemporary neural networks intended for natural language processing (NLP) are not designed with specific linguistic rules. It suggests that they may acquire a general understanding of language. This attribute has led to extensive research in deciphering their internal representations. A pioneering method involves an experimental setup using human brain data to explore if a translation between brain and neural network representations can be established. Since this technique emerged, more sophisticated NLP models have been developed. In our study, we apply this method to evaluate four new NLP models aiming to identify the one most compatible with brain activity. Additionally, to explore how the brain comprehends text semantically, we alter the text by removing punctuation in four different ways to understand its impact on semantic processing by the human brain. Our findings indicate that the RoBERTa model aligns best with brain activity, outperforming BERT in accuracy according to our metrics. Furthermore, for BERT, higher accuracy was noted when punctuation was excluded, and increased context length did not significantly diminish accuracy compared to the original results with punctuation.

Title: Autonomous Identity-Based Threat Segmentation in Zero Trust Architectures

• Authors: Sina Ahmadi
• Subjects: cs.CR, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.06281
• Pdf URL: https://arxiv.org/pdf/2501.06281
• Copy Paste: [[2501.06281]] Autonomous Identity-Based Threat Segmentation in Zero Trust Architectures(https://arxiv.org/abs/2501.06281)
• Keywords: security, privacy, attack, segmentation
• Abstract: Zero Trust Architectures (ZTA) fundamentally redefine network security by adopting a "trust nothing, verify everything" approach that requires identity verification for all access. Conventional discrete access control measures have proven inadequate since they do not consider evolving user activities and contextual threats, leading to internal threats and enhanced attacks. This research applies the proposed AI-driven, autonomous, identity-based threat segmentation in ZTA, along with real-time identity analytics for fine-grained, real-time mechanisms. Some of the sharp practices include using the behavioral analytics approach to provide real-time risk scores, such as analyzing the patterns used for logging into the system, the access sought, and the resources used. Permissions are adjusted using machine learning models that take into account context-aware factors like geolocation, device type, and access time. Automated threat segmentation helps analysts identify multiple compromised identities in real-time, thus minimizing the likelihood of a breach advancing. The system's use cases are based on real scenarios; for example, insider threats in global offices demonstrate how compromised accounts are detected and locked. This work outlines measures to address privacy issues, false positives, and scalability concerns. This research enhances the security of other critical areas of computer systems by providing dynamic access governance, minimizing insider threats, and supporting dynamic policy enforcement while ensuring that the needed balance between security and user productivity remains a top priority. We prove via comparative analyses that the model is precise and scalable.

Title: MinMo: A Multimodal Large Language Model for Seamless Voice Interaction

• Authors: Qian Chen, Yafeng Chen, Yanni Chen, Mengzhe Chen, Yingda Chen, Chong Deng, Zhihao Du, Ruize Gao, Changfeng Gao, Zhifu Gao, Yabin Li, Xiang Lv, Jiaqing Liu, Haoneng Luo, Bin Ma, Chongjia Ni, Xian Shi, Jialong Tang, Hui Wang, Hao Wang, Wen Wang, Yuxuan Wang, Yunlan Xu, Fan Yu, Zhijie Yan, Yexin Yang, Baosong Yang, Xian Yang, Guanrou Yang, Tianyu Zhao, Qinglin Zhang, Shiliang Zhang, Nan Zhao, Pei Zhang, Chong Zhang, Jinren Zhou
• Subjects: cs.CL, cs.AI, cs.HC, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2501.06282
• Pdf URL: https://arxiv.org/pdf/2501.06282
• Copy Paste: [[2501.06282]] MinMo: A Multimodal Large Language Model for Seamless Voice Interaction(https://arxiv.org/abs/2501.06282)
• Keywords: large language model
• Abstract: Recent advancements in large language models (LLMs) and multimodal speech-text models have laid the groundwork for seamless voice interactions, enabling real-time, natural, and human-like conversations. Previous models for voice interactions are categorized as native and aligned. Native models integrate speech and text processing in one framework but struggle with issues like differing sequence lengths and insufficient pre-training. Aligned models maintain text LLM capabilities but are often limited by small datasets and a narrow focus on speech tasks. In this work, we introduce MinMo, a Multimodal Large Language Model with approximately 8B parameters for seamless voice interaction. We address the main limitations of prior aligned multimodal models. We train MinMo through multiple stages of speech-to-text alignment, text-to-speech alignment, speech-to-speech alignment, and duplex interaction alignment, on 1.4 million hours of diverse speech data and a broad range of speech tasks. After the multi-stage training, MinMo achieves state-of-the-art performance across various benchmarks for voice comprehension and generation while maintaining the capabilities of text LLMs, and also facilitates full-duplex conversation, that is, simultaneous two-way communication between the user and the system. Moreover, we propose a novel and simple voice decoder that outperforms prior models in voice generation. The enhanced instruction-following capabilities of MinMo supports controlling speech generation based on user instructions, with various nuances including emotions, dialects, and speaking rates, and mimicking specific voices. For MinMo, the speech-to-text latency is approximately 100ms, full-duplex latency is approximately 600ms in theory and 800ms in practice. The MinMo project web page is this https URL, and the code and models will be released soon.

Title: Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks

• Authors: Iman Barati, Arash Ghafouri, Behrouz Minaei-Bidgoli
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06286
• Pdf URL: https://arxiv.org/pdf/2501.06286
• Copy Paste: [[2501.06286]] Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks(https://arxiv.org/abs/2501.06286)
• Keywords: large language model
• Abstract: In recent years, the use of large language models (LLMs) has significantly increased, and these models have demonstrated remarkable performance in a variety of general language tasks. However, the evaluation of their performance in domain-specific tasks, particularly those requiring deep natural language understanding, has received less attention. In this research, we evaluate the ability of large language models in performing domain-specific tasks, focusing on the multi-hop question answering (MHQA) problem using the HotpotQA dataset. This task, due to its requirement for reasoning and combining information from multiple textual sources, serves as a challenging benchmark for assessing the language comprehension capabilities of these models. To tackle this problem, we have designed a two-stage selector-reader architecture, where each stage utilizes an independent LLM. In addition, methods such as Chain of Thought (CoT) and question decomposition have been employed to investigate their impact on improving the model's performance. The results of the study show that the integration of large language models with these techniques can lead to up to a 4% improvement in F1 score for finding answers, providing evidence of the models' ability to handle domain-specific tasks and their understanding of complex language.

Title: Reinforcement Learning-Driven Adaptation Chains: A Robust Framework for Multi-Cloud Workflow Security

• Authors: Nafiseh Soveizi, Dimka Karastoyanova
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.06305
• Pdf URL: https://arxiv.org/pdf/2501.06305
• Copy Paste: [[2501.06305]] Reinforcement Learning-Driven Adaptation Chains: A Robust Framework for Multi-Cloud Workflow Security(https://arxiv.org/abs/2501.06305)
• Keywords: security, attack, robust
• Abstract: Cloud computing has emerged as a crucial solution for managing data- and compute-intensive workflows, offering scalability to address dynamic demands. However, security concerns persist, especially for workflows involving sensitive data and tasks. One of the main gaps in the literature is the lack of robust and flexible measures for reacting to these security violations. To address this, we propose an innovative approach leveraging Reinforcement Learning (RL) to formulate adaptation chains, responding effectively to security violations within cloud-based workflows. These chains consist of sequences of adaptation actions tailored to attack characteristics, workflow dependencies, and user-defined requirements. Unlike conventional single-task adaptations, adaptation chains provide a comprehensive mitigation strategy by taking into account both control and data dependencies between tasks, thereby accommodating conflicting objectives effectively. Moreover, our RL-based approach uses insights from past responses to mitigate uncertainties associated with adaptation costs. We evaluate the method using our jBPM and Cloudsim Plus based implementation and compare the impact of selected adaptation chains on workflows with the single adaptation approach. Results demonstrate that the adaptation chain approach outperforms in terms of total adaptation cost, offering resilience and adaptability against security threats.

Title: Uncertainty Estimation for Path Loss and Radio Metric Models

• Authors: Alexis Bose, Jonathan Ethier, Ryan G. Dempsey, Yifeng Qiu
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.06308
• Pdf URL: https://arxiv.org/pdf/2501.06308
• Copy Paste: [[2501.06308]] Uncertainty Estimation for Path Loss and Radio Metric Models(https://arxiv.org/abs/2501.06308)
• Keywords: robust
• Abstract: This research leverages Conformal Prediction (CP) in the form of Conformal Predictive Systems (CPS) to accurately estimate uncertainty in a suite of machine learning (ML)-based radio metric models [1] as well as in a 2-D map-based ML path loss model [2]. Utilizing diverse difficulty estimators, we construct 95% confidence prediction intervals (PIs) that are statistically robust. Our experiments demonstrate that CPS models, trained on Toronto datasets, generalize effectively to other cities such as Vancouver and Montreal, maintaining high coverage and reliability. Furthermore, the employed difficulty estimators identify challenging samples, leading to measurable reductions in RMSE as dataset difficulty decreases. These findings highlight the effectiveness of scalable and reliable uncertainty estimation through CPS in wireless network modeling, offering important potential insights for network planning, operations, and spectrum management.

Title: Towards Iris Presentation Attack Detection with Foundation Models

• Authors: Juan E. Tapia, Lázaro Janier González-Soler, Christoph Busch
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06312
• Pdf URL: https://arxiv.org/pdf/2501.06312
• Copy Paste: [[2501.06312]] Towards Iris Presentation Attack Detection with Foundation Models(https://arxiv.org/abs/2501.06312)
• Keywords: attack
• Abstract: Foundation models are becoming increasingly popular due to their strong generalization capabilities resulting from being trained on huge datasets. These generalization capabilities are attractive in areas such as NIR Iris Presentation Attack Detection (PAD), in which databases are limited in the number of subjects and diversity of attack instruments, and there is no correspondence between the bona fide and attack images because, most of the time, they do not belong to the same subjects. This work explores an iris PAD approach based on two foundation models, DinoV2 and VisualOpenClip. The results show that fine-tuning prediction with a small neural network as head overpasses the state-of-the-art performance based on deep learning approaches. However, systems trained from scratch have still reached better results if bona fide and attack images are available.

Title: Aggregating Low Rank Adapters in Federated Fine-tuning

• Authors: Evelyn Trautmann, Ian Hales, Martin F. Volk
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06332
• Pdf URL: https://arxiv.org/pdf/2501.06332
• Copy Paste: [[2501.06332]] Aggregating Low Rank Adapters in Federated Fine-tuning(https://arxiv.org/abs/2501.06332)
• Keywords: federate, large language model
• Abstract: Fine-tuning large language models requires high computational and memory resources, and is therefore associated with significant costs. When training on federated datasets, an increased communication effort is also needed. For this reason, parameter-efficient methods (PEFT) are becoming increasingly important. In this context, very good results have already been achieved by fine-tuning with low-rank adaptation methods (LoRA). The application of LoRA methods in Federated Learning, and especially the aggregation of adaptation matrices, is a current research field. In this article, we propose a novel aggregation method and compare it with different existing aggregation methods of low rank adapters trained in a federated fine-tuning of large machine learning models and evaluate their performance with respect to selected GLUE benchmark datasets.

Title: MEt3R: Measuring Multi-View Consistency in Generated Images

• Authors: Mohammad Asim, Christopher Wewer, Thomas Wimmer, Bernt Schiele, Jan Eric Lenssen
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.06336
• Pdf URL: https://arxiv.org/pdf/2501.06336
• Copy Paste: [[2501.06336]] MEt3R: Measuring Multi-View Consistency in Generated Images(https://arxiv.org/abs/2501.06336)
• Keywords: diffusion, generative
• Abstract: We introduce MEt3R, a metric for multi-view consistency in generated images. Large-scale generative models for multi-view image generation are rapidly advancing the field of 3D inference from sparse observations. However, due to the nature of generative modeling, traditional reconstruction metrics are not suitable to measure the quality of generated outputs and metrics that are independent of the sampling procedure are desperately needed. In this work, we specifically address the aspect of consistency between generated multi-view images, which can be evaluated independently of the specific scene. Our approach uses DUSt3R to obtain dense 3D reconstructions from image pairs in a feed-forward manner, which are used to warp image contents from one view into the other. Then, feature maps of these images are compared to obtain a similarity score that is invariant to view-dependent effects. Using MEt3R, we evaluate the consistency of a large set of previous methods for novel view and video generation, including our open, multi-view latent diffusion model.

Title: Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages

• Authors: Jannik Brinkmann, Chris Wendler, Christian Bartelt, Aaron Mueller
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06346
• Pdf URL: https://arxiv.org/pdf/2501.06346
• Copy Paste: [[2501.06346]] Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages(https://arxiv.org/abs/2501.06346)
• Keywords: robust, large language model
• Abstract: Human bilinguals often use similar brain regions to process multiple languages, depending on when they learned their second language and their proficiency. In large language models (LLMs), how are multiple languages learned and encoded? In this work, we explore the extent to which LLMs share representations of morphosyntactic concepts such as grammatical number, gender, and tense across languages. We train sparse autoencoders on Llama-3-8B and Aya-23-8B, and demonstrate that abstract grammatical concepts are often encoded in feature directions shared across many languages. We use causal interventions to verify the multilingual nature of these representations; specifically, we show that ablating only multilingual features decreases classifier performance to near-chance across languages. We then use these features to precisely modify model behavior in a machine translation task; this demonstrates both the generality and selectivity of these feature's roles in the network. Our findings suggest that even models trained predominantly on English data can develop robust, cross-lingual abstractions of morphosyntactic concepts.

Title: Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity

• Authors: Navin Ranjan, Andreas Savakis
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06357
• Pdf URL: https://arxiv.org/pdf/2501.06357
• Copy Paste: [[2501.06357]] Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity(https://arxiv.org/abs/2501.06357)
• Keywords: explainability, transformer
• Abstract: In this paper, we propose Mix-QViT, an explainability-driven MPQ framework that systematically allocates bit-widths to each layer based on two criteria: layer importance, assessed via Layer-wise Relevance Propagation (LRP), which identifies how much each layer contributes to the final classification, and quantization sensitivity, determined by evaluating the performance impact of quantizing each layer at various precision levels while keeping others layers at a baseline. Additionally, for post-training quantization (PTQ), we introduce a clipped channel-wise quantization method designed to reduce the effects of extreme outliers in post-LayerNorm activations by removing severe inter-channel variations. We validate our approach by applying Mix-QViT to ViT, DeiT, and Swin Transformer models across multiple datasets. Our experimental results for PTQ demonstrate that both fixed-bit and mixed-bit methods outperform existing techniques, particularly at 3-bit, 4-bit, and 6-bit precision. Furthermore, in quantization-aware training, Mix-QViT achieves superior performance with 2-bit mixed-precision.

Title: Gender-Neutral Large Language Models for Medical Applications: Reducing Bias in PubMed Abstracts

• Authors: Elizabeth Schaefer, Kirk Roberts
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.06365
• Pdf URL: https://arxiv.org/pdf/2501.06365
• Copy Paste: [[2501.06365]] Gender-Neutral Large Language Models for Medical Applications: Reducing Bias in PubMed Abstracts(https://arxiv.org/abs/2501.06365)
• Keywords: large language model
• Abstract: This paper presents a pipeline for mitigating gender bias in large language models (LLMs) used in medical literature by neutralizing gendered occupational pronouns. A dataset of 379,000 PubMed abstracts from 1965-1980 was processed to identify and modify pronouns tied to professions. We developed a BERT-based model, ``Modern Occupational Bias Elimination with Refined Training,'' or ``MOBERT,'' trained on these neutralized abstracts, and compared its performance with ``1965Bert,'' trained on the original dataset. MOBERT achieved a 70\% inclusive replacement rate, while 1965Bert reached only 4\%. A further analysis of MOBERT revealed that pronoun replacement accuracy correlated with the frequency of occupational terms in the training data. We propose expanding the dataset and refining the pipeline to improve performance and ensure more equitable language modeling in medical applications.

Title: Resilient Endurance-Aware NVM-based PUF against Learning-based Attacks

• Authors: Hassan Nassar, Ming-Liang Wei, Chia-Lin Yang, Jörg Henkel, Kuan-Hsun Chen
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06367
• Pdf URL: https://arxiv.org/pdf/2501.06367
• Copy Paste: [[2501.06367]] Resilient Endurance-Aware NVM-based PUF against Learning-based Attacks(https://arxiv.org/abs/2501.06367)
• Keywords: secure, protect, attack, robust
• Abstract: Physical Unclonable Functions (PUFs) based on Non-Volatile Memory (NVM) technology have emerged as a promising solution for secure authentication and cryptographic applications. By leveraging the multi-level cell (MLC) characteristic of NVMs, these PUFs can generate a wide range of unique responses, enhancing their resilience to machine learning (ML) modeling attacks. However, a significant issue with NVM-based PUFs is their endurance problem; frequent write operations lead to wear and degradation over time, reducing the reliability and lifespan of the PUF. This paper addresses these issues by offering a comprehensive model to predict and analyze the effects of endurance changes on NVM PUFs. This model provides insights into how wear impacts the PUF's quality and helps in designing more robust PUFs. Building on this model, we present a novel design for NVM PUFs that significantly improves endurance. Our design approach incorporates advanced techniques to distribute write operations more evenly and reduce stress on individual cells. The result is an NVM PUF that demonstrates a $62\times$ improvement in endurance compared to current state-of-the-art solutions while maintaining protection against learning-based attacks.

Title: Towards Robust Nonlinear Subspace Clustering: A Kernel Learning Approach

• Authors: Kunpeng Xu, Lifei Chen, Shengrui Wang
• Subjects: cs.LG, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.06368
• Pdf URL: https://arxiv.org/pdf/2501.06368
• Copy Paste: [[2501.06368]] Towards Robust Nonlinear Subspace Clustering: A Kernel Learning Approach(https://arxiv.org/abs/2501.06368)
• Keywords: robust
• Abstract: Kernel-based subspace clustering, which addresses the nonlinear structures in data, is an evolving area of research. Despite noteworthy progressions, prevailing methodologies predominantly grapple with limitations relating to (i) the influence of predefined kernels on model performance; (ii) the difficulty of preserving the original manifold structures in the nonlinear space; (iii) the dependency of spectral-type strategies on the ideal block diagonal structure of the affinity matrix. This paper presents DKLM, a novel paradigm for kernel-induced nonlinear subspace clustering. DKLM provides a data-driven approach that directly learns the kernel from the data's self-representation, ensuring adaptive weighting and satisfying the multiplicative triangle inequality constraint, which enhances the robustness of the learned kernel. By leveraging this learned kernel, DKLM preserves the local manifold structure of data in a nonlinear space while promoting the formation of an optimal block-diagonal affinity matrix. A thorough theoretical examination of DKLM reveals its relationship with existing clustering paradigms. Comprehensive experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Title: AFRIDOC-MT: Document-level MT Corpus for African Languages

• Authors: Jesujoba O. Alabi, Israel Abebe Azime, Miaoran Zhang, Cristina España-Bonet, Rachel Bawden, Dawei Zhu, David Ifeoluwa Adelani, Clement Oyeleke Odoje, Idris Akinade, Iffat Maab, Davis David, Shamsuddeen Hassan Muhammad, Neo Putini, David O. Ademuyiwa, Andrew Caines, Dietrich Klakow
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06374
• Pdf URL: https://arxiv.org/pdf/2501.06374
• Copy Paste: [[2501.06374]] AFRIDOC-MT: Document-level MT Corpus for African Languages(https://arxiv.org/abs/2501.06374)
• Keywords: large language model
• Abstract: This paper introduces AFRIDOC-MT, a document-level multi-parallel translation dataset covering English and five African languages: Amharic, Hausa, Swahili, Yorùbá, and Zulu. The dataset comprises 334 health and 271 information technology news documents, all human-translated from English to these languages. We conduct document-level translation benchmark experiments by evaluating neural machine translation (NMT) models and large language models (LLMs) for translations between English and these languages, at both the sentence and pseudo-document levels. These outputs are realigned to form complete documents for evaluation. Our results indicate that NLLB-200 achieved the best average performance among the standard NMT models, while GPT-4o outperformed general-purpose LLMs. Fine-tuning selected models led to substantial performance gains, but models trained on sentences struggled to generalize effectively to longer documents. Furthermore, our analysis reveals that some LLMs exhibit issues such as under-generation, repetition of words or phrases, and off-target translations, especially for African languages.

Title: Using Pre-trained LLMs for Multivariate Time Series Forecasting

• Authors: Malcolm L. Wolff, Shenghao Yang, Kari Torkkola, Michael W. Mahoney
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06386
• Pdf URL: https://arxiv.org/pdf/2501.06386
• Copy Paste: [[2501.06386]] Using Pre-trained LLMs for Multivariate Time Series Forecasting(https://arxiv.org/abs/2501.06386)
• Keywords: transformer, large language model
• Abstract: Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings.

Title: Has an AI model been trained on your images?

• Authors: Matyas Bohacek, Hany Farid
• Subjects: cs.CV, cs.AI, cs.CR, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06399
• Pdf URL: https://arxiv.org/pdf/2501.06399
• Copy Paste: [[2501.06399]] Has an AI model been trained on your images?(https://arxiv.org/abs/2501.06399)
• Keywords: membership infer, fair, generative
• Abstract: From a simple text prompt, generative-AI image models can create stunningly realistic and creative images bounded, it seems, by only our imagination. These models have achieved this remarkable feat thanks, in part, to the ingestion of billions of images collected from nearly every corner of the internet. Many creators have understandably expressed concern over how their intellectual property has been ingested without their permission or a mechanism to opt out of training. As a result, questions of fair use and copyright infringement have quickly emerged. We describe a method that allows us to determine if a model was trained on a specific image or set of images. This method is computationally efficient and assumes no explicit knowledge of the model architecture or weights (so-called black-box membership inference). We anticipate that this method will be crucial for auditing existing models and, looking ahead, ensuring the fairer development and deployment of generative AI models.

Title: Mathematics of Digital Twins and Transfer Learning for PDE Models

• Authors: Yifei Zong, Alexandre Tartakovsky
• Subjects: cs.LG, math.NA, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.06400
• Pdf URL: https://arxiv.org/pdf/2501.06400
• Copy Paste: [[2501.06400]] Mathematics of Digital Twins and Transfer Learning for PDE Models(https://arxiv.org/abs/2501.06400)
• Keywords: diffusion
• Abstract: We define a digital twin (DT) of a physical system governed by partial differential equations (PDEs) as a model for real-time simulations and control of the system behavior under changing conditions. We construct DTs using the Karhunen-Loève Neural Network (KL-NN) surrogate model and transfer learning (TL). The surrogate model allows fast inference and differentiability with respect to control parameters for control and optimization. TL is used to retrain the model for new conditions with minimal additional data. We employ the moment equations to analyze TL and identify parameters that can be transferred to new conditions. The proposed analysis also guides the control variable selection in DT to facilitate efficient TL. For linear PDE problems, the non-transferable parameters in the KL-NN surrogate model can be exactly estimated from a single solution of the PDE corresponding to the mean values of the control variables under new target conditions. Retraining an ML model with a single solution sample is known as one-shot learning, and our analysis shows that the one-shot TL is exact for linear PDEs. For nonlinear PDE problems, transferring of any parameters introduces errors. For a nonlinear diffusion PDE model, we find that for a relatively small range of control variables, some surrogate model parameters can be transferred without introducing a significant error, some can be approximately estimated from the mean-field equation, and the rest can be found using a linear residual least square problem or an ordinary linear least square problem if a small labeled dataset for new conditions is available. The former approach results in a one-shot TL while the latter approach is an example of a few-shot TL. Both methods are approximate for the nonlinear PDEs.

Title: FocusDD: Real-World Scene Infusion for Robust Dataset Distillation

• Authors: Youbing Hu, Yun Cheng, Olga Saukh, Firat Ozdemir, Anqi Lu, Zhiqiang Cao, Zhijun Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06405
• Pdf URL: https://arxiv.org/pdf/2501.06405
• Copy Paste: [[2501.06405]] FocusDD: Real-World Scene Infusion for Robust Dataset Distillation(https://arxiv.org/abs/2501.06405)
• Keywords: robust, transformer
• Abstract: Dataset distillation has emerged as a strategy to compress real-world datasets for efficient training. However, it struggles with large-scale and high-resolution datasets, limiting its practicality. This paper introduces a novel resolution-independent dataset distillation method Focus ed Dataset Distillation (FocusDD), which achieves diversity and realism in distilled data by identifying key information patches, thereby ensuring the generalization capability of the distilled dataset across different network architectures. Specifically, FocusDD leverages a pre-trained Vision Transformer (ViT) to extract key image patches, which are then synthesized into a single distilled image. These distilled images, which capture multiple targets, are suitable not only for classification tasks but also for dense tasks such as object detection. To further improve the generalization of the distilled dataset, each synthesized image is augmented with a downsampled view of the original image. Experimental results on the ImageNet-1K dataset demonstrate that, with 100 images per class (IPC), ResNet50 and MobileNet-v2 achieve validation accuracies of 71.0% and 62.6%, respectively, outperforming state-of-the-art methods by 2.8% and 4.7%. Notably, FocusDD is the first method to use distilled datasets for object detection tasks. On the COCO2017 dataset, with an IPC of 50, YOLOv11n and YOLOv11s achieve 24.4% and 32.1% mAP, respectively, further validating the effectiveness of our approach.

Title: Tensor Product Attention Is All You Need

• Authors: Yifan Zhang, Yifeng Liu, Huizhuo Yuan, Zhen Qin, Yang Yuan, Quanquan Gu, Andrew Chi-Chih Yao
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06425
• Pdf URL: https://arxiv.org/pdf/2501.06425
• Copy Paste: [[2501.06425]] Tensor Product Attention Is All You Need(https://arxiv.org/abs/2501.06425)
• Keywords: transformer
• Abstract: Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory overhead during inference. In this paper, we propose Tensor Product Attention (TPA), a novel attention mechanism that uses tensor decompositions to represent queries, keys, and values compactly, significantly shrinking KV cache size at inference time. By factorizing these representations into contextual low-rank components (contextual factorization) and seamlessly integrating with RoPE, TPA achieves improved model quality alongside memory efficiency. Based on TPA, we introduce the Tensor ProducT ATTenTion Transformer (T6), a new model architecture for sequence modeling. Through extensive empirical evaluation of language modeling tasks, we demonstrate that T6 exceeds the performance of standard Transformer baselines including MHA, MQA, GQA, and MLA across various metrics, including perplexity and a range of renowned evaluation benchmarks. Notably, TPAs memory efficiency enables the processing of significantly longer sequences under fixed resource constraints, addressing a critical scalability challenge in modern language models. The code is available at this https URL.

Title: Reliable Imputed-Sample Assisted Vertical Federated Learning

• Authors: Yaopei Zeng, Lei Liu, Shaoguo Liu, Hongjian Dou, Baoyuan Wu, Li Liu
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.06429
• Pdf URL: https://arxiv.org/pdf/2501.06429
• Copy Paste: [[2501.06429]] Reliable Imputed-Sample Assisted Vertical Federated Learning(https://arxiv.org/abs/2501.06429)
• Keywords: federate
• Abstract: Vertical Federated Learning (VFL) is a well-known FL variant that enables multiple parties to collaboratively train a model without sharing their raw data. Existing VFL approaches focus on overlapping samples among different parties, while their performance is constrained by the limited number of these samples, leaving numerous non-overlapping samples unexplored. Some previous work has explored techniques for imputing missing values in samples, but often without adequate attention to the quality of the imputed samples. To address this issue, we propose a Reliable Imputed-Sample Assisted (RISA) VFL framework to effectively exploit non-overlapping samples by selecting reliable imputed samples for training VFL models. Specifically, after imputing non-overlapping samples, we introduce evidence theory to estimate the uncertainty of imputed samples, and only samples with low uncertainty are selected. In this way, high-quality non-overlapping samples are utilized to improve VFL model. Experiments on two widely used datasets demonstrate the significant performance gains achieved by the RISA, especially with the limited overlapping samples, e.g., a 48% accuracy gain on CIFAR-10 with only 1% overlapping samples.

Title: Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs

• Authors: Shan Zhang, Aotian Chen, Yanpeng Sun, Jindong Gu, Yi-Yu Zheng, Piotr Koniusz, Kai Zou, Anton van den Hengel, Yuan Xue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06430
• Pdf URL: https://arxiv.org/pdf/2501.06430
• Copy Paste: [[2501.06430]] Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs(https://arxiv.org/abs/2501.06430)
• Keywords: large language model
• Abstract: Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding. The limitation is largely attributable to inadequate perception of geometric primitives during image-level contrastive pre-training (e.g., CLIP). While recent efforts to improve math MLLMs have focused on scaling up mathematical visual instruction datasets and employing stronger LLM backbones, they often overlook persistent errors in visual recognition. In this paper, we systematically evaluate the visual grounding capabilities of state-of-the-art MLLMs and reveal a significant negative correlation between visual grounding accuracy and problem-solving performance, underscoring the critical role of fine-grained visual understanding. Notably, advanced models like GPT-4o exhibit a 70% error rate when identifying geometric entities, highlighting that this remains a key bottleneck in visual mathematical reasoning. To address this, we propose a novel approach, SVE-Math (Selective Vision-Enhanced Mathematical MLLM), featuring a geometric-grounded vision encoder and a feature router that dynamically adjusts the contribution of hierarchical visual feature maps. Our model recognizes accurate visual primitives and generates precise visual prompts tailored to the language model's reasoning needs. In experiments, SVE-Math-Qwen2.5-7B outperforms other 7B models by 15% on MathVerse and is compatible with GPT-4V on MathVista. Despite being trained on smaller datasets, SVE-Math-7B achieves competitive performance on GeoQA, rivaling models trained on significantly larger datasets. Our findings emphasize the importance of incorporating fine-grained visual understanding into MLLMs and provide a promising direction for future research.

Title: Synthetic Feature Augmentation Improves Generalization Performance of Language Models

• Authors: Ashok Choudhary, Cornelius Thiels, Hojjat Salehinejad
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06434
• Pdf URL: https://arxiv.org/pdf/2501.06434
• Copy Paste: [[2501.06434]] Synthetic Feature Augmentation Improves Generalization Performance of Language Models(https://arxiv.org/abs/2501.06434)
• Keywords: robust, large language model
• Abstract: Training and fine-tuning deep learning models, especially large language models (LLMs), on limited and imbalanced datasets poses substantial challenges. These issues often result in poor generalization, where models overfit to dominant classes and underperform on minority classes, leading to biased predictions and reduced robustness in real-world applications. To overcome these challenges, we propose augmenting features in the embedding space by generating synthetic samples using a range of techniques. By upsampling underrepresented classes, this method improves model performance and alleviates data imbalance. We validate the effectiveness of this approach across multiple open-source text classification benchmarks, demonstrating its potential to enhance model robustness and generalization in imbalanced data scenarios.

Title: Qffusion: Controllable Portrait Video Editing via Quadrant-Grid Attention Learning

• Authors: Maomao Li, Lijian Lin, Yunfei Liu, Ye Zhu, Yu Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06438
• Pdf URL: https://arxiv.org/pdf/2501.06438
• Copy Paste: [[2501.06438]] Qffusion: Controllable Portrait Video Editing via Quadrant-Grid Attention Learning(https://arxiv.org/abs/2501.06438)
• Keywords: diffusion, generative
• Abstract: This paper presents Qffusion, a dual-frame-guided framework for portrait video editing. Specifically, we consider a design principle of ``animation for editing'', and train Qffusion as a general animation framework from two still reference images while we can use it for portrait video editing easily by applying modified start and end frames as references during inference. Leveraging the powerful generative power of Stable Diffusion, we propose a Quadrant-grid Arrangement (QGA) scheme for latent re-arrangement, which arranges the latent codes of two reference images and that of four facial conditions into a four-grid fashion, separately. Then, we fuse features of these two modalities and use self-attention for both appearance and temporal learning, where representations at different times are jointly modeled under QGA. Our Qffusion can achieve stable video editing without additional networks or complex training stages, where only the input format of Stable Diffusion is modified. Further, we propose a Quadrant-grid Propagation (QGP) inference strategy, which enjoys a unique advantage on stable arbitrary-length video generation by processing reference and condition frames recursively. Through extensive experiments, Qffusion consistently outperforms state-of-the-art techniques on portrait video editing.

Title: UCloudNet: A Residual U-Net with Deep Supervision for Cloud Image Segmentation

• Authors: Yijie Li, Hewei Wang, Shaofan Wang, Yee Hui Lee, Muhammad Salman Pathan, Soumyabrata Dev
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.06440
• Pdf URL: https://arxiv.org/pdf/2501.06440
• Copy Paste: [[2501.06440]] UCloudNet: A Residual U-Net with Deep Supervision for Cloud Image Segmentation(https://arxiv.org/abs/2501.06440)
• Keywords: extraction, segmentation
• Abstract: Recent advancements in meteorology involve the use of ground-based sky cameras for cloud observation. Analyzing images from these cameras helps in calculating cloud coverage and understanding atmospheric phenomena. Traditionally, cloud image segmentation relied on conventional computer vision techniques. However, with the advent of deep learning, convolutional neural networks (CNNs) are increasingly applied for this purpose. Despite their effectiveness, CNNs often require many epochs to converge, posing challenges for real-time processing in sky camera systems. In this paper, we introduce a residual U-Net with deep supervision for cloud segmentation which provides better accuracy than previous approaches, and with less training consumption. By utilizing residual connection in encoders of UCloudNet, the feature extraction ability is further improved.

Title: CPDR: Towards Highly-Efficient Salient Object Detection via Crossed Post-decoder Refinement

• Authors: Yijie Li, Hewei Wang, Aggelos Katsaggelos
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06441
• Pdf URL: https://arxiv.org/pdf/2501.06441
• Copy Paste: [[2501.06441]] CPDR: Towards Highly-Efficient Salient Object Detection via Crossed Post-decoder Refinement(https://arxiv.org/abs/2501.06441)
• Keywords: extraction
• Abstract: Most of the current salient object detection approaches use deeper networks with large backbones to produce more accurate predictions, which results in a significant increase in computational complexity. A great number of network designs follow the pure UNet and Feature Pyramid Network (FPN) architecture which has limited feature extraction and aggregation ability which motivated us to design a lightweight post-decoder refinement module, the crossed post-decoder refinement (CPDR) to enhance the feature representation of a standard FPN or U-Net framework. Specifically, we introduce the Attention Down Sample Fusion (ADF), which employs channel attention mechanisms with attention maps generated by high-level representation to refine the low-level features, and Attention Up Sample Fusion (AUF), leveraging the low-level information to guide the high-level features through spatial attention. Additionally, we proposed the Dual Attention Cross Fusion (DACF) upon ADFs and AUFs, which reduces the number of parameters while maintaining the performance. Experiments on five benchmark datasets demonstrate that our method outperforms previous state-of-the-art approaches.

Title: Automated Detection and Analysis of Minor Deformations in Flat Walls Due to Railway Vibrations Using LiDAR and Machine Learning

• Authors: Surjo Dey, Ankit Sharma, Hritu Raj, Susham Biswas
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06457
• Pdf URL: https://arxiv.org/pdf/2501.06457
• Copy Paste: [[2501.06457]] Automated Detection and Analysis of Minor Deformations in Flat Walls Due to Railway Vibrations Using LiDAR and Machine Learning(https://arxiv.org/abs/2501.06457)
• Keywords: extraction
• Abstract: This study introduces an advanced methodology for automatically identifying minor deformations in flat walls caused by vibrations from nearby railway tracks. It leverages high-density Terrestrial Laser Scanner (TLS) LiDAR surveys and AI/ML techniques to collect and analyze data. The scan data is processed into a detailed point cloud, which is segmented to distinguish ground points, trees, buildings, and other objects. The analysis focuses on identifying sections along flat walls and estimating their deformations relative to the ground orientation. Findings from the study, conducted at the RGIPT campus, reveal significant deformations in walls close to the railway corridor, with the highest deformations ranging from 7 to 8 cm and an average of 3 to 4 cm. In contrast, walls further from the corridor show negligible deformations. The developed automated process for feature extraction and deformation monitoring demonstrates potential for structural health monitoring. By integrating LiDAR data with machine learning, the methodology provides an efficient system for identifying and analyzing structural deformations, highlighting the importance of continuous monitoring for ensuring structural integrity and public safety in urban infrastructure. This approach represents a substantial advancement in automated feature extraction and deformation analysis, contributing to more effective management of urban infrastructure.

Title: O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning

• Authors: Zhongzhen Huang, Gui Geng, Shengyi Hua, Zhen Huang, Haoyang Zou, Shaoting Zhang, Pengfei Liu, Xiaofan Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06458
• Pdf URL: https://arxiv.org/pdf/2501.06458
• Copy Paste: [[2501.06458]] O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning(https://arxiv.org/abs/2501.06458)
• Keywords: large language model
• Abstract: Building upon our previous investigations of O1 replication (Part 1: Journey Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]), this work explores the potential of inference-time scaling in large language models (LLMs) for medical reasoning tasks, ranging from diagnostic decision-making to treatment planning. Through extensive experiments on medical benchmarks of varying complexity (MedQA, Medbullets, and JAMA Clinical Challenges), our investigation reveals several key insights: (1) Increasing inference time does lead to improved performance. With a modest training set of 500 samples, our model yields substantial performance improvements of 6%-11%. (2) Task complexity directly correlates with the required length of reasoning chains, confirming the necessity of extended thought processes for challenging problems. (3) The differential diagnoses generated by our model adhere to the principles of the hypothetico-deductive method, producing a list of potential conditions that may explain a patient's symptoms and systematically narrowing these possibilities by evaluating the evidence. These findings demonstrate the promising synergy between inference-time scaling and journey learning in advancing LLMs' real-world clinical reasoning capabilities.

Title: MedCT: A Clinical Terminology Graph for Generative AI Applications in Healthcare

• Authors: Ye Chen, Dongdong Huang, Haoyun Xu, Cong Fu, Lin Sheng, Qingli Zhou, Yuqiang Shen, Kai Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06465
• Pdf URL: https://arxiv.org/pdf/2501.06465
• Copy Paste: [[2501.06465]] MedCT: A Clinical Terminology Graph for Generative AI Applications in Healthcare(https://arxiv.org/abs/2501.06465)
• Keywords: generative, large language model
• Abstract: We introduce the world's first clinical terminology for the Chinese healthcare community, namely MedCT, accompanied by a clinical foundation model MedBERT and an entity linking model MedLink. The MedCT system enables standardized and programmable representation of Chinese clinical data, successively stimulating the development of new medicines, treatment pathways, and better patient outcomes for the populous Chinese community. Moreover, the MedCT knowledge graph provides a principled mechanism to minimize the hallucination problem of large language models (LLMs), therefore achieving significant levels of accuracy and safety in LLM-based clinical applications. By leveraging the LLMs' emergent capabilities of generativeness and expressiveness, we were able to rapidly built a production-quality terminology system and deployed to real-world clinical field within three months, while classical terminologies like SNOMED CT have gone through more than twenty years development. Our experiments show that the MedCT system achieves state-of-the-art (SOTA) performance in semantic matching and entity linking tasks, not only for Chinese but also for English. We also conducted a longitudinal field experiment by applying MedCT and LLMs in a representative spectrum of clinical tasks, including electronic health record (EHR) auto-generation and medical document search for diagnostic decision making. Our study shows a multitude of values of MedCT for clinical workflows and patient outcomes, especially in the new genre of clinical LLM applications. We present our approach in sufficient engineering detail, such that implementing a clinical terminology for other non-English societies should be readily reproducible. We openly release our terminology, models and algorithms, along with real-world clinical datasets for the development.

Title: First Token Probability Guided RAG for Telecom Question Answering

• Authors: Tingwei Chen, Jiayi Chen, Zijian Zhao, Haolong Chen, Liang Zhang, Guangxu Zhu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06468
• Pdf URL: https://arxiv.org/pdf/2501.06468
• Copy Paste: [[2501.06468]] First Token Probability Guided RAG for Telecom Question Answering(https://arxiv.org/abs/2501.06468)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have garnered significant attention for their impressive general-purpose capabilities. For applications requiring intricate domain knowledge, Retrieval-Augmented Generation (RAG) has shown a distinct advantage in incorporating domain-specific information into LLMs. However, existing RAG research has not fully addressed the challenges of Multiple Choice Question Answering (MCQA) in telecommunications, particularly in terms of retrieval quality and mitigating hallucinations. To tackle these challenges, we propose a novel first token probability guided RAG framework. This framework leverages confidence scores to optimize key hyperparameters, such as chunk number and chunk window size, while dynamically adjusting the context. Our method starts by retrieving the most relevant chunks and generates a single token as the potential answer. The probabilities of all options are then normalized to serve as confidence scores, which guide the dynamic adjustment of the context. By iteratively optimizing the hyperparameters based on these confidence scores, we can continuously improve RAG performance. We conducted experiments to validate the effectiveness of our framework, demonstrating its potential to enhance accuracy in domain-specific MCQA tasks.

Title: Flash Window Attention: speedup the attention computation for Swin Transformer

• Authors: Zhendong Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06480
• Pdf URL: https://arxiv.org/pdf/2501.06480
• Copy Paste: [[2501.06480]] Flash Window Attention: speedup the attention computation for Swin Transformer(https://arxiv.org/abs/2501.06480)
• Keywords: transformer
• Abstract: To address the high resolution of image pixels, the Swin Transformer introduces window attention. This mechanism divides an image into non-overlapping windows and restricts attention computation to within each window, significantly enhancing computational efficiency. To further optimize this process, one might consider replacing standard attention with flash attention, which has proven to be more efficient in language models. However, a direct substitution is ineffective. Flash attention is designed for long sequences, whereas window attention deals with shorter sequences but must handle numerous of them in parallel. In this report, we present an optimized solution called Flash Window Attention, tailored specifically for window attention. Flash Window Attention improves attention computation efficiency by up to 300% and enhances end-to-end runtime efficiency by up to 30%. Our code is available online.

Title: Analyzing the Role of Context in Forecasting with Large Language Models

• Authors: Gerrit Mutschlechner, Adam Jatowt
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.06496
• Pdf URL: https://arxiv.org/pdf/2501.06496
• Copy Paste: [[2501.06496]] Analyzing the Role of Context in Forecasting with Large Language Models(https://arxiv.org/abs/2501.06496)
• Keywords: large language model
• Abstract: This study evaluates the forecasting performance of recent language models (LLMs) on binary forecasting questions. We first introduce a novel dataset of over 600 binary forecasting questions, augmented with related news articles and their concise question-related summaries. We then explore the impact of input prompts with varying level of context on forecasting performance. The results indicate that incorporating news articles significantly improves performance, while using few-shot examples leads to a decline in accuracy. We find that larger models consistently outperform smaller models, highlighting the potential of LLMs in enhancing automated forecasting.

Title: On the Reliability of Biometric Datasets: How Much Test Data Ensures Reliability?

• Authors: Matin Fallahi, Ragini Ramesh, Pankaja Priya Ramasamy, Patricia Arias Cabarcos, Thorsten Strufe, Philipp Terhörst
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06504
• Pdf URL: https://arxiv.org/pdf/2501.06504
• Copy Paste: [[2501.06504]] On the Reliability of Biometric Datasets: How Much Test Data Ensures Reliability?(https://arxiv.org/abs/2501.06504)
• Keywords: biometric
• Abstract: Biometric authentication is increasingly popular for its convenience and accuracy. However, while recent advancements focus on reducing errors and expanding modalities, the reliability of reported performance metrics often remains overlooked. Understanding reliability is critical, as it communicates how accurately reported error rates represent a system's actual performance, considering the uncertainty in error-rate estimates from test data. Currently, there is no widely accepted standard for reporting these uncertainties and indeed biometric studies rarely provide reliability estimates, limiting comparability and interpretation. To address this gap, we introduce BioQuake--a measure to estimate uncertainty in biometric verification systems--and empirically validate it on four systems and three datasets. Based on BioQuake, we provide simple guidelines for estimating performance uncertainty and facilitating reliable reporting. Additionally, we apply BioQuake to analyze biometric recognition performance on 62 biometric datasets used in research across eight modalities: face, fingerprint, gait, iris, keystroke, eye movement, Electroencephalogram (EEG), and Electrocardiogram (ECG). Our analysis shows that reported state-of-the-art performance often deviates significantly from actual error rates, potentially leading to inaccurate conclusions. To support researchers and foster the development of more reliable biometric systems and datasets, we release BioQuake as an easy-to-use web tool for reliability calculations.

Title: Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering

• Authors: Yinghao Hu, Leilei Gan, Wenyi Xiao, Kun Kuang, Fei Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06521
• Pdf URL: https://arxiv.org/pdf/2501.06521
• Copy Paste: [[2501.06521]] Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering(https://arxiv.org/abs/2501.06521)
• Keywords: large language model
• Abstract: Hallucination, or the generation of incorrect or fabricated information, remains a critical challenge in large language models (LLMs), particularly in high-stake domains such as legal question answering (QA). In order to mitigate the hallucination rate in legal QA, we first introduce a benchmark called LegalHalBench and three automatic metrics to evaluate the common hallucinations when LLMs answer legal questions. We then propose a hallucination mitigation method that integrates behavior cloning and a novel Hard Sample-aware Iterative Direct Preference Optimization (HIPO). We conduct extensive real-data experiments to validate the effectiveness of our approach. Our results demonstrate remarkable improvements in various metrics, including the newly proposed Non-Hallucinated Statute Rate, Statute Relevance Rate, Legal Claim Truthfulness, as well as traditional metrics such as METEOR, BERTScore, ROUGE-L, and win rates.

Title: Multi-View Factorizing and Disentangling: A Novel Framework for Incomplete Multi-View Multi-Label Classification

• Authors: Wulin Xie, Lian Zhao, Jiang Long, Xiaohuan Lu, Bingyan Nie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06524
• Pdf URL: https://arxiv.org/pdf/2501.06524
• Copy Paste: [[2501.06524]] Multi-View Factorizing and Disentangling: A Novel Framework for Incomplete Multi-View Multi-Label Classification(https://arxiv.org/abs/2501.06524)
• Keywords: robust
• Abstract: Multi-view multi-label classification (MvMLC) has recently garnered significant research attention due to its wide range of real-world applications. However, incompleteness in views and labels is a common challenge, often resulting from data collection oversights and uncertainties in manual annotation. Furthermore, the task of learning robust multi-view representations that are both view-consistent and view-specific from diverse views still a challenge problem in MvMLC. To address these issues, we propose a novel framework for incomplete multi-view multi-label classification (iMvMLC). Our method factorizes multi-view representations into two independent sets of factors: view-consistent and view-specific, and we correspondingly design a graph disentangling loss to fully reduce redundancy between these representations. Additionally, our framework innovatively decomposes consistent representation learning into three key sub-objectives: (i) how to extract view-shared information across different views, (ii) how to eliminate intra-view redundancy in consistent representations, and (iii) how to preserve task-relevant information. To this end, we design a robust task-relevant consistency learning module that collaboratively learns high-quality consistent representations, leveraging a masked cross-view prediction (MCP) strategy and information theory. Notably, all modules in our framework are developed to function effectively under conditions of incomplete views and labels, making our method adaptable to various multi-view and multi-label datasets. Extensive experiments on five datasets demonstrate that our method outperforms other leading approaches.

Title: Stingray: Fast Concurrent Transactions Without Consensus

• Authors: Srivatsan Sridhar, Alberto Sonnino, Lefteris Kokoris-Kogias
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.06531
• Pdf URL: https://arxiv.org/pdf/2501.06531
• Copy Paste: [[2501.06531]] Stingray: Fast Concurrent Transactions Without Consensus(https://arxiv.org/abs/2501.06531)
• Keywords: security
• Abstract: Recent advances have improved the throughput and latency of blockchains by processing transactions accessing different parts of the state concurrently. However, these systems are unable to concurrently process (a) transactions accessing the same state, even if they are (almost) commutative, e.g., payments much smaller than an account's balance, and (b) multi-party transactions, e.g., asset swaps. Moreover, they are slow to recover from contention, requiring once-in-a-day synchronization. We present Stingray, a novel blockchain architecture that addresses these limitations. The key conceptual contributions are a replicated bounded counter that processes (almost) commutative transactions concurrently, and a FastUnlock protocol that uses a fallback consensus protocol for fast contention recovery. We prove Stingray's security in an asynchronous network with Byzantine faults and demonstrate on a global testbed that Stingray achieves 10,000 times the throughput of prior systems for commutative workloads.

Title: DivTrackee versus DynTracker: Promoting Diversity in Anti-Facial Recognition against Dynamic FR Strategy

• Authors: Wenshu Fan, Minxing Zhang, Hongwei Li, Wenbo Jiang, Hanxiao Chen, Xiangyu Yue, Michael Backes, Xiao Zhang
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06533
• Pdf URL: https://arxiv.org/pdf/2501.06533
• Copy Paste: [[2501.06533]] DivTrackee versus DynTracker: Promoting Diversity in Anti-Facial Recognition against Dynamic FR Strategy(https://arxiv.org/abs/2501.06533)
• Keywords: privacy, protect, attack
• Abstract: The widespread adoption of facial recognition (FR) models raises serious concerns about their potential misuse, motivating the development of anti-facial recognition (AFR) to protect user facial privacy. In this paper, we argue that the static FR strategy, predominantly adopted in prior literature for evaluating AFR efficacy, cannot faithfully characterize the actual capabilities of determined trackers who aim to track a specific target identity. In particular, we introduce \emph{\ourAttack}, a dynamic FR strategy where the model's gallery database is iteratively updated with newly recognized target identity images. Surprisingly, such a simple approach renders all the existing AFR protections ineffective. To mitigate the privacy threats posed by DynTracker, we advocate for explicitly promoting diversity in the AFR-protected images. We hypothesize that the lack of diversity is the primary cause of the failure of existing AFR methods. Specifically, we develop \emph{DivTrackee}, a novel method for crafting diverse AFR protections that builds upon a text-guided image generation framework and diversity-promoting adversarial losses. Through comprehensive experiments on various facial image benchmarks and feature extractors, we demonstrate DynTracker's strength in breaking existing AFR methods and the superiority of DivTrackee in preventing user facial images from being identified by dynamic FR strategies. We believe our work can act as an important initial step towards developing more effective AFR methods for protecting user facial privacy against determined trackers.

Title: CeViT: Copula-Enhanced Vision Transformer in multi-task learning and bi-group image covariates with an application to myopia screening

• Authors: Chong Zhong, Yang Li, Jinfeng Xu, Xiang Fu, Yunhao Liu, Qiuyi Huang, Danjuan Yang, Meiyan Li, Aiyi Liu, Alan H. Welsh, Xingtao Zhou, Bo Fu, Catherine C. Liu
• Subjects: cs.CV, math.ST, stat.AP, stat.ME
• Abstract URL: https://arxiv.org/abs/2501.06540
• Pdf URL: https://arxiv.org/pdf/2501.06540
• Copy Paste: [[2501.06540]] CeViT: Copula-Enhanced Vision Transformer in multi-task learning and bi-group image covariates with an application to myopia screening(https://arxiv.org/abs/2501.06540)
• Keywords: transformer
• Abstract: We aim to assist image-based myopia screening by resolving two longstanding problems, "how to integrate the information of ocular images of a pair of eyes" and "how to incorporate the inherent dependence among high-myopia status and axial length for both eyes." The classification-regression task is modeled as a novel 4-dimensional muti-response regression, where discrete responses are allowed, that relates to two dependent 3rd-order tensors (3D ultrawide-field fundus images). We present a Vision Transformer-based bi-channel architecture, named CeViT, where the common features of a pair of eyes are extracted via a shared Transformer encoder, and the interocular asymmetries are modeled through separated multilayer perceptron heads. Statistically, we model the conditional dependence among mixture of discrete-continuous responses given the image covariates by a so-called copula loss. We establish a new theoretical framework regarding fine-tuning on CeViT based on latent representations, allowing the black-box fine-tuning procedure interpretable and guaranteeing higher relative efficiency of fine-tuning weight estimation in the asymptotic setting. We apply CeViT to an annotated ultrawide-field fundus image dataset collected by Shanghai Eye \& ENT Hospital, demonstrating that CeViT enhances the baseline model in both accuracy of classifying high-myopia and prediction of AL on both eyes.

Title: Natural Language Supervision for Low-light Image Enhancement

• Authors: Jiahui Tang, Kaihua Zhou, Zhijian Luo, Yueen Hou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06546
• Pdf URL: https://arxiv.org/pdf/2501.06546
• Copy Paste: [[2501.06546]] Natural Language Supervision for Low-light Image Enhancement(https://arxiv.org/abs/2501.06546)
• Keywords: robust
• Abstract: With the development of deep learning, numerous methods for low-light image enhancement (LLIE) have demonstrated remarkable performance. Mainstream LLIE methods typically learn an end-to-end mapping based on pairs of low-light and normal-light images. However, normal-light images under varying illumination conditions serve as reference images, making it difficult to define a ``perfect'' reference image This leads to the challenge of reconciling metric-oriented and visual-friendly results. Recently, many cross-modal studies have found that side information from other related modalities can guide visual representation learning. Based on this, we introduce a Natural Language Supervision (NLS) strategy, which learns feature maps from text corresponding to images, offering a general and flexible interface for describing an image under different illumination. However, image distributions conditioned on textual descriptions are highly multimodal, which makes training difficult. To address this issue, we design a Textual Guidance Conditioning Mechanism (TCM) that incorporates the connections between image regions and sentence words, enhancing the ability to capture fine-grained cross-modal cues for images and text. This strategy not only utilizes a wider range of supervised sources, but also provides a new paradigm for LLIE based on visual and textual feature alignment. In order to effectively identify and merge features from various levels of image and textual information, we design an Information Fusion Attention (IFA) module to enhance different regions at different levels. We integrate the proposed TCM and IFA into a Natural Language Supervision network for LLIE, named NaLSuper. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NaLSuper.

Title: Recommending the right academic programs: An interest mining approach using BERTopic

• Authors: Alessandro Hill, Kalen Goo, Puneet Agarwal
• Subjects: cs.LG, cs.CY, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.06581
• Pdf URL: https://arxiv.org/pdf/2501.06581
• Copy Paste: [[2501.06581]] Recommending the right academic programs: An interest mining approach using BERTopic(https://arxiv.org/abs/2501.06581)
• Keywords: fair
• Abstract: Prospective students face the challenging task of selecting a university program that will shape their academic and professional careers. For decision-makers and support services, it is often time-consuming and extremely difficult to match personal interests with suitable programs due to the vast and complex catalogue information available. This paper presents the first information system that provides students with efficient recommendations based on both program content and personal preferences. BERTopic, a powerful topic modeling algorithm, is used that leverages text embedding techniques to generate topic representations. It enables us to mine interest topics from all course descriptions, representing the full body of knowledge taught at the institution. Underpinned by the student's individual choice of topics, a shortlist of the most relevant programs is computed through statistical backtracking in the knowledge map, a novel characterization of the program-course relationship. This approach can be applied to a wide range of educational settings, including professional and vocational training. A case study at a post-secondary school with 80 programs and over 5,000 courses shows that the system provides immediate and effective decision support. The presented interest topics are meaningful, leading to positive effects such as serendipity, personalization, and fairness, as revealed by a qualitative study involving 65 students. Over 98% of users indicated that the recommendations aligned with their interests, and about 94% stated they would use the tool in the future. Quantitative analysis shows the system can be configured to ensure fairness, achieving 98% program coverage while maintaining a personalization score of 0.77. These findings suggest that this real-time, user-centered, data-driven system could improve the program selection process.

Title: Boundary-enhanced time series data imputation with long-term dependency diffusion models

• Authors: Chunjing Xiao, Xue Jiang, Xianghe Du, Wei Yang, Wei Lu, Xiaomin Wang, Kevin Chetty
• Subjects: cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2501.06585
• Pdf URL: https://arxiv.org/pdf/2501.06585
• Copy Paste: [[2501.06585]] Boundary-enhanced time series data imputation with long-term dependency diffusion models(https://arxiv.org/abs/2501.06585)
• Keywords: diffusion
• Abstract: Data imputation is crucial for addressing challenges posed by missing values in multivariate time series data across various fields, such as healthcare, traffic, and economics, and has garnered significant attention. Among various methods, diffusion model-based approaches show notable performance improvements. However, existing methods often cause disharmonious boundaries between missing and known regions and overlook long-range dependencies in missing data estimation, leading to suboptimal results. To address these issues, we propose a Diffusion-based time Series Data Imputation (DSDI) framework. We develop a weight-reducing injection strategy that incorporates the predicted values of missing points with reducing weights into the reverse diffusion process to mitigate boundary inconsistencies. Further, we introduce a multi-scale S4-based U-Net, which combines hierarchical information from different levels via multi-resolution integration to capture long-term dependencies. Experimental results demonstrate that our model outperforms existing imputation methods.

Title: Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping

• Authors: Muru Zhang, Mayank Mishra, Zhongzhu Zhou, William Brandon, Jue Wang, Yoon Kim, Jonathan Ragan-Kelley, Shuaiwen Leon Song, Ben Athiwaratkun, Tri Dao
• Subjects: cs.LG, cs.CL, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.06589
• Pdf URL: https://arxiv.org/pdf/2501.06589
• Copy Paste: [[2501.06589]] Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping(https://arxiv.org/abs/2501.06589)
• Keywords: transformer, large language model
• Abstract: Large language model inference is both memory-intensive and time-consuming, often requiring distributed algorithms to efficiently scale. Various model parallelism strategies are used in multi-gpu training and inference to partition computation across multiple devices, reducing memory load and computation time. However, using model parallelism necessitates communication of information between GPUs, which has been a major bottleneck and limits the gains obtained by scaling up the number of devices. We introduce Ladder Residual, a simple architectural modification applicable to all residual-based models that enables straightforward overlapping that effectively hides the latency of communication. Our insight is that in addition to systems optimization, one can also redesign the model architecture to decouple communication from computation. While Ladder Residual can allow communication-computation decoupling in conventional parallelism patterns, we focus on Tensor Parallelism in this paper, which is particularly bottlenecked by its heavy communication. For a Transformer model with 70B parameters, applying Ladder Residual to all its layers can achieve 30% end-to-end wall clock speed up at inference time with TP sharding over 8 devices. We refer the resulting Transformer model as the Ladder Transformer. We train a 1B and 3B Ladder Transformer from scratch and observe comparable performance to a standard dense transformer baseline. We also show that it is possible to convert parts of the Llama-3.1 8B model to our Ladder Residual architecture with minimal accuracy degradation by only retraining for 3B tokens.

Title: ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning

• Authors: Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06590
• Pdf URL: https://arxiv.org/pdf/2501.06590
• Copy Paste: [[2501.06590]] ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning(https://arxiv.org/abs/2501.06590)
• Keywords: large language model
• Abstract: Chemical reasoning usually involves complex, multi-step processes that demand precise calculations, where even minor errors can lead to cascading failures. Furthermore, large language models (LLMs) encounter difficulties handling domain-specific formulas, executing reasoning steps accurately, and integrating code effectively when tackling chemical reasoning tasks. To address these challenges, we present ChemAgent, a novel framework designed to improve the performance of LLMs through a dynamic, self-updating library. This library is developed by decomposing chemical tasks into sub-tasks and compiling these sub-tasks into a structured collection that can be referenced for future queries. Then, when presented with a new problem, ChemAgent retrieves and refines pertinent information from the library, which we call memory, facilitating effective task decomposition and the generation of solutions. Our method designs three types of memory and a library-enhanced reasoning component, enabling LLMs to improve over time through experience. Experimental results on four chemical reasoning datasets from SciBench demonstrate that ChemAgent achieves performance gains of up to 46% (GPT-4), significantly outperforming existing methods. Our findings suggest substantial potential for future applications, including tasks such as drug discovery and materials science. Our code can be found at this https URL

Title: Exploring Pose-Based Anomaly Detection for Retail Security: A Real-World Shoplifting Dataset and Benchmark

• Authors: Narges Rashvand, Ghazal Alinezhad Noghre, Armin Danesh Pazho, Shanle Yao, Hamed Tabkhi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06591
• Pdf URL: https://arxiv.org/pdf/2501.06591
• Copy Paste: [[2501.06591]] Exploring Pose-Based Anomaly Detection for Retail Security: A Real-World Shoplifting Dataset and Benchmark(https://arxiv.org/abs/2501.06591)
• Keywords: security, privacy
• Abstract: Shoplifting poses a significant challenge for retailers, resulting in billions of dollars in annual losses. Traditional security measures often fall short, highlighting the need for intelligent solutions capable of detecting shoplifting behaviors in real time. This paper frames shoplifting detection as an anomaly detection problem, focusing on the identification of deviations from typical shopping patterns. We introduce PoseLift, a privacy-preserving dataset specifically designed for shoplifting detection, addressing challenges such as data scarcity, privacy concerns, and model biases. PoseLift is built in collaboration with a retail store and contains anonymized human pose data from real-world scenarios. By preserving essential behavioral information while anonymizing identities, PoseLift balances privacy and utility. We benchmark state-of-the-art pose-based anomaly detection models on this dataset, evaluating performance using a comprehensive set of metrics. Our results demonstrate that pose-based approaches achieve high detection accuracy while effectively addressing privacy and bias concerns inherent in traditional methods. As one of the first datasets capturing real-world shoplifting behaviors, PoseLift offers researchers a valuable tool to advance computer vision ethically and will be publicly available to foster innovation and collaboration. The dataset is available at this https URL.

Title: EmoXpt: Analyzing Emotional Variances in Human Comments and LLM-Generated Responses

• Authors: Shireesh Reddy Pyreddy, Tarannum Shaila Zaman
• Subjects: cs.LG, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2501.06597
• Pdf URL: https://arxiv.org/pdf/2501.06597
• Copy Paste: [[2501.06597]] EmoXpt: Analyzing Emotional Variances in Human Comments and LLM-Generated Responses(https://arxiv.org/abs/2501.06597)
• Keywords: generative
• Abstract: The widespread adoption of generative AI has generated diverse opinions, with individuals expressing both support and criticism of its applications. This study investigates the emotional dynamics surrounding generative AI by analyzing human tweets referencing terms such as ChatGPT, OpenAI, Copilot, and LLMs. To further understand the emotional intelligence of ChatGPT, we examine its responses to selected tweets, highlighting differences in sentiment between human comments and LLM-generated responses. We introduce EmoXpt, a sentiment analysis framework designed to assess both human perspectives on generative AI and the sentiment embedded in ChatGPT's responses. Unlike prior studies that focus exclusively on human sentiment, EmoXpt uniquely evaluates the emotional expression of ChatGPT. Experimental results demonstrate that LLM-generated responses are notably more efficient, cohesive, and consistently positive than human responses.

Title: A Comparative Performance Analysis of Classification and Segmentation Models on Bangladeshi Pothole Dataset

• Authors: Antara Firoz Parsa, S. M. Abdullah, Anika Hasan Talukder, Md. Asif Shahidullah Kabbya, Shakib Al Hasan, Md. Farhadul Islam, Jannatun Noor
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06602
• Pdf URL: https://arxiv.org/pdf/2501.06602
• Copy Paste: [[2501.06602]] A Comparative Performance Analysis of Classification and Segmentation Models on Bangladeshi Pothole Dataset(https://arxiv.org/abs/2501.06602)
• Keywords: transformer, segmentation
• Abstract: The study involves a comprehensive performance analysis of popular classification and segmentation models, applied over a Bangladeshi pothole dataset, being developed by the authors of this research. This custom dataset of 824 samples, collected from the streets of Dhaka and Bogura performs competitively against the existing industrial and custom datasets utilized in the present literature. The dataset was further augmented four-fold for segmentation and ten-fold for classification evaluation. We tested nine classification models (CCT, CNN, INN, Swin Transformer, ConvMixer, VGG16, ResNet50, DenseNet201, and Xception) and four segmentation models (U-Net, ResU-Net, U-Net++, and Attention-Unet) over both the datasets. Among the classification models, lightweight models namely CCT, CNN, INN, Swin Transformer, and ConvMixer were emphasized due to their low computational requirements and faster prediction times. The lightweight models performed respectfully, oftentimes equating to the performance of heavyweight models. In addition, augmentation was found to enhance the performance of all the tested models. The experimental results exhibit that, our dataset performs on par or outperforms the similar classification models utilized in the existing literature, reaching accuracy and f1-scores over 99%. The dataset also performed on par with the existing datasets for segmentation, achieving model Dice Similarity Coefficient up to 67.54% and IoU scores up to 59.39%.

Title: Dual-Modality Representation Learning for Molecular Property Prediction

• Authors: Anyin Zhao, Zuquan Chen, Zhengyu Fang, Xiaoge Zhang, Jing Li
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2501.06608
• Pdf URL: https://arxiv.org/pdf/2501.06608
• Copy Paste: [[2501.06608]] Dual-Modality Representation Learning for Molecular Property Prediction(https://arxiv.org/abs/2501.06608)
• Keywords: transformer
• Abstract: Molecular property prediction has attracted substantial attention recently. Accurate prediction of drug properties relies heavily on effective molecular representations. The structures of chemical compounds are commonly represented as graphs or SMILES sequences. Recent advances in learning drug properties commonly employ Graph Neural Networks (GNNs) based on the graph representation. For the SMILES representation, Transformer-based architectures have been adopted by treating each SMILES string as a sequence of tokens. Because each representation has its own advantages and disadvantages, combining both representations in learning drug properties is a promising direction. We propose a method named Dual-Modality Cross-Attention (DMCA) that can effectively combine the strengths of two representations by employing the cross-attention mechanism. DMCA was evaluated across eight datasets including both classification and regression tasks. Results show that our method achieves the best overall performance, highlighting its effectiveness in leveraging the complementary information from both graph and SMILES modalities.

Title: Differentially Private Distribution Estimation Using Functional Approximation

• Authors: Ye Tao, Anand D. Sarwate
• Subjects: cs.CR, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.06620
• Pdf URL: https://arxiv.org/pdf/2501.06620
• Copy Paste: [[2501.06620]] Differentially Private Distribution Estimation Using Functional Approximation(https://arxiv.org/abs/2501.06620)
• Keywords: privacy, protect
• Abstract: The cumulative distribution function (CDF) is fundamental due to its ability to reveal information about random variables, making it essential in studies that require privacy-preserving methods to protect sensitive data. This paper introduces a novel privacy-preserving CDF method inspired by the functional analysis and functional mechanism. Our approach projects the empirical CDF into a predefined space, approximating it using specific functions, and protects the coefficients to achieve a differentially private empirical CDF. Compared to existing methods like histogram queries and adaptive quantiles, our method is preferable in decentralized settings and scenarios where CDFs must be updated with newly collected data.

Title: Scaling Down Semantic Leakage: Investigating Associative Bias in Smaller Language Models

• Authors: Veronika Smilga
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06638
• Pdf URL: https://arxiv.org/pdf/2501.06638
• Copy Paste: [[2501.06638]] Scaling Down Semantic Leakage: Investigating Associative Bias in Smaller Language Models(https://arxiv.org/abs/2501.06638)
• Keywords: large language model
• Abstract: Semantic leakage is a phenomenon recently introduced by Gonen et al. (2024). It refers to a situation in which associations learnt from the training data emerge in language model generations in an unexpected and sometimes undesired way. Prior work has focused on leakage in large language models (7B+ parameters). In this study, I use Qwen2.5 model family to explore whether smaller models, ranging from 500M to 7B parameters, demonstrate less semantic leakage due to their limited capacity for capturing complex associations. Building on the previous dataset from Gonen et al. (2024), I introduce a new dataset of color-focused prompts, categorized into specific types of semantic associations, to systematically evaluate the models' performance. Results indicate that smaller models exhibit less semantic leakage overall, although this trend is not strictly linear, with medium-sized models sometimes surpassing larger ones in leaking behavior. The dataset, the model generations, and the evaluation code are publicly available at this https URL.

Title: FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings

• Authors: Tong Liu, Xiao Yu, Wenxuan Zhou, Jindong Gu, Volker Tresp
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06645
• Pdf URL: https://arxiv.org/pdf/2501.06645
• Copy Paste: [[2501.06645]] FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings(https://arxiv.org/abs/2501.06645)
• Keywords: large language model
• Abstract: Efficient preference optimization algorithms such as Direct Preference Optimization (DPO) have become a popular approach in aligning large language models (LLMs) with human preferences. These algorithms implicitly treat the LLM as a reward model, and focus on training it to correct misranked preference pairs. However, recent work~\citep{chen2024preference} empirically finds that DPO training \textit{rarely improves these misranked preference pairs}, despite its gradient emphasizing on these cases. We introduce FocalPO, a DPO variant that instead \textit{down-weighs} misranked preference pairs and prioritizes enhancing the model's understanding of pairs that it can already rank correctly. Inspired by Focal Loss used in vision tasks, FocalPO achieves this by adding a modulating factor to dynamically scale DPO loss. Our experiment demonstrates that FocalPO surpasses DPO and its variants on popular benchmarks like Alpaca Eval 2.0 using Mistral-Base-7B and Llama-3-Instruct-8B. Additionally, we empirically reveals how FocalPO affects training on correct and incorrect sample groups, further underscoring its effectiveness.

Title: RogueRFM: Attacking Refresh Management for Covert-Channel and Denial-of-Service

• Authors: Hritvik Taneja, Moinuddin Qureshi
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2501.06646
• Pdf URL: https://arxiv.org/pdf/2501.06646
• Copy Paste: [[2501.06646]] RogueRFM: Attacking Refresh Management for Covert-Channel and Denial-of-Service(https://arxiv.org/abs/2501.06646)
• Keywords: security, defense, attack, steal
• Abstract: With lowering thresholds, transparently defending against Rowhammer within DRAM is challenging due to the lack of time to perform mitigation. Commercially deployed in-DRAM defenses like TRR that steal time from normal refreshes~(REF) to perform mitigation have been proven ineffective against Rowhammer. In response, a new Refresh Management (RFM) interface has been added to the DDR5 specifications. RFM provides dedicated time to an in-DRAM defense to perform mitigation. Several recent works have used RFM for the intended purpose - building better Rowhammer defenses. However, to the best of our knowledge, no prior study has looked at the potential security implications of this new feature if an attacker subjects it to intentional misuse. Our paper shows that RFM introduces new side effects in the system - the activity of one bank causes interference with the operation of the other banks. Thus, the latency of a bank becomes dependent on the activity of other banks. We use these side effects to build two new attacks. First, a novel memory-based covert channel, which has a bandwidth of up to 31.3 KB/s, and is also effective even in a bank-partitioned system. Second, a new Denial-of-Service (DOS) attack pattern that exploits the activity within a single bank to reduce the performance of the other banks. Our experiments on SPEC2017, PARSEC, and LIGRA workloads show a slowdown of up to 67\% when running alongside our DOS pattern. We also discuss potential countermeasures for our attacks.

Title: SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in Split Learning

• Authors: Phillip Rieger, Alessandro Pegoraro, Kavita Kumari, Tigist Abera, Jonathan Knauer, Ahmad-Reza Sadeghi
• Subjects: cs.CR, cs.DC, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06650
• Pdf URL: https://arxiv.org/pdf/2501.06650
• Copy Paste: [[2501.06650]] SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in Split Learning(https://arxiv.org/abs/2501.06650)
• Keywords: defense, attack, federate
• Abstract: Split Learning (SL) is a distributed deep learning approach enabling multiple clients and a server to collaboratively train and infer on a shared deep neural network (DNN) without requiring clients to share their private local data. The DNN is partitioned in SL, with most layers residing on the server and a few initial layers and inputs on the client side. This configuration allows resource-constrained clients to participate in training and inference. However, the distributed architecture exposes SL to backdoor attacks, where malicious clients can manipulate local datasets to alter the DNN's behavior. Existing defenses from other distributed frameworks like Federated Learning are not applicable, and there is a lack of effective backdoor defenses specifically designed for SL. We present SafeSplit, the first defense against client-side backdoor attacks in Split Learning (SL). SafeSplit enables the server to detect and filter out malicious client behavior by employing circular backward analysis after a client's training is completed, iteratively reverting to a trained checkpoint where the model under examination is found to be benign. It uses a two-fold analysis to identify client-induced changes and detect poisoned models. First, a static analysis in the frequency domain measures the differences in the layer's parameters at the server. Second, a dynamic analysis introduces a novel rotational distance metric that assesses the orientation shifts of the server's layer parameters during training. Our comprehensive evaluation across various data distributions, client counts, and attack scenarios demonstrates the high efficacy of this dual analysis in mitigating backdoor attacks while preserving model utility.

Title: Parking Space Detection in the City of Granada

• Authors: Crespo-Orti Luis, Moreno-Cuadrado Isabel, Olivares-Martínez Pablo, Sanz-Tornero Ximo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06651
• Pdf URL: https://arxiv.org/pdf/2501.06651
• Copy Paste: [[2501.06651]] Parking Space Detection in the City of Granada(https://arxiv.org/abs/2501.06651)
• Keywords: segmentation
• Abstract: This paper addresses the challenge of parking space detection in urban areas, focusing on the city of Granada. Utilizing aerial imagery, we develop and apply semantic segmentation techniques to accurately identify parked cars, moving cars and roads. A significant aspect of our research is the creation of a proprietary dataset specific to Granada, which is instrumental in training our neural network model. We employ Fully Convolutional Networks, Pyramid Networks and Dilated Convolutions, demonstrating their effectiveness in urban semantic segmentation. Our approach involves comparative analysis and optimization of various models, including Dynamic U-Net, PSPNet and DeepLabV3+, tailored for the segmentation of aerial images. The study includes a thorough experimentation phase, using datasets such as UDD5 and UAVid, alongside our custom Granada dataset. We evaluate our models using metrics like Foreground Accuracy, Dice Coefficient and Jaccard Index. Our results indicate that DeepLabV3+ offers the most promising performance. We conclude with future directions, emphasizing the need for a dedicated neural network for parked car detection and the potential for application in other urban environments. This work contributes to the fields of urban planning and traffic management, providing insights into efficient utilization of parking spaces through advanced image processing techniques.

Title: Personalized Preference Fine-tuning of Diffusion Models

• Authors: Meihua Dang, Anikait Singh, Linqi Zhou, Stefano Ermon, Jiaming Song
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06655
• Pdf URL: https://arxiv.org/pdf/2501.06655
• Copy Paste: [[2501.06655]] Personalized Preference Fine-tuning of Diffusion Models(https://arxiv.org/abs/2501.06655)
• Keywords: diffusion
• Abstract: RLHF techniques like DPO can significantly improve the generation quality of text-to-image diffusion models. However, these methods optimize for a single reward that aligns model generation with population-level preferences, neglecting the nuances of individual users' beliefs or values. This lack of personalization limits the efficacy of these models. To bridge this gap, we introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences. With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way, enabling generalization to unseen users. Specifically, our approach (1) leverages a vision-language model (VLM) to extract personal preference embeddings from a small set of pairwise preference examples, and then (2) incorporates the embeddings into diffusion models through cross attention. Conditioning on user embeddings, the text-to-image models are fine-tuned with the DPO objective, simultaneously optimizing for alignment with the preferences of multiple users. Empirical results demonstrate that our method effectively optimizes for multiple reward functions and can interpolate between them during inference. In real-world user scenarios, with as few as four preference examples from a new user, our approach achieves an average win rate of 76\% over Stable Cascade, generating images that more accurately reflect specific user preferences.

Title: Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization

• Authors: Jiayi Tian, Jinming Lu, Hai Li, Xiangwei Wang, Cong (Callie)Hao, Ian Young, Zheng Zhang
• Subjects: cs.LG, cs.AR, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06663
• Pdf URL: https://arxiv.org/pdf/2501.06663
• Copy Paste: [[2501.06663]] Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization(https://arxiv.org/abs/2501.06663)
• Keywords: privacy, transformer
• Abstract: Transformer models have achieved state-of-the-art performance across a wide range of machine learning tasks. There is growing interest in training transformers on resource-constrained edge devices due to considerations such as privacy, domain adaptation, and on-device scientific machine learning. However, the significant computational and memory demands required for transformer training often exceed the capabilities of an edge device. Leveraging low-rank tensor compression, this paper presents the first on-FPGA accelerator for end-to-end transformer training. On the algorithm side, we present a bi-directional contraction flow for tensorized transformer training, significantly reducing the computational FLOPS and intra-layer memory costs compared to existing tensor operations. On the hardware side, we store all highly compressed model parameters and gradient information on chip, creating an on-chip-memory-only framework for each stage in training. This reduces off-chip communication and minimizes latency and energy costs. Additionally, we implement custom computing kernels for each training stage and employ intra-layer parallelism and pipe-lining to further enhance run-time and memory efficiency. Through experiments on transformer models within $36.7$ to $93.5$ MB using FP-32 data formats on the ATIS dataset, our tensorized FPGA accelerator could conduct single-batch end-to-end training on the AMD Alevo U50 FPGA, with a memory budget of less than $6$-MB BRAM and $22.5$-MB URAM. Compared to uncompressed training on the NVIDIA RTX 3090 GPU, our on-FPGA training achieves a memory reduction of $30\times$ to $51\times$. Our FPGA accelerator also achieves up to $3.6\times$ less energy cost per epoch compared with tensor Transformer training on an NVIDIA RTX 3090 GPU.

Title: Imbalanced Medical Image Segmentation with Pixel-dependent Noisy Labels

• Authors: Erjian Guo, Zicheng Wang, Zhen Zhao, Luping Zhou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06678
• Pdf URL: https://arxiv.org/pdf/2501.06678
• Copy Paste: [[2501.06678]] Imbalanced Medical Image Segmentation with Pixel-dependent Noisy Labels(https://arxiv.org/abs/2501.06678)
• Keywords: robust, segmentation
• Abstract: Accurate medical image segmentation is often hindered by noisy labels in training data, due to the challenges of annotating medical images. Prior research works addressing noisy labels tend to make class-dependent assumptions, overlooking the pixel-dependent nature of most noisy labels. Furthermore, existing methods typically apply fixed thresholds to filter out noisy labels, risking the removal of minority classes and consequently degrading segmentation performance. To bridge these gaps, our proposed framework, Collaborative Learning with Curriculum Selection (CLCS), addresses pixel-dependent noisy labels with class imbalance. CLCS advances the existing works by i) treating noisy labels as pixel-dependent and addressing them through a collaborative learning framework, and ii) employing a curriculum dynamic thresholding approach adapting to model learning progress to select clean data samples to mitigate the class imbalance issue, and iii) applying a noise balance loss to noisy data samples to improve data utilization instead of discarding them outright. Specifically, our CLCS contains two modules: Curriculum Noisy Label Sample Selection (CNS) and Noise Balance Loss (NBL). In the CNS module, we designed a two-branch network with discrepancy loss for collaborative learning so that different feature representations of the same instance could be extracted from distinct views and used to vote the class probabilities of pixels. Besides, a curriculum dynamic threshold is adopted to select clean-label samples through probability voting. In the NBL module, instead of directly dropping the suspiciously noisy labels, we further adopt a robust loss to leverage such instances to boost the performance.

Title: Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving

• Authors: Haoxiang Gao, Yu Zhao
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.06680
• Pdf URL: https://arxiv.org/pdf/2501.06680
• Copy Paste: [[2501.06680]] Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving(https://arxiv.org/abs/2501.06680)
• Keywords: large language model
• Abstract: Autonomous driving (AD) has experienced significant improvements in recent years and achieved promising 3D detection, classification, and localization results. However, many challenges remain, e.g. semantic understanding of pedestrians' behaviors, and downstream handling for pedestrian interactions. Recent studies in applications of Large Language Models (LLM) and Vision-Language Models (VLM) have achieved promising results in scene understanding and high-level maneuver planning in diverse traffic scenarios. However, deploying the billion-parameter LLMs to vehicles requires significant computation and memory resources. In this paper, we analyzed effective knowledge distillation of semantic labels to smaller Vision networks, which can be used for the semantic representation of complex scenes for downstream decision-making for planning and control.

Title: Understanding and Mitigating Membership Inference Risks of Neural Ordinary Differential Equations

• Authors: Sanghyun Hong, Fan Wu, Anthony Gruber, Kookjin Lee
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06686
• Pdf URL: https://arxiv.org/pdf/2501.06686
• Copy Paste: [[2501.06686]] Understanding and Mitigating Membership Inference Risks of Neural Ordinary Differential Equations(https://arxiv.org/abs/2501.06686)
• Keywords: privacy, attack, membership infer
• Abstract: Neural ordinary differential equations (NODEs) are an emerging paradigm in scientific computing for modeling dynamical systems. By accurately learning underlying dynamics in data in the form of differential equations, NODEs have been widely adopted in various domains, such as healthcare, finance, computer vision, and language modeling. However, there remains a limited understanding of the privacy implications of these fundamentally different models, particularly with regard to their membership inference risks. In this work, we study the membership inference risks associated with NODEs. We first comprehensively evaluate NODEs against membership inference attacks. We show that NODEs are twice as resistant to these privacy attacks compared to conventional feedforward models such as ResNets. By analyzing the variance in membership risks across different NODE models, we identify the factors that contribute to their lower risks. We then demonstrate, both theoretically and empirically, that membership inference risks can be further mitigated by utilizing a stochastic variant of NODEs: Neural stochastic differential equations (NSDEs). We show that NSDEs are differentially-private (DP) learners that provide the same provable privacy guarantees as DP-SGD, the de-facto mechanism for training private models. NSDEs are also effective in mitigating existing membership inference attacks, demonstrating risks comparable to private models trained with DP-SGD while offering an improved privacy-utility trade-off. Moreover, we propose a drop-in-replacement strategy that efficiently integrates NSDEs into conventional feedforward models to enhance their privacy.

Title: TAPO: Task-Referenced Adaptation for Prompt Optimization

• Authors: Wenxin Luo, Weirui Wang, Xiaopeng Li, Weibo Zhou, Pengyue Jia, Xiangyu Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06689
• Pdf URL: https://arxiv.org/pdf/2501.06689
• Copy Paste: [[2501.06689]] TAPO: Task-Referenced Adaptation for Prompt Optimization(https://arxiv.org/abs/2501.06689)
• Keywords: large language model
• Abstract: Prompt engineering can significantly improve the performance of large language models (LLMs), with automated prompt optimization (APO) gaining significant attention due to the time-consuming and laborious nature of manual prompt design. However, much of the existing work in APO overlooks task-specific characteristics, resulting in prompts that lack domain specificity and are not well-suited for task-specific optimization. In this paper, we introduce TAPO, a multitask-aware prompt optimization framework composed of three key modules. First, a task-aware metric selection module is proposed to enhance task-specific prompt generation capabilities. Second, we present a multi-metrics evaluation module to jointly evaluate prompts from multiple perspectives. Third, an evolution-based optimization framework is introduced for automatic prompt refinement, which improves adaptability across various tasks. Extensive experiments on six datasets demonstrate the effectiveness of our approach, and our code is publicly available.

Title: PGP-SAM: Prototype-Guided Prompt Learning for Efficient Few-Shot Medical Image Segmentation

• Authors: Zhonghao Yan, Zijin Yin, Tianyu Lin, Xiangzhu Zeng, Kongming Liang, Zhanyu Ma
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06692
• Pdf URL: https://arxiv.org/pdf/2501.06692
• Copy Paste: [[2501.06692]] PGP-SAM: Prototype-Guided Prompt Learning for Efficient Few-Shot Medical Image Segmentation(https://arxiv.org/abs/2501.06692)
• Keywords: segmentation
• Abstract: The Segment Anything Model (SAM) has demonstrated strong and versatile segmentation capabilities, along with intuitive prompt-based interactions. However, customizing SAM for medical image segmentation requires massive amounts of pixel-level annotations and precise point- or box-based prompt designs. To address these challenges, we introduce PGP-SAM, a novel prototype-based few-shot tuning approach that uses limited samples to replace tedious manual prompts. Our key idea is to leverage inter- and intra-class prototypes to capture class-specific knowledge and relationships. We propose two main components: (1) a plug-and-play contextual modulation module that integrates multi-scale information, and (2) a class-guided cross-attention mechanism that fuses prototypes and features for automatic prompt generation. Experiments on a public multi-organ dataset and a private ventricle dataset demonstrate that PGP-SAM achieves superior mean Dice scores compared with existing prompt-free SAM variants, while using only 10\% of the 2D slices.

Title: Mamba-MOC: A Multicategory Remote Object Counting via State Space Model

• Authors: Peng Liu, Sen Lei, Heng-Chao Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06697
• Pdf URL: https://arxiv.org/pdf/2501.06697
• Copy Paste: [[2501.06697]] Mamba-MOC: A Multicategory Remote Object Counting via State Space Model(https://arxiv.org/abs/2501.06697)
• Keywords: transformer
• Abstract: Multicategory remote object counting is a fundamental task in computer vision, aimed at accurately estimating the number of objects of various categories in remote images. Existing methods rely on CNNs and Transformers, but CNNs struggle to capture global dependencies, and Transformers are computationally expensive, which limits their effectiveness in remote applications. Recently, Mamba has emerged as a promising solution in the field of computer vision, offering a linear complexity for modeling global dependencies. To this end, we propose Mamba-MOC, a mamba-based network designed for multi-category remote object counting, which represents the first application of Mamba to remote sensing object counting. Specifically, we propose a cross-scale interaction module to facilitate the deep integration of hierarchical features. Then we design a context state space model to capture both global and local contextual information and provide local neighborhood information during the scan process. Experimental results in large-scale realistic scenarios demonstrate that our proposed method achieves state-of-the-art performance compared with some mainstream counting algorithms.

Title: Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints

• Authors: Ming Dai, Jian Li, Jiedong Zhuang, Xian Zhang, Wankou Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06710
• Pdf URL: https://arxiv.org/pdf/2501.06710
• Copy Paste: [[2501.06710]] Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints(https://arxiv.org/abs/2501.06710)
• Keywords: robust, transformer, segmentation
• Abstract: Multi-task visual grounding involves the simultaneous execution of localization and segmentation in images based on textual expressions. The majority of advanced methods predominantly focus on transformer-based multimodal fusion, aiming to extract robust multimodal representations. However, ambiguity between referring expression comprehension (REC) and referring image segmentation (RIS) is error-prone, leading to inconsistencies between multi-task predictions. Besides, insufficient multimodal understanding directly contributes to biased target perception. To overcome these challenges, we propose a Coarse-to-fine Consistency Constraints Visual Grounding architecture ($\text{C}^3\text{VG}$), which integrates implicit and explicit modeling approaches within a two-stage framework. Initially, query and pixel decoders are employed to generate preliminary detection and segmentation outputs, a process referred to as the Rough Semantic Perception (RSP) stage. These coarse predictions are subsequently refined through the proposed Mask-guided Interaction Module (MIM) and a novel explicit bidirectional consistency constraint loss to ensure consistent representations across tasks, which we term the Refined Consistency Interaction (RCI) stage. Furthermore, to address the challenge of insufficient multimodal understanding, we leverage pre-trained models based on visual-linguistic fusion representations. Empirical evaluations on the RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate the efficacy and soundness of $\text{C}^3\text{VG}$, which significantly outperforms state-of-the-art REC and RIS methods by a substantial margin. Code and model will be available at \url{this https URL}.

Title: F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting

• Authors: Yuxin Wang, Qianyi Wu, Dan Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06714
• Pdf URL: https://arxiv.org/pdf/2501.06714
• Copy Paste: [[2501.06714]] F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting(https://arxiv.org/abs/2501.06714)
• Keywords: robust
• Abstract: This paper tackles the problem of generalizable 3D-aware generation from monocular datasets, e.g., ImageNet. The key challenge of this task is learning a robust 3D-aware representation without multi-view or dynamic data, while ensuring consistent texture and geometry across different viewpoints. Although some baseline methods are capable of 3D-aware generation, the quality of the generated images still lags behind state-of-the-art 2D generation approaches, which excel in producing high-quality, detailed images. To address this severe limitation, we propose a novel feed-forward pipeline based on pixel-aligned Gaussian Splatting, coined as F3D-Gaus, which can produce more realistic and reliable 3D renderings from monocular inputs. In addition, we introduce a self-supervised cycle-consistent constraint to enforce cross-view consistency in the learned 3D representation. This training strategy naturally allows aggregation of multiple aligned Gaussian primitives and significantly alleviates the interpolation limitations inherent in single-view pixel-aligned Gaussian Splatting. Furthermore, we incorporate video model priors to perform geometry-aware refinement, enhancing the generation of fine details in wide-viewpoint scenarios and improving the model's capability to capture intricate 3D textures. Extensive experiments demonstrate that our approach not only achieves high-quality, multi-view consistent 3D-aware generation from monocular datasets, but also significantly improves training and inference efficiency.

Title: ZNO-Eval: Benchmarking reasoning capabilities of large language models in Ukrainian

• Authors: Mykyta Syromiatnikov, Victoria Ruvinskaya, Anastasiya Troynina
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06715
• Pdf URL: https://arxiv.org/pdf/2501.06715
• Copy Paste: [[2501.06715]] ZNO-Eval: Benchmarking reasoning capabilities of large language models in Ukrainian(https://arxiv.org/abs/2501.06715)
• Keywords: robust, large language model
• Abstract: As the usage of large language models for problems outside of simple text understanding or generation increases, assessing their abilities and limitations becomes crucial. While significant progress has been made in this area over the last few years, most research has focused on benchmarking English, leaving other languages underexplored. This makes evaluating the reasoning and robustness level of language models in Ukrainian particularly challenging. The purpose of this work is to establish a comprehensive benchmark for the reasoning capabilities evaluation of large language models in the Ukrainian language. This paper presents the ZNO-Eval benchmark based on real exam tasks from Ukraine's standardized educational testing system: the External Independent Evaluation and the National Multi-subject Test. With single-answer options, multiple-choice, matching, and open-ended questions from diverse subjects, including Ukrainian language, mathematics, history, and geography, this dataset paves the way toward a thorough analysis of reasoning capabilities across different domains and complexities. Evaluation of several well-known language models, such as GPT-3.5-Turbo, GPT-4o, GPT-4-Turbo, Mistral Large, Claude 3 Opus, and Gemini-1.5 Pro on this benchmark demonstrated the superiority of GPT-4o in both common knowledge reasoning and intricate language tasks. At the same time, Gemini Pro and GPT-4 Turbo excelled in the arithmetic domain, leading in single-answer and open-ended math problems. While all models were close to max performance in text-only common knowledge tasks like history and geography, there still is a gap for Ukrainian language and math, thus highlighting the importance of developing specialized language benchmarks for more accurate assessments of model capabilities and limitations across different languages and contexts.

Title: DRDT3: Diffusion-Refined Decision Test-Time Training Model

• Authors: Xingshuai Huang, Di Wu, Benoit Boulet
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06718
• Pdf URL: https://arxiv.org/pdf/2501.06718
• Copy Paste: [[2501.06718]] DRDT3: Diffusion-Refined Decision Test-Time Training Model(https://arxiv.org/abs/2501.06718)
• Keywords: diffusion, transformer, generative
• Abstract: Decision Transformer (DT), a trajectory modeling method, has shown competitive performance compared to traditional offline reinforcement learning (RL) approaches on various classic control tasks. However, it struggles to learn optimal policies from suboptimal, reward-labeled trajectories. In this study, we explore the use of conditional generative modeling to facilitate trajectory stitching given its high-quality data generation ability. Additionally, recent advancements in Recurrent Neural Networks (RNNs) have shown their linear complexity and competitive sequence modeling performance over Transformers. We leverage the Test-Time Training (TTT) layer, an RNN that updates hidden states during testing, to model trajectories in the form of DT. We introduce a unified framework, called Diffusion-Refined Decision TTT (DRDT3), to achieve performance beyond DT models. Specifically, we propose the Decision TTT (DT3) module, which harnesses the sequence modeling strengths of both self-attention and the TTT layer to capture recent contextual information and make coarse action predictions. We further integrate DT3 with the diffusion model using a unified optimization objective. With experiments on multiple tasks of Gym and AntMaze in the D4RL benchmark, our DT3 model without diffusion refinement demonstrates improved performance over standard DT, while DRDT3 further achieves superior results compared to state-of-the-art conventional offline RL and DT-based methods.

Title: Measuring the Robustness of Reference-Free Dialogue Evaluation Systems

• Authors: Justin Vasselli, Adam Nohejl, Taro Watanabe
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06728
• Pdf URL: https://arxiv.org/pdf/2501.06728
• Copy Paste: [[2501.06728]] Measuring the Robustness of Reference-Free Dialogue Evaluation Systems(https://arxiv.org/abs/2501.06728)
• Keywords: attack, robust, large language model
• Abstract: Advancements in dialogue systems powered by large language models (LLMs) have outpaced the development of reliable evaluation metrics, particularly for diverse and creative responses. We present a benchmark for evaluating the robustness of reference-free dialogue metrics against four categories of adversarial attacks: speaker tag prefixes, static responses, ungrammatical responses, and repeated conversational context. We analyze metrics such as DialogRPT, UniEval, and PromptEval -- a prompt-based method leveraging LLMs -- across grounded and ungrounded datasets. By examining both their correlation with human judgment and susceptibility to adversarial attacks, we find that these two axes are not always aligned; metrics that appear to be equivalent when judged by traditional benchmarks may, in fact, vary in their scores of adversarial responses. These findings motivate the development of nuanced evaluation frameworks to address real-world dialogue challenges.

Title: KeTS: Kernel-based Trust Segmentation against Model Poisoning Attacks

• Authors: Ankit Gangwal, Mauro Conti, Tommaso Pauselli
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06729
• Pdf URL: https://arxiv.org/pdf/2501.06729
• Copy Paste: [[2501.06729]] KeTS: Kernel-based Trust Segmentation against Model Poisoning Attacks(https://arxiv.org/abs/2501.06729)
• Keywords: defense, attack, robust, federate, segmentation
• Abstract: Federated Learning (FL) enables multiple users to collaboratively train a global model in a distributed manner without revealing their personal data. However, FL remains vulnerable to model poisoning attacks, where malicious actors inject crafted updates to compromise the global model's accuracy. These vulnerabilities are particularly severe in non-homogeneous environments, where clients exhibit varying proportions of class labels, resulting in heterogeneous updates. In such settings, benign outliers are often misclassified as false positives, while maliciously crafted uploads evade detection and are aggregated at the server. Existing defense mechanisms struggle in such real-world settings, resulting in significant declines in the global FL model's performance. We propose a novel defense mechanism, Kernel-based Trust Segmentation (KeTS), to counter model poisoning attacks. Unlike existing approaches, KeTS analyzes the evolution of each client's updates and effectively segments malicious clients using Kernel Density Estimation (KDE), even in the presence of benign outliers. We thoroughly evaluate KeTS's performance against the six most effective model poisoning attacks (i.e., Trim-Attack, Krum-Attack, Min-Max attack, Min-Sum attack, and their variants) on two different datasets (i.e., MNIST and Fashion-MNIST) and compare its performance with three classical robust schemes (i.e., Krum, Trim-Mean, and Median) and a state-of-the-art defense (i.e., FLTrust). Our results show that KeTS outperforms the existing defenses in every attack setting; beating the best-performing defense by an overall average of >24% (on MNIST) and >14% (on Fashion-MNIST). A series of further experiments (varying poisoning approaches, attacker population, etc.) reveal the consistent and superior performance of KeTS under diverse conditions.

Title: Better Prompt Compression Without Multi-Layer Perceptrons

• Authors: Edouardo Honig, Andrew Lizarraga, Zijun Frank Zhang, Ying Nian Wu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06730
• Pdf URL: https://arxiv.org/pdf/2501.06730
• Copy Paste: [[2501.06730]] Better Prompt Compression Without Multi-Layer Perceptrons(https://arxiv.org/abs/2501.06730)
• Keywords: transformer, generative
• Abstract: Prompt compression is a promising approach to speeding up language model inference without altering the generative model. Prior works compress prompts into smaller sequences of learned tokens using an encoder that is trained as a LowRank Adaptation (LoRA) of the inference language model. However, we show that the encoder does not need to keep the original language model's architecture to achieve useful compression. We introduce the Attention-Only Compressor (AOC), which learns a prompt compression encoder after removing the multilayer perceptron (MLP) layers in the Transformer blocks of a language model, resulting in an encoder with roughly 67% less parameters compared to the original model. Intriguingly we find that, across a range of compression ratios up to 480x, AOC can better regenerate prompts and outperform a baseline compression encoder that is a LoRA of the inference language model without removing MLP layers. These results demonstrate that the architecture of prompt compression encoders does not need to be identical to that of the original decoder language model, paving the way for further research into architectures and approaches for prompt compression.

Title: ZOQO: Zero-Order Quantized Optimization

• Authors: Noga Bar, Raja Giryes
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06736
• Pdf URL: https://arxiv.org/pdf/2501.06736
• Copy Paste: [[2501.06736]] ZOQO: Zero-Order Quantized Optimization(https://arxiv.org/abs/2501.06736)
• Keywords: attack, large language model
• Abstract: The increasing computational and memory demands in deep learning present significant challenges, especially in resource-constrained environments. We introduce a zero-order quantized optimization (ZOQO) method designed for training models with quantized parameters and operations. Our approach leverages zero-order approximations of the gradient sign and adapts the learning process to maintain the parameters' quantization without the need for full-precision gradient calculations. We demonstrate the effectiveness of ZOQO through experiments in fine-tuning of large language models and black-box adversarial attacks. Despite the limitations of zero-order and quantized operations training, our method achieves competitive performance compared to full-precision methods, highlighting its potential for low-resource environments.

Title: Rice Leaf Disease Detection: A Comparative Study Between CNN, Transformer and Non-neural Network Architectures

• Authors: Samia Mehnaz, Md. Touhidul Islam
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06740
• Pdf URL: https://arxiv.org/pdf/2501.06740
• Copy Paste: [[2501.06740]] Rice Leaf Disease Detection: A Comparative Study Between CNN, Transformer and Non-neural Network Architectures(https://arxiv.org/abs/2501.06740)
• Keywords: transformer
• Abstract: In nations such as Bangladesh, agriculture plays a vital role in providing livelihoods for a significant portion of the population. Identifying and classifying plant diseases early is critical to prevent their spread and minimize their impact on crop yield and quality. Various computer vision techniques can be used for such detection and classification. While CNNs have been dominant on such image classification tasks, vision transformers has become equally good in recent time also. In this paper we study the various computer vision techniques for Bangladeshi rice leaf disease detection. We use the Dhan-Shomadhan -- a Bangladeshi rice leaf disease dataset, to experiment with various CNN and ViT models. We also compared the performance of such deep neural network architecture with traditional machine learning architecture like Support Vector Machine(SVM). We leveraged transfer learning for better generalization with lower amount of training data. Among the models tested, ResNet50 exhibited the best performance over other CNN and transformer-based models making it the optimal choice for this task.

Title: Hierarchical Divide-and-Conquer for Fine-Grained Alignment in LLM-Based Medical Evaluation

• Authors: Shunfan Zheng, Xiechi Zhang, Gerard de Melo, Xiaoling Wang, Linlin Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06741
• Pdf URL: https://arxiv.org/pdf/2501.06741
• Copy Paste: [[2501.06741]] Hierarchical Divide-and-Conquer for Fine-Grained Alignment in LLM-Based Medical Evaluation(https://arxiv.org/abs/2501.06741)
• Keywords: large language model
• Abstract: In the rapidly evolving landscape of large language models (LLMs) for medical applications, ensuring the reliability and accuracy of these models in clinical settings is paramount. Existing benchmarks often focus on fixed-format tasks like multiple-choice QA, which fail to capture the complexity of real-world clinical diagnostics. Moreover, traditional evaluation metrics and LLM-based evaluators struggle with misalignment, often providing oversimplified assessments that do not adequately reflect human judgment. To address these challenges, we introduce HDCEval, a Hierarchical Divide-and-Conquer Evaluation framework tailored for fine-grained alignment in medical evaluation. HDCEval is built on a set of fine-grained medical evaluation guidelines developed in collaboration with professional doctors, encompassing Patient Question Relevance, Medical Knowledge Correctness, and Expression. The framework decomposes complex evaluation tasks into specialized subtasks, each evaluated by expert models trained through Attribute-Driven Token Optimization (ADTO) on a meticulously curated preference dataset. This hierarchical approach ensures that each aspect of the evaluation is handled with expert precision, leading to a significant improvement in alignment with human evaluators.

Title: Static Segmentation by Tracking: A Frustratingly Label-Efficient Approach to Fine-Grained Segmentation

• Authors: Zhenyang Feng, Zihe Wang, Saul Ibaven Bueno, Tomasz Frelek, Advikaa Ramesh, Jingyan Bai, Lemeng Wang, Zanming Huang, Jianyang Gu, Jinsu Yoo, Tai-Yu Pan, Arpita Chowdhury, Michelle Ramirez, Elizabeth G. Campolongo, Matthew J. Thompson, Christopher G. Lawrence, Sydne Record, Neil Rosser, Anuj Karpatne, Daniel Rubenstein, Hilmar Lapp, Charles V. Stewart, Tanya Berger-Wolf, Yu Su, Wei-Lun Chao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06749
• Pdf URL: https://arxiv.org/pdf/2501.06749
• Copy Paste: [[2501.06749]] Static Segmentation by Tracking: A Frustratingly Label-Efficient Approach to Fine-Grained Segmentation(https://arxiv.org/abs/2501.06749)
• Keywords: segmentation
• Abstract: We study image segmentation in the biological domain, particularly trait and part segmentation from specimen images (e.g., butterfly wing stripes or beetle body parts). This is a crucial, fine-grained task that aids in understanding the biology of organisms. The conventional approach involves hand-labeling masks, often for hundreds of images per species, and training a segmentation model to generalize these labels to other images, which can be exceedingly laborious. We present a label-efficient method named Static Segmentation by Tracking (SST). SST is built upon the insight: while specimens of the same species have inherent variations, the traits and parts we aim to segment show up consistently. This motivates us to concatenate specimen images into a ``pseudo-video'' and reframe trait and part segmentation as a tracking problem. Concretely, SST generates masks for unlabeled images by propagating annotated or predicted masks from the ``pseudo-preceding'' images. Powered by Segment Anything Model 2 (SAM~2) initially developed for video segmentation, we show that SST can achieve high-quality trait and part segmentation with merely one labeled image per species -- a breakthrough for analyzing specimen images. We further develop a cycle-consistent loss to fine-tune the model, again using one labeled image. Additionally, we highlight the broader potential of SST, including one-shot instance segmentation on images taken in the wild and trait-based image retrieval.

Title: Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models

• Authors: Michael Toker, Ido Galil, Hadas Orgad, Rinon Gal, Yoad Tewel, Gal Chechik, Yonatan Belinkov
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06751
• Pdf URL: https://arxiv.org/pdf/2501.06751
• Copy Paste: [[2501.06751]] Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models(https://arxiv.org/abs/2501.06751)
• Keywords: diffusion
• Abstract: Text-to-image (T2I) diffusion models rely on encoded prompts to guide the image generation process. Typically, these prompts are extended to a fixed length by adding padding tokens before text encoding. Despite being a default practice, the influence of padding tokens on the image generation process has not been investigated. In this work, we conduct the first in-depth analysis of the role padding tokens play in T2I models. We develop two causal techniques to analyze how information is encoded in the representation of tokens across different components of the T2I pipeline. Using these techniques, we investigate when and how padding tokens impact the image generation process. Our findings reveal three distinct scenarios: padding tokens may affect the model's output during text encoding, during the diffusion process, or be effectively ignored. Moreover, we identify key relationships between these scenarios and the model's architecture (cross or self-attention) and its training process (frozen or trained text encoder). These insights contribute to a deeper understanding of the mechanisms of padding tokens, potentially informing future model design and training practices in T2I systems.

Title: Procedural Fairness and Its Relationship with Distributive Fairness in Machine Learning

• Authors: Ziming Wang, Changwu Huang, Ke Tang, Xin Yao
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.06753
• Pdf URL: https://arxiv.org/pdf/2501.06753
• Copy Paste: [[2501.06753]] Procedural Fairness and Its Relationship with Distributive Fairness in Machine Learning(https://arxiv.org/abs/2501.06753)
• Keywords: fair
• Abstract: Fairness in machine learning (ML) has garnered significant attention in recent years. While existing research has predominantly focused on the distributive fairness of ML models, there has been limited exploration of procedural fairness. This paper proposes a novel method to achieve procedural fairness during the model training phase. The effectiveness of the proposed method is validated through experiments conducted on one synthetic and six real-world datasets. Additionally, this work studies the relationship between procedural fairness and distributive fairness in ML models. On one hand, the impact of dataset bias and the procedural fairness of ML model on its distributive fairness is examined. The results highlight a significant influence of both dataset bias and procedural fairness on distributive fairness. On the other hand, the distinctions between optimizing procedural and distributive fairness metrics are analyzed. Experimental results demonstrate that optimizing procedural fairness metrics mitigates biases introduced or amplified by the decision-making process, thereby ensuring fairness in the decision-making process itself, as well as improving distributive fairness. In contrast, optimizing distributive fairness metrics encourages the ML model's decision-making process to favor disadvantaged groups, counterbalancing the inherent preferences for advantaged groups present in the dataset and ultimately achieving distributive fairness.

Title: VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning

• Authors: Ji Soo Lee, Jongha Kim, Jeehye Na, Jinyoung Park, Hyunwoo J. Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06761
• Pdf URL: https://arxiv.org/pdf/2501.06761
• Copy Paste: [[2501.06761]] VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning(https://arxiv.org/abs/2501.06761)
• Keywords: large language model, segmentation
• Abstract: Despite the advancements of Video Large Language Models (VideoLLMs) in various tasks, they struggle with fine-grained temporal understanding, such as Dense Video Captioning (DVC). DVC is a complicated task of describing all events within a video while also temporally localizing them, which integrates multiple fine-grained tasks, including video segmentation, video captioning, and temporal video grounding. Previous VideoLLMs attempt to solve DVC in a single step, failing to utilize their reasoning capability. Moreover, previous training objectives for VideoLLMs do not fully reflect the evaluation metrics, therefore not providing supervision directly aligned to target tasks. To address such a problem, we propose a novel framework named VidChain comprised of Chain-of-Tasks (CoTasks) and Metric-based Direct Preference Optimization (M-DPO). CoTasks decompose a complex task into a sequence of sub-tasks, allowing VideoLLMs to leverage their reasoning capabilities more effectively. M-DPO aligns a VideoLLM with evaluation metrics, providing fine-grained supervision to each task that is well-aligned with metrics. Applied to two different VideoLLMs, VidChain consistently improves their fine-grained video understanding, thereby outperforming previous VideoLLMs on two different DVC benchmarks and also on the temporal video grounding task. Code is available at \url{this https URL}.

Title: ODPG: Outfitting Diffusion with Pose Guided Condition

• Authors: Seohyun Lee, Jintae Park, Sanghyeok Park
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06769
• Pdf URL: https://arxiv.org/pdf/2501.06769
• Copy Paste: [[2501.06769]] ODPG: Outfitting Diffusion with Pose Guided Condition(https://arxiv.org/abs/2501.06769)
• Keywords: diffusion, generative
• Abstract: Virtual Try-On (VTON) technology allows users to visualize how clothes would look on them without physically trying them on, gaining traction with the rise of digitalization and online shopping. Traditional VTON methods, often using Generative Adversarial Networks (GANs) and Diffusion models, face challenges in achieving high realism and handling dynamic poses. This paper introduces Outfitting Diffusion with Pose Guided Condition (ODPG), a novel approach that leverages a latent diffusion model with multiple conditioning inputs during the denoising process. By transforming garment, pose, and appearance images into latent features and integrating these features in a UNet-based denoising model, ODPG achieves non-explicit synthesis of garments on dynamically posed human images. Our experiments on the FashionTryOn and a subset of the DeepFashion dataset demonstrate that ODPG generates realistic VTON images with fine-grained texture details across various poses, utilizing an end-to-end architecture without the need for explicit garment warping processes. Future work will focus on generating VTON outputs in video format and on applying our attention mechanism, as detailed in the Method section, to other domains with limited data.

Title: Temporal-Aware Spiking Transformer Hashing Based on 3D-DWT

• Authors: Zihao Mei, Jianhao Li, Bolin Zhang, Chong Wang, Lijun Guo, Guoqi Li, Jiangbo Qian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06786
• Pdf URL: https://arxiv.org/pdf/2501.06786
• Copy Paste: [[2501.06786]] Temporal-Aware Spiking Transformer Hashing Based on 3D-DWT(https://arxiv.org/abs/2501.06786)
• Keywords: transformer
• Abstract: With the rapid growth of dynamic vision sensor (DVS) data, constructing a low-energy, efficient data retrieval system has become an urgent task. Hash learning is one of the most important retrieval technologies which can keep the distance between hash codes consistent with the distance between DVS data. As spiking neural networks (SNNs) can encode information through spikes, they demonstrate great potential in promoting energy efficiency. Based on the binary characteristics of SNNs, we first propose a novel supervised hashing method named Spikinghash with a hierarchical lightweight structure. Spiking WaveMixer (SWM) is deployed in shallow layers, utilizing a multilevel 3D discrete wavelet transform (3D-DWT) to decouple spatiotemporal features into various low-frequency and high frequency components, and then employing efficient spectral feature fusion. SWM can effectively capture the temporal dependencies and local spatial features. Spiking Self-Attention (SSA) is deployed in deeper layers to further extract global spatiotemporal information. We also design a hash layer utilizing binary characteristic of SNNs, which integrates information over multiple time steps to generate final hash codes. Furthermore, we propose a new dynamic soft similarity loss for SNNs, which utilizes membrane potentials to construct a learnable similarity matrix as soft labels to fully capture the similarity differences between classes and compensate information loss in SNNs, thereby improving retrieval performance. Experiments on multiple datasets demonstrate that Spikinghash can achieve state-of-the-art results with low energy consumption and fewer parameters.

Title: Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences

• Authors: Liu Yu, Ludie Guo, Ping Kuang, Fan Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06795
• Pdf URL: https://arxiv.org/pdf/2501.06795
• Copy Paste: [[2501.06795]] Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences(https://arxiv.org/abs/2501.06795)
• Keywords: fair, large language model
• Abstract: Pre-trained language models (PLMs) are trained on data that inherently contains gender biases, leading to undesirable impacts. Traditional debiasing methods often rely on external corpora, which may lack quality, diversity, or demographic balance, affecting the effectiveness of debiasing. With the rise of large language models and their extensive knowledge, we propose enhancing fairness (Fair-Gender) in PLMs by absorbing coherent, attribute-balanced, and semantically rich sentences. However, these sentences cannot be directly used for debiasing due to alignment issues and the risk of negative transfer. We address this by applying causal analysis to estimate causal effects, filtering out unaligned sentences, and identifying aligned ones for incorporation into PLMs, thereby ensuring positive transfer. Experiments show that our approach significantly reduces gender biases in PLMs while preserving their language expressiveness.

Title: OFDM-based JCAS under Attack: The Dual Threat of Spoofing and Jamming in WLAN Sensing

• Authors: Hasan Can Yildirim, Musa Furkan Keskin, Henk Wymeersch, Francois Horlin
• Subjects: cs.CR, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.06798
• Pdf URL: https://arxiv.org/pdf/2501.06798
• Copy Paste: [[2501.06798]] OFDM-based JCAS under Attack: The Dual Threat of Spoofing and Jamming in WLAN Sensing(https://arxiv.org/abs/2501.06798)
• Keywords: security, attack
• Abstract: This study reveals the vulnerabilities of Wireless Local Area Networks (WLAN) sensing, under the scope of joint communication and sensing (JCAS), focusing on target spoofing and deceptive jamming techniques. We use orthogonal frequency-division multiplexing (OFDM) to explore how adversaries can exploit WLAN's sensing capabilities to inject false targets and disrupt normal operations. Unlike traditional methods that require sophisticated digital radio-frequency memory hardware, we demonstrate that much simpler software-defined radios can effectively serve as deceptive jammers in WLAN settings. Through comprehensive modeling and practical experiments, we show how deceptive jammers can manipulate the range-Doppler map (RDM) by altering signal integrity, thereby posing significant security threats to OFDM-based JCAS systems. Our findings comprehensively evaluate jammer impact on RDMs and propose several jamming strategies that vary in complexity and detectability.

Title: A Pan-cancer Classification Model using Multi-view Feature Selection Method and Ensemble Classifier

• Authors: Tareque Mohmud Chowdhury, Farzana Tabassum, Sabrina Islam, Abu Raihan Mostofa Kamal
• Subjects: cs.LG, q-bio.GN
• Abstract URL: https://arxiv.org/abs/2501.06805
• Pdf URL: https://arxiv.org/pdf/2501.06805
• Copy Paste: [[2501.06805]] A Pan-cancer Classification Model using Multi-view Feature Selection Method and Ensemble Classifier(https://arxiv.org/abs/2501.06805)
• Keywords: robust
• Abstract: Accurately identifying cancer samples is crucial for precise diagnosis and effective patient treatment. Traditional methods falter with high-dimensional and high feature-to-sample count ratios, which are critical for classifying cancer samples. This study aims to develop a novel feature selection framework specifically for transcriptome data and propose two ensemble classifiers. For feature selection, we partition the transcriptome dataset vertically based on feature types. Then apply the Boruta feature selection process on each of the partitions, combine the results, and apply Boruta again on the combined result. We repeat the process with different parameters of Boruta and prepare the final feature set. Finally, we constructed two ensemble ML models based on LR, SVM and XGBoost classifiers with max voting and averaging probability approach. We used 10-fold cross-validation to ensure robust and reliable classification performance. With 97.11\% accuracy and 0.9996 AUC value, our approach performs better compared to existing state-of-the-art methods to classify 33 types of cancers. A set of 12 types of cancer is traditionally challenging to differentiate between each other due to their similarity in tissue of origin. Our method accurately identifies over 90\% of samples from these 12 types of cancers, which outperforms all known methods presented in existing literature. The gene set enrichment analysis reveals that our framework's selected features have enriched the pathways highly related to cancers. This study develops a feature selection framework to select features highly related to cancer development and leads to identifying different types of cancer samples with higher accuracy.

Title: MPCache: MPC-Friendly KV Cache Eviction for Efficient Private Large Language Model Inference

• Authors: Wenxuan Zeng, Ye Dong, Jinjin Zhou, Junming Ma, Jin Tan, Runsheng Wang, Meng Li
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06807
• Pdf URL: https://arxiv.org/pdf/2501.06807
• Copy Paste: [[2501.06807]] MPCache: MPC-Friendly KV Cache Eviction for Efficient Private Large Language Model Inference(https://arxiv.org/abs/2501.06807)
• Keywords: secure, protect, large language model
• Abstract: Private large language model (LLM) inference based on secure multi-party computation (MPC) offers cryptographically-secure protection for both user prompt and proprietary model weights. However, it suffers from large latency overhead especially for long input sequences. While key-value (KV) cache eviction algorithms have been proposed to reduce the computation and memory cost for plaintext inference, they are not designed for MPC and cannot benefit private inference easily. In this paper, we propose an accurate and MPC-friendly KV cache eviction framework, dubbed MPCache. MPCache is built on the observation that historical tokens in a long sequence may have different effects on the downstream decoding. Hence, MPCache combines a look-once static eviction algorithm to discard unimportant tokens and a query-aware dynamic selection algorithm to further select a small subset of tokens for attention computation. As existing dynamic selection algorithms incur too much latency, we propose a series of optimizations to drastically reduce the KV cache selection overhead, including MPC-friendly similarity approximation, hierarchical KV cache clustering, and cross-layer index sharing strategy. With extensive experiments, we demonstrate that MPCache consistently outperforms prior-art KV cache eviction baselines across different LLM generation tasks and achieves 1.8~2.01x and 3.39~8.37x decoding latency and communication reduction on different sequence lengths, respectively.

Title: Semantic-CD: Remote Sensing Image Semantic Change Detection towards Open-vocabulary Setting

• Authors: Yongshuo Zhu, Lu Li, Keyan Chen, Chenyang Liu, Fugen Zhou, Zhenwei Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06808
• Pdf URL: https://arxiv.org/pdf/2501.06808
• Copy Paste: [[2501.06808]] Semantic-CD: Remote Sensing Image Semantic Change Detection towards Open-vocabulary Setting(https://arxiv.org/abs/2501.06808)
• Keywords: segmentation
• Abstract: Remote sensing image semantic change detection is a method used to analyze remote sensing images, aiming to identify areas of change as well as categorize these changes within images of the same location taken at different times. Traditional change detection methods often face challenges in generalizing across semantic categories in practical scenarios. To address this issue, we introduce a novel approach called Semantic-CD, specifically designed for semantic change detection in remote sensing images. This method incorporates the open vocabulary semantics from the vision-language foundation model, CLIP. By utilizing CLIP's extensive vocabulary knowledge, our model enhances its ability to generalize across categories and improves segmentation through fully decoupled multi-task learning, which includes both binary change detection and semantic change detection tasks. Semantic-CD consists of four main components: a bi-temporal CLIP visual encoder for extracting features from bi-temporal images, an open semantic prompter for creating semantic cost volume maps with open vocabulary, a binary change detection decoder for generating binary change detection masks, and a semantic change detection decoder for producing semantic labels. Experimental results on the SECOND dataset demonstrate that Semantic-CD achieves more accurate masks and reduces semantic classification errors, illustrating its effectiveness in applying semantic priors from vision-language foundation models to SCD tasks.

Title: RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models

• Authors: Keyan Chen, Jiafan Zhang, Chenyang Liu, Zhengxia Zou, Zhenwei Shi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06809
• Pdf URL: https://arxiv.org/pdf/2501.06809
• Copy Paste: [[2501.06809]] RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models(https://arxiv.org/abs/2501.06809)
• Keywords: robust, extraction, segmentation
• Abstract: Referring remote sensing image segmentation is crucial for achieving fine-grained visual understanding through free-format textual input, enabling enhanced scene and object extraction in remote sensing applications. Current research primarily utilizes pre-trained language models to encode textual descriptions and align them with visual modalities, thereby facilitating the expression of relevant visual features. However, these approaches often struggle to establish robust alignments between fine-grained semantic concepts, leading to inconsistent representations across textual and visual information. To address these limitations, we introduce a referring remote sensing image segmentation foundational model, RSRefSeg. RSRefSeg leverages CLIP for visual and textual encoding, employing both global and local textual semantics as filters to generate referring-related visual activation features in the latent space. These activated features then serve as input prompts for SAM, which refines the segmentation masks through its robust visual generalization capabilities. Experimental results on the RRSIS-D dataset demonstrate that RSRefSeg outperforms existing methods, underscoring the effectiveness of foundational models in enhancing multimodal task comprehension. The code is available at \url{this https URL}.

Title: Event Argument Extraction with Enriched Prompts

• Authors: Chen Liang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06825
• Pdf URL: https://arxiv.org/pdf/2501.06825
• Copy Paste: [[2501.06825]] Event Argument Extraction with Enriched Prompts(https://arxiv.org/abs/2501.06825)
• Keywords: extraction, large language model
• Abstract: This work aims to delve deeper into prompt-based event argument extraction (EAE) models. We explore the impact of incorporating various types of information into the prompt on model performance, including trigger, other role arguments for the same event, and role arguments across multiple events within the same document. Further, we provide the best possible performance that the prompt-based EAE model can attain and demonstrate such models can be further optimized from the perspective of the training objective. Experiments are carried out on three small language models and two large language models in RAMS.

Title: GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing

• Authors: Ruizhe Ou, Yuan Hu, Fan Zhang, Jiaxin Chen, Yu Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06828
• Pdf URL: https://arxiv.org/pdf/2501.06828
• Copy Paste: [[2501.06828]] GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing(https://arxiv.org/abs/2501.06828)
• Keywords: large language model, segmentation
• Abstract: Multi-modal large language models (MLLMs) have achieved remarkable success in image- and region-level remote sensing (RS) image understanding tasks, such as image captioning, visual question answering, and visual grounding. However, existing RS MLLMs lack the pixel-level dialogue capability, which involves responding to user instructions with segmentation masks for specific instances. In this paper, we propose GeoPix, a RS MLLM that extends image understanding capabilities to the pixel level. This is achieved by equipping the MLLM with a mask predictor, which transforms visual features from the vision encoder into masks conditioned on the LLM's segmentation token embeddings. To facilitate the segmentation of multi-scale objects in RS imagery, a class-wise learnable memory module is integrated into the mask predictor to capture and store class-wise geo-context at the instance level across the entire dataset. In addition, to address the absence of large-scale datasets for training pixel-level RS MLLMs, we construct the GeoPixInstruct dataset, comprising 65,463 images and 140,412 instances, with each instance annotated with text descriptions, bounding boxes, and masks. Furthermore, we develop a two-stage training strategy to balance the distinct requirements of text generation and masks prediction in multi-modal multi-task optimization. Extensive experiments verify the effectiveness and superiority of GeoPix in pixel-level segmentation tasks, while also maintaining competitive performance in image- and region-level benchmarks.

Title: Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers

• Authors: Syed Ali Tariq, Tehseen Zia, Mubeen Ghafoor
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06831
• Pdf URL: https://arxiv.org/pdf/2501.06831
• Copy Paste: [[2501.06831]] Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers(https://arxiv.org/abs/2501.06831)
• Keywords: explainability
• Abstract: Explainability of deep convolutional neural networks (DCNNs) is an important research topic that tries to uncover the reasons behind a DCNN model's decisions and improve their understanding and reliability in high-risk environments. In this regard, we propose a novel method for generating interpretable counterfactual and contrastive explanations for DCNN models. The proposed method is model intrusive that probes the internal workings of a DCNN instead of altering the input image to generate explanations. Given an input image, we provide contrastive explanations by identifying the most important filters in the DCNN representing features and concepts that separate the model's decision between classifying the image to the original inferred class or some other specified alter class. On the other hand, we provide counterfactual explanations by specifying the minimal changes necessary in such filters so that a contrastive output is obtained. Using these identified filters and concepts, our method can provide contrastive and counterfactual reasons behind a model's decisions and makes the model more transparent. One of the interesting applications of this method is misclassification analysis, where we compare the identified concepts from a particular input image and compare them with class-specific concepts to establish the validity of the model's decisions. The proposed method is compared with state-of-the-art and evaluated on the Caltech-UCSD Birds (CUB) 2011 dataset to show the usefulness of the explanations provided.

Title: X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding

• Authors: Wenqi Zhou, Kai Cao, Hao Zheng, Xinyi Zheng, Miao Liu, Per Ola Kristensson, Walterio Mayol-Cuevas, Fan Zhang, Weizhe Lin, Junxiao Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06835
• Pdf URL: https://arxiv.org/pdf/2501.06835
• Copy Paste: [[2501.06835]] X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding(https://arxiv.org/abs/2501.06835)
• Keywords: large language model
• Abstract: Long-form egocentric video understanding provides rich contextual information and unique insights into long-term human behaviors, holding significant potential for applications in embodied intelligence, long-term activity analysis, and personalized assistive technologies. However, existing benchmark datasets primarily focus on single, short-duration videos or moderately long videos up to dozens of minutes, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. To address this, we introduce X-LeBench, a novel benchmark dataset specifically crafted for evaluating tasks on extremely long egocentric video recordings. Leveraging the advanced text processing capabilities of large language models (LLMs), X-LeBench develops a life-logging simulation pipeline that produces realistic, coherent daily plans aligned with real-world video data. This approach enables the flexible integration of synthetic daily plans with real-world footage from Ego4D-a massive-scale egocentric video dataset covers a wide range of daily life scenarios-resulting in 432 simulated video life logs that mirror realistic daily activities in contextually rich scenarios. The video life-log durations span from 23 minutes to 16.4 hours. The evaluation of several baseline systems and multimodal large language models (MLLMs) reveals their poor performance across the board, highlighting the inherent challenges of long-form egocentric video understanding and underscoring the need for more advanced models.

Title: SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation

• Authors: Javier Gamazo Tejero, Moritz Schmid, Pablo Márquez Neila, Martin S. Zinkernagel, Sebastian Wolf, Raphael Sznitman
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06836
• Pdf URL: https://arxiv.org/pdf/2501.06836
• Copy Paste: [[2501.06836]] SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation(https://arxiv.org/abs/2501.06836)
• Keywords: segmentation
• Abstract: This paper addresses the domain adaptation challenge for semantic segmentation in medical imaging. Despite the impressive performance of recent foundational segmentation models like SAM on natural images, they struggle with medical domain images. Beyond this, recent approaches that perform end-to-end fine-tuning of models are simply not computationally tractable. To address this, we propose a novel SAM adapter approach that minimizes the number of trainable parameters while achieving comparable performances to full fine-tuning. The proposed SAM adapter is strategically placed in the mask decoder, offering excellent and broad generalization capabilities and improved segmentation across both fully supervised and test-time domain adaptation tasks. Extensive validation on four datasets showcases the adapter's efficacy, outperforming existing methods while training less than 1% of SAM's total parameters.

Title: Faithful Counterfactual Visual Explanations (FCVE)

• Authors: Bismillah Khan, Syed Ali Tariq, Tehseen Zia, Muhammad Ahsan, David Windridge
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06841
• Pdf URL: https://arxiv.org/pdf/2501.06841
• Copy Paste: [[2501.06841]] Faithful Counterfactual Visual Explanations (FCVE)(https://arxiv.org/abs/2501.06841)
• Keywords: interpretability
• Abstract: Deep learning models in computer vision have made remarkable progress, but their lack of transparency and interpretability remains a challenge. The development of explainable AI can enhance the understanding and performance of these models. However, existing techniques often struggle to provide convincing explanations that non-experts easily understand, and they cannot accurately identify models' intrinsic decision-making processes. To address these challenges, we propose to develop a counterfactual explanation (CE) model that balances plausibility and faithfulness. This model generates easy-to-understand visual explanations by making minimum changes necessary in images without altering the pixel data. Instead, the proposed method identifies internal concepts and filters learned by models and leverages them to produce plausible counterfactual explanations. The provided explanations reflect the internal decision-making process of the model, thus ensuring faithfulness to the model.

Title: SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training

• Authors: Tianjin Huang, Ziquan Zhu, Gaojie Jin, Lu Liu, Zhangyang Wang, Shiwei Liu
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06842
• Pdf URL: https://arxiv.org/pdf/2501.06842
• Copy Paste: [[2501.06842]] SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training(https://arxiv.org/abs/2501.06842)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks, yet their training remains highly resource-intensive and susceptible to critical challenges such as training instability. A predominant source of this instability stems from gradient and loss spikes, which disrupt the learning process, often leading to costly interventions like checkpoint recovery and experiment restarts, further amplifying inefficiencies. This paper presents a comprehensive investigation into gradient spikes observed during LLM training, revealing their prevalence across multiple architectures and datasets. Our analysis shows that these spikes can be up to $1000\times$ larger than typical gradients, substantially deteriorating model performance. To address this issue, we propose Spike-Aware Adam with Momentum Reset SPAM, a novel optimizer designed to counteract gradient spikes through momentum reset and spike-aware gradient clipping. Extensive experiments, including both pre-training and fine-tuning, demonstrate that SPAM consistently surpasses Adam and its variants across various tasks, including (1) LLM pre-training from 60M to 1B, (2) 4-bit LLM pre-training,(3) reinforcement learning, and (4) Time Series Forecasting. Additionally, SPAM facilitates memory-efficient training by enabling sparse momentum, where only a subset of momentum terms are maintained and updated. When operating under memory constraints, SPAM outperforms state-of-the-art memory-efficient optimizers such as GaLore and Adam-Mini. Our work underscores the importance of mitigating gradient spikes in LLM training and introduces an effective optimization strategy that enhances both training stability and resource efficiency at scale. Code is available at this https URL

Title: A General Framework for Inference-time Scaling and Steering of Diffusion Models

• Authors: Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, Rajesh Ranganath
• Subjects: cs.LG, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06848
• Pdf URL: https://arxiv.org/pdf/2501.06848
• Copy Paste: [[2501.06848]] A General Framework for Inference-time Scaling and Steering of Diffusion Models(https://arxiv.org/abs/2501.06848)
• Keywords: diffusion
• Abstract: Diffusion models produce impressive results in modalities ranging from images and video to protein design and text. However, generating samples with user-specified properties remains a challenge. Recent research proposes fine-tuning models to maximize rewards that capture desired properties, but these methods require expensive training and are prone to mode collapse. In this work, we propose Feynman Kac (FK) steering, an inference-time framework for steering diffusion models with reward functions. FK steering works by sampling a system of multiple interacting diffusion processes, called particles, and resampling particles at intermediate steps based on scores computed using functions called potentials. Potentials are defined using rewards for intermediate states and are selected such that a high value indicates that the particle will yield a high-reward sample. We explore various choices of potentials, intermediate rewards, and samplers. We evaluate FK steering on text-to-image and text diffusion models. For steering text-to-image models with a human preference reward, we find that FK steering a 0.8B parameter model outperforms a 2.6B parameter fine-tuned model on prompt fidelity, with faster sampling and no training. For steering text diffusion models with rewards for text quality and specific text attributes, we find that FK steering generates lower perplexity, more linguistically acceptable outputs and enables gradient-free control of attributes like toxicity. Our results demonstrate that inference-time scaling and steering of diffusion models, even with off-the-shelf rewards, can provide significant sample quality gains and controllability benefits. Code is available at this https URL .

Title: A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic Context

• Authors: Noureldin Zahran, Aya E. Fouda, Radwa J. Hanafy, Mohammed E. Fouda
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06859
• Pdf URL: https://arxiv.org/pdf/2501.06859
• Copy Paste: [[2501.06859]] A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic Context(https://arxiv.org/abs/2501.06859)
• Keywords: large language model
• Abstract: Mental health disorders pose a growing public health concern in the Arab world, emphasizing the need for accessible diagnostic and intervention tools. Large language models (LLMs) offer a promising approach, but their application in Arabic contexts faces challenges including limited labeled datasets, linguistic complexity, and translation biases. This study comprehensively evaluates 8 LLMs, including general multi-lingual models, as well as bi-lingual ones, on diverse mental health datasets (such as AraDepSu, Dreaddit, MedMCQA), investigating the impact of prompt design, language configuration (native Arabic vs. translated English, and vice versa), and few-shot prompting on diagnostic performance. We find that prompt engineering significantly influences LLM scores mainly due to reduced instruction following, with our structured prompt outperforming a less structured variant on multi-class datasets, with an average difference of 14.5\%. While language influence on performance was modest, model selection proved crucial: Phi-3.5 MoE excelled in balanced accuracy, particularly for binary classification, while Mistral NeMo showed superior performance in mean absolute error for severity prediction tasks. Few-shot prompting consistently improved performance, with particularly substantial gains observed for GPT-4o Mini on multi-class classification, boosting accuracy by an average factor of 1.58. These findings underscore the importance of prompt optimization, multilingual analysis, and few-shot learning for developing culturally sensitive and effective LLM-based mental health tools for Arabic-speaking populations.

Title: LarvSeg: Exploring Image Classification Data For Large Vocabulary Semantic Segmentation via Category-wise Attentive Classifier

• Authors: Haojun Yu, Di Dai, Ziwei Zhao, Di He, Han Hu, Liwei Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06862
• Pdf URL: https://arxiv.org/pdf/2501.06862
• Copy Paste: [[2501.06862]] LarvSeg: Exploring Image Classification Data For Large Vocabulary Semantic Segmentation via Category-wise Attentive Classifier(https://arxiv.org/abs/2501.06862)
• Keywords: segmentation
• Abstract: Scaling up the vocabulary of semantic segmentation models is extremely challenging because annotating large-scale mask labels is labour-intensive and time-consuming. Recently, language-guided segmentation models have been proposed to address this challenge. However, their performance drops significantly when applied to out-of-distribution categories. In this paper, we propose a new large vocabulary semantic segmentation framework, called LarvSeg. Different from previous works, LarvSeg leverages image classification data to scale the vocabulary of semantic segmentation models as large-vocabulary classification datasets usually contain balanced categories and are much easier to obtain. However, for classification tasks, the category is image-level, while for segmentation we need to predict the label at pixel level. To address this issue, we first propose a general baseline framework to incorporate image-level supervision into the training process of a pixel-level segmentation model, making the trained network perform semantic segmentation on newly introduced categories in the classification data. We then observe that a model trained on segmentation data can group pixel features of categories beyond the training vocabulary. Inspired by this finding, we design a category-wise attentive classifier to apply supervision to the precise regions of corresponding categories to improve the model performance. Extensive experiments demonstrate that LarvSeg significantly improves the large vocabulary semantic segmentation performance, especially in the categories without mask labels. For the first time, we provide a 21K-category semantic segmentation model with the help of ImageNet21K. The code is available at this https URL.

Title: Transfer Learning of Tabular Data by Finetuning Large Language Models

• Authors: Shourav B. Rabbani, Ibna Kowsar, Manar D. Samad
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06863
• Pdf URL: https://arxiv.org/pdf/2501.06863
• Copy Paste: [[2501.06863]] Transfer Learning of Tabular Data by Finetuning Large Language Models(https://arxiv.org/abs/2501.06863)
• Keywords: generative, large language model
• Abstract: Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new era of generative AI, powered by large language models (LLM), brings unprecedented learning opportunities to diverse data and domains. This paper investigates the effectiveness of an LLM application programming interface (API) and transfer learning of LLM in tabular data classification. LLM APIs respond to input text prompts with tokenized data and instructions, whereas transfer learning finetunes an LLM for a target classification task. This paper proposes an end-to-end finetuning of LLM to demonstrate cross-data transfer learning on ten benchmark data sets when large pre-trained tabular data models do not exist to facilitate transfer learning. The proposed LLM finetuning method outperforms state-of-the-art machine and deep learning methods on tabular data with less than ten features - a standard feature size for tabular data sets. The transfer learning approach uses a fraction of the computational cost of other deep learning or API-based solutions while ensuring competitive or superior classification performance.

Title: Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach

• Authors: Mathieu Cocheteux, Julien Moreau, Franck Davoine
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06878
• Pdf URL: https://arxiv.org/pdf/2501.06878
• Copy Paste: [[2501.06878]] Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach(https://arxiv.org/abs/2501.06878)
• Keywords: robust
• Abstract: Accurate sensor calibration is crucial for autonomous systems, yet its uncertainty quantification remains underexplored. We present the first approach to integrate uncertainty awareness into online extrinsic calibration, combining Monte Carlo Dropout with Conformal Prediction to generate prediction intervals with a guaranteed level of coverage. Our method proposes a framework to enhance existing calibration models with uncertainty quantification, compatible with various network architectures. Validated on KITTI (RGB Camera-LiDAR) and DSEC (Event Camera-LiDAR) datasets, we demonstrate effectiveness across different visual sensor types, measuring performance with adapted metrics to evaluate the efficiency and reliability of the intervals. By providing calibration parameters with quantifiable confidence measures, we offer insights into the reliability of calibration estimates, which can greatly improve the robustness of sensor fusion in dynamic environments and usefully serve the Computer Vision community.

Title: Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning

• Authors: Hanwen Zhong, Jiaxin Chen, Yutong Zhang, Di Huang, Yunhong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06884
• Pdf URL: https://arxiv.org/pdf/2501.06884
• Copy Paste: [[2501.06884]] Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning(https://arxiv.org/abs/2501.06884)
• Keywords: transformer
• Abstract: Multi-Task Learning (MTL) for Vision Transformer aims at enhancing the model capability by tackling multiple tasks simultaneously. Most recent works have predominantly focused on designing Mixture-of-Experts (MoE) structures and in tegrating Low-Rank Adaptation (LoRA) to efficiently perform multi-task learning. However, their rigid combination hampers both the optimization of MoE and the ef fectiveness of reparameterization of LoRA, leading to sub-optimal performance and low inference speed. In this work, we propose a novel approach dubbed Efficient Multi-Task Learning (EMTAL) by transforming a pre-trained Vision Transformer into an efficient multi-task learner during training, and reparameterizing the learned structure for efficient inference. Specifically, we firstly develop the MoEfied LoRA structure, which decomposes the pre-trained Transformer into a low-rank MoE structure and employ LoRA to fine-tune the parameters. Subsequently, we take into account the intrinsic asynchronous nature of multi-task learning and devise a learning Quality Retaining (QR) optimization mechanism, by leveraging the historical high-quality class logits to prevent a well-trained task from performance degradation. Finally, we design a router fading strategy to integrate the learned parameters into the original Transformer, archiving efficient inference. Extensive experiments on public benchmarks demonstrate the superiority of our method, compared to the state-of-the-art multi-task learning approaches.

Title: MedGrad E-CLIP: Enhancing Trust and Transparency in AI-Driven Skin Lesion Diagnosis

• Authors: Sadia Kamal, Tim Oates
• Subjects: cs.CV, cs.AI, cs.ET, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06887
• Pdf URL: https://arxiv.org/pdf/2501.06887
• Copy Paste: [[2501.06887]] MedGrad E-CLIP: Enhancing Trust and Transparency in AI-Driven Skin Lesion Diagnosis(https://arxiv.org/abs/2501.06887)
• Keywords: robust, explainability
• Abstract: As deep learning models gain attraction in medical data, ensuring transparent and trustworthy decision-making is essential. In skin cancer diagnosis, while advancements in lesion detection and classification have improved accuracy, the black-box nature of these methods poses challenges in understanding their decision processes, leading to trust issues among physicians. This study leverages the CLIP (Contrastive Language-Image Pretraining) model, trained on different skin lesion datasets, to capture meaningful relationships between visual features and diagnostic criteria terms. To further enhance transparency, we propose a method called MedGrad E-CLIP, which builds on gradient-based E-CLIP by incorporating a weighted entropy mechanism designed for complex medical imaging like skin lesions. This approach highlights critical image regions linked to specific diagnostic descriptions. The developed integrated pipeline not only classifies skin lesions by matching corresponding descriptions but also adds an essential layer of explainability developed especially for medical data. By visually explaining how different features in an image relates to diagnostic criteria, this approach demonstrates the potential of advanced vision-language models in medical image analysis, ultimately improving transparency, robustness, and trust in AI-driven diagnostic systems.

Title: Synthetic Prior for Few-Shot Drivable Head Avatar Inversion

• Authors: Wojciech Zielonka, Stephan J. Garbin, Alexandros Lattas, George Kopanas, Paulo Gotardo, Thabo Beeler, Justus Thies, Timo Bolkart
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06903
• Pdf URL: https://arxiv.org/pdf/2501.06903
• Copy Paste: [[2501.06903]] Synthetic Prior for Few-Shot Drivable Head Avatar Inversion(https://arxiv.org/abs/2501.06903)
• Keywords: generative
• Abstract: We present SynShot, a novel method for the few-shot inversion of a drivable head avatar based on a synthetic prior. We tackle two major challenges. First, training a controllable 3D generative network requires a large number of diverse sequences, for which pairs of images and high-quality tracked meshes are not always available. Second, state-of-the-art monocular avatar models struggle to generalize to new views and expressions, lacking a strong prior and often overfitting to a specific viewpoint distribution. Inspired by machine learning models trained solely on synthetic data, we propose a method that learns a prior model from a large dataset of synthetic heads with diverse identities, expressions, and viewpoints. With few input images, SynShot fine-tunes the pretrained synthetic prior to bridge the domain gap, modeling a photorealistic head avatar that generalizes to novel expressions and viewpoints. We model the head avatar using 3D Gaussian splatting and a convolutional encoder-decoder that outputs Gaussian parameters in UV texture space. To account for the different modeling complexities over parts of the head (e.g., skin vs hair), we embed the prior with explicit control for upsampling the number of per-part primitives. Compared to state-of-the-art monocular methods that require thousands of real training images, SynShot significantly improves novel view and expression synthesis.

Title: Deep Learning and Foundation Models for Weather Prediction: A Survey

• Authors: Jimeng Shi, Azam Shirali, Bowen Jin, Sizhe Zhou, Wei Hu, Rahuul Rangaraj, Shaowen Wang, Jiawei Han, Zhaonan Wang, Upmanu Lall, Yanzhao Wu, Leonardo Bobadilla, Giri Narasimhan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06907
• Pdf URL: https://arxiv.org/pdf/2501.06907
• Copy Paste: [[2501.06907]] Deep Learning and Foundation Models for Weather Prediction: A Survey(https://arxiv.org/abs/2501.06907)
• Keywords: robust, generative
• Abstract: Physics-based numerical models have been the bedrock of atmospheric sciences for decades, offering robust solutions but often at the cost of significant computational resources. Deep learning (DL) models have emerged as powerful tools in meteorology, capable of analyzing complex weather and climate data by learning intricate dependencies and providing rapid predictions once trained. While these models demonstrate promising performance in weather prediction, often surpassing traditional physics-based methods, they still face critical challenges. This paper presents a comprehensive survey of recent deep learning and foundation models for weather prediction. We propose a taxonomy to classify existing models based on their training paradigms: deterministic predictive learning, probabilistic generative learning, and pre-training and fine-tuning. For each paradigm, we delve into the underlying model architectures, address major challenges, offer key insights, and propose targeted directions for future research. Furthermore, we explore real-world applications of these methods and provide a curated summary of open-source code repositories and widely used datasets, aiming to bridge research advancements with practical implementations while fostering open and trustworthy scientific practices in adopting cutting-edge artificial intelligence for weather prediction. The related sources are available at this https URL DL-Foundation-Models-Weather.

Title: Efficient Phishing URL Detection Using Graph-based Machine Learning and Loopy Belief Propagation

• Authors: Wenye Guo, Qun Wang, Hao Yue, Haijian Sun, Rose Qingyang Hu
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06912
• Pdf URL: https://arxiv.org/pdf/2501.06912
• Copy Paste: [[2501.06912]] Efficient Phishing URL Detection Using Graph-based Machine Learning and Loopy Belief Propagation(https://arxiv.org/abs/2501.06912)
• Keywords: security, attack, robust
• Abstract: The proliferation of mobile devices and online interactions have been threatened by different cyberattacks, where phishing attacks and malicious Uniform Resource Locators (URLs) pose significant risks to user security. Traditional phishing URL detection methods primarily rely on URL string-based features, which attackers often manipulate to evade detection. To address these limitations, we propose a novel graph-based machine learning model for phishing URL detection, integrating both URL structure and network-level features such as IP addresses and authoritative name servers. Our approach leverages Loopy Belief Propagation (LBP) with an enhanced convergence strategy to enable effective message passing and stable classification in the presence of complex graph structures. Additionally, we introduce a refined edge potential mechanism that dynamically adapts based on entity similarity and label relationships to further improve classification accuracy. Comprehensive experiments on real-world datasets demonstrate our model's effectiveness by achieving F1 score of up to 98.77\%. This robust and reproducible method advances phishing detection capabilities, offering enhanced reliability and valuable insights in the field of cybersecurity.

Title: Black-box optimization and quantum annealing for filtering out mislabeled training instances

• Authors: Makoto Otsuka, Kento Kodama, Keisuke Morita, Masayuki Ohzeki
• Subjects: cs.LG, cond-mat.stat-mech, quant-ph
• Abstract URL: https://arxiv.org/abs/2501.06916
• Pdf URL: https://arxiv.org/pdf/2501.06916
• Copy Paste: [[2501.06916]] Black-box optimization and quantum annealing for filtering out mislabeled training instances(https://arxiv.org/abs/2501.06916)
• Keywords: robust
• Abstract: This study proposes an approach for removing mislabeled instances from contaminated training datasets by combining surrogate model-based black-box optimization (BBO) with postprocessing and quantum annealing. Mislabeled training instances, a common issue in real-world datasets, often degrade model generalization, necessitating robust and efficient noise-removal strategies. The proposed method evaluates filtered training subsets based on validation loss, iteratively refines loss estimates through surrogate model-based BBO with postprocessing, and leverages quantum annealing to efficiently sample diverse training subsets with low validation error. Experiments on a noisy majority bit task demonstrate the method's ability to prioritize the removal of high-risk mislabeled instances. Integrating D-Wave's clique sampler running on a physical quantum annealer achieves faster optimization and higher-quality training subsets compared to OpenJij's simulated quantum annealing sampler or Neal's simulated annealing sampler, offering a scalable framework for enhancing dataset quality. This work highlights the effectiveness of the proposed method for supervised learning tasks, with future directions including its application to unsupervised learning, real-world datasets, and large-scale implementations.

Title: A Hybrid Virtual Element Method and Deep Learning Approach for Solving One-Dimensional Euler-Bernoulli Beams

• Authors: Paulo Akira F. Enabe, Rodrigo Provasi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06925
• Pdf URL: https://arxiv.org/pdf/2501.06925
• Copy Paste: [[2501.06925]] A Hybrid Virtual Element Method and Deep Learning Approach for Solving One-Dimensional Euler-Bernoulli Beams(https://arxiv.org/abs/2501.06925)
• Keywords: robust
• Abstract: A hybrid framework integrating the Virtual Element Method (VEM) with deep learning is presented as an initial step toward developing efficient and flexible numerical models for one-dimensional Euler-Bernoulli beams. The primary aim is to explore a data-driven surrogate model capable of predicting displacement fields across varying material and geometric parameters while maintaining computational efficiency. Building upon VEM's ability to handle higher-order polynomials and non-conforming discretizations, the method offers a robust numerical foundation for structural mechanics. A neural network architecture is introduced to separately process nodal and material-specific data, effectively capturing complex interactions with minimal reliance on large datasets. To address challenges in training, the model incorporates Sobolev training and GradNorm techniques, ensuring balanced loss contributions and enhanced generalization. While this framework is in its early stages, it demonstrates the potential for further refinement and development into a scalable alternative to traditional methods. The proposed approach lays the groundwork for advancing numerical and data-driven techniques in beam modeling, offering a foundation for future research in structural mechanics.

Title: CULTURE3D: Cultural Landmarks and Terrain Dataset for 3D Applications

• Authors: Xinyi Zheng, Steve Zhang, Weizhe Lin, Aaron Zhang, Walterio W. Mayol-Cuevas, Junxiao Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06927
• Pdf URL: https://arxiv.org/pdf/2501.06927
• Copy Paste: [[2501.06927]] CULTURE3D: Cultural Landmarks and Terrain Dataset for 3D Applications(https://arxiv.org/abs/2501.06927)
• Keywords: segmentation
• Abstract: In this paper, we present a large-scale fine-grained dataset using high-resolution images captured from locations worldwide. Compared to existing datasets, our dataset offers a significantly larger size and includes a higher level of detail, making it uniquely suited for fine-grained 3D applications. Notably, our dataset is built using drone-captured aerial imagery, which provides a more accurate perspective for capturing real-world site layouts and architectural structures. By reconstructing environments with these detailed images, our dataset supports applications such as the COLMAP format for Gaussian Splatting and the Structure-from-Motion (SfM) method. It is compatible with widely-used techniques including SLAM, Multi-View Stereo, and Neural Radiance Fields (NeRF), enabling accurate 3D reconstructions and point clouds. This makes it a benchmark for reconstruction and segmentation tasks. The dataset enables seamless integration with multi-modal data, supporting a range of 3D applications, from architectural reconstruction to virtual tourism. Its flexibility promotes innovation, facilitating breakthroughs in 3D modeling and analysis.

Title: Harnessing Large Language Models for Disaster Management: A Survey

• Authors: Zhenyu Lei, Yushun Dong, Weiyu Li, Rong Ding, Qi Wang, Jundong Li
• Subjects: cs.CL, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.06932
• Pdf URL: https://arxiv.org/pdf/2501.06932
• Copy Paste: [[2501.06932]] Harnessing Large Language Models for Disaster Management: A Survey(https://arxiv.org/abs/2501.06932)
• Keywords: large language model
• Abstract: Large language models (LLMs) have revolutionized scientific research with their exceptional capabilities and transformed various fields. Among their practical applications, LLMs have been playing a crucial role in mitigating threats to human life, infrastructure, and the environment. Despite growing research in disaster LLMs, there remains a lack of systematic review and in-depth analysis of LLMs for natural disaster management. To address the gap, this paper presents a comprehensive survey of existing LLMs in natural disaster management, along with a taxonomy that categorizes existing works based on disaster phases and application scenarios. By collecting public datasets and identifying key challenges and opportunities, this study aims to guide the professional community in developing advanced LLMs for disaster management to enhance the resilience against natural disasters.

Title: Comparison of Autoencoders for tokenization of ASL datasets

• Authors: Vouk Praun-Petrovic, Aadhvika Koundinya, Lavanya Prahallad
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.06942
• Pdf URL: https://arxiv.org/pdf/2501.06942
• Copy Paste: [[2501.06942]] Comparison of Autoencoders for tokenization of ASL datasets(https://arxiv.org/abs/2501.06942)
• Keywords: robust, extraction, diffusion, generative, large language model
• Abstract: Generative AI, powered by large language models (LLMs), has revolutionized applications across text, audio, images, and video. This study focuses on developing and evaluating encoder-decoder architectures for the American Sign Language (ASL) image dataset, consisting of 87,000 images across 29 hand sign classes. Three approaches were compared: Feedforward Autoencoders, Convolutional Autoencoders, and Diffusion Autoencoders. The Diffusion Autoencoder outperformed the others, achieving the lowest mean squared error (MSE) and highest Mean Opinion Score (MOS) due to its probabilistic noise modeling and iterative denoising capabilities. The Convolutional Autoencoder demonstrated effective spatial feature extraction but lacked the robustness of the diffusion process, while the Feedforward Autoencoder served as a baseline with limitations in handling complex image data. Objective and subjective evaluations confirmed the superiority of the Diffusion Autoencoder for high-fidelity image reconstruction, emphasizing its potential in multimodal AI applications such as sign language recognition and generation. This work provides critical insights into designing robust encoder-decoder systems to advance multimodal AI capabilities.

Title: ByzSFL: Achieving Byzantine-Robust Secure Federated Learning with Zero-Knowledge Proofs

• Authors: Yongming Fan, Rui Zhu, Zihao Wang, Chenghong Wang, Haixu Tang, Ye Dong, Hyunghoon Cho, Lucila Ohno-Machado
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06953
• Pdf URL: https://arxiv.org/pdf/2501.06953
• Copy Paste: [[2501.06953]] ByzSFL: Achieving Byzantine-Robust Secure Federated Learning with Zero-Knowledge Proofs(https://arxiv.org/abs/2501.06953)
• Keywords: secure, privacy, robust, federate
• Abstract: The advancement of AI models, especially those powered by deep learning, faces significant challenges in data-sensitive industries like healthcare and finance due to the distributed and private nature of data. Federated Learning (FL) and Secure Federated Learning (SFL) enable collaborative model training without data sharing, enhancing privacy by encrypting shared intermediate results. However, SFL currently lacks effective Byzantine robustness, a critical property that ensures model performance remains intact even when some participants act maliciously. Existing Byzantine-robust methods in FL are incompatible with SFL due to the inefficiency and limitations of encryption operations in handling complex aggregation calculations. This creates a significant gap in secure and robust model training. To address this gap, we propose ByzSFL, a novel SFL system that achieves Byzantine-robust secure aggregation with high efficiency. Our approach offloads aggregation weight calculations to individual parties and introduces a practical zero-knowledge proof (ZKP) protocol toolkit. This toolkit supports widely used operators for calculating aggregation weights, ensuring correct computations without compromising data privacy. Not only does this method maintain aggregation integrity, but it also significantly boosts computational efficiency, making ByzSFL approximately 100 times faster than existing solutions. Furthermore, our method aligns with open-source AI trends, enabling plaintext publication of the final model without additional information leakage, thereby enhancing the practicality and robustness of SFL in real-world applications.

Title: Compact Bayesian Neural Networks via pruned MCMC sampling

• Authors: Ratneel Deo, Scott Sisson, Jody M. Webster, Rohitash Chandra
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06962
• Pdf URL: https://arxiv.org/pdf/2501.06962
• Copy Paste: [[2501.06962]] Compact Bayesian Neural Networks via pruned MCMC sampling(https://arxiv.org/abs/2501.06962)
• Keywords: robust
• Abstract: Bayesian Neural Networks (BNNs) offer robust uncertainty quantification in model predictions, but training them presents a significant computational challenge. This is mainly due to the problem of sampling multimodal posterior distributions using Markov Chain Monte Carlo (MCMC) sampling and variational inference algorithms. Moreover, the number of model parameters scales exponentially with additional hidden layers, neurons, and features in the dataset. Typically, a significant portion of these densely connected parameters are redundant and pruning a neural network not only improves portability but also has the potential for better generalisation capabilities. In this study, we address some of the challenges by leveraging MCMC sampling with network pruning to obtain compact probabilistic models having removed redundant parameters. We sample the posterior distribution of model parameters (weights and biases) and prune weights with low importance, resulting in a compact model. We ensure that the compact BNN retains its ability to estimate uncertainty via the posterior distribution while retaining the model training and generalisation performance accuracy by adapting post-pruning resampling. We evaluate the effectiveness of our MCMC pruning strategy on selected benchmark datasets for regression and classification problems through empirical result analysis. We also consider two coral reef drill-core lithology classification datasets to test the robustness of the pruning model in complex real-world datasets. We further investigate if refining compact BNN can retain any loss of performance. Our results demonstrate the feasibility of training and pruning BNNs using MCMC whilst retaining generalisation performance with over 75% reduction in network size. This paves the way for developing compact BNN models that provide uncertainty estimates for real-world applications.

Title: Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot

• Authors: Antonio López Martínez, Alejandro Cano, Antonio Ruiz-Martínez
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06963
• Pdf URL: https://arxiv.org/pdf/2501.06963
• Copy Paste: [[2501.06963]] Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot(https://arxiv.org/abs/2501.06963)
• Keywords: security, generative
• Abstract: The advent of Generative Artificial Intelligence (GenAI) has brought a significant change to our society. GenAI can be applied across numerous fields, with particular relevance in cybersecurity. Among the various areas of application, its use in penetration testing (pentesting) or ethical hacking processes is of special interest. In this paper, we have analyzed the potential of leading generic-purpose GenAI tools-Claude Opus, GPT-4 from ChatGPT, and Copilot-in augmenting the penetration testing process as defined by the Penetration Testing Execution Standard (PTES). Our analysis involved evaluating each tool across all PTES phases within a controlled virtualized environment. The findings reveal that, while these tools cannot fully automate the pentesting process, they provide substantial support by enhancing efficiency and effectiveness in specific tasks. Notably, all tools demonstrated utility; however, Claude Opus consistently outperformed the others in our experimental scenarios.

Title: Combining LLM decision and RL action selection to improve RL policy for adaptive interventions

• Authors: Karine Karine, Benjamin M. Marlin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06980
• Pdf URL: https://arxiv.org/pdf/2501.06980
• Copy Paste: [[2501.06980]] Combining LLM decision and RL action selection to improve RL policy for adaptive interventions(https://arxiv.org/abs/2501.06980)
• Keywords: large language model
• Abstract: Reinforcement learning (RL) is increasingly being used in the healthcare domain, particularly for the development of personalized health adaptive interventions. Inspired by the success of Large Language Models (LLMs), we are interested in using LLMs to update the RL policy in real time, with the goal of accelerating personalization. We use the text-based user preference to influence the action selection on the fly, in order to immediately incorporate the user preference. We use the term "user preference" as a broad term to refer to a user personal preference, constraint, health status, or a statement expressing like or dislike, etc. Our novel approach is a hybrid method that combines the LLM response and the RL action selection to improve the RL policy. Given an LLM prompt that incorporates the user preference, the LLM acts as a filter in the typical RL action selection. We investigate different prompting strategies and action selection strategies. To evaluate our approach, we implement a simulation environment that generates the text-based user preferences and models the constraints that impact behavioral dynamics. We show that our approach is able to take into account the text-based user preferences, while improving the RL policy, thus improving personalization in adaptive intervention.

Title: LEO: Boosting Mixture of Vision Encoders for Multimodal Large Language Models

• Authors: Mozhgan Nasr Azadani, James Riddell, Sean Sedwards, Krzysztof Czarnecki
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.06986
• Pdf URL: https://arxiv.org/pdf/2501.06986
• Copy Paste: [[2501.06986]] LEO: Boosting Mixture of Vision Encoders for Multimodal Large Language Models(https://arxiv.org/abs/2501.06986)
• Keywords: large language model
• Abstract: Enhanced visual understanding serves as a cornerstone for multimodal large language models (MLLMs). Recent hybrid MLLMs incorporate a mixture of vision experts to address the limitations of using a single vision encoder and excessively long visual tokens. Despite the progress of these MLLMs, a research gap remains in effectively integrating diverse vision encoders. This work explores fusion strategies of visual tokens for hybrid MLLMs, leading to the design of LEO, a novel MLLM with a dual-branch vision encoder framework that incorporates a post-adaptation fusion strategy and adaptive tiling: for each segmented tile of the input images, LEO sequentially interleaves the visual tokens from its two vision encoders. Extensive evaluation across 13 vision-language benchmarks reveals that LEO outperforms state-of-the-art open-source MLLMs and hybrid MLLMs on the majority of tasks. Furthermore, we show that LEO can be adapted to the specialized domain of autonomous driving without altering the model architecture or training recipe, achieving competitive performance compared to existing baselines. The code and model will be publicly available.

Title: TFLAG:Towards Practical APT Detection via Deviation-Aware Learning on Temporal Provenance Graph

• Authors: Wenhan Jiang, Tingting Chai, Hongri Liu, Kai Wang, Hongke Zhang
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.06997
• Pdf URL: https://arxiv.org/pdf/2501.06997
• Copy Paste: [[2501.06997]] TFLAG:Towards Practical APT Detection via Deviation-Aware Learning on Temporal Provenance Graph(https://arxiv.org/abs/2501.06997)
• Keywords: attack, extraction
• Abstract: Advanced Persistent Threat (APT) have grown increasingly complex and concealed, posing formidable challenges to existing Intrusion Detection Systems in identifying and mitigating these attacks. Recent studies have incorporated graph learning techniques to extract detailed information from provenance graphs, enabling the detection of attacks with greater granularity. Nevertheless, existing studies have largely overlooked the continuous yet subtle temporal variations in the structure of provenance graphs, which may correspond to surreptitious perturbation anomalies in ongoing APT attacks. Therefore, we introduce TFLAG, an advanced anomaly detection framework that for the first time integrates the structural dynamic extraction capabilities of temporal graph model with the anomaly delineation abilities of deviation networks to pinpoint covert attack activities in provenance graphs. This self-supervised integration framework leverages the graph model to extract neighbor interaction data under continuous temporal changes from historical benign behaviors within provenance graphs, while simultaneously utilizing deviation networks to accurately distinguish authentic attack activities from false positive deviations due to unexpected subtle perturbations. The experimental results indicate that, through a comprehensive design that utilizes both attribute and temporal information, it can accurately identify the time windows associated with APT attack behaviors without prior knowledge (e.g., labeled data samples), demonstrating superior accuracy compared to current state-of-the-art methods in differentiating between attack events and system false positive events.

Title: Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps

• Authors: Henry Li, Ronen Basri, Yuval Kluger
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.06999
• Pdf URL: https://arxiv.org/pdf/2501.06999
• Copy Paste: [[2501.06999]] Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps(https://arxiv.org/abs/2501.06999)
• Keywords: diffusion, generative
• Abstract: Cascaded models are multi-scale generative models with a marked capacity for producing perceptually impressive samples at high resolutions. In this work, we show that they can also be excellent likelihood models, so long as we overcome a fundamental difficulty with probabilistic multi-scale models: the intractability of the likelihood function. Chiefly, in cascaded models each intermediary scale introduces extraneous variables that cannot be tractably marginalized out for likelihood evaluation. This issue vanishes by modeling the diffusion process on latent spaces induced by a class of transformations we call hierarchical volume-preserving maps, which decompose spatially structured data in a hierarchical fashion without introducing local distortions in the latent space. We demonstrate that two such maps are well-known in the literature for multiscale modeling: Laplacian pyramids and wavelet transforms. Not only do such reparameterizations allow the likelihood function to be directly expressed as a joint likelihood over the scales, we show that the Laplacian pyramid and wavelet transform also produces significant improvements to the state-of-the-art on a selection of benchmarks in likelihood modeling, including density estimation, lossless compression, and out-of-distribution detection. Investigating the theoretical basis of our empirical gains we uncover deep connections to score matching under the Earth Mover's Distance (EMD), which is a well-known surrogate for perceptual similarity. Code can be found at \href{this https URL}{this https url}.

Title: UNetVL: Enhancing 3D Medical Image Segmentation with Chebyshev KAN Powered Vision-LSTM

• Authors: Xuhui Guo, Tanmoy Dam, Rohan Dhamdhere, Gourav Modanwal, Anant Madabhushi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07017
• Pdf URL: https://arxiv.org/pdf/2501.07017
• Copy Paste: [[2501.07017]] UNetVL: Enhancing 3D Medical Image Segmentation with Chebyshev KAN Powered Vision-LSTM(https://arxiv.org/abs/2501.07017)
• Keywords: transformer, segmentation
• Abstract: 3D medical image segmentation has progressed considerably due to Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), yet these methods struggle to balance long-range dependency acquisition with computational efficiency. To address this challenge, we propose UNETVL (U-Net Vision-LSTM), a novel architecture that leverages recent advancements in temporal information processing. UNETVL incorporates Vision-LSTM (ViL) for improved scalability and memory functions, alongside an efficient Chebyshev Kolmogorov-Arnold Networks (KAN) to handle complex and long-range dependency patterns more effectively. We validated our method on the ACDC and AMOS2022 (post challenge Task 2) benchmark datasets, showing a significant improvement in mean Dice score compared to recent state-of-the-art approaches, especially over its predecessor, UNETR, with increases of 7.3% on ACDC and 15.6% on AMOS, respectively. Extensive ablation studies were conducted to demonstrate the impact of each component in UNETVL, providing a comprehensive understanding of its architecture. Our code is available at this https URL, facilitating further research and applications in this domain.

Title: ViSoLex: An Open-Source Repository for Vietnamese Social Media Lexical Normalization

• Authors: Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Kiet Van Nguyen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07020
• Pdf URL: https://arxiv.org/pdf/2501.07020
• Copy Paste: [[2501.07020]] ViSoLex: An Open-Source Repository for Vietnamese Social Media Lexical Normalization(https://arxiv.org/abs/2501.07020)
• Keywords: robust
• Abstract: ViSoLex is an open-source system designed to address the unique challenges of lexical normalization for Vietnamese social media text. The platform provides two core services: Non-Standard Word (NSW) Lookup and Lexical Normalization, enabling users to retrieve standard forms of informal language and standardize text containing NSWs. ViSoLex's architecture integrates pre-trained language models and weakly supervised learning techniques to ensure accurate and efficient normalization, overcoming the scarcity of labeled data in Vietnamese. This paper details the system's design, functionality, and its applications for researchers and non-technical users. Additionally, ViSoLex offers a flexible, customizable framework that can be adapted to various datasets and research requirements. By publishing the source code, ViSoLex aims to contribute to the development of more robust Vietnamese natural language processing tools and encourage further research in lexical normalization. Future directions include expanding the system's capabilities for additional languages and improving the handling of more complex non-standard linguistic patterns.

Title: Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions through Logical Reasoning

• Authors: Weixin Chen, Simon Yu, Huajie Shao, Lui Sha, Han Zhao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07021
• Pdf URL: https://arxiv.org/pdf/2501.07021
• Copy Paste: [[2501.07021]] Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions through Logical Reasoning(https://arxiv.org/abs/2501.07021)
• Keywords: interpretability
• Abstract: End-to-end deep neural networks have achieved remarkable success across various domains but are often criticized for their lack of interpretability. While post hoc explanation methods attempt to address this issue, they often fail to accurately represent these black-box models, resulting in misleading or incomplete explanations. To overcome these challenges, we propose an inherently transparent model architecture called Neural Probabilistic Circuits (NPCs), which enable compositional and interpretable predictions through logical reasoning. In particular, an NPC consists of two modules: an attribute recognition model, which predicts probabilities for various attributes, and a task predictor built on a probabilistic circuit, which enables logical reasoning over recognized attributes to make class predictions. To train NPCs, we introduce a three-stage training algorithm comprising attribute recognition, circuit construction, and joint optimization. Moreover, we theoretically demonstrate that an NPC's error is upper-bounded by a linear combination of the errors from its modules. To further demonstrate the interpretability of NPC, we provide both the most probable explanations and the counterfactual explanations. Empirical results on four benchmark datasets show that NPCs strike a balance between interpretability and performance, achieving results competitive even with those of end-to-end black-box models while providing enhanced interpretability.

Title: Hybrid Scheme of Post-Quantum Cryptography and Elliptic-Curve Cryptography for Certificates -- A Case Study of Security Credential Management System in Vehicle-to-Everything Communications

• Authors: Abel C. H. Chen, Bon-Yeh Lin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07028
• Pdf URL: https://arxiv.org/pdf/2501.07028
• Copy Paste: [[2501.07028]] Hybrid Scheme of Post-Quantum Cryptography and Elliptic-Curve Cryptography for Certificates -- A Case Study of Security Credential Management System in Vehicle-to-Everything Communications(https://arxiv.org/abs/2501.07028)
• Keywords: security, privacy, attack
• Abstract: Due to the current standard of Security Credential Management System (SCMS) for Vehicle-to-Everything (V2X) communications using asymmetric cryptography, specifically Elliptic-Curve Cryptography (ECC), which may be vulnerable to quantum computing attacks. Therefore, the V2X SCMS is threatened by quantum computing attacks. However, although the National Institute of Standards and Technology (NIST) has already selected Post-Quantum Cryptography (PQC) algorithms as the standard, the current PQC algorithms may have issues such as longer public key lengths, longer signature lengths, or lower signature generation and verification efficiency, which may not fully meet the requirements of V2X communication applications. In view of the challenges in V2X communication, such as packet length, signature generation and verification efficiency, security level, and vehicle privacy, this study proposes a hybrid certificate scheme of PQC and ECC. By leveraging the strengths of both PQC and ECC, this scheme aims to overcome the challenges in V2X communication. PQC is used to establish a security level resistant to quantum computing attacks, while ECC is utilized to establish anonymous certificates and reduce packet length to meet the requirements of V2X communication. In the practical experiments, the study implemented the SCMS end entity based on the Chunghwa Telecom SCMS and the Clientron On-Board Unit (OBU) to conduct field tests in Danhai New Town in New Taipei City. The performance of various existing hybrid certificate schemes combining PQC (e.g., Dilithium, Falcon, and SPHINCS+) and ECC is compared, and a practical solution is provided for V2X industries.

Title: PRKAN: Parameter-Reduced Kolmogorov-Arnold Networks

• Authors: Hoang-Thang Ta, Duy-Quy Thai, Anh Tran, Grigori Sidorov, Alexander Gelbukh
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07032
• Pdf URL: https://arxiv.org/pdf/2501.07032
• Copy Paste: [[2501.07032]] PRKAN: Parameter-Reduced Kolmogorov-Arnold Networks(https://arxiv.org/abs/2501.07032)
• Keywords: transformer
• Abstract: Kolmogorov-Arnold Networks (KANs) represent an innovation in neural network architectures, offering a compelling alternative to Multi-Layer Perceptrons (MLPs) in models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers. By advancing network design, KANs are driving groundbreaking research and enabling transformative applications across various scientific domains involving neural networks. However, existing KANs often require significantly more parameters in their network layers compared to MLPs. To address this limitation, this paper introduces PRKANs (\textbf{P}arameter-\textbf{R}educed \textbf{K}olmogorov-\textbf{A}rnold \textbf{N}etworks), which employ several methods to reduce the parameter count in KAN layers, making them comparable to MLP layers. Experimental results on the MNIST and Fashion-MNIST datasets demonstrate that PRKANs with attention mechanisms outperform several existing KANs and rival the performance of MLPs, albeit with slightly longer training times. Furthermore, the study highlights the advantages of Gaussian Radial Basis Functions (GRBFs) and layer normalization in KAN designs. The repository for this work is available at: \url{this https URL}.

Title: Detection of AI Deepfake and Fraud in Online Payments Using GAN-Based Models

• Authors: Zong Ke, Shicheng Zhou, Yining Zhou, Chia Hong Chang, Rong Zhang
• Subjects: cs.LG, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07033
• Pdf URL: https://arxiv.org/pdf/2501.07033
• Copy Paste: [[2501.07033]] Detection of AI Deepfake and Fraud in Online Payments Using GAN-Based Models(https://arxiv.org/abs/2501.07033)
• Keywords: security, robust, generative
• Abstract: This study explores the use of Generative Adversarial Networks (GANs) to detect AI deepfakes and fraudulent activities in online payment systems. With the growing prevalence of deepfake technology, which can manipulate facial features in images and videos, the potential for fraud in online transactions has escalated. Traditional security systems struggle to identify these sophisticated forms of fraud. This research proposes a novel GAN-based model that enhances online payment security by identifying subtle manipulations in payment images. The model is trained on a dataset consisting of real-world online payment images and deepfake images generated using advanced GAN architectures, such as StyleGAN and DeepFake. The results demonstrate that the proposed model can accurately distinguish between legitimate transactions and deepfakes, achieving a high detection rate above 95%. This approach significantly improves the robustness of payment systems against AI-driven fraud. The paper contributes to the growing field of digital security, offering insights into the application of GANs for fraud detection in financial services. Keywords- Payment Security, Image Recognition, Generative Adversarial Networks, AI Deepfake, Fraudulent Activities

Title: Protego: Detecting Adversarial Examples for Vision Transformers via Intrinsic Capabilities

• Authors: Jialin Wu, Kaikai Pan, Yanjiao Chen, Jiangyi Deng, Shengyuan Pang, Wenyuan Xu
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07044
• Pdf URL: https://arxiv.org/pdf/2501.07044
• Copy Paste: [[2501.07044]] Protego: Detecting Adversarial Examples for Vision Transformers via Intrinsic Capabilities(https://arxiv.org/abs/2501.07044)
• Keywords: security, attack, transformer
• Abstract: Transformer models have excelled in natural language tasks, prompting the vision community to explore their implementation in computer vision problems. However, these models are still influenced by adversarial examples. In this paper, we investigate the attack capabilities of six common adversarial attacks on three pretrained ViT models to reveal the vulnerability of ViT models. To understand and analyse the bias in neural network decisions when the input is adversarial, we use two visualisation techniques that are attention rollout and grad attention rollout. To prevent ViT models from adversarial attack, we propose Protego, a detection framework that leverages the transformer intrinsic capabilities to detection adversarial examples of ViT models. Nonetheless, this is challenging due to a diversity of attack strategies that may be adopted by adversaries. Inspired by the attention mechanism, we know that the token of prediction contains all the information from the input sample. Additionally, the attention region for adversarial examples differs from that of normal examples. Given these points, we can train a detector that achieves superior performance than existing detection methods to identify adversarial examples. Our experiments have demonstrated the high effectiveness of our detection method. For these six adversarial attack methods, our detector's AUC scores all exceed 0.95. Protego may advance investigations in metaverse security.

Title: Leveraging ASIC AI Chips for Homomorphic Encryption

• Authors: Jianming Tong, Tianhao Huang, Leo de Castro, Anirudh Itagi, Jingtian Dang, Anupam Golder, Asra Ali, Jevin Jiang, Arvind, G. Edward Suh, Tushar Krishna
• Subjects: cs.CR, cs.AR, cs.CL, cs.PL
• Abstract URL: https://arxiv.org/abs/2501.07047
• Pdf URL: https://arxiv.org/pdf/2501.07047
• Copy Paste: [[2501.07047]] Leveraging ASIC AI Chips for Homomorphic Encryption(https://arxiv.org/abs/2501.07047)
• Keywords: privacy
• Abstract: Cloud-based services are making the outsourcing of sensitive client data increasingly common. Although homomorphic encryption (HE) offers strong privacy guarantee, it requires substantially more resources than computing on plaintext, often leading to unacceptably large latencies in getting the results. HE accelerators have emerged to mitigate this latency issue, but with the high cost of ASICs. In this paper we show that HE primitives can be converted to AI operators and accelerated on existing ASIC AI accelerators, like TPUs, which are already widely deployed in the cloud. Adapting such accelerators for HE requires (1) supporting modular multiplication, (2) high-precision arithmetic in software, and (3) efficient mapping on matrix engines. We introduce the CROSS compiler (1) to adopt Barrett reduction to provide modular reduction support using multiplier and adder, (2) Basis Aligned Transformation (BAT) to convert high-precision multiplication as low-precision matrix-vector multiplication, (3) Matrix Aligned Transformation (MAT) to covert vectorized modular operation with reduction into matrix multiplication that can be efficiently processed on 2D spatial matrix engine. Our evaluation of CROSS on a Google TPUv4 demonstrates significant performance improvements, with up to 161x and 5x speedup compared to the previous work on many-core CPUs and V100. The kernel-level codes are open-sourced at this https URL.

Title: SFC-GAN: A Generative Adversarial Network for Brain Functional and Structural Connectome Translation

• Authors: Yee-Fan Tan, Jun Lin Liow, Pei-Sze Tan, Fuad Noman, Raphael C.-W. Phan, Hernando Ombao, Chee-Ming Ting
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07055
• Pdf URL: https://arxiv.org/pdf/2501.07055
• Copy Paste: [[2501.07055]] SFC-GAN: A Generative Adversarial Network for Brain Functional and Structural Connectome Translation(https://arxiv.org/abs/2501.07055)
• Keywords: diffusion, generative
• Abstract: Modern brain imaging technologies have enabled the detailed reconstruction of human brain connectomes, capturing structural connectivity (SC) from diffusion MRI and functional connectivity (FC) from functional MRI. Understanding the intricate relationships between SC and FC is vital for gaining deeper insights into the brain's functional and organizational mechanisms. However, obtaining both SC and FC modalities simultaneously remains challenging, hindering comprehensive analyses. Existing deep generative models typically focus on synthesizing a single modality or unidirectional translation between FC and SC, thereby missing the potential benefits of bi-directional translation, especially in scenarios where only one connectome is available. Therefore, we propose Structural-Functional Connectivity GAN (SFC-GAN), a novel framework for bidirectional translation between SC and FC. This approach leverages the CycleGAN architecture, incorporating convolutional layers to effectively capture the spatial structures of brain connectomes. To preserve the topological integrity of these connectomes, we employ a structure-preserving loss that guides the model in capturing both global and local connectome patterns while maintaining symmetry. Our framework demonstrates superior performance in translating between SC and FC, outperforming baseline models in similarity and graph property evaluations compared to ground truth data, each translated modality can be effectively utilized for downstream classification.

Title: Logic Meets Magic: LLMs Cracking Smart Contract Vulnerabilities

• Authors: ZeKe Xiao, Qin Wang, Hammond Pearce, Shiping Chen
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07058
• Pdf URL: https://arxiv.org/pdf/2501.07058
• Copy Paste: [[2501.07058]] Logic Meets Magic: LLMs Cracking Smart Contract Vulnerabilities(https://arxiv.org/abs/2501.07058)
• Keywords: large language model
• Abstract: Smart contract vulnerabilities caused significant economic losses in blockchain applications. Large Language Models (LLMs) provide new possibilities for addressing this time-consuming task. However, state-of-the-art LLM-based detection solutions are often plagued by high false-positive rates. In this paper, we push the boundaries of existing research in two key ways. First, our evaluation is based on Solidity v0.8, offering the most up-to-date insights compared to prior studies that focus on older versions (v0.4). Second, we leverage the latest five LLM models (across companies), ensuring comprehensive coverage across the most advanced capabilities in the field. We conducted a series of rigorous evaluations. Our experiments demonstrate that a well-designed prompt can reduce the false-positive rate by over 60%. Surprisingly, we also discovered that the recall rate for detecting some specific vulnerabilities in Solidity v0.8 has dropped to just 13% compared to earlier versions (i.e., v0.4). Further analysis reveals the root cause of this decline: the reliance of LLMs on identifying changes in newly introduced libraries and frameworks during detection.

Title: Hierarchical Superpixel Segmentation via Structural Information Theory

• Authors: Minhui Xie, Hao Peng, Pu Li, Guangjie Zeng, Shuhai Wang, Jia Wu, Peng Li, Philip S. Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07069
• Pdf URL: https://arxiv.org/pdf/2501.07069
• Copy Paste: [[2501.07069]] Hierarchical Superpixel Segmentation via Structural Information Theory(https://arxiv.org/abs/2501.07069)
• Keywords: segmentation
• Abstract: Superpixel segmentation is a foundation for many higher-level computer vision tasks, such as image segmentation, object recognition, and scene understanding. Existing graph-based superpixel segmentation methods typically concentrate on the relationships between a given pixel and its directly adjacent pixels while overlooking the influence of non-adjacent pixels. These approaches do not fully leverage the global information in the graph, leading to suboptimal segmentation quality. To address this limitation, we present SIT-HSS, a hierarchical superpixel segmentation method based on structural information theory. Specifically, we first design a novel graph construction strategy that incrementally explores the pixel neighborhood to add edges based on 1-dimensional structural entropy (1D SE). This strategy maximizes the retention of graph information while avoiding an overly complex graph structure. Then, we design a new 2D SE-guided hierarchical graph partitioning method, which iteratively merges pixel clusters layer by layer to reduce the graph's 2D SE until a predefined segmentation scale is achieved. Experimental results on three benchmark datasets demonstrate that the SIT-HSS performs better than state-of-the-art unsupervised superpixel segmentation algorithms. The source code is available at \url{this https URL}.

Title: Enhancing Image Generation Fidelity via Progressive Prompts

• Authors: Zhen Xiong, Yuqi Li, Chuanguang Yang, Tiao Tan, Zhihong Zhu, Siyuan Li, Yue Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07070
• Pdf URL: https://arxiv.org/pdf/2501.07070
• Copy Paste: [[2501.07070]] Enhancing Image Generation Fidelity via Progressive Prompts(https://arxiv.org/abs/2501.07070)
• Keywords: diffusion, transformer, large language model
• Abstract: The diffusion transformer (DiT) architecture has attracted significant attention in image generation, achieving better fidelity, performance, and diversity. However, most existing DiT - based image generation methods focus on global - aware synthesis, and regional prompt control has been less explored. In this paper, we propose a coarse - to - fine generation pipeline for regional prompt - following generation. Specifically, we first utilize the powerful large language model (LLM) to generate both high - level descriptions of the image (such as content, topic, and objects) and low - level descriptions (such as details and style). Then, we explore the influence of cross - attention layers at different depths. We find that deeper layers are always responsible for high - level content control, while shallow layers handle low - level content control. Various prompts are injected into the proposed regional cross - attention control for coarse - to - fine generation. By using the proposed pipeline, we enhance the controllability of DiT - based image generation. Extensive quantitative and qualitative results show that our pipeline can improve the performance of the generated images.

Title: Representation Learning of Point Cloud Upsampling in Global and Local Inputs

• Authors: Tongxu Zhang, Bei Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07076
• Pdf URL: https://arxiv.org/pdf/2501.07076
• Copy Paste: [[2501.07076]] Representation Learning of Point Cloud Upsampling in Global and Local Inputs(https://arxiv.org/abs/2501.07076)
• Keywords: interpretability
• Abstract: In recent years, point cloud upsampling has been widely applied in fields such as 3D reconstruction. Our study investigates the factors influencing point cloud upsampling on both global and local levels through representation learning. Specifically, the paper inputs global and local information of the same point cloud model object into two encoders to extract these features, fuses them, and then feeds the combined features into an upsampling decoder. The goal is to address issues of sparsity and noise in point clouds by leveraging prior knowledge from both global and local inputs. And the proposed framework can be applied to any state-of-the-art point cloud upsampling neural network. Experiments were conducted on a series of autoencoder-based models utilizing deep learning, yielding interpretability for both global and local inputs, and it has been proven in the results that our proposed framework can further improve the upsampling effect in previous SOTA works. At the same time, the Saliency Map reflects the differences between global and local feature inputs, as well as the effectiveness of training with both inputs in parallel.

Title: D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation

• Authors: Zhejun Zhang, Yuanping Chen, Shibing Chu
• Subjects: cs.LG, physics.chem-ph
• Abstract URL: https://arxiv.org/abs/2501.07077
• Pdf URL: https://arxiv.org/pdf/2501.07077
• Copy Paste: [[2501.07077]] D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation(https://arxiv.org/abs/2501.07077)
• Keywords: robust, diffusion, transformer, generative
• Abstract: Understanding and predicting the diverse conformational states of molecules is crucial for advancing fields such as chemistry, material science, and drug development. Despite significant progress in generative models, accurately generating complex and biologically or material-relevant molecular structures remains a major challenge. In this work, we introduce a diffusion model for three-dimensional (3D) molecule generation that combines a classifiable diffusion model, Diffusion Transformer, with multihead equivariant self-attention. This method addresses two key challenges: correctly attaching hydrogen atoms in generated molecules through learning representations of molecules after hydrogen atoms are removed; and overcoming the limitations of existing models that cannot generate molecules across multiple classes simultaneously. The experimental results demonstrate that our model not only achieves state-of-the-art performance across several key metrics but also exhibits robustness and versatility, making it highly suitable for early-stage large-scale generation processes in molecular design, followed by validation and further screening to obtain molecules with specific properties.

Title: AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR

• Authors: The Chuong Chu, Vu Tuan Dat Pham, Kien Dao, Hoang Nguyen, Quoc Hung Truong
• Subjects: cs.CL, cs.AI, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2501.07102
• Pdf URL: https://arxiv.org/pdf/2501.07102
• Copy Paste: [[2501.07102]] AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR(https://arxiv.org/abs/2501.07102)
• Keywords: robust
• Abstract: Intra-sentential code-switching (CS) refers to the alternation between languages that happens within a single utterance and is a significant challenge for Automatic Speech Recognition (ASR) systems. For example, when a Vietnamese speaker uses foreign proper names or specialized terms within their speech. ASR systems often struggle to accurately transcribe intra-sentential CS due to their training on monolingual data and the unpredictable nature of CS. This issue is even more pronounced for low-resource languages, where limited data availability hinders the development of robust models. In this study, we propose AdaCS, a normalization model integrates an adaptive bias attention module (BAM) into encoder-decoder network. This novel approach provides a robust solution to CS ASR in unseen domains, thereby significantly enhancing our contribution to the field. By utilizing BAM to both identify and normalize CS phrases, AdaCS enhances its adaptive capabilities with a biased list of words provided during inference. Our method demonstrates impressive performance and the ability to handle unseen CS phrases across various domains. Experiments show that AdaCS outperforms previous state-of-the-art method on Vietnamese CS ASR normalization by considerable WER reduction of 56.2% and 36.8% on the two proposed test sets.

Title: The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering

• Authors: Anupam Pandey, Deepjyoti Bodo, Arpan Phukan, Asif Ekbal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07109
• Pdf URL: https://arxiv.org/pdf/2501.07109
• Copy Paste: [[2501.07109]] The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering(https://arxiv.org/abs/2501.07109)
• Keywords: interpretability, transformer
• Abstract: Visual Question Answering (VQA) is an interdisciplinary field that bridges the gap between computer vision (CV) and natural language processing(NLP), enabling Artificial Intelligence(AI) systems to answer questions about images. Since its inception in 2015, VQA has rapidly evolved, driven by advances in deep learning, attention mechanisms, and transformer-based models. This survey traces the journey of VQA from its early days, through major breakthroughs, such as attention mechanisms, compositional reasoning, and the rise of vision-language pre-training methods. We highlight key models, datasets, and techniques that shaped the development of VQA systems, emphasizing the pivotal role of transformer architectures and multimodal pre-training in driving recent progress. Additionally, we explore specialized applications of VQA in domains like healthcare and discuss ongoing challenges, such as dataset bias, model interpretability, and the need for common-sense reasoning. Lastly, we discuss the emerging trends in large multimodal language models and the integration of external knowledge, offering insights into the future directions of VQA. This paper aims to provide a comprehensive overview of the evolution of VQA, highlighting both its current state and potential advancements.

Title: Duplex: Dual Prototype Learning for Compositional Zero-Shot Learning

• Authors: Zhong Peng, Yishi Xu, Gerong Wang, Wenchao Chen, Bo Chen, Jing Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07114
• Pdf URL: https://arxiv.org/pdf/2501.07114
• Copy Paste: [[2501.07114]] Duplex: Dual Prototype Learning for Compositional Zero-Shot Learning(https://arxiv.org/abs/2501.07114)
• Keywords: robust
• Abstract: Compositional Zero-Shot Learning (CZSL) aims to enable models to recognize novel compositions of visual states and objects that were absent during training. Existing methods predominantly focus on learning semantic representations of seen compositions but often fail to disentangle the independent features of states and objects in images, thereby limiting their ability to generalize to unseen compositions. To address this challenge, we propose Duplex, a novel dual-prototype learning method that integrates semantic and visual prototypes through a carefully designed dual-branch architecture, enabling effective representation learning for compositional tasks. Duplex utilizes a Graph Neural Network (GNN) to adaptively update visual prototypes, capturing complex interactions between states and objects. Additionally, it leverages the strong visual-semantic alignment of pre-trained Vision-Language Models (VLMs) and employs a multi-path architecture combined with prompt engineering to align image and text representations, ensuring robust generalization. Extensive experiments on three benchmark datasets demonstrate that Duplex outperforms state-of-the-art methods in both closed-world and open-world settings.

Title: LLM360 K2: Scaling Up 360-Open-Source Large Language Models

• Authors: Zhengzhong Liu, Bowen Tan, Hongyi Wang, Willie Neiswanger, Tianhua Tao, Haonan Li, Fajri Koto, Yuqi Wang, Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Liqun Ma, Liping Tang, Nikhil Ranjan, Yonghao Zhuang, Guowei He, Renxi Wang, Mingkai Deng, Robin Algayres, Yuanzhi Li, Zhiqiang Shen, Preslav Nakov, Eric Xing
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07124
• Pdf URL: https://arxiv.org/pdf/2501.07124
• Copy Paste: [[2501.07124]] LLM360 K2: Scaling Up 360-Open-Source Large Language Models(https://arxiv.org/abs/2501.07124)
• Keywords: protect, large language model
• Abstract: We detail the training of the LLM360 K2-65B model, scaling up our 360-degree OPEN SOURCE approach to the largest and most powerful models under project LLM360. While open-source LLMs continue to advance, the answer to "How are the largest LLMs trained?" remains unclear within the community. The implementation details for such high-capacity models are often protected due to business considerations associated with their high cost. This lack of transparency prevents LLM researchers from leveraging valuable insights from prior experience, e.g., "What are the best practices for addressing loss spikes?" The LLM360 K2 project addresses this gap by providing full transparency and access to resources accumulated during the training of LLMs at the largest scale. This report highlights key elements of the K2 project, including our first model, K2 DIAMOND, a 65 billion-parameter LLM that surpasses LLaMA-65B and rivals LLaMA2-70B, while requiring fewer FLOPs and tokens. We detail the implementation steps and present a longitudinal analysis of K2 DIAMOND's capabilities throughout its training process. We also outline ongoing projects such as TXT360, setting the stage for future models in the series. By offering previously unavailable resources, the K2 project also resonates with the 360-degree OPEN SOURCE principles of transparency, reproducibility, and accessibility, which we believe are vital in the era of resource-intensive AI research.

Title: Beyond the Surface: An NLP-based Methodology to Automatically Estimate CVE Relevance for CAPEC Attack Patterns

• Authors: Silvia Bonomi, Andrea Ciavotta, Simone Lenti, Alessandro Palma
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07131
• Pdf URL: https://arxiv.org/pdf/2501.07131
• Copy Paste: [[2501.07131]] Beyond the Surface: An NLP-based Methodology to Automatically Estimate CVE Relevance for CAPEC Attack Patterns(https://arxiv.org/abs/2501.07131)
• Keywords: security, attack
• Abstract: Threat analysis is continuously growing in importance due to the always-increasing complexity and frequency of cyber attacks. Analyzing threats demands significant effort from security experts, leading to delays in the security analysis process. Different cybersecurity knowledge bases are currently available to support this task but manual efforts are often required to correlate such heterogenous sources into a unified view that would enable a more comprehensive assessment. To address this gap, we propose a methodology leveraging Natural Language Processing (NLP) to effectively and efficiently associate Common Vulnerabilities and Exposure (CVE) vulnerabilities with Common Attack Pattern Enumeration and Classification (CAPEC) attack patterns. The proposed technique combines semantic similarity with keyword analysis to improve the accuracy of association estimations. Experimental evaluations demonstrate superior performance compared to state-of-the-art models, reducing manual effort and analysis time, and enabling cybersecurity professionals to prioritize critical tasks.

Title: Robust Single Object Tracking in LiDAR Point Clouds under Adverse Weather Conditions

• Authors: Xiantong Zhao, Xiuping Liu, Shengjing Tian, Yinan Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07133
• Pdf URL: https://arxiv.org/pdf/2501.07133
• Copy Paste: [[2501.07133]] Robust Single Object Tracking in LiDAR Point Clouds under Adverse Weather Conditions(https://arxiv.org/abs/2501.07133)
• Keywords: robust
• Abstract: 3D single object tracking (3DSOT) in LiDAR point clouds is a critical task for outdoor perception, enabling real-time perception of object location, orientation, and motion. Despite the impressive performance of current 3DSOT methods, evaluating them on clean datasets inadequately reflects their comprehensive performance, as the adverse weather conditions in real-world surroundings has not been considered. One of the main obstacles is the lack of adverse weather benchmarks for the evaluation of 3DSOT. To this end, this work proposes a challenging benchmark for LiDAR-based 3DSOT in adverse weather, which comprises two synthetic datasets (KITTI-A and nuScenes-A) and one real-world dataset (CADC-SOT) spanning three weather types: rain, fog, and snow. Based on this benchmark, five representative 3D trackers from different tracking frameworks conducted robustness evaluation, resulting in significant performance degradations. This prompts the question: What are the factors that cause current advanced methods to fail on such adverse weather samples? Consequently, we explore the impacts of adverse weather and answer the above question from three perspectives: 1) target distance; 2) template shape corruption; and 3) target shape corruption. Finally, based on domain randomization and contrastive learning, we designed a dual-branch tracking framework for adverse weather, named DRCT, achieving excellent performance in benchmarks.

Title: TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments

• Authors: Chenyang Qi, Huiping Li, Panfeng Huang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07146
• Pdf URL: https://arxiv.org/pdf/2501.07146
• Copy Paste: [[2501.07146]] TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments(https://arxiv.org/abs/2501.07146)
• Keywords: transformer
• Abstract: In recent years, meta-reinforcement learning (meta-RL) algorithm has been proposed to improve sample efficiency in the field of decision-making and control, enabling agents to learn new knowledge from a small number of samples. However, most research uses the Gaussian distribution to extract task representation, which is poorly adapted to tasks that change in non-stationary environment. To address this problem, we propose a novel meta-reinforcement learning method by leveraging Gaussian mixture model and the transformer network to construct task inference model. The Gaussian mixture model is utilized to extend the task representation and conduct explicit encoding of tasks. Specifically, the classification of tasks is encoded through transformer network to determine the Gaussian component corresponding to the task. By leveraging task labels, the transformer network is trained using supervised learning. We validate our method on MuJoCo benchmarks with non-stationary and multi-task environments. Experimental results demonstrate that the proposed method dramatically improves sample efficiency and accurately recognizes the classification of the tasks, while performing excellently in the environment.

Title: Pantomime: Towards the Anonymization of Motion Data using Foundation Motion Models

• Authors: Simon Hanisch, Julian Todt, Thorsten Strufe
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07149
• Pdf URL: https://arxiv.org/pdf/2501.07149
• Copy Paste: [[2501.07149]] Pantomime: Towards the Anonymization of Motion Data using Foundation Motion Models(https://arxiv.org/abs/2501.07149)
• Keywords: privacy, protect, biometric, extraction
• Abstract: Human motion is a behavioral biometric trait that can be used to identify individuals and infer private attributes such as medical conditions. This poses a serious privacy threat as motion extraction from video and motion capture are increasingly used for a variety of applications, including mixed reality, robotics, medicine, and the quantified self. In order to protect the privacy of the tracked individuals, anonymization techniques that preserve the utility of the data are required. However, anonymizing motion data is a challenging task because there are many dependencies in motion sequences (such as physiological constraints) that, if ignored, make the anonymized motion sequence appear unnatural. In this paper, we propose Pantomime, a full-body anonymization technique for motion data, which uses foundation motion models to generate motion sequences that adhere to the dependencies in the data, thus keeping the utility of the anonymized data high. Our results show that Pantomime can maintain the naturalness of the motion sequences while reducing the identification accuracy to 10%.

Title: Eye Sclera for Fair Face Image Quality Assessment

• Authors: Wassim Kabbani, Kiran Raja, Raghavendra Ramachandra, Christoph Busch
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07158
• Pdf URL: https://arxiv.org/pdf/2501.07158
• Copy Paste: [[2501.07158]] Eye Sclera for Fair Face Image Quality Assessment(https://arxiv.org/abs/2501.07158)
• Keywords: fair
• Abstract: Fair operational systems are crucial in gaining and maintaining society's trust in face recognition systems (FRS). FRS start with capturing an image and assessing its quality before using it further for enrollment or verification. Fair Face Image Quality Assessment (FIQA) schemes therefore become equally important in the context of fair FRS. This work examines the sclera as a quality assessment region for obtaining a fair FIQA. The sclera region is agnostic to demographic variations and skin colour for assessing the quality of a face image. We analyze three skin tone related ISO/IEC face image quality assessment measures and assess the sclera region as an alternative area for assessing FIQ. Our analysis of the face dataset of individuals from different demographic groups representing different skin tones indicates sclera as an alternative to measure dynamic range, over- and under-exposure of face using sclera region alone. The sclera region being agnostic to skin tone, i.e., demographic factors, provides equal utility as a fair FIQA as shown by our Error-vs-Discard Characteristic (EDC) curve analysis.

Title: Adaptive Noise-Tolerant Network for Image Segmentation

• Authors: Weizhi Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07163
• Pdf URL: https://arxiv.org/pdf/2501.07163
• Copy Paste: [[2501.07163]] Adaptive Noise-Tolerant Network for Image Segmentation(https://arxiv.org/abs/2501.07163)
• Keywords: segmentation
• Abstract: Unlike image classification and annotation, for which deep network models have achieved dominating superior performances compared to traditional computer vision algorithms, deep learning for automatic image segmentation still faces critical challenges. One of such hurdles is to obtain ground-truth segmentations as the training labels for deep network training. Especially when we study biomedical images, such as histopathological images (histo-images), it is unrealistic to ask for manual segmentation labels as the ground truth for training due to the fine image resolution as well as the large image size and complexity. In this paper, instead of relying on clean segmentation labels, we study whether and how integrating imperfect or noisy segmentation results from off-the-shelf segmentation algorithms may help achieve better segmentation results through a new Adaptive Noise-Tolerant Network (ANTN) model. We extend the noisy label deep learning to image segmentation with two novel aspects: (1) multiple noisy labels can be integrated into one deep learning model; (2) noisy segmentation modeling, including probabilistic parameters, is adaptive, depending on the given testing image appearance. Implementation of the new ANTN model on both the synthetic data and real-world histo-images demonstrates its effectiveness and superiority over off-the-shelf and other existing deep-learning-based image segmentation algorithms.

Title: Knowledge Distillation and Enhanced Subdomain Adaptation Using Graph Convolutional Network for Resource-Constrained Bearing Fault Diagnosis

• Authors: Mohammadreza Kavianpour, Parisa Kavianpour, Amin Ramezani, Mohammad TH Beheshti
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.07173
• Pdf URL: https://arxiv.org/pdf/2501.07173
• Copy Paste: [[2501.07173]] Knowledge Distillation and Enhanced Subdomain Adaptation Using Graph Convolutional Network for Resource-Constrained Bearing Fault Diagnosis(https://arxiv.org/abs/2501.07173)
• Keywords: robust
• Abstract: Bearing fault diagnosis under varying working conditions faces challenges, including a lack of labeled data, distribution discrepancies, and resource constraints. To address these issues, we propose a progressive knowledge distillation framework that transfers knowledge from a complex teacher model, utilizing a Graph Convolutional Network (GCN) with Autoregressive moving average (ARMA) filters, to a compact and efficient student model. To mitigate distribution discrepancies and labeling uncertainty, we introduce Enhanced Local Maximum Mean Squared Discrepancy (ELMMSD), which leverages mean and variance statistics in the Reproducing Kernel Hilbert Space (RKHS) and incorporates a priori probability distributions between labels. This approach increases the distance between clustering centers, bridges subdomain gaps, and enhances subdomain alignment reliability. Experimental results on benchmark datasets (CWRU and JNU) demonstrate that the proposed method achieves superior diagnostic accuracy while significantly reducing computational costs. Comprehensive ablation studies validate the effectiveness of each component, highlighting the robustness and adaptability of the approach across diverse working conditions.

Title: Generalizable Graph Neural Networks for Robust Power Grid Topology Control

• Authors: Matthijs de Jong, Jan Viebahn, Yuliya Shapovalova
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.07186
• Pdf URL: https://arxiv.org/pdf/2501.07186
• Copy Paste: [[2501.07186]] Generalizable Graph Neural Networks for Robust Power Grid Topology Control(https://arxiv.org/abs/2501.07186)
• Keywords: robust
• Abstract: The energy transition necessitates new congestion management methods. One such method is controlling the grid topology with machine learning (ML). This approach has gained popularity following the Learning to Run a Power Network (L2RPN) competitions. Graph neural networks (GNNs) are a class of ML models that reflect graph structure in their computation, which makes them suitable for power grid modeling. Various GNN approaches for topology control have thus been proposed. We propose the first GNN model for grid topology control that uses only GNN layers. Additionally, we identify the busbar information asymmetry problem that the popular homogeneous graph representation suffers from, and propose a heterogeneous graph representation to resolve it. We train both homogeneous and heterogeneous GNNs and fully connected neural networks (FCNN) baselines on an imitation learning task. We evaluate the models according to their classification accuracy and grid operation ability. We find that the heterogeneous GNNs perform best on in-distribution networks, followed by the FCNNs, and lastly, the homogeneous GNNs. We also find that both GNN types generalize better to out-of-distribution networks than FCNNs.

Title: A4O: All Trigger for One sample

• Authors: Duc Anh Vu, Anh Tuan Tran, Cong Tran, Cuong Pham
• Subjects: cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07192
• Pdf URL: https://arxiv.org/pdf/2501.07192
• Copy Paste: [[2501.07192]] A4O: All Trigger for One sample(https://arxiv.org/abs/2501.07192)
• Keywords: defense, attack, steal
• Abstract: Backdoor attacks have become a critical threat to deep neural networks (DNNs), drawing many research interests. However, most of the studied attacks employ a single type of trigger. Consequently, proposed backdoor defenders often rely on the assumption that triggers would appear in a unified way. In this paper, we show that this naive assumption can create a loophole, allowing more sophisticated backdoor attacks to bypass. We design a novel backdoor attack mechanism that incorporates multiple types of backdoor triggers, focusing on stealthiness and effectiveness. Our journey begins with the intriguing observation that the performance of a backdoor attack in deep learning models, as well as its detectability and removability, are all proportional to the magnitude of the trigger. Based on this correlation, we propose reducing the magnitude of each trigger type and combining them to achieve a strong backdoor relying on the combined trigger while still staying safely under the radar of defenders. Extensive experiments on three standard datasets demonstrate that our method can achieve high attack success rates (ASRs) while consistently bypassing state-of-the-art defenses.

Title: An Enhanced Zeroth-Order Stochastic Frank-Wolfe Framework for Constrained Finite-Sum Optimization

• Authors: Haishan Ye, Yinghui Huang, Hao Di, Xiangyu Chang
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/2501.07201
• Pdf URL: https://arxiv.org/pdf/2501.07201
• Copy Paste: [[2501.07201]] An Enhanced Zeroth-Order Stochastic Frank-Wolfe Framework for Constrained Finite-Sum Optimization(https://arxiv.org/abs/2501.07201)
• Keywords: attack, robust
• Abstract: We propose an enhanced zeroth-order stochastic Frank-Wolfe framework to address constrained finite-sum optimization problems, a structure prevalent in large-scale machine-learning applications. Our method introduces a novel double variance reduction framework that effectively reduces the gradient approximation variance induced by zeroth-order oracles and the stochastic sampling variance from finite-sum objectives. By leveraging this framework, our algorithm achieves significant improvements in query efficiency, making it particularly well-suited for high-dimensional optimization tasks. Specifically, for convex objectives, the algorithm achieves a query complexity of O(d \sqrt{n}/\epsilon ) to find an epsilon-suboptimal solution, where d is the dimensionality and n is the number of functions in the finite-sum objective. For non-convex objectives, it achieves a query complexity of O(d^{3/2}\sqrt{n}/\epsilon^2 ) without requiring the computation ofd partial derivatives at each iteration. These complexities are the best known among zeroth-order stochastic Frank-Wolfe algorithms that avoid explicit gradient calculations. Empirical experiments on convex and non-convex machine learning tasks, including sparse logistic regression, robust classification, and adversarial attacks on deep networks, validate the computational efficiency and scalability of our approach. Our algorithm demonstrates superior performance in both convergence rate and query complexity compared to existing methods.

Title: A data-driven approach to discover and quantify systemic lupus erythematosus etiological heterogeneity from electronic health records

• Authors: Marco Barbero Mota, John M. Still, Jorge L. Gamboa, Eric V. Strobl, Charles M. Stein, Vivian K. Kawai, Thomas A. Lasko
• Subjects: cs.LG, stat.AP
• Abstract URL: https://arxiv.org/abs/2501.07206
• Pdf URL: https://arxiv.org/pdf/2501.07206
• Copy Paste: [[2501.07206]] A data-driven approach to discover and quantify systemic lupus erythematosus etiological heterogeneity from electronic health records(https://arxiv.org/abs/2501.07206)
• Keywords: interpretability
• Abstract: Systemic lupus erythematosus (SLE) is a complex heterogeneous disease with many manifestational facets. We propose a data-driven approach to discover probabilistic independent sources from multimodal imperfect EHR data. These sources represent exogenous variables in the data generation process causal graph that estimate latent root causes of the presence of SLE in the health record. We objectively evaluated the sources against the original variables from which they were discovered by training supervised models to discriminate SLE from negative health records using a reduced set of labelled instances. We found 19 predictive sources with high clinical validity and whose EHR signatures define independent factors of SLE heterogeneity. Using the sources as input patient data representation enables models to provide with rich explanations that better capture the clinical reasons why a particular record is (not) an SLE case. Providers may be willing to trade patient-level interpretability for discrimination especially in challenging cases.

Title: Beyond Security-by-design: Securing a compromised system

• Authors: Awais Rashid, Sana Belguith, Matthew Bradbury, Sadie Creese, Ivan Flechais, Neeraj Suri
• Subjects: cs.CR, cs.DC, cs.HC, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.07207
• Pdf URL: https://arxiv.org/pdf/2501.07207
• Copy Paste: [[2501.07207]] Beyond Security-by-design: Securing a compromised system(https://arxiv.org/abs/2501.07207)
• Keywords: secure, security
• Abstract: Digital infrastructures are seeing convergence and connectivity at unprecedented scale. This is true for both current critical national infrastructures and emerging future systems that are highly cyber-physical in nature with complex intersections between humans and technologies, e.g., smart cities, intelligent transportation, high-value manufacturing and Industry 4.0. Diverse legacy and non-legacy software systems underpinned by heterogeneous hardware compose on-the-fly to deliver services to millions of users with varying requirements and unpredictable actions. This complexity is compounded by intricate and complicated supply-chains with many digital assets and services outsourced to third parties. The reality is that, at any particular point in time, there will be untrusted, partially-trusted or compromised elements across the infrastructure. Given this reality, and the societal scale of digital infrastructures, delivering secure and resilient operations is a major challenge. We argue that this requires us to move beyond the paradigm of security-by-design and embrace the challenge of securing-a-compromised-system.

Title: A Secure Remote Password Protocol From The Learning With Errors Problem

• Authors: Huapeng Li, Baocheng Wang
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07208
• Pdf URL: https://arxiv.org/pdf/2501.07208
• Copy Paste: [[2501.07208]] A Secure Remote Password Protocol From The Learning With Errors Problem(https://arxiv.org/abs/2501.07208)
• Keywords: secure, security, attack
• Abstract: Secure Remote Password (SRP) protocol is an essential password-authenticated key exchange (PAKE) protocol based on the discrete logarithm problem (DLP). The protocol is specifically designed to obtain a session key and it has been widely used in various scenarios due to its attractive security features. In the SRP protocol, the server is not required to save any data directly associated with passwords. And this makes attackers who manage to corrupt the server fail to impersonate the client unless performing a brute-force search for the password. However, the development of quantum computing has potentially made classic DLP-based public-key cryptography schemes not secure, including the SRP protocol. So it is significant to design a quantum-resistant SRP protocol. In this paper, based on the original scheme, we propose a post-quantum SRP protocol from the learning with errors (LWE) problem. And we give rigorous proof and analyses on the correctness and security of the scheme. Besides being resistant to known quantum attacks, it maintains the various secure qualities of the original protocol.

Title: Privacy-Preserving Authentication: Theory vs. Practice

• Authors: Daniel Slamanig (Research Institute CODE, Universität der Bundeswehr München)
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07209
• Pdf URL: https://arxiv.org/pdf/2501.07209
• Copy Paste: [[2501.07209]] Privacy-Preserving Authentication: Theory vs. Practice(https://arxiv.org/abs/2501.07209)
• Keywords: privacy, protect
• Abstract: With the increasing use of online services, the protection of the privacy of users becomes more and more important. This is particularly critical as authentication and authorization as realized on the Internet nowadays, typically relies on centralized identity management solutions. Although those are very convenient from a user's perspective, they are quite intrusive from a privacy perspective and are currently far from implementing the concept of data minimization. Fortunately, cryptography offers exciting primitives such as zero-knowledge proofs and advanced signature schemes to realize various forms of so-called anonymous credentials. Such primitives allow to realize online authentication and authorization with a high level of built-in privacy protection (what we call privacy-preserving authentication). Though these primitives have already been researched for various decades and are well understood in the research community, unfortunately, they lack widespread adoption. In this paper, we look at the problems, what cryptography can do, some deployment examples, and barriers to widespread adoption. Latter using the example of the EU Digital Identity Wallet (EUDIW) and the recent discussion and feedback from cryptography experts around this topic. We also briefly comment on the transition to post-quantum cryptography.

Title: TimeLogic: A Temporal Logic Benchmark for Video QA

• Authors: Sirnam Swetha, Hilde Kuehne, Mubarak Shah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07214
• Pdf URL: https://arxiv.org/pdf/2501.07214
• Copy Paste: [[2501.07214]] TimeLogic: A Temporal Logic Benchmark for Video QA(https://arxiv.org/abs/2501.07214)
• Keywords: segmentation
• Abstract: Temporal logical understanding, a core facet of human cognition, plays a pivotal role in capturing complex sequential events and their temporal relationships within videos. This capability is particularly crucial in tasks like Video Question Answering (VideoQA), where the goal is to process visual data over time together with textual data to provide coherent answers. However, current VideoQA benchmarks devote little focus to evaluating this critical skill due to the challenge of annotating temporal logic. Despite the advancement of vision-language models, assessing their temporal logical reasoning powers remains a challenge, primarily due to the lack QA pairs that demand formal, complex temporal reasoning. To bridge this gap, we introduce the TimeLogic QA (TLQA) framework to automatically generate the QA pairs, specifically designed to evaluate the temporal logical understanding. To this end, TLQA leverages temporal annotations from existing video datasets together with temporal operators derived from logic theory to construct questions that test understanding of event sequences and their temporal relationships. TLQA framework is generic and scalable, capable of leveraging both, existing video action datasets with temporal action segmentation annotations, or video datasets with temporal scene graph annotations, to automatically generate temporal logical questions. We leverage 4 datasets, STAR, Breakfast, AGQA, and CrossTask, and generate two VideoQA dataset variants - small (TLQA-S) and large (TLQA-L) - containing 2k and 10k QA pairs for each category, resulting in 32k and 160k total pairs per dataset. We undertake a comprehensive evaluation of leading-edge VideoQA models, employing the TLQA to benchmark their temporal logical understanding capabilities. We assess the VideoQA model's temporal reasoning performance on 16 categories of temporal logic with varying temporal complexity.

Title: When lies are mostly truthful: automated verbal deception detection for embedded lies

• Authors: Riccardo Loconte, Bennett Kleinberg
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.07217
• Pdf URL: https://arxiv.org/pdf/2501.07217
• Copy Paste: [[2501.07217]] When lies are mostly truthful: automated verbal deception detection for embedded lies(https://arxiv.org/abs/2501.07217)
• Keywords: explainability
• Abstract: Background: Verbal deception detection research relies on narratives and commonly assumes statements as truthful or deceptive. A more realistic perspective acknowledges that the veracity of statements exists on a continuum with truthful and deceptive parts being embedded within the same statement. However, research on embedded lies has been lagging behind. Methods: We collected a novel dataset of 2,088 truthful and deceptive statements with annotated embedded lies. Using a within-subjects design, participants provided a truthful account of an autobiographical event. They then rewrote their statement in a deceptive manner by including embedded lies, which they highlighted afterwards and judged on lie centrality, deceptiveness, and source. Results: We show that a fined-tuned language model (Llama-3-8B) can classify truthful statements and those containing embedded lies with 64% accuracy. Individual differences, linguistic properties and explainability analysis suggest that the challenge of moving the dial towards embedded lies stems from their resemblance to truthful statements. Typical deceptive statements consisted of 2/3 truthful information and 1/3 embedded lies, largely derived from past personal experiences and with minimal linguistic differences with their truthful counterparts. Conclusion: We present this dataset as a novel resource to address this challenge and foster research on embedded lies in verbal deception detection.

Title: MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning

• Authors: Tieyuan Chen, Huabin Liu, Yi Wang, Yihang Chen, Tianyao He, Chaofan Gan, Huanyu He, Weiyao Lin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07227
• Pdf URL: https://arxiv.org/pdf/2501.07227
• Copy Paste: [[2501.07227]] MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning(https://arxiv.org/abs/2501.07227)
• Keywords: robust
• Abstract: Video causal reasoning aims to achieve a high-level understanding of videos from a causal perspective. However, it exhibits limitations in its scope, primarily executed in a question-answering paradigm and focusing on brief video segments containing isolated events and basic causal relations, lacking comprehensive and structured causality analysis for videos with multiple interconnected events. To fill this gap, we introduce a new task and dataset, Multi-Event Causal Discovery (MECD). It aims to uncover the causal relations between events distributed chronologically across long videos. Given visual segments and textual descriptions of events, MECD identifies the causal associations between these events to derive a comprehensive and structured event-level video causal graph explaining why and how the result event occurred. To address the challenges of MECD, we devise a novel framework inspired by the Granger Causality method, incorporating an efficient mask-based event prediction model to perform an Event Granger Test. It estimates causality by comparing the predicted result event when premise events are masked versus unmasked. Furthermore, we integrate causal inference techniques such as front-door adjustment and counterfactual inference to mitigate challenges in MECD like causality confounding and illusory causality. Additionally, context chain reasoning is introduced to conduct more robust and generalized reasoning. Experiments validate the effectiveness of our framework in reasoning complete causal relations, outperforming GPT-4o and VideoChat2 by 5.77% and 2.70%, respectively. Further experiments demonstrate that causal relation graphs can also contribute to downstream video understanding tasks such as video question answering and video event prediction.

Title: Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training

• Authors: Ziqing Wen, Ping Luo, Jiahuan Wang, Xiaoge Deng, Jinping Zou, Kun Yuan, Tao Sun, Dongsheng Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07237
• Pdf URL: https://arxiv.org/pdf/2501.07237
• Copy Paste: [[2501.07237]] Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training(https://arxiv.org/abs/2501.07237)
• Keywords: large language model
• Abstract: Large language models (LLMs) have shown impressive performance across a range of natural language processing tasks. However, their vast number of parameters introduces significant memory challenges during training, particularly when using memory-intensive optimizers like Adam. Existing memory-efficient algorithms often rely on techniques such as singular value decomposition projection or weight freezing. While these approaches help alleviate memory constraints, they generally produce suboptimal results compared to full-rank updates. In this paper, we investigate the memory-efficient method beyond low-rank training, proposing a novel solution called Gradient Wavelet Transform (GWT), which applies wavelet transforms to gradients in order to significantly reduce the memory requirements for maintaining optimizer states. We demonstrate that GWT can be seamlessly integrated with memory-intensive optimizers, enabling efficient training without sacrificing performance. Through extensive experiments on both pre-training and fine-tuning tasks, we show that GWT achieves state-of-the-art performance compared with advanced memory-efficient optimizers and full-rank approaches in terms of both memory usage and training performance.

Title: Depth and Image Fusion for Road Obstacle Detection Using Stereo Camera

• Authors: Oleg Perezyabov, Mikhail Gavrilenkov, Ilya Afanasyev
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2501.07245
• Pdf URL: https://arxiv.org/pdf/2501.07245
• Copy Paste: [[2501.07245]] Depth and Image Fusion for Road Obstacle Detection Using Stereo Camera(https://arxiv.org/abs/2501.07245)
• Keywords: segmentation
• Abstract: This paper is devoted to the detection of objects on a road, performed with a combination of two methods based on both the use of depth information and video analysis of data from a stereo camera. Since neither the time of the appearance of an object on the road, nor its size and shape is known in advance, ML/DL-based approaches are not applicable. The task becomes more complicated due to variations in artificial illumination, inhomogeneous road surface texture, and unknown character and features of the object. To solve this problem we developed the depth and image fusion method that complements a search of small contrast objects by RGB-based method, and obstacle detection by stereo image-based approach with SLIC superpixel segmentation. We conducted experiments with static and low speed obstacles in an underground parking lot and demonstrated the successful work of the developed technique for detecting and even tracking small objects, which can be parking infrastructure objects, things left on the road, wheels, dropped boxes, etc.

Title: MOS-Attack: A Scalable Multi-objective Adversarial Attack Framework

• Authors: Ping Guo, Cheng Gong, Xi Lin, Fei Liu, Zhichao Lu, Qingfu Zhang, Zhenkun Wang
• Subjects: cs.LG, cs.AI, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07251
• Pdf URL: https://arxiv.org/pdf/2501.07251
• Copy Paste: [[2501.07251]] MOS-Attack: A Scalable Multi-objective Adversarial Attack Framework(https://arxiv.org/abs/2501.07251)
• Keywords: attack, robust
• Abstract: Crafting adversarial examples is crucial for evaluating and enhancing the robustness of Deep Neural Networks (DNNs), presenting a challenge equivalent to maximizing a non-differentiable 0-1 loss function. However, existing single objective methods, namely adversarial attacks focus on a surrogate loss function, do not fully harness the benefits of engaging multiple loss functions, as a result of insufficient understanding of their synergistic and conflicting nature. To overcome these limitations, we propose the Multi-Objective Set-based Attack (MOS Attack), a novel adversarial attack framework leveraging multiple loss functions and automatically uncovering their interrelations. The MOS Attack adopts a set-based multi-objective optimization strategy, enabling the incorporation of numerous loss functions without additional parameters. It also automatically mines synergistic patterns among various losses, facilitating the generation of potent adversarial attacks with fewer objectives. Extensive experiments have shown that our MOS Attack outperforms single-objective attacks. Furthermore, by harnessing the identified synergistic patterns, MOS Attack continues to show superior results with a reduced number of loss functions.

Title: EdgeTAM: On-Device Track Anything Model

• Authors: Chong Zhou, Chenchen Zhu, Yunyang Xiong, Saksham Suri, Fanyi Xiao, Lemeng Wu, Raghuraman Krishnamoorthi, Bo Dai, Chen Change Loy, Vikas Chandra, Bilge Soran
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07256
• Pdf URL: https://arxiv.org/pdf/2501.07256
• Copy Paste: [[2501.07256]] EdgeTAM: On-Device Track Anything Model(https://arxiv.org/abs/2501.07256)
• Keywords: transformer, segmentation
• Abstract: On top of Segment Anything Model (SAM), SAM 2 further extends its capability from image to video inputs through a memory bank mechanism and obtains a remarkable performance compared with previous methods, making it a foundation model for video segmentation task. In this paper, we aim at making SAM 2 much more efficient so that it even runs on mobile devices while maintaining a comparable performance. Despite several works optimizing SAM for better efficiency, we find they are not sufficient for SAM 2 because they all focus on compressing the image encoder, while our benchmark shows that the newly introduced memory attention blocks are also the latency bottleneck. Given this observation, we propose EdgeTAM, which leverages a novel 2D Spatial Perceiver to reduce the computational cost. In particular, the proposed 2D Spatial Perceiver encodes the densely stored frame-level memories with a lightweight Transformer that contains a fixed set of learnable queries. Given that video segmentation is a dense prediction task, we find preserving the spatial structure of the memories is essential so that the queries are split into global-level and patch-level groups. We also propose a distillation pipeline that further improves the performance without inference overhead. As a result, EdgeTAM achieves 87.7, 70.0, 72.3, and 71.7 J&F on DAVIS 2017, MOSE, SA-V val, and SA-V test, while running at 16 FPS on iPhone 15 Pro Max.

Title: Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion

• Authors: Li Liang, Naveed Akhtar, Jordan Vice, Xiangrui Kong, Ajmal Saeed Mian
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07260
• Pdf URL: https://arxiv.org/pdf/2501.07260
• Copy Paste: [[2501.07260]] Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion(https://arxiv.org/abs/2501.07260)
• Keywords: diffusion, generative, segmentation
• Abstract: 3D semantic scene completion is critical for multiple downstream tasks in autonomous systems. It estimates missing geometric and semantic information in the acquired scene data. Due to the challenging real-world conditions, this task usually demands complex models that process multi-modal data to achieve acceptable performance. We propose a unique neural model, leveraging advances from the state space and diffusion generative modeling to achieve remarkable 3D semantic scene completion performance with monocular image input. Our technique processes the data in the conditioned latent space of a variational autoencoder where diffusion modeling is carried out with an innovative state space technique. A key component of our neural network is the proposed Skimba (Skip Mamba) denoiser, which is adept at efficiently processing long-sequence data. The Skimba diffusion model is integral to our 3D scene completion network, incorporating a triple Mamba structure, dimensional decomposition residuals and varying dilations along three directions. We also adopt a variant of this network for the subsequent semantic segmentation stage of our method. Extensive evaluation on the standard SemanticKITTI and SSCBench-KITTI360 datasets show that our approach not only outperforms other monocular techniques by a large margin, it also achieves competitive performance against stereo methods. The code is available at this https URL

Title: OblivCDN: A Practical Privacy-preserving CDN with Oblivious Content Access

• Authors: Viet Vo, Shangqi Lai, Xingliang Yuan, Surya Nepal, Qi Li
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07262
• Pdf URL: https://arxiv.org/pdf/2501.07262
• Copy Paste: [[2501.07262]] OblivCDN: A Practical Privacy-preserving CDN with Oblivious Content Access(https://arxiv.org/abs/2501.07262)
• Keywords: security, privacy
• Abstract: Content providers increasingly utilise Content Delivery Networks (CDNs) to enhance users' content download experience. However, this deployment scenario raises significant security concerns regarding content confidentiality and user privacy due to the involvement of third-party providers. Prior proposals using private information retrieval (PIR) and oblivious RAM (ORAM) have proven impractical due to high computation and communication costs, as well as integration challenges within distributed CDN architectures. In response, we present \textsf{OblivCDN}, a practical privacy-preserving system meticulously designed for seamless integration with the existing real-world Internet-CDN infrastructure. Our design strategically adapts Range ORAM primitives to optimise memory and disk seeks when accessing contiguous blocks of CDN content, both at the origin and edge servers, while preserving both content confidentiality and user access pattern hiding features. Also, we carefully customise several oblivious building blocks that integrate the distributed trust model into the ORAM client, thereby eliminating the computational bottleneck in the origin server and reducing communication costs between the origin server and edge servers. Moreover, the newly-designed ORAM client also eliminates the need for trusted hardware on edge servers, and thus significantly ameliorates the compatibility towards networks with massive legacy this http URL real-world streaming evaluations, OblivCDN} demonstrates remarkable performance, downloading a $256$ MB video in just $5.6$ seconds. This achievement represents a speedup of $90\times$ compared to a strawman approach (direct ORAM adoption) and a $366\times$ improvement over the prior art, OblivP2P.

Title: Generating Poisoning Attacks against Ridge Regression Models with Categorical Features

• Authors: Monse Guedes-Ayala, Lars Schewe, Zeynep Suvak, Miguel Anjos
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2501.07275
• Pdf URL: https://arxiv.org/pdf/2501.07275
• Copy Paste: [[2501.07275]] Generating Poisoning Attacks against Ridge Regression Models with Categorical Features(https://arxiv.org/abs/2501.07275)
• Keywords: attack
• Abstract: Machine Learning (ML) models have become a very powerful tool to extract information from large datasets and use it to make accurate predictions and automated decisions. However, ML models can be vulnerable to external attacks, causing them to underperform or deviate from their expected tasks. One way to attack ML models is by injecting malicious data to mislead the algorithm during the training phase, which is referred to as a poisoning attack. We can prepare for such situations by designing anticipated attacks, which are later used for creating and testing defence strategies. In this paper, we propose an algorithm to generate strong poisoning attacks for a ridge regression model containing both numerical and categorical features that explicitly models and poisons categorical features. We model categorical features as SOS-1 sets and formulate the problem of designing poisoning attacks as a bilevel optimization problem that is nonconvex mixed-integer in the upper-level and unconstrained convex quadratic in the lower-level. We present the mathematical formulation of the problem, introduce a single-level reformulation based on the Karush-Kuhn-Tucker (KKT) conditions of the lower level, find bounds for the lower-level variables to accelerate solver performance, and propose a new algorithm to poison categorical features. Numerical experiments show that our method improves the mean squared error of all datasets compared to the previous benchmark in the literature.

Title: Event-based Video Person Re-identification via Cross-Modality and Temporal Collaboration

• Authors: Renkai Li, Xin Yuan, Wei Liu, Xin Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07296
• Pdf URL: https://arxiv.org/pdf/2501.07296
• Copy Paste: [[2501.07296]] Event-based Video Person Re-identification via Cross-Modality and Temporal Collaboration(https://arxiv.org/abs/2501.07296)
• Keywords: privacy, attack
• Abstract: Video-based person re-identification (ReID) has become increasingly important due to its applications in video surveillance applications. By employing events in video-based person ReID, more motion information can be provided between continuous frames to improve recognition accuracy. Previous approaches have assisted by introducing event data into the video person ReID task, but they still cannot avoid the privacy leakage problem caused by RGB images. In order to avoid privacy attacks and to take advantage of the benefits of event data, we consider using only event data. To make full use of the information in the event stream, we propose a Cross-Modality and Temporal Collaboration (CMTC) network for event-based video person ReID. First, we design an event transform network to obtain corresponding auxiliary information from the input of raw events. Additionally, we propose a differential modality collaboration module to balance the roles of events and auxiliaries to achieve complementary effects. Furthermore, we introduce a temporal collaboration module to exploit motion information and appearance cues. Experimental results demonstrate that our method outperforms others in the task of event-based video person ReID.

Title: Toward Realistic Camouflaged Object Detection: Benchmarks and Method

• Authors: Zhimeng Xin, Tianxu Wu, Shiming Chen, Shuo Ye, Zijing Xie, Yixiong Zou, Xinge You, Yufei Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07297
• Pdf URL: https://arxiv.org/pdf/2501.07297
• Copy Paste: [[2501.07297]] Toward Realistic Camouflaged Object Detection: Benchmarks and Method(https://arxiv.org/abs/2501.07297)
• Keywords: transformer, segmentation
• Abstract: Camouflaged object detection (COD) primarily relies on semantic or instance segmentation methods. While these methods have made significant advancements in identifying the contours of camouflaged objects, they may be inefficient or cost-effective for tasks that only require the specific location of the object. Object detection algorithms offer an optimized solution for Realistic Camouflaged Object Detection (RCOD) in such cases. However, detecting camouflaged objects remains a formidable challenge due to the high degree of similarity between the features of the objects and their backgrounds. Unlike segmentation methods that perform pixel-wise comparisons to differentiate between foreground and background, object detectors omit this analysis, further aggravating the challenge. To solve this problem, we propose a camouflage-aware feature refinement (CAFR) strategy. Since camouflaged objects are not rare categories, CAFR fully utilizes a clear perception of the current object within the prior knowledge of large models to assist detectors in deeply understanding the distinctions between background and foreground. Specifically, in CAFR, we introduce the Adaptive Gradient Propagation (AGP) module that fine-tunes all feature extractor layers in large detection models to fully refine class-specific features from camouflaged contexts. We then design the Sparse Feature Refinement (SFR) module that optimizes the transformer-based feature extractor to focus primarily on capturing class-specific features in camouflaged scenarios. To facilitate the assessment of RCOD tasks, we manually annotate the labels required for detection on three existing segmentation COD datasets, creating a new benchmark for RCOD tasks. Code and datasets are available at: this https URL.

Title: The Lessons of Developing Process Reward Models in Mathematical Reasoning

• Authors: Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07301
• Pdf URL: https://arxiv.org/pdf/2501.07301
• Copy Paste: [[2501.07301]] The Lessons of Developing Process Reward Models in Mathematical Reasoning(https://arxiv.org/abs/2501.07301)
• Keywords: large language model
• Abstract: Process Reward Models (PRMs) emerge as a promising approach for process supervision in mathematical reasoning of Large Language Models (LLMs), which aim to identify and mitigate intermediate errors in the reasoning processes. However, the development of effective PRMs faces significant challenges, particularly in data annotation and evaluation methodologies. In this paper, through extensive experiments, we demonstrate that commonly used Monte Carlo (MC) estimation-based data synthesis for PRMs typically yields inferior performance and generalization compared to LLM-as-a-judge and human annotation methods. MC estimation relies on completion models to evaluate current-step correctness, leading to inaccurate step verification. Furthermore, we identify potential biases in conventional Best-of-N (BoN) evaluation strategies for PRMs: (1) The unreliable policy models generate responses with correct answers but flawed processes, leading to a misalignment between the evaluation criteria of BoN and the PRM objectives of process verification. (2) The tolerance of PRMs of such responses leads to inflated BoN scores. (3) Existing PRMs have a significant proportion of minimum scores concentrated on the final answer steps, revealing the shift from process to outcome-based assessment in BoN Optimized PRMs. To address these challenges, we develop a consensus filtering mechanism that effectively integrates MC estimation with LLM-as-a-judge and advocates a more comprehensive evaluation framework that combines response-level and step-level metrics. Based on the mechanisms, we significantly improve both model performance and data efficiency in the BoN evaluation and the step-wise error identification task. Finally, we release a new state-of-the-art PRM that outperforms existing open-source alternatives and provides practical guidelines for future research in building process supervision models.

Title: Code and Pixels: Multi-Modal Contrastive Pre-training for Enhanced Tabular Data Analysis

• Authors: Kankana Roy, Lars Krämer, Sebastian Domaschke, Malik Haris, Roland Aydin, Fabian Isensee, Martin Held
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07304
• Pdf URL: https://arxiv.org/pdf/2501.07304
• Copy Paste: [[2501.07304]] Code and Pixels: Multi-Modal Contrastive Pre-training for Enhanced Tabular Data Analysis(https://arxiv.org/abs/2501.07304)
• Keywords: robust
• Abstract: Learning from tabular data is of paramount importance, as it complements the conventional analysis of image and video data by providing a rich source of structured information that is often critical for comprehensive understanding and decision-making processes. We present Multi-task Contrastive Masked Tabular Modeling (MT-CMTM), a novel method aiming to enhance tabular models by leveraging the correlation between tabular data and corresponding images. MT-CMTM employs a dual strategy combining contrastive learning with masked tabular modeling, optimizing the synergy between these data modalities. Central to our approach is a 1D Convolutional Neural Network with residual connections and an attention mechanism (1D-ResNet-CBAM), designed to efficiently process tabular data without relying on images. This enables MT-CMTM to handle purely tabular data for downstream tasks, eliminating the need for potentially costly image acquisition and processing. We evaluated MT-CMTM on the DVM car dataset, which is uniquely suited for this particular scenario, and the newly developed HIPMP dataset, which connects membrane fabrication parameters with image data. Our MT-CMTM model outperforms the proposed tabular 1D-ResNet-CBAM, which is trained from scratch, achieving a relative 1.48% improvement in relative MSE on HIPMP and a 2.38% increase in absolute accuracy on DVM. These results demonstrate MT-CMTM's robustness and its potential to advance the field of multi-modal learning.

Title: The Devil is in the Spurious Correlation: Boosting Moment Retrieval via Temporal Dynamic Learning

• Authors: Xinyang Zhou, Fanyue Wei, Lixin Duan, Wen Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07305
• Pdf URL: https://arxiv.org/pdf/2501.07305
• Copy Paste: [[2501.07305]] The Devil is in the Spurious Correlation: Boosting Moment Retrieval via Temporal Dynamic Learning(https://arxiv.org/abs/2501.07305)
• Keywords: transformer
• Abstract: Given a textual query along with a corresponding video, the objective of moment retrieval aims to localize the moments relevant to the query within the video. While commendable results have been demonstrated by existing transformer-based approaches, predicting the accurate temporal span of the target moment is currently still a major challenge. In this paper, we reveal that a crucial reason stems from the spurious correlation between the text queries and the moment context. Namely, the model may associate the textual query with the background frames rather than the target moment. To address this issue, we propose a temporal dynamic learning approach for moment retrieval, where two strategies are designed to mitigate the spurious correlation. First, we introduce a novel video synthesis approach to construct a dynamic context for the relevant moment. With separate yet similar videos mixed up, the synthesis approach empowers our model to attend to the target moment of the corresponding query under various dynamic contexts. Second, we enhance the representation by learning temporal dynamics. Besides the visual representation, text queries are aligned with temporal dynamic representations, which enables our model to establish a non-spurious correlation between the query-related moment and context. With the aforementioned proposed method, the spurious correlation issue in moment retrieval can be largely alleviated. Our method establishes a new state-of-the-art performance on two popular benchmarks of moment retrieval, \ie, QVHighlights and Charades-STA. In addition, the detailed ablation analyses demonstrate the effectiveness of the proposed strategies. Our code will be publicly available.

Title: Localization-Aware Multi-Scale Representation Learning for Repetitive Action Counting

• Authors: Sujia Wang, Xiangwei Shen, Yansong Tang, Xin Dong, Wenjia Geng, Lei Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07312
• Pdf URL: https://arxiv.org/pdf/2501.07312
• Copy Paste: [[2501.07312]] Localization-Aware Multi-Scale Representation Learning for Repetitive Action Counting(https://arxiv.org/abs/2501.07312)
• Keywords: robust
• Abstract: Repetitive action counting (RAC) aims to estimate the number of class-agnostic action occurrences in a video without exemplars. Most current RAC methods rely on a raw frame-to-frame similarity representation for period prediction. However, this approach can be significantly disrupted by common noise such as action interruptions and inconsistencies, leading to sub-optimal counting performance in realistic scenarios. In this paper, we introduce a foreground localization optimization objective into similarity representation learning to obtain more robust and efficient video features. We propose a Localization-Aware Multi-Scale Representation Learning (LMRL) framework. Specifically, we apply a Multi-Scale Period-Aware Representation (MPR) with a scale-specific design to accommodate various action frequencies and learn more flexible temporal correlations. Furthermore, we introduce the Repetition Foreground Localization (RFL) method, which enhances the representation by coarsely identifying periodic actions and incorporating global semantic information. These two modules can be jointly optimized, resulting in a more discerning periodic action representation. Our approach significantly reduces the impact of noise, thereby improving counting accuracy. Additionally, the framework is designed to be scalable and adaptable to different types of video content. Experimental results on the RepCountA and UCFRep datasets demonstrate that our proposed method effectively handles repetitive action counting.

Title: FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering

• Authors: Erik Henriksson, Otto Tarkka, Filip Ginter
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.07314
• Pdf URL: https://arxiv.org/pdf/2501.07314
• Copy Paste: [[2501.07314]] FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering(https://arxiv.org/abs/2501.07314)
• Keywords: large language model
• Abstract: Data quality is crucial for training Large Language Models (LLMs). Traditional heuristic filters often miss low-quality text or mistakenly remove valuable content. In this paper, we introduce an LLM-based line-level filtering method to enhance training data quality. We use GPT-4o mini to label a 20,000-document sample from FineWeb at the line level, allowing the model to create descriptive labels for low-quality lines. These labels are grouped into nine main categories, and we train a DeBERTa-v3 classifier to scale the filtering to a 10B-token subset of FineWeb. To test the impact of our filtering, we train GPT-2 models on both the original and the filtered datasets. The results show that models trained on the filtered data achieve higher accuracy on the HellaSwag benchmark and reach their performance targets faster, even with up to 25\% less data. This demonstrates that LLM-based line-level filtering can significantly improve data quality and training efficiency for LLMs. We release our quality-annotated dataset, FinerWeb-10BT, and the codebase to support further work in this area.

Title: Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring

• Authors: Buse Sibel Korkmaz, Rahul Nair, Elizabeth M. Daly, Evangelos Anagnostopoulos, Christos Varytimidis, Antonio del Rio Chanona
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07324
• Pdf URL: https://arxiv.org/pdf/2501.07324
• Copy Paste: [[2501.07324]] Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring(https://arxiv.org/abs/2501.07324)
• Keywords: fair, generative, large language model
• Abstract: Foundation models require fine-tuning to ensure their generative outputs align with intended results for specific tasks. Automating this fine-tuning process is challenging, as it typically needs human feedback that can be expensive to acquire. We present AutoRefine, a method that leverages reinforcement learning for targeted fine-tuning, utilizing direct feedback from measurable performance improvements in specific downstream tasks. We demonstrate the method for a problem arising in algorithmic hiring platforms where linguistic biases influence a recommendation system. In this setting, a generative model seeks to rewrite given job specifications to receive more diverse candidate matches from a recommendation engine which matches jobs to candidates. Our model detects and regulates biases in job descriptions to meet diversity and fairness criteria. The experiments on a public hiring dataset and a real-world hiring platform showcase how large language models can assist in identifying and mitigation biases in the real world.

Title: Am I Infected? Lessons from Operating a Large-Scale IoT Security Diagnostic Service

• Authors: Takayuki Sasaki, Tomoya Inazawa, Youhei Yamaguchi, Simon Parkin, Michel van Eeten, Katsunari Yoshioka, Tsutomu Matsumoto
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07326
• Pdf URL: https://arxiv.org/pdf/2501.07326
• Copy Paste: [[2501.07326]] Am I Infected? Lessons from Operating a Large-Scale IoT Security Diagnostic Service(https://arxiv.org/abs/2501.07326)
• Keywords: secure, security
• Abstract: There is an expectation that users of home IoT devices will be able to secure those devices, but they may lack information about what they need to do. In February 2022, we launched a web service that scans users' IoT devices to determine how secure they are. The service aims to diagnose and remediate vulnerabilities and malware infections of IoT devices of Japanese users. This paper reports on findings from operating this service drawn from three studies: (1) the engagement of 114,747 users between February, 2022 - May, 2024; (2) a large-scale evaluation survey among service users (n=4,103), and; (3) an investigation and targeted survey (n=90) around the remediation actions of users of non-secure devices. During the operation, we notified 417 (0.36%) users that one or more of their devices were detected as vulnerable, and 171 (0.15%) users that one of their devices was infected with malware. The service found no issues for 99% of users. Still, 96% of all users evaluated the service positively, most often for it providing reassurance, being free of charge, and short diagnosis time. Of the 171 users with malware infections, 67 returned to the service later for a new check, with 59 showing improvement. Of the 417 users with vulnerable devices, 151 users revisited and re-diagnosed, where 75 showed improvement. We report on lessons learned, including a consideration of the capabilities that non-expert users will assume of a security scan.

Title: A method for estimating roadway billboard salience

• Authors: Zuzana Berger Haladova, Michal Zrubec, Zuzana Cernekova
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07342
• Pdf URL: https://arxiv.org/pdf/2501.07342
• Copy Paste: [[2501.07342]] A method for estimating roadway billboard salience(https://arxiv.org/abs/2501.07342)
• Keywords: extraction
• Abstract: Roadside billboards and other forms of outdoor advertising play a crucial role in marketing initiatives; however, they can also distract drivers, potentially contributing to accidents. This study delves into the significance of roadside advertising in images captured from a driver's perspective. Firstly, it evaluates the effectiveness of neural networks in detecting advertising along roads, focusing on the YOLOv5 and Faster R-CNN models. Secondly, the study addresses the determination of billboard significance using methods for saliency extraction. The UniSal and SpectralResidual methods were employed to create saliency maps for each image. The study establishes a database of eye tracking sessions captured during city highway driving to assess the saliency models.

Title: Deep Generative Clustering with VAEs and Expectation-Maximization

• Authors: Michael Adipoetra, Ségolène Martin
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.07358
• Pdf URL: https://arxiv.org/pdf/2501.07358
• Copy Paste: [[2501.07358]] Deep Generative Clustering with VAEs and Expectation-Maximization(https://arxiv.org/abs/2501.07358)
• Keywords: generative
• Abstract: We propose a novel deep clustering method that integrates Variational Autoencoders (VAEs) into the Expectation-Maximization (EM) framework. Our approach models the probability distribution of each cluster with a VAE and alternates between updating model parameters by maximizing the Evidence Lower Bound (ELBO) of the log-likelihood and refining cluster assignments based on the learned distributions. This enables effective clustering and generation of new samples from each cluster. Unlike existing VAE-based methods, our approach eliminates the need for a Gaussian Mixture Model (GMM) prior or additional regularization techniques. Experiments on MNIST and FashionMNIST demonstrate superior clustering performance compared to state-of-the-art methods.

Title: Emergent effects of scaling on the functional hierarchies within large language models

• Authors: Paul C. Bogdan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07359
• Pdf URL: https://arxiv.org/pdf/2501.07359
• Copy Paste: [[2501.07359]] Emergent effects of scaling on the functional hierarchies within large language models(https://arxiv.org/abs/2501.07359)
• Keywords: large language model
• Abstract: Large language model (LLM) architectures are often described as functionally hierarchical: Early layers process syntax, middle layers begin to parse semantics, and late layers integrate information. The present work revisits these ideas. This research submits simple texts to an LLM (e.g., "A church and organ") and extracts the resulting activations. Then, for each layer, support vector machines and ridge regressions are fit to predict a text's label and thus examine whether a given layer encodes some information. Analyses using a small model (Llama-3.2-3b; 28 layers) partly bolster the common hierarchical perspective: Item-level semantics are most strongly represented early (layers 2-7), then two-item relations (layers 8-12), and then four-item analogies (layers 10-15). Afterward, the representation of items and simple relations gradually decreases in deeper layers that focus on more global information. However, several findings run counter to a steady hierarchy view: First, although deep layers can represent document-wide abstractions, deep layers also compress information from early portions of the context window without meaningful abstraction. Second, when examining a larger model (Llama-3.3-70b-Instruct), stark fluctuations in abstraction level appear: As depth increases, two-item relations and four-item analogies initially increase in their representation, then markedly decrease, and afterward increase again momentarily. This peculiar pattern consistently emerges across several experiments. Third, another emergent effect of scaling is coordination between the attention mechanisms of adjacent layers. Across multiple experiments using the larger model, adjacent layers fluctuate between what information they each specialize in representing. In sum, an abstraction hierarchy often manifests across layers, but large models also deviate from this structure in curious ways.

Title: TimberVision: A Multi-Task Dataset and Framework for Log-Component Segmentation and Tracking in Autonomous Forestry Operations

• Authors: Daniel Steininger, Julia Simon, Andreas Trondl, Markus Murschitz
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07360
• Pdf URL: https://arxiv.org/pdf/2501.07360
• Copy Paste: [[2501.07360]] TimberVision: A Multi-Task Dataset and Framework for Log-Component Segmentation and Tracking in Autonomous Forestry Operations(https://arxiv.org/abs/2501.07360)
• Keywords: robust, segmentation
• Abstract: Timber represents an increasingly valuable and versatile resource. However, forestry operations such as harvesting, handling and measuring logs still require substantial human labor in remote environments posing significant safety risks. Progressively automating these tasks has the potential of increasing their efficiency as well as safety, but requires an accurate detection of individual logs as well as live trees and their context. Although initial approaches have been proposed for this challenging application domain, specialized data and algorithms are still too scarce to develop robust solutions. To mitigate this gap, we introduce the TimberVision dataset, consisting of more than 2k annotated RGB images containing a total of 51k trunk components including cut and lateral surfaces, thereby surpassing any existing dataset in this domain in terms of both quantity and detail by a large margin. Based on this data, we conduct a series of ablation experiments for oriented object detection and instance segmentation and evaluate the influence of multiple scene parameters on model performance. We introduce a generic framework to fuse the components detected by our models for both tasks into unified trunk representations. Furthermore, we automatically derive geometric properties and apply multi-object tracking to further enhance robustness. Our detection and tracking approach provides highly descriptive and accurate trunk representations solely from RGB image data, even under challenging environmental conditions. Our solution is suitable for a wide range of application scenarios and can be readily combined with other sensor modalities.

Title: Dynami-CAL GraphNet: A Physics-Informed Graph Neural Network Conserving Linear and Angular Momentum for Dynamical Systems

• Authors: Vinay Sharma, Olga Fink
• Subjects: cs.LG, cs.CE, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2501.07373
• Pdf URL: https://arxiv.org/pdf/2501.07373
• Copy Paste: [[2501.07373]] Dynami-CAL GraphNet: A Physics-Informed Graph Neural Network Conserving Linear and Angular Momentum for Dynamical Systems(https://arxiv.org/abs/2501.07373)
• Keywords: robust, interpretability
• Abstract: Accurate, interpretable, and real-time modeling of multi-body dynamical systems is essential for predicting behaviors and inferring physical properties in natural and engineered environments. Traditional physics-based models face scalability challenges and are computationally demanding, while data-driven approaches like Graph Neural Networks (GNNs) often lack physical consistency, interpretability, and generalization. In this paper, we propose Dynami-CAL GraphNet, a Physics-Informed Graph Neural Network that integrates the learning capabilities of GNNs with physics-based inductive biases to address these limitations. Dynami-CAL GraphNet enforces pairwise conservation of linear and angular momentum for interacting nodes using edge-local reference frames that are equivariant to rotational symmetries, invariant to translations, and equivariant to node permutations. This design ensures physically consistent predictions of node dynamics while offering interpretable, edge-wise linear and angular impulses resulting from pairwise interactions. Evaluated on a 3D granular system with inelastic collisions, Dynami-CAL GraphNet demonstrates stable error accumulation over extended rollouts, effective extrapolations to unseen configurations, and robust handling of heterogeneous interactions and external forces. Dynami-CAL GraphNet offers significant advantages in fields requiring accurate, interpretable, and real-time modeling of complex multi-body dynamical systems, such as robotics, aerospace engineering, and materials science. By providing physically consistent and scalable predictions that adhere to fundamental conservation laws, it enables the inference of forces and moments while efficiently handling heterogeneous interactions and external forces.

Title: FedSemiDG: Domain Generalized Federated Semi-supervised Medical Image Segmentation

• Authors: Zhipeng Deng, Zhe Xu, Tsuyoshi Isshiki, Yefeng Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07378
• Pdf URL: https://arxiv.org/pdf/2501.07378
• Copy Paste: [[2501.07378]] FedSemiDG: Domain Generalized Federated Semi-supervised Medical Image Segmentation(https://arxiv.org/abs/2501.07378)
• Keywords: robust, federate, segmentation
• Abstract: Medical image segmentation is challenging due to the diversity of medical images and the lack of labeled data, which motivates recent developments in federated semi-supervised learning (FSSL) to leverage a large amount of unlabeled data from multiple centers for model training without sharing raw data. However, what remains under-explored in FSSL is the domain shift problem which may cause suboptimal model aggregation and low effectivity of the utilization of unlabeled data, eventually leading to unsatisfactory performance in unseen domains. In this paper, we explore this previously ignored scenario, namely domain generalized federated semi-supervised learning (FedSemiDG), which aims to learn a model in a distributed manner from multiple domains with limited labeled data and abundant unlabeled data such that the model can generalize well to unseen domains. We present a novel framework, Federated Generalization-Aware SemiSupervised Learning (FGASL), to address the challenges in FedSemiDG by effectively tackling critical issues at both global and local levels. Globally, we introduce Generalization-Aware Aggregation (GAA), assigning adaptive weights to local models based on their generalization performance. Locally, we use a Dual-Teacher Adaptive Pseudo Label Refinement (DR) strategy to combine global and domain-specific knowledge, generating more reliable pseudo labels. Additionally, Perturbation-Invariant Alignment (PIA) enforces feature consistency under perturbations, promoting domain-invariant learning. Extensive experiments on three medical segmentation tasks (cardiac MRI, spine MRI and bladder cancer MRI) demonstrate that our method significantly outperforms state-of-the-art FSSL and domain generalization approaches, achieving robust generalization on unseen domains.

Title: Device-Bound vs. Synced Credentials: A Comparative Evaluation of Passkey Authentication

• Authors: Andre Büttner, Nils Gruschka
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07380
• Pdf URL: https://arxiv.org/pdf/2501.07380
• Copy Paste: [[2501.07380]] Device-Bound vs. Synced Credentials: A Comparative Evaluation of Passkey Authentication(https://arxiv.org/abs/2501.07380)
• Keywords: security
• Abstract: With passkeys, the FIDO Alliance introduces the ability to sync FIDO2 credentials across a user's devices through passkey providers. This aims to mitigate user concerns about losing their devices and promotes the shift toward password-less authentication. As a consequence, many major online services have adopted passkeys. However, credential syncing has also created a debate among experts about their security guarantees. In this paper, we categorize the different access levels of passkeys to show how syncing credentials impacts their security and availability. Moreover, we use the established framework from Bonneau et al.'s Quest to Replace Passwords and apply it to different types of device-bound and synced passkeys. By this, we reveal relevant differences, particularly in their usability and security, and show that the security of synced passkeys is mainly concentrated in the passkey provider. We further provide practical recommendations for end users, passkey providers, and relying parties.

Title: Kolmogorov-Arnold Network for Remote Sensing Image Semantic Segmentation

• Authors: Xianping Ma, Ziyao Wang, Yin Hu, Xiaokang Zhang, Man-On Pun
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07390
• Pdf URL: https://arxiv.org/pdf/2501.07390
• Copy Paste: [[2501.07390]] Kolmogorov-Arnold Network for Remote Sensing Image Semantic Segmentation(https://arxiv.org/abs/2501.07390)
• Keywords: extraction, interpretability, segmentation
• Abstract: Semantic segmentation plays a crucial role in remote sensing applications, where the accurate extraction and representation of features are essential for high-quality results. Despite the widespread use of encoder-decoder architectures, existing methods often struggle with fully utilizing the high-dimensional features extracted by the encoder and efficiently recovering detailed information during decoding. To address these problems, we propose a novel semantic segmentation network, namely DeepKANSeg, including two key innovations based on the emerging Kolmogorov Arnold Network (KAN). Notably, the advantage of KAN lies in its ability to decompose high-dimensional complex functions into univariate transformations, enabling efficient and flexible representation of intricate relationships in data. First, we introduce a KAN-based deep feature refinement module, namely DeepKAN to effectively capture complex spatial and rich semantic relationships from high-dimensional features. Second, we replace the traditional multi-layer perceptron (MLP) layers in the global-local combined decoder with KAN-based linear layers, namely GLKAN. This module enhances the decoder's ability to capture fine-grained details during decoding. To evaluate the effectiveness of the proposed method, experiments are conducted on two well-known fine-resolution remote sensing benchmark datasets, namely ISPRS Vaihingen and ISPRS Potsdam. The results demonstrate that the KAN-enhanced segmentation model achieves superior performance in terms of accuracy compared to state-of-the-art methods. They highlight the potential of KANs as a powerful alternative to traditional architectures in semantic segmentation tasks. Moreover, the explicit univariate decomposition provides improved interpretability, which is particularly beneficial for applications requiring explainable learning in remote sensing.

Title: Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models

• Authors: Yasiru Ranasinghe, Vibashan VS, James Uplinger, Celso De Melo, Vishal M. Patel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07396
• Pdf URL: https://arxiv.org/pdf/2501.07396
• Copy Paste: [[2501.07396]] Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models(https://arxiv.org/abs/2501.07396)
• Keywords: robust
• Abstract: Automatic target recognition (ATR) plays a critical role in tasks such as navigation and surveillance, where safety and accuracy are paramount. In extreme use cases, such as military applications, these factors are often challenged due to the presence of unknown terrains, environmental conditions, and novel object categories. Current object detectors, including open-world detectors, lack the ability to confidently recognize novel objects or operate in unknown environments, as they have not been exposed to these new conditions. However, Large Vision-Language Models (LVLMs) exhibit emergent properties that enable them to recognize objects in varying conditions in a zero-shot manner. Despite this, LVLMs struggle to localize objects effectively within a scene. To address these limitations, we propose a novel pipeline that combines the detection capabilities of open-world detectors with the recognition confidence of LVLMs, creating a robust system for zero-shot ATR of novel classes and unknown domains. In this study, we compare the performance of various LVLMs for recognizing military vehicles, which are often underrepresented in training datasets. Additionally, we examine the impact of factors such as distance range, modality, and prompting methods on the recognition performance, providing insights into the development of more reliable ATR systems for novel conditions and classes.

Title: OCORD: Open-Campus Object Removal Dataset

• Authors: Shuo Zhang, Runpu Wei, Kongming Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07397
• Pdf URL: https://arxiv.org/pdf/2501.07397
• Copy Paste: [[2501.07397]] OCORD: Open-Campus Object Removal Dataset(https://arxiv.org/abs/2501.07397)
• Keywords: diffusion, generative
• Abstract: The rapid advancements in generative models, particularly diffusion-based techniques, have revolutionized image inpainting tasks by enabling the generation of high-fidelity and diverse content. However, object removal remains under-explored as a specific subset of inpainting, facing challenges such as inadequate semantic understanding and the unintended generation of artifacts. Existing datasets for object removal often rely on synthetic data, which fails to align with real-world scenarios, limiting model performance. Although some real-world datasets address these issues partially, they suffer from scalability, annotation inefficiencies, and limited realism in physical phenomena such as lighting and shadows. To address these limitations, this paper introduces a novel approach to object removal by constructing a high-resolution real-world dataset through long-duration video capture with fixed camera settings. Leveraging advanced tools such as Grounding-DINO, Segment-Anything-Model, and MASA for automated annotation, we provides image, background, and mask pairs while significantly reducing annotation time and labor. With our efficient annotation pipeline, we release the first fully open, high-resolution real-world dataset for object removal, and improved performance in object removal tasks through fine-tuning of pre-trained diffusion models.

Title: Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning

• Authors: Thomas Chen
• Subjects: cs.LG, cs.AI, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.07400
• Pdf URL: https://arxiv.org/pdf/2501.07400
• Copy Paste: [[2501.07400]] Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning(https://arxiv.org/abs/2501.07400)
• Keywords: interpretability
• Abstract: We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean cost in the input layer, and under the assumption that the weights are, in a precise sense, adapted to the coordinate system distinguished by the activations. We show that gradient descent corresponds to a dynamical process in the input layer, whereby clusters of data are progressively reduced in complexity ("truncated") at an exponential rate that increases with the number of data points that have already been truncated. We provide a detailed discussion of several types of solutions to the gradient flow equations. A main motivation for this work is to shed light on the interpretability question in supervised learning.

Title: PROTECT: Protein circadian time prediction using unsupervised learning

• Authors: Aram Ansary Ogholbake, Qiang Cheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07405
• Pdf URL: https://arxiv.org/pdf/2501.07405
• Copy Paste: [[2501.07405]] PROTECT: Protein circadian time prediction using unsupervised learning(https://arxiv.org/abs/2501.07405)
• Keywords: protect, robust
• Abstract: Circadian rhythms regulate the physiology and behavior of humans and animals. Despite advancements in understanding these rhythms and predicting circadian phases at the transcriptional level, predicting circadian phases from proteomic data remains elusive. This challenge is largely due to the scarcity of time labels in proteomic datasets, which are often characterized by small sample sizes, high dimensionality, and significant noise. Furthermore, existing methods for predicting circadian phases from transcriptomic data typically rely on prior knowledge of known rhythmic genes, making them unsuitable for proteomic datasets. To address this gap, we developed a novel computational method using unsupervised deep learning techniques to predict circadian sample phases from proteomic data without requiring time labels or prior knowledge of proteins or genes. Our model involves a two-stage training process optimized for robust circadian phase prediction: an initial greedy one-layer-at-a-time pre-training which generates informative initial parameters followed by fine-tuning. During fine-tuning, a specialized loss function guides the model to align protein expression levels with circadian patterns, enabling it to accurately capture the underlying rhythmic structure within the data. We tested our method on both time-labeled and unlabeled proteomic data. For labeled data, we compared our predictions to the known time labels, achieving high accuracy, while for unlabeled human datasets, including postmortem brain regions and urine samples, we explored circadian disruptions. Notably, our analysis identified disruptions in rhythmic proteins between Alzheimer's disease and control subjects across these samples.

Title: An Investigation into Seasonal Variations in Energy Forecasting for Student Residences

• Authors: Muhammad Umair Danish, Mathumitha Sureshkumar, Thanuri Fonseka, Umeshika Uthayakumar, Vinura Galwaduge
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07423
• Pdf URL: https://arxiv.org/pdf/2501.07423
• Copy Paste: [[2501.07423]] An Investigation into Seasonal Variations in Energy Forecasting for Student Residences(https://arxiv.org/abs/2501.07423)
• Keywords: transformer
• Abstract: This research provides an in-depth evaluation of various machine learning models for energy forecasting, focusing on the unique challenges of seasonal variations in student residential settings. The study assesses the performance of baseline models, such as LSTM and GRU, alongside state-of-the-art forecasting methods, including Autoregressive Feedforward Neural Networks, Transformers, and hybrid approaches. Special attention is given to predicting energy consumption amidst challenges like seasonal patterns, vacations, meteorological changes, and irregular human activities that cause sudden fluctuations in usage. The findings reveal that no single model consistently outperforms others across all seasons, emphasizing the need for season-specific model selection or tailored designs. Notably, the proposed Hyper Network based LSTM and MiniAutoEncXGBoost models exhibit strong adaptability to seasonal variations, effectively capturing abrupt changes in energy consumption during summer months. This study advances the energy forecasting field by emphasizing the critical role of seasonal dynamics and model-specific behavior in achieving accurate predictions.

Title: Diff-Ensembler: Learning to Ensemble 2D Diffusion Models for Volume-to-Volume Medical Image Translation

• Authors: Xiyue Zhu, Dou Hoon Kwark, Ruike Zhu, Kaiwen Hong, Yiqi Tao, Shirui Luo, Yudu Li, Zhi-Pei Liang, Volodymyr Kindratenko
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07430
• Pdf URL: https://arxiv.org/pdf/2501.07430
• Copy Paste: [[2501.07430]] Diff-Ensembler: Learning to Ensemble 2D Diffusion Models for Volume-to-Volume Medical Image Translation(https://arxiv.org/abs/2501.07430)
• Keywords: diffusion, segmentation
• Abstract: Despite success in volume-to-volume translations in medical images, most existing models struggle to effectively capture the inherent volumetric distribution using 3D representations. The current state-of-the-art approach combines multiple 2D-based networks through weighted averaging, thereby neglecting the 3D spatial structures. Directly training 3D models in medical imaging presents significant challenges due to high computational demands and the need for large-scale datasets. To address these challenges, we introduce Diff-Ensembler, a novel hybrid 2D-3D model for efficient and effective volumetric translations by ensembling perpendicularly trained 2D diffusion models with a 3D network in each diffusion step. Moreover, our model can naturally be used to ensemble diffusion models conditioned on different modalities, allowing flexible and accurate fusion of input conditions. Extensive experiments demonstrate that Diff-Ensembler attains superior accuracy and volumetric realism in 3D medical image super-resolution and modality translation. We further demonstrate the strength of our model's volumetric realism using tumor segmentation as a downstream task.

Title: Guided SAM: Label-Efficient Part Segmentation

• Authors: S.B. van Rooij, G.J. Burghouts
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07434
• Pdf URL: https://arxiv.org/pdf/2501.07434
• Copy Paste: [[2501.07434]] Guided SAM: Label-Efficient Part Segmentation(https://arxiv.org/abs/2501.07434)
• Keywords: segmentation
• Abstract: Localizing object parts precisely is essential for tasks such as object recognition and robotic manipulation. Recent part segmentation methods require extensive training data and labor-intensive annotations. Segment-Anything Model (SAM) has demonstrated good performance on a wide range of segmentation problems, but requires (manual) positional prompts to guide it where to segment. Furthermore, since it has been trained on full objects instead of object parts, it is prone to over-segmentation of parts. To address this, we propose a novel approach that guides SAM towards the relevant object parts. Our method learns positional prompts from coarse patch annotations that are easier and cheaper to acquire. We train classifiers on image patches to identify part classes and aggregate patches into regions of interest (ROIs) with positional prompts. SAM is conditioned on these ROIs and prompts. This approach, termed `Guided SAM', enhances efficiency and reduces manual effort, allowing effective part segmentation with minimal labeled data. We demonstrate the efficacy of Guided SAM on a dataset of car parts, improving the average IoU on state of the art models from 0.37 to 0.49 with annotations that are on average five times more efficient to acquire.

Title: Union: A Trust-minimized Bridge for Bitcoin

• Authors: Ramon Amela (1), Shreemoy Mishra (1), Sergio Demian Lerner (1 and 2), Javier Álvarez Cid-Fuentes (1) ((1) Rootstock Labs, (2) Fairgate Labs)
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.07435
• Pdf URL: https://arxiv.org/pdf/2501.07435
• Copy Paste: [[2501.07435]] Union: A Trust-minimized Bridge for Bitcoin(https://arxiv.org/abs/2501.07435)
• Keywords: secure, security, federate
• Abstract: We present Union, a trust-minimized bridge protocol that enables secure transfer of BTC between Bitcoin and a secondary blockchain. The growing ecosystem of blockchain systems built around Bitcoin has created a pressing need for secure and efficient bridges to transfer BTC between networks while preserving Bitcoin's security guarantees. Union employs a multi-party variant of BitVMX, an optimistic proving system on Bitcoin, to create a bridge that operates securely under the assumption that at least one participant remains honest. This 1-of-n honest approach is strikingly different from the conventional honest-majority assumption adopted by practically all federated systems. The protocol introduces several innovations: a packet-based architecture that allows security bonds to be reused for multiple bridge operations, improving capital efficiency; a system of enablers to manage functionaries participation and to enforce penalties; a flexible light client framework adaptable to various blockchain architectures; and an efficient stop watch mechanism to optimize time-lock management. Union is a practical and scalable solution for Bitcoin interoperability that maintains strong security guarantees and minimizes trust assumptions.

Title: PrecipDiff: Leveraging image diffusion models to enhance satellite-based precipitation observations

• Authors: Ting-Yu Dai, Hayato Ushijima-Mwesigwa
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07447
• Pdf URL: https://arxiv.org/pdf/2501.07447
• Copy Paste: [[2501.07447]] PrecipDiff: Leveraging image diffusion models to enhance satellite-based precipitation observations(https://arxiv.org/abs/2501.07447)
• Keywords: diffusion
• Abstract: A recent report from the World Meteorological Organization (WMO) highlights that water-related disasters have caused the highest human losses among natural disasters over the past 50 years, with over 91\% of deaths occurring in low-income countries. This disparity is largely due to the lack of adequate ground monitoring stations, such as weather surveillance radars (WSR), which are expensive to install. For example, while the US and Europe combined possess over 600 WSRs, Africa, despite having almost one and half times their landmass, has fewer than 40. To address this issue, satellite-based observations offer a global, near-real-time monitoring solution. However, they face several challenges like accuracy, bias, and low spatial resolution. This study leverages the power of diffusion models and residual learning to address these limitations in a unified framework. We introduce the first diffusion model for correcting the inconsistency between different precipitation products. Our method demonstrates the effectiveness in downscaling satellite precipitation estimates from 10 km to 1 km resolution. Extensive experiments conducted in the Seattle region demonstrate significant improvements in accuracy, bias reduction, and spatial detail. Importantly, our approach achieves these results using only precipitation data, showcasing the potential of a purely computer vision-based approach for enhancing satellite precipitation products and paving the way for further advancements in this domain.

Title: A Novel Approach to Network Traffic Analysis: the HERA tool

• Authors: Daniela Pinto, Ivone Amorim, Eva Maia, Isabel Praça
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07475
• Pdf URL: https://arxiv.org/pdf/2501.07475
• Copy Paste: [[2501.07475]] A Novel Approach to Network Traffic Analysis: the HERA tool(https://arxiv.org/abs/2501.07475)
• Keywords: security, robust, extraction
• Abstract: Cybersecurity threats highlight the need for robust network intrusion detection systems to identify malicious behaviour. These systems rely heavily on large datasets to train machine learning models capable of detecting patterns and predicting threats. In the past two decades, researchers have produced a multitude of datasets, however, some widely utilised recent datasets generated with CICFlowMeter contain inaccuracies. These result in flow generation and feature extraction inconsistencies, leading to skewed results and reduced system effectiveness. Other tools in this context lack ease of use, customizable feature sets, and flow labelling options. In this work, we introduce HERA, a new open-source tool that generates flow files and labelled or unlabelled datasets with user-defined features. Validated and tested with the UNSW-NB15 dataset, HERA demonstrated accurate flow and label generation.

Title: Encrypted Computation of Collision Probability for Secure Satellite Conjunction Analysis

• Authors: Jihoon Suh, Michael Hibbard, Kaoru Teranishi, Takashi Tanaka, Moriba Jah, Maruthi Akella
• Subjects: cs.CR, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.07476
• Pdf URL: https://arxiv.org/pdf/2501.07476
• Copy Paste: [[2501.07476]] Encrypted Computation of Collision Probability for Secure Satellite Conjunction Analysis(https://arxiv.org/abs/2501.07476)
• Keywords: secure, privacy
• Abstract: The computation of collision probability ($\mathcal{P}_c$) is crucial for space environmentalism and sustainability by providing decision-making knowledge that can prevent collisions between anthropogenic space objects. However, the accuracy and precision of $\mathcal{P}_c$ computations is often compromised by limitations in computational resources and data availability. While significant improvements have been made in the computational aspects, the rising concerns regarding the privacy of collaborative data sharing can be a major limiting factor in the future conjunction analysis and risk assessment, especially as the space environment grows increasingly privatized, competitive, and fraught with conflicting strategic interests. This paper argues that the importance of privacy measures in space situational awareness (SSA) is underappreciated, and regulatory and compliance measures currently in place are not sufficient by themselves, presenting a significant gap. To address this gap, we introduce a novel encrypted architecture that leverages advanced cryptographic techniques, including homomorphic encryption (HE) and multi-party computation (MPC), to safeguard the privacy of entities computing space sustainability metrics, inter alia, $\mathcal{P}_c$. Our proposed protocol, Encrypted $\mathcal{P}_c$, integrates the Monte Carlo estimation algorithm with cryptographic solutions, enabling secure collision probability computation without exposing sensitive or proprietary information. This research advances secure conjunction analysis by developing a secure MPC protocol for $\mathcal{P}_c$ computation and highlights the need for innovative protocols to ensure a more secure and cooperative SSA landscape.

Title: TiEBe: A Benchmark for Assessing the Current Knowledge of Large Language Models

• Authors: Thales Sales Almeida, Giovana Kerche Bonás, João Guilherme Alves Santos, Hugo Abonizio, Rodrigo Nogueira
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07482
• Pdf URL: https://arxiv.org/pdf/2501.07482
• Copy Paste: [[2501.07482]] TiEBe: A Benchmark for Assessing the Current Knowledge of Large Language Models(https://arxiv.org/abs/2501.07482)
• Keywords: fair, large language model
• Abstract: In a rapidly evolving knowledge landscape and the increasing adoption of large language models, a need has emerged to keep these models continuously updated with current events. While existing benchmarks evaluate general factual recall, they often overlook two critical aspects: the ability of models to integrate evolving knowledge through continual learning and the significant regional disparities in their performance. To address these gaps, we introduce the Timely Events Benchmark (TiEBe), a dataset containing over 11,000 question-answer pairs focused on globally and regionally significant events. TiEBe leverages structured retrospective data from Wikipedia, enabling continuous updates to assess LLMs' knowledge of evolving global affairs and their understanding of events across different regions. Our benchmark demonstrates that LLMs exhibit substantial geographic disparities in factual recall, emphasizing the need for more balanced global knowledge representation. Furthermore, TiEBe serves as a tool for evaluating continual learning strategies, providing insights into models' ability to acquire new information without forgetting past knowledge.

Title: Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards

• Authors: Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramer, Chiyuan Zhang
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.07493
• Pdf URL: https://arxiv.org/pdf/2501.07493
• Copy Paste: [[2501.07493]] Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards(https://arxiv.org/abs/2501.07493)
• Keywords: security, protect, defense, attack, robust, fair, large language model
• Abstract: It is now common to evaluate Large Language Models (LLMs) by having humans manually vote to evaluate model outputs, in contrast to typical benchmarks that evaluate knowledge or skill at some particular task. Chatbot Arena, the most popular benchmark of this type, ranks models by asking users to select the better response between two randomly selected models (without revealing which model was responsible for the generations). These platforms are widely trusted as a fair and accurate measure of LLM capabilities. In this paper, we show that if bot protection and other defenses are not implemented, these voting-based benchmarks are potentially vulnerable to adversarial manipulation. Specifically, we show that an attacker can alter the leaderboard (to promote their favorite model or demote competitors) at the cost of roughly a thousand votes (verified in a simulated, offline version of Chatbot Arena). Our attack consists of two steps: first, we show how an attacker can determine which model was used to generate a given reply with more than $95\%$ accuracy; and then, the attacker can use this information to consistently vote for (or against) a target model. Working with the Chatbot Arena developers, we identify, propose, and implement mitigations to improve the robustness of Chatbot Arena against adversarial manipulation, which, based on our analysis, substantially increases the cost of such attacks. Some of these defenses were present before our collaboration, such as bot protection with Cloudflare, malicious user detection, and rate limiting. Others, including reCAPTCHA and login are being integrated to strengthen the security in Chatbot Arena.

Title: RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment

• Authors: Difei Gu, Yunhe Gao, Yang Zhou, Mu Zhou, Dimitris Metaxas
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07525
• Pdf URL: https://arxiv.org/pdf/2501.07525
• Copy Paste: [[2501.07525]] RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment(https://arxiv.org/abs/2501.07525)
• Keywords: interpretability, generative, large language model
• Abstract: Automated chest radiographs interpretation requires both accurate disease classification and detailed radiology report generation, presenting a significant challenge in the clinical workflow. Current approaches either focus on classification accuracy at the expense of interpretability or generate detailed but potentially unreliable reports through image captioning techniques. In this study, we present RadAlign, a novel framework that combines the predictive accuracy of vision-language models (VLMs) with the reasoning capabilities of large language models (LLMs). Inspired by the radiologist's workflow, RadAlign first employs a specialized VLM to align visual features with key medical concepts, achieving superior disease classification with an average AUC of 0.885 across multiple diseases. These recognized medical conditions, represented as text-based concepts in the aligned visual-language space, are then used to prompt LLM-based report generation. Enhanced by a retrieval-augmented generation mechanism that grounds outputs in similar historical cases, RadAlign delivers superior report quality with a GREEN score of 0.678, outperforming state-of-the-art methods' 0.634. Our framework maintains strong clinical interpretability while reducing hallucinations, advancing automated medical imaging and report analysis through integrated predictive and generative AI. Code is available at this https URL.

Title: IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion

• Authors: Tharun Anand, Aryan Garg, Kaushik Mitra
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07530
• Pdf URL: https://arxiv.org/pdf/2501.07530
• Copy Paste: [[2501.07530]] IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion(https://arxiv.org/abs/2501.07530)
• Keywords: diffusion
• Abstract: Facial video editing has become increasingly important for content creators, enabling the manipulation of facial expressions and attributes. However, existing models encounter challenges such as poor editing quality, high computational costs and difficulties in preserving facial identity across diverse edits. Additionally, these models are often constrained to editing predefined facial attributes, limiting their flexibility to diverse editing prompts. To address these challenges, we propose a novel facial video editing framework that leverages the rich latent space of pre-trained text-to-image (T2I) diffusion models and fine-tune them specifically for facial video editing tasks. Our approach introduces a targeted fine-tuning scheme that enables high quality, localized, text-driven edits while ensuring identity preservation across video frames. Additionally, by using pre-trained T2I models during inference, our approach significantly reduces editing time by 80%, while maintaining temporal consistency throughout the video sequence. We evaluate the effectiveness of our approach through extensive testing across a wide range of challenging scenarios, including varying head poses, complex action sequences, and diverse facial expressions. Our method consistently outperforms existing techniques, demonstrating superior performance across a broad set of metrics and benchmarks.

Title: Investigating Large Language Models in Inferring Personality Traits from User Conversations

• Authors: Jianfeng Zhu, Ruoming Jin, Karin G. Coifman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.07532
• Pdf URL: https://arxiv.org/pdf/2501.07532
• Copy Paste: [[2501.07532]] Investigating Large Language Models in Inferring Personality Traits from User Conversations(https://arxiv.org/abs/2501.07532)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are demonstrating remarkable human like capabilities across diverse domains, including psychological assessment. This study evaluates whether LLMs, specifically GPT-4o and GPT-4o mini, can infer Big Five personality traits and generate Big Five Inventory-10 (BFI-10) item scores from user conversations under zero-shot prompting conditions. Our findings reveal that incorporating an intermediate step--prompting for BFI-10 item scores before calculating traits--enhances accuracy and aligns more closely with the gold standard than direct trait inference. This structured approach underscores the importance of leveraging psychological frameworks in improving predictive precision. Additionally, a group comparison based on depressive symptom presence revealed differential model performance. Participants were categorized into two groups: those experiencing at least one depressive symptom and those without symptoms. GPT-4o mini demonstrated heightened sensitivity to depression-related shifts in traits such as Neuroticism and Conscientiousness within the symptom-present group, whereas GPT-4o exhibited strengths in nuanced interpretation across groups. These findings underscore the potential of LLMs to analyze real-world psychological data effectively, offering a valuable foundation for interdisciplinary research at the intersection of artificial intelligence and psychology.

Title: Confident Pseudo-labeled Diffusion Augmentation for Canine Cardiomegaly Detection

• Authors: Shiman Zhang, Lakshmikar Reddy Polamreddy, Youshan Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07533
• Pdf URL: https://arxiv.org/pdf/2501.07533
• Copy Paste: [[2501.07533]] Confident Pseudo-labeled Diffusion Augmentation for Canine Cardiomegaly Detection(https://arxiv.org/abs/2501.07533)
• Keywords: diffusion
• Abstract: Canine cardiomegaly, marked by an enlarged heart, poses serious health risks if undetected, requiring accurate diagnostic methods. Current detection models often rely on small, poorly annotated datasets and struggle to generalize across diverse imaging conditions, limiting their real-world applicability. To address these issues, we propose a Confident Pseudo-labeled Diffusion Augmentation (CDA) model for identifying canine cardiomegaly. Our approach addresses the challenge of limited high-quality training data by employing diffusion models to generate synthetic X-ray images and annotate Vertebral Heart Score key points, thereby expanding the dataset. We also employ a pseudo-labeling strategy with Monte Carlo Dropout to select high-confidence labels, refine the synthetic dataset, and improve accuracy. Iteratively incorporating these labels enhances the model's performance, overcoming the limitations of existing approaches. Experimental results show that the CDA model outperforms traditional methods, achieving state-of-the-art accuracy in canine cardiomegaly detection. The code implementation is available at this https URL.

Title: ML Mule: Mobile-Driven Context-Aware Collaborative Learning

• Authors: Haoxiang Yu, Javier Berrocal, Christine Julien
• Subjects: cs.LG, cs.HC
• Abstract URL: https://arxiv.org/abs/2501.07536
• Pdf URL: https://arxiv.org/pdf/2501.07536
• Copy Paste: [[2501.07536]] ML Mule: Mobile-Driven Context-Aware Collaborative Learning(https://arxiv.org/abs/2501.07536)
• Keywords: privacy, protect, robust, federate, large language model
• Abstract: Artificial intelligence has been integrated into nearly every aspect of daily life, powering applications from object detection with computer vision to large language models for writing emails and compact models in smart homes. These machine learning models cater to individual users but are often detached from them, as they are typically stored and processed in centralized data centers. This centralized approach raises privacy concerns, incurs high infrastructure costs, and struggles with personalization. Federated and fully decentralized learning methods have been proposed to address these issues, but they still depend on centralized servers or face slow convergence due to communication constraints. To overcome these challenges, we propose ML Mule, a approach that utilizes individual mobile devices as 'Mules' to train and transport model snapshots as they move through physical spaces, sharing these models with the physical 'Spaces' they inhabit. This method implicitly forms affinity groups among devices associated with users who share particular spaces, enabling collaborative model evolution, and protecting users' privacy. Our approach addresses several major shortcomings of traditional, federated, and fully decentralized learning systems. The proposed framework represents a new class of machine learning methods that are more robust, distributed, and personalized, bringing the field closer to realizing the original vision of intelligent, adaptive, and genuinely context-aware smart environments. The results show that ML Mule converges faster and achieves higher model accuracy compared to other existing methods.

Title: Imagine while Reasoning in Space: Multimodal Visualization-of-Thought

• Authors: Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, Furu Wei
• Subjects: cs.CL, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07542
• Pdf URL: https://arxiv.org/pdf/2501.07542
• Copy Paste: [[2501.07542]] Imagine while Reasoning in Space: Multimodal Visualization-of-Thought(https://arxiv.org/abs/2501.07542)
• Keywords: robust, large language model
• Abstract: Chain-of-Thought (CoT) prompting has proven highly effective for enhancing complex reasoning in Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs). Yet, it struggles in complex spatial reasoning tasks. Nonetheless, human cognition extends beyond language alone, enabling the remarkable capability to think in both words and images. Inspired by this mechanism, we propose a new reasoning paradigm, Multimodal Visualization-of-Thought (MVoT). It enables visual thinking in MLLMs by generating image visualizations of their reasoning traces. To ensure high-quality visualization, we introduce token discrepancy loss into autoregressive MLLMs. This innovation significantly improves both visual coherence and fidelity. We validate this approach through several dynamic spatial reasoning tasks. Experimental results reveal that MVoT demonstrates competitive performance across tasks. Moreover, it exhibits robust and reliable improvements in the most challenging scenarios where CoT fails. Ultimately, MVoT establishes new possibilities for complex reasoning tasks where visual thinking can effectively complement verbal reasoning.

Title: SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editing

• Authors: Varun Biyyala, Bharat Chanderprakash Kathuria, Jialu Li, Youshan Zhang
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.07554
• Pdf URL: https://arxiv.org/pdf/2501.07554
• Copy Paste: [[2501.07554]] SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editing(https://arxiv.org/abs/2501.07554)
• Keywords: extraction, transformer
• Abstract: Video editing models have advanced significantly, but evaluating their performance remains challenging. Traditional metrics, such as CLIP text and image scores, often fall short: text scores are limited by inadequate training data and hierarchical dependencies, while image scores fail to assess temporal consistency. We present SST-EM (Semantic, Spatial, and Temporal Evaluation Metric), a novel evaluation framework that leverages modern Vision-Language Models (VLMs), Object Detection, and Temporal Consistency checks. SST-EM comprises four components: (1) semantic extraction from frames using a VLM, (2) primary object tracking with Object Detection, (3) focused object refinement via an LLM agent, and (4) temporal consistency assessment using a Vision Transformer (ViT). These components are integrated into a unified metric with weights derived from human evaluations and regression analysis. The name SST-EM reflects its focus on Semantic, Spatial, and Temporal aspects of video evaluation. SST-EM provides a comprehensive evaluation of semantic fidelity and temporal smoothness in video editing. The source code is available in the \textbf{\href{this https URL}{GitHub Repository}}.

Title: Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss

• Authors: Xinyu Zhang, Zicheng Duan, Dong Gong, Lingqiao Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.07563
• Pdf URL: https://arxiv.org/pdf/2501.07563
• Copy Paste: [[2501.07563]] Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss(https://arxiv.org/abs/2501.07563)
• Keywords: diffusion
• Abstract: In this paper, we address the challenge of generating temporally consistent videos with motion guidance. While many existing methods depend on additional control modules or inference-time fine-tuning, recent studies suggest that effective motion guidance is achievable without altering the model architecture or requiring extra training. Such approaches offer promising compatibility with various video generation foundation models. However, existing training-free methods often struggle to maintain consistent temporal coherence across frames or to follow guided motion accurately. In this work, we propose a simple yet effective solution that combines an initial-noise-based approach with a novel motion consistency loss, the latter being our key innovation. Specifically, we capture the inter-frame feature correlation patterns of intermediate features from a video diffusion model to represent the motion pattern of the reference video. We then design a motion consistency loss to maintain similar feature correlation patterns in the generated video, using the gradient of this loss in the latent space to guide the generation process for precise motion control. This approach improves temporal consistency across various motion control tasks while preserving the benefits of a training-free setup. Extensive experiments show that our method sets a new standard for efficient, temporally coherent video generation.

Title: E2ESlack: An End-to-End Graph-Based Framework for Pre-Routing Slack Prediction

• Authors: Saurabh Bodhe, Zhanguang Zhang, Atia Hamidizadeh, Shixiong Kai, Yingxue Zhang, Mingxuan Yuan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.07564
• Pdf URL: https://arxiv.org/pdf/2501.07564
• Copy Paste: [[2501.07564]] E2ESlack: An End-to-End Graph-Based Framework for Pre-Routing Slack Prediction(https://arxiv.org/abs/2501.07564)
• Keywords: extraction
• Abstract: Pre-routing slack prediction remains a critical area of research in Electronic Design Automation (EDA). Despite numerous machine learning-based approaches targeting this task, there is still a lack of a truly end-to-end framework that engineers can use to obtain TNS/WNS metrics from raw circuit data at the placement stage. Existing works have demonstrated effectiveness in Arrival Time (AT) prediction but lack a mechanism for Required Arrival Time (RAT) prediction, which is essential for slack prediction and obtaining TNS/WNS metrics. In this work, we propose E2ESlack, an end-to-end graph-based framework for pre-routing slack prediction. The framework includes a TimingParser that supports DEF, SDF and LIB files for feature extraction and graph construction, an arrival time prediction model and a fast RAT estimation module. To the best of our knowledge, this is the first work capable of predicting path-level slacks at the pre-routing stage. We perform extensive experiments and demonstrate that our proposed RAT estimation method outperforms the SOTA ML-based prediction method and also pre-routing STA tool. Additionally, the proposed E2ESlack framework achieves TNS/WNS values comparable to post-routing STA results while saving up to 23x runtime.

Title: UnCommon Objects in 3D

• Authors: Xingchen Liu, Piyush Tayal, Jianyuan Wang, Jesus Zarzar, Tom Monnier, Konstantinos Tertikas, Jiali Duan, Antoine Toisoul, Jason Y. Zhang, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, David Novotny
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2501.07574
• Pdf URL: https://arxiv.org/pdf/2501.07574
• Copy Paste: [[2501.07574]] UnCommon Objects in 3D(https://arxiv.org/abs/2501.07574)
• Keywords: generative
• Abstract: We introduce Uncommon Objects in 3D (uCO3D), a new object-centric dataset for 3D deep learning and 3D generative AI. uCO3D is the largest publicly-available collection of high-resolution videos of objects with 3D annotations that ensures full-360$^{\circ}$ coverage. uCO3D is significantly more diverse than MVImgNet and CO3Dv2, covering more than 1,000 object categories. It is also of higher quality, due to extensive quality checks of both the collected videos and the 3D annotations. Similar to analogous datasets, uCO3D contains annotations for 3D camera poses, depth maps and sparse point clouds. In addition, each object is equipped with a caption and a 3D Gaussian Splat reconstruction. We train several large 3D models on MVImgNet, CO3Dv2, and uCO3D and obtain superior results using the latter, showing that uCO3D is better for learning applications.

Title: Dataset Distillation via Committee Voting

• Authors: Jiacheng Cui, Zhaoyi Li, Xiaochen Ma, Xinyue Bi, Yaxin Luo, Zhiqiang Shen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.07575
• Pdf URL: https://arxiv.org/pdf/2501.07575
• Copy Paste: [[2501.07575]] Dataset Distillation via Committee Voting(https://arxiv.org/abs/2501.07575)
• Keywords: robust
• Abstract: Dataset distillation aims to synthesize a smaller, representative dataset that preserves the essential properties of the original data, enabling efficient model training with reduced computational resources. Prior work has primarily focused on improving the alignment or matching process between original and synthetic data, or on enhancing the efficiency of distilling large datasets. In this work, we introduce ${\bf C}$ommittee ${\bf V}$oting for ${\bf D}$ataset ${\bf D}$istillation (CV-DD), a novel and orthogonal approach that leverages the collective wisdom of multiple models or experts to create high-quality distilled datasets. We start by showing how to establish a strong baseline that already achieves state-of-the-art accuracy through leveraging recent advancements and thoughtful adjustments in model design and optimization processes. By integrating distributions and predictions from a committee of models while generating high-quality soft labels, our method captures a wider spectrum of data features, reduces model-specific biases and the adverse effects of distribution shifts, leading to significant improvements in generalization. This voting-based strategy not only promotes diversity and robustness within the distilled dataset but also significantly reduces overfitting, resulting in improved performance on post-eval tasks. Extensive experiments across various datasets and IPCs (images per class) demonstrate that Committee Voting leads to more reliable and adaptable distilled data compared to single/multi-model distillation methods, demonstrating its potential for efficient and accurate dataset distillation. Code is available at: this https URL.