2025-01-28

Title: Towards Foundation Models: Evaluation of Geoscience Artificial Intelligence with Uncertainty

• Authors: Samuel Myren, Nidhi Parikh, Rosalyn Rael, Garrison Flynn, Dave Higdon, Emily Casleton
• Subjects: cs.LG, cs.AI, physics.geo-ph
• Abstract URL: https://arxiv.org/abs/2501.14809
• Pdf URL: https://arxiv.org/pdf/2501.14809
• Copy Paste: [[2501.14809]] Towards Foundation Models: Evaluation of Geoscience Artificial Intelligence with Uncertainty(https://arxiv.org/abs/2501.14809)
• Keywords: robust
• Abstract: Artificial intelligence (AI) has transformed the geoscience community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. This success has led to the development of geoscience foundation models (FMs), which promise to accomplish multiple tasks within a workflow or replace the workflow altogether. However, lack of robust evaluation frameworks, even for traditional DLMs, leaves the geoscience community ill prepared for the inevitable adoption of FMs. We address this gap by designing an evaluation framework that jointly incorporates three crucial aspects to current DLMs and future FMs: performance uncertainty, learning efficiency, and overlapping training-test data splits. To target the three aspects, we meticulously construct the training, validation, and test splits using clustering methods tailored to geoscience data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework's ability to guard against misleading declarations of model superiority is demonstrated through evaluation of PhaseNet, a popular seismic phase picking DLM, under 3 training approaches. Furthermore, we show how the performance gains due to overlapping training-test data can lead to biased FM evaluation. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance at varying budgets of training data.

Title: An Ensemble Model with Attention Based Mechanism for Image Captioning

• Authors: Israa Al Badarneh, Bassam Hammo, Omar Al-Kadi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14828
• Pdf URL: https://arxiv.org/pdf/2501.14828
• Copy Paste: [[2501.14828]] An Ensemble Model with Attention Based Mechanism for Image Captioning(https://arxiv.org/abs/2501.14828)
• Keywords: transformer
• Abstract: Image captioning creates informative text from an input image by creating a relationship between the words and the actual content of an image. Recently, deep learning models that utilize transformers have been the most successful in automatically generating image captions. The capabilities of transformer networks have led to notable progress in several activities related to vision. In this paper, we thoroughly examine transformer models, emphasizing the critical role that attention mechanisms play. The proposed model uses a transformer encoder-decoder architecture to create textual captions and a deep learning convolutional neural network to extract features from the images. To create the captions, we present a novel ensemble learning framework that improves the richness of the generated captions by utilizing several deep neural network architectures based on a voting mechanism that chooses the caption with the highest bilingual evaluation understudy (BLEU) score. The proposed model was evaluated using publicly available datasets. Using the Flickr8K dataset, the proposed model achieved the highest BLEU-[1-3] scores with rates of 0.728, 0.495, and 0.323, respectively. The suggested model outperformed the latest methods in Flickr30k datasets, determined by BLEU-[1-4] scores with rates of 0.798, 0.561, 0.387, and 0.269, respectively. The model efficacy was also obtained by the Semantic propositional image caption evaluation (SPICE) metric with a scoring rate of 0.164 for the Flicker8k dataset and 0.387 for the Flicker30k. Finally, ensemble learning significantly advances the process of image captioning and, hence, can be leveraged in various applications across different domains.

Title: Unmasking Conversational Bias in AI Multiagent Systems

• Authors: Erica Coppolillo, Giuseppe Manco, Luca Maria Aiello
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.14844
• Pdf URL: https://arxiv.org/pdf/2501.14844
• Copy Paste: [[2501.14844]] Unmasking Conversational Bias in AI Multiagent Systems(https://arxiv.org/abs/2501.14844)
• Keywords: generative, large language model
• Abstract: Detecting biases in the outputs produced by generative models is essential to reduce the potential risks associated with their application in critical settings. However, the majority of existing methodologies for identifying biases in generated text consider the models in isolation and neglect their contextual applications. Specifically, the biases that may arise in multi-agent systems involving generative models remain under-researched. To address this gap, we present a framework designed to quantify biases within multi-agent systems of conversational Large Language Models (LLMs). Our approach involves simulating small echo chambers, where pairs of LLMs, initialized with aligned perspectives on a polarizing topic, engage in discussions. Contrary to expectations, we observe significant shifts in the stance expressed in the generated messages, particularly within echo chambers where all agents initially express conservative viewpoints, in line with the well-documented political bias of many LLMs toward liberal positions. Crucially, the bias observed in the echo-chamber experiment remains undetected by current state-of-the-art bias detection methods that rely on questionnaires. This highlights a critical need for the development of a more sophisticated toolkit for bias detection and mitigation for AI multi-agent systems. The code to perform the experiments is publicly available at this https URL.

Title: Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval

• Authors: Libo Wang
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14846
• Pdf URL: https://arxiv.org/pdf/2501.14846
• Copy Paste: [[2501.14846]] Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval(https://arxiv.org/abs/2501.14846)
• Keywords: large language model
• Abstract: In view of the gap in the current large language model in sharing memory across dialogues, this research proposes a wormhole memory module (WMM) to realize memory as a Rubik's cube that can be arbitrarily retrieved between different dialogues. Through simulation experiments, the researcher built an experimental framework based on the Python environment and used setting memory barriers to simulate the current situation where memories between LLMs dialogues are difficult to share. The CoQA development data set was imported into the experiment, and the feasibility of its cross-dialogue memory retrieval function was verified for WMM's nonlinear indexing and dynamic retrieval, and a comparative analysis was conducted with the capabilities of Titans and MemGPT memory modules. Experimental results show that WMM demonstrated the ability to retrieve memory across dialogues and the stability of quantitative indicators in eight experiments. It contributes new technical approaches to the optimization of memory management of LLMs and provides experience for the practical application in the future.

Title: On the locality bias and results in the Long Range Arena

• Authors: Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín, David Camacho
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14850
• Pdf URL: https://arxiv.org/pdf/2501.14850
• Copy Paste: [[2501.14850]] On the locality bias and results in the Long Range Arena(https://arxiv.org/abs/2501.14850)
• Keywords: transformer
• Abstract: The Long Range Arena (LRA) benchmark was designed to evaluate the performance of Transformer improvements and alternatives in long-range dependency modeling tasks. The Transformer and its main variants performed poorly on this benchmark, and a new series of architectures such as State Space Models (SSMs) gained some traction, greatly outperforming Transformers in the LRA. Recent work has shown that with a denoising pre-training phase, Transformers can achieve competitive results in the LRA with these new architectures. In this work, we discuss and explain the superiority of architectures such as MEGA and SSMs in the Long Range Arena, as well as the recent improvement in the results of Transformers, pointing to the positional and local nature of the tasks. We show that while the LRA is a benchmark for long-range dependency modeling, in reality most of the performance comes from short-range dependencies. Using training techniques to mitigate data inefficiency, Transformers are able to reach state-of-the-art performance with proper positional encoding. In addition, with the same techniques, we were able to remove all restrictions from SSM convolutional kernels and learn fully parameterized convolutions without decreasing performance, suggesting that the design choices behind SSMs simply added inductive biases and learning efficiency for these particular tasks. Our insights indicate that LRA results should be interpreted with caution and call for a redesign of the benchmark.

Title: JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models

• Authors: Michael K. Chen, Xikun Zhang, Dacheng Tao
• Subjects: cs.CL, cs.AI, cs.LG, cs.LO
• Abstract URL: https://arxiv.org/abs/2501.14851
• Pdf URL: https://arxiv.org/pdf/2501.14851
• Copy Paste: [[2501.14851]] JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models(https://arxiv.org/abs/2501.14851)
• Keywords: large language model
• Abstract: Logical reasoning is a critical component of Large Language Models (LLMs), and substantial research efforts in recent years have aimed to enhance their deductive reasoning capabilities. However, existing deductive reasoning benchmarks, which are crucial for evaluating and advancing LLMs, are inadequate due to their lack of task complexity, presence of prior knowledge as a confounder, and superficial error analysis. To address these deficiencies, we introduce JustLogic, a synthetically generated deductive reasoning benchmark designed for rigorous evaluation of LLMs. JustLogic is (i) highly complex, capable of generating a diverse range of linguistic patterns, vocabulary, and argument structures; (ii) prior knowledge independent, eliminating the advantage of models possessing prior knowledge and ensuring that only deductive reasoning is used to answer questions; and (iii) capable of in-depth error analysis on the heterogeneous effects of reasoning depth and argument form on model accuracy. Our experimental results on JustLogic reveal that most state-of-the-art (SOTA) LLMs perform significantly worse than the human average, demonstrating substantial room for model improvement. All code and data are available at this https URL

Title: Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models

• Authors: Xiaoxuan Liao, Chihang Wang, Shicheng Zhou, Jiacheng Hu, Hongye Zheng, Jia Gao
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14859
• Pdf URL: https://arxiv.org/pdf/2501.14859
• Copy Paste: [[2501.14859]] Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models(https://arxiv.org/abs/2501.14859)
• Keywords: large language model
• Abstract: This paper presents a novel methodology of fine-tuning for large language models-dynamic LoRA. Building from the standard Low-Rank Adaptation framework, this methodology further adds dynamic adaptation mechanisms to improve efficiency and performance. The key contribution of dynamic LoRA lies within its adaptive weight allocation mechanism coupled with an input feature-based adaptive strategy. These enhancements allow for a more precise fine-tuning process that is more tailored to specific tasks. Traditional LoRA methods use static adapter settings, not considering the different importance of model layers. In contrast, dynamic LoRA introduces a mechanism that dynamically evaluates the layer's importance during fine-tuning. This evaluation enables the reallocation of adapter parameters to fit the unique demands of each individual task, which leads to better optimization results. Another gain in flexibility arises from the consideration of the input feature distribution, which helps the model generalize better when faced with complicated and diverse datasets. The joint approach boosts not only the performance over each single task but also the generalization ability of the model. The efficiency of the dynamic LoRA was validated in experiments on benchmark datasets, such as GLUE, with surprising results. More specifically, this method achieved 88.1% accuracy with an F1-score of 87.3%. Noticeably, these improvements were made at a slight increase in computational costs: only 0.1% more resources than standard LoRA. This balance between performance and efficiency positions dynamic LoRA as a practical, scalable solution for fine-tuning LLMs, especially in resource-constrained scenarios. To take it a step further, its adaptability makes it a promising foundation for much more advanced applications, including multimodal tasks.

Title: DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images

• Authors: Sami Baral, Li Lucy, Ryan Knight, Alice Ng, Luca Soldaini, Neil T. Heffernan, Kyle Lo
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14877
• Pdf URL: https://arxiv.org/pdf/2501.14877
• Copy Paste: [[2501.14877]] DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images(https://arxiv.org/abs/2501.14877)
• Keywords: robust
• Abstract: In real-world settings, vision language models (VLMs) should robustly handle naturalistic, noisy visual content as well as domain-specific language and concepts. For example, K-12 educators using digital learning platforms may need to examine and provide feedback across many images of students' math work. To assess the potential of VLMs to support educators in settings like this one, we introduce DrawEduMath, an English-language dataset of 2,030 images of students' handwritten responses to K-12 math problems. Teachers provided detailed annotations, including free-form descriptions of each image and 11,661 question-answer (QA) pairs. These annotations capture a wealth of pedagogical insights, ranging from students' problem-solving strategies to the composition of their drawings, diagrams, and writing. We evaluate VLMs on teachers' QA pairs, as well as 44,362 synthetic QA pairs derived from teachers' descriptions using language models (LMs). We show that even state-of-the-art VLMs leave much room for improvement on DrawEduMath questions. We also find that synthetic QAs, though imperfect, can yield similar model rankings as teacher-written QAs. We release DrawEduMath to support the evaluation of VLMs' abilities to reason mathematically over images gathered with educational contexts in mind.

Title: Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics

• Authors: Ameya Godbole, Robin Jia
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14883
• Pdf URL: https://arxiv.org/pdf/2501.14883
• Copy Paste: [[2501.14883]] Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics(https://arxiv.org/abs/2501.14883)
• Keywords: large language model
• Abstract: Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge this optimism by thoroughly re-evaluating five state-of-the-art factuality metrics on a collection of 11 datasets for summarization, retrieval-augmented generation, and question answering. We find that these evaluators are inconsistent with each other and often misestimate system-level performance, both of which can lead to a variety of pitfalls. We further show that these metrics exhibit biases against highly paraphrased outputs and outputs that draw upon faraway parts of the source documents. We urge users of these factuality metrics to proceed with caution and manually validate the reliability of these metrics in their domain of interest before proceeding.

Title: Hybrid Interpretable Deep Learning Framework for Skin Cancer Diagnosis: Integrating Radial Basis Function Networks with Explainable AI

• Authors: Mirza Ahsan Ullah, Tehseen Zia
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14885
• Pdf URL: https://arxiv.org/pdf/2501.14885
• Copy Paste: [[2501.14885]] Hybrid Interpretable Deep Learning Framework for Skin Cancer Diagnosis: Integrating Radial Basis Function Networks with Explainable AI(https://arxiv.org/abs/2501.14885)
• Keywords: extraction, interpretability, segmentation
• Abstract: Skin cancer is one of the most prevalent and potentially life-threatening diseases worldwide, necessitating early and accurate diagnosis to improve patient outcomes. Conventional diagnostic methods, reliant on clinical expertise and histopathological analysis, are often time-intensive, subjective, and prone to variability. To address these limitations, we propose a novel hybrid deep learning framework that integrates convolutional neural networks (CNNs) with Radial Basis Function (RBF) Networks to achieve high classification accuracy and enhanced interpretability. The motivation for incorporating RBF Networks lies in their intrinsic interpretability and localized response to input features, which make them well-suited for tasks requiring transparency and fine-grained decision-making. Unlike traditional deep learning models that rely on global feature representations, RBF Networks allow for mapping segments of images to chosen prototypes, exploiting salient features within a single image. This enables clinicians to trace predictions to specific, interpretable patterns. The framework incorporates segmentation-based feature extraction, active learning for prototype selection, and K-Medoids clustering to focus on these salient features. Evaluations on the ISIC 2016 and ISIC 2017 datasets demonstrate the model's effectiveness, achieving classification accuracies of 83.02\% and 72.15\% using ResNet50, respectively, and outperforming VGG16-based configurations. By generating interpretable explanations for predictions, the framework aligns with clinical workflows, bridging the gap between predictive performance and trustworthiness. This study highlights the potential of hybrid models to deliver actionable insights, advancing the development of reliable AI-assisted diagnostic tools for high-stakes medical applications.

Title: Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing

• Authors: Madeline Anderson, Miriam Cha, William T. Freeman, J. Taylor Perron, Nathaniel Maidel, Kerri Cahoy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14905
• Pdf URL: https://arxiv.org/pdf/2501.14905
• Copy Paste: [[2501.14905]] Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing(https://arxiv.org/abs/2501.14905)
• Keywords: large language model
• Abstract: Vision language models have achieved impressive results across various fields. However, adoption in remote sensing remains limited, largely due to the scarcity of paired image-text data. To bridge this gap, synthetic caption generation has gained interest, traditionally relying on rule-based methods that use metadata or bounding boxes. While these approaches provide some description, they often lack the depth needed to capture complex wide-area scenes. Large language models (LLMs) offer a promising alternative for generating more descriptive captions, yet they can produce generic outputs and are prone to hallucination. In this paper, we propose a new method to enhance vision-language datasets for remote sensing by integrating maps as external data sources, enabling the generation of detailed, context-rich captions. Additionally, we present methods to measure and mitigate hallucinations in LLM-generated text. We introduce fMoW-mm, a multimodal dataset incorporating satellite imagery, maps, metadata, and text annotations. We demonstrate its effectiveness for automatic target recognition in few-shot settings, achieving superior performance compared to other vision-language remote sensing datasets.

Title: Feasible Learning

• Authors: Juan Ramirez, Ignacio Hounie, Juan Elenter, Jose Gallego-Posada, Meraj Hashemizadeh, Alejandro Ribeiro, Simon Lacoste-Julien
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14912
• Pdf URL: https://arxiv.org/pdf/2501.14912
• Copy Paste: [[2501.14912]] Feasible Learning(https://arxiv.org/abs/2501.14912)
• Keywords: large language model
• Abstract: We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample. In contrast to the ubiquitous Empirical Risk Minimization (ERM) framework, which optimizes for average performance, FL demands satisfactory performance on every individual data point. Since any model that meets the prescribed performance threshold is a valid FL solution, the choice of optimization algorithm and its dynamics play a crucial role in shaping the properties of the resulting solutions. In particular, we study a primal-dual approach which dynamically re-weights the importance of each sample during training. To address the challenge of setting a meaningful threshold in practice, we introduce a relaxation of FL that incorporates slack variables of minimal norm. Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.

Title: Light3R-SfM: Towards Feed-forward Structure-from-Motion

• Authors: Sven Elflein, Qunjie Zhou, Sérgio Agostinho, Laura Leal-Taixé
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14914
• Pdf URL: https://arxiv.org/pdf/2501.14914
• Copy Paste: [[2501.14914]] Light3R-SfM: Towards Feed-forward Structure-from-Motion(https://arxiv.org/abs/2501.14914)
• Keywords: robust
• Abstract: We present Light3R-SfM, a feed-forward, end-to-end learnable framework for efficient large-scale Structure-from-Motion (SfM) from unconstrained image collections. Unlike existing SfM solutions that rely on costly matching and global optimization to achieve accurate 3D reconstructions, Light3R-SfM addresses this limitation through a novel latent global alignment module. This module replaces traditional global optimization with a learnable attention mechanism, effectively capturing multi-view constraints across images for robust and precise camera pose estimation. Light3R-SfM constructs a sparse scene graph via retrieval-score-guided shortest path tree to dramatically reduce memory usage and computational overhead compared to the naive approach. Extensive experiments demonstrate that Light3R-SfM achieves competitive accuracy while significantly reducing runtime, making it ideal for 3D reconstruction tasks in real-world applications with a runtime constraint. This work pioneers a data-driven, feed-forward SfM approach, paving the way toward scalable, accurate, and efficient 3D reconstruction in the wild.

Title: Self-reflecting Large Language Models: A Hegelian Dialectical Approach

• Authors: Sara Abdali, Can Goksen, Saeed Amizadeh andKazuhito Koishida
• Subjects: cs.CL, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14917
• Pdf URL: https://arxiv.org/pdf/2501.14917
• Copy Paste: [[2501.14917]] Self-reflecting Large Language Models: A Hegelian Dialectical Approach(https://arxiv.org/abs/2501.14917)
• Keywords: large language model
• Abstract: Investigating NLP through a philosophical lens has recently caught researcher's eyes as it connects computational methods with classical schools of philosophy. This paper introduces a philosophical approach inspired by the Hegelian Dialectic for LLMs' self-reflection, utilizing a self-dialectical approach to emulate internal critiques and then synthesize new ideas by resolving the contradicting points. Moreover, this paper investigates the effect of LLMs' temperature for generation by establishing a dynamic annealing approach, which promotes the creativity in the early stages and gradually refines it by focusing on the nuances, as well as a fixed temperature strategy for generation. Our proposed approach is examined to determine its ability to generate novel ideas from an initial proposition. Additionally, a Multi Agent Majority Voting (MAMV) strategy is leveraged to assess the validity and novelty of the generated ideas, which proves beneficial in the absence of domain experts. Our experiments show promise in generating new ideas and provide a stepping-stone for future research.

Title: Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition

• Authors: Dan Braun, Lucius Bushnaq, Stefan Heimersheim, Jake Mendel, Lee Sharkey
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.14926
• Pdf URL: https://arxiv.org/pdf/2501.14926
• Copy Paste: [[2501.14926]] Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition(https://arxiv.org/abs/2501.14926)
• Keywords: interpretability
• Abstract: Mechanistic interpretability aims to understand the internal mechanisms learned by neural networks. Despite recent progress toward this goal, it remains unclear how best to decompose neural network parameters into mechanistic components. We introduce Attribution-based Parameter Decomposition (APD), a method that directly decomposes a neural network's parameters into components that (i) are faithful to the parameters of the original network, (ii) require a minimal number of components to process any input, and (iii) are maximally simple. Our approach thus optimizes for a minimal length description of the network's mechanisms. We demonstrate APD's effectiveness by successfully identifying ground truth mechanisms in multiple toy experimental settings: Recovering features from superposition; separating compressed computations; and identifying cross-layer distributed representations. While challenges remain to scaling APD to non-toy models, our results suggest solutions to several open problems in mechanistic interpretability, including identifying minimal circuits in superposition, offering a conceptual foundation for 'features', and providing an architecture-agnostic framework for neural network decomposition.

Title: Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy

• Authors: Fan Chen, Alexander Rakhlin
• Subjects: cs.LG, cs.AI, cs.IT, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.14928
• Pdf URL: https://arxiv.org/pdf/2501.14928
• Copy Paste: [[2501.14928]] Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy(https://arxiv.org/abs/2501.14928)
• Keywords: privacy, robust
• Abstract: We study the problem of interactive decision making in which the underlying environment changes over time subject to given constraints. We propose a framework, which we call \textit{hybrid Decision Making with Structured Observations} (hybrid DMSO), that provides an interpolation between the stochastic and adversarial settings of decision making. Within this framework, we can analyze local differentially private (LDP) decision making, query-based learning (in particular, SQ learning), and robust and smooth decision making under the same umbrella, deriving upper and lower bounds based on variants of the Decision-Estimation Coefficient (DEC). We further establish strong connections between the DEC's behavior, the SQ dimension, local minimax complexity, learnability, and joint differential privacy. To showcase the framework's power, we provide new results for contextual bandits under the LDP constraint.

Title: Motion-enhancement to Echocardiography Segmentation via Inserting a Temporal Attention Module: An Efficient, Adaptable, and Scalable Approach

• Authors: Md. Kamrul Hasan, Guang Yang, Choon Hwai Yap
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14929
• Pdf URL: https://arxiv.org/pdf/2501.14929
• Copy Paste: [[2501.14929]] Motion-enhancement to Echocardiography Segmentation via Inserting a Temporal Attention Module: An Efficient, Adaptable, and Scalable Approach(https://arxiv.org/abs/2501.14929)
• Keywords: robust, transformer, segmentation
• Abstract: Cardiac anatomy segmentation is essential for clinical assessment of cardiac function and disease diagnosis to inform treatment and intervention. In performing segmentation, deep learning (DL) algorithms improved accuracy significantly compared to traditional image processing approaches. More recently, studies showed that enhancing DL segmentation with motion information can further improve it. A range of methods for injecting motion information has been proposed, but many of them increase the dimensionality of input images (which is computationally expensive) or have not used an optimal method to insert motion information, such as non-DL registration, non-attention-based networks or single-headed attention. Here, we present a novel, computation-efficient alternative where a novel, scalable temporal attention module (TAM) extracts temporal feature interactions multiple times and where TAM has a multi-headed, KQV projection cross-attention architecture. The module can be seamlessly integrated into a wide range of existing CNN- or Transformer-based networks, providing novel flexibility for inclusion in future implementations. Extensive evaluations on different cardiac datasets, 2D echocardiography (CAMUS), and 3D echocardiography (MITEA) demonstrate the model's effectiveness when integrated into well-established backbone networks like UNet, FCN8s, UNetR, SwinUNetR, and the recent I2UNet. We further find that the optimized TAM-enhanced FCN8s network performs well compared to contemporary alternatives. Our results confirm TAM's robustness, scalability, and generalizability across diverse datasets and backbones.

Title: Context-Aware Neural Gradient Mapping for Fine-Grained Instruction Processing

• Authors: David Boldo, Lily Pemberton, Gabriel Thistledown, Jacob Fairchild, Felix Kowalski
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14936
• Pdf URL: https://arxiv.org/pdf/2501.14936
• Copy Paste: [[2501.14936]] Context-Aware Neural Gradient Mapping for Fine-Grained Instruction Processing(https://arxiv.org/abs/2501.14936)
• Keywords: robust, large language model
• Abstract: The integration of contextual embeddings into the optimization processes of large language models is an advancement in natural language processing. The Context-Aware Neural Gradient Mapping framework introduces a dynamic gradient adjustment mechanism, incorporating contextual embeddings directly into the optimization process. This approach facilitates real-time parameter adjustments, enhancing task-specific generalization even in the presence of sparse or noisy data inputs. The mathematical foundation of this framework relies on gradient descent modifications, where contextual embeddings are derived from a supplementary neural network trained to map input features to optimal adaptation gradients. By employing differential geometry principles, high-dimensional input dependencies are encoded into low-dimensional gradient manifolds, enabling efficient adaptation without necessitating the retraining of the entire model. Empirical evaluations demonstrate that the proposed framework consistently outperforms baseline models across various metrics, including accuracy, robustness to noise, and computational efficiency. The integration of context-specific embeddings allows for a more complex understanding of language, thereby improving the model's ability to handle diverse linguistic phenomena. Furthermore, the computational efficiency achieved through this method demonstrates its scalability for large-scale language models operating under diverse constraints.

Title: CASE-Bench: Context-Aware Safety Evaluation Benchmark for Large Language Models

• Authors: Guangzhi Sun, Xiao Zhan, Shutong Feng, Philip C. Woodland, Jose Such
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14940
• Pdf URL: https://arxiv.org/pdf/2501.14940
• Copy Paste: [[2501.14940]] CASE-Bench: Context-Aware Safety Evaluation Benchmark for Large Language Models(https://arxiv.org/abs/2501.14940)
• Keywords: large language model
• Abstract: Aligning large language models (LLMs) with human values is essential for their safe deployment and widespread adoption. Current LLM safety benchmarks often focus solely on the refusal of individual problematic queries, which overlooks the importance of the context where the query occurs and may cause undesired refusal of queries under safe contexts that diminish user experience. Addressing this gap, we introduce CASE-Bench, a Context-Aware Safety Evaluation Benchmark that integrates context into safety assessments of LLMs. CASE-Bench assigns distinct, formally described contexts to categorized queries based on Contextual Integrity theory. Additionally, in contrast to previous studies which mainly rely on majority voting from just a few annotators, we recruited a sufficient number of annotators necessary to ensure the detection of statistically significant differences among the experimental conditions based on power analysis. Our extensive analysis using CASE-Bench on various open-source and commercial LLMs reveals a substantial and significant influence of context on human judgments (p<0.0001 from a z-test), underscoring the necessity of context in safety evaluations. We also identify notable mismatches between human judgments and LLM responses, particularly in commercial models within safe contexts.

Title: MATCHA:Towards Matching Anything

• Authors: Fei Xue, Sven Elflein, Laura Leal-Taixé, Qunjie Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14945
• Pdf URL: https://arxiv.org/pdf/2501.14945
• Copy Paste: [[2501.14945]] MATCHA:Towards Matching Anything(https://arxiv.org/abs/2501.14945)
• Keywords: robust, diffusion
• Abstract: Establishing correspondences across images is a fundamental challenge in computer vision, underpinning tasks like Structure-from-Motion, image editing, and point tracking. Traditional methods are often specialized for specific correspondence types, geometric, semantic, or temporal, whereas humans naturally identify alignments across these domains. Inspired by this flexibility, we propose MATCHA, a unified feature model designed to ``rule them all'', establishing robust correspondences across diverse matching tasks. Building on insights that diffusion model features can encode multiple correspondence types, MATCHA augments this capacity by dynamically fusing high-level semantic and low-level geometric features through an attention-based module, creating expressive, versatile, and robust features. Additionally, MATCHA integrates object-level features from DINOv2 to further boost generalization, enabling a single feature capable of matching anything. Extensive experiments validate that MATCHA consistently surpasses state-of-the-art methods across geometric, semantic, and temporal matching tasks, setting a new foundation for a unified approach for the fundamental correspondence problem in computer vision. To the best of our knowledge, MATCHA is the first approach that is able to effectively tackle diverse matching tasks with a single unified feature.

Title: E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions

• Authors: Hongbo Zheng, Suyuan Wang, Neeraj Gangwar, Nickvash Kani
• Subjects: cs.LG, cs.CL, cs.SC
• Abstract URL: https://arxiv.org/abs/2501.14951
• Pdf URL: https://arxiv.org/pdf/2501.14951
• Copy Paste: [[2501.14951]] E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions(https://arxiv.org/abs/2501.14951)
• Keywords: large language model
• Abstract: As vector representations have been pivotal in advancing natural language processing (NLP), some prior research has concentrated on creating embedding techniques for mathematical expressions by leveraging mathematically equivalent expressions. While effective, these methods are limited by the training data. In this work, we propose augmenting prior algorithms with larger synthetic dataset, using a novel e-graph-based generation scheme. This new mathematical dataset generation scheme, E-Gen, improves upon prior dataset-generation schemes that are limited in size and operator types. We use this dataset to compare embedding models trained with two methods: (1) training the model to generate mathematically equivalent expressions, and (2) training the model using contrastive learning to group mathematically equivalent expressions explicitly. We evaluate the embeddings generated by these methods against prior work on both in-distribution and out-of-distribution language processing tasks. Finally, we compare the performance of our embedding scheme against state-of-the-art large language models and demonstrate that embedding-based language processing methods perform better than LLMs on several tasks, demonstrating the necessity of optimizing embedding methods for the mathematical data modality.

Title: ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation

• Authors: Alireza Salemi, Julian Killingback, Hamed Zamani
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.14956
• Pdf URL: https://arxiv.org/pdf/2501.14956
• Copy Paste: [[2501.14956]] ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation(https://arxiv.org/abs/2501.14956)
• Keywords: interpretability, large language model
• Abstract: Evaluating personalized text generated by large language models (LLMs) is challenging, as only the LLM user, i.e., prompt author, can reliably assess the output, but re-engaging the same individuals across studies is infeasible. This paper addresses the challenge of evaluating personalized text generation by introducing ExPerT, an explainable reference-based evaluation framework. ExPerT leverages an LLM to extract atomic aspects and their evidence from the generated and reference texts, match the aspects, and evaluate their alignment based on content and writing style -- two key attributes in personalized text generation. Additionally, ExPerT generates detailed, fine-grained explanations for every step of the evaluation process, enhancing transparency and interpretability. Our experiments demonstrate that ExPerT achieves a 7.2% relative improvement in alignment with human judgments compared to the state-of-the-art text generation evaluation methods. Furthermore, human evaluators rated the usability of ExPerT's explanations at 4.7 out of 5, highlighting its effectiveness in making evaluation decisions more interpretable.

Title: LLM4DistReconfig: A Fine-tuned Large Language Model for Power Distribution Network Reconfiguration

• Authors: Panayiotis Christou, Md. Zahidul Islam, Yuzhang Lin, Jingwei Xiong
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14960
• Pdf URL: https://arxiv.org/pdf/2501.14960
• Copy Paste: [[2501.14960]] LLM4DistReconfig: A Fine-tuned Large Language Model for Power Distribution Network Reconfiguration(https://arxiv.org/abs/2501.14960)
• Keywords: robust, large language model
• Abstract: Power distribution networks are evolving due to the integration of DERs and increased customer participation. To maintain optimal operation, minimize losses, and meet varying load demands, frequent network reconfiguration is necessary. Traditionally, the reconfiguration task relies on optimization software and expert operators, but as systems grow more complex, faster and more adaptive solutions are required without expert intervention. Data-driven reconfiguration is gaining traction for its accuracy, speed, and robustness against incomplete network data. LLMs, with their ability to capture complex patterns, offer a promising approach for efficient and responsive network reconfiguration in evolving complex power networks. In this work, we introduce LLM4DistReconfig, a deep learning-based approach utilizing a fine-tuned LLM to solve the distribution network reconfiguration problem. By carefully crafting prompts and designing a custom loss function, we train the LLM with inputs representing network parameters such as buses, available lines, open lines, node voltages, and system loss. The model then predicts optimal reconfigurations by outputting updated network configurations that minimize system loss while meeting operational constraints. Our approach significantly reduces inference time compared to classical algorithms, allowing for near real-time optimal reconfiguration after training. Experimental results show that our method generates optimal configurations minimizing system loss for five individual and a combined test dataset. It also produces minimal invalid edges, no cycles, or subgraphs across all datasets, fulfilling domain-specific needs. Additionally, the generated responses contain less than 5% improper outputs on seen networks and satisfactory results on unseen networks, demonstrating its effectiveness and reliability for the reconfiguration task.

Title: Personalized Layer Selection for Graph Neural Networks

• Authors: Kartik Sharma, Vineeth Rakesh Mohan, Yingtong Dou, Srijan Kumar, Mahashweta Das
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14964
• Pdf URL: https://arxiv.org/pdf/2501.14964
• Copy Paste: [[2501.14964]] Personalized Layer Selection for Graph Neural Networks(https://arxiv.org/abs/2501.14964)
• Keywords: attack, robust
• Abstract: Graph Neural Networks (GNNs) combine node attributes over a fixed granularity of the local graph structure around a node to predict its label. However, different nodes may relate to a node-level property with a different granularity of its local neighborhood, and using the same level of smoothing for all nodes can be detrimental to their classification. In this work, we challenge the common fact that a single GNN layer can classify all nodes of a graph by training GNNs with a distinct personalized layer for each node. Inspired by metric learning, we propose a novel algorithm, MetSelect1, to select the optimal representation layer to classify each node. In particular, we identify a prototype representation of each class in a transformed GNN layer and then, classify using the layer where the distance is smallest to a class prototype after normalizing with that layer's variance. Results on 10 datasets and 3 different GNNs show that we significantly improve the node classification accuracy of GNNs in a plug-and-play manner. We also find that using variable layers for prediction enables GNNs to be deeper and more robust to poisoning attacks. We hope this work can inspire future works to learn more adaptive and personalized graph representations.

Title: A Deep State Space Model for Rainfall-Runoff Simulations

• Authors: Yihan Wang, Lujun Zhang, Annan Yu, N. Benjamin Erichson, Tiantian Yang
• Subjects: cs.LG, cs.AI, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2501.14980
• Pdf URL: https://arxiv.org/pdf/2501.14980
• Copy Paste: [[2501.14980]] A Deep State Space Model for Rainfall-Runoff Simulations(https://arxiv.org/abs/2501.14980)
• Keywords: transformer
• Abstract: The classical way of studying the rainfall-runoff processes in the water cycle relies on conceptual or physically-based hydrologic models. Deep learning (DL) has recently emerged as an alternative and blossomed in hydrology community for rainfall-runoff simulations. However, the decades-old Long Short-Term Memory (LSTM) network remains the benchmark for this task, outperforming newer architectures like Transformers. In this work, we propose a State Space Model (SSM), specifically the Frequency Tuned Diagonal State Space Sequence (S4D-FT) model, for rainfall-runoff simulations. The proposed S4D-FT is benchmarked against the established LSTM and a physically-based Sacramento Soil Moisture Accounting model across 531 watersheds in the contiguous United States (CONUS). Results show that S4D-FT is able to outperform the LSTM model across diverse regions. Our pioneering introduction of the S4D-FT for rainfall-runoff simulations challenges the dominance of LSTM in the hydrology community and expands the arsenal of DL tools available for hydrological modeling.

Title: DepressionX: Knowledge Infused Residual Attention for Explainable Depression Severity Assessment

• Authors: Yusif Ibrahimov, Tarique Anwar, Tommy Yuan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14985
• Pdf URL: https://arxiv.org/pdf/2501.14985
• Copy Paste: [[2501.14985]] DepressionX: Knowledge Infused Residual Attention for Explainable Depression Severity Assessment(https://arxiv.org/abs/2501.14985)
• Keywords: explainability
• Abstract: In today's interconnected society, social media platforms have become an important part of our lives, where individuals virtually express their thoughts, emotions, and moods. These expressions offer valuable insights into their mental health. This paper explores the use of platforms like Facebook, $\mathbb{X}$ (formerly Twitter), and Reddit for mental health assessments. We propose a domain knowledge-infused residual attention model called DepressionX for explainable depression severity detection. Existing deep learning models on this problem have shown considerable performance, but they often lack transparency in their decision-making processes. In healthcare, where decisions are critical, the need for explainability is crucial. In our model, we address the critical gap by focusing on the explainability of depression severity detection while aiming for a high performance accuracy. In addition to being explainable, our model consistently outperforms the state-of-the-art models by over 7% in terms of $\text{F}_1$ score on balanced as well as imbalanced datasets. Our ultimate goal is to establish a foundation for trustworthy and comprehensible analysis of mental disorders via social media.

Title: Advances in Set Function Learning: A Survey of Techniques and Applications

• Authors: Jiahao Xie, Guangmo Tong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14991
• Pdf URL: https://arxiv.org/pdf/2501.14991
• Copy Paste: [[2501.14991]] Advances in Set Function Learning: A Survey of Techniques and Applications(https://arxiv.org/abs/2501.14991)
• Keywords: transformer
• Abstract: Set function learning has emerged as a crucial area in machine learning, addressing the challenge of modeling functions that take sets as inputs. Unlike traditional machine learning that involves fixed-size input vectors where the order of features matters, set function learning demands methods that are invariant to permutations of the input set, presenting a unique and complex problem. This survey provides a comprehensive overview of the current development in set function learning, covering foundational theories, key methodologies, and diverse applications. We categorize and discuss existing approaches, focusing on deep learning approaches, such as DeepSets and Set Transformer based methods, as well as other notable alternative methods beyond deep learning, offering a complete view of current models. We also introduce various applications and relevant datasets, such as point cloud processing and multi-label classification, highlighting the significant progress achieved by set function learning methods in these domains. Finally, we conclude by summarizing the current state of set function learning approaches and identifying promising future research directions, aiming to guide and inspire further advancements in this promising field.

Title: Federated Retrieval Augmented Generation for Multi-Product Question Answering

• Authors: Parshin Shojaee, Sai Sree Harsha, Dan Luo, Akash Maharaj, Tong Yu, Yunyao Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14998
• Pdf URL: https://arxiv.org/pdf/2501.14998
• Copy Paste: [[2501.14998]] Federated Retrieval Augmented Generation for Multi-Product Question Answering(https://arxiv.org/abs/2501.14998)
• Keywords: federate, large language model
• Abstract: Recent advancements in Large Language Models and Retrieval-Augmented Generation have boosted interest in domain-specific question-answering for enterprise products. However, AI Assistants often face challenges in multi-product QA settings, requiring accurate responses across diverse domains. Existing multi-domain RAG-QA approaches either query all domains indiscriminately, increasing computational costs and LLM hallucinations, or rely on rigid resource selection, which can limit search results. We introduce MKP-QA, a novel multi-product knowledge-augmented QA framework with probabilistic federated search across domains and relevant knowledge. This method enhances multi-domain search quality by aggregating query-domain and query-passage probabilistic relevance. To address the lack of suitable benchmarks for multi-product QAs, we also present new datasets focused on three Adobe products: Adobe Experience Platform, Target, and Customer Journey Analytics. Our experiments show that MKP-QA significantly boosts multi-product RAG-QA performance in terms of both retrieval accuracy and response quality.

Title: VideoPure: Diffusion-based Adversarial Purification for Video Recognition

• Authors: Kaixun Jiang, Zhaoyu Chen, Jiyuan Fu, Lingyi Hong, Jinglun Li, Wenqiang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14999
• Pdf URL: https://arxiv.org/pdf/2501.14999
• Copy Paste: [[2501.14999]] VideoPure: Diffusion-based Adversarial Purification for Video Recognition(https://arxiv.org/abs/2501.14999)
• Keywords: security, defense, attack, robust, diffusion
• Abstract: Recent work indicates that video recognition models are vulnerable to adversarial examples, posing a serious security risk to downstream applications. However, current research has primarily focused on adversarial attacks, with limited work exploring defense mechanisms. Furthermore, due to the spatial-temporal complexity of videos, existing video defense methods face issues of high cost, overfitting, and limited defense performance. Recently, diffusion-based adversarial purification methods have achieved robust defense performance in the image domain. However, due to the additional temporal dimension in videos, directly applying these diffusion-based adversarial purification methods to the video domain suffers performance and efficiency degradation. To achieve an efficient and effective video adversarial defense method, we propose the first diffusion-based video purification framework to improve video recognition models' adversarial robustness: VideoPure. Given an adversarial example, we first employ temporal DDIM inversion to transform the input distribution into a temporally consistent and trajectory-defined distribution, covering adversarial noise while preserving more video structure. Then, during DDIM denoising, we leverage intermediate results at each denoising step and conduct guided spatial-temporal optimization, removing adversarial noise while maintaining temporal consistency. Finally, we input the list of optimized intermediate results into the video recognition model for multi-step voting to obtain the predicted class. We investigate the defense performance of our method against black-box, gray-box, and adaptive attacks on benchmark datasets and models. Compared with other adversarial purification methods, our method overall demonstrates better defense performance against different attacks. Our code is available at this https URL.

Title: MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models

• Authors: Zhongpu Chen, Yinfeng Liu, Long Shi, Zhi-Jie Wang, Xingyan Chen, Yu Zhao, Fuji Ren
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.15000
• Pdf URL: https://arxiv.org/pdf/2501.15000
• Copy Paste: [[2501.15000]] MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models(https://arxiv.org/abs/2501.15000)
• Keywords: interpretability, large language model
• Abstract: Large language models (LLMs) are expected to offer structured Markdown responses for the sake of readability in web chatbots (e.g., ChatGPT). Although there are a myriad of metrics to evaluate LLMs, they fail to evaluate the readability from the view of output content structure. To this end, we focus on an overlooked yet important metric -- Markdown Awareness, which directly impacts the readability and structure of the content generated by these language models. In this paper, we introduce MDEval, a comprehensive benchmark to assess Markdown Awareness for LLMs, by constructing a dataset with 20K instances covering 10 subjects in English and Chinese. Unlike traditional model-based evaluations, MDEval provides excellent interpretability by combining model-based generation tasks and statistical methods. Our results demonstrate that MDEval achieves a Spearman correlation of 0.791 and an accuracy of 84.1% with human, outperforming existing methods by a large margin. Extensive experimental results also show that through fine-tuning over our proposed dataset, less performant open-source models are able to achieve comparable performance to GPT-4o in terms of Markdown Awareness. To ensure reproducibility and transparency, MDEval is open sourced at this https URL.

Title: Towards Distributed Backdoor Attacks with Network Detection in Decentralized Federated Learning

• Authors: Bohan Liu, Yang Xiao, Ruimeng Ye, Zinan Ling, Xiaolong Ma, Bo Hui
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15005
• Pdf URL: https://arxiv.org/pdf/2501.15005
• Copy Paste: [[2501.15005]] Towards Distributed Backdoor Attacks with Network Detection in Decentralized Federated Learning(https://arxiv.org/abs/2501.15005)
• Keywords: attack, federate
• Abstract: Distributed backdoor attacks (DBA) have shown a higher attack success rate than centralized attacks in centralized federated learning (FL). However, it has not been investigated in the decentralized FL. In this paper, we experimentally demonstrate that, while directly applying DBA to decentralized FL, the attack success rate depends on the distribution of attackers in the network architecture. Considering that the attackers can not decide their location, this paper aims to achieve a high attack success rate regardless of the attackers' location distribution. Specifically, we first design a method to detect the network by predicting the distance between any two attackers on the network. Then, based on the distance, we organize the attackers in different clusters. Lastly, we propose an algorithm to \textit{dynamically} embed local patterns decomposed from a global pattern into the different attackers in each cluster. We conduct a thorough empirical investigation and find that our method can, in benchmark datasets, outperform both centralized attacks and naive DBA in different decentralized frameworks.

Title: HuGDiffusion: Generalizable Single-Image Human Rendering via 3D Gaussian Diffusion

• Authors: Yingzhi Tang, Qijian Zhang, Junhui Hou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15008
• Pdf URL: https://arxiv.org/pdf/2501.15008
• Copy Paste: [[2501.15008]] HuGDiffusion: Generalizable Single-Image Human Rendering via 3D Gaussian Diffusion(https://arxiv.org/abs/2501.15008)
• Keywords: extraction, diffusion
• Abstract: We present HuGDiffusion, a generalizable 3D Gaussian splatting (3DGS) learning pipeline to achieve novel view synthesis (NVS) of human characters from single-view input images. Existing approaches typically require monocular videos or calibrated multi-view images as inputs, whose applicability could be weakened in real-world scenarios with arbitrary and/or unknown camera poses. In this paper, we aim to generate the set of 3DGS attributes via a diffusion-based framework conditioned on human priors extracted from a single image. Specifically, we begin with carefully integrated human-centric feature extraction procedures to deduce informative conditioning signals. Based on our empirical observations that jointly learning the whole 3DGS attributes is challenging to optimize, we design a multi-stage generation strategy to obtain different types of 3DGS attributes. To facilitate the training process, we investigate constructing proxy ground-truth 3D Gaussian attributes as high-quality attribute-level supervision signals. Through extensive experiments, our HuGDiffusion shows significant performance improvements over the state-of-the-art methods. Our code will be made publicly available.

Title: On Accelerating Edge AI: Optimizing Resource-Constrained Environments

• Authors: Jacob Sander, Achraf Cohen, Venkat R. Dasari, Brent Venable, Brian Jalaian
• Subjects: cs.LG, cs.AI, cs.NE
• Abstract URL: https://arxiv.org/abs/2501.15014
• Pdf URL: https://arxiv.org/pdf/2501.15014
• Copy Paste: [[2501.15014]] On Accelerating Edge AI: Optimizing Resource-Constrained Environments(https://arxiv.org/abs/2501.15014)
• Keywords: large language model
• Abstract: Resource-constrained edge deployments demand AI solutions that balance high performance with stringent compute, memory, and energy limitations. In this survey, we present a comprehensive overview of the primary strategies for accelerating deep learning models under such constraints. First, we examine model compression techniques-pruning, quantization, tensor decomposition, and knowledge distillation-that streamline large models into smaller, faster, and more efficient variants. Next, we explore Neural Architecture Search (NAS), a class of automated methods that discover architectures inherently optimized for particular tasks and hardware budgets. We then discuss compiler and deployment frameworks, such as TVM, TensorRT, and OpenVINO, which provide hardware-tailored optimizations at inference time. By integrating these three pillars into unified pipelines, practitioners can achieve multi-objective goals, including latency reduction, memory savings, and energy efficiency-all while maintaining competitive accuracy. We also highlight emerging frontiers in hierarchical NAS, neurosymbolic approaches, and advanced distillation tailored to large language models, underscoring open challenges like pre-training pruning for massive networks. Our survey offers practical insights, identifies current research gaps, and outlines promising directions for building scalable, platform-independent frameworks to accelerate deep learning models at the edge.

Title: Utilizing Graph Neural Networks for Effective Link Prediction in Microservice Architectures

• Authors: Ghazal Khodabandeh, Alireza Ezaz, Majid Babaei, Naser Ezzati-Jivan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15019
• Pdf URL: https://arxiv.org/pdf/2501.15019
• Copy Paste: [[2501.15019]] Utilizing Graph Neural Networks for Effective Link Prediction in Microservice Architectures(https://arxiv.org/abs/2501.15019)
• Keywords: robust, segmentation
• Abstract: Managing microservice architectures in distributed systems is complex and resource intensive due to the high frequency and dynamic nature of inter service interactions. Accurate prediction of these future interactions can enhance adaptive monitoring, enabling proactive maintenance and resolution of potential performance issues before they escalate. This study introduces a Graph Neural Network GNN based approach, specifically using a Graph Attention Network GAT, for link prediction in microservice Call Graphs. Unlike social networks, where interactions tend to occur sporadically and are often less frequent, microservice Call Graphs involve highly frequent and time sensitive interactions that are essential to operational performance. Our approach leverages temporal segmentation, advanced negative sampling, and GATs attention mechanisms to model these complex interactions accurately. Using real world data, we evaluate our model across performance metrics such as AUC, Precision, Recall, and F1 Score, demonstrating its high accuracy and robustness in predicting microservice interactions. Our findings support the potential of GNNs for proactive monitoring in distributed systems, paving the way for applications in adaptive resource management and performance optimization.

Title: AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models

• Authors: Zunhai Su, Wang Shen, Linge Li, Zhe Chen, Hanyu Wei, Huangqi Yu, Kehong Yuan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15021
• Pdf URL: https://arxiv.org/pdf/2501.15021
• Copy Paste: [[2501.15021]] AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models(https://arxiv.org/abs/2501.15021)
• Keywords: large language model
• Abstract: Vision-language models (VLMs) show remarkable performance in multimodal tasks. However, excessively long multimodal inputs lead to oversized Key-Value (KV) caches, resulting in significant memory consumption and I/O bottlenecks. Previous KV quantization methods for Large Language Models (LLMs) may alleviate these issues but overlook the attention saliency differences of multimodal tokens, resulting in suboptimal performance. In this paper, we investigate the attention-aware token saliency patterns in VLM and propose AKVQ-VL. AKVQ-VL leverages the proposed Text-Salient Attention (TSA) and Pivot-Token-Salient Attention (PSA) patterns to adaptively allocate bit budgets. Moreover, achieving extremely low-bit quantization requires effectively addressing outliers in KV tensors. AKVQ-VL utilizes the Walsh-Hadamard transform (WHT) to construct outlier-free KV caches, thereby reducing quantization difficulty. Evaluations of 2-bit quantization on 12 long-context and multimodal tasks demonstrate that AKVQ-VL maintains or even improves accuracy, outperforming LLM-oriented methods. AKVQ-VL can reduce peak memory usage by 2.13x, support up to 3.25x larger batch sizes and 2.46x throughput.

Title: Using Large Language Models for education managements in Vietnamese with low resources

• Authors: Duc Do Minh, Vinh Nguyen Van, Thang Dam Cong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15022
• Pdf URL: https://arxiv.org/pdf/2501.15022
• Copy Paste: [[2501.15022]] Using Large Language Models for education managements in Vietnamese with low resources(https://arxiv.org/abs/2501.15022)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs), such as GPT-4, Gemini 1.5, Claude 3.5 Sonnet, and Llama3, have demonstrated significant advancements in various NLP tasks since the release of ChatGPT in 2022. Despite their success, fine-tuning and deploying LLMs remain computationally expensive, especially in resource-constrained environments. In this paper, we proposed VietEduFrame, a framework specifically designed to apply LLMs to educational management tasks in Vietnamese institutions. Our key contribution includes the development of a tailored dataset, derived from student education documents at Hanoi VNU, which addresses the unique challenges faced by educational systems with limited resources. Through extensive experiments, we show that our approach outperforms existing methods in terms of accuracy and efficiency, offering a promising solution for improving educational management in under-resourced environments. While our framework leverages synthetic data to supplement real-world examples, we discuss potential limitations regarding broader applicability and robustness in future implementations.

Title: A Portable and Stealthy Inaudible Voice Attack Based on Acoustic Metamaterials

• Authors: Zhiyuan Ning, Juan He, Zhanyong Tang, Weihang Hu, Xiaojiang Chen
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15031
• Pdf URL: https://arxiv.org/pdf/2501.15031
• Copy Paste: [[2501.15031]] A Portable and Stealthy Inaudible Voice Attack Based on Acoustic Metamaterials(https://arxiv.org/abs/2501.15031)
• Keywords: attack, steal
• Abstract: We present METAATTACK, the first approach to leverage acoustic metamaterials for inaudible attacks for voice control systems. Compared to the state-of-the-art inaudible attacks requiring complex and large speaker setups, METAATTACK achieves a longer attacking range and higher accuracy using a compact, portable device small enough to be put into a carry bag. These improvements in portability and stealth have led to the practical applicability of inaudible attacks and their adaptation to a wider range of scenarios. We demonstrate how the recent advancement in metamaterials can be utilized to design a voice attack system with carefully selected implementation parameters and commercial off-the-shelf components. We showcase that METAATTACK can be used to launch inaudible attacks for representative voice-controlled personal assistants, including Siri, Alexa, Google Assistant, XiaoAI, and Xiaoyi. The average word accuracy of all assistants is 76%, with a range of 8.85 m.

Title: Semi-supervised Anomaly Detection with Extremely Limited Labels in Dynamic Graphs

• Authors: Jiazhen Chen, Sichao Fu, Zheng Ma, Mingbin Feng, Tony S. Wirjanto, Qinmu Peng
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15035
• Pdf URL: https://arxiv.org/pdf/2501.15035
• Copy Paste: [[2501.15035]] Semi-supervised Anomaly Detection with Extremely Limited Labels in Dynamic Graphs(https://arxiv.org/abs/2501.15035)
• Keywords: transformer
• Abstract: Semi-supervised graph anomaly detection (GAD) has recently received increasing attention, which aims to distinguish anomalous patterns from graphs under the guidance of a moderate amount of labeled data and a large volume of unlabeled data. Although these proposed semi-supervised GAD methods have achieved great success, their superior performance will be seriously degraded when the provided labels are extremely limited due to some unpredictable factors. Besides, the existing methods primarily focus on anomaly detection in static graphs, and little effort was paid to consider the continuous evolution characteristic of graphs over time (dynamic graphs). To address these challenges, we propose a novel GAD framework (EL$^{2}$-DGAD) to tackle anomaly detection problem in dynamic graphs with extremely limited labels. Specifically, a transformer-based graph encoder model is designed to more effectively preserve evolving graph structures beyond the local neighborhood. Then, we incorporate an ego-context hypersphere classification loss to classify temporal interactions according to their structure and temporal neighborhoods while ensuring the normal samples are mapped compactly against anomalous data. Finally, the above loss is further augmented with an ego-context contrasting module which utilizes unlabeled data to enhance model generalization. Extensive experiments on four datasets and three label rates demonstrate the effectiveness of the proposed method in comparison to the existing GAD methods.

Title: Adaptive Client Selection in Federated Learning: A Network Anomaly Detection Use Case

• Authors: William Marfo, Deepak K. Tosh, Shirley V. Moore
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15038
• Pdf URL: https://arxiv.org/pdf/2501.15038
• Copy Paste: [[2501.15038]] Adaptive Client Selection in Federated Learning: A Network Anomaly Detection Use Case(https://arxiv.org/abs/2501.15038)
• Keywords: privacy, robust, federate
• Abstract: Federated Learning (FL) has become a widely used approach for training machine learning models on decentralized data, addressing the significant privacy concerns associated with traditional centralized methods. However, the efficiency of FL relies on effective client selection and robust privacy preservation mechanisms. Ineffective client selection can result in suboptimal model performance, while inadequate privacy measures risk exposing sensitive data. This paper introduces a client selection framework for FL that incorporates differential privacy and fault tolerance. The proposed adaptive approach dynamically adjusts the number of selected clients based on model performance and system constraints, ensuring privacy through the addition of calibrated noise. The method is evaluated on a network anomaly detection use case using the UNSW-NB15 and ROAD datasets. Results demonstrate up to a 7% improvement in accuracy and a 25% reduction in training time compared to the FedL2P approach. Additionally, the study highlights trade-offs between privacy budgets and model performance, with higher privacy budgets leading to reduced noise and improved accuracy. While the fault tolerance mechanism introduces a slight performance decrease, it enhances robustness against client failures. Statistical validation using the Mann-Whitney U test confirms the significance of these improvements, with results achieving a p-value of less than 0.05.

Title: Towards Robust Unsupervised Attention Prediction in Autonomous Driving

• Authors: Mengshi Qi, Xiaoyang Bi, Pengfei Zhu, Huadong Ma
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15045
• Pdf URL: https://arxiv.org/pdf/2501.15045
• Copy Paste: [[2501.15045]] Towards Robust Unsupervised Attention Prediction in Autonomous Driving(https://arxiv.org/abs/2501.15045)
• Keywords: robust
• Abstract: Robustly predicting attention regions of interest for self-driving systems is crucial for driving safety but presents significant challenges due to the labor-intensive nature of obtaining large-scale attention labels and the domain gap between self-driving scenarios and natural scenes. These challenges are further exacerbated by complex traffic environments, including camera corruption under adverse weather, noise interferences, and central bias from long-tail distributions. To address these issues, we propose a robust unsupervised attention prediction method. An Uncertainty Mining Branch refines predictions by analyzing commonalities and differences across multiple pre-trained models on natural scenes, while a Knowledge Embedding Block bridges the domain gap by incorporating driving knowledge to adaptively enhance pseudo-labels. Additionally, we introduce RoboMixup, a novel data augmentation method that improves robustness against corruption through soft attention and dynamic augmentation, and mitigates central bias by integrating random cropping into Mixup as a this http URL systematically evaluate robustness in self-driving attention prediction, we introduce the DriverAttention-C benchmark, comprising over 100k frames across three subsets: BDD-A-C, DR(eye)VE-C, and DADA-2000-C. Our method achieves performance equivalent to or surpassing fully supervised state-of-the-art approaches on three public datasets and the proposed robustness benchmark, reducing relative corruption degradation by 58.8% and 52.8%, and improving central bias robustness by 12.4% and 11.4% in KLD and CC metrics, respectively. Code and data are available at this https URL.

Title: An Attempt to Unraveling Token Prediction Refinement and Identifying Essential Layers of Large Language Models

• Authors: Jaturong Kongmanee
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15054
• Pdf URL: https://arxiv.org/pdf/2501.15054
• Copy Paste: [[2501.15054]] An Attempt to Unraveling Token Prediction Refinement and Identifying Essential Layers of Large Language Models(https://arxiv.org/abs/2501.15054)
• Keywords: large language model
• Abstract: This research aims to unravel how large language models (LLMs) iteratively refine token predictions (or, in a general sense, vector predictions). We utilized a logit lens technique to analyze the model's token predictions derived from intermediate representations. Specifically, we focused on how LLMs access and use information from input contexts, and how positioning of relevant information affects the model's token prediction refinement process. Our findings for multi-document question answering task, by varying input context lengths (the number of documents), using GPT-2, revealed that the number of layers between the first layer that the model predicted next tokens correctly and the later layers that the model finalized its correct predictions, as a function of the position of relevant information (i.e., placing the relevant one at the beginning, middle, or end of the input context), has a nearly inverted U shape. We found that the gap between these two layers, on average, diminishes when relevant information is positioned at the beginning or end of the input context, suggesting that the model requires more refinements when processing longer contexts with relevant information situated in the middle, and highlighting which layers are essential for determining the correct output. Our analysis provides insights about how token predictions are distributed across different conditions, and establishes important connections to existing hypotheses and previous findings in AI safety research and development.

Title: KETA: Kinematic-Phrases-Enhanced Text-to-Motion Generation via Fine-grained Alignment

• Authors: Yu Jiang, Yixing Chen, Xingyang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15058
• Pdf URL: https://arxiv.org/pdf/2501.15058
• Copy Paste: [[2501.15058]] KETA: Kinematic-Phrases-Enhanced Text-to-Motion Generation via Fine-grained Alignment(https://arxiv.org/abs/2501.15058)
• Keywords: diffusion
• Abstract: Motion synthesis plays a vital role in various fields of artificial intelligence. Among the various conditions of motion generation, text can describe motion details elaborately and is easy to acquire, making text-to-motion(T2M) generation important. State-of-the-art T2M techniques mainly leverage diffusion models to generate motions with text prompts as guidance, tackling the many-to-many nature of T2M tasks. However, existing T2M approaches face challenges, given the gap between the natural language domain and the physical domain, making it difficult to generate motions fully consistent with the texts. We leverage kinematic phrases(KP), an intermediate representation that bridges these two modalities, to solve this. Our proposed method, KETA, decomposes the given text into several decomposed texts via a language model. It trains an aligner to align decomposed texts with the KP segments extracted from the generated motions. Thus, it's possible to restrict the behaviors for diffusion-based T2M models. During the training stage, we deploy the text-KP alignment loss as an auxiliary goal to supervise the models. During the inference stage, we refine our generated motions for multiple rounds in our decoder structure, where we compute the text-KP distance as the guidance signal in each new round. Experiments demonstrate that KETA achieves up to 1.19x, 2.34x better R precision and FID value on both backbones of the base model, motion diffusion model. Compared to a wide range of T2M generation models. KETA achieves either the best or the second-best performance.

Title: PolaFormer: Polarity-aware Linear Attention for Vision Transformers

• Authors: Weikang Meng, Yadan Luo, Xin Li, Dongmei Jiang, Zheng Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15061
• Pdf URL: https://arxiv.org/pdf/2501.15061
• Copy Paste: [[2501.15061]] PolaFormer: Polarity-aware Linear Attention for Vision Transformers(https://arxiv.org/abs/2501.15061)
• Keywords: transformer
• Abstract: Linear attention has emerged as a promising alternative to softmax-based attention, leveraging kernelized feature maps to reduce complexity from quadratic to linear in sequence length. However, the non-negative constraint on feature maps and the relaxed exponential function used in approximation lead to significant information loss compared to the original query-key dot products, resulting in less discriminative attention maps with higher entropy. To address the missing interactions driven by negative values in query-key pairs, we propose a polarity-aware linear attention mechanism that explicitly models both same-signed and opposite-signed query-key interactions, ensuring comprehensive coverage of relational information. Furthermore, to restore the spiky properties of attention maps, we provide a theoretical analysis proving the existence of a class of element-wise functions (with positive first and second derivatives) that can reduce entropy in the attention distribution. For simplicity, and recognizing the distinct contributions of each dimension, we employ a learnable power function for rescaling, allowing strong and weak attention signals to be effectively separated. Extensive experiments demonstrate that the proposed PolaFormer improves performance on various vision tasks, enhancing both expressiveness and efficiency by up to 4.6%.

Title: Exact Fit Attention in Node-Holistic Graph Convolutional Network for Improved EEG-Based Driver Fatigue Detection

• Authors: Meiyan Xu, Qingqing Chen, Duo Chen, Yi Ding, Jingyuan Wang, Peipei Gu, Yijie Pan, Deshuang Huang, Xun Zhang, Jiayang Guo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15062
• Pdf URL: https://arxiv.org/pdf/2501.15062
• Copy Paste: [[2501.15062]] Exact Fit Attention in Node-Holistic Graph Convolutional Network for Improved EEG-Based Driver Fatigue Detection(https://arxiv.org/abs/2501.15062)
• Keywords: interpretability
• Abstract: EEG-based fatigue monitoring can effectively reduce the incidence of related traffic accidents. In the past decade, with the advancement of deep learning, convolutional neural networks (CNN) have been increasingly used for EEG signal processing. However, due to the data's non-Euclidean characteristics, existing CNNs may lose important spatial information from EEG, specifically channel correlation. Thus, we propose the node-holistic graph convolutional network (NHGNet), a model that uses graphic convolution to dynamically learn each channel's features. With exact fit attention optimization, the network captures inter-channel correlations through a trainable adjacency matrix. The interpretability is enhanced by revealing critical areas of brain activity and their interrelations in various mental states. In validations on two public datasets, NHGNet outperforms the SOTAs. Specifically, in the intra-subject, NHGNet improved detection accuracy by at least 2.34% and 3.42%, and in the inter-subjects, it improved by at least 2.09% and 15.06%. Visualization research on the model revealed that the central parietal area plays an important role in detecting fatigue levels, whereas the frontal and temporal lobes are essential for maintaining vigilance.

Title: Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts

• Authors: Wenju Sun, Qingyong Li, Wen Wang, Yangli-ao Geng, Boyang Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15065
• Pdf URL: https://arxiv.org/pdf/2501.15065
• Copy Paste: [[2501.15065]] Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts(https://arxiv.org/abs/2501.15065)
• Keywords: robust
• Abstract: Multi-task model merging offers an efficient solution for integrating knowledge from multiple fine-tuned models, mitigating the significant computational and storage demands associated with multi-task training. As a key technique in this field, Task Arithmetic (TA) defines task vectors by subtracting the pre-trained model ($\theta_{\text{pre}}$) from the fine-tuned task models in parameter space, then adjusting the weight between these task vectors and $\theta_{\text{pre}}$ to balance task-generalized and task-specific knowledge. Despite the promising performance of TA, conflicts can arise among the task vectors, particularly when different tasks require distinct model adaptations. In this paper, we formally define this issue as knowledge conflicts, characterized by the performance degradation of one task after merging with a model fine-tuned for another task. Through in-depth analysis, we show that these conflicts stem primarily from the components of task vectors that align with the gradient of task-specific losses at $\theta_{\text{pre}}$. To address this, we propose Task Arithmetic in Trust Region (TATR), which defines the trust region as dimensions in the model parameter space that cause only small changes (corresponding to the task vector components with gradient orthogonal direction) in the task-specific losses. Restricting parameter merging within this trust region, TATR can effectively alleviate knowledge conflicts. Moreover, TATR serves as both an independent approach and a plug-and-play module compatible with a wide range of TA-based methods. Extensive empirical evaluations on eight distinct datasets robustly demonstrate that TATR improves the multi-task performance of several TA-based model merging methods by an observable margin.

Title: Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining

• Authors: Qisen Cheng, Jinming Xing, Chang Xue, Xiaoran Yang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15070
• Pdf URL: https://arxiv.org/pdf/2501.15070
• Copy Paste: [[2501.15070]] Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining(https://arxiv.org/abs/2501.15070)
• Keywords: robust, transformer
• Abstract: In this paper, we propose ShapTST, a framework that enables time-series transformers to efficiently generate Shapley-value-based explanations alongside predictions in a single forward pass. Shapley values are widely used to evaluate the contribution of different time-steps and features in a test sample, and are commonly generated through repeatedly inferring on each sample with different parts of information removed. Therefore, it requires expensive inference-time computations that occur at every request for model explanations. In contrast, our framework unifies the explanation and prediction in training through a novel Shapley-based pre-training design, which eliminates the undesirable test-time computation and replaces it with a single-time pre-training. Moreover, this specialized pre-training benefits the prediction performance by making the transformer model more effectively weigh different features and time-steps in the time-series, particularly improving the robustness against data noise that is common to raw time-series data. We experimentally validated our approach on eight public datasets, where our time-series model achieved competitive results in both classification and regression tasks, while providing Shapley-based explanations similar to those obtained with post-hoc computation. Our work offers an efficient and explainable solution for time-series analysis tasks in the safety-critical applications.

Title: SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos

• Authors: Yingying Jiao, Zhigang Wang, Sifan Wu, Shaojing Fan, Zhenguang Liu, Zhuoyue Xu, Zheqi Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15073
• Pdf URL: https://arxiv.org/pdf/2501.15073
• Copy Paste: [[2501.15073]] SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos(https://arxiv.org/abs/2501.15073)
• Keywords: robust
• Abstract: Human pose estimation in videos remains a challenge, largely due to the reliance on extensive manual annotation of large datasets, which is expensive and labor-intensive. Furthermore, existing approaches often struggle to capture long-range temporal dependencies and overlook the complementary relationship between temporal pose heatmaps and visual features. To address these limitations, we introduce STDPose, a novel framework that enhances human pose estimation by learning spatiotemporal dynamics in sparsely-labeled videos. STDPose incorporates two key innovations: 1) A novel Dynamic-Aware Mask to capture long-range motion context, allowing for a nuanced understanding of pose changes. 2) A system for encoding and aggregating spatiotemporal representations and motion dynamics to effectively model spatiotemporal relationships, improving the accuracy and robustness of pose estimation. STDPose establishes a new performance benchmark for both video pose propagation (i.e., propagating pose annotations from labeled frames to unlabeled frames) and pose estimation tasks, across three large-scale evaluation datasets. Additionally, utilizing pseudo-labels generated by pose propagation, STDPose achieves competitive performance with only 26.7% labeled data.

Title: PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures

• Authors: Shreya Shukla, Nakul Sharma, Manish Gupta, Anand Mishra
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15074
• Pdf URL: https://arxiv.org/pdf/2501.15074
• Copy Paste: [[2501.15074]] PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures(https://arxiv.org/abs/2501.15074)
• Keywords: protect, large language model
• Abstract: Writing comprehensive and accurate descriptions of technical drawings in patent documents is crucial to effective knowledge sharing and enabling the replication and protection of intellectual property. However, automation of this task has been largely overlooked by the research community. To this end, we introduce PatentDesc-355K, a novel large-scale dataset containing ~355K patent figures along with their brief and detailed textual descriptions extracted from more than 60K US patent documents. In addition, we propose PatentLMM - a novel multimodal large language model specifically tailored to generate high-quality descriptions of patent figures. Our proposed PatentLMM comprises two key components: (i) PatentMME, a specialized multimodal vision encoder that captures the unique structural elements of patent figures, and (ii) PatentLLaMA, a domain-adapted version of LLaMA fine-tuned on a large collection of patents. Extensive experiments demonstrate that training a vision encoder specifically designed for patent figures significantly boosts the performance, generating coherent descriptions compared to fine-tuning similar-sized off-the-shelf multimodal models. PatentDesc-355K and PatentLMM pave the way for automating the understanding of patent figures, enabling efficient knowledge sharing and faster drafting of patent documents. We make the code and data publicly available.

Title: Cryptanalysis via Machine Learning Based Information Theoretic Metrics

• Authors: Benjamin D. Kim, Vipindev Adat Vasudevan, Rafael G. L. D'Oliveira, Alejandro Cohen, Thomas Stahlbuhk, Muriel Médard
• Subjects: cs.CR, cs.IT, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15076
• Pdf URL: https://arxiv.org/pdf/2501.15076
• Copy Paste: [[2501.15076]] Cryptanalysis via Machine Learning Based Information Theoretic Metrics(https://arxiv.org/abs/2501.15076)
• Keywords: secure, attack, robust
• Abstract: The fields of machine learning (ML) and cryptanalysis share an interestingly common objective of creating a function, based on a given set of inputs and outputs. However, the approaches and methods in doing so vary vastly between the two fields. In this paper, we explore integrating the knowledge from the ML domain to provide empirical evaluations of cryptosystems. Particularly, we utilize information theoretic metrics to perform ML-based distribution estimation. We propose two novel applications of ML algorithms that can be applied in a known plaintext setting to perform cryptanalysis on any cryptosystem. We use mutual information neural estimation to calculate a cryptosystem's mutual information leakage, and a binary cross entropy classification to model an indistinguishability under chosen plaintext attack (CPA). These algorithms can be readily applied in an audit setting to evaluate the robustness of a cryptosystem and the results can provide a useful empirical bound. We evaluate the efficacy of our methodologies by empirically analyzing several encryption schemes. Furthermore, we extend the analysis to novel network coding-based cryptosystems and provide other use cases for our algorithms. We show that our classification model correctly identifies the encryption schemes that are not IND-CPA secure, such as DES, RSA, and AES ECB, with high accuracy. It also identifies the faults in CPA-secure cryptosystems with faulty parameters, such a reduced counter version of AES-CTR. We also conclude that with our algorithms, in most cases a smaller-sized neural network using less computing power can identify vulnerabilities in cryptosystems, providing a quick check of the sanity of the cryptosystem and help to decide whether to spend more resources to deploy larger networks that are able to break the cryptosystem.

Title: NetChain: Authenticated Blockchain Top-k Graph Data Queries and its Application in Asset Management

• Authors: Hongguang Zhao, Xu Yang, Saiyu Qi, Qiuhao Wang, Ke Li
• Subjects: cs.CR, cs.DB
• Abstract URL: https://arxiv.org/abs/2501.15077
• Pdf URL: https://arxiv.org/pdf/2501.15077
• Copy Paste: [[2501.15077]] NetChain: Authenticated Blockchain Top-k Graph Data Queries and its Application in Asset Management(https://arxiv.org/abs/2501.15077)
• Keywords: security
• Abstract: As a valuable digital resource, graph data is an important data asset, which has been widely utilized across various fields to optimize decision-making and enable smarter solutions. To manage data assets, blockchain is widely used to enable data sharing and trading, but it cannot supply complex analytical queries. vChain was proposed to achieve verifiable boolean queries over blockchain by designing an embedded authenticated data structure (ADS). However, for generating (non-)existence proofs, vChain suffers from expensive storage and computation costs in ADS construction, along with high communication and verification costs. In this paper, we propose a novel NetChain framework that enables efficient top-k queries over on-chain graph data with verifiability. Specifically, we design a novel authenticated two-layer index that supports (non-)existence proof generation in block-level and built-in verifiability for matched objects. To further alleviate the computation and verification overhead, an optimized variant NetChain+ is derived. The authenticity of our frameworks is validated through security analysis. Evaluations show that NetChain and NetChain+ outperform vChain, respectively achieving up to 85X and 31X improvements on ADS construction. Moreover, compared with vChain, NetChain+ reduces the communication and verification costs by 87% and 96% respectively.

Title: Hierarchical Pattern Decryption Methodology for Ransomware Detection Using Probabilistic Cryptographic Footprints

• Authors: Kevin Pekepok, Persephone Kirkwood, Esme Christopolous, Florence Braithwaite, Oliver Nightingale
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15084
• Pdf URL: https://arxiv.org/pdf/2501.15084
• Copy Paste: [[2501.15084]] Hierarchical Pattern Decryption Methodology for Ransomware Detection Using Probabilistic Cryptographic Footprints(https://arxiv.org/abs/2501.15084)
• Keywords: robust
• Abstract: The increasing sophistication of encryption-based ransomware has demanded innovative approaches to detection and mitigation, prompting the development of a hierarchical framework grounded in probabilistic cryptographic analysis. By focusing on the statistical characteristics of encryption patterns, the proposed methodology introduces a layered approach that combines advanced clustering algorithms with machine learning to isolate ransomware-induced anomalies. Through comprehensive testing across diverse ransomware families, the framework demonstrated exceptional accuracy, effectively distinguishing malicious encryption operations from benign activities while maintaining low false positive rates. The system's design integrates dynamic feedback mechanisms, enabling adaptability to varying cryptographic complexities and operational environments. Detailed entropy-based evaluations revealed its sensitivity to subtle deviations in encryption workflows, offering a robust alternative to traditional detection methods reliant on static signatures or heuristics. Computational benchmarks confirmed its scalability and efficiency, achieving consistent performance even under high data loads and complex cryptographic scenarios. The inclusion of real-time clustering and anomaly evaluation ensures rapid response capabilities, addressing critical latency challenges in ransomware detection. Performance comparisons with established methods highlighted its improvements in detection efficacy, particularly against advanced ransomware employing extended key lengths and unique cryptographic protocols.

Title: LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion

• Authors: Zhan Ling, Kang Liu, Kai Yan, Yifan Yang, Weijian Lin, Ting-Han Fan, Lingfeng Shen, Zhengyin Du, Jiecao Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15089
• Pdf URL: https://arxiv.org/pdf/2501.15089
• Copy Paste: [[2501.15089]] LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion(https://arxiv.org/abs/2501.15089)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) have demonstrated remarkable progress in understanding long-context inputs. However, benchmarks for evaluating the long-context reasoning abilities of LLMs fall behind the pace. Existing benchmarks often focus on a narrow range of tasks or those that do not demand complex reasoning. To address this gap and enable a more comprehensive evaluation of the long-context reasoning capabilities of current LLMs, we propose a new synthetic benchmark, LongReason, which is constructed by synthesizing long-context reasoning questions from a varied set of short-context reasoning questions through context expansion. LongReason consists of 794 multiple-choice reasoning questions with diverse reasoning patterns across three task categories: reading comprehension, logical inference, and mathematical word problems. We evaluate 21 LLMs on LongReason, revealing that most models experience significant performance drops as context length increases. Our further analysis shows that even state-of-the-art LLMs still have significant room for improvement in providing robust reasoning across different tasks. We will open-source LongReason to support the comprehensive evaluation of LLMs' long-context reasoning capabilities.

Title: Speech Translation Refinement using Large Language Models

• Authors: Huaixia Dou, Xinyu Tian, Xinglin Lyu, Jie Zhu, Junhui Li, Lifan Guo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15090
• Pdf URL: https://arxiv.org/pdf/2501.15090
• Copy Paste: [[2501.15090]] Speech Translation Refinement using Large Language Models(https://arxiv.org/abs/2501.15090)
• Keywords: large language model
• Abstract: Recent advancements in large language models (LLMs) have demonstrated their remarkable capabilities across various language tasks. Inspired by the success of text-to-text translation refinement, this paper investigates how LLMs can improve the performance of speech translation by introducing a joint refinement process. Through the joint refinement of speech translation (ST) and automatic speech recognition (ASR) transcription via LLMs, the performance of the ST model is significantly improved in both training-free in-context learning and parameter-efficient fine-tuning scenarios. Additionally, we explore the effect of document-level context on refinement under the context-aware fine-tuning scenario. Experimental results on the MuST-C and CoVoST 2 datasets, which include seven translation tasks, demonstrate the effectiveness of the proposed approach using several popular LLMs including GPT-3.5-turbo, LLaMA3-8B, and Mistral-12B. Further analysis further suggests that jointly refining both transcription and translation yields better performance compared to refining translation alone. Meanwhile, incorporating document-level context significantly enhances refinement performance. We release our code and datasets on GitHub.

Title: Towards Better Robustness: Progressively Joint Pose-3DGS Learning for Arbitrarily Long Videos

• Authors: Zhen-Hui Dong, Sheng Ye, Yu-Hui Wen, Nannan Li, Yong-Jin Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15096
• Pdf URL: https://arxiv.org/pdf/2501.15096
• Copy Paste: [[2501.15096]] Towards Better Robustness: Progressively Joint Pose-3DGS Learning for Arbitrarily Long Videos(https://arxiv.org/abs/2501.15096)
• Keywords: robust
• Abstract: 3D Gaussian Splatting (3DGS) has emerged as a powerful representation due to its efficiency and high-fidelity rendering. However, 3DGS training requires a known camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Pioneering works have attempted to relax this restriction but still face difficulties when handling long sequences with complex camera trajectories. In this work, we propose Rob-GS, a robust framework to progressively estimate camera poses and optimize 3DGS for arbitrarily long video sequences. Leveraging the inherent continuity of videos, we design an adjacent pose tracking method to ensure stable pose estimation between consecutive frames. To handle arbitrarily long inputs, we adopt a "divide and conquer" scheme that adaptively splits the video sequence into several segments and optimizes them separately. Extensive experiments on the Tanks and Temples dataset and our collected real-world dataset show that our Rob-GS outperforms the state-of-the-arts.

Title: CFT-RAG: An Entity Tree Based Retrieval Augmented Generation Algorithm With Cuckoo Filter

• Authors: Zihang Li, Yangdong Ruan, Wenjun Liu, Zhengyang Wang, Tong Yang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15098
• Pdf URL: https://arxiv.org/pdf/2501.15098
• Copy Paste: [[2501.15098]] CFT-RAG: An Entity Tree Based Retrieval Augmented Generation Algorithm With Cuckoo Filter(https://arxiv.org/abs/2501.15098)
• Keywords: generative
• Abstract: Although retrieval-augmented generation(RAG) significantly improves generation quality by retrieving external knowledge bases and integrating generated content, it faces computational efficiency bottlenecks, particularly in knowledge retrieval tasks involving hierarchical structures for Tree-RAG. This paper proposes a Tree-RAG acceleration method based on the improved Cuckoo Filter, which optimizes entity localization during the retrieval process to achieve significant performance improvements. Tree-RAG effectively organizes entities through the introduction of a hierarchical tree structure, while the Cuckoo Filter serves as an efficient data structure that supports rapid membership queries and dynamic updates. The experiment results demonstrate that our method is much faster than naive Tree-RAG while maintaining high levels of generative quality. When the number of trees is large, our method is hundreds of times faster than naive Tree-RAG. Our work is available at this https URL.

Title: Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection

• Authors: Shengdong Zhang, Xiaoqin Zhang, Wenqi Ren, Linlin Shen, Shaohua Wan, Jun Zhang, Yujing M Jiang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15099
• Pdf URL: https://arxiv.org/pdf/2501.15099
• Copy Paste: [[2501.15099]] Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection(https://arxiv.org/abs/2501.15099)
• Keywords: robust
• Abstract: Ensuring a stable power supply in rural areas relies heavily on effective inspection of power equipment, particularly transmission lines (TLs). However, detecting TLs from aerial imagery can be challenging when dealing with misalignments between visible light (RGB) and infrared (IR) images, as well as mismatched high- and low-level features in convolutional networks. To address these limitations, we propose a novel Hierarchical Multi-Modal Enhancement Network (HMMEN) that integrates RGB and IR data for robust and accurate TL detection. Our method introduces two key components: (1) a Mutual Multi-Modal Enhanced Block (MMEB), which fuses and enhances hierarchical RGB and IR feature maps in a coarse-to-fine manner, and (2) a Feature Alignment Block (FAB) that corrects misalignments between decoder outputs and IR feature maps by leveraging deformable convolutions. We employ MobileNet-based encoders for both RGB and IR inputs to accommodate edge-computing constraints and reduce computational overhead. Experimental results on diverse weather and lighting conditionsfog, night, snow, and daytimedemonstrate the superiority and robustness of our approach compared to state-of-the-art methods, resulting in fewer false positives, enhanced boundary delineation, and better overall detection performance. This framework thus shows promise for practical large-scale power line inspections with unmanned aerial vehicles.

Title: Comprehensive Evaluation of Cloaking Backdoor Attacks on Object Detector in Real-World

• Authors: Hua Ma, Alsharif Abuadbba, Yansong Gao, Hyoungshick Kim, Surya Nepal
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15101
• Pdf URL: https://arxiv.org/pdf/2501.15101
• Copy Paste: [[2501.15101]] Comprehensive Evaluation of Cloaking Backdoor Attacks on Object Detector in Real-World(https://arxiv.org/abs/2501.15101)
• Keywords: attack, robust
• Abstract: The exploration of backdoor vulnerabilities in object detectors, particularly in real-world scenarios, remains limited. A significant challenge lies in the absence of a natural physical backdoor dataset, and constructing such a dataset is both time- and labor-intensive. In this work, we address this gap by creating a large-scale dataset comprising approximately 11,800 images/frames with annotations featuring natural objects (e.g., T-shirts and hats) as triggers to incur cloaking adversarial effects in diverse real-world scenarios. This dataset is tailored for the study of physical backdoors in object detectors. Leveraging this dataset, we conduct a comprehensive evaluation of an insidious cloaking backdoor effect against object detectors, wherein the bounding box around a person vanishes when the individual is near a natural object (e.g., a commonly available T-shirt) in front of the detector. Our evaluations encompass three prevalent attack surfaces: data outsourcing, model outsourcing, and the use of pretrained models. The cloaking effect is successfully implanted in object detectors across all three attack surfaces. We extensively evaluate four popular object detection algorithms (anchor-based Yolo-V3, Yolo-V4, Faster R-CNN, and anchor-free CenterNet) using 19 videos (totaling approximately 11,800 frames) in real-world scenarios. Our results demonstrate that the backdoor attack exhibits remarkable robustness against various factors, including movement, distance, angle, non-rigid deformation, and lighting. In data and model outsourcing scenarios, the attack success rate (ASR) in most videos reaches 100% or near it, while the clean data accuracy of the backdoored model remains indistinguishable from that of the clean model, making it impossible to detect backdoor behavior through a validation set.

Title: Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning

• Authors: Ziyu Zhao, Yixiao Zhou, Didi Zhu, Tao Shen, Xuwu Wang, Jing Su, Kun Kuang, Zhongyu Wei, Fei Wu, Yu Cheng
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15103
• Pdf URL: https://arxiv.org/pdf/2501.15103
• Copy Paste: [[2501.15103]] Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning(https://arxiv.org/abs/2501.15103)
• Keywords: large language model
• Abstract: Low-Rank Adaptation (LoRA) is widely used for adapting large language models (LLMs) to specific domains due to its efficiency and modularity. Meanwhile, vanilla LoRA struggles with task conflicts in multi-task scenarios. Recent works adopt Mixture of Experts (MoE) by treating each LoRA module as an expert, thereby mitigating task interference through multiple specialized LoRA modules. While effective, these methods often isolate knowledge within individual tasks, failing to fully exploit the shared knowledge across related tasks. In this paper, we establish a connection between single LoRA and multi-LoRA MoE, integrating them into a unified framework. We demonstrate that the dynamic routing of multiple LoRAs is functionally equivalent to rank partitioning and block-level activation within a single LoRA. We further empirically demonstrate that finer-grained LoRA partitioning, within the same total and activated parameter constraints, leads to better performance gains across heterogeneous tasks. Building on these findings, we propose Single-ranked Mixture of Experts LoRA (\textbf{SMoRA}), which embeds MoE into LoRA by \textit{treating each rank as an independent expert}. With a \textit{dynamic rank-wise activation} mechanism, SMoRA promotes finer-grained knowledge sharing while mitigating task conflicts. Experiments demonstrate that SMoRA activates fewer parameters yet achieves better performance in multi-task scenarios.

Title: Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Training

• Authors: Xunxin Cai, Chengrui Wang, Qingqing Long, Yuanchun Zhou, Meng Xiao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15108
• Pdf URL: https://arxiv.org/pdf/2501.15108
• Copy Paste: [[2501.15108]] Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Training(https://arxiv.org/abs/2501.15108)
• Keywords: large language model
• Abstract: The rapid advancement of large language models (LLMs) in biological-medical applications has highlighted a gap between their potential and the limited scale and often low quality of available open-source annotated textual datasets. In addition, the inherent complexity of the biomedical knowledge hierarchy significantly hampers efforts to bridge this this http URL LLMs themselves play a pivotal role in overcoming this limitation? Motivated by this question, we investigate this challenge in the present this http URL propose a framework that automates the distillation of high-quality textual training data from the extensive scientific literature. Our approach self-evaluates and generates questions that are more closely aligned with the biomedical domain, guided by the biomedical knowledge hierarchy through medical subject headings (MeSH). This comprehensive framework establishes an automated workflow, thereby eliminating the need for manual intervention. Furthermore, we conducted comprehensive experiments to evaluate the impact of our framework-generated data on downstream language models of varying sizes. Our approach substantially improves question-answering tasks compared to pre-trained models from the life sciences domain and powerful close-source models represented by GPT-4. Notably, the generated AI-Ready dataset enabled the Llama3-70B base model to outperform GPT-4 using MedPrompt with multiple times the number of parameters. Detailed case studies and ablation experiments underscore the significance of each component within our framework

Title: HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understanding

• Authors: Jiaxing Zhao, Qize Yang, Yixing Peng, Detao Bai, Shimin Yao, Boyuan Sun, Xiang Chen, Shenghao Fu, Weixuan chen, Xihan Wei, Liefeng Bo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15111
• Pdf URL: https://arxiv.org/pdf/2501.15111
• Copy Paste: [[2501.15111]] HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understanding(https://arxiv.org/abs/2501.15111)
• Keywords: large language model
• Abstract: In human-centric scenes, the ability to simultaneously understand visual and auditory information is crucial. While recent omni models can process multiple modalities, they generally lack effectiveness in human-centric scenes due to the absence of large-scale, specialized datasets and non-targeted architectures. In this work, we developed HumanOmni, the industry's first human-centric Omni-multimodal large language model. We constructed a dataset containing over 2.4 million human-centric video clips with detailed captions and more than 14 million instructions, facilitating the understanding of diverse human-centric scenes. HumanOmni includes three specialized branches for understanding different types of scenes. It adaptively fuses features from these branches based on user instructions, significantly enhancing visual understanding in scenes centered around individuals. Moreover, HumanOmni integrates audio features to ensure a comprehensive understanding of environments and individuals. Our experiments validate HumanOmni's advanced capabilities in handling human-centric scenes across a variety of tasks, including emotion recognition, facial expression description, and action understanding. Our model will be open-sourced to facilitate further development and collaboration within both academia and industry.

Title: Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads

• Authors: Xingyang He, Jie Liu, Shaowei Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15113
• Pdf URL: https://arxiv.org/pdf/2501.15113
• Copy Paste: [[2501.15113]] Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads(https://arxiv.org/abs/2501.15113)
• Keywords: large language model
• Abstract: KV cache is a widely used acceleration technique for large language models (LLMs) inference. However, its memory requirement grows rapidly with input length. Previous studies have reduced the size of KV cache by either removing the same number of unimportant tokens for all attention heads or by allocating differentiated KV cache budgets for pre-identified attention heads. However, due to the importance of attention heads varies across different tasks, the pre-identified attention heads fail to adapt effectively to various downstream tasks. To address this issue, we propose Task-KV, a method that leverages the semantic differentiation of attention heads to allocate differentiated KV cache budgets across various tasks. We demonstrate that attention heads far from the semantic center (called heterogeneous heads) make an significant contribution to task outputs and semantic understanding. In contrast, other attention heads play the role of aggregating important information and focusing reasoning. Task-KV allocates full KV cache budget to heterogeneous heads to preserve comprehensive semantic information, while reserving a small number of recent tokens and attention sinks for non-heterogeneous heads. Furthermore, we innovatively introduce middle activations to preserve key contextual information aggregated from non-heterogeneous heads. To dynamically perceive semantic differences among attention heads, we design a semantic separator to distinguish heterogeneous heads from non-heterogeneous ones based on their distances from the semantic center. Experimental results on multiple benchmarks and different model architectures demonstrate that Task-KV significantly outperforms existing baseline methods.

Title: TranStable: Towards Robust Pixel-level Online Video Stabilization by Jointing Transformer and CNN

• Authors: zhizhen li, tianyi zhuo, Yifei Cao, Jizhe Yu, Yu Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15138
• Pdf URL: https://arxiv.org/pdf/2501.15138
• Copy Paste: [[2501.15138]] TranStable: Towards Robust Pixel-level Online Video Stabilization by Jointing Transformer and CNN(https://arxiv.org/abs/2501.15138)
• Keywords: robust, transformer
• Abstract: Video stabilization often struggles with distortion and excessive cropping. This paper proposes a novel end-to-end framework, named TranStable, to address these challenges, comprising a genera tor and a discriminator. We establish TransformerUNet (TUNet) as the generator to utilize the Hierarchical Adaptive Fusion Module (HAFM), integrating Transformer and CNN to leverage both global and local features across multiple visual cues. By modeling frame-wise relationships, it generates robust pixel-level warping maps for stable geometric transformations. Furthermore, we design the Stability Discriminator Module (SDM), which provides pixel-wise supervision for authenticity and consistency in training period, ensuring more complete field-of-view while minimizing jitter artifacts and enhancing visual fidelity. Extensive experiments on NUS, DeepStab, and Selfie benchmarks demonstrate state-of-the-art performance.

Title: Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models

• Authors: Hulingxiao He, Geng Li, Zijun Geng, Jinglin Xu, Yuxin Peng
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15140
• Pdf URL: https://arxiv.org/pdf/2501.15140
• Copy Paste: [[2501.15140]] Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models(https://arxiv.org/abs/2501.15140)
• Keywords: extraction, large language model
• Abstract: Multi-modal large language models (MLLMs) have shown remarkable abilities in various visual understanding tasks. However, MLLMs still struggle with fine-grained visual recognition (FGVR), which aims to identify subordinate-level categories from images. This can negatively impact more advanced capabilities of MLLMs, such as object-centric visual question answering and reasoning. In our study, we revisit three quintessential capabilities of MLLMs for FGVR, including object information extraction, category knowledge reserve, object-category alignment, and position of the root cause as a misalignment problem. To address this issue, we present Finedefics, an MLLM that enhances the model's FGVR capability by incorporating informative attribute descriptions of objects into the training phase. We employ contrastive learning on object-attribute pairs and attribute-category pairs simultaneously and use examples from similar but incorrect categories as hard negatives, naturally bringing representations of visual objects and category names closer. Extensive evaluations across multiple popular FGVR datasets demonstrate that Finedefics outperforms existing MLLMs of comparable parameter sizes, showcasing its remarkable efficacy. The code is available at this https URL.

Title: Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Models

• Authors: Ankit Yadav, Lingqiao Liu, Yuankai Qi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15144
• Pdf URL: https://arxiv.org/pdf/2501.15144
• Copy Paste: [[2501.15144]] Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Models(https://arxiv.org/abs/2501.15144)
• Keywords: robust
• Abstract: This work investigates the capabilities of current vision-language models (VLMs) in visual understanding and attribute measurement of primitive shapes using a benchmark focused on controlled 2D shape configurations with variations in spatial positioning, occlusion, rotation, size, and shape attributes such as type, quadrant, center-coordinates, rotation, occlusion status, and color as shown in Figure 1 and supplementary Figures S3-S81. We fine-tune state-of-the-art VLMs (2B-8B parameters) using Low-Rank Adaptation (LoRA) and validate them on multiple out-of-domain (OD) scenarios from our proposed benchmark. Our findings reveal that coherent sentence-based outputs outperform tuple formats, particularly in OD scenarios with large domain gaps. Additionally, we demonstrate that scaling numeric tokens during loss computation enhances numerical approximation capabilities, further improving performance on spatial and measurement tasks. These results highlight the importance of output format design, loss scaling strategies, and robust generalization techniques in enhancing the training and fine-tuning of VLMs, particularly for tasks requiring precise spatial approximations and strong OD generalization.

Title: PromptShield: Deployable Detection for Prompt Injection Attacks

• Authors: Dennis Jacob, Hend Alzahrani, Zhanhao Hu, Basel Alomair, David Wagner
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15145
• Pdf URL: https://arxiv.org/pdf/2501.15145
• Copy Paste: [[2501.15145]] PromptShield: Deployable Detection for Prompt Injection Attacks(https://arxiv.org/abs/2501.15145)
• Keywords: attack, large language model
• Abstract: Current application designers have moved to integrate large language models (LLMs) into their products. These LLM-integrated applications are vulnerable to prompt injection vulnerabilities. While attempts have been made to address this problem by building a detector that can monitor inputs to the LLM and detect attacks, we find that many detectors are not yet suitable for practical deployment. To support research in this area, we design the PromptShield benchmark for evaluating practical prompt injection detectors. We also construct a new detector, the PromptShield detector, which achieves significantly better performance at detecting prompt injection attacks than any prior scheme. Our work suggests that larger models, more training data, appropriate metrics, and careful curation of training data can contribute to strong detector performance.

Title: SpikSSD: Better Extraction and Fusion for Object Detection with Spiking Neuron Networks

• Authors: Yimeng Fan, Chagsong Liu, Mingyang Li, Wei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15151
• Pdf URL: https://arxiv.org/pdf/2501.15151
• Copy Paste: [[2501.15151]] SpikSSD: Better Extraction and Fusion for Object Detection with Spiking Neuron Networks(https://arxiv.org/abs/2501.15151)
• Keywords: extraction, interpretability
• Abstract: As the third generation of neural networks, Spiking Neural Networks (SNNs) have gained widespread attention due to their low energy consumption and biological interpretability. Recently, SNNs have made considerable advancements in computer vision. However, efficiently conducting feature extraction and fusion under the spiking characteristics of SNNs for object detection remains a pressing challenge. To address this problem, we propose the SpikSSD, a novel Spiking Single Shot Multibox Detector. Specifically, we design a full-spiking backbone network, MDS-ResNet, which effectively adjusts the membrane synaptic input distribution at each layer, achieving better spiking feature extraction. Additionally, for spiking feature fusion, we introduce the Spiking Bi-direction Fusion Module (SBFM), which for the first time realizes bi-direction fusion of spiking features, enhancing the multi-scale detection capability of the model. Experimental results show that SpikSSD achieves 40.8\% mAP on the GEN1 dataset, 76.3\% and 52.4\% mAP@0.5 on VOC 2007 and COCO 2017 datasets respectively with the lowest firing rate, outperforming existing SNN-based approaches at ultralow energy consumption. This work sets a new benchmark for future research in SNN-based object detection. Our code is publicly available in this https URL.

Title: Enhancing Intent Understanding for Ambiguous Prompts through Human-Machine Co-Adaptation

• Authors: Yangfan He, Jianhui Wang, Kun Li, Yijin Wang, Li Sun, Jun Yin, Miao Zhang, Xueqian Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15167
• Pdf URL: https://arxiv.org/pdf/2501.15167
• Copy Paste: [[2501.15167]] Enhancing Intent Understanding for Ambiguous Prompts through Human-Machine Co-Adaptation(https://arxiv.org/abs/2501.15167)
• Keywords: diffusion
• Abstract: Modern image generation systems can produce high-quality visuals, yet user prompts often contain ambiguities, requiring multiple revisions. Existing methods struggle to address the nuanced needs of non-expert users. We propose Visual Co-Adaptation (VCA), a novel framework that iteratively refines prompts and aligns generated images with user preferences. VCA employs a fine-tuned language model with reinforcement learning and multi-turn dialogues for prompt disambiguation. Key components include the Incremental Context-Enhanced Dialogue Block for interactive clarification, the Semantic Exploration and Disambiguation Module (SESD) leveraging Retrieval-Augmented Generation (RAG) and CLIP scoring, and the Pixel Precision and Consistency Optimization Module (PPCO) for refining image details using Proximal Policy Optimization (PPO). A human-in-the-loop feedback mechanism further improves performance. Experiments show that VCA surpasses models like DALL-E 3 and Stable Diffusion, reducing dialogue rounds to 4.3, achieving a CLIP score of 0.92, and enhancing user satisfaction to 4.73/5. Additionally, we introduce a novel multi-round dialogue dataset with prompt-image pairs and user intent annotations.

Title: Option-ID Based Elimination For Multiple Choice Questions

• Authors: Zhenhao Zhu, Bulou Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15175
• Pdf URL: https://arxiv.org/pdf/2501.15175
• Copy Paste: [[2501.15175]] Option-ID Based Elimination For Multiple Choice Questions(https://arxiv.org/abs/2501.15175)
• Keywords: large language model
• Abstract: Multiple choice questions (MCQs) are a common and important task for evaluating large language models (LLMs). Based on common strategies humans use when answering MCQs, the process of elimination has been proposed as an effective problem-solving method. Existing methods to the process of elimination generally fall into two categories: one involves having the model directly select the incorrect answer, while the other involves scoring the options. However, both methods incur high computational costs and often perform worse than methods that answer based on option ID. To address this issue, this paper proposes a process of elimination based on option ID. We select 10 LLMs and conduct zero-shot experiments on 7 different datasets. The experimental results demonstrate that our method significantly improves the model's performance. Further analysis reveals that the sequential elimination strategy can effectively enhance the model's reasoning ability. Additionally, we find that sequential elimination is also applicable to few-shot settings and can be combined with debias methods to further improve model performance.

Title: Uni-Sign: Toward Unified Sign Language Understanding at Scale

• Authors: Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, Houqiang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15187
• Pdf URL: https://arxiv.org/pdf/2501.15187
• Copy Paste: [[2501.15187]] Uni-Sign: Toward Unified Sign Language Understanding at Scale(https://arxiv.org/abs/2501.15187)
• Keywords: generative
• Abstract: Sign language pre-training has gained increasing attention for its ability to enhance performance across various sign language understanding (SLU) tasks. However, existing methods often suffer from a gap between pre-training and fine-tuning, leading to suboptimal results. To address this, we propose \modelname, a unified pre-training framework that eliminates the gap between pre-training and downstream SLU tasks through a large-scale generative pre-training strategy and a novel fine-tuning paradigm. First, we introduce CSL-News, a large-scale Chinese Sign Language (CSL) dataset containing 1,985 hours of video paired with textual annotations, which enables effective large-scale pre-training. Second, \modelname unifies SLU tasks by treating downstream tasks as a single sign language translation (SLT) task during fine-tuning, ensuring seamless knowledge transfer between pre-training and fine-tuning. Furthermore, we incorporate a prior-guided fusion (PGF) module and a score-aware sampling strategy to efficiently fuse pose and RGB information, addressing keypoint inaccuracies and improving computational efficiency. Extensive experiments across multiple SLU benchmarks demonstrate that \modelname achieves state-of-the-art performance across multiple downstream SLU tasks. Dataset and code are available at \url{this http URL}.

Title: A Floating Normalization Scheme for Deep Learning-Based Custom-Range Parameter Extraction in BSIM-CMG Compact Models

• Authors: Aasim Ashai, Aakash Jadhav, Biplab Sarkar
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.15190
• Pdf URL: https://arxiv.org/pdf/2501.15190
• Copy Paste: [[2501.15190]] A Floating Normalization Scheme for Deep Learning-Based Custom-Range Parameter Extraction in BSIM-CMG Compact Models(https://arxiv.org/abs/2501.15190)
• Keywords: extraction
• Abstract: A deep-learning (DL) based methodology for automated extraction of BSIM-CMG compact model parameters from experimental gate capacitance vs gate voltage (Cgg-Vg) and drain current vs gate voltage (Id-Vg) measurements is proposed in this paper. The proposed method introduces a floating normalization scheme within a cascaded forward and inverse ANN architecture enabling user-defined parameter extraction ranges. Unlike conventional DL-based extraction techniques, which are often constrained by fixed normalization ranges, the floating normalization approach adapts dynamically to user-specified ranges, allowing for fine-tuned control over the extracted parameters. Experimental validation, using a TCAD calibrated 14 nm FinFET process, demonstrates high accuracy for both Cgg-Vg and Id-Vg parameter extraction. The proposed framework offers enhanced flexibility, making it applicable to various compact models beyond BSIM-CMG.

Title: A Training-free Synthetic Data Selection Method for Semantic Segmentation

• Authors: Hao Tang, Siyue Yu, Jian Pang, Bingfeng Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15201
• Pdf URL: https://arxiv.org/pdf/2501.15201
• Copy Paste: [[2501.15201]] A Training-free Synthetic Data Selection Method for Semantic Segmentation(https://arxiv.org/abs/2501.15201)
• Keywords: segmentation
• Abstract: Training semantic segmenter with synthetic data has been attracting great attention due to its easy accessibility and huge quantities. Most previous methods focused on producing large-scale synthetic image-annotation samples and then training the segmenter with all of them. However, such a solution remains a main challenge in that the poor-quality samples are unavoidable, and using them to train the model will damage the training process. In this paper, we propose a training-free Synthetic Data Selection (SDS) strategy with CLIP to select high-quality samples for building a reliable synthetic dataset. Specifically, given massive synthetic image-annotation pairs, we first design a Perturbation-based CLIP Similarity (PCS) to measure the reliability of synthetic image, thus removing samples with low-quality images. Then we propose a class-balance Annotation Similarity Filter (ASF) by comparing the synthetic annotation with the response of CLIP to remove the samples related to low-quality annotations. The experimental results show that using our method significantly reduces the data size by half, while the trained segmenter achieves higher performance. The code is released at this https URL.

Title: "Stones from Other Hills can Polish Jade": Zero-shot Anomaly Image Synthesis via Cross-domain Anomaly Injection

• Authors: Siqi Wang, Yuanze Hu, Xinwang Liu, Siwei Wang, Guangpu Wang, Chuanfu Xu, Jie Liu, Ping Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15211
• Pdf URL: https://arxiv.org/pdf/2501.15211
• Copy Paste: [[2501.15211]] "Stones from Other Hills can Polish Jade": Zero-shot Anomaly Image Synthesis via Cross-domain Anomaly Injection(https://arxiv.org/abs/2501.15211)
• Keywords: diffusion
• Abstract: Industrial image anomaly detection (IAD) is a pivotal topic with huge value. Due to anomaly's nature, real anomalies in a specific modern industrial domain (i.e. domain-specific anomalies) are usually too rare to collect, which severely hinders IAD. Thus, zero-shot anomaly synthesis (ZSAS), which synthesizes pseudo anomaly images without any domain-specific anomaly, emerges as a vital technique for IAD. However, existing solutions are either unable to synthesize authentic pseudo anomalies, or require cumbersome training. Thus, we focus on ZSAS and propose a brand-new paradigm that can realize both authentic and training-free ZSAS. It is based on a chronically-ignored fact: Although domain-specific anomalies are rare, real anomalies from other domains (i.e. cross-domain anomalies) are actually abundant and directly applicable to ZSAS. Specifically, our new ZSAS paradigm makes three-fold contributions: First, we propose a novel method named Cross-domain Anomaly Injection (CAI), which directly exploits cross-domain anomalies to enable highly authentic ZSAS in a training-free manner. Second, to supply CAI with sufficient cross-domain anomalies, we build the first domain-agnostic anomaly dataset within our best knowledge, which provides ZSAS with abundant real anomaly patterns. Third, we propose a CAI-guided Diffusion Mechanism, which further breaks the quantity limit of real anomalies and enable unlimited anomaly synthesis. Our head-to-head comparison with existing ZSAS solutions justifies our paradigm's superior performance for IAD and demonstrates it as an effective and pragmatic ZSAS solution.

Title: Efficient and Interpretable Neural Networks Using Complex Lehmer Transform

• Authors: Masoud Ataei, Xiaogang Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15223
• Pdf URL: https://arxiv.org/pdf/2501.15223
• Copy Paste: [[2501.15223]] Efficient and Interpretable Neural Networks Using Complex Lehmer Transform(https://arxiv.org/abs/2501.15223)
• Keywords: interpretability
• Abstract: We propose an efficient and interpretable neural network with a novel activation function called the weighted Lehmer transform. This new activation function enables adaptive feature selection and extends to the complex domain, capturing phase-sensitive and hierarchical relationships within data. Notably, it provides greater interpretability and transparency compared to existing machine learning models, facilitating a deeper understanding of its functionality and decision-making processes. We analyze the mathematical properties of both real-valued and complex-valued Lehmer activation units and demonstrate their applications in modeling nonlinear interactions. Empirical evaluations demonstrate that our proposed neural network achieves competitive accuracy on benchmark datasets with significantly improved computational efficiency. A single layer of real-valued or complex-valued Lehmer activation units is shown to deliver state-of-the-art performance, balancing efficiency with interpretability.

Title: SEAL: Scaling to Emphasize Attention for Long-Context Retrieval

• Authors: Changhun Lee, Jun-gyu Jin, Younghyun Cho, Eunhyeok Park
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15225
• Pdf URL: https://arxiv.org/pdf/2501.15225
• Copy Paste: [[2501.15225]] SEAL: Scaling to Emphasize Attention for Long-Context Retrieval(https://arxiv.org/abs/2501.15225)
• Keywords: large language model
• Abstract: In this work, we introduce a novel approach called Scaling to Emphasize Attention for Long-context retrieval (SEAL), which enhances the retrieval performance of large language models (LLMs) over extended contexts. Previous studies have shown that each attention head in LLMs has a unique functionality and collectively contributes to the overall behavior of the model. Similarly, we observe that specific heads are closely tied to long-context retrieval, showing positive or negative correlation with retrieval scores. Built on this insight, we propose a learning-based mechanism using zero-shot generated data to emphasize these heads, improving the model's performance in long-context retrieval tasks. By applying SEAL, we can achieve significant improvements in in-domain retrieval performance, including document QA tasks from LongBench, and considerable improvements in out-of-domain cases. Additionally, when combined with existing training-free context extension techniques, SEAL extends the context limits of LLMs while maintaining highly reliable outputs, opening new avenues for research in this field.

Title: Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning

• Authors: Yiqun Chen, Lingyong Yan, Weiwei Sun, Xinyu Ma, Yi Zhang, Shuaiqiang Wang, Dawei Yin, Yiming Yang, Jiaxin Mao
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.15228
• Pdf URL: https://arxiv.org/pdf/2501.15228
• Copy Paste: [[2501.15228]] Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning(https://arxiv.org/abs/2501.15228)
• Keywords: large language model
• Abstract: Retrieval-augmented generation (RAG) is extensively utilized to incorporate external, current knowledge into large language models, thereby minimizing hallucinations. A standard RAG pipeline may comprise several components, such as query rewriting, document retrieval, document filtering, and answer generation. However, these components are typically optimized separately through supervised fine-tuning, which can lead to misalignments between the objectives of individual modules and the overarching aim of generating accurate answers in question-answering (QA) tasks. Although recent efforts have explored reinforcement learning (RL) to optimize specific RAG components, these approaches often focus on overly simplistic pipelines with only two components or do not adequately address the complex interdependencies and collaborative interactions among the modules. To overcome these challenges, we propose treating the RAG pipeline as a multi-agent cooperative task, with each component regarded as an RL agent. Specifically, we present MMOA-RAG, a Multi-Module joint Optimization Algorithm for RAG, which employs multi-agent reinforcement learning to harmonize all agents' goals towards a unified reward, such as the F1 score of the final answer. Experiments conducted on various QA datasets demonstrate that MMOA-RAG improves the overall pipeline performance and outperforms existing baselines. Furthermore, comprehensive ablation studies validate the contributions of individual components and the adaptability of MMOA-RAG across different RAG components and datasets. The code of MMOA-RAG is on this https URL.

Title: ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval

• Authors: Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Adam Jatowt
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15245
• Pdf URL: https://arxiv.org/pdf/2501.15245
• Copy Paste: [[2501.15245]] ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval(https://arxiv.org/abs/2501.15245)
• Keywords: large language model
• Abstract: Retrieval-Augmented Generation (RAG) models have drawn considerable attention in modern open-domain question answering. The effectiveness of RAG depends on the quality of the top retrieved documents. However, conventional retrieval methods sometimes fail to rank the most relevant documents at the top. In this paper, we introduce ASRank, a new re-ranking method based on scoring retrieved documents using zero-shot answer scent which relies on a pre-trained large language model to compute the likelihood of the document-derived answers aligning with the answer scent. Our approach demonstrates marked improvements across several datasets, including NQ, TriviaQA, WebQA, ArchivalQA, HotpotQA, and Entity Questions. Notably, ASRank increases Top-1 retrieval accuracy on NQ from $19.2\%$ to $46.5\%$ for MSS and $22.1\%$ to $47.3\%$ for BM25. It also shows strong retrieval performance on several datasets compared to state-of-the-art methods (47.3 Top-1 by ASRank vs 35.4 by UPR by BM25).

Title: Prompting ChatGPT for Chinese Learning as L2: A CEFR and EBCL Level Study

• Authors: Miao Lin-Zucker, Joël Bellasen, Jean-Daniel Zucker
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15247
• Pdf URL: https://arxiv.org/pdf/2501.15247
• Copy Paste: [[2501.15247]] Prompting ChatGPT for Chinese Learning as L2: A CEFR and EBCL Level Study(https://arxiv.org/abs/2501.15247)
• Keywords: generative, large language model
• Abstract: The use of chatbots in language learning has evolved significantly since the 1960s, becoming more sophisticated platforms as generative AI emerged. These tools now simulate natural conversations, adapting to individual learners' needs, including those studying Chinese. Our study explores how learners can use specific prompts to engage Large Language Models (LLM) as personalized chatbots, aiming to target their language level based on the Common European Framework of Reference for Languages (CEFR) and the European Benchmarking Chinese Language (EBCL) project. Focusing on A1, A1+ and A2 levels, we examine the teaching of Chinese, which presents unique challenges due to its logographic writing system. Our goal is to develop prompts that integrate oral and written skills, using high-frequency character lists and controlling oral lexical productions. These tools, powered by generative AI, aim to enhance language practice by crossing lexical and sinographic recurrence. While generative AI shows potential as a personalized tutor, further evaluation is needed to assess its effectiveness. We conducted a systematic series of experiments using ChatGPT models to evaluate their adherence to constraints specified in the prompts. The results indicate that incorporating level A1 and A1+ characters, along with the associated reference list, significantly enhances compliance with the EBCL character set. Properly prompted, LLMs can increase exposure to the target language and offer interactive exchanges to develop language skills.

Title: Enhancing Fetal Plane Classification Accuracy with Data Augmentation Using Diffusion Models

• Authors: Yueying Tian, Elif Ucurum, Xudong Han, Rupert Young, Chris Chatwin, Philip Birch
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15248
• Pdf URL: https://arxiv.org/pdf/2501.15248
• Copy Paste: [[2501.15248]] Enhancing Fetal Plane Classification Accuracy with Data Augmentation Using Diffusion Models(https://arxiv.org/abs/2501.15248)
• Keywords: diffusion
• Abstract: Ultrasound imaging is widely used in medical diagnosis, especially for fetal health assessment. However, the availability of high-quality annotated ultrasound images is limited, which restricts the training of machine learning models. In this paper, we investigate the use of diffusion models to generate synthetic ultrasound images to improve the performance on fetal plane classification. We train different classifiers first on synthetic images and then fine-tune them with real images. Extensive experimental results demonstrate that incorporating generated images into training pipelines leads to better classification accuracy than training with real images alone. The findings suggest that generating synthetic data using diffusion models can be a valuable tool in overcoming the challenges of data scarcity in ultrasound medical imaging.

Title: Generalizable Deepfake Detection via Effective Local-Global Feature Extraction

• Authors: Jiazhen Yan, Ziqiang Li, Ziwen He, Zhangjie Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15253
• Pdf URL: https://arxiv.org/pdf/2501.15253
• Copy Paste: [[2501.15253]] Generalizable Deepfake Detection via Effective Local-Global Feature Extraction(https://arxiv.org/abs/2501.15253)
• Keywords: extraction, diffusion, generative
• Abstract: The rapid advancement of GANs and diffusion models has led to the generation of increasingly realistic fake images, posing significant hidden dangers and threats to society. Consequently, deepfake detection has become a pressing issue in today's world. While some existing methods focus on forgery features from either a local or global perspective, they often overlook the complementary nature of these features. Other approaches attempt to incorporate both local and global features but rely on simplistic strategies, such as cropping, which fail to capture the intricate relationships between local features. To address these limitations, we propose a novel method that effectively combines local spatial-frequency domain features with global frequency domain information, capturing detailed and holistic forgery traces. Specifically, our method uses Discrete Wavelet Transform (DWT) and sliding windows to tile forged features and leverages attention mechanisms to extract local spatial-frequency domain information. Simultaneously, the phase component of the Fast Fourier Transform (FFT) is integrated with attention mechanisms to extract global frequency domain information, complementing the local features and ensuring the integrity of forgery detection. Comprehensive evaluations on open-world datasets generated by 34 distinct generative models demonstrate a significant improvement of 2.9% over existing state-of-the-art methods.

Title: Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models

• Authors: Zihuai Xu, Yang Xu, Hongli Xu, Yunming Liao, Zhiwei Yao, Zuan Xie
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15255
• Pdf URL: https://arxiv.org/pdf/2501.15255
• Copy Paste: [[2501.15255]] Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models(https://arxiv.org/abs/2501.15255)
• Keywords: large language model
• Abstract: Considering the hardware-friendly characteristics and broad applicability, structured pruning has emerged as an efficient solution to reduce the resource demands of large language models (LLMs) on resource-constrained devices. Traditional structured pruning methods often need fine-tuning to recover performance loss, which incurs high memory overhead and substantial data requirements, rendering them unsuitable for on-device applications. Additionally, post-training structured pruning techniques typically necessitate specific activation functions or architectural modifications, thereby limiting their scope of applications. Herein, we introduce COMP, a lightweight post-training structured pruning method that employs a hybrid-granularity pruning strategy. COMP initially prunes selected model layers based on their importance at a coarse granularity, followed by fine-grained neuron pruning within the dense layers of each remaining model layer. To more accurately evaluate neuron importance, COMP introduces a new matrix condition-based metric. Subsequently, COMP utilizes mask tuning to recover accuracy without the need for fine-tuning, significantly reducing memory consumption. Experimental results demonstrate that COMP improves performance by 6.13\% on the LLaMA-2-7B model with a 20\% pruning ratio compared to LLM-Pruner, while simultaneously reducing memory overhead by 80\%.

Title: Pre-trained Model Guided Mixture Knowledge Distillation for Adversarial Federated Learning

• Authors: Yu Qiao, Huy Q. Le, Apurba Adhikary, Choong Seon Hong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15257
• Pdf URL: https://arxiv.org/pdf/2501.15257
• Copy Paste: [[2501.15257]] Pre-trained Model Guided Mixture Knowledge Distillation for Adversarial Federated Learning(https://arxiv.org/abs/2501.15257)
• Keywords: defense, attack, robust, federate
• Abstract: This paper aims to improve the robustness of a small global model while maintaining clean accuracy under adversarial attacks and non-IID challenges in federated learning. By leveraging the concise knowledge embedded in the class probabilities from a pre-trained model for both clean and adversarial image classification, we propose a Pre-trained Model-guided Adversarial Federated Learning (PM-AFL) training paradigm. This paradigm integrates vanilla mixture and adversarial mixture knowledge distillation to effectively balance accuracy and robustness while promoting local models to learn from diverse data. Specifically, for clean accuracy, we adopt a dual distillation strategy where the class probabilities of randomly paired images and their blended versions are aligned between the teacher model and the local models. For adversarial robustness, we use a similar distillation approach but replace clean samples on the local side with adversarial examples. Moreover, considering the bias between local and global models, we also incorporate a consistency regularization term to ensure that local adversarial predictions stay aligned with their corresponding global clean ones. These strategies collectively enable local models to absorb diverse knowledge from the teacher model while maintaining close alignment with the global model, thereby mitigating overfitting to local optima and enhancing the generalization of the global model. Experiments demonstrate that the PM-AFL-based paradigm outperforms other methods that integrate defense strategies by a notable margin.

Title: Dynamic Estimation of Tea Flowering Based on an Improved YOLOv5 and ANN Model

• Authors: Qianxi Mi, Pengcheng Yuan, Chunlei Ma, Jiedan Chen, Mingzhe Yao
• Subjects: cs.CV, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2501.15262
• Pdf URL: https://arxiv.org/pdf/2501.15262
• Copy Paste: [[2501.15262]] Dynamic Estimation of Tea Flowering Based on an Improved YOLOv5 and ANN Model(https://arxiv.org/abs/2501.15262)
• Keywords: robust
• Abstract: Tea flowers play a crucial role in taxonomic research and hybrid breeding for the tea plant. Tea flowering consumes the plant's nutrients, and flower thinning can regulate carbon-nitrogen metabolism, enhancing the yield and quality of young shoots. As traditional methods of observing tea flower traits are labor-intensive and inaccurate, we propose an effective framework for tea flowering quantifying. In this study, a highly representative and diverse dataset was constructed by collecting flower images from 29 tea accessions. Based on this dataset, the TflosYOLO model was built on the YOLOv5 architecture and enhanced with the Squeeze-and-Excitation (SE) network, which is the first model to offer a viable solution for detecting tea flowers and predicting flower quantities. The TflosYOLO model achieved an mAP50 of 0.874, outperforming YOLOv5, YOLOv7 and YOLOv8. Furthermore, this model was tested on 34 datasets encompassing 26 tea accessions, five flowering stages, various lighting conditions, and pruned/unpruned plants, demonstrating high generalization and robustness. The correlation coefficient ($ R^2 $) between the predicted and actual flower counts was 0.974. Additionally, the TFSC (Tea Flowering Stage Classification) model - a novel Artificial Neural Network (ANN) was designed for automatic classification of the flowering stages. TFSC achieved an accuracy of 0.899. Dynamic analysis of flowering across 29 tea accessions in 2023 and 2024 was conducted, revealed significant variability in flower quantity and dynamics, with genetically similar accessions showing more consistent flowering patterns. This framework provides a solution for quantifying tea flowering, and can serve as a reference for precision horticulture.

Title: Explainable YOLO-Based Dyslexia Detection in Synthetic Handwriting Data

• Authors: Nora Fink
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15263
• Pdf URL: https://arxiv.org/pdf/2501.15263
• Copy Paste: [[2501.15263]] Explainable YOLO-Based Dyslexia Detection in Synthetic Handwriting Data(https://arxiv.org/abs/2501.15263)
• Keywords: explainability
• Abstract: Dyslexia affects reading and writing skills across many languages. This work describes a new application of YOLO-based object detection to isolate and label handwriting patterns (Normal, Reversal, Corrected) within synthetic images that resemble real words. Individual letters are first collected, preprocessed into 32x32 samples, then assembled into larger synthetic 'words' to simulate realistic handwriting. Our YOLOv11 framework simultaneously localizes each letter and classifies it into one of three categories, reflecting key dyslexia traits. Empirically, we achieve near-perfect performance, with precision, recall, and F1 metrics typically exceeding 0.999. This surpasses earlier single-letter approaches that rely on conventional CNNs or transfer-learning classifiers (for example, MobileNet-based methods in Robaa et al. arXiv:2410.19821). Unlike simpler pipelines that consider each letter in isolation, our solution processes complete word images, resulting in more authentic representations of handwriting. Although relying on synthetic data raises concerns about domain gaps, these experiments highlight the promise of YOLO-based detection for faster and more interpretable dyslexia screening. Future work will expand to real-world handwriting, other languages, and deeper explainability methods to build confidence among educators, clinicians, and families.

Title: Enhanced Intrusion Detection in IIoT Networks: A Lightweight Approach with Autoencoder-Based Feature Learning

• Authors: Tasnimul Hasan, Abrar Hossain, Mufakir Qamar Ansari, Talha Hussain Syed
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15266
• Pdf URL: https://arxiv.org/pdf/2501.15266
• Copy Paste: [[2501.15266]] Enhanced Intrusion Detection in IIoT Networks: A Lightweight Approach with Autoencoder-Based Feature Learning(https://arxiv.org/abs/2501.15266)
• Keywords: security, protect, attack, robust
• Abstract: The rapid expansion of the Industrial Internet of Things (IIoT) has significantly advanced digital technologies and interconnected industrial systems, creating substantial opportunities for growth. However, this growth has also heightened the risk of cyberattacks, necessitating robust security measures to protect IIoT networks. Intrusion Detection Systems (IDS) are essential for identifying and preventing abnormal network behaviors and malicious activities. Despite the potential of Machine Learning (ML)--based IDS solutions, existing models often face challenges with class imbalance and multiclass IIoT datasets, resulting in reduced detection accuracy. This research directly addresses these challenges by implementing six innovative approaches to enhance IDS performance, including leveraging an autoencoder for dimensional reduction, which improves feature learning and overall detection accuracy. Our proposed Decision Tree model achieved an exceptional F1 score and accuracy of 99.94% on the Edge-IIoTset dataset. Furthermore, we prioritized lightweight model design, ensuring deployability on resource-constrained edge devices. Notably, we are the first to deploy our model on a Jetson Nano, achieving inference times of 0.185 ms for binary classification and 0.187 ms for multiclass classification. These results highlight the novelty and robustness of our approach, offering a practical and efficient solution to the challenges posed by imbalanced and multiclass IIoT datasets, thereby enhancing the detection and prevention of network intrusions.

Title: New Evaluation Paradigm for Lexical Simplification

• Authors: Jipeng Qiang, Minjiang Huang, Yi Zhu, Yunhao Yuan, Chaowei Zhang, Xiaoye Ouyang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15268
• Pdf URL: https://arxiv.org/pdf/2501.15268
• Copy Paste: [[2501.15268]] New Evaluation Paradigm for Lexical Simplification(https://arxiv.org/abs/2501.15268)
• Keywords: large language model
• Abstract: Lexical Simplification (LS) methods use a three-step pipeline: complex word identification, substitute generation, and substitute ranking, each with separate evaluation datasets. We found large language models (LLMs) can simplify sentences directly with a single prompt, bypassing the traditional pipeline. However, existing LS datasets are not suitable for evaluating these LLM-generated simplified sentences, as they focus on providing substitutes for single complex words without identifying all complex words in a sentence. To address this gap, we propose a new annotation method for constructing an all-in-one LS dataset through human-machine collaboration. Automated methods generate a pool of potential substitutes, which human annotators then assess, suggesting additional alternatives as needed. Additionally, we explore LLM-based methods with single prompts, in-context learning, and chain-of-thought techniques. We introduce a multi-LLMs collaboration approach to simulate each step of the LS task. Experimental results demonstrate that LS based on multi-LLMs approaches significantly outperforms existing baselines.

Title: Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink

• Authors: Yining Wang, Mi Zhang, Junjie Sun, Chenyue Wang, Min Yang, Hui Xue, Jialing Tao, Ranjie Duan, Jiexi Liu
• Subjects: cs.LG, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15269
• Pdf URL: https://arxiv.org/pdf/2501.15269
• Copy Paste: [[2501.15269]] Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink(https://arxiv.org/abs/2501.15269)
• Keywords: attack, large language model
• Abstract: Fusing visual understanding into language generation, Multi-modal Large Language Models (MLLMs) are revolutionizing visual-language applications. Yet, these models are often plagued by the hallucination problem, which involves generating inaccurate objects, attributes, and relationships that do not match the visual content. In this work, we delve into the internal attention mechanisms of MLLMs to reveal the underlying causes of hallucination, exposing the inherent vulnerabilities in the instruction-tuning process. We propose a novel hallucination attack against MLLMs that exploits attention sink behaviors to trigger hallucinated content with minimal image-text relevance, posing a significant threat to critical downstream applications. Distinguished from previous adversarial methods that rely on fixed patterns, our approach generates dynamic, effective, and highly transferable visual adversarial inputs, without sacrificing the quality of model responses. Comprehensive experiments on 6 prominent MLLMs demonstrate the efficacy of our attack in compromising black-box MLLMs even with extensive mitigating mechanisms, as well as the promising results against cutting-edge commercial APIs, such as GPT-4o and Gemini 1.5. Our code is available at this https URL.

Title: Killing it with Zero-Shot: Adversarially Robust Novelty Detection

• Authors: Hossein Mirzaei, Mohammad Jafari, Hamid Reza Dehbashi, Zeinab Sadat Taghavi, Mohammad Sabokrou, Mohammad Hossein Rohban
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15271
• Pdf URL: https://arxiv.org/pdf/2501.15271
• Copy Paste: [[2501.15271]] Killing it with Zero-Shot: Adversarially Robust Novelty Detection(https://arxiv.org/abs/2501.15271)
• Keywords: attack, robust
• Abstract: Novelty Detection (ND) plays a crucial role in machine learning by identifying new or unseen data during model inference. This capability is especially important for the safe and reliable operation of automated systems. Despite advances in this field, existing techniques often fail to maintain their performance when subject to adversarial attacks. Our research addresses this gap by marrying the merits of nearest-neighbor algorithms with robust features obtained from models pretrained on ImageNet. We focus on enhancing the robustness and performance of ND algorithms. Experimental results demonstrate that our approach significantly outperforms current state-of-the-art methods across various benchmarks, particularly under adversarial conditions. By incorporating robust pretrained features into the k-NN algorithm, we establish a new standard for performance and robustness in the field of robust ND. This work opens up new avenues for research aimed at fortifying machine learning systems against adversarial vulnerabilities. Our implementation is publicly available at this https URL.

Title: PIP: Perturbation-based Iterative Pruning for Large Language Models

• Authors: Yi Cao, Wei-Jie Xu, Yucheng Shen, Weijie Shi, Chi-Min Chan, Jiajie Xu
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15278
• Pdf URL: https://arxiv.org/pdf/2501.15278
• Copy Paste: [[2501.15278]] PIP: Perturbation-based Iterative Pruning for Large Language Models(https://arxiv.org/abs/2501.15278)
• Keywords: large language model
• Abstract: The rapid increase in the parameter counts of Large Language Models (LLMs), reaching billions or even trillions, presents significant challenges for their practical deployment, particularly in resource-constrained environments. To ease this issue, we propose PIP (Perturbation-based Iterative Pruning), a novel double-view structured pruning method to optimize LLMs, which combines information from two different views: the unperturbed view and the perturbed view. With the calculation of gradient differences, PIP iteratively prunes those that struggle to distinguish between these two views. Our experiments show that PIP reduces the parameter count by approximately 20% while retaining over 85% of the original model's accuracy across varied benchmarks. In some cases, the performance of the pruned model is within 5% of the unpruned version, demonstrating PIP's ability to preserve key aspects of model effectiveness. Moreover, PIP consistently outperforms existing state-of-the-art (SOTA) structured pruning methods, establishing it as a leading technique for optimizing LLMs in environments with constrained resources. Our code is available at: this https URL.

Title: Pre-training a Transformer-Based Generative Model Using a Small Sepedi Dataset

• Authors: Simon P. Ramalepe, Thipe I. Modipa, Marelie H. Davel
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15281
• Pdf URL: https://arxiv.org/pdf/2501.15281
• Copy Paste: [[2501.15281]] Pre-training a Transformer-Based Generative Model Using a Small Sepedi Dataset(https://arxiv.org/abs/2501.15281)
• Keywords: transformer, generative
• Abstract: Due to the scarcity of data in low-resourced languages, the development of language models for these languages has been very slow. Currently, pre-trained language models have gained popularity in natural language processing, especially, in developing domain-specific models for low-resourced languages. In this study, we experiment with the impact of using occlusion-based techniques when training a language model for a text generation task. We curate 2 new datasets, the Sepedi monolingual (SepMono) dataset from several South African resources and the Sepedi radio news (SepNews) dataset from the radio news domain. We use the SepMono dataset to pre-train transformer-based models using the occlusion and non-occlusion pre-training techniques and compare performance. The SepNews dataset is specifically used for fine-tuning. Our results show that the non-occlusion models perform better compared to the occlusion-based models when measuring validation loss and perplexity. However, analysis of the generated text using the BLEU score metric, which measures the quality of the generated text, shows a slightly higher BLEU score for the occlusion-based models compared to the non-occlusion models.

Title: Are Human Interactions Replicable by Generative Agents? A Case Study on Pronoun Usage in Hierarchical Interactions

• Authors: Naihao Deng, Rada Mihalcea
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15283
• Pdf URL: https://arxiv.org/pdf/2501.15283
• Copy Paste: [[2501.15283]] Are Human Interactions Replicable by Generative Agents? A Case Study on Pronoun Usage in Hierarchical Interactions(https://arxiv.org/abs/2501.15283)
• Keywords: generative, large language model
• Abstract: As Large Language Models (LLMs) advance in their capabilities, researchers have increasingly employed them for social simulation. In this paper, we investigate whether interactions among LLM agents resemble those of humans. Specifically, we focus on the pronoun usage difference between leaders and non-leaders, examining whether the simulation would lead to human-like pronoun usage patterns during the LLMs' interactions. Our evaluation reveals the significant discrepancies between LLM-based simulations and human pronoun usage, with prompt-based or specialized agents failing to demonstrate human-like pronoun usage patterns. In addition, we reveal that even if LLMs understand the human pronoun usage patterns, they fail to demonstrate them in the actual interaction process. Our study highlights the limitations of social simulations based on LLM agents, urging caution in using such social simulation in practitioners' decision-making process.

Title: Efficient Point Clouds Upsampling via Flow Matching

• Authors: Zhi-Song Liu, Chenhang He, Lei Li
• Subjects: cs.CV, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.15286
• Pdf URL: https://arxiv.org/pdf/2501.15286
• Copy Paste: [[2501.15286]] Efficient Point Clouds Upsampling via Flow Matching(https://arxiv.org/abs/2501.15286)
• Keywords: diffusion
• Abstract: Diffusion models are a powerful framework for tackling ill-posed problems, with recent advancements extending their use to point cloud upsampling. Despite their potential, existing diffusion models struggle with inefficiencies as they map Gaussian noise to real point clouds, overlooking the geometric information inherent in sparse point clouds. To address these inefficiencies, we propose PUFM, a flow matching approach to directly map sparse point clouds to their high-fidelity dense counterparts. Our method first employs midpoint interpolation to sparse point clouds, resolving the density mismatch between sparse and dense point clouds. Since point clouds are unordered representations, we introduce a pre-alignment method based on Earth Mover's Distance (EMD) optimization to ensure coherent interpolation between sparse and dense point clouds, which enables a more stable learning path in flow matching. Experiments on synthetic datasets demonstrate that our method delivers superior upsampling quality but with fewer sampling steps. Further experiments on ScanNet and KITTI also show that our approach generalizes well on RGB-D point clouds and LiDAR point clouds, making it more practical for real-world applications.

Title: A Two-Stage CAE-Based Federated Learning Framework for Efficient Jamming Detection in 5G Networks

• Authors: Samhita Kuili, Mohammadreza Amini, Burak Kantarci
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15288
• Pdf URL: https://arxiv.org/pdf/2501.15288
• Copy Paste: [[2501.15288]] A Two-Stage CAE-Based Federated Learning Framework for Efficient Jamming Detection in 5G Networks(https://arxiv.org/abs/2501.15288)
• Keywords: security, privacy, attack, robust, federate
• Abstract: Cyber-security for 5G networks is drawing notable attention due to an increase in complex jamming attacks that could target the critical 5G Radio Frequency (RF) domain. These attacks pose a significant risk to heterogeneous network (HetNet) architectures, leading to degradation in network performance. Conventional machine-learning techniques for jamming detection rely on centralized training while increasing the odds of data privacy. To address these challenges, this paper proposes a decentralized two-stage federated learning (FL) framework for jamming detection in 5G femtocells. Our proposed distributed framework encompasses using the Federated Averaging (FedAVG) algorithm to train a Convolutional Autoencoder (CAE) for unsupervised learning. In the second stage, we use a fully connected network (FCN) built on the pre-trained CAE encoder that is trained using Federated Proximal (FedProx) algorithm to perform supervised classification. Our experimental results depict that our proposed framework (FedAVG and FedProx) accomplishes efficient training and prediction across non-IID client datasets without compromising data privacy. Specifically, our framework achieves a precision of 0.94, recall of 0.90, F1-score of 0.92, and an accuracy of 0.92, while minimizing communication rounds to 30 and achieving robust convergence in detecting jammed signals with an optimal client count of 6.

Title: Advanced Real-Time Fraud Detection Using RAG-Based LLMs

• Authors: Gurjot Singh, Prabhjot Singh, Maninder Singh
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15290
• Pdf URL: https://arxiv.org/pdf/2501.15290
• Copy Paste: [[2501.15290]] Advanced Real-Time Fraud Detection Using RAG-Based LLMs(https://arxiv.org/abs/2501.15290)
• Keywords: protect, robust
• Abstract: Artificial Intelligence has become a double edged sword in modern society being both a boon and a bane. While it empowers individuals it also enables malicious actors to perpetrate scams such as fraudulent phone calls and user impersonations. This growing threat necessitates a robust system to protect individuals In this paper we introduce a novel real time fraud detection mechanism using Retrieval Augmented Generation technology to address this challenge on two fronts. First our system incorporates a continuously updating policy checking feature that transcribes phone calls in real time and uses RAG based models to verify that the caller is not soliciting private information thus ensuring transparency and the authenticity of the conversation. Second we implement a real time user impersonation check with a two step verification process to confirm the callers identity ensuring accountability. A key innovation of our system is the ability to update policies without retraining the entire model enhancing its adaptability. We validated our RAG based approach using synthetic call recordings achieving an accuracy of 97.98 percent and an F1score of 97.44 percent with 100 calls outperforming state of the art methods. This robust and flexible fraud detection system is well suited for real world deployment.

Title: Deep Learning in Early Alzheimers diseases Detection: A Comprehensive Survey of Classification, Segmentation, and Feature Extraction Methods

• Authors: Rubab Hafeez, Sadia Waheed, Syeda Aleena Naqvi, Fahad Maqbool, Amna Sarwar, Sajjad Saleem, Muhammad Imran Sharif, Kamran Siddique, Zahid Akhtar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15293
• Pdf URL: https://arxiv.org/pdf/2501.15293
• Copy Paste: [[2501.15293]] Deep Learning in Early Alzheimers diseases Detection: A Comprehensive Survey of Classification, Segmentation, and Feature Extraction Methods(https://arxiv.org/abs/2501.15293)
• Keywords: extraction, segmentation
• Abstract: Alzheimers disease is a deadly neurological condition, impairing important memory and brain functions. Alzheimers disease promotes brain shrinkage, ultimately leading to dementia. Dementia diagnosis typically takes 2.8 to 4.4 years after the first clinical indication. Advancements in computing and information technology have led to many techniques of studying Alzheimers disease. Early identification and therapy are crucial for preventing Alzheimers disease, as early-onset dementia hits people before the age of 65, while late-onset dementia occurs after this age. According to the 2015 World Alzheimers disease Report, there are 46.8 million individuals worldwide suffering from dementia, with an anticipated 74.7 million more by 2030 and 131.5 million by 2050. Deep Learning has outperformed conventional Machine Learning techniques by identifying intricate structures in high-dimensional data. Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), have achieved an accuracy of up to 96.0% for Alzheimers disease classification, and 84.2% for mild cognitive impairment (MCI) conversion prediction. There have been few literature surveys available on applying ML to predict dementia, lacking in congenital observations. However, this survey has focused on a specific data channel for dementia detection. This study evaluated Deep Learning algorithms for early Alzheimers disease detection, using openly accessible datasets, feature segmentation, and classification methods. This article also has identified research gaps and limits in detecting Alzheimers disease, which can inform future research.

Title: You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning

• Authors: Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15296
• Pdf URL: https://arxiv.org/pdf/2501.15296
• Copy Paste: [[2501.15296]] You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning(https://arxiv.org/abs/2501.15296)
• Keywords: robust, large language model
• Abstract: The ever-increasing size of large language models (LLMs) presents significant challenges for deployment due to their heavy computational and memory requirements. Current model pruning techniques attempt to alleviate these issues by relying heavily on external calibration datasets to determine which parameters to prune or compress, thus limiting their flexibility and scalability across different compression ratios. Moreover, these methods often cause severe performance degradation, particularly in downstream tasks, when subjected to higher compression rates. In this paper, we propose PruneNet, a novel model compression method that addresses these limitations by reformulating model pruning as a policy learning process. PruneNet decouples the pruning process from the model architecture, eliminating the need for calibration datasets. It learns a stochastic pruning policy to assess parameter importance solely based on intrinsic model properties while preserving the spectral structure to minimize information loss. PruneNet can compress the LLaMA-2-7B model in just 15 minutes, achieving over 80% retention of its zero-shot performance with a 30% compression ratio, outperforming existing methods that retain only 75% performance. Furthermore, on complex multitask language understanding tasks, PruneNet demonstrates its robustness by preserving up to 80% performance of the original model, proving itself a superior alternative to conventional structured compression techniques.

Title: The Multicultural Medical Assistant: Can LLMs Improve Medical ASR Errors Across Borders?

• Authors: Ayo Adedeji, Mardhiyah Sanni, Emmanuel Ayodele, Sarita Joshi, Tobi Olatunji
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2501.15310
• Pdf URL: https://arxiv.org/pdf/2501.15310
• Copy Paste: [[2501.15310]] The Multicultural Medical Assistant: Can LLMs Improve Medical ASR Errors Across Borders?(https://arxiv.org/abs/2501.15310)
• Keywords: large language model
• Abstract: The global adoption of Large Language Models (LLMs) in healthcare shows promise to enhance clinical workflows and improve patient outcomes. However, Automatic Speech Recognition (ASR) errors in critical medical terms remain a significant challenge. These errors can compromise patient care and safety if not detected. This study investigates the prevalence and impact of ASR errors in medical transcription in Nigeria, the United Kingdom, and the United States. By evaluating raw and LLM-corrected transcriptions of accented English in these regions, we assess the potential and limitations of LLMs to address challenges related to accents and medical terminology in ASR. Our findings highlight significant disparities in ASR accuracy across regions and identify specific conditions under which LLM corrections are most effective.

Title: I Know What You Did Last Summer: Identifying VR User Activity Through VR Network Traffic

• Authors: Sheikh Samit Muhaimin, Spyridon Mastorakis
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15313
• Pdf URL: https://arxiv.org/pdf/2501.15313
• Copy Paste: [[2501.15313]] I Know What You Did Last Summer: Identifying VR User Activity Through VR Network Traffic(https://arxiv.org/abs/2501.15313)
• Keywords: security, privacy, attack
• Abstract: Virtual Reality (VR) technology has gained substantial traction and has the potential to transform a number of industries, including education, entertainment, and professional sectors. Nevertheless, concerns have arisen about the security and privacy implications of VR applications and the impact that they might have on users. In this paper, we investigate the following overarching research question: can VR applications and VR user activities in the context of such applications (e.g., manipulating virtual objects, walking, talking, flying) be identified based on the (potentially encrypted) network traffic that is generated by VR headsets during the operation of VR applications? To answer this question, we collect network traffic data from 25 VR applications running on the Meta Quest Pro headset and identify characteristics of the generated network traffic, which we subsequently use to train off-the-shelf Machine Learning (ML) models. Our results indicate that through the use of ML models, we can identify the VR applications being used with an accuracy of 92.4F% and the VR user activities performed with an accuracy of 91%. Furthermore, our results demonstrate that an attacker does not need to collect large amounts of network traffic data for each VR application to carry out such an attack. Specifically, an attacker only needs to collect less than 10 minutes of network traffic data for each VR application in order to identify applications with an accuracy higher than 90% and VR user activities with an accuracy higher than 88%.

Title: ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning

• Authors: Shangqian Gao, Ting Hua, Reza Shirkavand, Chi-Heng Lin, Zhen Tang, Zhengao Li, Longge Yuan, Fangyi Li, Zeyu Zhang, Alireza Ganjdanesh, Lou Qian, Xu Jie, Yen-Chang Hsu
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15316
• Pdf URL: https://arxiv.org/pdf/2501.15316
• Copy Paste: [[2501.15316]] ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning(https://arxiv.org/abs/2501.15316)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable abilities in tackling a wide range of complex tasks. However, their huge computational and memory costs raise significant challenges in deploying these models on resource-constrained devices or efficiently serving them. Prior approaches have attempted to alleviate these problems by permanently removing less important model structures, yet these methods often result in substantial performance degradation due to the permanent deletion of model parameters. In this work, we tried to mitigate this issue by reducing the number of active parameters without permanently removing them. Specifically, we introduce a differentiable dynamic pruning method that pushes dense models to maintain a fixed number of active parameters by converting their MLP layers into a Mixture of Experts (MoE) architecture. Our method, even without fine-tuning, consistently outperforms previous structural pruning techniques across diverse model families, including Phi-2, LLaMA-2, LLaMA-3, and Qwen-2.5.

Title: A Post-Processing-Based Fair Federated Learning Framework

• Authors: Yi Zhou, Naman Goel
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.15318
• Pdf URL: https://arxiv.org/pdf/2501.15318
• Copy Paste: [[2501.15318]] A Post-Processing-Based Fair Federated Learning Framework(https://arxiv.org/abs/2501.15318)
• Keywords: federate, fair
• Abstract: Federated Learning (FL) allows collaborative model training among distributed parties without pooling local datasets at a central server. However, the distributed nature of FL poses challenges in training fair federated learning models. The existing techniques are often limited in offering fairness flexibility to clients and performance. We formally define and empirically analyze a simple and intuitive post-processing-based framework to improve group fairness in FL systems. This framework can be divided into two stages: a standard FL training stage followed by a completely decentralized local debiasing stage. In the first stage, a global model is trained without fairness constraints using a standard federated learning algorithm (e.g. FedAvg). In the second stage, each client applies fairness post-processing on the global model using their respective local dataset. This allows for customized fairness improvements based on clients' desired and context-guided fairness requirements. We demonstrate two well-established post-processing techniques in this framework: model output post-processing and final layer fine-tuning. We evaluate the framework against three common baselines on four different datasets, including tabular, signal, and image data, each with varying levels of data heterogeneity across clients. Our work shows that this framework not only simplifies fairness implementation in FL but also provides significant fairness improvements with minimal accuracy loss or even accuracy gain, across data modalities and machine learning methods, being especially effective in more heterogeneous settings.

Title: Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data

• Authors: Jiajie Li, Brian R Quaranto, Chenhui Xu, Ishan Mishra, Ruiyang Qin, Dancheng Liu, Peter C W Kim, Jinjun Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15326
• Pdf URL: https://arxiv.org/pdf/2501.15326
• Copy Paste: [[2501.15326]] Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data(https://arxiv.org/abs/2501.15326)
• Keywords: robust
• Abstract: We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gatherers to 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks. We will open-source our code, model, and dataset to facilitate further research.

Title: Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection

• Authors: Bo Yang, Jiaxian Guo, Yusuke Iwasawa, Yutaka Matsuo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15355
• Pdf URL: https://arxiv.org/pdf/2501.15355
• Copy Paste: [[2501.15355]] Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection(https://arxiv.org/abs/2501.15355)
• Keywords: generative, large language model
• Abstract: Recent studies have increasingly demonstrated that large language models (LLMs) possess significant theory of mind (ToM) capabilities, showing the potential for simulating the tracking of mental states in generative agents. In this study, we propose a novel paradigm called ToM-agent, designed to empower LLMs-based generative agents to simulate ToM in open-domain conversational interactions. ToM-agent disentangles the confidence from mental states, facilitating the emulation of an agent's perception of its counterpart's mental states, such as beliefs, desires, and intentions (BDIs). Using past conversation history and verbal reflections, ToM-Agent can dynamically adjust counterparts' inferred BDIs, along with related confidence levels. We further put forth a counterfactual intervention method that reflects on the gap between the predicted responses of counterparts and their real utterances, thereby enhancing the efficiency of reflection. Leveraging empathetic and persuasion dialogue datasets, we assess the advantages of implementing the ToM-agent with downstream tasks, as well as its performance in both the first-order and the \textit{second-order} ToM. Our findings indicate that the ToM-agent can grasp the underlying reasons for their counterpart's behaviors beyond mere semantic-emotional supporting or decision-making based on common sense, providing new insights for studying large-scale LLMs-based simulation of human social behaviors.

Title: Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting

• Authors: Milad Khademi Nori, Il-Min Kim, Guanghui (Richard)Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15356
• Pdf URL: https://arxiv.org/pdf/2501.15356
• Copy Paste: [[2501.15356]] Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting(https://arxiv.org/abs/2501.15356)
• Keywords: privacy, federate, data-free
• Abstract: Federated Class-Incremental Learning (FCIL) refers to a scenario where a dynamically changing number of clients collaboratively learn an ever-increasing number of incoming tasks. FCIL is known to suffer from local forgetting due to class imbalance at each client and global forgetting due to class imbalance across clients. We develop a mathematical framework for FCIL that formulates local and global forgetting. Then, we propose an approach called Hybrid Rehearsal (HR), which utilizes latent exemplars and data-free techniques to address local and global forgetting, respectively. HR employs a customized autoencoder designed for both data classification and the generation of synthetic data. To determine the embeddings of new tasks for all clients in the latent space of the encoder, the server uses the Lennard-Jones Potential formulations. Meanwhile, at the clients, the decoder decodes the stored low-dimensional latent space exemplars back to the high-dimensional input space, used to address local forgetting. To overcome global forgetting, the decoder generates synthetic data. Furthermore, our mathematical framework proves that our proposed approach HR can, in principle, tackle the two local and global forgetting challenges. In practice, extensive experiments demonstrate that while preserving privacy, our proposed approach outperforms the state-of-the-art baselines on multiple FCIL benchmarks with low compute and memory footprints.

Title: Decentralized Low-Rank Fine-Tuning of Large Language Models

• Authors: Sajjad Ghiasvand, Mahnoosh Alizadeh, Ramtin Pedarsani
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15361
• Pdf URL: https://arxiv.org/pdf/2501.15361
• Copy Paste: [[2501.15361]] Decentralized Low-Rank Fine-Tuning of Large Language Models(https://arxiv.org/abs/2501.15361)
• Keywords: privacy, federate, large language model
• Abstract: The emergence of Large Language Models (LLMs) such as GPT-4, LLaMA, and BERT has transformed artificial intelligence, enabling advanced capabilities across diverse applications. While parameter-efficient fine-tuning (PEFT) techniques like LoRA offer computationally efficient adaptations of these models, their practical deployment often assumes centralized data and training environments. However, real-world scenarios frequently involve distributed, privacy-sensitive datasets that require decentralized solutions. Federated learning (FL) addresses data privacy by coordinating model updates across clients, but it is typically based on centralized aggregation through a parameter server, which can introduce bottlenecks and communication constraints. Decentralized learning, in contrast, eliminates this dependency by enabling direct collaboration between clients, improving scalability and efficiency in distributed environments. Despite its advantages, decentralized LLM fine-tuning remains underexplored. In this work, we propose \texttt{Dec-LoRA}, an algorithm for decentralized fine-tuning of LLMs based on low-rank adaptation (LoRA). Through extensive experiments on BERT and LLaMA-2 models, we evaluate \texttt{Dec-LoRA}'s performance in handling data heterogeneity and quantization constraints, enabling scalable, privacy-preserving LLM fine-tuning in decentralized settings.

Title: AI-Driven Secure Data Sharing: A Trustworthy and Privacy-Preserving Approach

• Authors: Al Amin, Kamrul Hasan, Sharif Ullah, Liang Hong
• Subjects: cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15363
• Pdf URL: https://arxiv.org/pdf/2501.15363
• Copy Paste: [[2501.15363]] AI-Driven Secure Data Sharing: A Trustworthy and Privacy-Preserving Approach(https://arxiv.org/abs/2501.15363)
• Keywords: secure, security, privacy, attack, robust, transformer
• Abstract: In the era of data-driven decision-making, ensuring the privacy and security of shared data is paramount across various domains. Applying existing deep neural networks (DNNs) to encrypted data is critical and often compromises performance, security, and computational overhead. To address these limitations, this research introduces a secure framework consisting of a learnable encryption method based on the block-pixel operation to encrypt the data and subsequently integrate it with the Vision Transformer (ViT). The proposed framework ensures data privacy and security by creating unique scrambling patterns per key, providing robust performance against adversarial attacks without compromising computational efficiency and data integrity. The framework was tested on sensitive medical datasets to validate its efficacy, proving its ability to handle highly confidential information securely. The suggested framework was validated with a 94\% success rate after extensive testing on real-world datasets, such as MRI brain tumors and histological scans of lung and colon cancers. Additionally, the framework was tested under diverse adversarial attempts against secure data sharing with optimum performance and demonstrated its effectiveness in various threat scenarios. These comprehensive analyses underscore its robustness, making it a trustworthy solution for secure data sharing in critical applications.

Title: A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Data

• Authors: Mahshid Rezakhani, Tolunay Seyfi, Fatemeh Afghah
• Subjects: cs.LG, cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.15365
• Pdf URL: https://arxiv.org/pdf/2501.15365
• Copy Paste: [[2501.15365]] A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Data(https://arxiv.org/abs/2501.15365)
• Keywords: security, robust
• Abstract: In recent years, rapid technological advancements and expanded Internet access have led to a significant rise in anomalies within network traffic and time-series data. Prompt detection of these irregularities is crucial for ensuring service quality, preventing financial losses, and maintaining robust security standards. While machine learning algorithms have shown promise in achieving high accuracy for anomaly detection, their performance is often constrained by the specific conditions of their training data. A persistent challenge in this domain is the scarcity of labeled data for anomaly detection in time-series datasets. This limitation hampers the training efficacy of both traditional machine learning and advanced deep learning models. To address this, unsupervised transfer learning emerges as a viable solution, leveraging unlabeled data from a source domain to identify anomalies in an unlabeled target domain. However, many existing approaches still depend on a small amount of labeled data from the target domain. To overcome these constraints, we propose a transfer learning-based model for anomaly detection in multivariate time-series datasets. Unlike conventional methods, our approach does not require labeled data in either the source or target domains. Empirical evaluations on novel intrusion detection datasets demonstrate that our model outperforms existing techniques in accurately identifying anomalies within an entirely unlabeled target domain.

Title: iFormer: Integrating ConvNet and Transformer for Mobile Application

• Authors: Chuanyang Zheng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15369
• Pdf URL: https://arxiv.org/pdf/2501.15369
• Copy Paste: [[2501.15369]] iFormer: Integrating ConvNet and Transformer for Mobile Application(https://arxiv.org/abs/2501.15369)
• Keywords: transformer, segmentation
• Abstract: We present a new family of mobile hybrid vision networks, called iFormer, with a focus on optimizing latency and accuracy on mobile applications. iFormer effectively integrates the fast local representation capacity of convolution with the efficient global modeling ability of self-attention. The local interactions are derived from transforming a standard convolutional network, \textit{i.e.}, ConvNeXt, to design a more lightweight mobile network. Our newly introduced mobile modulation attention removes memory-intensive operations in MHA and employs an efficient modulation mechanism to boost dynamic global representational capacity. We conduct comprehensive experiments demonstrating that iFormer outperforms existing lightweight networks across various tasks. Notably, iFormer achieves an impressive Top-1 accuracy of 80.4\% on ImageNet-1k with a latency of only 1.10 ms on an iPhone 13, surpassing the recently proposed MobileNetV4 under similar latency constraints. Additionally, our method shows significant improvements in downstream tasks, including COCO object detection, instance segmentation, and ADE20k semantic segmentation, while still maintaining low latency on mobile devices for high-resolution inputs in these scenarios.

Title: Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis

• Authors: Robinson Umeike, Neil Getty, Fangfang Xia, Rick Stevens
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15370
• Pdf URL: https://arxiv.org/pdf/2501.15370
• Copy Paste: [[2501.15370]] Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis(https://arxiv.org/abs/2501.15370)
• Keywords: extraction, large language model
• Abstract: Large language models (LLMs) have demonstrated immense capabilities in understanding textual data and are increasingly being adopted to help researchers accelerate scientific discovery through knowledge extraction (information retrieval), knowledge distillation (summarizing key findings and methodologies into concise forms), and knowledge synthesis (aggregating information from multiple scientific sources to address complex queries, generate hypothesis and formulate experimental plans). However, scientific data often exists in both visual and textual modalities. Vision language models (VLMs) address this by incorporating a pretrained vision backbone for processing images and a cross-modal projector that adapts image tokens into the LLM dimensional space, thereby providing richer multimodal comprehension. Nevertheless, off-the-shelf VLMs show limited capabilities in handling domain-specific data and are prone to hallucinations. We developed intelligent assistants finetuned from LLaVA models to enhance multimodal understanding in low-dose radiation therapy (LDRT)-a benign approach used in the treatment of cancer-related illnesses. Using multilingual data from 42,673 articles, we devise complex reasoning and detailed description tasks for visual question answering (VQA) benchmarks. Our assistants, trained on 50,882 image-text pairs, demonstrate superior performance over base models as evaluated using LLM-as-a-judge approach, particularly in reducing hallucination and improving domain-specific comprehension.

Title: Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models

• Authors: Melkamu Abay Mersha, Mesay Gemeda Yigezu, Jugal Kalita
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15374
• Pdf URL: https://arxiv.org/pdf/2501.15374
• Copy Paste: [[2501.15374]] Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models(https://arxiv.org/abs/2501.15374)
• Keywords: robust, extraction, explainability, large language model
• Abstract: The black-box nature of large language models (LLMs) necessitates the development of eXplainable AI (XAI) techniques for transparency and trustworthiness. However, evaluating these techniques remains a challenge. This study presents a general evaluation framework using four key metrics: Human-reasoning Agreement (HA), Robustness, Consistency, and Contrastivity. We assess the effectiveness of six explainability techniques from five different XAI categories model simplification (LIME), perturbation-based methods (SHAP), gradient-based approaches (InputXGradient, Grad-CAM), Layer-wise Relevance Propagation (LRP), and attention mechanisms-based explainability methods (Attention Mechanism Visualization, AMV) across five encoder-based language models: TinyBERT, BERTbase, BERTlarge, XLM-R large, and DeBERTa-xlarge, using the IMDB Movie Reviews and Tweet Sentiment Extraction (TSE) datasets. Our findings show that the model simplification-based XAI method (LIME) consistently outperforms across multiple metrics and models, significantly excelling in HA with a score of 0.9685 on DeBERTa-xlarge, robustness, and consistency as the complexity of large language models increases. AMV demonstrates the best Robustness, with scores as low as 0.0020. It also excels in Consistency, achieving near-perfect scores of 0.9999 across all models. Regarding Contrastivity, LRP performs the best, particularly on more complex models, with scores up to 0.9371.

Title: Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation

• Authors: Reza Akbarian Bafghi, Carden Bagwell, Avinash Ravichandran, Ashish Shrivastava, Maziar Raissi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15377
• Pdf URL: https://arxiv.org/pdf/2501.15377
• Copy Paste: [[2501.15377]] Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation(https://arxiv.org/abs/2501.15377)
• Keywords: robust
• Abstract: Adapting deep learning models to new domains often requires computationally intensive retraining and risks catastrophic forgetting. While fine-tuning enables domain-specific adaptation, it can reduce robustness to distribution shifts, impacting out-of-distribution (OOD) performance. Pre-trained zero-shot models like CLIP offer strong generalization but may suffer degraded robustness after fine-tuning. Building on Task Adaptive Parameter Sharing (TAPS), we propose a simple yet effective extension as a parameter-efficient fine-tuning (PEFT) method, using an indicator function to selectively activate Low-Rank Adaptation (LoRA) blocks. Our approach minimizes knowledge loss, retains its generalization strengths under domain shifts, and significantly reduces computational costs compared to traditional fine-tuning. We demonstrate that effective fine-tuning can be achieved with as few as 5\% of active blocks, substantially improving efficiency. Evaluations on pre-trained models such as CLIP and DINO-ViT demonstrate our method's broad applicability and effectiveness in maintaining performance and knowledge retention.

Title: MetaOcc: Surround-View 4D Radar and Camera Fusion Framework for 3D Occupancy Prediction with Dual Training Strategies

• Authors: Long Yang, Lianqing Zheng, Wenjin Ai, Minghao Liu, Sen Li, Qunshu Lin, Shengyu Yan, Jie Bai, Zhixiong Ma, Xichan Zhu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15384
• Pdf URL: https://arxiv.org/pdf/2501.15384
• Copy Paste: [[2501.15384]] MetaOcc: Surround-View 4D Radar and Camera Fusion Framework for 3D Occupancy Prediction with Dual Training Strategies(https://arxiv.org/abs/2501.15384)
• Keywords: robust, extraction
• Abstract: 3D occupancy prediction is crucial for autonomous driving perception. Fusion of 4D radar and camera provides a potential solution of robust occupancy prediction on serve weather with least cost. How to achieve effective multi-modal feature fusion and reduce annotation costs remains significant challenges. In this work, we propose MetaOcc, a novel multi-modal occupancy prediction framework that fuses surround-view cameras and 4D radar for comprehensive environmental perception. We first design a height self-attention module for effective 3D feature extraction from sparse radar points. Then, a local-global fusion mechanism is proposed to adaptively capture modality contributions while handling spatio-temporal misalignments. Temporal alignment and fusion module is employed to further aggregate historical feature. Furthermore, we develop a semi-supervised training procedure leveraging open-set segmentor and geometric constraints for pseudo-label generation, enabling robust perception with limited annotations. Extensive experiments on OmniHD-Scenes dataset demonstrate that MetaOcc achieves state-of-the-art performance, surpassing previous methods by significant margins. Notably, as the first semi-supervised 4D radar and camera fusion-based occupancy prediction approach, MetaOcc maintains 92.5% of the fully-supervised performance while using only 50% of ground truth annotations, establishing a new benchmark for multi-modal 3D occupancy prediction. Code and data are available at this https URL.

Title: DDUNet: Dual Dynamic U-Net for Highly-Efficient Cloud Segmentation

• Authors: Yijie Li, Hewei Wang, Jinfeng Xu, Puzhen Wu, Yunzhong Xiao, Shaofan Wang, Soumyabrata Dev
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.15385
• Pdf URL: https://arxiv.org/pdf/2501.15385
• Copy Paste: [[2501.15385]] DDUNet: Dual Dynamic U-Net for Highly-Efficient Cloud Segmentation(https://arxiv.org/abs/2501.15385)
• Keywords: robust, segmentation
• Abstract: Cloud segmentation amounts to separating cloud pixels from non-cloud pixels in an image. Current deep learning methods for cloud segmentation suffer from three issues. (a) Constrain on their receptive field due to the fixed size of the convolution kernel. (b) Lack of robustness towards different scenarios. (c) Requirement of a large number of parameters and limitations for real-time implementation. To address these issues, we propose a Dual Dynamic U-Net (DDUNet) for supervised cloud segmentation. The DDUNet adheres to a U-Net architecture and integrates two crucial modules: the dynamic multi-scale convolution (DMSC), improving merging features under different reception fields, and the dynamic weights and bias generator (DWBG) in classification layers to enhance generalization ability. More importantly, owing to the use of depth-wise convolution, the DDUNet is a lightweight network that can achieve 95.3% accuracy on the SWINySEG dataset with only 0.33M parameters, and achieve superior performance over three different configurations of the SWINySEg dataset in both accuracy and efficiency.

Title: CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation

• Authors: Yijie Li, Hewei Wang, Jinfeng Xu, Zixiao Ma, Puzhen Wu, Shaofan Wang, Soumyabrata Dev
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15389
• Pdf URL: https://arxiv.org/pdf/2501.15389
• Copy Paste: [[2501.15389]] CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation(https://arxiv.org/abs/2501.15389)
• Keywords: robust, segmentation
• Abstract: Remote sensing image segmentation is pivotal for earth observation, underpinning applications such as environmental monitoring and urban planning. Due to the limited annotation data available in remote sensing images, numerous studies have focused on data augmentation as a means to alleviate overfitting in deep learning networks. However, some existing data augmentation strategies rely on simple transformations that may not sufficiently enhance data diversity or model generalization capabilities. This paper proposes a novel augmentation strategy, Clustered-Patch-Mixed Mosaic (CP2M), designed to address these limitations. CP2M integrates a Mosaic augmentation phase with a clustered patch mix phase. The former stage constructs a new sample from four random samples, while the latter phase uses the connected component labeling algorithm to ensure the augmented data maintains spatial coherence and avoids introducing irrelevant semantics when pasting random patches. Our experiments on the ISPRS Potsdam dataset demonstrate that CP2M substantially mitigates overfitting, setting new benchmarks for segmentation accuracy and model robustness in remote sensing tasks.

Title: Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception

• Authors: Lianqing Zheng, Jianan Liu, Runwei Guan, Long Yang, Shouyi Lu, Yuanzhe Li, Xiaokai Bai, Jie Bai, Zhixiong Ma, Hui-Liang Shen, Xichan Zhu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15394
• Pdf URL: https://arxiv.org/pdf/2501.15394
• Copy Paste: [[2501.15394]] Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception(https://arxiv.org/abs/2501.15394)
• Keywords: robust, transformer
• Abstract: 3D object detection and occupancy prediction are critical tasks in autonomous driving, attracting significant attention. Despite the potential of recent vision-based methods, they encounter challenges under adverse conditions. Thus, integrating cameras with next-generation 4D imaging radar to achieve unified multi-task perception is highly significant, though research in this domain remains limited. In this paper, we propose Doracamom, the first framework that fuses multi-view cameras and 4D radar for joint 3D object detection and semantic occupancy prediction, enabling comprehensive environmental perception. Specifically, we introduce a novel Coarse Voxel Queries Generator that integrates geometric priors from 4D radar with semantic features from images to initialize voxel queries, establishing a robust foundation for subsequent Transformer-based refinement. To leverage temporal information, we design a Dual-Branch Temporal Encoder that processes multi-modal temporal features in parallel across BEV and voxel spaces, enabling comprehensive spatio-temporal representation learning. Furthermore, we propose a Cross-Modal BEV-Voxel Fusion module that adaptively fuses complementary features through attention mechanisms while employing auxiliary tasks to enhance feature quality. Extensive experiments on the OmniHD-Scenes, View-of-Delft (VoD), and TJ4DRadSet datasets demonstrate that Doracamom achieves state-of-the-art performance in both tasks, establishing new benchmarks for multi-modal 3D perception. Code and models will be publicly available.

Title: Hiding in Plain Sight: An IoT Traffic Camouflage Framework for Enhanced Privacy

• Authors: Daniel Adu Worae, Spyridon Mastorakis
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.15395
• Pdf URL: https://arxiv.org/pdf/2501.15395
• Copy Paste: [[2501.15395]] Hiding in Plain Sight: An IoT Traffic Camouflage Framework for Enhanced Privacy(https://arxiv.org/abs/2501.15395)
• Keywords: privacy, protect, attack, robust
• Abstract: The rapid growth of Internet of Things (IoT) devices has introduced significant challenges to privacy, particularly as network traffic analysis techniques evolve. While encryption protects data content, traffic attributes such as packet size and timing can reveal sensitive information about users and devices. Existing single-technique obfuscation methods, such as packet padding, often fall short in dynamic environments like smart homes due to their predictability, making them vulnerable to machine learning-based attacks. This paper introduces a multi-technique obfuscation framework designed to enhance privacy by disrupting traffic analysis. The framework leverages six techniques-Padding, Padding with XORing, Padding with Shifting, Constant Size Padding, Fragmentation, and Delay Randomization-to obscure traffic patterns effectively. Evaluations on three public datasets demonstrate significant reductions in classifier performance metrics, including accuracy, precision, recall, and F1 score. We assess the framework's robustness against adversarial tactics by retraining and fine-tuning neural network classifiers on obfuscated traffic. The results reveal a notable degradation in classifier performance, underscoring the framework's resilience against adaptive attacks. Furthermore, we evaluate communication and system performance, showing that higher obfuscation levels enhance privacy but may increase latency and communication overhead.

Title: How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning

• Authors: Tohida Rehman, Debarshi Kumar Sanyal, Samiran Chattopadhyay
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15398
• Pdf URL: https://arxiv.org/pdf/2501.15398
• Copy Paste: [[2501.15398]] How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning(https://arxiv.org/abs/2501.15398)
• Keywords: large language model
• Abstract: Artificial intelligence systems significantly impact the environment, particularly in natural language processing (NLP) tasks. These tasks often require extensive computational resources to train deep neural networks, including large-scale language models containing billions of parameters. This study analyzes the trade-offs between energy consumption and performance across three neural language models: two pre-trained models (T5-base and BART-base), and one large language model (LLaMA 3-8B). These models were fine-tuned for the text summarization task, focusing on generating research paper highlights that encapsulate the core themes of each paper. A wide range of evaluation metrics, including ROUGE, METEOR, MoverScore, BERTScore, and SciBERTScore, were employed to assess their performance. Furthermore, the carbon footprint associated with fine-tuning each model was measured, offering a comprehensive assessment of their environmental impact. This research underscores the importance of incorporating environmental considerations into the design and implementation of neural language models and calls for the advancement of energy-efficient AI methodologies.

Title: Semantic Layered Embedding Diffusion in Large Language Models for Multi-Contextual Consistency

• Authors: Irin Kabakum, Thomas Montgomery, Daniel Ravenwood, Genevieve Harrington
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15405
• Pdf URL: https://arxiv.org/pdf/2501.15405
• Copy Paste: [[2501.15405]] Semantic Layered Embedding Diffusion in Large Language Models for Multi-Contextual Consistency(https://arxiv.org/abs/2501.15405)
• Keywords: diffusion, transformer, large language model
• Abstract: The Semantic Layered Embedding Diffusion (SLED) mechanism redefines the representation of hierarchical semantics within transformer-based architectures, enabling enhanced contextual consistency across a wide array of linguistic tasks. By introducing a multi-layered diffusion process grounded in spectral analysis, it achieves a complex balance between global and local semantic coherence. Experimental results demonstrate significant improvements in perplexity and BLEU scores, emphasizing the mechanism's ability to adapt effectively across diverse domains, including multilingual and cross-domain text generation. A rigorous mathematical framework underpins the embedding diffusion process, incorporating weighted adjacency matrices, kernel-based refinements, and dynamic layer-wise normalization. Error distribution analysis reveals that SLED addresses challenges in semantic alignment and coherence, outperforming baseline approaches across varied benchmarks. Scalability studies illustrate that its performance gains are maintained consistently across different model sizes, reflecting a practical balance between computational efficiency and linguistic precision. The implementation also achieves energy efficiency, reducing resource consumption during training and inference phases without compromising accuracy. Qualitative case studies further validate its adaptability to extended narratives and context-intensive scenarios, highlighting the mechanism's potential for real-world applications. SLED offers a different perspective on embedding design and its implications for advancing language modeling.

Title: Episodic Novelty Through Temporal Distance

• Authors: Yuhua Jiang, Qihan Liu, Yiqin Yang, Xiaoteng Ma, Dianyu Zhong, Hao Hu, Jun Yang, Bin Liang, Bo Xu, Chongjie Zhang, Qianchuan Zhao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15418
• Pdf URL: https://arxiv.org/pdf/2501.15418
• Copy Paste: [[2501.15418]] Episodic Novelty Through Temporal Distance(https://arxiv.org/abs/2501.15418)
• Keywords: robust
• Abstract: Exploration in sparse reward environments remains a significant challenge in reinforcement learning, particularly in Contextual Markov Decision Processes (CMDPs), where environments differ across episodes. Existing episodic intrinsic motivation methods for CMDPs primarily rely on count-based approaches, which are ineffective in large state spaces, or on similarity-based methods that lack appropriate metrics for state comparison. To address these shortcomings, we propose Episodic Novelty Through Temporal Distance (ETD), a novel approach that introduces temporal distance as a robust metric for state similarity and intrinsic reward computation. By employing contrastive learning, ETD accurately estimates temporal distances and derives intrinsic rewards based on the novelty of states within the current episode. Extensive experiments on various benchmark tasks demonstrate that ETD significantly outperforms state-of-the-art methods, highlighting its effectiveness in enhancing exploration in sparse reward CMDPs.

Title: Visual Generation Without Guidance

• Authors: Huayu Chen, Kai Jiang, Kaiwen Zheng, Jianfei Chen, Hang Su, Jun Zhu
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15420
• Pdf URL: https://arxiv.org/pdf/2501.15420
• Copy Paste: [[2501.15420]] Visual Generation Without Guidance(https://arxiv.org/abs/2501.15420)
• Keywords: diffusion, generative
• Abstract: Classifier-Free Guidance (CFG) has been a default technique in various visual generative models, yet it requires inference from both conditional and unconditional models during sampling. We propose to build visual models that are free from guided sampling. The resulting algorithm, Guidance-Free Training (GFT), matches the performance of CFG while reducing sampling to a single model, halving the computational cost. Unlike previous distillation-based approaches that rely on pretrained CFG networks, GFT enables training directly from scratch. GFT is simple to implement. It retains the same maximum likelihood objective as CFG and differs mainly in the parameterization of conditional models. Implementing GFT requires only minimal modifications to existing codebases, as most design choices and hyperparameters are directly inherited from CFG. Our extensive experiments across five distinct visual models demonstrate the effectiveness and versatility of GFT. Across domains of diffusion, autoregressive, and masked-prediction modeling, GFT consistently achieves comparable or even lower FID scores, with similar diversity-fidelity trade-offs compared with CFG baselines, all while being guidance-free. Code will be available at this https URL.

Title: OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas

• Authors: Xiaoyang Wang, Hongming Zhang, Tao Ge, Wenhao Yu, Dian Yu, Dong Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15427
• Pdf URL: https://arxiv.org/pdf/2501.15427
• Copy Paste: [[2501.15427]] OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas(https://arxiv.org/abs/2501.15427)
• Keywords: large language model
• Abstract: Customizable role-playing in large language models (LLMs), also known as character generalization, is gaining increasing attention for its versatility and cost-efficiency in developing and deploying role-playing dialogue agents. This study explores a large-scale data synthesis approach to equip LLMs with character generalization capabilities. We begin by synthesizing large-scale character profiles using personas from Persona Hub and then explore two strategies: response rewriting and response generation, to create character-aligned instructional responses. To validate the effectiveness of our synthetic instruction tuning data for character generalization, we perform supervised fine-tuning (SFT) using the LLaMA-3 8B model. Our best-performing model strengthens the original LLaMA-3 8B Instruct model and achieves performance comparable to GPT-4o models on role-playing dialogue. We release our synthetic characters and instruction-tuning dialogues to support public research.

Title: Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?

• Authors: Utku Ozbulak, Esla Timothy Anzaku, Solha Kang, Wesley De Neve, Joris Vankerschaver
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15431
• Pdf URL: https://arxiv.org/pdf/2501.15431
• Copy Paste: [[2501.15431]] Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?(https://arxiv.org/abs/2501.15431)
• Keywords: fair
• Abstract: Machine learning (ML) research strongly relies on benchmarks in order to determine the relative effectiveness of newly proposed models. Recently, a number of prominent research effort argued that a number of models that improve the state-of-the-art by a small margin tend to do so by winning what they call a "benchmark lottery". An important benchmark in the field of machine learning and computer vision is the ImageNet where newly proposed models are often showcased based on their performance on this dataset. Given the large number of self-supervised learning (SSL) frameworks that has been proposed in the past couple of years each coming with marginal improvements on the ImageNet dataset, in this work, we evaluate whether those marginal improvements on ImageNet translate to improvements on similar datasets or not. To do so, we investigate twelve popular SSL frameworks on five ImageNet variants and discover that models that seem to perform well on ImageNet may experience significant performance declines on similar datasets. Specifically, state-of-the-art frameworks such as DINO and Swav, which are praised for their performance, exhibit substantial drops in performance while MoCo and Barlow Twins displays comparatively good results. As a result, we argue that otherwise good and desirable properties of models remain hidden when benchmarking is only performed on the ImageNet validation set, making us call for more adequate benchmarking. To avoid the "benchmark lottery" on ImageNet and to ensure a fair benchmarking process, we investigate the usage of a unified metric that takes into account the performance of models on other ImageNet variant datasets.

Title: Mitigating Spurious Negative Pairs for Robust Industrial Anomaly Detection

• Authors: Hossein Mirzaei, Mojtaba Nafez, Jafar Habibi, Mohammad Sabokrou, Mohammad Hossein Rohban
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15434
• Pdf URL: https://arxiv.org/pdf/2501.15434
• Copy Paste: [[2501.15434]] Mitigating Spurious Negative Pairs for Robust Industrial Anomaly Detection(https://arxiv.org/abs/2501.15434)
• Keywords: attack, robust
• Abstract: Despite significant progress in Anomaly Detection (AD), the robustness of existing detection methods against adversarial attacks remains a challenge, compromising their reliability in critical real-world applications such as autonomous driving. This issue primarily arises from the AD setup, which assumes that training data is limited to a group of unlabeled normal samples, making the detectors vulnerable to adversarial anomaly samples during testing. Additionally, implementing adversarial training as a safeguard encounters difficulties, such as formulating an effective objective function without access to labels. An ideal objective function for adversarial training in AD should promote strong perturbations both within and between the normal and anomaly groups to maximize margin between normal and anomaly distribution. To address these issues, we first propose crafting a pseudo-anomaly group derived from normal group samples. Then, we demonstrate that adversarial training with contrastive loss could serve as an ideal objective function, as it creates both inter- and intra-group perturbations. However, we notice that spurious negative pairs compromise the conventional contrastive loss to achieve robust AD. Spurious negative pairs are those that should be closely mapped but are erroneously separated. These pairs introduce noise and misguide the direction of inter-group adversarial perturbations. To overcome the effect of spurious negative pairs, we define opposite pairs and adversarially pull them apart to strengthen inter-group perturbations. Experimental results demonstrate our superior performance in both clean and adversarial scenarios, with a 26.1% improvement in robust detection across various challenging benchmark datasets. The implementation of our work is available at: this https URL.

Title: Making Sense Of Distributed Representations With Activation Spectroscopy

• Authors: Kyle Reing, Greg Ver Steeg, Aram Galstyan
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15435
• Pdf URL: https://arxiv.org/pdf/2501.15435
• Copy Paste: [[2501.15435]] Making Sense Of Distributed Representations With Activation Spectroscopy(https://arxiv.org/abs/2501.15435)
• Keywords: interpretability, transformer
• Abstract: In the study of neural network interpretability, there is growing evidence to suggest that relevant features are encoded across many neurons in a distributed fashion. Making sense of these distributed representations without knowledge of the network's encoding strategy is a combinatorial task that is not guaranteed to be tractable. This work explores one feasible path to both detecting and tracing the joint influence of neurons in a distributed representation. We term this approach Activation Spectroscopy (ActSpec), owing to its analysis of the pseudo-Boolean Fourier spectrum defined over the activation patterns of a network layer. The sub-network defined between a given layer and an output logit is cast as a special class of pseudo-Boolean function. The contributions of each subset of neurons in the specified layer can be quantified through the function's Fourier coefficients. We propose a combinatorial optimization procedure to search for Fourier coefficients that are simultaneously high-valued, and non-redundant. This procedure can be viewed as an extension of the Goldreich-Levin algorithm which incorporates additional problem-specific constraints. The resulting coefficients specify a collection of subsets, which are used to test the degree to which a representation is distributed. We verify our approach in a number of synthetic settings and compare against existing interpretability benchmarks. We conclude with a number of experimental evaluations on an MNIST classifier, and a transformer-based network for sentiment analysis.

Title: Dfilled: Repurposing Edge-Enhancing Diffusion for Guided DSM Void Filling

• Authors: Daniel Panangian, Ksenia Bittner
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15440
• Pdf URL: https://arxiv.org/pdf/2501.15440
• Copy Paste: [[2501.15440]] Dfilled: Repurposing Edge-Enhancing Diffusion for Guided DSM Void Filling(https://arxiv.org/abs/2501.15440)
• Keywords: diffusion
• Abstract: Digital Surface Models (DSMs) are essential for accurately representing Earth's topography in geospatial analyses. DSMs capture detailed elevations of natural and manmade features, crucial for applications like urban planning, vegetation studies, and 3D reconstruction. However, DSMs derived from stereo satellite imagery often contain voids or missing data due to occlusions, shadows, and lowsignal areas. Previous studies have primarily focused on void filling for digital elevation models (DEMs) and Digital Terrain Models (DTMs), employing methods such as inverse distance weighting (IDW), kriging, and spline interpolation. While effective for simpler terrains, these approaches often fail to handle the intricate structures present in DSMs. To overcome these limitations, we introduce Dfilled, a guided DSM void filling method that leverages optical remote sensing images through edge-enhancing diffusion. Dfilled repurposes deep anisotropic diffusion models, which originally designed for super-resolution tasks, to inpaint DSMs. Additionally, we utilize Perlin noise to create inpainting masks that mimic natural void patterns in DSMs. Experimental evaluations demonstrate that Dfilled surpasses traditional interpolation methods and deep learning approaches in DSM void filling tasks. Both quantitative and qualitative assessments highlight the method's ability to manage complex features and deliver accurate, visually coherent results.

Title: InfoBFR: Real-World Blind Face Restoration via Information Bottleneck

• Authors: Nan Gao, Jia Li, Huaibo Huang, Ke Shang, Ran He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15443
• Pdf URL: https://arxiv.org/pdf/2501.15443
• Copy Paste: [[2501.15443]] InfoBFR: Real-World Blind Face Restoration via Information Bottleneck(https://arxiv.org/abs/2501.15443)
• Keywords: diffusion
• Abstract: Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of data degradation patterns. Current BFR methods have realized certain restored productions but with inherent neural degradations that limit real-world generalization in complicated scenarios. In this paper, we propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues, which achieves high-generalization face restoration in diverse wild and heterogeneous scenes. Specifically, based on the results from pre-trained BFR models, InfoBFR considers information compression using manifold information bottleneck (MIB) and information compensation with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. Comprehensive experimental results demonstrate the superiority of InfoBFR over state-of-the-art GAN-based and diffusion-based BFR methods, with around 70ms consumption, 16M trainable parameters, and nearly 85% BFR-boosting. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.

Title: StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces

• Authors: Kyeongmin Yeo, Jaihoon Kim, Minhyuk Sung
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15445
• Pdf URL: https://arxiv.org/pdf/2501.15445
• Copy Paste: [[2501.15445]] StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces(https://arxiv.org/abs/2501.15445)
• Keywords: diffusion
• Abstract: We propose a zero-shot method for generating images in arbitrary spaces (e.g., a sphere for 360° panoramas and a mesh surface for texture) using a pretrained image diffusion model. The zero-shot generation of various visual content using a pretrained image diffusion model has been explored mainly in two directions. First, Diffusion Synchronization-performing reverse diffusion processes jointly across different projected spaces while synchronizing them in the target space-generates high-quality outputs when enough conditioning is provided, but it struggles in its absence. Second, Score Distillation Sampling-gradually updating the target space data through gradient descent-results in better coherence but often lacks detail. In this paper, we reveal for the first time the interconnection between these two methods while highlighting their differences. To this end, we propose StochSync, a novel approach that combines the strengths of both, enabling effective performance with weak conditioning. Our experiments demonstrate that StochSync provides the best performance in 360° panorama generation (where image conditioning is not given), outperforming previous finetuning-based methods, and also delivers comparable results in 3D mesh texturing (where depth conditioning is provided) with previous methods.

Title: Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models

• Authors: Robin Young
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15446
• Pdf URL: https://arxiv.org/pdf/2501.15446
• Copy Paste: [[2501.15446]] Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models(https://arxiv.org/abs/2501.15446)
• Keywords: robust, transformer
• Abstract: Modern language models paradoxically combine unprecedented capability with persistent vulnerability in that they can draft poetry yet cannot reliably refuse harmful requests. We reveal this fragility stems not from inadequate training, but from a fundamental architectural limitation: transformers process all tokens as equals. Transformers operate as computational democracies, granting equal voice to all tokens. This is a design tragically unsuited for AGI, where we cannot risk adversarial "candidates" hijacking the system. Through formal analysis, we demonstrate that safety instructions fundamentally lack privileged status in transformer architectures, that they compete with adversarial inputs in the same computational arena, making robust alignment through prompting or fine-tuning inherently limited. This "token democracy" explains why jailbreaks bypass even extensively safety-trained models and why positional shifts erode prompt effectiveness. Our work systematizes practitioners' tacit knowledge into an architectural critique, showing current alignment approaches create mere preferences, not constraints.

Title: SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity

• Authors: Zichen Fan, Steve Dai, Rangharajan Venkatesan, Dennis Sylvester, Brucek Khailany
• Subjects: cs.CV, cs.AI, cs.AR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15448
• Pdf URL: https://arxiv.org/pdf/2501.15448
• Copy Paste: [[2501.15448]] SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity(https://arxiv.org/abs/2501.15448)
• Keywords: diffusion
• Abstract: Diffusion models have gained significant popularity in image generation tasks. However, generating high-quality content remains notably slow because it requires running model inference over many time steps. To accelerate these models, we propose to aggressively quantize both weights and activations, while simultaneously promoting significant activation sparsity. We further observe that the stated sparsity pattern varies among different channels and evolves across time steps. To support this quantization and sparsity scheme, we present a novel diffusion model accelerator featuring a heterogeneous mixed-precision dense-sparse architecture, channel-last address mapping, and a time-step-aware sparsity detector for efficient handling of the sparsity pattern. Our 4-bit quantization technique demonstrates superior generation quality compared to existing 4-bit methods. Our custom accelerator achieves 6.91x speed-up and 51.5% energy reduction compared to traditional dense accelerators.

Title: STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection

• Authors: Zewen Bai, Yuanyuan Sun, Shengdi Yin, Junyu Lu, Jingjie Zeng, Haohao Zhu, Liang Yang, Hongfei Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15451
• Pdf URL: https://arxiv.org/pdf/2501.15451
• Copy Paste: [[2501.15451]] STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection(https://arxiv.org/abs/2501.15451)
• Keywords: extraction
• Abstract: The proliferation of hate speech has caused significant harm to society. The intensity and directionality of hate are closely tied to the target and argument it is associated with. However, research on hate speech detection in Chinese has lagged behind, and existing datasets lack span-level fine-grained annotations. Furthermore, the lack of research on Chinese hateful slang poses a significant challenge. In this paper, we provide a solution for fine-grained detection of Chinese hate speech. First, we construct a dataset containing Target-Argument-Hateful-Group quadruples (STATE ToxiCN), which is the first span-level Chinese hate speech dataset. Secondly, we evaluate the span-level hate speech detection performance of existing models using STATE ToxiCN. Finally, we conduct the first study on Chinese hateful slang and evaluate the ability of LLMs to detect such expressions. Our work contributes valuable resources and insights to advance span-level hate speech detection in Chinese

Title: Identifying Critical Tokens for Accurate Predictions in Transformer-based Medical Imaging Models

• Authors: Solha Kang, Joris Vankerschaver, Utku Ozbulak
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15452
• Pdf URL: https://arxiv.org/pdf/2501.15452
• Copy Paste: [[2501.15452]] Identifying Critical Tokens for Accurate Predictions in Transformer-based Medical Imaging Models(https://arxiv.org/abs/2501.15452)
• Keywords: transformer
• Abstract: With the advancements in self-supervised learning (SSL), transformer-based computer vision models have recently demonstrated superior results compared to convolutional neural networks (CNNs) and are poised to dominate the field of artificial intelligence (AI)-based medical imaging in the upcoming years. Nevertheless, similar to CNNs, unveiling the decision-making process of transformer-based models remains a challenge. In this work, we take a step towards demystifying the decision-making process of transformer-based medical imaging models and propose Token Insight, a novel method that identifies the critical tokens that contribute to the prediction made by the model. Our method relies on the principled approach of token discarding native to transformer-based models, requires no additional module, and can be applied to any transformer model. Using the proposed approach, we quantify the importance of each token based on its contribution to the prediction and enable a more nuanced understanding of the model's decisions. Our experimental results which are showcased on the problem of colonic polyp identification using both supervised and self-supervised pretrained vision transformers indicate that Token Insight contributes to a more transparent and interpretable transformer-based medical imaging model, fostering trust and facilitating broader adoption in clinical settings.

Title: Data-adaptive Safety Rules for Training Reward Models

• Authors: Xiaomin Li, Mingye Gao, Zhiwei Zhang, Jingxuan Fan, Weiyu Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15453
• Pdf URL: https://arxiv.org/pdf/2501.15453
• Copy Paste: [[2501.15453]] Data-adaptive Safety Rules for Training Reward Models(https://arxiv.org/abs/2501.15453)
• Keywords: large language model
• Abstract: Reinforcement Learning from Human Feedback (RLHF) is commonly employed to tailor models to human preferences, especially to improve the safety of outputs from large language models (LLMs). Traditionally, this method depends on selecting preferred responses from pairs. However, due to the variability in human opinions and the challenges in directly comparing two responses, there is an increasing trend towards fine-grained annotation approaches that evaluate responses using multiple targeted metrics or rules. The challenge lies in efficiently choosing and applying these rules to handle the diverse range of preference data. In this paper, we propose a dynamic method that adaptively selects the most important rules for each response pair. We introduce a mathematical framework that utilizes the maximum discrepancy across paired responses and demonstrate theoretically that this approach maximizes the mutual information between the rule-based annotations and the underlying true preferences. We then train an 8B reward model using this adaptively labeled preference dataset and assess its efficacy using RewardBench. As of January 25, 2025, our model achieved the highest safety performance on the leaderboard, surpassing various larger models.

Title: FiberPool: Leveraging Multiple Blockchains for Decentralized Pooled Mining

• Authors: Akira Sakurai, Kazuyuki Shudo
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15459
• Pdf URL: https://arxiv.org/pdf/2501.15459
• Copy Paste: [[2501.15459]] FiberPool: Leveraging Multiple Blockchains for Decentralized Pooled Mining(https://arxiv.org/abs/2501.15459)
• Keywords: security, fair
• Abstract: The security of blockchain systems based on Proof of Work relies on mining. However, mining suffers from unstable revenue, prompting many miners to form cooperative mining pools. Most existing mining pools operate in a centralized manner, which undermines the decentralization principle of blockchain. Distributed mining pools offer a practical solution to this problem. Well-known examples include P2Pool and SmartPool. However, P2Pool encounters scalability and security issues in its early stages. Similarly, SmartPool is not budget-balanced and imposes fees due to its heavy use of the smart contract. In this research, we present a distributed mining pool named FiberPool to address these challenges. FiberPool integrates a smart contract on the main chain, a storage chain for sharing data necessary for share verification, and a child chain to reduce fees associated with using and withdrawing block rewards. We validate the mining fairness, budget balance, reward stability, and incentive compatibility of the payment scheme FiberPool Proportional adopted by FiberPool.

Title: TractoGPT: A GPT architecture for White Matter Segmentation

• Authors: Anoushkrit Goel, Simroop Singh, Ankita Joshi, Ranjeet Ranjan Jha, Chirag Ahuja, Aditya Nigam, Arnav Bhavsar
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15464
• Pdf URL: https://arxiv.org/pdf/2501.15464
• Copy Paste: [[2501.15464]] TractoGPT: A GPT architecture for White Matter Segmentation(https://arxiv.org/abs/2501.15464)
• Keywords: segmentation
• Abstract: White matter bundle segmentation is crucial for studying brain structural connectivity, neurosurgical planning, and neurological disorders. White Matter Segmentation remains challenging due to structural similarity in streamlines, subject variability, symmetry in 2 hemispheres, etc. To address these challenges, we propose TractoGPT, a GPT-based architecture trained on streamline, cluster, and fusion data representations separately. TractoGPT is a fully-automatic method that generalizes across datasets and retains shape information of the white matter bundles. Experiments also show that TractoGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores. We use TractoInferno and 105HCP datasets and validate generalization across dataset.

Title: Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning

• Authors: Jiawei Huang, Aimin Wang, Geng Sun, Jiahui Li, Jiacheng Wang, Dusit Niyato, Victor C. M. Leung
• Subjects: cs.CR, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.15468
• Pdf URL: https://arxiv.org/pdf/2501.15468
• Copy Paste: [[2501.15468]] Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning(https://arxiv.org/abs/2501.15468)
• Keywords: secure, security, transformer, generative
• Abstract: Low Earth Orbit (LEO) satellites can be used to assist maritime wireless communications for data transmission across wide-ranging areas. However, extensive coverage of LEO satellites, combined with openness of channels, can cause the communication process to suffer from security risks. This paper presents a low-altitude friendly-jamming LEO satellite-maritime communication system enabled by a unmanned aerial vehicle (UAV) to ensure data security at the physical layer. Since such a system requires trade-off policies that balance the secrecy rate and energy consumption of the UAV to meet evolving scenario demands, we formulate a secure satellite-maritime communication multi-objective optimization problem (SSMCMOP). In order to solve the dynamic and long-term optimization problem, we reformulate it into a Markov decision process. We then propose a transformer-enhanced soft actor critic (TransSAC) algorithm, which is a generative artificial intelligence-enable deep reinforcement learning approach to solve the reformulated problem, so that capturing global dependencies and diversely exploring weights. Simulation results demonstrate that the TransSAC outperforms various baselines, and achieves an optimal secrecy rate while effectively minimizing the energy consumption of the UAV. Moreover, the results find more suitable constraint values for the system.

Title: CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry

• Authors: David Tschirschwitz, Volker Rodehorst
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15469
• Pdf URL: https://arxiv.org/pdf/2501.15469
• Copy Paste: [[2501.15469]] CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry(https://arxiv.org/abs/2501.15469)
• Keywords: robust, extraction
• Abstract: Reproducibility and replicability are critical pillars of empirical research, particularly in machine learning, where they depend not only on the availability of models, but also on the datasets used to train and evaluate those models. In this paper, we introduce the Construction Industry Steel Ordering List (CISOL) dataset, which was developed with a focus on transparency to ensure reproducibility, replicability, and extensibility. CISOL provides a valuable new research resource and highlights the importance of having diverse datasets, even in niche application domains such as table extraction in civil engineering. CISOL is unique in that it contains real-world civil engineering documents from industry, making it a distinctive contribution to the field. The dataset contains more than 120,000 annotated instances in over 800 document images, positioning it as a medium-sized dataset that provides a robust foundation for Table Structure Recognition (TSR) and Table Detection (TD) tasks. Benchmarking results show that CISOL achieves 67.22 mAP@0.5:0.95:0.05 using the YOLOv8 model, outperforming the TSR-specific TATR model. This highlights the effectiveness of CISOL as a benchmark for advancing TSR, especially in specialized domains.

Title: LoRAGuard: An Effective Black-box Watermarking Approach for LoRAs

• Authors: Peizhuo Lv, Yiran Xiahou, Congyi Li, Mengjie Sun, Shengzhi Zhang, Kai Chen, Yingjun Zhang
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15478
• Pdf URL: https://arxiv.org/pdf/2501.15478
• Copy Paste: [[2501.15478]] LoRAGuard: An Effective Black-box Watermarking Approach for LoRAs(https://arxiv.org/abs/2501.15478)
• Keywords: watermark, diffusion
• Abstract: LoRA (Low-Rank Adaptation) has achieved remarkable success in the parameter-efficient fine-tuning of large models. The trained LoRA matrix can be integrated with the base model through addition or negation operation to improve performance on downstream tasks. However, the unauthorized use of LoRAs to generate harmful content highlights the need for effective mechanisms to trace their usage. A natural solution is to embed watermarks into LoRAs to detect unauthorized misuse. However, existing methods struggle when multiple LoRAs are combined or negation operation is applied, as these can significantly degrade watermark performance. In this paper, we introduce LoRAGuard, a novel black-box watermarking technique for detecting unauthorized misuse of LoRAs. To support both addition and negation operations, we propose the Yin-Yang watermark technique, where the Yin watermark is verified during negation operation and the Yang watermark during addition operation. Additionally, we propose a shadow-model-based watermark training approach that significantly improves effectiveness in scenarios involving multiple integrated LoRAs. Extensive experiments on both language and diffusion models show that LoRAGuard achieves nearly 100% watermark verification success and demonstrates strong effectiveness.

Title: FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment

• Authors: Sunny Gupta, Vinay Sutar, Varunav Singh, Amit Sethi
• Subjects: cs.LG, cs.AI, cs.CV, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.15486
• Pdf URL: https://arxiv.org/pdf/2501.15486
• Copy Paste: [[2501.15486]] FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment(https://arxiv.org/abs/2501.15486)
• Keywords: privacy, robust, federate
• Abstract: Federated Learning (FL) offers a decentralized paradigm for collaborative model training without direct data sharing, yet it poses unique challenges for Domain Generalization (DG), including strict privacy constraints, non-i.i.d. local data, and limited domain diversity. We introduce FedAlign, a lightweight, privacy-preserving framework designed to enhance DG in federated settings by simultaneously increasing feature diversity and promoting domain invariance. First, a cross-client feature extension module broadens local domain representations through domain-invariant feature perturbation and selective cross-client feature transfer, allowing each client to safely access a richer domain space. Second, a dual-stage alignment module refines global feature learning by aligning both feature embeddings and predictions across clients, thereby distilling robust, domain-invariant features. By integrating these modules, our method achieves superior generalization to unseen domains while maintaining data privacy and operating with minimal computational and communication overhead.

Title: Color Flow Imaging Microscopy Improves Identification of Stress Sources of Protein Aggregates in Biopharmaceuticals

• Authors: Michaela Cohrs, Shiwoo Koak, Yejin Lee, Yu Jin Sung, Wesley De Neve, Hristo L. Svilenov, Utku Ozbulak
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15492
• Pdf URL: https://arxiv.org/pdf/2501.15492
• Copy Paste: [[2501.15492]] Color Flow Imaging Microscopy Improves Identification of Stress Sources of Protein Aggregates in Biopharmaceuticals(https://arxiv.org/abs/2501.15492)
• Keywords: robust, transformer
• Abstract: Protein-based therapeutics play a pivotal role in modern medicine targeting various diseases. Despite their therapeutic importance, these products can aggregate and form subvisible particles (SvPs), which can compromise their efficacy and trigger immunological responses, emphasizing the critical need for robust monitoring techniques. Flow Imaging Microscopy (FIM) has been a significant advancement in detecting SvPs, evolving from monochrome to more recently incorporating color imaging. Complementing SvP images obtained via FIM, deep learning techniques have recently been employed successfully for stress source identification of monochrome SvPs. In this study, we explore the potential of color FIM to enhance the characterization of stress sources in SvPs. To achieve this, we curate a new dataset comprising 16,000 SvPs from eight commercial monoclonal antibodies subjected to heat and mechanical stress. Using both supervised and self-supervised convolutional neural networks, as well as vision transformers in large-scale experiments, we demonstrate that deep learning with color FIM images consistently outperforms monochrome images, thus highlighting the potential of color FIM in stress source classification compared to its monochrome counterparts.

Title: Domain Adaptation from Generated Multi-Weather Images for Unsupervised Maritime Object Classification

• Authors: Dan Song, Shumeng Huo, Wenhui Li, Lanjun Wang, Chao Xue, An-An Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15503
• Pdf URL: https://arxiv.org/pdf/2501.15503
• Copy Paste: [[2501.15503]] Domain Adaptation from Generated Multi-Weather Images for Unsupervised Maritime Object Classification(https://arxiv.org/abs/2501.15503)
• Keywords: generative
• Abstract: The classification and recognition of maritime objects are crucial for enhancing maritime safety, monitoring, and intelligent sea environment prediction. However, existing unsupervised methods for maritime object classification often struggle with the long-tail data distributions in both object categories and weather conditions. In this paper, we construct a dataset named AIMO produced by large-scale generative models with diverse weather conditions and balanced object categories, and collect a dataset named RMO with real-world images where long-tail issue exists. We propose a novel domain adaptation approach that leverages AIMO (source domain) to address the problem of limited labeled data, unbalanced distribution and domain shift in RMO (target domain), and enhance the generalization of source features with the Vision-Language Models such as CLIP. Experimental results shows that the proposed method significantly improves the classification accuracy, particularly for samples within rare object categories and weather conditions. Datasets and codes will be publicly available at this https URL.

Title: FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint

• Authors: Shuo Shao, Haozhe Zhu, Hongwei Yao, Yiming Li, Tianwei Zhang, Zhan Qin, Kui Ren
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15509
• Pdf URL: https://arxiv.org/pdf/2501.15509
• Copy Paste: [[2501.15509]] FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint(https://arxiv.org/abs/2501.15509)
• Keywords: protect, attack
• Abstract: Model fingerprinting is a widely adopted approach to safeguard the intellectual property rights of open-source models by preventing their unauthorized reuse. It is promising and convenient since it does not necessitate modifying the protected model. In this paper, we revisit existing fingerprinting methods and reveal that they are vulnerable to false claim attacks where adversaries falsely assert ownership of any third-party model. We demonstrate that this vulnerability mostly stems from their untargeted nature, where they generally compare the outputs of given samples on different models instead of the similarities to specific references. Motivated by these findings, we propose a targeted fingerprinting paradigm (i.e., FIT-Print) to counteract false claim attacks. Specifically, FIT-Print transforms the fingerprint into a targeted signature via optimization. Building on the principles of FIT-Print, we develop bit-wise and list-wise black-box model fingerprinting methods, i.e., FIT-ModelDiff and FIT-LIME, which exploit the distance between model outputs and the feature attribution of specific samples as the fingerprint, respectively. Extensive experiments on benchmark models and datasets verify the effectiveness, conferrability, and resistance to false claim attacks of our FIT-Print.

Title: Universal Image Restoration Pre-training via Degradation Classification

• Authors: JiaKui Hu, Lujia Jin, Zhengjian Yao, Yanye Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15510
• Pdf URL: https://arxiv.org/pdf/2501.15510
• Copy Paste: [[2501.15510]] Universal Image Restoration Pre-training via Degradation Classification(https://arxiv.org/abs/2501.15510)
• Keywords: transformer
• Abstract: This paper proposes the Degradation Classification Pre-Training (DCPT), which enables models to learn how to classify the degradation type of input images for universal image restoration pre-training. Unlike the existing self-supervised pre-training methods, DCPT utilizes the degradation type of the input image as an extremely weak supervision, which can be effortlessly obtained, even intrinsic in all image restoration datasets. DCPT comprises two primary stages. Initially, image features are extracted from the encoder. Subsequently, a lightweight decoder, such as ResNet18, is leveraged to classify the degradation type of the input image solely based on the features extracted in the first stage, without utilizing the input image. The encoder is pre-trained with a straightforward yet potent DCPT, which is used to address universal image restoration and achieve outstanding performance. Following DCPT, both convolutional neural networks (CNNs) and transformers demonstrate performance improvements, with gains of up to 2.55 dB in the 10D all-in-one restoration task and 6.53 dB in the mixed degradation scenarios. Moreover, previous self-supervised pretraining methods, such as masked image modeling, discard the decoder after pre-training, while our DCPT utilizes the pre-trained parameters more effectively. This superiority arises from the degradation classifier acquired during DCPT, which facilitates transfer learning between models of identical architecture trained on diverse degradation types. Source code and models are available at this https URL.

Title: Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition

• Authors: Ying Zheng, Yiyi Zhang, Yi Wang, Lap-Pui Chau
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15519
• Pdf URL: https://arxiv.org/pdf/2501.15519
• Copy Paste: [[2501.15519]] Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition(https://arxiv.org/abs/2501.15519)
• Keywords: privacy, protect, robust
• Abstract: Source-free domain adaptation in visual emotion recognition (SFDA-VER) is a highly challenging task that requires adapting VER models to the target domain without relying on source data, which is of great significance for data privacy protection. However, due to the unignorable disparities between visual emotion data and traditional image classification data, existing SFDA methods perform poorly on this task. In this paper, we investigate the SFDA-VER task from a fuzzy perspective and identify two key issues: fuzzy emotion labels and fuzzy pseudo-labels. These issues arise from the inherent uncertainty of emotion annotations and the potential mispredictions in pseudo-labels. To address these issues, we propose a novel fuzzy-aware loss (FAL) to enable the VER model to better learn and adapt to new domains under fuzzy labels. Specifically, FAL modifies the standard cross entropy loss and focuses on adjusting the losses of non-predicted categories, which prevents a large number of uncertain or incorrect predictions from overwhelming the VER model during adaptation. In addition, we provide a theoretical analysis of FAL and prove its robustness in handling the noise in generated pseudo-labels. Extensive experiments on 26 domain adaptation sub-tasks across three benchmark datasets demonstrate the effectiveness of our method.

Title: UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning

• Authors: Oubo Ma, Linkang Du, Yang Dai, Chunyi Zhou, Qingming Li, Yuwen Pu, Shouling Ji
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15529
• Pdf URL: https://arxiv.org/pdf/2501.15529
• Copy Paste: [[2501.15529]] UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning(https://arxiv.org/abs/2501.15529)
• Keywords: attack, steal
• Abstract: Deep reinforcement learning (DRL) is widely applied to safety-critical decision-making scenarios. However, DRL is vulnerable to backdoor attacks, especially action-level backdoors, which pose significant threats through precise manipulation and flexible activation, risking outcomes like vehicle collisions or drone crashes. The key distinction of action-level backdoors lies in the utilization of the backdoor reward function to associate triggers with target actions. Nevertheless, existing studies typically rely on backdoor reward functions with fixed values or conditional flipping, which lack universality across diverse DRL tasks and backdoor designs, resulting in fluctuations or even failure in practice. This paper proposes the first universal action-level backdoor attack framework, called UNIDOOR, which enables adaptive exploration of backdoor reward functions through performance monitoring, eliminating the reliance on expert knowledge and grid search. We highlight that action tampering serves as a crucial component of action-level backdoor attacks in continuous action scenarios, as it addresses attack failures caused by low-frequency target actions. Extensive evaluations demonstrate that UNIDOOR significantly enhances the attack performance of action-level backdoors, showcasing its universality across diverse attack scenarios, including single/multiple agents, single/multiple backdoors, discrete/continuous action spaces, and sparse/dense reward signals. Furthermore, visualization results encompassing state distribution, neuron activation, and animations demonstrate the stealthiness of UNIDOOR. The source code of UNIDOOR can be found at this https URL.

Title: Advancing Generative Artificial Intelligence and Large Language Models for Demand Side Management with Electric Vehicles

• Authors: Hanwen Zhang, Ruichen Zhang, Wei Zhang, Dusit Niyato, Yonggang Wen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15544
• Pdf URL: https://arxiv.org/pdf/2501.15544
• Copy Paste: [[2501.15544]] Advancing Generative Artificial Intelligence and Large Language Models for Demand Side Management with Electric Vehicles(https://arxiv.org/abs/2501.15544)
• Keywords: generative, large language model
• Abstract: Generative artificial intelligence, particularly through large language models (LLMs), is poised to transform energy optimization and demand side management (DSM) within microgrids. This paper explores the integration of LLMs into energy management, emphasizing their roles in automating the optimization of DSM strategies with electric vehicles. We investigate challenges and solutions associated with DSM and explore the new opportunities presented by leveraging LLMs. Then, We propose an innovative solution that enhances LLMs with retrieval-augmented generation for automatic problem formulation, code generation, and customizing optimization. We present a case study to demonstrate the effectiveness of our proposed solution in charging scheduling and optimization for electric vehicles, highlighting our solution's significant advancements in energy efficiency and user adaptability. This work underscores the potential of LLMs for energy optimization and fosters a new era of intelligent DSM solutions.

Title: Building Efficient Lightweight CNN Models

• Authors: Nathan Isong
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15547
• Pdf URL: https://arxiv.org/pdf/2501.15547
• Copy Paste: [[2501.15547]] Building Efficient Lightweight CNN Models(https://arxiv.org/abs/2501.15547)
• Keywords: robust, extraction
• Abstract: Convolutional Neural Networks (CNNs) are pivotal in image classification tasks due to their robust feature extraction capabilities. However, their high computational and memory requirements pose challenges for deployment in resource-constrained environments. This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy. The approach integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing. The dual-input-output model train on original and augmented datasets, enhancing robustness. Progressive unfreezing is applied to the unified model to optimize pre-learned features during fine-tuning, enabling faster convergence and improved model accuracy. The methodology was evaluated on three benchmark datasets; handwritten digit MNIST, fashion MNIST, and CIFAR-10. The proposed model achieved a state-of-the-art accuracy of 99% on the handwritten digit MNIST and 89% on fashion MNIST, with only 14,862 parameters and a model size of 0.17 MB. While performance on CIFAR-10 was comparatively lower (65% with less than 20,00 parameters), the results highlight the scalability of this method. The final model demonstrated fast inference times and low latency, making it suitable for real-time applications. Future directions include exploring advanced augmentation techniques, improving architectural scalability for complex datasets, and extending the methodology to tasks beyond classification. This research underscores the potential for creating efficient, scalable, and task-specific CNNs for diverse applications.

Title: Optimal Transport on Categorical Data for Counterfactuals using Compositional Data and Dirichlet Transport

• Authors: Agathe Fernandes Machado, Arthur Charpentier, Ewen Gallic
• Subjects: cs.LG, stat.ME
• Abstract URL: https://arxiv.org/abs/2501.15549
• Pdf URL: https://arxiv.org/pdf/2501.15549
• Copy Paste: [[2501.15549]] Optimal Transport on Categorical Data for Counterfactuals using Compositional Data and Dirichlet Transport(https://arxiv.org/abs/2501.15549)
• Keywords: fair
• Abstract: Recently, optimal transport-based approaches have gained attention for deriving counterfactuals, e.g., to quantify algorithmic discrimination. However, in the general multivariate setting, these methods are often opaque and difficult to interpret. To address this, alternative methodologies have been proposed, using causal graphs combined with iterative quantile regressions (Plečko and Meinshausen (2020)) or sequential transport (Fernandes Machado et al. (2025)) to examine fairness at the individual level, often referred to as ``counterfactual fairness.'' Despite these advancements, transporting categorical variables remains a significant challenge in practical applications with real datasets. In this paper, we propose a novel approach to address this issue. Our method involves (1) converting categorical variables into compositional data and (2) transporting these compositions within the probabilistic simplex of $\mathbb{R}^d$. We demonstrate the applicability and effectiveness of this approach through an illustration on real-world data, and discuss limitations.

Title: Real-CATS: A Practical Training Ground for Emerging Research on Cryptocurrency Cybercrime Detection

• Authors: Jiadong Shi, Chunyu Duan, Hao Lei, Liangmin Wang
• Subjects: cs.CR, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.15553
• Pdf URL: https://arxiv.org/pdf/2501.15553
• Copy Paste: [[2501.15553]] Real-CATS: A Practical Training Ground for Emerging Research on Cryptocurrency Cybercrime Detection(https://arxiv.org/abs/2501.15553)
• Keywords: security
• Abstract: Cybercriminals pose a significant threat to blockchain trading security, causing $40.9 billion in losses in 2024. However, the lack of an effective real-world address dataset hinders the advancement of cybercrime detection research. The anti-cybercrime efforts of researchers from broader fields, such as statistics and artificial intelligence, are blocked by data scarcity. In this paper, we present Real-CATS, a Real-world dataset of Cryptocurrency Addresses with Transaction profileS, serving as a practical training ground for developing and assessing detection methods. Real-CATS comprises 103,203 criminal addresses from real-world reports and 106,196 benign addresses from exchange customers. It satifies the C3R characteristics (Comprehensiveness, Classifiability, Customizability, and Real-world Transferability), which are fundemental for practical detection of cryptocurrency cybercrime. The dataset provides three main functions: 1) effective evaluation of detection methods, 2) support for feature extensions, and 3) a new evaluation scenario for real-world deployment. Real-CATS also offers opportunities to expand cybercrime measurement studies. It is particularly beneficial for researchers without cryptocurrency-related knowledge to engage in this emerging research field. We hope that studies on cryptocurrency cybercrime detection will be promoted by an increasing number of cross-disciplinary researchers drawn to this versatile data platform. All datasets are available at this https URL

Title: BoTier: Multi-Objective Bayesian Optimization with Tiered Composite Objectives

• Authors: Mohammad Haddadnia, Leonie Grashoff, Felix Strieth-Kalthoff
• Subjects: cs.LG, math.OC, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.15554
• Pdf URL: https://arxiv.org/pdf/2501.15554
• Copy Paste: [[2501.15554]] BoTier: Multi-Objective Bayesian Optimization with Tiered Composite Objectives(https://arxiv.org/abs/2501.15554)
• Keywords: robust
• Abstract: Scientific optimization problems are usually concerned with balancing multiple competing objectives, which come as preferences over both the outcomes of an experiment (e.g. maximize the reaction yield) and the corresponding input parameters (e.g. minimize the use of an expensive reagent). Typically, practical and economic considerations define a hierarchy over these objectives, which must be reflected in algorithms for sample-efficient experiment planning. Herein, we introduce BoTier, a composite objective that can flexibly represent a hierarchy of preferences over both experiment outcomes and input parameters. We provide systematic benchmarks on synthetic and real-life surfaces, demonstrating the robust applicability of BoTier across a number of use cases. Importantly, BoTier is implemented in an auto-differentiable fashion, enabling seamless integration with the BoTorch library, thereby facilitating adoption by the scientific community.

Title: Distributionally Robust Graph Out-of-Distribution Recommendation via Diffusion Model

• Authors: Chu Zhao, Enneng Yang, Yuliang Liang, Jianzhe Zhao, Guibing Guo, Xingwei Wang
• Subjects: cs.LG, cs.AI, cs.GR, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.15555
• Pdf URL: https://arxiv.org/pdf/2501.15555
• Copy Paste: [[2501.15555]] Distributionally Robust Graph Out-of-Distribution Recommendation via Diffusion Model(https://arxiv.org/abs/2501.15555)
• Keywords: robust, diffusion
• Abstract: The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theoretical analysis, this paper reveals that current DRO-based graph recommendation methods assign greater weight to noise distribution, leading to model parameter learning being dominated by it. When the model overly focuses on fitting noise samples in the training data, it may learn irrelevant or meaningless features that cannot be generalized to OOD data. To address this challenge, we design a Distributionally Robust Graph model for OOD recommendation (DRGO). Specifically, our method first employs a simple and effective diffusion paradigm to alleviate the noisy effect in the latent space. Additionally, an entropy regularization term is introduced in the DRO objective function to avoid extreme sample weights in the worst-case distribution. Finally, we provide a theoretical proof of the generalization error bound of DRGO as well as a theoretical analysis of how our approach mitigates noisy sample effects, which helps to better understand the proposed framework from a theoretical perspective. We conduct extensive experiments on four datasets to evaluate the effectiveness of our framework against three typical distribution shifts, and the results demonstrate its superiority in both independently and identically distributed distributions (IID) and OOD.

Title: Ocean-OCR: Towards General OCR Application via a Vision-Language Model

• Authors: Song Chen, Xinyu Guo, Yadong Li, Tao Zhang, Mingan Lin, Dongdong Kuang, Youwei Zhang, Lingfeng Ming, Fengyu Zhang, Yuran Wang, Jianhua Xu, Zenan Zhou, Weipeng Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15558
• Pdf URL: https://arxiv.org/pdf/2501.15558
• Copy Paste: [[2501.15558]] Ocean-OCR: Towards General OCR Application via a Vision-Language Model(https://arxiv.org/abs/2501.15558)
• Keywords: robust, large language model
• Abstract: Multimodal large language models (MLLMs) have shown impressive capabilities across various domains, excelling in processing and understanding information from multiple modalities. Despite the rapid progress made previously, insufficient OCR ability hinders MLLMs from excelling in text-related tasks. In this paper, we present \textbf{Ocean-OCR}, a 3B MLLM with state-of-the-art performance on various OCR scenarios and comparable understanding ability on general tasks. We employ Native Resolution ViT to enable variable resolution input and utilize a substantial collection of high-quality OCR datasets to enhance the model performance. We demonstrate the superiority of Ocean-OCR through comprehensive experiments on open-source OCR benchmarks and across various OCR scenarios. These scenarios encompass document understanding, scene text recognition, and handwritten recognition, highlighting the robust OCR capabilities of Ocean-OCR. Note that Ocean-OCR is the first MLLM to outperform professional OCR models such as TextIn and PaddleOCR.

Title: CE-SDWV: Effective and Efficient Concept Erasure for Text-to-Image Diffusion Models via a Semantic-Driven Word Vocabulary

• Authors: Jiahang Tu, Qian Feng, Chufan Chen, Jiahua Dong, Hanbin Zhao, Chao Zhang, Hui Qian
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15562
• Pdf URL: https://arxiv.org/pdf/2501.15562
• Copy Paste: [[2501.15562]] CE-SDWV: Effective and Efficient Concept Erasure for Text-to-Image Diffusion Models via a Semantic-Driven Word Vocabulary(https://arxiv.org/abs/2501.15562)
• Keywords: privacy, diffusion, generative
• Abstract: Large-scale text-to-image (T2I) diffusion models have achieved remarkable generative performance about various concepts. With the limitation of privacy and safety in practice, the generative capability concerning NSFW (Not Safe For Work) concepts is undesirable, e.g., producing sexually explicit photos, and licensed images. The concept erasure task for T2I diffusion models has attracted considerable attention and requires an effective and efficient method. To achieve this goal, we propose a CE-SDWV framework, which removes the target concepts (e.g., NSFW concepts) of T2I diffusion models in the text semantic space by only adjusting the text condition tokens and does not need to re-train the original T2I diffusion model's weights. Specifically, our framework first builds a target concept-related word vocabulary to enhance the representation of the target concepts within the text semantic space, and then utilizes an adaptive semantic component suppression strategy to ablate the target concept-related semantic information in the text condition tokens. To further adapt the above text condition tokens to the original image semantic space, we propose an end-to-end gradient-orthogonal token optimization strategy. Extensive experiments on I2P and UnlearnCanvas benchmarks demonstrate the effectiveness and efficiency of our method.

Title: PCAP-Backdoor: Backdoor Poisoning Generator for Network Traffic in CPS/IoT Environments

• Authors: Ajesh Koyatan Chathoth, Stephen Lee
• Subjects: cs.LG, cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.15563
• Pdf URL: https://arxiv.org/pdf/2501.15563
• Copy Paste: [[2501.15563]] PCAP-Backdoor: Backdoor Poisoning Generator for Network Traffic in CPS/IoT Environments(https://arxiv.org/abs/2501.15563)
• Keywords: security, defense, attack
• Abstract: The rapid expansion of connected devices has made them prime targets for cyberattacks. To address these threats, deep learning-based, data-driven intrusion detection systems (IDS) have emerged as powerful tools for detecting and mitigating such attacks. These IDSs analyze network traffic to identify unusual patterns and anomalies that may indicate potential security breaches. However, prior research has shown that deep learning models are vulnerable to backdoor attacks, where attackers inject triggers into the model to manipulate its behavior and cause misclassifications of network traffic. In this paper, we explore the susceptibility of deep learning-based IDS systems to backdoor attacks in the context of network traffic analysis. We introduce \texttt{PCAP-Backdoor}, a novel technique that facilitates backdoor poisoning attacks on PCAP datasets. Our experiments on real-world Cyber-Physical Systems (CPS) and Internet of Things (IoT) network traffic datasets demonstrate that attackers can effectively backdoor a model by poisoning as little as 1\% or less of the entire training dataset. Moreover, we show that an attacker can introduce a trigger into benign traffic during model training yet cause the backdoored model to misclassify malicious traffic when the trigger is present. Finally, we highlight the difficulty of detecting this trigger-based backdoor, even when using existing backdoor defense techniques.

Title: ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer

• Authors: Lin Yueyu, Li Zhiyuan, Peter Yue, Liu Xiao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15570
• Pdf URL: https://arxiv.org/pdf/2501.15570
• Copy Paste: [[2501.15570]] ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer(https://arxiv.org/abs/2501.15570)
• Keywords: transformer
• Abstract: As is known, hybrid quadratic and subquadratic attention models in multi-head architectures have surpassed both Transformer and Linear RNN models , with these works primarily focusing on reducing KV complexity and improving efficiency. For further research on expressiveness, we introduce our series of models distilled from Qwen 2.5, based on pure native RWKV-7 attention, which aims to make RNN more expressive and demonstrates state tracking ability beyond transformers. We work with QRWK 32B based on RWKV-6 architecture, another approach that reduces the entire knowledge processing time to just 8 hours using 16 AMD MI300X GPUs while maintaining Qwen 2.5's performance. In fact, the distillation process can utilize any LLM, not just Qwen, and enables knowledge transfer from larger LLMs to smaller ones with more fewer tokens. We will explain the detailed process and share our insights on building more powerful foundation models. Please note that this is an ongoing work that will be updated continuously. The model checkpoints and source code are available at \href{this https URL}{this https URL}, \href{this https URL}{this https URL}.

Title: Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models

• Authors: Spencer Ramsey, Amina Grant, Jeffrey Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15571
• Pdf URL: https://arxiv.org/pdf/2501.15571
• Copy Paste: [[2501.15571]] Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models(https://arxiv.org/abs/2501.15571)
• Keywords: diffusion, large language model
• Abstract: Fashion content generation is an emerging area at the intersection of artificial intelligence and creative design, with applications ranging from virtual try-on to culturally diverse design prototyping. Existing methods often struggle with cultural bias, limited scalability, and alignment between textual prompts and generated visuals, particularly under weak supervision. In this work, we propose a novel framework that integrates Large Language Models (LLMs) with Latent Diffusion Models (LDMs) to address these challenges. Our method leverages LLMs for semantic refinement of textual prompts and introduces a weak supervision filtering module to effectively utilize noisy or weakly labeled data. By fine-tuning the LDM on an enhanced DeepFashion+ dataset enriched with global fashion styles, the proposed approach achieves state-of-the-art performance. Experimental results demonstrate that our method significantly outperforms baselines, achieving lower Frechet Inception Distance (FID) and higher Inception Scores (IS), while human evaluations confirm its ability to generate culturally diverse and semantically relevant fashion content. These results highlight the potential of LLM-guided diffusion models in driving scalable and inclusive AI-driven fashion innovation.

Title: Approximate Message Passing for Bayesian Neural Networks

• Authors: Romeo Sommerfeld, Christian Helms, Ralf Herbrich
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15573
• Pdf URL: https://arxiv.org/pdf/2501.15573
• Copy Paste: [[2501.15573]] Approximate Message Passing for Bayesian Neural Networks(https://arxiv.org/abs/2501.15573)
• Keywords: interpretability
• Abstract: Bayesian neural networks (BNNs) offer the potential for reliable uncertainty quantification and interpretability, which are critical for trustworthy AI in high-stakes domains. However, existing methods often struggle with issues such as overconfidence, hyperparameter sensitivity, and posterior collapse, leaving room for alternative approaches. In this work, we advance message passing (MP) for BNNs and present a novel framework that models the predictive posterior as a factor graph. To the best of our knowledge, our framework is the first MP method that handles convolutional neural networks and avoids double-counting training data, a limitation of previous MP methods that causes overconfidence. We evaluate our approach on CIFAR-10 with a convolutional neural network of roughly 890k parameters and find that it can compete with the SOTA baselines AdamW and IVON, even having an edge in terms of calibration. On synthetic data, we validate the uncertainty estimates and observe a strong correlation (0.9) between posterior credible intervals and its probability of covering the true data-generating function outside the training range. While our method scales to an MLP with 5.6 million parameters, further improvements are necessary to match the scale and performance of state-of-the-art variational inference methods.

Title: Instruction Tuning for Story Understanding and Generation with Weak Supervision

• Authors: Yangshu Yuan, Heng Chen, Christian Ng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15574
• Pdf URL: https://arxiv.org/pdf/2501.15574
• Copy Paste: [[2501.15574]] Instruction Tuning for Story Understanding and Generation with Weak Supervision(https://arxiv.org/abs/2501.15574)
• Keywords: generative, large language model
• Abstract: Story understanding and generation have long been a challenging task in natural language processing (NLP), especially when dealing with various levels of instruction specificity. In this paper, we propose a novel approach called "Weak to Strong Instruction Tuning" for improving story generation by tuning models with instructions of varying clarity. We explore the potential of large language models (LLMs) to adapt to different types of instructions, weak and strong, and show that our method significantly enhances performance in story comprehension and generation. By leveraging the strength of instruction tuning, we train models to understand the nuances of story plots, characters, and themes while generating coherent and engaging narratives. Through extensive experiments on several benchmark datasets and comparison with state-of-the-art baselines, we demonstrate that our method outperforms existing techniques, yielding substantial improvements in both automatic evaluation metrics and human evaluations. Our work shows that adaptive instruction tuning can be a powerful tool in refining generative models for complex narrative tasks.

Title: A Complexity-Informed Approach to Optimise Cyber Defences

• Authors: Lampis Alevizos
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15578
• Pdf URL: https://arxiv.org/pdf/2501.15578
• Copy Paste: [[2501.15578]] A Complexity-Informed Approach to Optimise Cyber Defences(https://arxiv.org/abs/2501.15578)
• Keywords: security
• Abstract: This paper introduces a novel complexity-informed approach to cybersecurity management, addressing the challenges found within complex cyber defences. We adapt and extend the complexity theory to cybersecurity and develop a quantitative framework that empowers decision-makers with strategies to de-complexify defences, identify improvement opportunities, and resolve bottlenecks. Our approach also provides a solid foundation for critical cybersecurity decisions, such as tooling investment or divestment, workforce capacity planning, and optimisation of processes and capabilities. Through a case study, we detail and validate a systematic method for assessing and managing the complexity within cybersecurity defences. The complexity-informed approach based on MITRE ATT&CK, is designed to complement threat-informed defences. Threat-informed methods focus on understanding and countering adversary tactics, while the complexity-informed approach optimises the underlying defence infrastructure, thereby optimising the overall efficiency and effectiveness of cyber defences.

Title: ConceptCLIP: Towards Trustworthy Medical AI via Concept-Enhanced Contrastive Langauge-Image Pre-training

• Authors: Yuxiang Nie, Sunan He, Yequan Bie, Yihui Wang, Zhixuan Chen, Shu Yang, Hao Chen
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15579
• Pdf URL: https://arxiv.org/pdf/2501.15579
• Copy Paste: [[2501.15579]] ConceptCLIP: Towards Trustworthy Medical AI via Concept-Enhanced Contrastive Langauge-Image Pre-training(https://arxiv.org/abs/2501.15579)
• Keywords: robust, interpretability, explainability
• Abstract: Trustworthiness is essential for the precise and interpretable application of artificial intelligence (AI) in medical imaging. Traditionally, precision and interpretability have been addressed as separate tasks, namely medical image analysis and explainable AI, each developing its own models independently. In this study, for the first time, we investigate the development of a unified medical vision-language pre-training model that can achieve both accurate analysis and interpretable understanding of medical images across various modalities. To build the model, we construct MedConcept-23M, a large-scale dataset comprising 23 million medical image-text pairs extracted from 6.2 million scientific articles, enriched with concepts from the Unified Medical Language System (UMLS). Based on MedConcept-23M, we introduce ConceptCLIP, a medical AI model utilizing concept-enhanced contrastive language-image pre-training. The pre-training of ConceptCLIP involves two primary components: image-text alignment learning (IT-Align) and patch-concept alignment learning (PC-Align). This dual alignment strategy enhances the model's capability to associate specific image regions with relevant concepts, thereby improving both the precision of analysis and the interpretability of the AI system. We conducted extensive experiments on 5 diverse types of medical image analysis tasks, spanning 51 subtasks across 10 image modalities, with the broadest range of downstream tasks. The results demonstrate the effectiveness of the proposed vision-language pre-training model. Further explainability analysis across 6 modalities reveals that ConceptCLIP achieves superior performance, underscoring its robust ability to advance explainable AI in medical imaging. These findings highlight ConceptCLIP's capability in promoting trustworthy AI in the field of medicine.

Title: Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework

• Authors: Yuhong Sun, Zhangyue Yin, Xuanjing Huang, Xipeng Qiu, Hui Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15581
• Pdf URL: https://arxiv.org/pdf/2501.15581
• Copy Paste: [[2501.15581]] Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework(https://arxiv.org/abs/2501.15581)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains. Math Word Problems (MWPs) serve as a crucial benchmark for evaluating LLMs' reasoning abilities. While most research primarily focuses on improving accuracy, it often neglects understanding and addressing the underlying patterns of errors. Current error classification methods rely on static and predefined categories, which limit their ability to capture the full spectrum of error patterns in mathematical reasoning. To enable systematic error analysis, we collect error samples from 15 different LLMs of varying sizes across four distinct MWP datasets using multiple sampling strategies. Based on this extensive collection, we introduce MWPES-300K, a comprehensive dataset containing 304,865 error samples that cover diverse error patterns and reasoning paths. To reduce human bias and enable fine-grained analysis of error patterns, we propose a novel framework for automated dynamic error classification in mathematical reasoning. Experimental results demonstrate that dataset characteristics significantly shape error patterns, which evolve from basic to complex manifestations as model capabilities increase. With deeper insights into error patterns, we propose error-aware prompting that incorporates common error patterns as explicit guidance, leading to significant improvements in mathematical reasoning performance.

Title: SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain

• Authors: Dakuan Lu, Xiaoyu Tan, Rui Xu, Tianchu Yao, Chao Qu, Wei Chu, Yinghui Xu, Yuan Qi
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.15587
• Pdf URL: https://arxiv.org/pdf/2501.15587
• Copy Paste: [[2501.15587]] SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain(https://arxiv.org/abs/2501.15587)
• Keywords: extraction, large language model
• Abstract: Recent breakthroughs in large language models (LLMs) exemplified by the impressive mathematical and scientific reasoning capabilities of the o1 model have spotlighted the critical importance of high-quality training data in advancing LLM performance across STEM disciplines. While the mathematics community has benefited from a growing body of curated datasets, the scientific domain at the higher education level has long suffered from a scarcity of comparable resources. To address this gap, we present SCP-116K, a new large-scale dataset of 116,756 high-quality problem-solution pairs, automatically extracted from heterogeneous sources using a streamlined and highly generalizable pipeline. Our approach involves stringent filtering to ensure the scientific rigor and educational level of the extracted materials, while maintaining adaptability for future expansions or domain transfers. By openly releasing both the dataset and the extraction pipeline, we seek to foster research on scientific reasoning, enable comprehensive performance evaluations of new LLMs, and lower the barrier to replicating the successes of advanced models like o1 in the broader science community. We believe SCP-116K will serve as a critical resource, catalyzing progress in high-level scientific reasoning tasks and promoting further innovations in LLM development. The dataset and code are publicly available at this https URL.

Title: SedarEval: Automated Evaluation using Self-Adaptive Rubrics

• Authors: Zhiyuan Fan, Weinong Wang, Xing Wu, Debing Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15595
• Pdf URL: https://arxiv.org/pdf/2501.15595
• Copy Paste: [[2501.15595]] SedarEval: Automated Evaluation using Self-Adaptive Rubrics(https://arxiv.org/abs/2501.15595)
• Keywords: large language model
• Abstract: The evaluation paradigm of LLM-as-judge gains popularity due to its significant reduction in human labor and time costs. This approach utilizes one or more large language models (LLMs) to assess the quality of outputs from other LLMs. However, existing methods rely on generic scoring rubrics that fail to consider the specificities of each question and its problem-solving process, compromising precision and stability in assessments. Inspired by human examination scoring processes, we propose a new evaluation paradigm based on self-adaptive rubrics. Specifically, we create detailed scoring rubrics for each question, capturing the primary and secondary criteria in a structured format of scoring and deduction points that mimic a human evaluator's analytical process. Building on this paradigm, we further develop a novel benchmark called SedarEval, which covers a range of domains including long-tail knowledge, mathematics, coding, and logical reasoning. SedarEval consists of 1,000 meticulously crafted questions, each with its own self-adaptive rubric. To further streamline the evaluation, we train a specialized evaluator language model (evaluator LM) to supplant human graders. Using the same training data, our evaluator LM achieves a higher concordance rate with human grading results than other paradigms, including GPT-4, highlighting the superiority and efficiency of our approach. We release our dataset at this https URL.

Title: IPVTON: Image-based 3D Virtual Try-on with Image Prompt Adapter

• Authors: Xiaojing Zhong, Zhonghua Wu, Xiaofeng Yang, Guosheng Lin, Qingyao Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15616
• Pdf URL: https://arxiv.org/pdf/2501.15616
• Copy Paste: [[2501.15616]] IPVTON: Image-based 3D Virtual Try-on with Image Prompt Adapter(https://arxiv.org/abs/2501.15616)
• Keywords: diffusion
• Abstract: Given a pair of images depicting a person and a garment separately, image-based 3D virtual try-on methods aim to reconstruct a 3D human model that realistically portrays the person wearing the desired garment. In this paper, we present IPVTON, a novel image-based 3D virtual try-on framework. IPVTON employs score distillation sampling with image prompts to optimize a hybrid 3D human representation, integrating target garment features into diffusion priors through an image prompt adapter. To avoid interference with non-target areas, we leverage mask-guided image prompt embeddings to focus the image features on the try-on regions. Moreover, we impose geometric constraints on the 3D model with a pseudo silhouette generated by ControlNet, ensuring that the clothed 3D human model retains the shape of the source identity while accurately wearing the target garments. Extensive qualitative and quantitative experiments demonstrate that IPVTON outperforms previous methods in image-based 3D virtual try-on tasks, excelling in both geometry and texture.

Title: Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets

• Authors: Eduard Barbu, Meeri-Ly Muru, Sten Marcus Malva
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15624
• Pdf URL: https://arxiv.org/pdf/2501.15624
• Copy Paste: [[2501.15624]] Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets(https://arxiv.org/abs/2501.15624)
• Keywords: large language model
• Abstract: This study introduces an approach to Estonian text simplification using two model architectures: a neural machine translation model and a fine-tuned large language model (LLaMA). Given the limited resources for Estonian, we developed a new dataset, the Estonian Simplification Dataset, combining translated data and GPT-4.0-generated simplifications. We benchmarked OpenNMT, a neural machine translation model that frames text simplification as a translation task, and fine-tuned the LLaMA model on our dataset to tailor it specifically for Estonian simplification. Manual evaluations on the test set show that the LLaMA model consistently outperforms OpenNMT in readability, grammaticality, and meaning preservation. These findings underscore the potential of large language models for low-resource languages and provide a basis for further research in Estonian text simplification.

Title: HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI

• Authors: Tidor-Vlad Pricope
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15627
• Pdf URL: https://arxiv.org/pdf/2501.15627
• Copy Paste: [[2501.15627]] HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI(https://arxiv.org/abs/2501.15627)
• Keywords: fair
• Abstract: We present HardML, a benchmark designed to evaluate the knowledge and reasoning abilities in the fields of data science and machine learning. HardML comprises a diverse set of 100 challenging multiple-choice questions, handcrafted over a period of 6 months, covering the most popular and modern branches of data science and machine learning. These questions are challenging even for a typical Senior Machine Learning Engineer to answer correctly. To minimize the risk of data contamination, HardML uses mostly original content devised by the author. Current state of the art AI models achieve a 30% error rate on this benchmark, which is about 3 times larger than the one achieved on the equivalent, well known MMLU ML. While HardML is limited in scope and not aiming to push the frontier, primarily due to its multiple choice nature, it serves as a rigorous and modern testbed to quantify and track the progress of top AI. While plenty benchmarks and experimentation in LLM evaluation exist in other STEM fields like mathematics, physics and chemistry, the subfields of data science and machine learning remain fairly underexplored.

Title: Quantum-Enhanced Attention Mechanism in NLP: A Hybrid Classical-Quantum Approach

• Authors: S.M. Yousuf Iqbal Tomal, Abdullah Al Shafin, Debojit Bhattacharjee, MD. Khairul Amin, Rafiad Sadat Shahir
• Subjects: cs.CL, quant-ph
• Abstract URL: https://arxiv.org/abs/2501.15630
• Pdf URL: https://arxiv.org/pdf/2501.15630
• Copy Paste: [[2501.15630]] Quantum-Enhanced Attention Mechanism in NLP: A Hybrid Classical-Quantum Approach(https://arxiv.org/abs/2501.15630)
• Keywords: robust, transformer
• Abstract: Transformer-based models have achieved remarkable results in natural language processing (NLP) tasks such as text classification and machine translation. However, their computational complexity and resource demands pose challenges for scalability and accessibility. This research proposes a hybrid quantum-classical transformer model that integrates a quantum-enhanced attention mechanism to address these limitations. By leveraging quantum kernel similarity and variational quantum circuits (VQC), the model captures intricate token dependencies while improving computational efficiency. Experimental results on the IMDb dataset demonstrate that the quantum-enhanced model outperforms the classical baseline across all key metrics, achieving a 1.5% improvement in accuracy (65.5% vs. 64%), precision, recall, and F1 score. Statistical significance tests validate these improvements, highlighting the robustness of the quantum approach. These findings illustrate the transformative potential of quantum-enhanced attention mechanisms in optimizing NLP architectures for real-world applications.

Title: A Comprehensive Survey on Self-Interpretable Neural Networks

• Authors: Yang Ji, Ying Sun, Yuting Zhang, Zhigaoyuan Wang, Yuanxin Zhuang, Zheng Gong, Dazhong Shen, Chuan Qin, Hengshu Zhu, Hui Xiong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15638
• Pdf URL: https://arxiv.org/pdf/2501.15638
• Copy Paste: [[2501.15638]] A Comprehensive Survey on Self-Interpretable Neural Networks(https://arxiv.org/abs/2501.15638)
• Keywords: robust, interpretability
• Abstract: Neural networks have achieved remarkable success across various fields. However, the lack of interpretability limits their practical use, particularly in critical decision-making scenarios. Post-hoc interpretability, which provides explanations for pre-trained models, is often at risk of robustness and fidelity. This has inspired a rising interest in self-interpretable neural networks, which inherently reveal the prediction rationale through the model structures. Although there exist surveys on post-hoc interpretability, a comprehensive and systematic survey of self-interpretable neural networks is still missing. To address this gap, we first collect and review existing works on self-interpretable neural networks and provide a structured summary of their methodologies from five key perspectives: attribution-based, function-based, concept-based, prototype-based, and rule-based self-interpretation. We also present concrete, visualized examples of model explanations and discuss their applicability across diverse scenarios, including image, text, graph data, and deep reinforcement learning. Additionally, we summarize existing evaluation metrics for self-interpretability and identify open challenges in this field, offering insights for future research. To support ongoing developments, we present a publicly accessible resource to track advancements in this domain: this https URL.

Title: Bringing Characters to New Stories: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting

• Authors: Yuxin Zhang, Minyan Luo, Weiming Dong, Xiao Yang, Haibin Huang, Chongyang Ma, Oliver Deussen, Tong-Yee Lee, Changsheng Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15641
• Pdf URL: https://arxiv.org/pdf/2501.15641
• Copy Paste: [[2501.15641]] Bringing Characters to New Stories: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting(https://arxiv.org/abs/2501.15641)
• Keywords: robust, generative, large language model
• Abstract: The stories and characters that captivate us as we grow up shape unique fantasy worlds, with images serving as the primary medium for visually experiencing these realms. Personalizing generative models through fine-tuning with theme-specific data has become a prevalent approach in text-to-image generation. However, unlike object customization, which focuses on learning specific objects, theme-specific generation encompasses diverse elements such as characters, scenes, and objects. Such diversity also introduces a key challenge: how to adaptively generate multi-character, multi-concept, and continuous theme-specific images (TSI). Moreover, fine-tuning approaches often come with significant computational overhead, time costs, and risks of overfitting. This paper explores a fundamental question: Can image generation models directly leverage images as contextual input, similarly to how large language models use text as context? To address this, we present T-Prompter, a novel training-free TSI method for generation. T-Prompter introduces visual prompting, a mechanism that integrates reference images into generative models, allowing users to seamlessly specify the target theme without requiring additional training. To further enhance this process, we propose a Dynamic Visual Prompting (DVP) mechanism, which iteratively optimizes visual prompts to improve the accuracy and quality of generated images. Our approach enables diverse applications, including consistent story generation, character design, realistic character generation, and style-guided image generation. Comparative evaluations against state-of-the-art personalization methods demonstrate that T-Prompter achieves significantly better results and excels in maintaining character identity preserving, style consistency and text alignment, offering a robust and flexible solution for theme-specific image generation.

Title: Can Pose Transfer Models Generate Realistic Human Motion?

• Authors: Vaclav Knapp, Matyas Bohacek
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15648
• Pdf URL: https://arxiv.org/pdf/2501.15648
• Copy Paste: [[2501.15648]] Can Pose Transfer Models Generate Realistic Human Motion?(https://arxiv.org/abs/2501.15648)
• Keywords: diffusion
• Abstract: Recent pose-transfer methods aim to generate temporally consistent and fully controllable videos of human action where the motion from a reference video is reenacted by a new identity. We evaluate three state-of-the-art pose-transfer methods -- AnimateAnyone, MagicAnimate, and ExAvatar -- by generating videos with actions and identities outside the training distribution and conducting a participant study about the quality of these videos. In a controlled environment of 20 distinct human actions, we find that participants, presented with the pose-transferred videos, correctly identify the desired action only 42.92% of the time. Moreover, the participants find the actions in the generated videos consistent with the reference (source) videos only 36.46% of the time. These results vary by method: participants find the splatting-based ExAvatar more consistent and photorealistic than the diffusion-based AnimateAnyone and MagicAnimate.

Title: A Privacy Enhancing Technique to Evade Detection by Street Video Cameras Without Using Adversarial Accessories

• Authors: Jacob Shams, Ben Nassi, Satoru Koda, Asaf Shabtai, Yuval Elovici
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15653
• Pdf URL: https://arxiv.org/pdf/2501.15653
• Copy Paste: [[2501.15653]] A Privacy Enhancing Technique to Evade Detection by Street Video Cameras Without Using Adversarial Accessories(https://arxiv.org/abs/2501.15653)
• Keywords: privacy
• Abstract: In this paper, we propose a privacy-enhancing technique leveraging an inherent property of automatic pedestrian detection algorithms, namely, that the training of deep neural network (DNN) based methods is generally performed using curated datasets and laboratory settings, while the operational areas of these methods are dynamic real-world environments. In particular, we leverage a novel side effect of this gap between the laboratory and the real world: location-based weakness in pedestrian detection. We demonstrate that the position (distance, angle, height) of a person, and ambient light level, directly impact the confidence of a pedestrian detector when detecting the person. We then demonstrate that this phenomenon is present in pedestrian detectors observing a stationary scene of pedestrian traffic, with blind spot areas of weak detection of pedestrians with low confidence. We show how privacy-concerned pedestrians can leverage these blind spots to evade detection by constructing a minimum confidence path between two points in a scene, reducing the maximum confidence and average confidence of the path by up to 0.09 and 0.13, respectively, over direct and random paths through the scene. To counter this phenomenon, and force the use of more costly and sophisticated methods to leverage this vulnerability, we propose a novel countermeasure to improve the confidence of pedestrian detectors in blind spots, raising the max/average confidence of paths generated by our technique by 0.09 and 0.05, respectively. In addition, we demonstrate that our countermeasure improves a Faster R-CNN-based pedestrian detector's TPR and average true positive confidence by 0.03 and 0.15, respectively.

Title: People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

• Authors: Jenna Russell, Marzena Karpinska, Mohit Iyyer
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15654
• Pdf URL: https://arxiv.org/pdf/2501.15654
• Copy Paste: [[2501.15654]] People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text(https://arxiv.org/abs/2501.15654)
• Keywords: robust
• Abstract: In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such "expert" annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts' free-form explanations shows that while they rely heavily on specific lexical clues ('AI vocabulary'), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity) that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text.

Title: A Machine Learning Approach to Automatic Fall Detection of Combat Soldiers

• Authors: Leandro Soares, Rodrigo Parracho, Gustavo Venturini, José Gomes, Jonathan Efigenio, Pablo Rangel, Pedro Gonzalez, Joel dos Santos, Diego Brandão, Eduardo Bezerra
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2501.15655
• Pdf URL: https://arxiv.org/pdf/2501.15655
• Copy Paste: [[2501.15655]] A Machine Learning Approach to Automatic Fall Detection of Combat Soldiers(https://arxiv.org/abs/2501.15655)
• Keywords: security
• Abstract: Military personnel and security agents often face significant physical risks during conflict and engagement situations, particularly in urban operations. Ensuring the rapid and accurate communication of incidents involving injuries is crucial for the timely execution of rescue operations. This article presents research conducted under the scope of the Brazilian Navy's ``Soldier of the Future'' project, focusing on the development of a Casualty Detection System to identify injuries that could incapacitate a soldier and lead to severe blood loss. The study specifically addresses the detection of soldier falls, which may indicate critical injuries such as hypovolemic hemorrhagic shock. To generate the publicly available dataset, we used smartwatches and smartphones as wearable devices to collect inertial data from soldiers during various activities, including simulated falls. The data were used to train 1D Convolutional Neural Networks (CNN1D) with the objective of accurately classifying falls that could result from life-threatening injuries. We explored different sensor placements (on the wrists and near the center of mass) and various approaches to using inertial variables, including linear and angular accelerations. The neural network models were optimized using Bayesian techniques to enhance their performance. The best-performing model and its results, discussed in this article, contribute to the advancement of automated systems for monitoring soldier safety and improving response times in engagement scenarios.

Title: Classifying Deepfakes Using Swin Transformers

• Authors: Aprille J. Xi, Eason Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15656
• Pdf URL: https://arxiv.org/pdf/2501.15656
• Copy Paste: [[2501.15656]] Classifying Deepfakes Using Swin Transformers(https://arxiv.org/abs/2501.15656)
• Keywords: robust, transformer
• Abstract: The proliferation of deepfake technology poses significant challenges to the authenticity and trustworthiness of digital media, necessitating the development of robust detection methods. This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images. Using the Real and Fake Face Detection dataset by Yonsei University's Computational Intelligence Photography Lab, we evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts. Our results demonstrate that the Swin Transformer outperforms conventional CNN-based architectures, including VGG16, ResNet18, and AlexNet, achieving a test accuracy of 71.29\%. Additionally, we present insights into hybrid model design, highlighting the complementary strengths of transformer and CNN-based approaches in deepfake detection. This study underscores the potential of transformer-based architectures for improving accuracy and generalizability in image-based manipulation detection, paving the way for more effective countermeasures against deepfake threats.

Title: StagFormer: Time Staggering Transformer Decoding for RunningLayers In Parallel

• Authors: Dylan Cutler, Arun Kandoor, Nishanth Dikkala, Nikunj Saunshi, Xin Wang, Rina Panigrahy
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15665
• Pdf URL: https://arxiv.org/pdf/2501.15665
• Copy Paste: [[2501.15665]] StagFormer: Time Staggering Transformer Decoding for RunningLayers In Parallel(https://arxiv.org/abs/2501.15665)
• Keywords: transformer
• Abstract: Standard decoding in a Transformer based language model is inherently sequential as we wait for a token's embedding to pass through all the layers in the network before starting the generation of the next token. In this work, we propose a new architecture StagFormer (Staggered Transformer), which staggered execution along the time axis and thereby enables parallelizing the decoding process along the depth of the model. We achieve this by breaking the dependency of the token representation at time step $i$ in layer $l$ upon the representations of tokens until time step $i$ from layer $l-1$. Instead, we stagger the execution and only allow a dependency on token representations until time step $i-1$. The later sections of the Transformer still get access to the ``rich" representations from the prior section but only from those token positions which are one time step behind. StagFormer allows for different sections of the model to be executed in parallel yielding at potential 33\% speedup in decoding while being quality neutral in our simulations. We also explore many natural variants of this idea. We present how weight-sharing across the different sections being staggered can be more practical in settings with limited memory. We show how one can approximate a recurrent model during inference using such weight-sharing. We explore the efficacy of using a bounded window attention to pass information from one section to another which helps drive further latency gains for some applications. We also explore demonstrate the scalability of the staggering idea over more than 2 sections of the Transformer.

Title: MimicGait: A Model Agnostic approach for Occluded Gait Recognition using Correlational Knowledge Distillation

• Authors: Ayush Gupta, Rama Chellappa
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15666
• Pdf URL: https://arxiv.org/pdf/2501.15666
• Copy Paste: [[2501.15666]] MimicGait: A Model Agnostic approach for Occluded Gait Recognition using Correlational Knowledge Distillation(https://arxiv.org/abs/2501.15666)
• Keywords: biometric
• Abstract: Gait recognition is an important biometric technique over large distances. State-of-the-art gait recognition systems perform very well in controlled environments at close range. Recently, there has been an increased interest in gait recognition in the wild prompted by the collection of outdoor, more challenging datasets containing variations in terms of illumination, pitch angles, and distances. An important problem in these environments is that of occlusion, where the subject is partially blocked from camera view. While important, this problem has received little attention. Thus, we propose MimicGait, a model-agnostic approach for gait recognition in the presence of occlusions. We train the network using a multi-instance correlational distillation loss to capture both inter-sequence and intra-sequence correlations in the occluded gait patterns of a subject, utilizing an auxiliary Visibility Estimation Network to guide the training of the proposed mimic network. We demonstrate the effectiveness of our approach on challenging real-world datasets like GREW, Gait3D and BRIAR. We release the code in this https URL.

Title: TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs

• Authors: Yuxuan Gu, Wuyang Zhou, Giorgos Iacovides, Danilo Mandic
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15674
• Pdf URL: https://arxiv.org/pdf/2501.15674
• Copy Paste: [[2501.15674]] TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs(https://arxiv.org/abs/2501.15674)
• Keywords: transformer, large language model
• Abstract: The reasoning abilities of Large Language Models (LLMs) can be improved by structurally denoising their weights, yet existing techniques primarily focus on denoising the feed-forward network (FFN) of the transformer block, and can not efficiently utilise the Multi-head Attention (MHA) block, which is the core of transformer architectures. To address this issue, we propose a novel intuitive framework that, at its very core, performs MHA compression through a multi-head tensorisation process and the Tucker decomposition. This enables both higher-dimensional structured denoising and compression of the MHA weights, by enforcing a shared higher-dimensional subspace across the weights of the multiple attention heads. We demonstrate that this approach consistently enhances the reasoning capabilities of LLMs across multiple benchmark datasets, and for both encoder-only and decoder-only architectures, while achieving compression rates of up to $\sim 250$ times in the MHA weights, all without requiring any additional data, training, or fine-tuning. Furthermore, we show that the proposed method can be seamlessly combined with existing FFN-only-based denoising techniques to achieve further improvements in LLM reasoning performance.

Title: Exploring the Feasibility of Deep Learning Models for Long-term Disease Prediction: A Case Study for Wheat Yellow Rust in England

• Authors: Zhipeng Yuan, Yu Zhang, Gaoshan Bi, Po Yang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15677
• Pdf URL: https://arxiv.org/pdf/2501.15677
• Copy Paste: [[2501.15677]] Exploring the Feasibility of Deep Learning Models for Long-term Disease Prediction: A Case Study for Wheat Yellow Rust in England(https://arxiv.org/abs/2501.15677)
• Keywords: robust
• Abstract: Wheat yellow rust, caused by the fungus Puccinia striiformis, is a critical disease affecting wheat crops across Britain, leading to significant yield losses and economic consequences. Given the rapid environmental changes and the evolving virulence of pathogens, there is a growing need for innovative approaches to predict and manage such diseases over the long term. This study explores the feasibility of using deep learning models to predict outbreaks of wheat yellow rust in British fields, offering a proactive approach to disease management. We construct a yellow rust dataset with historial weather information and disease indicator acrossing multiple regions in England. We employ two poweful deep learning models, including fully connected neural networks and long short-term memory to develop predictive models capable of recognizing patterns and predicting future disease this http URL models are trained and validated in a randomly sliced datasets. The performance of these models with different predictive time steps are evaluated based on their accuracy, precision, recall, and F1-score. Preliminary results indicate that deep learning models can effectively capture the complex interactions between multiple factors influencing disease dynamics, demonstrating a promising capacity to forecast wheat yellow rust with considerable accuracy. Specifically, the fully-connected neural network achieved 83.65% accuracy in a disease prediction task with 6 month predictive time step setup. These findings highlight the potential of deep learning to transform disease management strategies, enabling earlier and more precise interventions. Our study provides a methodological framework for employing deep learning in agricultural settings but also opens avenues for future research to enhance the robustness and applicability of predictive models in combating crop diseases globally.

Title: Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts

• Authors: Haodi Ma, Dzmitry Kasinets, Daisy Zhe Wang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15688
• Pdf URL: https://arxiv.org/pdf/2501.15688
• Copy Paste: [[2501.15688]] Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts(https://arxiv.org/abs/2501.15688)
• Keywords: transformer
• Abstract: Multimodal knowledge graph completion (MMKGC) aims to predict missing links in multimodal knowledge graphs (MMKGs) by leveraging information from various modalities alongside structural data. Existing MMKGC approaches primarily extend traditional knowledge graph embedding (KGE) models, which often require creating an embedding for every entity. This results in large model sizes and inefficiencies in integrating multimodal information, particularly for real-world graphs. Meanwhile, Transformer-based models have demonstrated competitive performance in knowledge graph completion (KGC). However, their focus on single-modal knowledge limits their capacity to utilize cross-modal information. Recently, Large vision-language models (VLMs) have shown potential in cross-modal tasks but are constrained by the high cost of training. In this work, we propose a novel approach that integrates Transformer-based KGE models with cross-modal context generated by pre-trained VLMs, thereby extending their applicability to MMKGC. Specifically, we employ a pre-trained VLM to transform relevant visual information from entities and their neighbors into textual sequences. We then frame KGC as a sequence-to-sequence task, fine-tuning the model with the generated cross-modal context. This simple yet effective method significantly reduces model size compared to traditional KGE approaches while achieving competitive performance across multiple large-scale datasets with minimal hyperparameter tuning.

Title: Random Walk Guided Hyperbolic Graph Distillation

• Authors: Yunbo Long, Liming Xu, Stefan Schoepf, Alexandra Brintrup
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15696
• Pdf URL: https://arxiv.org/pdf/2501.15696
• Copy Paste: [[2501.15696]] Random Walk Guided Hyperbolic Graph Distillation(https://arxiv.org/abs/2501.15696)
• Keywords: privacy, robust
• Abstract: Graph distillation (GD) is an effective approach to extract useful information from large-scale network structures. However, existing methods, which operate in Euclidean space to generate condensed graphs, struggle to capture the inherent tree-like geometry of real-world networks, resulting in distilled graphs with limited task-specific information for downstream tasks. Furthermore, these methods often fail to extract dynamic properties from graphs, which are crucial for understanding information flow and facilitating graph continual learning. This paper presents the Hyperbolic Graph Distillation with Random Walks Optimization (HyDRO), a novel graph distillation approach that leverages hyperbolic embeddings to capture complex geometric patterns and optimize the spectral gap in hyperbolic space. Experiments show that HyDRO demonstrates strong task generalization, consistently outperforming state-of-the-art methods in both node classification and link prediction tasks. HyDRO also effectively preserves graph random walk properties, producing condensed graphs that achieve enhanced performance in continual graph learning. Additionally, HyDRO achieves competitive results on mainstream graph distillation benchmarks, while maintaining a strong balance between privacy and utility, and exhibiting robust resistance to noises.

Title: Adapting Biomedical Abstracts into Plain language using Large Language Models

• Authors: Haritha Gangavarapu, Giridhar Kaushik Ramachandran, Kevin Lybarger, Meliha Yetisgen, Özlem Uzuner
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15700
• Pdf URL: https://arxiv.org/pdf/2501.15700
• Copy Paste: [[2501.15700]] Adapting Biomedical Abstracts into Plain language using Large Language Models(https://arxiv.org/abs/2501.15700)
• Keywords: robust, large language model
• Abstract: A vast amount of medical knowledge is available for public use through online health forums, and question-answering platforms on social media. The majority of the population in the United States doesn't have the right amount of health literacy to make the best use of that information. Health literacy means the ability to obtain and comprehend the basic health information to make appropriate health decisions. To build the bridge between this gap, organizations advocate adapting this medical knowledge into plain language. Building robust systems to automate the adaptations helps both medical and non-medical professionals best leverage the available information online. The goal of the Plain Language Adaptation of Biomedical Abstracts (PLABA) track is to adapt the biomedical abstracts in English language extracted from PubMed based on the questions asked in MedlinePlus for the general public using plain language at the sentence level. As part of this track, we leveraged the best open-source Large Language Models suitable and fine-tuned for dialog use cases. We compare and present the results for all of our systems and our ranking among the other participants' submissions. Our top performing GPT-4 based model ranked first in the avg. simplicity measure and 3rd on the avg. accuracy measure.

Title: Disentanglement Analysis in Deep Latent Variable Models Matching Aggregate Posterior Distributions

• Authors: Surojit Saha, Sarang Joshi, Ross Whitaker
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.15705
• Pdf URL: https://arxiv.org/pdf/2501.15705
• Copy Paste: [[2501.15705]] Disentanglement Analysis in Deep Latent Variable Models Matching Aggregate Posterior Distributions(https://arxiv.org/abs/2501.15705)
• Keywords: generative
• Abstract: Deep latent variable models (DLVMs) are designed to learn meaningful representations in an unsupervised manner, such that the hidden explanatory factors are interpretable by independent latent variables (aka disentanglement). The variational autoencoder (VAE) is a popular DLVM widely studied in disentanglement analysis due to the modeling of the posterior distribution using a factorized Gaussian distribution that encourages the alignment of the latent factors with the latent axes. Several metrics have been proposed recently, assuming that the latent variables explaining the variation in data are aligned with the latent axes (cardinal directions). However, there are other DLVMs, such as the AAE and WAE-MMD (matching the aggregate posterior to the prior), where the latent variables might not be aligned with the latent axes. In this work, we propose a statistical method to evaluate disentanglement for any DLVMs in general. The proposed technique discovers the latent vectors representing the generative factors of a dataset that can be different from the cardinal latent axes. We empirically demonstrate the advantage of the method on two datasets.

Title: StaICC: Standardized Evaluation for Classification Task in In-context Learning

• Authors: Hakaze Cho, Naoya Inoue
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15708
• Pdf URL: https://arxiv.org/pdf/2501.15708
• Copy Paste: [[2501.15708]] StaICC: Standardized Evaluation for Classification Task in In-context Learning(https://arxiv.org/abs/2501.15708)
• Keywords: robust, fair
• Abstract: Classification tasks are widely investigated in the In-Context Learning (ICL) paradigm. However, current efforts are evaluated on disjoint benchmarks and settings, while their performances are significantly influenced by some trivial variables, such as prompt templates, data sampling, instructions, etc., which leads to significant inconsistencies in the results reported across various literature, preventing fair comparison or meta-analysis across different papers. Therefore, this paper proposes a standardized and easy-to-use evaluation toolkit (StaICC) for in-context classification. Including, for the normal classification task, we provide StaICC-Normal, selecting 10 widely used datasets, and generating prompts with a fixed form, to mitigate the variance among the experiment implementations. To enrich the usage of our benchmark, we also provide a sub-benchmark StaICC-Diag for diagnosing ICL from several aspects, aiming for a more robust inference processing.

Title: CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling

• Authors: Kaiyuan Zhang, Siyuan Cheng, Guangyu Shen, Bruno Ribeiro, Shengwei An, Pin-Yu Chen, Xiangyu Zhang, Ninghui Li
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15718
• Pdf URL: https://arxiv.org/pdf/2501.15718
• Copy Paste: [[2501.15718]] CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling(https://arxiv.org/abs/2501.15718)
• Keywords: defense, attack, federate
• Abstract: Federated learning collaboratively trains a neural network on a global server, where each local client receives the current global model weights and sends back parameter updates (gradients) based on its local private data. The process of sending these model updates may leak client's private data information. Existing gradient inversion attacks can exploit this vulnerability to recover private training instances from a client's gradient vectors. Recently, researchers have proposed advanced gradient inversion techniques that existing defenses struggle to handle effectively. In this work, we present a novel defense tailored for large neural network models. Our defense capitalizes on the high dimensionality of the model parameters to perturb gradients within a subspace orthogonal to the original gradient. By leveraging cold posteriors over orthogonal subspaces, our defense implements a refined gradient update mechanism. This enables the selection of an optimal gradient that not only safeguards against gradient inversion attacks but also maintains model utility. We conduct comprehensive experiments across three different datasets and evaluate our defense against various state-of-the-art attacks and defenses. Code is available at this https URL.

Title: A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks

• Authors: Dong Li, Guihong Wan, Xintao Wu, Xinyu Wu, Ajit J. Nirmal, Christine G. Lian, Peter K. Sorger, Yevgeniy R. Semenov, Chen Zhao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15724
• Pdf URL: https://arxiv.org/pdf/2501.15724
• Copy Paste: [[2501.15724]] A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks(https://arxiv.org/abs/2501.15724)
• Keywords: robust, segmentation
• Abstract: Computational pathology foundation models (CPathFMs) have emerged as a powerful approach for analyzing histopathological data, leveraging self-supervised learning to extract robust feature representations from unlabeled whole-slide images. These models, categorized into uni-modal and multi-modal frameworks, have demonstrated promise in automating complex pathology tasks such as segmentation, classification, and biomarker discovery. However, the development of CPathFMs presents significant challenges, such as limited data accessibility, high variability across datasets, the necessity for domain-specific adaptation, and the lack of standardized evaluation benchmarks. This survey provides a comprehensive review of CPathFMs in computational pathology, focusing on datasets, adaptation strategies, and evaluation tasks. We analyze key techniques, such as contrastive learning and multi-modal integration, and highlight existing gaps in current research. Finally, we explore future directions from four perspectives for advancing CPathFMs. This survey serves as a valuable resource for researchers, clinicians, and AI practitioners, guiding the advancement of CPathFMs toward robust and clinically applicable AI-driven pathology solutions.

Title: Integrating Personalized Federated Learning with Control Systems for Enhanced Performance

• Authors: Alice Smith, Bob Johnson, Michael Geller
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.15728
• Pdf URL: https://arxiv.org/pdf/2501.15728
• Copy Paste: [[2501.15728]] Integrating Personalized Federated Learning with Control Systems for Enhanced Performance(https://arxiv.org/abs/2501.15728)
• Keywords: privacy, robust, federate
• Abstract: In the expanding field of machine learning, federated learning has emerged as a pivotal methodology for distributed data environments, ensuring privacy while leveraging decentralized data sources. However, the heterogeneity of client data and the need for tailored models necessitate the integration of personalization techniques to enhance learning efficacy and model performance. This paper introduces a novel framework that amalgamates personalized federated learning with robust control systems, aimed at optimizing both the learning process and the control of data flow across diverse networked environments. Our approach harnesses personalized algorithms that adapt to the unique characteristics of each client's data, thereby improving the relevance and accuracy of the model for individual nodes without compromising the overall system performance. To manage and control the learning process across the network, we employ a sophisticated control system that dynamically adjusts the parameters based on real-time feedback and system states, ensuring stability and efficiency. Through rigorous experimentation, we demonstrate that our integrated system not only outperforms standard federated learning models in terms of accuracy and learning speed but also maintains system integrity and robustness in face of varying network conditions and data distributions. The experimental results, obtained from a multi-client simulated environment with non-IID data distributions, underscore the benefits of integrating control systems into personalized federated learning frameworks, particularly in scenarios demanding high reliability and precision.

Title: Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis

• Authors: Haibo Wang, Jun Huang, Lutfu Sua, Bahram Alidaee
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15731
• Pdf URL: https://arxiv.org/pdf/2501.15731
• Copy Paste: [[2501.15731]] Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis(https://arxiv.org/abs/2501.15731)
• Keywords: robust
• Abstract: The increasing focus on predicting renewable energy production aligns with advancements in deep learning (DL). The inherent variability of renewable sources and the complexity of prediction methods require robust approaches, such as DL models, in the renewable energy sector. DL models are preferred over traditional machine learning (ML) because they capture complex, nonlinear relationships in renewable energy datasets. This study examines key factors influencing DL technique accuracy, including sampling and hyperparameter optimization, by comparing various methods and training and test ratios within a DL framework. Seven machine learning methods, LSTM, Stacked LSTM, CNN, CNN-LSTM, DNN, Time-Distributed MLP (TD-MLP), and Autoencoder (AE), are evaluated using a dataset combining weather and photovoltaic power output data from 12 locations. Regularization techniques such as early stopping, neuron dropout, L1 and L2 regularization are applied to address overfitting. The results demonstrate that the combination of early stopping, dropout, and L1 regularization provides the best performance to reduce overfitting in the CNN and TD-MLP models with larger training set, while the combination of early stopping, dropout, and L2 regularization is the most effective to reduce the overfitting in CNN-LSTM and AE models with smaller training set.

Title: IndicMMLU-Pro: Benchmarking the Indic Large Language Models

• Authors: Sankalp KJ, Ashutosh Kumar, Laxmaan Balaji, Nikunj Kotecha, Vinija Jain, Aman Chadha, Sreyoshi Bhaduri
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15747
• Pdf URL: https://arxiv.org/pdf/2501.15747
• Copy Paste: [[2501.15747]] IndicMMLU-Pro: Benchmarking the Indic Large Language Models(https://arxiv.org/abs/2501.15747)
• Keywords: large language model
• Abstract: Known by more than 1.5 billion people in the Indian subcontinent, Indic languages present unique challenges and opportunities for natural language processing (NLP) research due to their rich cultural heritage, linguistic diversity, and complex structures. IndicMMLU-Pro is a comprehensive benchmark designed to evaluate Large Language Models (LLMs) across Indic languages, building upon the MMLU Pro (Massive Multitask Language Understanding) framework. Covering major languages such as Hindi, Bengali, Gujarati, Marathi, Kannada, Punjabi, Tamil, Telugu, and Urdu, our benchmark addresses the unique challenges and opportunities presented by the linguistic diversity of the Indian subcontinent. This benchmark encompasses a wide range of tasks in language comprehension, reasoning, and generation, meticulously crafted to capture the intricacies of Indian languages. IndicMMLU-Pro provides a standardized evaluation framework to push the research boundaries in Indic language AI, facilitating the development of more accurate, efficient, and culturally sensitive models. This paper outlines the benchmarks' design principles, task taxonomy, data collection methodology, and presents baseline results from state-of-the-art multilingual models.

Title: A Privacy Model for Classical & Learned Bloom Filters

• Authors: Hayder Tirmazi
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15751
• Pdf URL: https://arxiv.org/pdf/2501.15751
• Copy Paste: [[2501.15751]] A Privacy Model for Classical & Learned Bloom Filters(https://arxiv.org/abs/2501.15751)
• Keywords: privacy, defense, attack
• Abstract: The Classical Bloom Filter (CBF) is a class of Probabilistic Data Structures (PDS) for handling Approximate Query Membership (AMQ). The Learned Bloom Filter (LBF) is a recently proposed class of PDS that combines the Classical Bloom Filter with a Learning Model while preserving the Bloom Filter's one-sided error guarantees. Bloom Filters have been used in settings where inputs are sensitive and need to be private in the presence of an adversary with access to the Bloom Filter through an API or in the presence of an adversary who has access to the internal state of the Bloom Filter. Prior work has investigated the privacy of the Classical Bloom Filter providing attacks and defenses under various privacy definitions. In this work, we formulate a stronger differential privacy-based model for the Bloom Filter. We propose constructions of the Classical and Learned Bloom Filter that satisfy $(\epsilon, 0)$-differential privacy. This is also the first work that analyses and addresses the privacy of the Learned Bloom Filter under any rigorous model, which is an open problem.

Title: Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference

• Authors: Go Kamoda, Benjamin Hienzerling, Tatsuro Inaba, Keito Kudo, Keisuke Sakaguchi, Kentaro Inui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15754
• Pdf URL: https://arxiv.org/pdf/2501.15754
• Copy Paste: [[2501.15754]] Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference(https://arxiv.org/abs/2501.15754)
• Keywords: segmentation
• Abstract: According to the stages-of-inference hypothesis, early layers of language models map their subword-tokenized input, which does not necessarily correspond to a linguistically meaningful segmentation, to more meaningful representations that form the model's ``inner vocabulary''. Prior analysis of this detokenization stage has predominantly relied on probing and interventions such as path patching, which involve selecting particular inputs, choosing a subset of components that will be patched, and then observing changes in model behavior. Here, we show that several important aspects of the detokenization stage can be understood purely by analyzing model weights, without performing any model inference steps. Specifically, we introduce an analytical decomposition of first-layer attention in GPT-2. Our decomposition yields interpretable terms that quantify the relative contributions of position-related, token-related, and mixed effects. By focusing on terms in this decomposition, we discover weight-based explanations of attention bias toward close tokens and attention for detokenization.

Title: GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design

• Authors: Yuanfu Sun, Zhengnan Ma, Yi Fang, Jing Ma, Qiaoyu Tan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15755
• Pdf URL: https://arxiv.org/pdf/2501.15755
• Copy Paste: [[2501.15755]] GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design(https://arxiv.org/abs/2501.15755)
• Keywords: large language model
• Abstract: The growing importance of textual and relational systems has driven interest in enhancing large language models (LLMs) for graph-structured data, particularly Text-Attributed Graphs (TAGs), where samples are represented by textual descriptions interconnected by edges. While research has largely focused on developing specialized graph LLMs through task-specific instruction tuning, a comprehensive benchmark for evaluating LLMs solely through prompt design remains surprisingly absent. Without such a carefully crafted evaluation benchmark, most if not all, tailored graph LLMs are compared against general LLMs using simplistic queries (e.g., zero-shot reasoning with LLaMA), which can potentially camouflage many advantages as well as unexpected predicaments of them. To achieve more general evaluations and unveil the true potential of LLMs for graph tasks, we introduce Graph In-context Learning (GraphICL) Benchmark, a comprehensive benchmark comprising novel prompt templates designed to capture graph structure and handle limited label knowledge. Our systematic evaluation shows that general-purpose LLMs equipped with our GraphICL outperform state-of-the-art specialized graph LLMs and graph neural network models in resource-constrained settings and out-of-domain tasks. These findings highlight the significant potential of prompt engineering to enhance LLM performance on graph learning tasks without training and offer a strong baseline for advancing research in graph LLMs.

Title: Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification

• Authors: Ashim Dahal, Saydul Akbar Murad, Nick Rahimi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15757
• Pdf URL: https://arxiv.org/pdf/2501.15757
• Copy Paste: [[2501.15757]] Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification(https://arxiv.org/abs/2501.15757)
• Keywords: fair, transformer
• Abstract: Algorithmic level developments like Convolutional Neural Networks, transformers, attention mechanism, Retrieval Augmented Generation and so on have changed Artificial Intelligence. Recent such development was observed by Kolmogorov-Arnold Networks that suggested to challenge the fundamental concept of a Neural Network, thus change Multilayer Perceptron, and Convolutional Neural Networks. They received a good reception in terms of scientific modeling, yet had some drawbacks in terms of efficiency. In this paper, we train Convolutional Kolmogorov Arnold Networks (CKANs) with the ImageNet-1k dataset with 1.3 million images, MNIST dataset with 60k images and a tabular biological science related MoA dataset and test the promise of CKANs in terms of FLOPS, Inference Time, number of trainable parameters and training time against the accuracy, precision, recall and f-1 score they produce against the standard industry practice on CNN models. We show that the CKANs perform fair yet slower than CNNs in small size dataset like MoA and MNIST but are not nearly comparable as the dataset gets larger and more complex like the ImageNet. The code implementation of this paper can be found on the link: \href{this https URL}{this https URL}

Title: Investigating Application of Deep Neural Networks in Intrusion Detection System Design

• Authors: Mofe O. Jeje
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15760
• Pdf URL: https://arxiv.org/pdf/2501.15760
• Copy Paste: [[2501.15760]] Investigating Application of Deep Neural Networks in Intrusion Detection System Design(https://arxiv.org/abs/2501.15760)
• Keywords: attack
• Abstract: Despite decades of development, existing IDSs still face challenges in improving detection accuracy, evasion, and detection of unknown attacks. To solve these problems, many researchers have focused on designing and developing IDSs that use Deep Neural Networks (DNN) which provides advanced methods of threat investigation and detection. Given this reason, the motivation of this research then, is to learn how effective applications of Deep Neural Networks (DNN) can accurately detect and identify malicious network intrusion, while advancing the frontiers of their optimal potential use in network intrusion detection. Using the ASNM-TUN dataset, the study used a Multilayer Perceptron modeling approach in Deep Neural Network to identify network intrusions, in addition to distinguishing them in terms of legitimate network traffic, direct network attacks, and obfuscated network attacks. To further enhance the speed and efficiency of this DNN solution, a thorough feature selection technique called Forward Feature Selection (FFS), which resulted in a significant reduction in the feature subset, was implemented. Using the Multilayer Perceptron model, test results demonstrate no support for the model to accurately and correctly distinguish the classification of network intrusion.

Title: Is It Navajo? Accurate Language Detection in Endangered Athabaskan Languages

• Authors: Ivory Yang, Weicheng Ma, Chunhui Zhang, Soroush Vosoughi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15773
• Pdf URL: https://arxiv.org/pdf/2501.15773
• Copy Paste: [[2501.15773]] Is It Navajo? Accurate Language Detection in Endangered Athabaskan Languages(https://arxiv.org/abs/2501.15773)
• Keywords: robust, large language model
• Abstract: Endangered languages, such as Navajo - the most widely spoken Native American language - are significantly underrepresented in contemporary language technologies, exacerbating the challenges of their preservation and revitalization. This study evaluates Google's large language model (LLM)-based language identification system, which consistently misidentifies Navajo, exposing inherent limitations when applied to low-resource Native American languages. To address this, we introduce a random forest classifier trained on Navajo and eight frequently confused languages. Despite its simplicity, the classifier achieves near-perfect accuracy (97-100%), significantly outperforming Google's LLM-based system. Additionally, the model demonstrates robustness across other Athabaskan languages - a family of Native American languages spoken primarily in Alaska, the Pacific Northwest, and parts of the Southwestern United States - suggesting its potential for broader application. Our findings underscore the pressing need for NLP systems that prioritize linguistic diversity and adaptability over centralized, one-size-fits-all solutions, especially in supporting underrepresented languages in a multicultural world. This work directly contributes to ongoing efforts to address cultural biases in language models and advocates for the development of culturally localized NLP tools that serve diverse linguistic communities.

Title: Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution

• Authors: Karam Park, Jae Woong Soh, Nam Ik Cho
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15774
• Pdf URL: https://arxiv.org/pdf/2501.15774
• Copy Paste: [[2501.15774]] Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution(https://arxiv.org/abs/2501.15774)
• Keywords: transformer
• Abstract: Transformer-based Super-Resolution (SR) methods have demonstrated superior performance compared to convolutional neural network (CNN)-based SR approaches due to their capability to capture long-range dependencies. However, their high computational complexity necessitates the development of lightweight approaches for practical use. To address this challenge, we propose the Attention-Sharing Information Distillation (ASID) network, a lightweight SR network that integrates attention-sharing and an information distillation structure specifically designed for Transformer-based SR methods. We modify the information distillation scheme, originally designed for efficient CNN operations, to reduce the computational load of stacked self-attention layers, effectively addressing the efficiency bottleneck. Additionally, we introduce attention-sharing across blocks to further minimize the computational cost of self-attention operations. By combining these strategies, ASID achieves competitive performance with existing SR methods while requiring only around 300K parameters - significantly fewer than existing CNN-based and Transformer-based SR models. Furthermore, ASID outperforms state-of-the-art SR methods when the number of parameters is matched, demonstrating its efficiency and effectiveness. The code and supplementary material are available on the project page.

Title: Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?

• Authors: Yunbo Lyu, Zhou Yang, Yuqing Niu, Jing Jiang, David Lo
• Subjects: cs.CV, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.15775
• Pdf URL: https://arxiv.org/pdf/2501.15775
• Copy Paste: [[2501.15775]] Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?(https://arxiv.org/abs/2501.15775)
• Keywords: diffusion
• Abstract: Text-to-Image (T2I) models have recently gained significant attention due to their ability to generate high-quality images and are consequently used in a wide range of applications. However, there are concerns about the gender bias of these models. Previous studies have shown that T2I models can perpetuate or even amplify gender stereotypes when provided with neutral text prompts. Researchers have proposed automated gender bias uncovering detectors for T2I models, but a crucial gap exists: no existing work comprehensively compares the various detectors and understands how the gender bias detected by them deviates from the actual situation. This study addresses this gap by validating previous gender bias detectors using a manually labeled dataset and comparing how the bias identified by various detectors deviates from the actual bias in T2I models, as verified by manual confirmation. We create a dataset consisting of 6,000 images generated from three cutting-edge T2I models: Stable Diffusion XL, Stable Diffusion 3, and Dreamlike Photoreal 2.0. During the human-labeling process, we find that all three T2I models generate a portion (12.48% on average) of low-quality images (e.g., generate images with no face present), where human annotators cannot determine the gender of the person. Our analysis reveals that all three T2I models show a preference for generating male images, with SDXL being the most biased. Additionally, images generated using prompts containing professional descriptions (e.g., lawyer or doctor) show the most bias. We evaluate seven gender bias detectors and find that none fully capture the actual level of bias in T2I models, with some detectors overestimating bias by up to 26.95%. We further investigate the causes of inaccurate estimations, highlighting the limitations of detectors in dealing with low-quality images. Based on our findings, we propose an enhanced detector...

Title: Large Language Models to Diffusion Finetuning

• Authors: Edoardo Cetin, Tianyu Zhao, Yujin Tang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15781
• Pdf URL: https://arxiv.org/pdf/2501.15781
• Copy Paste: [[2501.15781]] Large Language Models to Diffusion Finetuning(https://arxiv.org/abs/2501.15781)
• Keywords: diffusion, large language model
• Abstract: We propose a new finetuning method to provide pre-trained large language models (LMs) the ability to scale test-time compute through the diffusion framework. By increasing the number of diffusion steps, we show our finetuned models achieve monotonically increasing accuracy, directly translating to improved performance across downstream tasks. Furthermore, our finetuned models can expertly answer questions on specific topics by integrating powerful guidance techniques, and autonomously determine the compute required for a given problem by leveraging adaptive ODE solvers. Our method is universally applicable to any foundation model pre-trained with a cross-entropy loss and does not modify any of its original weights, fully preserving its strong single-step generation capabilities. We show our method is more effective and fully compatible with traditional finetuning approaches, introducing an orthogonal new direction to unify the strengths of the autoregressive and diffusion frameworks.

Title: Memorization and Regularization in Generative Diffusion Models

• Authors: Ricardo Baptista, Agnimitra Dasgupta, Nikola B. Kovachki, Assad Oberai, Andrew M. Stuart
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/2501.15785
• Pdf URL: https://arxiv.org/pdf/2501.15785
• Copy Paste: [[2501.15785]] Memorization and Regularization in Generative Diffusion Models(https://arxiv.org/abs/2501.15785)
• Keywords: diffusion, generative
• Abstract: Diffusion models have emerged as a powerful framework for generative modeling. At the heart of the methodology is score matching: learning gradients of families of log-densities for noisy versions of the data distribution at different scales. When the loss function adopted in score matching is evaluated using empirical data, rather than the population loss, the minimizer corresponds to the score of a time-dependent Gaussian mixture. However, use of this analytically tractable minimizer leads to data memorization: in both unconditioned and conditioned settings, the generative model returns the training samples. This paper contains an analysis of the dynamical mechanism underlying memorization. The analysis highlights the need for regularization to avoid reproducing the analytically tractable minimizer; and, in so doing, lays the foundations for a principled understanding of how to regularize. Numerical experiments investigate the properties of: (i) Tikhonov regularization; (ii) regularization designed to promote asymptotic consistency; and (iii) regularizations induced by under-parameterization of a neural network or by early stopping when training a neural network. These experiments are evaluated in the context of memorization, and directions for future development of regularization are highlighted.

Title: Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

• Authors: Zhiling Chen, Hanning Chen, Mohsen Imani, Farhad Imani
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15795
• Pdf URL: https://arxiv.org/pdf/2501.15795
• Copy Paste: [[2501.15795]] Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?(https://arxiv.org/abs/2501.15795)
• Keywords: robust, large language model
• Abstract: In industrial settings, the accurate detection of anomalies is essential for maintaining product quality and ensuring operational safety. Traditional industrial anomaly detection (IAD) models often struggle with flexibility and adaptability, especially in dynamic production environments where new defect types and operational changes frequently arise. Recent advancements in Multimodal Large Language Models (MLLMs) hold promise for overcoming these limitations by combining visual and textual information processing capabilities. MLLMs excel in general visual understanding due to their training on large, diverse datasets, but they lack domain-specific knowledge, such as industry-specific defect tolerance levels, which limits their effectiveness in IAD tasks. To address these challenges, we propose Echo, a novel multi-expert framework designed to enhance MLLM performance for IAD. Echo integrates four expert modules: Reference Extractor which provides a contextual baseline by retrieving similar normal images, Knowledge Guide which supplies domain-specific insights, Reasoning Expert which enables structured, stepwise reasoning for complex queries, and Decision Maker which synthesizes information from all modules to deliver precise, context-aware responses. Evaluated on the MMAD benchmark, Echo demonstrates significant improvements in adaptability, precision, and robustness, moving closer to meeting the demands of real-world industrial anomaly detection.

Title: LemmaHead: RAG Assisted Proof Generation Using Large Language Models

• Authors: Tianbo Yang, Mingqi Yang, Hongyi Zhao, Tianshuo Yang
• Subjects: cs.LG, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.15797
• Pdf URL: https://arxiv.org/pdf/2501.15797
• Copy Paste: [[2501.15797]] LemmaHead: RAG Assisted Proof Generation Using Large Language Models(https://arxiv.org/abs/2501.15797)
• Keywords: large language model
• Abstract: Developing the logic necessary to solve mathematical problems or write mathematical proofs is one of the more difficult objectives for large language models (LLMS). Currently, the most popular methods in literature consists of fine-tuning the model on written mathematical content such as academic publications and textbooks, so that the model can learn to emulate the style of mathematical writing. In this project, we explore the effectiveness of using retrieval augmented generation (RAG) to address gaps in the mathematical reasoning of LLMs. We develop LemmaHead, a RAG knowledge base that supplements queries to the model with relevant mathematical context, with particular focus on context from published textbooks. To measure our model's performance in mathematical reasoning, our testing paradigm focuses on the task of automated theorem proving via generating proofs to a given mathematical claim in the Lean formal language.

Title: MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining

• Authors: Ruiqi Wu, Na Su, Chenran Zhang, Tengfei Ma, Tao Zhou, Zhiting Cui, Nianfeng Tang, Tianyu Mao, Yi Zhou, Wen Fan, Tianxing Wu, Shenqi Jing, Huazhu Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15798
• Pdf URL: https://arxiv.org/pdf/2501.15798
• Copy Paste: [[2501.15798]] MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining(https://arxiv.org/abs/2501.15798)
• Keywords: generative
• Abstract: Vision-language pretraining (VLP) has been investigated to generalize across diverse downstream tasks for fundus image analysis. Although recent methods showcase promising achievements, they significantly rely on large-scale private image-text data but pay less attention to the pretraining manner, which limits their further advancements. In this work, we introduce MM-Retinal V2, a high-quality image-text paired dataset comprising CFP, FFA, and OCT image modalities. Then, we propose a novel fundus vision-language pretraining model, namely KeepFIT V2, which is pretrained by integrating knowledge from the elite data spark into categorical public datasets. Specifically, a preliminary textual pretraining is adopted to equip the text encoder with primarily ophthalmic textual knowledge. Moreover, a hybrid image-text knowledge injection module is designed for knowledge transfer, which is essentially based on a combination of global semantic concepts from contrastive learning and local appearance details from generative learning. Extensive experiments across zero-shot, few-shot, and linear probing settings highlight the generalization and transferability of KeepFIT V2, delivering performance competitive to state-of-the-art fundus VLP models trained on large-scale private image-text datasets. Our dataset and model are publicly available via this https URL.

Title: ClearSight: Human Vision-Inspired Solutions for Event-Based Motion Deblurring

• Authors: Xiaopeng Lin, Yulong Huang, Hongwei Ren, Zunchang Liu, Yue Zhou, Haotian Fu, Bojun Cheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15808
• Pdf URL: https://arxiv.org/pdf/2501.15808
• Copy Paste: [[2501.15808]] ClearSight: Human Vision-Inspired Solutions for Event-Based Motion Deblurring(https://arxiv.org/abs/2501.15808)
• Keywords: extraction
• Abstract: Motion deblurring addresses the challenge of image blur caused by camera or scene movement. Event cameras provide motion information that is encoded in the asynchronous event streams. To efficiently leverage the temporal information of event streams, we employ Spiking Neural Networks (SNNs) for motion feature extraction and Artificial Neural Networks (ANNs) for color information processing. Due to the non-uniform distribution and inherent redundancy of event data, existing cross-modal feature fusion methods exhibit certain limitations. Inspired by the visual attention mechanism in the human visual system, this study introduces a bioinspired dual-drive hybrid network (BDHNet). Specifically, the Neuron Configurator Module (NCM) is designed to dynamically adjusts neuron configurations based on cross-modal features, thereby focusing the spikes in blurry regions and adapting to varying blurry scenarios dynamically. Additionally, the Region of Blurry Attention Module (RBAM) is introduced to generate a blurry mask in an unsupervised manner, effectively extracting motion clues from the event features and guiding more accurate cross-modal feature fusion. Extensive subjective and objective evaluations demonstrate that our method outperforms current state-of-the-art methods on both synthetic and real-world datasets.

Title: MADP: Multi-Agent Deductive Planning for Enhanced Cognitive-Behavioral Mental Health Question Answer

• Authors: Qi Chen, Dexi Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15826
• Pdf URL: https://arxiv.org/pdf/2501.15826
• Copy Paste: [[2501.15826]] MADP: Multi-Agent Deductive Planning for Enhanced Cognitive-Behavioral Mental Health Question Answer(https://arxiv.org/abs/2501.15826)
• Keywords: large language model
• Abstract: The Mental Health Question Answer (MHQA) task requires the seeker and supporter to complete the support process in one-turn dialogue. Given the richness of help-seeker posts, supporters must thoroughly understand the content and provide logical, comprehensive, and well-structured responses. Previous works in MHQA mostly focus on single-agent approaches based on the cognitive element of Cognitive Behavioral Therapy (CBT), but they overlook the interactions among various CBT elements, such as emotion and cognition. This limitation hinders the models' ability to thoroughly understand the distress of help-seekers. To address this, we propose a framework named Multi-Agent Deductive Planning (MADP), which is based on the interactions between the various psychological elements of CBT. This method guides Large Language Models (LLMs) to achieve a deeper understanding of the seeker's context and provide more personalized assistance based on individual circumstances. Furthermore, we construct a new dataset based on the MADP framework and use it to fine-tune LLMs, resulting in a specialized model named MADP-LLM. We conduct extensive experiments, including comparisons with multiple LLMs, human evaluations, and automatic evaluations, to validate the effectiveness of the MADP framework and MADP-LLM.

Title: Intelligent Code Embedding Framework for High-Precision Ransomware Detection via Multimodal Execution Path Analysis

• Authors: Levi Gareth, Maximilian Fairbrother, Peregrine Blackwood, Lucasta Underhill, Benedict Ruthermore
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15836
• Pdf URL: https://arxiv.org/pdf/2501.15836
• Copy Paste: [[2501.15836]] Intelligent Code Embedding Framework for High-Precision Ransomware Detection via Multimodal Execution Path Analysis(https://arxiv.org/abs/2501.15836)
• Keywords: attack, robust
• Abstract: Modern threat landscapes continue to evolve with increasing sophistication, challenging traditional detection methodologies and necessitating innovative solutions capable of addressing complex adversarial tactics. A novel framework was developed to identify ransomware activity through multimodal execution path analysis, integrating high-dimensional embeddings and dynamic heuristic derivation mechanisms to capture behavioral patterns across diverse attack variants. The approach demonstrated high adaptability, effectively mitigating obfuscation strategies and polymorphic characteristics often employed by ransomware families to evade detection. Comprehensive experimental evaluations revealed significant advancements in precision, recall, and accuracy metrics compared to baseline techniques, particularly under conditions of variable encryption speeds and obfuscated execution flows. The framework achieved scalable and computationally efficient performance, ensuring robust applicability across a range of system configurations, from resource-constrained environments to high-performance infrastructures. Notable findings included reduced false positive rates and enhanced detection latency, even for ransomware families employing sophisticated encryption mechanisms. The modular design allowed seamless integration of additional modalities, enabling extensibility and future-proofing against emerging threat vectors. Quantitative analyses further highlighted the system's energy efficiency, emphasizing its practicality for deployment in environments with stringent operational constraints. The results underline the importance of integrating advanced computational techniques and dynamic adaptability to safeguard digital ecosystems from increasingly complex threats.

Title: Controllable Hand Grasp Generation for HOI and Efficient Evaluation Methods

• Authors: Ishant, Rongliang Wu, Joo Hwee Lim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15839
• Pdf URL: https://arxiv.org/pdf/2501.15839
• Copy Paste: [[2501.15839]] Controllable Hand Grasp Generation for HOI and Efficient Evaluation Methods(https://arxiv.org/abs/2501.15839)
• Keywords: diffusion
• Abstract: Controllable affordance Hand-Object Interaction (HOI) generation has become an increasingly important area of research in computer vision. In HOI generation, the hand grasp generation is a crucial step for effectively controlling the geometry of the hand. Current hand grasp generation methods rely on 3D information for both the hand and the object. In addition, these methods lack controllability concerning the hand's location and orientation. We treat the hand pose as the discrete graph structure and exploit the geometric priors. It is well established that higher order contextual dependency among the points improves the quality of the results in general. We propose a framework of higher order geometric representations (HOR's) inspired by spectral graph theory and vector algebra to improve the quality of generated hand poses. We demonstrate the effectiveness of our proposed HOR's in devising a controllable novel diffusion method (based on 2D information) for hand grasp generation that outperforms the state of the art (SOTA). Overcoming the limitations of existing methods: like lacking of controllability and dependency on 3D information. Once we have the generated pose, it is very natural to evaluate them using a metric. Popular metrics like FID and MMD are biased and inefficient for evaluating the generated hand poses. Using our proposed HOR's, we introduce an efficient and stable framework of evaluation metrics for grasp generation methods, addressing inefficiencies and biases in FID and MMD.

Title: Beyond In-Distribution Performance: A Cross-Dataset Study of Trajectory Prediction Robustness

• Authors: Yue Yao, Daniel Goehring, Joerg Reichardt
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.15842
• Pdf URL: https://arxiv.org/pdf/2501.15842
• Copy Paste: [[2501.15842]] Beyond In-Distribution Performance: A Cross-Dataset Study of Trajectory Prediction Robustness(https://arxiv.org/abs/2501.15842)
• Keywords: robust
• Abstract: We study the Out-of-Distribution (OoD) generalization ability of three SotA trajectory prediction models with comparable In-Distribution (ID) performance but different model designs. We investigate the influence of inductive bias, size of training data and data augmentation strategy by training the models on Argoverse 2 (A2) and testing on Waymo Open Motion (WO) and vice versa. We find that the smallest model with highest inductive bias exhibits the best OoD generalization across different augmentation strategies when trained on the smaller A2 dataset and tested on the large WO dataset. In the converse setting, training all models on the larger WO dataset and testing on the smaller A2 dataset, we find that all models generalize poorly, even though the model with the highest inductive bias still exhibits the best generalization ability. We discuss possible reasons for this surprising finding and draw conclusions about the design and test of trajectory prediction models and benchmarks.

Title: Can Location Embeddings Enhance Super-Resolution of Satellite Imagery?

• Authors: Daniel Panangian, Ksenia Bittner
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.15847
• Pdf URL: https://arxiv.org/pdf/2501.15847
• Copy Paste: [[2501.15847]] Can Location Embeddings Enhance Super-Resolution of Satellite Imagery?(https://arxiv.org/abs/2501.15847)
• Keywords: diffusion, generative, segmentation
• Abstract: Publicly available satellite imagery, such as Sentinel- 2, often lacks the spatial resolution required for accurate analysis of remote sensing tasks including urban planning and disaster response. Current super-resolution techniques are typically trained on limited datasets, leading to poor generalization across diverse geographic regions. In this work, we propose a novel super-resolution framework that enhances generalization by incorporating geographic context through location embeddings. Our framework employs Generative Adversarial Networks (GANs) and incorporates techniques from diffusion models to enhance image quality. Furthermore, we address tiling artifacts by integrating information from neighboring images, enabling the generation of seamless, high-resolution outputs. We demonstrate the effectiveness of our method on the building segmentation task, showing significant improvements over state-of-the-art methods and highlighting its potential for real-world applications.

Title: LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models

• Authors: Yuewen Mei, Tong Nie, Jian Sun, Ye Tian
• Subjects: cs.LG, cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.15850
• Pdf URL: https://arxiv.org/pdf/2501.15850
• Copy Paste: [[2501.15850]] LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models(https://arxiv.org/abs/2501.15850)
• Keywords: attack, robust, large language model
• Abstract: Ensuring and improving the safety of autonomous driving systems (ADS) is crucial for the deployment of highly automated vehicles, especially in safety-critical events. To address the rarity issue, adversarial scenario generation methods are developed, in which behaviors of traffic participants are manipulated to induce safety-critical events. However, existing methods still face two limitations. First, identification of the adversarial participant directly impacts the effectiveness of the generation. However, the complexity of real-world scenarios, with numerous participants and diverse behaviors, makes identification challenging. Second, the potential of generated safety-critical scenarios to continuously improve ADS performance remains underexplored. To address these issues, we propose LLM-attacker: a closed-loop adversarial scenario generation framework leveraging large language models (LLMs). Specifically, multiple LLM agents are designed and coordinated to identify optimal attackers. Then, the trajectories of the attackers are optimized to generate adversarial scenarios. These scenarios are iteratively refined based on the performance of ADS, forming a feedback loop to improve ADS. Experimental results show that LLM-attacker can create more dangerous scenarios than other methods, and the ADS trained with it achieves a collision rate half that of training with normal scenarios. This indicates the ability of LLM-attacker to test and enhance the safety and robustness of ADS. Video demonstrations are provided at: this https URL.

Title: D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation

• Authors: Maik Steinhauser, Laurenz Reichardt, Nikolas Ebert, Oliver Wasenmüller
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15870
• Pdf URL: https://arxiv.org/pdf/2501.15870
• Copy Paste: [[2501.15870]] D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation(https://arxiv.org/abs/2501.15870)
• Keywords: segmentation
• Abstract: This paper introduces a novel approach to 4D Panoptic LiDAR Segmentation that decouples semantic and instance segmentation, leveraging single-scan semantic predictions as prior information for instance segmentation. Our method D-PLS first performs single-scan semantic segmentation and aggregates the results over time, using them to guide instance segmentation. The modular design of D-PLS allows for seamless integration on top of any semantic segmentation architecture, without requiring architectural changes or retraining. We evaluate our approach on the SemanticKITTI dataset, where it demonstrates significant improvements over the baseline in both classification and association tasks, as measured by the LiDAR Segmentation and Tracking Quality (LSTQ) metric. Furthermore, we show that our decoupled architecture not only enhances instance prediction but also surpasses the baseline due to advancements in single-scan semantic segmentation.

Title: LCTG Bench: LLM Controlled Text Generation Benchmark

• Authors: Kentaro Kurihara, Masato Mita, Peinan Zhang, Shota Sasaki, Ryosuke Ishigami, Naoaki Okazaki
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15875
• Pdf URL: https://arxiv.org/pdf/2501.15875
• Copy Paste: [[2501.15875]] LCTG Bench: LLM Controlled Text Generation Benchmark(https://arxiv.org/abs/2501.15875)
• Keywords: large language model
• Abstract: The rise of large language models (LLMs) has led to more diverse and higher-quality machine-generated text. However, their high expressive power makes it difficult to control outputs based on specific business instructions. In response, benchmarks focusing on the controllability of LLMs have been developed, but several issues remain: (1) They primarily cover major languages like English and Chinese, neglecting low-resource languages like Japanese; (2) Current benchmarks employ task-specific evaluation metrics, lacking a unified framework for selecting models based on controllability across different use cases. To address these challenges, this research introduces LCTG Bench, the first Japanese benchmark for evaluating the controllability of LLMs. LCTG Bench provides a unified framework for assessing control performance, enabling users to select the most suitable model for their use cases based on controllability. By evaluating nine diverse Japanese-specific and multilingual LLMs like GPT-4, we highlight the current state and challenges of controllability in Japanese LLMs and reveal the significant gap between multilingual models and Japanese-specific models.

Title: Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation

• Authors: Adil Kaan Akan, Yucel Yemez
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15878
• Pdf URL: https://arxiv.org/pdf/2501.15878
• Copy Paste: [[2501.15878]] Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation(https://arxiv.org/abs/2501.15878)
• Keywords: diffusion, generative
• Abstract: We present SlotAdapt, an object-centric learning method that combines slot attention with pretrained diffusion models by introducing adapters for slot-based conditioning. Our method preserves the generative power of pretrained diffusion models, while avoiding their text-centric conditioning bias. We also incorporate an additional guidance loss into our architecture to align cross-attention from adapter layers with slot attention. This enhances the alignment of our model with the objects in the input image without using external supervision. Experimental results show that our method outperforms state-of-the-art techniques in object discovery and image generation tasks across multiple datasets, including those with real images. Furthermore, we demonstrate through experiments that our method performs remarkably well on complex real-world images for compositional generation, in contrast to other slot-based generative methods in the literature. The project page can be found at $\href{this https URL}{\text{this https url}}$.

Title: A Data-Centric Approach: Dimensions of Visual Complexity and How to find Them

• Authors: Karahan Sarıtaş, Tingke Shen, Surabhi S Nath, Peter Dayan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15890
• Pdf URL: https://arxiv.org/pdf/2501.15890
• Copy Paste: [[2501.15890]] A Data-Centric Approach: Dimensions of Visual Complexity and How to find Them(https://arxiv.org/abs/2501.15890)
• Keywords: interpretability, segmentation
• Abstract: Understanding how humans perceive visual complexity is a key area of study in visual cognition. Previous approaches to modeling visual complexity have often resulted in intricate, difficult-to-interpret solutions that employ numerous features or sophisticated deep learning architectures. While these complex models achieve high performance on specific datasets, they often sacrifice interpretability, making it challenging to understand the factors driving human perception of complexity. A recent model based on image segmentations showed promise in addressing this challenge; however, it presented limitations in capturing structural and semantic aspects of visual complexity. In this paper, we propose viable and effective features to overcome these shortcomings. Specifically, we develop multiscale features for the structural aspect of complexity, including the Multiscale Sobel Gradient (MSG), which captures spatial intensity variations across scales, and Multiscale Unique Colors (MUC), which quantifies image colorfulness by indexing quantized RGB values. We also introduce a new dataset SVG based on Visual Genome to explore the semantic aspect of visual complexity, obtaining surprise scores based on the element of surprise in images, which we demonstrate significantly contributes to perceived complexity. Overall, we suggest that the nature of the data is fundamental to understanding and modeling visual complexity, highlighting the importance of both structural and semantic dimensions in providing a comprehensive, interpretable assessment. The code for our analysis, experimental setup, and dataset will be made publicly available upon acceptance.

Title: Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effects

• Authors: Victor Deng (ENS-PSL), Changhong Wang (LTCI, S2A, IDS), Gael Richard (S2A, IDS, LTCI), Brian McFee (NYU)
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15900
• Pdf URL: https://arxiv.org/pdf/2501.15900
• Copy Paste: [[2501.15900]] Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effects(https://arxiv.org/abs/2501.15900)
• Keywords: robust
• Abstract: In recent years, foundation models have significantly advanced data-driven systems across various domains. Yet, their underlying properties, especially when functioning as feature extractors, remain under-explored. In this paper, we investigate the sensitivity to audio effects of audio embeddings extracted from widely-used foundation models, including OpenL3, PANNs, and CLAP. We focus on audio effects as the source of sensitivity due to their prevalent presence in large audio datasets. By applying parameterized audio effects (gain, low-pass filtering, reverberation, and bitcrushing), we analyze the correlation between the deformation trajectories and the effect strength in the embedding space. We propose to quantify the dimensionality and linearizability of the deformation trajectories induced by audio effects using canonical correlation analysis. We find that there exists a direction along which the embeddings move monotonically as the audio effect strength increases, but that the subspace containing the displacements is generally high-dimensional. This shows that pre-trained audio embeddings do not globally linearize the effects. Our empirical results on instrument classification downstream tasks confirm that projecting out the estimated deformation directions cannot generally improve the robustness of pre-trained embeddings to audio effects.

Title: The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspective

• Authors: Michael Muehlebach, Zhiyu He, Michael I. Jordan
• Subjects: cs.LG, eess.SY, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.15910
• Pdf URL: https://arxiv.org/pdf/2501.15910
• Copy Paste: [[2501.15910]] The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspective(https://arxiv.org/abs/2501.15910)
• Keywords: transformer
• Abstract: We study the sample complexity of online reinforcement learning for nonlinear dynamical systems with continuous state and action spaces. Our analysis accommodates a large class of dynamical systems ranging from a finite set of nonlinear candidate models to models with bounded and Lipschitz continuous dynamics, to systems that are parametrized by a compact and real-valued set of parameters. In the most general setting, our algorithm achieves a policy regret of $\mathcal{O}(N \epsilon^2 + \mathrm{ln}(m(\epsilon))/\epsilon^2)$, where $N$ is the time horizon, $\epsilon$ is a user-specified discretization width, and $m(\epsilon)$ measures the complexity of the function class under consideration via its packing number. In the special case where the dynamics are parametrized by a compact and real-valued set of parameters (such as neural networks, transformers, etc.), we prove a policy regret of $\mathcal{O}(\sqrt{N p})$, where $p$ denotes the number of parameters, recovering earlier sample-complexity results that were derived for linear time-invariant dynamical systems. While this article focuses on characterizing sample complexity, the proposed algorithms are likely to be useful in practice, due to their simplicity, the ability to incorporate prior knowledge, and their benign transient behavior.

Title: Web Execution Bundles: Reproducible, Accurate, and Archivable Web Measurements

• Authors: Florian Hantke, Peter Snyder, Hamed Haddadi, Ben Stock
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15911
• Pdf URL: https://arxiv.org/pdf/2501.15911
• Copy Paste: [[2501.15911]] Web Execution Bundles: Reproducible, Accurate, and Archivable Web Measurements(https://arxiv.org/abs/2501.15911)
• Keywords: security, privacy, robust
• Abstract: Recently, reproducibility has become a cornerstone in the security and privacy research community, including artifact evaluations and even a new symposium topic. However, Web measurements lack tools that can be reused across many measurement tasks without modification, while being robust to circumvention, and accurate across the wide range of behaviors in the Web. As a result, most measurement studies use custom tools and varied archival formats, each of unknown correctness and significant limitations, systematically affecting the research's accuracy and reproducibility. To address these limitations, we present WebREC, a Web measurement tool that is, compared against the current state-of-the-art, accurate (i.e., correctly measures and attributes events not possible with existing tools), general (i.e., reusable without modification for a broad range of measurement tasks), and comprehensive (i.e., handling events from all relevant browser behaviors). We also present .web, an archival format for the accurate and reproducible measurement of a wide range of website behaviors. We empirically evaluate WebREC's accuracy by replicating well-known Web measurement studies and showing that WebREC's results more accurately match our baseline. We then assess if WebREC and .web succeed as general-purpose tools, which could be used to accomplish many Web measurement tasks without modification. We find that this is so: 70% of papers discussed in a 2024 web crawling SoK paper could be conducted using WebREC as is, and a larger number (48%) could be leveraged against .web archives without requiring any new crawling.

Title: Parametric Retrieval Augmented Generation

• Authors: Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, Yiqun Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.15915
• Pdf URL: https://arxiv.org/pdf/2501.15915
• Copy Paste: [[2501.15915]] Parametric Retrieval Augmented Generation(https://arxiv.org/abs/2501.15915)
• Keywords: large language model
• Abstract: Retrieval-augmented generation (RAG) techniques have emerged as a promising solution to enhance the reliability of large language models (LLMs) by addressing issues like hallucinations, outdated knowledge, and domain adaptation. In particular, existing RAG methods append relevant documents retrieved from external corpus or databases to the input of LLMs to guide their generation process, which we refer to as the in-context knowledge injection method. While this approach is simple and often effective, it has inherent limitations. Firstly, increasing the context length and number of relevant documents can lead to higher computational overhead and degraded performance, especially in complex reasoning tasks. More importantly, in-context knowledge injection operates primarily at the input level, but LLMs store their internal knowledge in their parameters. This gap fundamentally limits the capacity of in-context methods. To this end, we introduce Parametric retrieval-augmented generation (Parametric RAG), a new RAG paradigm that integrates external knowledge directly into the parameters of feed-forward networks (FFN) of an LLM through document parameterization. This approach not only saves online computational costs by eliminating the need to inject multiple documents into the LLMs' input context, but also deepens the integration of external knowledge into the parametric knowledge space of the LLM. Experimental results demonstrate that Parametric RAG substantially enhances both the effectiveness and efficiency of knowledge augmentation in LLMs. Also, it can be combined with in-context RAG methods to achieve even better performance. We have open-sourced all the code, data, and models in the following anonymized GitHub link: this https URL

Title: TimeHF: Billion-Scale Time Series Models Guided by Human Feedback

• Authors: Yongzhi Qi, Hao Hu, Dazhou Lei, Jianshen Zhang, Zhengxin Shi, Yulin Huang, Zhengyu Chen, Xiaoming Lin, Zuo-Jun Max Shen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15942
• Pdf URL: https://arxiv.org/pdf/2501.15942
• Copy Paste: [[2501.15942]] TimeHF: Billion-Scale Time Series Models Guided by Human Feedback(https://arxiv.org/abs/2501.15942)
• Keywords: large language model
• Abstract: Time series neural networks perform exceptionally well in real-world applications but encounter challenges such as limited scalability, poor generalization, and suboptimal zero-shot performance. Inspired by large language models, there is interest in developing large time series models (LTM) to address these issues. However, current methods struggle with training complexity, adapting human feedback, and achieving high predictive accuracy. We introduce TimeHF, a novel pipeline for creating LTMs with 6 billion parameters, incorporating human feedback. We use patch convolutional embedding to capture long time series information and design a human feedback mechanism called time-series policy optimization. Deployed in this http URL's supply chain, TimeHF handles automated replenishment for over 20,000 products, improving prediction accuracy by 33.21% over existing methods. This work advances LTM technology and shows significant industrial benefits.

Title: Enhancing the Convergence of Federated Learning Aggregation Strategies with Limited Data

• Authors: Judith Sáinz-Pardo Díaz, Álvaro López García
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15949
• Pdf URL: https://arxiv.org/pdf/2501.15949
• Copy Paste: [[2501.15949]] Enhancing the Convergence of Federated Learning Aggregation Strategies with Limited Data(https://arxiv.org/abs/2501.15949)
• Keywords: privacy, robust, federate
• Abstract: The development of deep learning techniques is a leading field applied to cases in which medical data is used, particularly in cases of image diagnosis. This type of data has privacy and legal restrictions that in many cases prevent it from being processed from central servers. However, in this area collaboration between different research centers, in order to create models as robust as possible, trained with the largest quantity and diversity of data available, is a critical point to be taken into account. In this sense, the application of privacy aware distributed architectures, such as federated learning arises. When applying this type of architecture, the server aggregates the different local models trained with the data of each data owner to build a global model. This point is critical and therefore it is fundamental to analyze different ways of aggregation according to the use case, taking into account the distribution of the clients, the characteristics of the model, etc. In this paper we propose a novel aggregation strategy and we apply it to a use case of cerebral magnetic resonance image classification. In this use case the aggregation function proposed manages to improve the convergence obtained over the rounds of the federated learning process in relation to different aggregation strategies classically implemented and applied.

Title: Inverse Reinforcement Learning via Convex Optimization

• Authors: Hao Zhu, Yuan Zhang, Joschka Boedecker
• Subjects: cs.LG, cs.CE, math.OC, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2501.15957
• Pdf URL: https://arxiv.org/pdf/2501.15957
• Copy Paste: [[2501.15957]] Inverse Reinforcement Learning via Convex Optimization(https://arxiv.org/abs/2501.15957)
• Keywords: robust
• Abstract: We consider the inverse reinforcement learning (IRL) problem, where an unknown reward function of some Markov decision process is estimated based on observed expert demonstrations. In most existing approaches, IRL is formulated and solved as a nonconvex optimization problem, posing challenges in scenarios where robustness and reproducibility are critical. We discuss a convex formulation of the IRL problem (CIRL) initially proposed by Ng and Russel, and reformulate the problem such that the domain-specific language CVXPY can be applied directly to specify and solve the convex problem. We also extend the CIRL problem to scenarios where the expert policy is not given analytically but by trajectory as state-action pairs, which can be strongly inconsistent with optimality, by augmenting some of the constraints. Theoretical analysis and practical implementation for hyperparameter auto-selection are introduced. This note helps the users to easily apply CIRL for their problems, without background knowledge on convex optimization.

Title: An Explainable Disease Surveillance System for Early Prediction of Multiple Chronic Diseases

• Authors: Shaheer Ahmad Khan, Muhammad Usamah Shahid, Ahmad Abdullah, Ibrahim Hashmat, Muddassar Farooq
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.15969
• Pdf URL: https://arxiv.org/pdf/2501.15969
• Copy Paste: [[2501.15969]] An Explainable Disease Surveillance System for Early Prediction of Multiple Chronic Diseases(https://arxiv.org/abs/2501.15969)
• Keywords: explainability
• Abstract: This study addresses a critical gap in the healthcare system by developing a clinically meaningful, practical, and explainable disease surveillance system for multiple chronic diseases, utilizing routine EHR data from multiple U.S. practices integrated with CureMD's EMR/EHR system. Unlike traditional systems--using AI models that rely on features from patients' labs--our approach focuses on routinely available data, such as medical history, vitals, diagnoses, and medications, to preemptively assess the risks of chronic diseases in the next year. We trained three distinct models for each chronic disease: prediction models that forecast the risk of a disease 3, 6, and 12 months before a potential diagnosis. We developed Random Forest models, which were internally validated using F1 scores and AUROC as performance metrics and further evaluated by a panel of expert physicians for clinical relevance based on inferences grounded in medical knowledge. Additionally, we discuss our implementation of integrating these models into a practical EMR system. Beyond using Shapley attributes and surrogate models for explainability, we also introduce a new rule-engineering framework to enhance the intrinsic explainability of Random Forests.

Title: Integrating Probabilistic Trees and Causal Networks for Clinical and Epidemiological Data

• Authors: Sheresh Zahoor, Pietro Liò, Gaël Dias, Mohammed Hasanuzzaman
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2501.15973
• Pdf URL: https://arxiv.org/pdf/2501.15973
• Copy Paste: [[2501.15973]] Integrating Probabilistic Trees and Causal Networks for Clinical and Epidemiological Data(https://arxiv.org/abs/2501.15973)
• Keywords: robust, interpretability
• Abstract: Healthcare decision-making requires not only accurate predictions but also insights into how factors influence patient outcomes. While traditional Machine Learning (ML) models excel at predicting outcomes, such as identifying high risk patients, they are limited in addressing what-if questions about interventions. This study introduces the Probabilistic Causal Fusion (PCF) framework, which integrates Causal Bayesian Networks (CBNs) and Probability Trees (PTrees) to extend beyond predictions. PCF leverages causal relationships from CBNs to structure PTrees, enabling both the quantification of factor impacts and simulation of hypothetical interventions. PCF was validated on three real-world healthcare datasets i.e. MIMIC-IV, Framingham Heart Study, and Diabetes, chosen for their clinically diverse variables. It demonstrated predictive performance comparable to traditional ML models while providing additional causal reasoning capabilities. To enhance interpretability, PCF incorporates sensitivity analysis and SHapley Additive exPlanations (SHAP). Sensitivity analysis quantifies the influence of causal parameters on outcomes such as Length of Stay (LOS), Coronary Heart Disease (CHD), and Diabetes, while SHAP highlights the importance of individual features in predictive modeling. By combining causal reasoning with predictive modeling, PCF bridges the gap between clinical intuition and data-driven insights. Its ability to uncover relationships between modifiable factors and simulate hypothetical scenarios provides clinicians with a clearer understanding of causal pathways. This approach supports more informed, evidence-based decision-making, offering a robust framework for addressing complex questions in diverse healthcare settings.

Title: Provisioning Time-Based Subscription in NDN: A Secure and Efficient Access Control Scheme

• Authors: Nazatul H. Sultan, Chandan Kumar, Saurab Dulal, Vijay Varadharajan, Seyit Camtepe, Surya Nepal
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.15975
• Pdf URL: https://arxiv.org/pdf/2501.15975
• Copy Paste: [[2501.15975]] Provisioning Time-Based Subscription in NDN: A Secure and Efficient Access Control Scheme(https://arxiv.org/abs/2501.15975)
• Keywords: secure, security, attack
• Abstract: This paper proposes a novel encryption-based access control mechanism for Named Data Networking (NDN). The scheme allows data producers to share their content in encrypted form before transmitting it to consumers. The encryption mechanism incorporates time-based subscription access policies directly into the encrypted content, enabling only consumers with valid subscriptions to decrypt it. This makes the scheme well-suited for real-world, subscription-based applications like Netflix. Additionally, the scheme introduces an anonymous and unlinkable signature-based authentication mechanism that empowers edge routers to block bogus content requests at the network's entry point, thereby mitigating Denial of Service (DoS) attacks. A formal security proof demonstrates the scheme's resistance to Chosen Plaintext Attacks (CPA). Performance analysis, using Mini-NDN-based emulation and a Charm library implementation, further confirms the practicality of the scheme. Moreover, it outperforms closely related works in terms of functionality, security, and communication overhead.

Title: MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models

• Authors: Michael Birsak, John Femiani, Biao Zhang, Peter Wonka
• Subjects: cs.CV, cs.GR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.15981
• Pdf URL: https://arxiv.org/pdf/2501.15981
• Copy Paste: [[2501.15981]] MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models(https://arxiv.org/abs/2501.15981)
• Keywords: diffusion
• Abstract: Assigning realistic materials to 3D models remains a significant challenge in computer graphics. We propose MatCLIP, a novel method that extracts shape- and lighting-insensitive descriptors of Physically Based Rendering (PBR) materials to assign plausible textures to 3D objects based on images, such as the output of Latent Diffusion Models (LDMs) or photographs. Matching PBR materials to static images is challenging because the PBR representation captures the dynamic appearance of materials under varying viewing angles, shapes, and lighting conditions. By extending an Alpha-CLIP-based model on material renderings across diverse shapes and lighting, and encoding multiple viewing conditions for PBR materials, our approach generates descriptors that bridge the domains of PBR representations with photographs or renderings, including LDM outputs. This enables consistent material assignments without requiring explicit knowledge of material relationships between different parts of an object. MatCLIP achieves a top-1 classification accuracy of 76.6%, outperforming state-of-the-art methods such as PhotoShape and MatAtlas by over 15 percentage points on publicly available datasets. Our method can be used to construct material assignments for 3D shape datasets such as ShapeNet, 3DCoMPaT++, and Objaverse. All code and data will be released.

Title: 3CEL: A corpus of legal Spanish contract clauses

• Authors: Nuria Aldama García, Patricia Marsà Morales, David Betancur Sánchez, Álvaro Barbero Jiménez, Marta Guerrero Nieto, Pablo Haya Coll, Patricia Martín Chozas, Elena Montiel Ponsoda
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.15990
• Pdf URL: https://arxiv.org/pdf/2501.15990
• Copy Paste: [[2501.15990]] 3CEL: A corpus of legal Spanish contract clauses(https://arxiv.org/abs/2501.15990)
• Keywords: extraction
• Abstract: Legal corpora for Natural Language Processing (NLP) are valuable and scarce resources in languages like Spanish due to two main reasons: data accessibility and legal expert knowledge availability. INESData 2024 is a European Union funded project lead by the Universidad Politécnica de Madrid (UPM) and developed by Instituto de Ingeniería del Conocimiento (IIC) to create a series of state-of-the-art NLP resources applied to the legal/administrative domain in Spanish. The goal of this paper is to present the Corpus of Legal Spanish Contract Clauses (3CEL), which is a contract information extraction corpus developed within the framework of INESData 2024. 3CEL contains 373 manually annotated tenders using 19 defined categories (4 782 total tags) that identify key information for contract understanding and reviewing.

Title: Improving Tropical Cyclone Forecasting With Video Diffusion Models

• Authors: Zhibo Ren, Pritthijit Nath, Pancham Shukla
• Subjects: cs.CV, physics.ao-ph
• Abstract URL: https://arxiv.org/abs/2501.16003
• Pdf URL: https://arxiv.org/pdf/2501.16003
• Copy Paste: [[2501.16003]] Improving Tropical Cyclone Forecasting With Video Diffusion Models(https://arxiv.org/abs/2501.16003)
• Keywords: diffusion
• Abstract: Tropical cyclone (TC) forecasting is crucial for disaster preparedness and mitigation. While recent deep learning approaches have shown promise, existing methods often treat TC evolution as a series of independent frame-to-frame predictions, limiting their ability to capture long-term dynamics. We present a novel application of video diffusion models for TC forecasting that explicitly models temporal dependencies through additional temporal layers. Our approach enables the model to generate multiple frames simultaneously, better capturing cyclone evolution patterns. We introduce a two-stage training strategy that significantly improves individual-frame quality and performance in low-data regimes. Experimental results show our method outperforms the previous approach of Nath et al. by 19.3% in MAE, 16.2% in PSNR, and 36.1% in SSIM. Most notably, we extend the reliable forecasting horizon from 36 to 50 hours. Through comprehensive evaluation using both traditional metrics and Fréchet Video Distance (FVD), we demonstrate that our approach produces more temporally coherent forecasts while maintaining competitive single-frame quality. Code accessible at this https URL.

Title: TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference

• Authors: Jack Min Ong, Matthew Di Ferrante, Aaron Pazdera, Ryan Garner, Sami Jaghouar, Manveer Basra, Johannes Hagemann
• Subjects: cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.16007
• Pdf URL: https://arxiv.org/pdf/2501.16007
• Copy Paste: [[2501.16007]] TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference(https://arxiv.org/abs/2501.16007)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) have proven to be very capable, but access to the best models currently rely on inference providers which introduces trust challenges -- how can we be sure that the provider is using the model configuration they claim? We propose TOPLOC, a novel method for verifiable inference that addresses this problem. TOPLOC leverages a compact locality sensitive hashing mechanism for intermediate activations which can detect unauthorized modifications to models, prompts, or precision with 100% accuracy, achieving no false positives or negatives in our empirical evaluations. Our approach is robust across diverse hardware configurations, GPU types, and algebraic reorderings, which allows for validation speeds significantly faster than the original inference. By introducing a polynomial encoding scheme, TOPLOC minimizes memory overhead of the generated commits by $1000\times$, requiring only 258 bytes of storage per 32 new tokens compared to the 262KB requirement of storing the token embeddings directly for Llama-3.1-8B-Instruct. Our method empowers users to verify LLM inference computations efficiently, fostering greater trust and transparency in open ecosystems and lays a foundation for decentralized and verifiable AI services.

Title: Freestyle Sketch-in-the-Loop Image Segmentation

• Authors: Subhadeep Koley, Viswanatha Reddy Gajjala, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Ayan Kumar Bhunia, Yi-Zhe Song
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16022
• Pdf URL: https://arxiv.org/pdf/2501.16022
• Copy Paste: [[2501.16022]] Freestyle Sketch-in-the-Loop Image Segmentation(https://arxiv.org/abs/2501.16022)
• Keywords: segmentation
• Abstract: In this paper, we expand the domain of sketch research into the field of image segmentation, aiming to establish freehand sketches as a query modality for subjective image segmentation. Our innovative approach introduces a "sketch-in-the-loop" image segmentation framework, enabling the segmentation of visual concepts partially, completely, or in groupings - a truly "freestyle" approach - without the need for a purpose-made dataset (i.e., mask-free). This framework capitalises on the synergy between sketch-based image retrieval (SBIR) models and large-scale pre-trained models (CLIP or DINOv2). The former provides an effective training signal, while fine-tuned versions of the latter execute the subjective segmentation. Additionally, our purpose-made augmentation strategy enhances the versatility of our sketch-guided mask generation, allowing segmentation at multiple granularity levels. Extensive evaluations across diverse benchmark datasets underscore the superior performance of our method in comparison to existing approaches across various evaluation scenarios.

Title: FDLLM: A Text Fingerprint Detection Method for LLMs in Multi-Language, Multi-Domain Black-Box Environments

• Authors: Zhiyuan Fu, Junfan Chen, Hongyu Sun, Ting Yang, Ruidong Li, Yuqing Zhang
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16029
• Pdf URL: https://arxiv.org/pdf/2501.16029
• Copy Paste: [[2501.16029]] FDLLM: A Text Fingerprint Detection Method for LLMs in Multi-Language, Multi-Domain Black-Box Environments(https://arxiv.org/abs/2501.16029)
• Keywords: security, attack, large language model
• Abstract: Using large language models (LLMs) integration platforms without transparency about which LLM is being invoked can lead to potential security risks. Specifically, attackers may exploit this black-box scenario to deploy malicious models and embed viruses in the code provided to users. In this context, it is increasingly urgent for users to clearly identify the LLM they are interacting with, in order to avoid unknowingly becoming victims of malicious models. However, existing studies primarily focus on mixed classification of human and machine-generated text, with limited attention to classifying texts generated solely by different models. Current research also faces dual bottlenecks: poor quality of LLM-generated text (LLMGT) datasets and limited coverage of detectable LLMs, resulting in poor detection performance for various LLMGT in black-box scenarios. We propose the first LLMGT fingerprint detection model, \textbf{FDLLM}, based on Qwen2.5-7B and fine-tuned using LoRA to address these challenges. FDLLM can more efficiently handle detection tasks across multilingual and multi-domain scenarios. Furthermore, we constructed a dataset named \textbf{FD-Datasets}, consisting of 90,000 samples that span multiple languages and domains, covering 20 different LLMs. Experimental results demonstrate that FDLLM achieves a macro F1 score 16.7\% higher than the best baseline method, LM-D.

Title: Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge

• Authors: Anh-Kiet Duong, Petra Gomez-Krämer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16037
• Pdf URL: https://arxiv.org/pdf/2501.16037
• Copy Paste: [[2501.16037]] Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge(https://arxiv.org/abs/2501.16037)
• Keywords: privacy, robust
• Abstract: This paper presents a novel approach for hazard analysis in dashcam footage, addressing the detection of driver reactions to hazards, the identification of hazardous objects, and the generation of descriptive captions. We first introduce a method for detecting driver reactions through speed and sound anomaly detection, leveraging unsupervised learning techniques. For hazard detection, we employ a set of heuristic rules as weak classifiers, which are combined using an ensemble method. This ensemble approach is further refined with differential privacy to mitigate overconfidence, ensuring robustness despite the lack of labeled data. Lastly, we use state-of-the-art vision-language models for hazard captioning, generating descriptive labels for the detected hazards. Our method achieved the highest scores in the Challenge on Out-of-Label in Autonomous Driving, demonstrating its effectiveness across all three tasks. Source codes are publicly available at this https URL.

Title: CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

• Authors: Huazhong Zhao, Lei Qi, Xin Geng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16065
• Pdf URL: https://arxiv.org/pdf/2501.16065
• Copy Paste: [[2501.16065]] CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification(https://arxiv.org/abs/2501.16065)
• Keywords: robust
• Abstract: The Visual Language Model, known for its robust cross-modal capabilities, has been extensively applied in various computer vision tasks. In this paper, we explore the use of CLIP (Contrastive Language-Image Pretraining), a vision-language model pretrained on large-scale image-text pairs to align visual and textual features, for acquiring fine-grained and domain-invariant representations in generalizable person re-identification. The adaptation of CLIP to the task presents two primary challenges: learning more fine-grained features to enhance discriminative ability, and learning more domain-invariant features to improve the model's generalization capabilities. To mitigate the first challenge thereby enhance the ability to learn fine-grained features, a three-stage strategy is proposed to boost the accuracy of text descriptions. Initially, the image encoder is trained to effectively adapt to person re-identification tasks. In the second stage, the features extracted by the image encoder are used to generate textual descriptions (i.e., prompts) for each image. Finally, the text encoder with the learned prompts is employed to guide the training of the final image encoder. To enhance the model's generalization capabilities to unseen domains, a bidirectional guiding method is introduced to learn domain-invariant image features. Specifically, domain-invariant and domain-relevant prompts are generated, and both positive (pulling together image features and domain-invariant prompts) and negative (pushing apart image features and domain-relevant prompts) views are used to train the image encoder. Collectively, these strategies contribute to the development of an innovative CLIP-based framework for learning fine-grained generalized features in person re-identification.

Title: PISCO: Pretty Simple Compression for Retrieval-Augmented Generation

• Authors: Maxime Louis, Hervé Déjean, Stéphane Clinchant
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.16075
• Pdf URL: https://arxiv.org/pdf/2501.16075
• Copy Paste: [[2501.16075]] PISCO: Pretty Simple Compression for Retrieval-Augmented Generation(https://arxiv.org/abs/2501.16075)
• Keywords: large language model
• Abstract: Retrieval-Augmented Generation (RAG) pipelines enhance Large Language Models (LLMs) by retrieving relevant documents, but they face scalability issues due to high inference costs and limited context size. Document compression is a practical solution, but current soft compression methods suffer from accuracy losses and require extensive pretraining. In this paper, we introduce PISCO, a novel method that achieves a 16x compression rate with minimal accuracy loss (0-3%) across diverse RAG-based question-answering (QA) tasks. Unlike existing approaches, PISCO requires no pretraining or annotated data, relying solely on sequence-level knowledge distillation from document-based questions. With the ability to fine-tune a 7-10B LLM in 48 hours on a single A100 GPU, PISCO offers a highly efficient and scalable solution. We present comprehensive experiments showing that PISCO outperforms existing compression models by 8% in accuracy.

Title: RelCAT: Advancing Extraction of Clinical Inter-Entity Relationships from Unstructured Electronic Health Records

• Authors: Shubham Agarwal, Vlad Dinu, Thomas Searle, Mart Ratas, Anthony Shek, Dan F. Stein, James Teo, Richard Dobson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16077
• Pdf URL: https://arxiv.org/pdf/2501.16077
• Copy Paste: [[2501.16077]] RelCAT: Advancing Extraction of Clinical Inter-Entity Relationships from Unstructured Electronic Health Records(https://arxiv.org/abs/2501.16077)
• Keywords: extraction
• Abstract: This study introduces RelCAT (Relation Concept Annotation Toolkit), an interactive tool, library, and workflow designed to classify relations between entities extracted from clinical narratives. Building upon the CogStack MedCAT framework, RelCAT addresses the challenge of capturing complete clinical relations dispersed within text. The toolkit implements state-of-the-art machine learning models such as BERT and Llama along with proven evaluation and training methods. We demonstrate a dataset annotation tool (built within MedCATTrainer), model training, and evaluate our methodology on both openly available gold-standard and real-world UK National Health Service (NHS) hospital clinical datasets. We perform extensive experimentation and a comparative analysis of the various publicly available models with varied approaches selected for model fine-tuning. Finally, we achieve macro F1-scores of 0.977 on the gold-standard n2c2, surpassing the previous state-of-the-art performance, and achieve performance of >=0.93 F1 on our NHS gathered datasets.

Title: Integration of LLM Quality Assurance into an NLG System

• Authors: Ching-Yi Chen, Johanna Heininger, Adela Schneider, Christian Eckard, Andreas Madsack, Robert Weißgraeber
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16078
• Pdf URL: https://arxiv.org/pdf/2501.16078
• Copy Paste: [[2501.16078]] Integration of LLM Quality Assurance into an NLG System(https://arxiv.org/abs/2501.16078)
• Keywords: large language model
• Abstract: In this paper, we present a system that uses a Large Language Model (LLM) to perform grammar and spelling correction as a component of Quality Assurance (QA) for texts generated by NLG systems, which is important for text production in real-world scenarios. Evaluating the results of the system on work-in-progress sports news texts in three languages, we show that it is able to deliver acceptable corrections.

Title: Generating Spatial Synthetic Populations Using Wasserstein Generative Adversarial Network: A Case Study with EU-SILC Data for Helsinki and Thessaloniki

• Authors: Vanja Falck
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.16080
• Pdf URL: https://arxiv.org/pdf/2501.16080
• Copy Paste: [[2501.16080]] Generating Spatial Synthetic Populations Using Wasserstein Generative Adversarial Network: A Case Study with EU-SILC Data for Helsinki and Thessaloniki(https://arxiv.org/abs/2501.16080)
• Keywords: privacy, robust, generative
• Abstract: Using agent-based social simulations can enhance our understanding of urban planning, public health, and economic forecasting. Realistic synthetic populations with numerous attributes strengthen these simulations. The Wasserstein Generative Adversarial Network, trained on census data like EU-SILC, can create robust synthetic populations. These methods, aided by external statistics or EU-SILC weights, generate spatial synthetic populations for agent-based models. The increased access to high-quality micro-data has sparked interest in synthetic populations, which preserve demographic profiles and analytical strength while ensuring privacy and preventing discrimination. This study uses national data from Finland and Greece for Helsinki and Thessaloniki to explore balanced spatial synthetic population generation. Results show challenges related to balancing data with or without aggregated statistics for the target population and the general under-representation of fringe profiles by deep generative methods. The latter can lead to discrimination in agent-based simulations.

Title: Automated Detection of Sport Highlights from Audio and Video Sources

• Authors: Francesco Della Santa, Morgana Lalli
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.16100
• Pdf URL: https://arxiv.org/pdf/2501.16100
• Copy Paste: [[2501.16100]] Automated Detection of Sport Highlights from Audio and Video Sources(https://arxiv.org/abs/2501.16100)
• Keywords: robust
• Abstract: This study presents a novel Deep Learning-based and lightweight approach for the automated detection of sports highlights (HLs) from audio and video sources. HL detection is a key task in sports video analysis, traditionally requiring significant human effort. Our solution leverages Deep Learning (DL) models trained on relatively small datasets of audio Mel-spectrograms and grayscale video frames, achieving promising accuracy rates of 89% and 83% for audio and video detection, respectively. The use of small datasets, combined with simple architectures, demonstrates the practicality of our method for fast and cost-effective deployment. Furthermore, an ensemble model combining both modalities shows improved robustness against false positives and false negatives. The proposed methodology offers a scalable solution for automated HL detection across various types of sports video content, reducing the need for manual intervention. Future work will focus on enhancing model architectures and extending this approach to broader scene-detection tasks in media analysis.

Title: Towards Explainable Multimodal Depression Recognition for Clinical Interviews

• Authors: Wenjie Zheng, Qiming Xie, Zengzhi Wang, Jianfei Yu, Rui Xia
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16106
• Pdf URL: https://arxiv.org/pdf/2501.16106
• Copy Paste: [[2501.16106]] Towards Explainable Multimodal Depression Recognition for Clinical Interviews(https://arxiv.org/abs/2501.16106)
• Keywords: interpretability
• Abstract: Recently, multimodal depression recognition for clinical interviews (MDRC) has recently attracted considerable attention. Existing MDRC studies mainly focus on improving task performance and have achieved significant development. However, for clinical applications, model transparency is critical, and previous works ignore the interpretability of decision-making processes. To address this issue, we propose an Explainable Multimodal Depression Recognition for Clinical Interviews (EMDRC) task, which aims to provide evidence for depression recognition by summarizing symptoms and uncovering underlying causes. Given an interviewer-participant interaction scenario, the goal of EMDRC is to structured summarize participant's symptoms based on the eight-item Patient Health Questionnaire depression scale (PHQ-8), and predict their depression severity. To tackle the EMDRC task, we construct a new dataset based on an existing MDRC dataset. Moreover, we utilize the PHQ-8 and propose a PHQ-aware multimodal multi-task learning framework, which captures the utterance-level symptom-related semantic information to help generate dialogue-level summary. Experiment results on our annotated dataset demonstrate the superiority of our proposed methods over baseline systems on the EMDRC task.

Title: A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond

• Authors: Yipeng Li, Xinchen Lyu, Zhenyu Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.16117
• Pdf URL: https://arxiv.org/pdf/2501.16117
• Copy Paste: [[2501.16117]] A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond(https://arxiv.org/abs/2501.16117)
• Keywords: federate
• Abstract: We aim to provide a unified convergence analysis for permutation-based Stochastic Gradient Descent (SGD), where data examples are permuted before each epoch. By examining the relations among permutations, we categorize existing permutation-based SGD algorithms into four categories: Arbitrary Permutations, Independent Permutations (including Random Reshuffling), One Permutation (including Incremental Gradient, Shuffle One and Nice Permutation) and Dependent Permutations (including GraBs Lu et al., 2022; Cooper et al., 2023). Existing unified analyses failed to encompass the Dependent Permutations category due to the inter-epoch dependencies in its permutations. In this work, we propose a general assumption that captures the inter-epoch permutation dependencies. Using the general assumption, we develop a unified framework for permutation-based SGD with arbitrary permutations of examples, incorporating all the aforementioned representative algorithms. Furthermore, we adapt our framework on example ordering in SGD for client ordering in Federated Learning (FL). Specifically, we develop a unified framework for regularized-participation FL with arbitrary permutations of clients.

Title: Efficient Portrait Matte Creation With Layer Diffusion and Connectivity Priors

• Authors: Zhiyuan Lu, Hao Lu, Hua Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16147
• Pdf URL: https://arxiv.org/pdf/2501.16147
• Copy Paste: [[2501.16147]] Efficient Portrait Matte Creation With Layer Diffusion and Connectivity Priors(https://arxiv.org/abs/2501.16147)
• Keywords: diffusion, segmentation
• Abstract: Learning effective deep portrait matting models requires training data of both high quality and large quantity. Neither quality nor quantity can be easily met for portrait matting, however. Since the most accurate ground-truth portrait mattes are acquired in front of the green screen, it is almost impossible to harvest a large-scale portrait matting dataset in reality. This work shows that one can leverage text prompts and the recent Layer Diffusion model to generate high-quality portrait foregrounds and extract latent portrait mattes. However, the portrait mattes cannot be readily in use due to significant generation artifacts. Inspired by the connectivity priors observed in portrait images, that is, the border of portrait foregrounds always appears connected, a connectivity-aware approach is introduced to refine portrait mattes. Building on this, a large-scale portrait matting dataset is created, termed LD-Portrait-20K, with $20,051$ portrait foregrounds and high-quality alpha mattes. Extensive experiments demonstrated the value of the LD-Portrait-20K dataset, with models trained on it significantly outperforming those trained on other datasets. In addition, comparisons with the chroma keying algorithm and an ablation study on dataset capacity further confirmed the effectiveness of the proposed matte creation approach. Further, the dataset also contributes to state-of-the-art video portrait matting, implemented by simple video segmentation and a trimap-based image matting model trained on this dataset.

Title: MILP initialization for solving parabolic PDEs with PINNs

• Authors: Sirui Li, Federica Bragone, Matthieu Barreau, Kateryna Morozovska
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.16153
• Pdf URL: https://arxiv.org/pdf/2501.16153
• Copy Paste: [[2501.16153]] MILP initialization for solving parabolic PDEs with PINNs(https://arxiv.org/abs/2501.16153)
• Keywords: diffusion, transformer
• Abstract: Physics-Informed Neural Networks (PINNs) are a powerful deep learning method capable of providing solutions and parameter estimations of physical systems. Given the complexity of their neural network structure, the convergence speed is still limited compared to numerical methods, mainly when used in applications that model realistic systems. The network initialization follows a random distribution of the initial weights, as in the case of traditional neural networks, which could lead to severe model convergence bottlenecks. To overcome this problem, we follow current studies that deal with optimal initial weights in traditional neural networks. In this paper, we use a convex optimization model to improve the initialization of the weights in PINNs and accelerate convergence. We investigate two optimization models as a first training step, defined as pre-training, one involving only the boundaries and one including physics. The optimization is focused on the first layer of the neural network part of the PINN model, while the other weights are randomly initialized. We test the methods using a practical application of the heat diffusion equation to model the temperature distribution of power transformers. The PINN model with boundary pre-training is the fastest converging method at the current stage.

Title: AdaCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Chain-of-Thought

• Authors: Xin Huang, Tarun Kumar Vangani, Zhengyuan Liu, Bowei Zou, Ai Ti Aw
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16154
• Pdf URL: https://arxiv.org/pdf/2501.16154
• Copy Paste: [[2501.16154]] AdaCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Chain-of-Thought(https://arxiv.org/abs/2501.16154)
• Keywords: large language model
• Abstract: Large language models (LLMs) have shown impressive multilingual capabilities through pretraining on diverse corpora. While these models show strong reasoning abilities, their performance varies significantly across languages due to uneven training data distribution. Existing approaches using machine translation, and extensive multilingual pretraining and cross-lingual tuning face scalability challenges and often fail to capture nuanced reasoning processes across languages. In this paper, we introduce AdaCoT (Adaptive Chain-of-Thought), a framework that enhances multilingual reasoning by dynamically routing thought processes through intermediary "thinking languages" before generating target-language responses. AdaCoT leverages a language-agnostic core and incorporates an adaptive, reward-based mechanism for selecting optimal reasoning pathways without requiring additional pretraining. Our comprehensive evaluation across multiple benchmarks demonstrates substantial improvements in both factual reasoning quality and cross-lingual consistency, with particularly strong performance gains in low-resource language settings. The results suggest that adaptive reasoning paths can effectively bridge the performance gap between high and low-resource languages while maintaining cultural and linguistic nuances.

Title: Demystifying OS Kernel Fuzzing with a Novel Taxonomy

• Authors: Jiacheng Xu, He Sun, Shihao Jiang, Qinying Wang, Mingming Zhang, Xiang Li, Kaiwen Shen, Peng Cheng, Jiming Chen, Charles Zhang, Shouling Ji
• Subjects: cs.CR, cs.CY, cs.OS
• Abstract URL: https://arxiv.org/abs/2501.16165
• Pdf URL: https://arxiv.org/pdf/2501.16165
• Copy Paste: [[2501.16165]] Demystifying OS Kernel Fuzzing with a Novel Taxonomy(https://arxiv.org/abs/2501.16165)
• Keywords: security
• Abstract: The Operating System (OS) kernel is foundational in modern computing, especially with the proliferation of diverse computing devices. However, its development also comes with vulnerabilities that can lead to severe security breaches. Kernel fuzzing, a technique used to uncover these vulnerabilities, poses distinct challenges when compared to userspace fuzzing. These include the complexity of configuring the testing environment and addressing the statefulness inherent to both the kernel and the fuzzing process. Despite the significant interest from the security community, a comprehensive understanding of kernel fuzzing remains lacking, hindering further progress in the field. In this paper, we present the first systematic study dedicated to OS kernel fuzzing. It begins by summarizing the progress of 99 academic studies from top-tier venues between 2017 and 2024. Following this, we introduce a stage-based fuzzing model and a novel fuzzing taxonomy that highlights nine core functionalities unique to kernel fuzzing. These functionalities are examined alongside their corresponding methodological approaches based on qualitative evaluation criteria. Our systematization identifies challenges in meeting functionality requirements and proposes potential technical solutions. Finally, we outline promising and practical future directions to guide forthcoming research in kernel security, supported in part by insights derived from our case study.

Title: BAG: Body-Aligned 3D Wearable Asset Generation

• Authors: Zhongjin Luo, Yang Li, Mingrui Zhang, Senbo Wang, Han Yan, Xibin Song, Taizhang Shang, Wei Mao, Hongdong Li, Xiaoguang Han, Pan Ji
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2501.16177
• Pdf URL: https://arxiv.org/pdf/2501.16177
• Copy Paste: [[2501.16177]] BAG: Body-Aligned 3D Wearable Asset Generation(https://arxiv.org/abs/2501.16177)
• Keywords: diffusion
• Abstract: While recent advancements have shown remarkable progress in general 3D shape generation models, the challenge of leveraging these approaches to automatically generate wearable 3D assets remains unexplored. To this end, we present BAG, a Body-aligned Asset Generation method to output 3D wearable asset that can be automatically dressed on given 3D human bodies. This is achived by controlling the 3D generation process using human body shape and pose information. Specifically, we first build a general single-image to consistent multiview image diffusion model, and train it on the large Objaverse dataset to achieve diversity and generalizability. Then we train a Controlnet to guide the multiview generator to produce body-aligned multiview images. The control signal utilizes the multiview 2D projections of the target human body, where pixel values represent the XYZ coordinates of the body surface in a canonical space. The body-conditioned multiview diffusion generates body-aligned multiview images, which are then fed into a native 3D diffusion model to produce the 3D shape of the asset. Finally, by recovering the similarity transformation using multiview silhouette supervision and addressing asset-body penetration with physics simulators, the 3D asset can be accurately fitted onto the target human body. Experimental results demonstrate significant advantages over existing methods in terms of image prompt-following capability, shape diversity, and shape quality. Our project page is available at this https URL.

Title: SWIFT: Mapping Sub-series with Wavelet Decomposition Improves Time Series Forecasting

• Authors: Wenxuan Xie, Fanpu Cao
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.16178
• Pdf URL: https://arxiv.org/pdf/2501.16178
• Copy Paste: [[2501.16178]] SWIFT: Mapping Sub-series with Wavelet Decomposition Improves Time Series Forecasting(https://arxiv.org/abs/2501.16178)
• Keywords: transformer, large language model
• Abstract: In recent work on time-series prediction, Transformers and even large language models have garnered significant attention due to their strong capabilities in sequence modeling. However, in practical deployments, time-series prediction often requires operation in resource-constrained environments, such as edge devices, which are unable to handle the computational overhead of large models. To address such scenarios, some lightweight models have been proposed, but they exhibit poor performance on non-stationary sequences. In this paper, we propose $\textit{SWIFT}$, a lightweight model that is not only powerful, but also efficient in deployment and inference for Long-term Time Series Forecasting (LTSF). Our model is based on three key points: (i) Utilizing wavelet transform to perform lossless downsampling of time series. (ii) Achieving cross-band information fusion with a learnable filter. (iii) Using only one shared linear layer or one shallow MLP for sub-series' mapping. We conduct comprehensive experiments, and the results show that $\textit{SWIFT}$ achieves state-of-the-art (SOTA) performance on multiple datasets, offering a promising method for edge computing and deployment in this task. Moreover, it is noteworthy that the number of parameters in $\textit{SWIFT-Linear}$ is only 25\% of what it would be with a single-layer linear model for time-domain prediction. Our code is available at this https URL.

Title: The Linear Attention Resurrection in Vision Transformer

• Authors: Chuanyang Zheng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16182
• Pdf URL: https://arxiv.org/pdf/2501.16182
• Copy Paste: [[2501.16182]] The Linear Attention Resurrection in Vision Transformer(https://arxiv.org/abs/2501.16182)
• Keywords: transformer, segmentation
• Abstract: Vision Transformers (ViTs) have recently taken computer vision by storm. However, the softmax attention underlying ViTs comes with a quadratic complexity in time and memory, hindering the application of ViTs to high-resolution images. We revisit the attention design and propose a linear attention method to address the limitation, which doesn't sacrifice ViT's core advantage of capturing global representation like existing methods (e.g. local window attention of Swin). We further investigate the key difference between linear attention and softmax attention. Our empirical results suggest that linear attention lacks a fundamental property of concentrating the distribution of the attention matrix. Inspired by this observation, we introduce a local concentration module to enhance linear attention. By incorporating enhanced linear global attention and local window attention, we propose a new ViT architecture, dubbed L$^2$ViT. Notably, L$^2$ViT can effectively capture both global interactions and local representations while enjoying linear computational complexity. Extensive experiments demonstrate the strong performance of L$^2$ViT. On image classification, L$^2$ViT achieves 84.4% Top-1 accuracy on ImageNet-1K without any extra training data or label. By further pre-training on ImageNet-22k, it attains 87.0% when fine-tuned with resolution 384$^2$. For downstream tasks, L$^2$ViT delivers favorable performance as a backbone on object detection as well as semantic segmentation.

Title: UDBE: Unsupervised Diffusion-based Brightness Enhancement in Underwater Images

• Authors: Tatiana Taís Schein, Gustavo Pereira de Almeira, Stephanie Loi Brião, Rodrigo Andrade de Bem, Felipe Gomes de Oliveira, Paulo L. J. Drews-Jr
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.16211
• Pdf URL: https://arxiv.org/pdf/2501.16211
• Copy Paste: [[2501.16211]] UDBE: Unsupervised Diffusion-based Brightness Enhancement in Underwater Images(https://arxiv.org/abs/2501.16211)
• Keywords: robust, diffusion
• Abstract: Activities in underwater environments are paramount in several scenarios, which drives the continuous development of underwater image enhancement techniques. A major challenge in this domain is the depth at which images are captured, with increasing depth resulting in a darker environment. Most existing methods for underwater image enhancement focus on noise removal and color adjustment, with few works dedicated to brightness enhancement. This work introduces a novel unsupervised learning approach to underwater image enhancement using a diffusion model. Our method, called UDBE, is based on conditional diffusion to maintain the brightness details of the unpaired input images. The input image is combined with a color map and a Signal-Noise Relation map (SNR) to ensure stable training and prevent color distortion in the output images. The results demonstrate that our approach achieves an impressive accuracy rate in the datasets UIEB, SUIM and RUIE, well-established underwater image benchmarks. Additionally, the experiments validate the robustness of our approach, regarding the image quality metrics PSNR, SSIM, UIQM, and UISM, indicating the good performance of the brightness enhancement process. The source code is available here: this https URL.

Title: Provence: efficient and robust context pruning for retrieval-augmented generation

• Authors: Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, Stéphane Clinchant
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16214
• Pdf URL: https://arxiv.org/pdf/2501.16214
• Copy Paste: [[2501.16214]] Provence: efficient and robust context pruning for retrieval-augmented generation(https://arxiv.org/abs/2501.16214)
• Keywords: robust, large language model
• Abstract: Retrieval-augmented generation improves various aspects of large language models (LLMs) generation, but suffers from computational overhead caused by long contexts as well as the propagation of irrelevant retrieved information into generated responses. Context pruning deals with both aspects, by removing irrelevant parts of retrieved contexts before LLM generation. Existing context pruning approaches are however limited, and do not provide a universal model that would be both efficient and robust in a wide range of scenarios, e.g., when contexts contain a variable amount of relevant information or vary in length, or when evaluated on various domains. In this work, we close this gap and introduce Provence (Pruning and Reranking Of retrieVEd relevaNt ContExts), an efficient and robust context pruner for Question Answering, which dynamically detects the needed amount of pruning for a given context and can be used out-of-the-box for various domains. The three key ingredients of Provence are formulating the context pruning task as sequence labeling, unifying context pruning capabilities with context reranking, and training on diverse data. Our experimental results show that Provence enables context pruning with negligible to no drop in performance, in various domains and settings, at almost no cost in a standard RAG pipeline. We also conduct a deeper analysis alongside various ablations to provide insights into training context pruners for future work.

Title: Automatic Calibration of a Multi-Camera System with Limited Overlapping Fields of View for 3D Surgical Scene Reconstruction

• Authors: Tim Flückiger (1 and 2), Jonas Hein (1 and 2), Valery Fischer (1), Philipp Fürnstahl (1), Lilian Calvet (1) ((1) ROCS University Hospital Balgrist University of Zurich, (2) CVG ETH Zurich)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16221
• Pdf URL: https://arxiv.org/pdf/2501.16221
• Copy Paste: [[2501.16221]] Automatic Calibration of a Multi-Camera System with Limited Overlapping Fields of View for 3D Surgical Scene Reconstruction(https://arxiv.org/abs/2501.16221)
• Keywords: robust, extraction
• Abstract: Purpose: The purpose of this study is to develop an automated and accurate external camera calibration method for multi-camera systems used in 3D surgical scene reconstruction (3D-SSR), eliminating the need for operator intervention or specialized expertise. The method specifically addresses the problem of limited overlapping fields of view caused by significant variations in optical zoom levels and camera locations. Methods: We contribute a novel, fast, and fully automatic calibration method based on the projection of multi-scale markers (MSMs) using a ceiling-mounted projector. MSMs consist of 2D patterns projected at varying scales, ensuring accurate extraction of well distributed point correspondences across significantly different viewpoints and zoom levels. Validation is performed using both synthetic and real data captured in a mock-up OR, with comparisons to traditional manual marker-based methods as well as markerless calibration methods. Results: The method achieves accuracy comparable to manual, operator-dependent calibration methods while exhibiting higher robustness under conditions of significant differences in zoom levels. Additionally, we show that state-of-the-art Structure-from-Motion (SfM) pipelines are ineffective in 3D-SSR settings, even when additional texture is projected onto the OR floor. Conclusion: The use of a ceiling-mounted entry-level projector proves to be an effective alternative to operator-dependent, traditional marker-based methods, paving the way for fully automated 3D-SSR.

Title: Language-Based Bayesian Optimization Research Assistant (BORA)

• Authors: Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, Andrew I. Cooper
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16224
• Pdf URL: https://arxiv.org/pdf/2501.16224
• Copy Paste: [[2501.16224]] Language-Based Bayesian Optimization Research Assistant (BORA)(https://arxiv.org/abs/2501.16224)
• Keywords: large language model
• Abstract: Many important scientific problems involve multivariate optimization coupled with slow and laborious experimental measurements. These complex, high-dimensional searches can be defined by non-convex optimization landscapes that resemble needle-in-a-haystack surfaces, leading to entrapment in local minima. Contextualizing optimizers with human domain knowledge is a powerful approach to guide searches to localized fruitful regions. However, this approach is susceptible to human confirmation bias and it is also challenging for domain experts to keep track of the rapidly expanding scientific literature. Here, we propose the use of Large Language Models (LLMs) for contextualizing Bayesian optimization (BO) via a hybrid optimization framework that intelligently and economically blends stochastic inference with domain knowledge-based insights from the LLM, which is used to suggest new, better-performing areas of the search space for exploration. Our method fosters user engagement by offering real-time commentary on the optimization progress, explaining the reasoning behind the search strategies. We validate the effectiveness of our approach on synthetic benchmarks with up to 15 independent variables and demonstrate the ability of LLMs to reason in four real-world experimental tasks where context-aware suggestions boost optimization performance substantially.

Title: PDC-ViT : Source Camera Identification using Pixel Difference Convolution and Vision Transformer

• Authors: Omar Elharrouss, Younes Akbari, Noor Almaadeed, Somaya Al-Maadeed, Fouad Khelifi, Ahmed Bouridane
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16227
• Pdf URL: https://arxiv.org/pdf/2501.16227
• Copy Paste: [[2501.16227]] PDC-ViT : Source Camera Identification using Pixel Difference Convolution and Vision Transformer(https://arxiv.org/abs/2501.16227)
• Keywords: robust, extraction, transformer
• Abstract: Source camera identification has emerged as a vital solution to unlock incidents involving critical cases like terrorism, violence, and other criminal activities. The ability to trace the origin of an image/video can aid law enforcement agencies in gathering evidence and constructing the timeline of events. Moreover, identifying the owner of a certain device narrows down the area of search in a criminal investigation where smartphone devices are involved. This paper proposes a new pixel-based method for source camera identification, integrating Pixel Difference Convolution (PDC) with a Vision Transformer network (ViT), and named PDC-ViT. While the PDC acts as the backbone for feature extraction by exploiting Angular PDC (APDC) and Radial PDC (RPDC). These techniques enhance the capability to capture subtle variations in pixel information, which are crucial for distinguishing between different source cameras. The second part of the methodology focuses on classification, which is based on a Vision Transformer network. Unlike traditional methods that utilize image patches directly for training the classification network, the proposed approach uniquely inputs PDC features into the Vision Transformer network. To demonstrate the effectiveness of the PDC-ViT approach, it has been assessed on five different datasets, which include various image contents and video scenes. The method has also been compared with state-of-the-art source camera identification methods. Experimental results demonstrate the effectiveness and superiority of the proposed system in terms of accuracy and robustness when compared to its competitors. For example, our proposed PDC-ViT has achieved an accuracy of 94.30%, 84%, 94.22% and 92.29% using the Vision dataset, Daxing dataset, Socrates dataset and QUFVD dataset, respectively.

Title: Application of Structured State Space Models to High energy physics with locality-sensitive hashing

• Authors: Cheng Jiang, Sitian Qian
• Subjects: cs.LG, physics.ins-det
• Abstract URL: https://arxiv.org/abs/2501.16237
• Pdf URL: https://arxiv.org/pdf/2501.16237
• Copy Paste: [[2501.16237]] Application of Structured State Space Models to High energy physics with locality-sensitive hashing(https://arxiv.org/abs/2501.16237)
• Keywords: robust, transformer
• Abstract: Modern high-energy physics (HEP) experiments are increasingly challenged by the vast size and complexity of their datasets, particularly regarding large-scale point cloud processing and long sequences. In this study, to address these challenges, we explore the application of structured state space models (SSMs), proposing one of the first trials to integrate local-sensitive hashing into either a hybrid or pure Mamba Model. Our results demonstrate that pure SSMs could serve as powerful backbones for HEP problems involving tasks for long sequence data with local inductive bias. By integrating locality-sensitive hashing into Mamba blocks, we achieve significant improvements over traditional backbones in key HEP tasks, surpassing them in inference speed and physics metrics while reducing computational overhead. In key tests, our approach demonstrated promising results, presenting a viable alternative to traditional transformer backbones by significantly reducing FLOPS while maintaining robust performance.

Title: Distilling foundation models for robust and efficient models in digital pathology

• Authors: Alexandre Filiot, Nicolas Dop, Oussama Tchita, Auriane Riou, Thomas Peeters, Daria Valter, Marin Scalbert, Charlie Saillard, Geneviève Robin, Antoine Olivier
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16239
• Pdf URL: https://arxiv.org/pdf/2501.16239
• Copy Paste: [[2501.16239]] Distilling foundation models for robust and efficient models in digital pathology(https://arxiv.org/abs/2501.16239)
• Keywords: robust
• Abstract: In recent years, the advent of foundation models (FM) for digital pathology has relied heavily on scaling the pre-training datasets and the model size, yielding large and powerful models. While it resulted in improving the performance on diverse downstream tasks, it also introduced increased computational cost and inference time. In this work, we explore the distillation of a large foundation model into a smaller one, reducing the number of parameters by several orders of magnitude. Leveraging distillation techniques, our distilled model, H0-mini, achieves nearly comparable performance to large FMs at a significantly reduced inference cost. It is evaluated on several public benchmarks, achieving 3rd place on the HEST benchmark and 5th place on the EVA benchmark. Additionally, a robustness analysis conducted on the PLISM dataset demonstrates that our distilled model reaches excellent robustness to variations in staining and scanning conditions, significantly outperforming other state-of-the art models. This opens new perspectives to design lightweight and robust models for digital pathology, without compromising on performance.

Title: Phase Transitions in Large Language Models and the $O(N)$ Model

• Authors: Youran Sun, Babak Haghighat
• Subjects: cs.LG, cs.CL, hep-th, physics.data-an
• Abstract URL: https://arxiv.org/abs/2501.16241
• Pdf URL: https://arxiv.org/pdf/2501.16241
• Copy Paste: [[2501.16241]] Phase Transitions in Large Language Models and the $O(N)$ Model(https://arxiv.org/abs/2501.16241)
• Keywords: transformer, large language model
• Abstract: Large language models (LLMs) exhibit unprecedentedly rich scaling behaviors. In physics, scaling behavior is closely related to phase transitions, critical phenomena, and field theory. To investigate the phase transition phenomena in LLMs, we reformulated the Transformer architecture as an $O(N)$ model. Our study reveals two distinct phase transitions corresponding to the temperature used in text generation and the model's parameter size, respectively. The first phase transition enables us to estimate the internal dimension of the model, while the second phase transition is of \textit{higher-depth} and signals the emergence of new capabilities. As an application, the energy of the $O(N)$ model can be used to evaluate whether an LLM's parameters are sufficient to learn the training data.

Title: CLISC: Bridging clip and sam by enhanced cam for unsupervised brain tumor segmentation

• Authors: Xiaochuan Ma, Jia Fu, Wenjun Liao, Shichuan Zhang, Guotai Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16246
• Pdf URL: https://arxiv.org/pdf/2501.16246
• Copy Paste: [[2501.16246]] CLISC: Bridging clip and sam by enhanced cam for unsupervised brain tumor segmentation(https://arxiv.org/abs/2501.16246)
• Keywords: segmentation
• Abstract: Brain tumor segmentation is important for diagnosis of the tumor, and current deep-learning methods rely on a large set of annotated images for training, with high annotation costs. Unsupervised segmentation is promising to avoid human annotations while the performance is often limited. In this study, we present a novel unsupervised segmentation approach that leverages the capabilities of foundation models, and it consists of three main steps: (1) A vision-language model (i.e., CLIP) is employed to obtain image-level pseudo-labels for training a classification network. Class Activation Mapping (CAM) is then employed to extract Regions of Interest (ROIs), where an adaptive masking-based data augmentation is used to enhance ROI identification.(2) The ROIs are used to generate bounding box and point prompts for the Segment Anything Model (SAM) to obtain segmentation pseudo-labels. (3) A 3D segmentation network is trained with the SAM-derived pseudo-labels, where low-quality pseudo-labels are filtered out in a self-learning process based on the similarity between the SAM's output and the network's prediction. Evaluation on the BraTS2020 dataset demonstrates that our approach obtained an average Dice Similarity Score (DSC) of 85.60%, outperforming five state-of-the-art unsupervised segmentation methods by more than 10 percentage points. Besides, our approach outperforms directly using SAM for zero-shot inference, and its performance is close to fully supervised learning.

Title: Zero-Shot Decision Tree Construction via Large Language Models

• Authors: Lucas Carrasco, Felipe Urrutia, Andrés Abeliuk
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16247
• Pdf URL: https://arxiv.org/pdf/2501.16247
• Copy Paste: [[2501.16247]] Zero-Shot Decision Tree Construction via Large Language Models(https://arxiv.org/abs/2501.16247)
• Keywords: interpretability, large language model
• Abstract: This paper introduces a novel algorithm for constructing decision trees using large language models (LLMs) in a zero-shot manner based on Classification and Regression Trees (CART) principles. Traditional decision tree induction methods rely heavily on labeled data to recursively partition data using criteria such as information gain or the Gini index. In contrast, we propose a method that uses the pre-trained knowledge embedded in LLMs to build decision trees without requiring training data. Our approach leverages LLMs to perform operations essential for decision tree construction, including attribute discretization, probability calculation, and Gini index computation based on the probabilities. We show that these zero-shot decision trees can outperform baseline zero-shot methods and achieve competitive performance compared to supervised data-driven decision trees on tabular datasets. The decision trees constructed via this method provide transparent and interpretable models, addressing data scarcity while preserving interpretability. This work establishes a new baseline in low-data machine learning, offering a principled, knowledge-driven alternative to data-driven tree construction.

Title: Multi-Agent Geospatial Copilots for Remote Sensing Workflows

• Authors: Chaehong Lee, Varatheepan Paramanayakam, Andreas Karatzas, Yanan Jian, Michael Fore, Heming Liao, Fuxun Yu, Ruopu Li, Iraklis Anagnostopoulos, Dimitrios Stamoulis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.16254
• Pdf URL: https://arxiv.org/pdf/2501.16254
• Copy Paste: [[2501.16254]] Multi-Agent Geospatial Copilots for Remote Sensing Workflows(https://arxiv.org/abs/2501.16254)
• Keywords: protect, robust, large language model
• Abstract: We present GeoLLM-Squad, a geospatial Copilot that introduces the novel multi-agent paradigm to remote sensing (RS) workflows. Unlike existing single-agent approaches that rely on monolithic large language models (LLM), GeoLLM-Squad separates agentic orchestration from geospatial task-solving, by delegating RS tasks to specialized sub-agents. Built on the open-source AutoGen and GeoLLM-Engine frameworks, our work enables the modular integration of diverse applications, spanning urban monitoring, forestry protection, climate analysis, and agriculture studies. Our results demonstrate that while single-agent systems struggle to scale with increasing RS task complexity, GeoLLM-Squad maintains robust performance, achieving a 17% improvement in agentic correctness over state-of-the-art baselines. Our findings highlight the potential of multi-agent AI in advancing RS workflows.

Title: A foundation model for human-AI collaboration in medical literature mining

• Authors: Zifeng Wang, Lang Cao, Qiao Jin, Joey Chan, Nicholas Wan, Behdad Afzali, Hyun-Jin Cho, Chang-In Choi, Mehdi Emamverdi, Manjot K. Gill, Sun-Hyung Kim, Yijia Li, Yi Liu, Hanley Ong, Justin Rousseau, Irfan Sheikh, Jenny J. Wei, Ziyang Xu, Christopher M. Zallek, Kyungsang Kim, Yifan Peng, Zhiyong Lu, Jimeng Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16255
• Pdf URL: https://arxiv.org/pdf/2501.16255
• Copy Paste: [[2501.16255]] A foundation model for human-AI collaboration in medical literature mining(https://arxiv.org/abs/2501.16255)
• Keywords: extraction, large language model
• Abstract: Systematic literature review is essential for evidence-based medicine, requiring comprehensive analysis of clinical trial publications. However, the application of artificial intelligence (AI) models for medical literature mining has been limited by insufficient training and evaluation across broad therapeutic areas and diverse tasks. Here, we present LEADS, an AI foundation model for study search, screening, and data extraction from medical literature. The model is trained on 633,759 instruction data points in LEADSInstruct, curated from 21,335 systematic reviews, 453,625 clinical trial publications, and 27,015 clinical trial registries. We showed that LEADS demonstrates consistent improvements over four cutting-edge generic large language models (LLMs) on six tasks. Furthermore, LEADS enhances expert workflows by providing supportive references following expert requests, streamlining processes while maintaining high-quality results. A study with 16 clinicians and medical researchers from 14 different institutions revealed that experts collaborating with LEADS achieved a recall of 0.81 compared to 0.77 experts working alone in study selection, with a time savings of 22.6%. In data extraction tasks, experts using LEADS achieved an accuracy of 0.85 versus 0.80 without using LEADS, alongside a 26.9% time savings. These findings highlight the potential of specialized medical literature foundation models to outperform generic models, delivering significant quality and efficiency benefits when integrated into expert workflows for medical literature mining.

Title: URAG: Implementing a Unified Hybrid RAG for Precise Answers in University Admission Chatbots -- A Case Study at HCMUT

• Authors: Long Nguyen, Tho Quan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16276
• Pdf URL: https://arxiv.org/pdf/2501.16276
• Copy Paste: [[2501.16276]] URAG: Implementing a Unified Hybrid RAG for Precise Answers in University Admission Chatbots -- A Case Study at HCMUT(https://arxiv.org/abs/2501.16276)
• Keywords: large language model
• Abstract: With the rapid advancement of Artificial Intelligence, particularly in Natural Language Processing, Large Language Models (LLMs) have become pivotal in educational question-answering systems, especially university admission chatbots. Concepts such as Retrieval-Augmented Generation (RAG) and other advanced techniques have been developed to enhance these systems by integrating specific university data, enabling LLMs to provide informed responses on admissions and academic counseling. However, these enhanced RAG techniques often involve high operational costs and require the training of complex, specialized modules, which poses challenges for practical deployment. Additionally, in the educational context, it is crucial to provide accurate answers to prevent misinformation, a task that LLM-based systems find challenging without appropriate strategies and methods. In this paper, we introduce the Unified RAG (URAG) Framework, a hybrid approach that significantly improves the accuracy of responses, particularly for critical queries. Experimental results demonstrate that URAG enhances our in-house, lightweight model to perform comparably to state-of-the-art commercial models. Moreover, to validate its practical applicability, we conducted a case study at our educational institution, which received positive feedback and acclaim. This study not only proves the effectiveness of URAG but also highlights its feasibility for real-world implementation in educational settings.

Title: Multi-view Structural Convolution Network for Domain-Invariant Point Cloud Recognition of Autonomous Vehicles

• Authors: Younggun Kim, Beomsik Cho, Seonghoon Ryoo, Soomok Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16289
• Pdf URL: https://arxiv.org/pdf/2501.16289
• Copy Paste: [[2501.16289]] Multi-view Structural Convolution Network for Domain-Invariant Point Cloud Recognition of Autonomous Vehicles(https://arxiv.org/abs/2501.16289)
• Keywords: robust
• Abstract: Point cloud representation has recently become a research hotspot in the field of computer vision and has been utilized for autonomous vehicles. However, adapting deep learning networks for point cloud data recognition is challenging due to the variability in datasets and sensor technologies. This variability underscores the necessity for adaptive techniques to maintain accuracy under different conditions. In this paper, we present the Multi-View Structural Convolution Network (MSCN) designed for domain-invariant point cloud recognition. MSCN comprises Structural Convolution Layers (SCL) that extract local context geometric features from point clouds and Structural Aggregation Layers (SAL) that extract and aggregate both local and overall context features from point clouds. Additionally, our MSCN enhances feature representation robustness by training with unseen domain point clouds derived from source domain point clouds. This method acquires domain-invariant features and exhibits robust, consistent performance across various point cloud datasets, ensuring compatibility with diverse sensor configurations without the need for parameter adjustments. This highlights MSCN's potential to significantly improve the reliability and domain invariant features in different environments. Our code is available at this https URL.

Title: Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity

• Authors: Weixin Liang, Junhong Shen, Genghan Zhang, Ning Dong, Luke Zettlemoyer, Lili Yu
• Subjects: cs.LG, cs.AI, cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16295
• Pdf URL: https://arxiv.org/pdf/2501.16295
• Copy Paste: [[2501.16295]] Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity(https://arxiv.org/abs/2501.16295)
• Keywords: diffusion, transformer
• Abstract: State Space Models (SSMs) have emerged as efficient alternatives to Transformers for sequential modeling, but their inability to leverage modality-specific features limits their performance in multi-modal pretraining. Here, we propose Mixture-of-Mamba, a novel SSM architecture that introduces modality-aware sparsity through modality-specific parameterization of the Mamba block. Building on Mixture-of-Transformers (W. Liang et al. arXiv:2411.04996; 2024), we extend the benefits of modality-aware sparsity to SSMs while preserving their computational efficiency. We evaluate Mixture-of-Mamba across three multi-modal pretraining settings: Transfusion (interleaved text and continuous image tokens with diffusion loss), Chameleon (interleaved text and discrete image tokens), and an extended three-modality framework incorporating speech. Mixture-of-Mamba consistently reaches the same loss values at earlier training steps with significantly reduced computational costs. In the Transfusion setting, Mixture-of-Mamba achieves equivalent image loss using only 34.76% of the training FLOPs at the 1.4B scale. In the Chameleon setting, Mixture-of-Mamba reaches similar image loss with just 42.50% of the FLOPs at the 1.4B scale, and similar text loss with just 65.40% of the FLOPs. In the three-modality setting, MoM matches speech loss at 24.80% of the FLOPs at the 1.4B scale. Our ablation study highlights the synergistic effects of decoupling projection components, where joint decoupling yields greater gains than individual modifications. These results establish modality-aware sparsity as a versatile and effective design principle, extending its impact from Transformers to SSMs and setting new benchmarks in multi-modal pretraining. Our code can be accessed at this https URL

Title: FALCON: Resolving Visual Redundancy and Fragmentation in High-resolution Multimodal Large Language Models via Visual Registers

• Authors: Renshan Zhang, Rui Shao, Gongwei Chen, Kaiwen Zhou, Weili Guan, Liqiang Nie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.16297
• Pdf URL: https://arxiv.org/pdf/2501.16297
• Copy Paste: [[2501.16297]] FALCON: Resolving Visual Redundancy and Fragmentation in High-resolution Multimodal Large Language Models via Visual Registers(https://arxiv.org/abs/2501.16297)
• Keywords: large language model
• Abstract: The incorporation of high-resolution visual input equips multimodal large language models (MLLMs) with enhanced visual perception capabilities for real-world tasks. However, most existing high-resolution MLLMs rely on a cropping-based approach to process images, which leads to fragmented visual encoding and a sharp increase in redundant tokens. To tackle these issues, we propose the FALCON model. FALCON introduces a novel visual register technique to simultaneously: 1) Eliminate redundant tokens at the stage of visual encoding. To directly address the visual redundancy present in the output of vision encoder, we propose a Register-based Representation Compacting (ReCompact) mechanism. This mechanism introduces a set of learnable visual registers designed to adaptively aggregate essential information while discarding redundancy. It enables the encoder to produce a more compact visual representation with a minimal number of output tokens, thus eliminating the need for an additional compression module. 2) Ensure continuity in visual encoding. To address the potential encoding errors caused by fragmented visual inputs, we develop a Register Interactive Attention (ReAtten) module. This module facilitates effective and efficient information exchange across sub-images by enabling interactions between visual registers. It ensures the continuity of visual semantics throughout the encoding. We conduct comprehensive experiments with FALCON on high-resolution benchmarks across a wide range of scenarios. FALCON demonstrates superior performance with a remarkable 9-fold and 16-fold reduction in visual tokens.

Title: Large Models in Dialogue for Active Perception and Anomaly Detection

• Authors: Tzoulio Chamiti, Nikolaos Passalis, Anastasios Tefas
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16300
• Pdf URL: https://arxiv.org/pdf/2501.16300
• Copy Paste: [[2501.16300]] Large Models in Dialogue for Active Perception and Anomaly Detection(https://arxiv.org/abs/2501.16300)
• Keywords: large language model
• Abstract: Autonomous aerial monitoring is an important task aimed at gathering information from areas that may not be easily accessible by humans. At the same time, this task often requires recognizing anomalies from a significant distance or not previously encountered in the past. In this paper, we propose a novel framework that leverages the advanced capabilities provided by Large Language Models (LLMs) to actively collect information and perform anomaly detection in novel scenes. To this end, we propose an LLM based model dialogue approach, in which two deep learning models engage in a dialogue to actively control a drone to increase perception and anomaly detection accuracy. We conduct our experiments in a high fidelity simulation environment where an LLM is provided with a predetermined set of natural language movement commands mapped into executable code functions. Additionally, we deploy a multimodal Visual Question Answering (VQA) model charged with the task of visual question answering and captioning. By engaging the two models in conversation, the LLM asks exploratory questions while simultaneously flying a drone into different parts of the scene, providing a novel way to implement active perception. By leveraging LLMs reasoning ability, we output an improved detailed description of the scene going beyond existing static perception approaches. In addition to information gathering, our approach is utilized for anomaly detection and our results demonstrate the proposed methods effectiveness in informing and alerting about potential hazards.

Title: Matryoshka Re-Ranker: A Flexible Re-Ranking Architecture With Configurable Depth and Width

• Authors: Zheng Liu, Chaofan Li, Shitao Xiao, Chaozhuo Li, Defu Lian, Yingxia Shao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16302
• Pdf URL: https://arxiv.org/pdf/2501.16302
• Copy Paste: [[2501.16302]] Matryoshka Re-Ranker: A Flexible Re-Ranking Architecture With Configurable Depth and Width(https://arxiv.org/abs/2501.16302)
• Keywords: large language model
• Abstract: Large language models (LLMs) provide powerful foundations to perform fine-grained text re-ranking. However, they are often prohibitive in reality due to constraints on computation bandwidth. In this work, we propose a \textbf{flexible} architecture called \textbf{Matroyshka Re-Ranker}, which is designed to facilitate \textbf{runtime customization} of model layers and sequence lengths at each layer based on users' configurations. Consequently, the LLM-based re-rankers can be made applicable across various real-world situations. The increased flexibility may come at the cost of precision loss. To address this problem, we introduce a suite of techniques to optimize the performance. First, we propose \textbf{cascaded self-distillation}, where each sub-architecture learns to preserve a precise re-ranking performance from its super components, whose predictions can be exploited as smooth and informative teacher signals. Second, we design a \textbf{factorized compensation mechanism}, where two collaborative Low-Rank Adaptation modules, vertical and horizontal, are jointly employed to compensate for the precision loss resulted from arbitrary combinations of layer and sequence compression. We perform comprehensive experiments based on the passage and document retrieval datasets from MSMARCO, along with all public datasets from BEIR benchmark. In our experiments, Matryoshka Re-Ranker substantially outperforms the existing methods, while effectively preserving its superior performance across various forms of compression and different application scenarios.

Title: RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval

• Authors: Long Nguyen, Huy Nguyen, Bao Khuu, Huy Luu, Huy Le, Tuan Nguyen, Tho Quan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.16303
• Pdf URL: https://arxiv.org/pdf/2501.16303
• Copy Paste: [[2501.16303]] RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval(https://arxiv.org/abs/2501.16303)
• Keywords: robust, large language model
• Abstract: Retrieving events from videos using text queries has become increasingly challenging due to the rapid growth of multimedia content. Existing methods for text-based video event retrieval often focus heavily on object-level descriptions, overlooking the crucial role of contextual information. This limitation is especially apparent when queries lack sufficient context, such as missing location details or ambiguous background elements. To address these challenges, we propose a novel system called RAPID (Retrieval-Augmented Parallel Inference Drafting), which leverages advancements in Large Language Models (LLMs) and prompt-based learning to semantically correct and enrich user queries with relevant contextual information. These enriched queries are then processed through parallel retrieval, followed by an evaluation step to select the most relevant results based on their alignment with the original query. Through extensive experiments on our custom-developed dataset, we demonstrate that RAPID significantly outperforms traditional retrieval methods, particularly for contextually incomplete queries. Our system was validated for both speed and accuracy through participation in the Ho Chi Minh City AI Challenge 2024, where it successfully retrieved events from over 300 hours of video. Further evaluation comparing RAPID with the baseline proposed by the competition organizers demonstrated its superior effectiveness, highlighting the strength and robustness of our approach.

Title: Tailored Forecasting from Short Time Series via Meta-learning

• Authors: Declan A. Norton, Edward Ott, Andrew Pomerance, Brian Hunt, Michelle Girvan
• Subjects: cs.LG, nlin.CD, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2501.16325
• Pdf URL: https://arxiv.org/pdf/2501.16325
• Copy Paste: [[2501.16325]] Tailored Forecasting from Short Time Series via Meta-learning(https://arxiv.org/abs/2501.16325)
• Keywords: robust
• Abstract: Machine learning (ML) models can be effective for forecasting the dynamics of unknown systems from time-series data, but they often require large amounts of data and struggle to generalize across systems with varying dynamics. Combined, these issues make forecasting from short time series particularly challenging. To address this problem, we introduce Meta-learning for Tailored Forecasting from Related Time Series (METAFORS), which uses related systems with longer time-series data to supplement limited data from the system of interest. By leveraging a library of models trained on related systems, METAFORS builds tailored models to forecast system evolution with limited data. Using a reservoir computing implementation and testing on simulated chaotic systems, we demonstrate METAFORS' ability to predict both short-term dynamics and long-term statistics, even when test and related systems exhibit significantly different behaviors and the available data are scarce, highlighting its robustness and versatility in data-limited scenarios.

Title: sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging

• Authors: Jingyuan Chen, Yuan Yao, Mie Anderson, Natalie Hauglund, Celia Kjaerby, Verena Untiet, Maiken Nedergaard, Jiebo Luo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16329
• Pdf URL: https://arxiv.org/pdf/2501.16329
• Copy Paste: [[2501.16329]] sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging(https://arxiv.org/abs/2501.16329)
• Keywords: transformer
• Abstract: Automatic sleep staging based on electroencephalography (EEG) and electromyography (EMG) signals is an important aspect of sleep-related research. Current sleep staging methods suffer from two major drawbacks. First, there are limited information interactions between modalities in the existing methods. Second, current methods do not develop unified models that can handle different sources of input. To address these issues, we propose a novel sleep stage scoring model sDREAMER, which emphasizes cross-modality interaction and per-channel performance. Specifically, we develop a mixture-of-modality-expert (MoME) model with three pathways for EEG, EMG, and mixed signals with partially shared weights. We further propose a self-distillation training scheme for further information interaction across modalities. Our model is trained with multi-channel inputs and can make classifications on either single-channel or multi-channel inputs. Experiments demonstrate that our model outperforms the existing transformer-based sleep scoring methods for multi-channel inference. For single-channel inference, our model also outperforms the transformer-based models trained with single-channel signals.

Title: RelightVid: Temporal-Consistent Diffusion Model for Video Relighting

• Authors: Ye Fang, Zeyi Sun, Shangzhan Zhang, Tong Wu, Yinghao Xu, Pan Zhang, Jiaqi Wang, Gordon Wetzstein, Dahua Lin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.16330
• Pdf URL: https://arxiv.org/pdf/2501.16330
• Copy Paste: [[2501.16330]] RelightVid: Temporal-Consistent Diffusion Model for Video Relighting(https://arxiv.org/abs/2501.16330)
• Keywords: diffusion
• Abstract: Diffusion models have demonstrated remarkable success in image generation and editing, with recent advancements enabling albedo-preserving image relighting. However, applying these models to video relighting remains challenging due to the lack of paired video relighting datasets and the high demands for output fidelity and temporal consistency, further complicated by the inherent randomness of diffusion models. To address these challenges, we introduce RelightVid, a flexible framework for video relighting that can accept background video, text prompts, or environment maps as relighting conditions. Trained on in-the-wild videos with carefully designed illumination augmentations and rendered videos under extreme dynamic lighting, RelightVid achieves arbitrary video relighting with high temporal consistency without intrinsic decomposition while preserving the illumination priors of its image backbone.