2025-01-31

Title: Explainable Machine Learning: An Illustration of Kolmogorov-Arnold Network Model for Airfoil Lift Prediction

• Authors: Sudhanva Kulkarni
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.17896
• Pdf URL: https://arxiv.org/pdf/2501.17896
• Copy Paste: [[2501.17896]] Explainable Machine Learning: An Illustration of Kolmogorov-Arnold Network Model for Airfoil Lift Prediction(https://arxiv.org/abs/2501.17896)
• Keywords: explainability
• Abstract: Data science has emerged as fourth paradigm of scientific exploration. However many machine learning models operate as black boxes offering limited insight into the reasoning behind their predictions. This lack of transparency is one of the drawbacks to generate new knowledge from data. Recently Kolmogorov-Arnold Network or KAN has been proposed as an alternative model which embeds explainable AI. This study demonstrates the potential of KAN for new scientific exploration. KAN along with five other popular supervised machine learning models are applied to the well-known problem of airfoil lift prediction in aerospace engineering. Standard data generated from an earlier study on 2900 different airfoils is used. KAN performed the best with an R2 score of 96.17 percent on the test data, surpassing both the baseline model and Multi Layer Perceptron. Explainability of KAN is shown by pruning and symbolizing the model resulting in an equation for coefficient of lift in terms of input variables. The explainable information retrieved from KAN model is found to be consistent with the known physics of lift generation by airfoil thus demonstrating its potential to aid in scientific exploration.

Title: Shared DIFF Transformer

• Authors: Yueyang Cang, Yuhang Liu, Xiaoteng Zhang, Xiangju Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.17900
• Pdf URL: https://arxiv.org/pdf/2501.17900
• Copy Paste: [[2501.17900]] Shared DIFF Transformer(https://arxiv.org/abs/2501.17900)
• Keywords: robust, transformer
• Abstract: DIFF Transformer improves attention allocation by enhancing focus on relevant context while suppressing noise. It introduces a differential attention mechanism that calculates the difference between two independently generated attention distributions, effectively reducing noise and promoting sparse attention patterns. However, the independent signal generation in DIFF Transformer results in parameter redundancy and suboptimal utilization of information. In this work, we propose Shared DIFF Transformer, which draws on the idea of a differential amplifier by introducing a shared base matrix to model global patterns and incorporating low-rank updates to enhance task-specific flexibility. This design significantly reduces parameter redundancy, improves efficiency, and retains strong noise suppression capabilities. Experimental results show that, compared to DIFF Transformer, our method achieves better performance in tasks such as long-sequence modeling, key information retrieval, and in-context learning. Our work provides a novel and efficient approach to optimizing differential attention mechanisms and advancing robust Transformer architectures.

Title: DReSS: Data-driven Regularized Structured Streamlining for Large Language Models

• Authors: Mingkuan Feng, Jinyang Wu, Shuai Zhang, Pengpeng Shao, Ruihan Jin, Zhengqi Wen, Jianhua Tao, Feihu Che
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.17905
• Pdf URL: https://arxiv.org/pdf/2501.17905
• Copy Paste: [[2501.17905]] DReSS: Data-driven Regularized Structured Streamlining for Large Language Models(https://arxiv.org/abs/2501.17905)
• Keywords: large language model
• Abstract: Large language models (LLMs) have achieved significant progress across various domains, but their increasing scale results in high computational and memory costs. Recent studies have revealed that LLMs exhibit sparsity, providing the potential to reduce model size through pruning techniques. However, existing pruning methods typically follow a prune-then-finetune paradigm. Since the pruned components still contain valuable information, their direct removal often leads to irreversible performance degradation, imposing a substantial computational burden to recover performance during finetuning. In this paper, we propose a novel paradigm that first applies regularization, then prunes, and finally finetunes. Based on this paradigm, we introduce DReSS, a simple and effective Data-driven Regularized Structured Streamlining method for LLMs. By leveraging a small amount of data to regularize the components to be pruned, DReSS explicitly transfers the important information to the remaining parts of the model in advance. Compared to direct pruning, this can reduce the information loss caused by parameter removal, thereby enhancing its language modeling capabilities. Experimental results demonstrate that DReSS significantly outperforms existing pruning methods even under extreme pruning ratios, significantly reducing latency and increasing throughput.

Title: TransRAD: Retentive Vision Transformer for Enhanced Radar Object Detection

• Authors: Lei Cheng, Siyang Cao
• Subjects: cs.CV, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.17977
• Pdf URL: https://arxiv.org/pdf/2501.17977
• Copy Paste: [[2501.17977]] TransRAD: Retentive Vision Transformer for Enhanced Radar Object Detection(https://arxiv.org/abs/2501.17977)
• Keywords: transformer
• Abstract: Despite significant advancements in environment perception capabilities for autonomous driving and intelligent robotics, cameras and LiDARs remain notoriously unreliable in low-light conditions and adverse weather, which limits their effectiveness. Radar serves as a reliable and low-cost sensor that can effectively complement these limitations. However, radar-based object detection has been underexplored due to the inherent weaknesses of radar data, such as low resolution, high noise, and lack of visual information. In this paper, we present TransRAD, a novel 3D radar object detection model designed to address these challenges by leveraging the Retentive Vision Transformer (RMT) to more effectively learn features from information-dense radar Range-Azimuth-Doppler (RAD) data. Our approach leverages the Retentive Manhattan Self-Attention (MaSA) mechanism provided by RMT to incorporate explicit spatial priors, thereby enabling more accurate alignment with the spatial saliency characteristics of radar targets in RAD data and achieving precise 3D radar detection across Range-Azimuth-Doppler dimensions. Furthermore, we propose Location-Aware NMS to effectively mitigate the common issue of duplicate bounding boxes in deep radar object detection. The experimental results demonstrate that TransRAD outperforms state-of-the-art methods in both 2D and 3D radar detection tasks, achieving higher accuracy, faster inference speed, and reduced computational complexity. Code is available at this https URL

Title: Efficient Feature Fusion for UAV Object Detection

• Authors: Xudong Wang, Chaomin Shen, Yaxin Peng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.17983
• Pdf URL: https://arxiv.org/pdf/2501.17983
• Copy Paste: [[2501.17983]] Efficient Feature Fusion for UAV Object Detection(https://arxiv.org/abs/2501.17983)
• Keywords: robust
• Abstract: Object detection in unmanned aerial vehicle (UAV) remote sensing images poses significant challenges due to unstable image quality, small object sizes, complex backgrounds, and environmental occlusions. Small objects, in particular, occupy minimal portions of images, making their accurate detection highly difficult. Existing multi-scale feature fusion methods address these challenges to some extent by aggregating features across different resolutions. However, these methods often fail to effectively balance classification and localization performance for small objects, primarily due to insufficient feature representation and imbalanced network information flow. In this paper, we propose a novel feature fusion framework specifically designed for UAV object detection tasks to enhance both localization accuracy and classification performance. The proposed framework integrates hybrid upsampling and downsampling modules, enabling feature maps from different network depths to be flexibly adjusted to arbitrary resolutions. This design facilitates cross-layer connections and multi-scale feature fusion, ensuring improved representation of small objects. Our approach leverages hybrid downsampling to enhance fine-grained feature representation, improving spatial localization of small targets, even under complex conditions. Simultaneously, the upsampling module aggregates global contextual information, optimizing feature consistency across scales and enhancing classification robustness in cluttered scenes. Experimental results on two public UAV datasets demonstrate the effectiveness of the proposed framework. Integrated into the YOLO-V10 model, our method achieves a 2\% improvement in average precision (AP) compared to the baseline YOLO-V10 model, while maintaining the same number of parameters. These results highlight the potential of our framework for accurate and efficient UAV object detection.

Title: Pressure Field Reconstruction with SIREN: A Mesh-Free Approach for Image Velocimetry in Complex Noisy Environments

• Authors: Renato F. Miotto, William R. Wolf, Fernando Zigunov
• Subjects: cs.CV, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2501.17987
• Pdf URL: https://arxiv.org/pdf/2501.17987
• Copy Paste: [[2501.17987]] Pressure Field Reconstruction with SIREN: A Mesh-Free Approach for Image Velocimetry in Complex Noisy Environments(https://arxiv.org/abs/2501.17987)
• Keywords: robust
• Abstract: This work presents a novel approach for pressure field reconstruction from image velocimetry data using SIREN (Sinusoidal Representation Network), emphasizing its effectiveness as an implicit neural representation in noisy environments and its mesh-free nature. While we briefly assess two recently proposed methods - one-shot matrix-omnidirectional integration (OS-MODI) and Green's function integral (GFI) - the primary focus is on the advantages of the SIREN approach. The OS-MODI technique performs well in noise-free conditions and with structured meshes but struggles when applied to unstructured meshes with high aspect ratio. Similarly, the GFI method encounters difficulties due to singularities inherent from the Newtonian kernel. In contrast, the proposed SIREN approach is a mesh-free method that directly reconstructs the pressure field, bypassing the need for an intrinsic grid connectivity and, hence, avoiding the challenges associated with ill-conditioned cells and unstructured meshes. This provides a distinct advantage over traditional mesh-based methods. Moreover, it is shown that changes in the architecture of the SIREN can be used to filter out inherent noise from velocimetry data. This work positions SIREN as a robust and versatile solution for pressure reconstruction, particularly in noisy environments characterized by the absence of mesh structure, opening new avenues for innovative applications in this field.

Title: InnerThoughts: Disentangling Representations and Predictions in Large Language Models

• Authors: Didier Chételat, Joseph Cotnareanu, Rylee Thompson, Yingxue Zhang, Mark Coates
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.17994
• Pdf URL: https://arxiv.org/pdf/2501.17994
• Copy Paste: [[2501.17994]] InnerThoughts: Disentangling Representations and Predictions in Large Language Models(https://arxiv.org/abs/2501.17994)
• Keywords: transformer, large language model
• Abstract: Large language models (LLMs) contain substantial factual knowledge which is commonly elicited by multiple-choice question-answering prompts. Internally, such models process the prompt through multiple transformer layers, building varying representations of the problem within its hidden states. Ultimately, however, only the hidden state corresponding to the final layer and token position are used to predict the answer label. In this work, we propose instead to learn a small separate neural network predictor module on a collection of training questions, that take the hidden states from all the layers at the last temporal position as input and outputs predictions. In effect, such a framework disentangles the representational abilities of LLMs from their predictive abilities. On a collection of hard benchmarks, our method achieves considerable improvements in performance, sometimes comparable to supervised fine-tuning procedures, but at a fraction of the computational cost.

Title: Topological Signatures of Adversaries in Multimodal Alignments

• Authors: Minh Vu, Geigh Zollicoffer, Huy Mai, Ben Nebgen, Boian Alexandrov, Manish Bhattarai
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18006
• Pdf URL: https://arxiv.org/pdf/2501.18006
• Copy Paste: [[2501.18006]] Topological Signatures of Adversaries in Multimodal Alignments(https://arxiv.org/abs/2501.18006)
• Keywords: defense, attack, robust
• Abstract: Multimodal Machine Learning systems, particularly those aligning text and image data like CLIP/BLIP models, have become increasingly prevalent, yet remain susceptible to adversarial attacks. While substantial research has addressed adversarial robustness in unimodal contexts, defense strategies for multimodal systems are underexplored. This work investigates the topological signatures that arise between image and text embeddings and shows how adversarial attacks disrupt their alignment, introducing distinctive signatures. We specifically leverage persistent homology and introduce two novel Topological-Contrastive losses based on Total Persistence and Multi-scale kernel methods to analyze the topological signatures introduced by adversarial perturbations. We observe a pattern of monotonic changes in the proposed topological losses emerging in a wide range of attacks on image-text alignments, as more adversarial samples are introduced in the data. By designing an algorithm to back-propagate these signatures to input samples, we are able to integrate these signatures into Maximum Mean Discrepancy tests, creating a novel class of tests that leverage topological signatures for better adversarial detection.

Title: A Proximal Operator for Inducing 2:4-Sparsity

• Authors: Jonas M Kübler, Yu-Xiang Wang, Shoham Sabach, Navid Ansari, Matthäus Kleindessner, Kailash Budhathoki, Volkan Cevher, George Karypis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18015
• Pdf URL: https://arxiv.org/pdf/2501.18015
• Copy Paste: [[2501.18015]] A Proximal Operator for Inducing 2:4-Sparsity(https://arxiv.org/abs/2501.18015)
• Keywords: large language model
• Abstract: Recent hardware advancements in AI Accelerators and GPUs allow to efficiently compute sparse matrix multiplications, especially when 2 out of 4 consecutive weights are set to zero. However, this so-called 2:4 sparsity usually comes at a decreased accuracy of the model. We derive a regularizer that exploits the local correlation of features to find better sparsity masks in trained models. We minimize the regularizer jointly with a local squared loss by deriving the proximal operator for which we show that it has an efficient solution in the 2:4-sparse case. After optimizing the mask, we use maskedgradient updates to further minimize the local squared loss. We illustrate our method on toy problems and apply it to pruning entire large language models up to 70B parameters. On models up to 13B we improve over previous state of the art algorithms, whilst on 70B models we match their performance.

Title: KNN and K-means in Gini Prametric Spaces

• Authors: Cassandra Mussard, Arthur Charpentier, Stéphane Mussard
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18028
• Pdf URL: https://arxiv.org/pdf/2501.18028
• Copy Paste: [[2501.18028]] KNN and K-means in Gini Prametric Spaces(https://arxiv.org/abs/2501.18028)
• Keywords: robust
• Abstract: This paper introduces innovative enhancements to the K-means and K-nearest neighbors (KNN) algorithms based on the concept of Gini prametric spaces. Unlike traditional distance metrics, Gini-based measures incorporate both value-based and rank-based information, improving robustness to noise and outliers. The main contributions of this work include: proposing a Gini-based measure that captures both rank information and value distances; presenting a Gini K-means algorithm that is proven to converge and demonstrates resilience to noisy data; and introducing a Gini KNN method that performs competitively with state-of-the-art approaches such as Hassanat's distance in noisy environments. Experimental evaluations on 14 datasets from the UCI repository demonstrate the superior performance and efficiency of Gini-based algorithms in clustering and classification tasks. This work opens new avenues for leveraging rank-based measures in machine learning and statistical analysis.

Title: Generative AI for Vision: A Comprehensive Study of Frameworks and Applications

• Authors: Fouad Bousetouane
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18033
• Pdf URL: https://arxiv.org/pdf/2501.18033
• Copy Paste: [[2501.18033]] Generative AI for Vision: A Comprehensive Study of Frameworks and Applications(https://arxiv.org/abs/2501.18033)
• Keywords: diffusion, generative
• Abstract: Generative AI is transforming image synthesis, enabling the creation of high-quality, diverse, and photorealistic visuals across industries like design, media, healthcare, and autonomous systems. Advances in techniques such as image-to-image translation, text-to-image generation, domain transfer, and multimodal alignment have broadened the scope of automated visual content creation, supporting a wide spectrum of applications. These advancements are driven by models like Generative Adversarial Networks (GANs), conditional frameworks, and diffusion-based approaches such as Stable Diffusion. This work presents a structured classification of image generation techniques based on the nature of the input, organizing methods by input modalities like noisy vectors, latent representations, and conditional inputs. We explore the principles behind these models, highlight key frameworks including DALL-E, ControlNet, and DeepSeek Janus-Pro, and address challenges such as computational costs, data biases, and output alignment with user intent. By offering this input-centric perspective, this study bridges technical depth with practical insights, providing researchers and practitioners with a comprehensive resource to harness generative AI for real-world applications.

Title: SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders

• Authors: Bartosz Cywiński, Kamil Deja
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18052
• Pdf URL: https://arxiv.org/pdf/2501.18052
• Copy Paste: [[2501.18052]] SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders(https://arxiv.org/abs/2501.18052)
• Keywords: attack, diffusion
• Abstract: Recent machine unlearning approaches offer promising solution for removing unwanted concepts from diffusion models. However, traditional methods, which largely rely on fine-tuning, provide little insight into the changes they introduce to the base model, making it unclear whether concepts are truly removed or only masked. In this work, we introduce SAeUron, a novel method leveraging features learned by sparse autoencoders (SAEs) to unlearn unwanted concepts in text-to-image diffusion models. First, we demonstrate that SAEs, trained in an unsupervised manner on activations from multiple denoising timesteps of the diffusion model, capture sparse and interpretable features corresponding to specific concepts. Building on this, we propose a method of selecting concept-specific features. This enables precise interventions on the model's activations to block targeted content while preserving the model's overall performance. Evaluation on the competitive UnlearnCanvas benchmark on object and style unlearning highlights SAeUron's state-of-the-art performance. Moreover, we show that with a single SAE, we can remove multiple concepts simultaneously and that in contrast to other methods, SAeUron dismisses the possibility of generating unwanted content, even under adversarial attack.

Title: Current Pathology Foundation Models are unrobust to Medical Center Differences

• Authors: Edwin D. de Jong, Eric Marcus, Jonas Teuwen
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18055
• Pdf URL: https://arxiv.org/pdf/2501.18055
• Copy Paste: [[2501.18055]] Current Pathology Foundation Models are unrobust to Medical Center Differences(https://arxiv.org/abs/2501.18055)
• Keywords: robust
• Abstract: Pathology Foundation Models (FMs) hold great promise for healthcare. Before they can be used in clinical practice, it is essential to ensure they are robust to variations between medical centers. We measure whether pathology FMs focus on biological features like tissue and cancer type, or on the well known confounding medical center signatures introduced by staining procedure and other differences. We introduce the Robustness Index. This novel robustness metric reflects to what degree biological features dominate confounding features. Ten current publicly available pathology FMs are evaluated. We find that all current pathology foundation models evaluated represent the medical center to a strong degree. Significant differences in the robustness index are observed. Only one model so far has a robustness index greater than one, meaning biological features dominate confounding features, but only slightly. A quantitative approach to measure the influence of medical center differences on FM-based prediction performance is described. We analyze the impact of unrobustness on classification performance of downstream models, and find that cancer-type classification errors are not random, but specifically attributable to same-center confounders: images of other classes from the same medical center. We visualize FM embedding spaces, and find these are more strongly organized by medical centers than by biological factors. As a consequence, the medical center of origin is predicted more accurately than the tissue source and cancer type. The robustness index introduced here is provided with the aim of advancing progress towards clinical adoption of robust and reliable pathology FMs.

Title: FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models

• Authors: Spencer Mateega, Carlos Georgescu, Danny Tang
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18062
• Pdf URL: https://arxiv.org/pdf/2501.18062
• Copy Paste: [[2501.18062]] FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models(https://arxiv.org/abs/2501.18062)
• Keywords: large language model
• Abstract: FinanceQA is a testing suite that evaluates LLMs' performance on complex numerical financial analysis tasks that mirror real-world investment work. Despite recent advances, current LLMs fail to meet the strict accuracy requirements of financial institutions, with models failing approximately 60% of realistic tasks that mimic on-the-job analyses at hedge funds, private equity firms, investment banks, and other financial institutions. The primary challenges include hand-spreading metrics, adhering to standard accounting and corporate valuation conventions, and performing analysis under incomplete information - particularly in multi-step tasks requiring assumption generation. This performance gap highlights the disconnect between existing LLM capabilities and the demands of professional financial analysis that are inadequately tested by current testing architectures. Results show that higher-quality training data is needed to support such tasks, which we experiment with using OpenAI's fine-tuning API. FinanceQA is publicly released at [this https URL](this https URL).

Title: Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence

• Authors: Pir Bakhsh Khokhar, Viviana Pentangelo, Fabio Palomba, Carmine Gravino
• Subjects: cs.LG, cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.18071
• Pdf URL: https://arxiv.org/pdf/2501.18071
• Copy Paste: [[2501.18071]] Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence(https://arxiv.org/abs/2501.18071)
• Keywords: interpretability
• Abstract: Diabetes mellitus (DM) is a global health issue of significance that must be diagnosed as early as possible and managed well. This study presents a framework for diabetes prediction using Machine Learning (ML) models, complemented with eXplainable Artificial Intelligence (XAI) tools, to investigate both the predictive accuracy and interpretability of the predictions from ML models. Data Preprocessing is based on the Synthetic Minority Oversampling Technique (SMOTE) and feature scaling used on the Diabetes Binary Health Indicators dataset to deal with class imbalance and variability of clinical features. The ensemble model provided high accuracy, with a test accuracy of 92.50% and an ROC-AUC of 0.975. BMI, Age, General Health, Income, and Physical Activity were the most influential predictors obtained from the model explanations. The results of this study suggest that ML combined with XAI is a promising means of developing accurate and computationally transparent tools for use in healthcare systems.

Title: AlphaAdam:Asynchronous Masked Optimization with Dynamic Alpha for Selective Updates

• Authors: Da Chang, Yu Li, Ganzhao Yuan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18094
• Pdf URL: https://arxiv.org/pdf/2501.18094
• Copy Paste: [[2501.18094]] AlphaAdam:Asynchronous Masked Optimization with Dynamic Alpha for Selective Updates(https://arxiv.org/abs/2501.18094)
• Keywords: large language model
• Abstract: In the training of large language models (LLMs), updating parameters more efficiently and stably has always been an important challenge. To achieve efficient parameter updates, existing methods usually achieve performance comparable to full parameter updates through methods such as low-dimensional decomposition or layer-wise selective updates. In this work, we propose AlphaAdam, an optimization framework for LLM from the perspective of intra-layer parameter updates. By decoupling parameter updates and dynamically adjusting their strength, AlphaAdam accelerates convergence and improves training stability. We construct parameter masks based on the consistency of historical momentum and gradient direction and combine them with an adaptive mask strength strategy to ensure efficient optimization and theoretical convergence guarantees, which is also applicable to most momentum-based optimizers. Extensive experiments show that AlphaAdam outperforms state-of-the-art methods such as AdamW in terms of convergence speed and computational efficiency across tasks, including GPT-2 pre-trained and fine-tuned RoBERTa and Llama-7B. Our AlphaAdam implements an optimizer enhancement framework for LLMs through intra-layer asynchronous masked adaptive updates. Our code is available in this \href{this https URL}{link}

Title: Disentangling Safe and Unsafe Corruptions via Anisotropy and Locality

• Authors: Ramchandran Muthukumar, Ambar Pal, Jeremias Sulam, Rene Vidal
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18098
• Pdf URL: https://arxiv.org/pdf/2501.18098
• Copy Paste: [[2501.18098]] Disentangling Safe and Unsafe Corruptions via Anisotropy and Locality(https://arxiv.org/abs/2501.18098)
• Keywords: attack, robust
• Abstract: State-of-the-art machine learning systems are vulnerable to small perturbations to their input, where ``small'' is defined according to a threat model that assigns a positive threat to each perturbation. Most prior works define a task-agnostic, isotropic, and global threat, like the $\ell_p$ norm, where the magnitude of the perturbation fully determines the degree of the threat and neither the direction of the attack nor its position in space matter. However, common corruptions in computer vision, such as blur, compression, or occlusions, are not well captured by such threat models. This paper proposes a novel threat model called \texttt{Projected Displacement} (PD) to study robustness beyond existing isotropic and global threat models. The proposed threat model measures the threat of a perturbation via its alignment with \textit{unsafe directions}, defined as directions in the input space along which a perturbation of sufficient magnitude changes the ground truth class label. Unsafe directions are identified locally for each input based on observed training data. In this way, the PD threat model exhibits anisotropy and locality. Experiments on Imagenet-1k data indicate that, for any input, the set of perturbations with small PD threat includes \textit{safe} perturbations of large $\ell_p$ norm that preserve the true label, such as noise, blur and compression, while simultaneously excluding \textit{unsafe} perturbations that alter the true label. Unlike perceptual threat models based on embeddings of large-vision models, the PD threat model can be readily computed for arbitrary classification tasks without pre-training or finetuning. Further additional task annotation such as sensitivity to image regions or concept hierarchies can be easily integrated into the assessment of threat and thus the PD threat model presents practitioners with a flexible, task-driven threat specification.

Title: Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation

• Authors: Yibo Wang, Tiansheng Huang, Li Shen, Huanjin Yao, Haotian Luo, Rui Liu, Naiqiang Tan, Jiaxing Huang, Dacheng Tao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18100
• Pdf URL: https://arxiv.org/pdf/2501.18100
• Copy Paste: [[2501.18100]] Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation(https://arxiv.org/abs/2501.18100)
• Keywords: security, defense, attack, large language model
• Abstract: Harmful fine-tuning attack introduces significant security risks to the fine-tuning services. Mainstream defenses aim to vaccinate the model such that the later harmful fine-tuning attack is less effective. However, our evaluation results show that such defenses are fragile -- with a few fine-tuning steps, the model still can learn the harmful knowledge. To this end, we do further experiment and find that an embarrassingly simple solution -- adding purely random perturbations to the fine-tuned model, can recover the model from harmful behavior, though it leads to a degradation in the model's fine-tuning performance. To address the degradation of fine-tuning performance, we further propose Panacea, which optimizes an adaptive perturbation that will be applied to the model after fine-tuning. Panacea maintains model's safety alignment performance without compromising downstream fine-tuning performance. Comprehensive experiments are conducted on different harmful ratios, fine-tuning tasks and mainstream LLMs, where the average harmful scores are reduced by up-to 21.5%, while maintaining fine-tuning performance. As a by-product, we analyze the optimized perturbation and show that different layers in various LLMs have distinct safety coefficients. Source code available at this https URL

Title: Diverse Preference Optimization

• Authors: Jack Lanchantin, Angelica Chen, Shehzaad Dhuliawala, Ping Yu, Jason Weston, Sainbayar Sukhbaatar, Ilia Kulikov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18101
• Pdf URL: https://arxiv.org/pdf/2501.18101
• Copy Paste: [[2501.18101]] Diverse Preference Optimization(https://arxiv.org/abs/2501.18101)
• Keywords: generative
• Abstract: Post-training of language models, either through reinforcement learning, preference optimization or supervised finetuning, tends to sharpen the output probability distribution and reduce the diversity of generated responses. This is particularly a problem for creative generative tasks where varied responses are desired. %This impacts the ability to generate high quality synthetic data which is becoming a vital component of model training. In this work we introduce Diverse Preference Optimization (DivPO), an online optimization method which learns to generate much more diverse responses than standard pipelines, while maintaining the quality of the generations. In DivPO, preference pairs are selected by first considering a pool of responses, and a measure of diversity among them, and selecting chosen examples as being more rare but high quality, while rejected examples are more common, but low quality. DivPO results in generating 45.6% more diverse persona attributes, and an 74.6% increase in story diversity, while maintaining similar win rates as standard baselines.

Title: Security for IEEE P1451.1.6-based Sensor Networks for IoT Applications

• Authors: Hiroaki Nishi, Janaka Wijekoon, Eugene Y. Song, Kang B. Lee
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18102
• Pdf URL: https://arxiv.org/pdf/2501.18102
• Copy Paste: [[2501.18102]] Security for IEEE P1451.1.6-based Sensor Networks for IoT Applications(https://arxiv.org/abs/2501.18102)
• Keywords: security, privacy, robust
• Abstract: There are many challenges for Internet of Things (IoT) sensor networks including the lack of robust standards, diverse wireline and wireless connectivity, interoperability, security, and privacy. Addressing these challenges, the Institute of Electrical and Electronics Engineers (IEEE) P1451.0 standard defines network services, transducer services, transducer electronic data sheets (TEDS) format, and a security framework to achieve sensor data security and interoperability for IoT applications. This paper proposes a security solution for IEEE P1451.1.6-based sensor networks for IoT applications utilizing the security framework defined in IEEE P1451.0. The proposed solution includes an architecture, a security policy with six security levels, security standards, and security TEDS. Further, this paper introduces a new service to update access control lists (ACLs) to regulate the access for topic names by the applications and provides an implementation of the security TEDS for IEEE P1451.1.6-based sensor networks. The paper also illustrates how to access security TEDS that contain metadata on security standards to achieve sensor data security and interoperability.

Title: Scaling Inference-Efficient Language Models

• Authors: Song Bian, Minghao Yan, Shivaram Venkataraman
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18107
• Pdf URL: https://arxiv.org/pdf/2501.18107
• Copy Paste: [[2501.18107]] Scaling Inference-Efficient Language Models(https://arxiv.org/abs/2501.18107)
• Keywords: large language model
• Abstract: Scaling laws are powerful tools to predict the performance of large language models. However, current scaling laws fall short of accounting for inference costs. In this work, we first show that model architecture affects inference latency, where models of the same size can have up to 3.5x difference in latency. To tackle this challenge, we modify the Chinchilla scaling laws to co-optimize the model parameter count, the number of training tokens, and the model architecture. Due to the reason that models of similar training loss exhibit gaps in downstream evaluation, we also propose a novel method to train inference-efficient models based on the revised scaling laws. We perform extensive empirical studies to fit and evaluate our inference-aware scaling laws. We vary model parameters from 80M to 1B, training tokens from 1.6B to 30B, and model shapes, training a total of 63 models. Guided by our inference-efficient scaling law and model selection method, we release the Morph-1B model, which improves inference latency by 1.8x while maintaining accuracy on downstream tasks compared to open-source models, pushing the Pareto frontier of accuracy-latency tradeoff.

Title: ACTGNN: Assessment of Clustering Tendency with Synthetically-Trained Graph Neural Networks

• Authors: Yiran Luo, Evangelos E. Papalexakis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18112
• Pdf URL: https://arxiv.org/pdf/2501.18112
• Copy Paste: [[2501.18112]] ACTGNN: Assessment of Clustering Tendency with Synthetically-Trained Graph Neural Networks(https://arxiv.org/abs/2501.18112)
• Keywords: robust
• Abstract: Determining clustering tendency in datasets is a fundamental but challenging task, especially in noisy or high-dimensional settings where traditional methods, such as the Hopkins Statistic and Visual Assessment of Tendency (VAT), often struggle to produce reliable results. In this paper, we propose ACTGNN, a graph-based framework designed to assess clustering tendency by leveraging graph representations of data. Node features are constructed using Locality-Sensitive Hashing (LSH), which captures local neighborhood information, while edge features incorporate multiple similarity metrics, such as the Radial Basis Function (RBF) kernel, to model pairwise relationships. A Graph Neural Network (GNN) is trained exclusively on synthetic datasets, enabling robust learning of clustering structures under controlled conditions. Extensive experiments demonstrate that ACTGNN significantly outperforms baseline methods on both synthetic and real-world datasets, exhibiting superior performance in detecting faint clustering structures, even in high-dimensional or noisy data. Our results highlight the generalizability and effectiveness of the proposed approach, making it a promising tool for robust clustering tendency assessment.

Title: Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models

• Authors: Qika Lin, Tianzhe Zhao, Kai He, Zhen Peng, Fangzhi Xu, Ling Huang, Jingying Ma, Mengling Feng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18119
• Pdf URL: https://arxiv.org/pdf/2501.18119
• Copy Paste: [[2501.18119]] Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models(https://arxiv.org/abs/2501.18119)
• Keywords: large language model
• Abstract: Due to the presence of the natural gap between Knowledge Graph (KG) structures and the natural language, the effective integration of holistic structural information of KGs with Large Language Models (LLMs) has emerged as a significant question. To this end, we propose a two-stage framework to learn and apply quantized codes for each entity, aiming for the seamless integration of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR) method is proposed to compress both KG structural and semantic knowledge into discrete codes (\ie, tokens) that align the format of language sentences. We further design KG instruction-following data by viewing these learned codes as features to directly input to LLMs, thereby achieving seamless integration. The experiment results demonstrate that SSQR outperforms existing unsupervised quantized methods, producing more distinguishable codes. Further, the fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link prediction and triple classification tasks, utilizing only 16 tokens per entity instead of thousands in conventional prompting methods.

Title: Battery State of Health Estimation Using LLM Framework

• Authors: Aybars Yunusoglu, Dexter Le, Karn Tiwari, Murat Isik, I. Can Dikmen
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.18123
• Pdf URL: https://arxiv.org/pdf/2501.18123
• Copy Paste: [[2501.18123]] Battery State of Health Estimation Using LLM Framework(https://arxiv.org/abs/2501.18123)
• Keywords: transformer
• Abstract: Battery health monitoring is critical for the efficient and reliable operation of electric vehicles (EVs). This study introduces a transformer-based framework for estimating the State of Health (SoH) and predicting the Remaining Useful Life (RUL) of lithium titanate (LTO) battery cells by utilizing both cycle-based and instantaneous discharge data. Testing on eight LTO cells under various cycling conditions over 500 cycles, we demonstrate the impact of charge durations on energy storage trends and apply Differential Voltage Analysis (DVA) to monitor capacity changes (dQ/dV) across voltage ranges. Our LLM model achieves superior performance, with a Mean Absolute Error (MAE) as low as 0.87\% and varied latency metrics that support efficient processing, demonstrating its strong potential for real-time integration into EVs. The framework effectively identifies early signs of degradation through anomaly detection in high-resolution data, facilitating predictive maintenance to prevent sudden battery failures and enhance energy efficiency.

Title: Unraveling the Capabilities of Language Models in News Summarization

• Authors: Abdurrahman Odabaşı, Göksel Biricik
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18128
• Pdf URL: https://arxiv.org/pdf/2501.18128
• Copy Paste: [[2501.18128]] Unraveling the Capabilities of Language Models in News Summarization(https://arxiv.org/abs/2501.18128)
• Keywords: robust
• Abstract: Given the recent introduction of multiple language models and the ongoing demand for improved Natural Language Processing tasks, particularly summarization, this work provides a comprehensive benchmarking of 20 recent language models, focusing on smaller ones for the news summarization task. In this work, we systematically test the capabilities and effectiveness of these models in summarizing news article texts which are written in different styles and presented in three distinct datasets. Specifically, we focus in this study on zero-shot and few-shot learning settings and we apply a robust evaluation methodology that combines different evaluation concepts including automatic metrics, human evaluation, and LLM-as-a-judge. Interestingly, including demonstration examples in the few-shot learning setting did not enhance models' performance and, in some cases, even led to worse quality of the generated summaries. This issue arises mainly due to the poor quality of the gold summaries that have been used as reference summaries, which negatively impacts the models' performance. Furthermore, our study's results highlight the exceptional performance of GPT-3.5-Turbo and GPT-4, which generally dominate due to their advanced capabilities. However, among the public models evaluated, certain models such as Qwen1.5-7B, SOLAR-10.7B-Instruct-v1.0, Meta-Llama-3-8B and Zephyr-7B-Beta demonstrated promising results. These models showed significant potential, positioning them as competitive alternatives to large models for the task of news summarization.

Title: Entropy-Synchronized Neural Hashing for Unsupervised Ransomware Detection

• Authors: Peter Idliman, Wilfred Balfour, Benedict Featheringham, Hugo Chesterfield
• Subjects: cs.CR, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18131
• Pdf URL: https://arxiv.org/pdf/2501.18131
• Copy Paste: [[2501.18131]] Entropy-Synchronized Neural Hashing for Unsupervised Ransomware Detection(https://arxiv.org/abs/2501.18131)
• Keywords: robust
• Abstract: Entropy-based detection methodologies have gained significant attention due to their ability to analyze structural irregularities within executable files, particularly in the identification of malicious software employing advanced obfuscation techniques. The Entropy-Synchronized Neural Hashing (ESNH) framework introduces a novel approach that leverages entropy-driven hash representations to classify software binaries based on their underlying entropy characteristics. Through the synchronization of entropy profiles with neural network architectures, the model generates robust and unique hash values that maintain stability even when faced with polymorphic and metamorphic transformations. Comparative analysis against traditional detection approaches revealed superior performance in identifying novel threats, reducing false-positive rates, and achieving consistent classification across diverse ransomware families. The incorporation of a self-regulating hash convergence mechanism further ensured that entropy-synchronized hashes remained invariant across executions, minimizing classification inconsistencies that often arise due to dynamic modifications in ransomware payloads. Experimental results demonstrated high detection rates across contemporary ransomware strains, with the model exhibiting resilience against encryption-based evasion mechanisms, code injection strategies, and reflective loading techniques. Unlike conventional detection mechanisms that rely on static signatures and heuristic analysis, the proposed entropy-aware classification framework adapts to emerging threats through an inherent ability to capture entropy anomalies within executable structures. The findings reinforce the potential of entropy-based detection in addressing the limitations of traditional methodologies while enhancing detection robustness against obfuscation and adversarial evasion techniques.

Title: Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models

• Authors: Wanlong Liu, Yichen Xiao, Dingyi Zeng, Hongyang Zhao, Wenyu Chen, Malu Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18154
• Pdf URL: https://arxiv.org/pdf/2501.18154
• Copy Paste: [[2501.18154]] Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models(https://arxiv.org/abs/2501.18154)
• Keywords: large language model
• Abstract: Post-Training Quantization (PTQ) is pivotal for deploying large language models (LLMs) within resource-limited settings by significantly reducing resource demands. However, existing PTQ strategies underperform at low bit levels < 3 bits due to the significant difference between the quantized and original weights. To enhance the quantization performance at low bit widths, we introduce a Mixed-precision Graph Neural PTQ (MG-PTQ) approach, employing a graph neural network (GNN) module to capture dependencies among weights and adaptively assign quantization bit-widths. Through the information propagation of the GNN module, our method more effectively captures dependencies among target weights, leading to a more accurate assessment of weight importance and optimized allocation of quantization strategies. Extensive experiments on the WikiText2 and C4 datasets demonstrate that our MG-PTQ method outperforms previous state-of-the-art PTQ method GPTQ, setting new benchmarks for quantization performance under low-bit conditions.

Title: Large Language Models for Cryptocurrency Transaction Analysis: A Bitcoin Case Study

• Authors: Yuchen Lei, Yuexin Xiang, Qin Wang, Rafael Dowsley, Tsz Hon Yuen, Jiangshan Yu
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18158
• Pdf URL: https://arxiv.org/pdf/2501.18158
• Copy Paste: [[2501.18158]] Large Language Models for Cryptocurrency Transaction Analysis: A Bitcoin Case Study(https://arxiv.org/abs/2501.18158)
• Keywords: robust, interpretability, large language model
• Abstract: Cryptocurrencies are widely used, yet current methods for analyzing transactions heavily rely on opaque, black-box models. These lack interpretability and adaptability, failing to effectively capture behavioral patterns. Many researchers, including us, believe that Large Language Models (LLMs) could bridge this gap due to their robust reasoning abilities for complex tasks. In this paper, we test this hypothesis by applying LLMs to real-world cryptocurrency transaction graphs, specifically within the Bitcoin network. We introduce a three-tiered framework to assess LLM capabilities: foundational metrics, characteristic overview, and contextual interpretation. This includes a new, human-readable graph representation format, LLM4TG, and a connectivity-enhanced sampling algorithm, CETraS, which simplifies larger transaction graphs. Experimental results show that LLMs excel at foundational metrics and offer detailed characteristic overviews. Their effectiveness in contextual interpretation suggests they can provide useful explanations of transaction behaviors, even with limited labeled data.

Title: IROAM: Improving Roadside Monocular 3D Object Detection Learning from Autonomous Vehicle Data Domain

• Authors: Zhe Wang, Xiaoliang Huo, Siqi Fan, Jingjing Liu, Ya-Qin Zhang, Yan Wang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.18162
• Pdf URL: https://arxiv.org/pdf/2501.18162
• Copy Paste: [[2501.18162]] IROAM: Improving Roadside Monocular 3D Object Detection Learning from Autonomous Vehicle Data Domain(https://arxiv.org/abs/2501.18162)
• Keywords: transformer
• Abstract: In autonomous driving, The perception capabilities of the ego-vehicle can be improved with roadside sensors, which can provide a holistic view of the environment. However, existing monocular detection methods designed for vehicle cameras are not suitable for roadside cameras due to viewpoint domain gaps. To bridge this gap and Improve ROAdside Monocular 3D object detection, we propose IROAM, a semantic-geometry decoupled contrastive learning framework, which takes vehicle-side and roadside data as input simultaneously. IROAM has two significant modules. In-Domain Query Interaction module utilizes a transformer to learn content and depth information for each domain and outputs object queries. Cross-Domain Query Enhancement To learn better feature representations from two domains, Cross-Domain Query Enhancement decouples queries into semantic and geometry parts and only the former is used for contrastive learning. Experiments demonstrate the effectiveness of IROAM in improving roadside detector's performance. The results validate that IROAM has the capabilities to learn cross-domain information.

Title: Continually Evolved Multimodal Foundation Models for Cancer Prognosis

• Authors: Jie Peng, Shuang Zhou, Longwei Yang, Yiran Song, Mohan Zhang, Kaixiong Zhou, Feng Xie, Mingquan Lin, Rui Zhang, Tianlong Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18170
• Pdf URL: https://arxiv.org/pdf/2501.18170
• Copy Paste: [[2501.18170]] Continually Evolved Multimodal Foundation Models for Cancer Prognosis(https://arxiv.org/abs/2501.18170)
• Keywords: robust
• Abstract: Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates. To enhance prediction accuracy, previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information. However, existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals, thus rendering sub-optimal generalizability and limited utility in real-world applications. Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities. To address these, we propose a continually evolving multi-modal foundation model. Extensive experiments on the TCGA dataset demonstrate the effectiveness of our approach, highlighting its potential to advance cancer prognosis by enabling robust and adaptive multimodal integration.

Title: Advancing Personalized Federated Learning: Integrative Approaches with AI for Enhanced Privacy and Customization

• Authors: Kevin Cooper, Michael Geller
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.18174
• Pdf URL: https://arxiv.org/pdf/2501.18174
• Copy Paste: [[2501.18174]] Advancing Personalized Federated Learning: Integrative Approaches with AI for Enhanced Privacy and Customization(https://arxiv.org/abs/2501.18174)
• Keywords: privacy, protect, robust, federate
• Abstract: In the age of data-driven decision making, preserving privacy while providing personalized experiences has become paramount. Personalized Federated Learning (PFL) offers a promising framework by decentralizing the learning process, thus ensuring data privacy and reducing reliance on centralized data repositories. However, the integration of advanced Artificial Intelligence (AI) techniques within PFL remains underexplored. This paper proposes a novel approach that enhances PFL with cutting-edge AI methodologies including adaptive optimization, transfer learning, and differential privacy. We present a model that not only boosts the performance of individual client models but also ensures robust privacy-preserving mechanisms and efficient resource utilization across heterogeneous networks. Empirical results demonstrate significant improvements in model accuracy and personalization, along with stringent privacy adherence, as compared to conventional federated learning models. This work paves the way for a new era of truly personalized and privacy-conscious AI systems, offering significant implications for industries requiring compliance with stringent data protection regulations.

Title: In-Context Learning of Polynomial Kernel Regression in Transformers with GLU Layers

• Authors: Haoyuan Sun, Ali Jadbabaie, Navid Azizan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18187
• Pdf URL: https://arxiv.org/pdf/2501.18187
• Copy Paste: [[2501.18187]] In-Context Learning of Polynomial Kernel Regression in Transformers with GLU Layers(https://arxiv.org/abs/2501.18187)
• Keywords: transformer
• Abstract: Transformer-based models have demonstrated remarkable ability in in-context learning (ICL), where they can adapt to unseen tasks from a prompt with a few examples, without requiring parameter updates. Recent research has provided insight into how linear Transformers can perform ICL by implementing gradient descent estimators. In particular, it has been shown that the optimal linear self-attention (LSA) mechanism can implement one step of gradient descent with respect to a linear least-squares objective when trained on random linear regression tasks. However, the theoretical understanding of ICL for nonlinear function classes remains limited. In this work, we address this gap by first showing that LSA is inherently restricted to solving linear least-squares objectives and thus, the solutions in prior works cannot readily extend to nonlinear ICL tasks. To overcome this limitation, drawing inspiration from modern architectures, we study a mechanism that combines LSA with GLU-like feed-forward layers and show that this allows the model to perform one step of gradient descent on a polynomial kernel regression. Further, we characterize the scaling behavior of the resulting Transformer model, highlighting the necessary model size to effectively handle quadratic ICL tasks. Our findings highlight the distinct roles of attention and feed-forward layers in nonlinear ICL and identify key challenges when extending ICL to nonlinear function classes.

Title: Machine Learning Fairness for Depression Detection using EEG Data

• Authors: Angus Man Ho Kwok, Jiaee Cheong, Sinan Kalkan, Hatice Gunes
• Subjects: cs.CV, cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.18192
• Pdf URL: https://arxiv.org/pdf/2501.18192
• Copy Paste: [[2501.18192]] Machine Learning Fairness for Depression Detection using EEG Data(https://arxiv.org/abs/2501.18192)
• Keywords: fair
• Abstract: This paper presents the very first attempt to evaluate machine learning fairness for depression detection using electroencephalogram (EEG) data. We conduct experiments using different deep learning architectures such as Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks across three EEG datasets: Mumtaz, MODMA and Rest. We employ five different bias mitigation strategies at the pre-, in- and post-processing stages and evaluate their effectiveness. Our experimental results show that bias exists in existing EEG datasets and algorithms for depression detection, and different bias mitigation methods address bias at different levels across different fairness measures.

Title: GDformer: Going Beyond Subsequence Isolation for Multivariate Time Series Anomaly Detection

• Authors: Qingxiang Liu, Chenghao Liu, Sheng Sun, Di Yao, Yuxuan Liang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18196
• Pdf URL: https://arxiv.org/pdf/2501.18196
• Copy Paste: [[2501.18196]] GDformer: Going Beyond Subsequence Isolation for Multivariate Time Series Anomaly Detection(https://arxiv.org/abs/2501.18196)
• Keywords: transformer
• Abstract: Unsupervised anomaly detection of multivariate time series is a challenging task, given the requirements of deriving a compact detection criterion without accessing the anomaly points. The existing methods are mainly based on reconstruction error or association divergence, which are both confined to isolated subsequences with limited horizons, hardly promising unified series-level criterion. In this paper, we propose the Global Dictionary-enhanced Transformer (GDformer) with a renovated dictionary-based cross attention mechanism to cultivate the global representations shared by all normal points in the entire series. Accordingly, the cross-attention maps reflect the correlation weights between the point and global representations, which naturally leads to the representation-wise similarity-based detection criterion. To foster more compact detection boundary, prototypes are introduced to capture the distribution of normal point-global correlation weights. GDformer consistently achieves state-of-the-art unsupervised anomaly detection performance on five real-world benchmark datasets. Further experiments validate the global dictionary has great transferability among various datasets. The code is available at this https URL.

Title: HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation

• Authors: Grzegorz Dudek, Tomasz Rodak
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18199
• Pdf URL: https://arxiv.org/pdf/2501.18199
• Copy Paste: [[2501.18199]] HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation(https://arxiv.org/abs/2501.18199)
• Keywords: robust
• Abstract: This paper introduces the Hierarchical Kolmogorov-Arnold Network (HKAN), a novel network architecture that offers a competitive alternative to the recently proposed Kolmogorov-Arnold Network (KAN). Unlike KAN, which relies on backpropagation, HKAN adopts a randomized learning approach, where the parameters of its basis functions are fixed, and linear aggregations are optimized using least-squares regression. HKAN utilizes a hierarchical multi-stacking framework, with each layer refining the predictions from the previous one by solving a series of linear regression problems. This non-iterative training method simplifies computation and eliminates sensitivity to local minima in the loss function. Empirical results show that HKAN delivers comparable, if not superior, accuracy and stability relative to KAN across various regression tasks, while also providing insights into variable importance. The proposed approach seamlessly integrates theoretical insights with practical applications, presenting a robust and efficient alternative for neural network modeling.

Title: Contextually Structured Token Dependency Encoding for Large Language Models

• Authors: James Blades, Frederick Somerfield, William Langley, Susan Everingham, Maurice Witherington
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18205
• Pdf URL: https://arxiv.org/pdf/2501.18205
• Copy Paste: [[2501.18205]] Contextually Structured Token Dependency Encoding for Large Language Models(https://arxiv.org/abs/2501.18205)
• Keywords: transformer, large language model
• Abstract: Token representation strategies within large-scale neural architectures often rely on contextually refined embeddings, yet conventional approaches seldom encode structured relationships explicitly within token interactions. Self-attention mechanisms effectively capture dynamic contextual dependencies, but their reliance on learned weight distributions limits the preservation of long-range hierarchical structures in generated sequences. Dependency-aware token encoding introduces a structured approach to embedding initialization, ensuring that relational constraints are embedded within token representations rather than inferred solely through attention dynamics. The proposed encoding mechanism refines token interactions through dependency-weighted attention computations, ensuring that syntactic and semantic dependencies are retained across multiple processing layers. Empirical evaluations indicate reductions in perplexity across diverse linguistic benchmarks, suggesting improvements in contextual coherence and predictive consistency in autoregressive text generation. Computational efficiency assessments reveal a moderate increase in memory consumption and training time, attributed to additional matrix computations within the encoding module, yet scalability remains feasible within conventional transformer architectures. Structured encoding enhances lexical variation and dependency retention, reinforcing linguistic coherence without requiring external syntactic annotations or auxiliary training objectives. Statistical comparisons highlight improvements in dependency alignment, particularly in longer sequences where conventional self-attention models exhibit degradation in hierarchical consistency. Sentence length distributions indicate a reduction in abrupt phrase transitions, further supporting the hypothesis that explicit dependency encoding facilitates more structured phrase generation.

Title: Free-T2M: Frequency Enhanced Text-to-Motion Diffusion Model With Consistency Loss

• Authors: Wenshuo Chen, Haozhe Jia, Songning Lai, Keming Wu, Hongru Xiao, Lijie Hu, Yutao Yue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18232
• Pdf URL: https://arxiv.org/pdf/2501.18232
• Copy Paste: [[2501.18232]] Free-T2M: Frequency Enhanced Text-to-Motion Diffusion Model With Consistency Loss(https://arxiv.org/abs/2501.18232)
• Keywords: robust, diffusion
• Abstract: Rapid progress in text-to-motion generation has been largely driven by diffusion models. However, existing methods focus solely on temporal modeling, thereby overlooking frequency-domain analysis. We identify two key phases in motion denoising: the **semantic planning stage** and the **fine-grained improving stage**. To address these phases effectively, we propose **Fre**quency **e**nhanced **t**ext-**to**-**m**otion diffusion model (**Free-T2M**), incorporating stage-specific consistency losses that enhance the robustness of static features and improve fine-grained accuracy. Extensive experiments demonstrate the effectiveness of our method. Specifically, on StableMoFusion, our method reduces the FID from **0.189** to **0.051**, establishing a new SOTA performance within the diffusion architecture. These findings highlight the importance of incorporating frequency-domain insights into text-to-motion generation for more precise and robust results.

Title: Arbitrary Data as Images: Fusion of Patient Data Across Modalities and Irregular Intervals with Vision Transformers

• Authors: Malte Tölle, Mohamad Scharaf, Samantha Fischer, Christoph Reich, Silav Zeid, Christoph Dieterich, Benjamin Meder, Norbert Frey, Philipp Wild, Sandy Engelhardt
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18237
• Pdf URL: https://arxiv.org/pdf/2501.18237
• Copy Paste: [[2501.18237]] Arbitrary Data as Images: Fusion of Patient Data Across Modalities and Irregular Intervals with Vision Transformers(https://arxiv.org/abs/2501.18237)
• Keywords: transformer
• Abstract: A patient undergoes multiple examinations in each hospital stay, where each provides different facets of the health status. These assessments include temporal data with varying sampling rates, discrete single-point measurements, therapeutic interventions such as medication administration, and images. While physicians are able to process and integrate diverse modalities intuitively, neural networks need specific modeling for each modality complicating the training procedure. We demonstrate that this complexity can be significantly reduced by visualizing all information as images along with unstructured text and subsequently training a conventional vision-text transformer. Our approach, Vision Transformer for irregular sampled Multi-modal Measurements (ViTiMM), not only simplifies data preprocessing and modeling but also outperforms current state-of-the-art methods in predicting in-hospital mortality and phenotyping, as evaluated on 6,175 patients from the MIMIC-IV dataset. The modalities include patient's clinical measurements, medications, X-ray images, and electrocardiography scans. We hope our work inspires advancements in multi-modal medical AI by reducing the training complexity to (visual) prompt engineering, thus lowering entry barriers and enabling no-code solutions for training. The source code will be made publicly available.

Title: Ground Awareness in Deep Learning for Large Outdoor Point Cloud Segmentation

• Authors: Kevin Qiu, Dimitri Bulatov, Dorota Iwaszczuk
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18246
• Pdf URL: https://arxiv.org/pdf/2501.18246
• Copy Paste: [[2501.18246]] Ground Awareness in Deep Learning for Large Outdoor Point Cloud Segmentation(https://arxiv.org/abs/2501.18246)
• Keywords: segmentation
• Abstract: This paper presents an analysis of utilizing elevation data to aid outdoor point cloud semantic segmentation through existing machine-learning networks in remote sensing, specifically in urban, built-up areas. In dense outdoor point clouds, the receptive field of a machine learning model may be too small to accurately determine the surroundings and context of a point. By computing Digital Terrain Models (DTMs) from the point clouds, we extract the relative elevation feature, which is the vertical distance from the terrain to a point. RandLA-Net is employed for efficient semantic segmentation of large-scale point clouds. We assess its performance across three diverse outdoor datasets captured with varying sensor technologies and sensor locations. Integration of relative elevation data leads to consistent performance improvements across all three datasets, most notably in the Hessigheim dataset, with an increase of 3.7 percentage points in average F1 score from 72.35% to 76.01%, by establishing long-range dependencies between ground and objects. We also explore additional local features such as planarity, normal vectors, and 2D features, but their efficacy varied based on the characteristics of the point cloud. Ultimately, this study underscores the important role of the non-local relative elevation feature for semantic segmentation of point clouds in remote sensing applications.

Title: PDE-DKL: PDE-constrained deep kernel learning in high dimensionality

• Authors: Weihao Yan, Christoph Brune, Mengwu Guo
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.18258
• Pdf URL: https://arxiv.org/pdf/2501.18258
• Copy Paste: [[2501.18258]] PDE-DKL: PDE-constrained deep kernel learning in high dimensionality(https://arxiv.org/abs/2501.18258)
• Keywords: robust
• Abstract: Many physics-informed machine learning methods for PDE-based problems rely on Gaussian processes (GPs) or neural networks (NNs). However, both face limitations when data are scarce and the dimensionality is high. Although GPs are known for their robust uncertainty quantification in low-dimensional settings, their computational complexity becomes prohibitive as the dimensionality increases. In contrast, while conventional NNs can accommodate high-dimensional input, they often require extensive training data and do not offer uncertainty quantification. To address these challenges, we propose a PDE-constrained Deep Kernel Learning (PDE-DKL) framework that combines DL and GPs under explicit PDE constraints. Specifically, NNs learn a low-dimensional latent representation of the high-dimensional PDE problem, reducing the complexity of the problem. GPs then perform kernel regression subject to the governing PDEs, ensuring accurate solutions and principled uncertainty quantification, even when available data are limited. This synergy unifies the strengths of both NNs and GPs, yielding high accuracy, robust uncertainty estimates, and computational efficiency for high-dimensional PDEs. Numerical experiments demonstrate that PDE-DKL achieves high accuracy with reduced data requirements. They highlight its potential as a practical, reliable, and scalable solver for complex PDE-based applications in science and engineering.

Title: MAMS: Model-Agnostic Module Selection Framework for Video Captioning

• Authors: Sangho Lee, Il Yong Chun, Hogun Park
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18269
• Pdf URL: https://arxiv.org/pdf/2501.18269
• Copy Paste: [[2501.18269]] MAMS: Model-Agnostic Module Selection Framework for Video Captioning(https://arxiv.org/abs/2501.18269)
• Keywords: transformer
• Abstract: Multi-modal transformers are rapidly gaining attention in video captioning tasks. Existing multi-modal video captioning methods typically extract a fixed number of frames, which raises critical challenges. When a limited number of frames are extracted, important frames with essential information for caption generation may be missed. Conversely, extracting an excessive number of frames includes consecutive frames, potentially causing redundancy in visual tokens extracted from consecutive video frames. To extract an appropriate number of frames for each video, this paper proposes the first model-agnostic module selection framework in video captioning that has two main functions: (1) selecting a caption generation module with an appropriate size based on visual tokens extracted from video frames, and (2) constructing subsets of visual tokens for the selected caption generation module. Furthermore, we propose a new adaptive attention masking scheme that enhances attention on important visual tokens. Our experiments on three different benchmark datasets demonstrate that the proposed framework significantly improves the performance of three recent video captioning models.

Title: SoK: Measuring Blockchain Decentralization

• Authors: Christina Ovezik, Dimitris Karakostas, Aggelos Kiayias, Daniel Woods
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18279
• Pdf URL: https://arxiv.org/pdf/2501.18279
• Copy Paste: [[2501.18279]] SoK: Measuring Blockchain Decentralization(https://arxiv.org/abs/2501.18279)
• Keywords: extraction
• Abstract: In the context of blockchain systems, the importance of decentralization is undermined by the lack of a widely accepted methodology to measure it. To address this gap, we set out a systematization effort targeting the decentralization measurement workflow. To facilitate our systematization, we put forth a framework that categorizes all measurement techniques used in previous work based on the resource they target, the methods they use to extract resource allocation, and the functions they apply to produce the final measurements. We complement this framework with an empirical analysis designed to evaluate whether the various pre-processing steps and metrics used in prior work capture the same underlying concept of decentralization. Our analysis brings about a number of novel insights and observations. First, the seemingly innocuous choices performed during data extraction, such as the size of estimation windows or the application of thresholds that affect the resource distribution, have important repercussions when calculating the level of decentralization. Second, exploratory factor analysis suggests that in Proof-of-Work (PoW) blockchains, participation on the consensus layer is not correlated with decentralization, but rather captures a distinct signal, unlike in Proof-of-Stake (PoS) systems, where the different metrics align under a single factor. These findings challenge the long-held assumption within the blockchain community that higher participation drives higher decentralization. Finally, we combine the results of our empirical analysis with first-principles reasoning to derive practical recommendations for researchers that set out to measure blockchain decentralization, and we further systematize the existing literature in line with these recommendations.

Title: Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

• Authors: Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, Bo Zhang
• Subjects: cs.CL, cs.AI, cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2501.18280
• Pdf URL: https://arxiv.org/pdf/2501.18280
• Copy Paste: [[2501.18280]] Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models(https://arxiv.org/abs/2501.18280)
• Keywords: security, defense, attack, large language model
• Abstract: The security issue of large language models (LLMs) has gained significant attention recently, with various defense mechanisms developed to prevent harmful outputs, among which safeguards based on text embedding models serve as a fundamental defense. Through testing, we discover that the distribution of text embedding model outputs is significantly biased with a large mean. Inspired by this observation, we propose novel efficient methods to search for universal magic words that can attack text embedding models. The universal magic words as suffixes can move the embedding of any text towards the bias direction, therefore manipulate the similarity of any text pair and mislead safeguards. By appending magic words to user prompts and requiring LLMs to end answers with magic words, attackers can jailbreak the safeguard. To eradicate this security risk, we also propose defense mechanisms against such attacks, which can correct the biased distribution of text embeddings in a train-free manner.

Title: Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models

• Authors: Jennifer D'Souza, Zachary Laubach, Tarek Al Mustafa, Sina Zarrieß, Robert Frühstückl, Phyllis Illari
• Subjects: cs.CL, cs.AI, cs.DL
• Abstract URL: https://arxiv.org/abs/2501.18287
• Pdf URL: https://arxiv.org/pdf/2501.18287
• Copy Paste: [[2501.18287]] Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models(https://arxiv.org/abs/2501.18287)
• Keywords: extraction, large language model
• Abstract: This paper presents an exploratory study that harnesses the capabilities of large language models (LLMs) to mine key ecological entities from invasion biology literature. Specifically, we focus on extracting species names, their locations, associated habitats, and ecosystems, information that is critical for understanding species spread, predicting future invasions, and informing conservation efforts. Traditional text mining approaches often struggle with the complexity of ecological terminology and the subtle linguistic patterns found in these texts. By applying general-purpose LLMs without domain-specific fine-tuning, we uncover both the promise and limitations of using these models for ecological entity extraction. In doing so, this study lays the groundwork for more advanced, automated knowledge extraction tools that can aid researchers and practitioners in understanding and managing biological invasions.

Title: Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices

• Authors: Furkan Bagci, Busra Tegin, Mohammad Kazemi, Tolga M. Duman
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.18298
• Pdf URL: https://arxiv.org/pdf/2501.18298
• Copy Paste: [[2501.18298]] Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices(https://arxiv.org/abs/2501.18298)
• Keywords: federate
• Abstract: We study over-the-air (OTA) federated learning (FL) for energy harvesting devices with heterogeneous data distribution over wireless fading multiple access channel (MAC). To address the impact of low energy arrivals and data heterogeneity on global learning, we propose user scheduling strategies. Specifically, we develop two approaches: 1) entropy-based scheduling for known data distributions and 2) least-squares-based user representation estimation for scheduling with unknown data distributions at the parameter server. Both methods aim to select diverse users, mitigating bias and enhancing convergence. Numerical and analytical results demonstrate improved learning performance by reducing redundancy and conserving energy.

Title: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis

• Authors: Haoxiong Liu, Jiacheng Sun, Zhenguo Li, Andrew C Yao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18310
• Pdf URL: https://arxiv.org/pdf/2501.18310
• Copy Paste: [[2501.18310]] Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis(https://arxiv.org/abs/2501.18310)
• Keywords: robust
• Abstract: The synergy between deep learning models and traditional automation tools plays a pivotal role in developing robust neural theorem provers (NTPs). However, for proof synthesis with LLMs, previous work applies automation tools either only when the model explicitly calls the method, or only at a single granularity level, failing to fully exploit the power of built-in tactics and off-the-shelf automated theorem provers. In this work, we propose ProofAug, a novel theorem proving method that enjoys superior sample efficiency through equipping proof-generation LLMs with automation methods in different granularities via fine-grained structure analysis of model-generated proof proposals. Furthermore, ProofAug serves as a versatile plug-and-play module that seamlessly integrates with any tree-search algorithm, enabling our construction of an efficient recursive proving (ERP) module to further enhance performance. The superiority of our method is validated on the miniF2F-test benchmark using the open-source deepseek-math-7b-base model and the Isabelle proof assistant. Notably, by additionally employing a mixed prompting strategy, we achieve a cumulative pass rate of 66.0% after curation of the dataset (61.9% for the original version), setting a new SOTA across all proof languages with a total sample budget of only 2100. Our code is available at this https URL.

Title: Simulation of microstructures and machine learning

• Authors: Katja Schladitz, Claudia Redenbach, Tin Barisin, Christian Jung, Natascha Jeziorski, Lovro Bosnar, Juraj Fulir, Petra Gospodnetić
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18313
• Pdf URL: https://arxiv.org/pdf/2501.18313
• Copy Paste: [[2501.18313]] Simulation of microstructures and machine learning(https://arxiv.org/abs/2501.18313)
• Keywords: segmentation
• Abstract: Machine learning offers attractive solutions to challenging image processing tasks. Tedious development and parametrization of algorithmic solutions can be replaced by training a convolutional neural network or a random forest with a high potential to generalize. However, machine learning methods rely on huge amounts of representative image data along with a ground truth, usually obtained by manual annotation. Thus, limited availability of training data is a critical bottleneck. We discuss two use cases: optical quality control in industrial production and segmenting crack structures in 3D images of concrete. For optical quality control, all defect types have to be trained but are typically not evenly represented in the training data. Additionally, manual annotation is costly and often inconsistent. It is nearly impossible in the second case: segmentation of crack systems in 3D images of concrete. Synthetic images, generated based on realizations of stochastic geometry models, offer an elegant way out. A wide variety of structure types can be generated. The within structure variation is naturally captured by the stochastic nature of the models and the ground truth is for free. Many new questions arise. In particular, which characteristics of the real image data have to be met to which degree of fidelity.

Title: A Unified Perspective on the Dynamics of Deep Transformers

• Authors: Valérie Castin, Pierre Ablin, José Antonio Carrillo, Gabriel Peyré
• Subjects: cs.LG, math.AP
• Abstract URL: https://arxiv.org/abs/2501.18322
• Pdf URL: https://arxiv.org/pdf/2501.18322
• Copy Paste: [[2501.18322]] A Unified Perspective on the Dynamics of Deep Transformers(https://arxiv.org/abs/2501.18322)
• Keywords: transformer
• Abstract: Transformers, which are state-of-the-art in most machine learning tasks, represent the data as sequences of vectors called tokens. This representation is then exploited by the attention function, which learns dependencies between tokens and is key to the success of Transformers. However, the iterative application of attention across layers induces complex dynamics that remain to be fully understood. To analyze these dynamics, we identify each input sequence with a probability measure and model its evolution as a Vlasov equation called Transformer PDE, whose velocity field is non-linear in the probability measure. Our first set of contributions focuses on compactly supported initial data. We show the Transformer PDE is well-posed and is the mean-field limit of an interacting particle system, thus generalizing and extending previous analysis to several variants of self-attention: multi-head attention, L2 attention, Sinkhorn attention, Sigmoid attention, and masked attention--leveraging a conditional Wasserstein framework. In a second set of contributions, we are the first to study non-compactly supported initial conditions, by focusing on Gaussian initial data. Again for different types of attention, we show that the Transformer PDE preserves the space of Gaussian measures, which allows us to analyze the Gaussian case theoretically and numerically to identify typical behaviors. This Gaussian analysis captures the evolution of data anisotropy through a deep Transformer. In particular, we highlight a clustering phenomenon that parallels previous results in the non-normalized discrete case.

Title: Stream-Based Monitoring of Algorithmic Fairness

• Authors: Jan Baumeister, Bernd Finkbeiner, Frederik Scheerer, Julian Siber, Tobias Wagenpfeil
• Subjects: cs.LG, cs.LO, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.18331
• Pdf URL: https://arxiv.org/pdf/2501.18331
• Copy Paste: [[2501.18331]] Stream-Based Monitoring of Algorithmic Fairness(https://arxiv.org/abs/2501.18331)
• Keywords: fair
• Abstract: Automatic decision and prediction systems are increasingly deployed in applications where they significantly impact the livelihood of people, such as for predicting the creditworthiness of loan applicants or the recidivism risk of defendants. These applications have given rise to a new class of algorithmic-fairness specifications that require the systems to decide and predict without bias against social groups. Verifying these specifications statically is often out of reach for realistic systems, since the systems may, e.g., employ complex learning components, and reason over a large input space. In this paper, we therefore propose stream-based monitoring as a solution for verifying the algorithmic fairness of decision and prediction systems at runtime. Concretely, we present a principled way to formalize algorithmic fairness over temporal data streams in the specification language RTLola and demonstrate the efficacy of this approach on a number of benchmarks. Besides synthetic scenarios that particularly highlight its efficiency on streams with a scaling amount of data, we notably evaluate the monitor on real-world data from the recidivism prediction tool COMPAS.

Title: State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence

• Authors: Thea Aviss
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18356
• Pdf URL: https://arxiv.org/pdf/2501.18356
• Copy Paste: [[2501.18356]] State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence(https://arxiv.org/abs/2501.18356)
• Keywords: transformer
• Abstract: We introduce the State Stream Transformer (SST), a novel LLM architecture that reveals emergent reasoning behaviours and capabilities latent in pretrained weights through addressing a fundamental limitation in traditional transformer models: the lack of latent computational continuity across autoregressive generations in the state space. SST introduces a sliding window latent state (FFN) cache with weighted decay that maintains and evolves persistent latent processes throughout autoregressive generations. Through controlled experiments comparing base and SST architectures using the same frozen weights, we demonstrate that this architectural modification alone enables enhanced reasoning capabilities which appear best explained by some form of potential higher-order processing, as evidenced by emergent metacognitive behaviours. These behaviours persist under controlled conditions designed to eliminate confounding factors such as stochastic variation or learned response patterns. Analysis of latent state distributions and processing dynamics provides evidence that it is solely the 'state stream' that is responsible for these phenomena. In quantitative evaluations, the SST achieves substantial performance improvements over the base model on two reasoning benchmarks, reaching 89.01\% accuracy on GSM-8K (0-shot) and 91.04\% on ARC Challenge (0-shot CoT). These findings indicate that persistent computation in the latent state space enables fundamentally different information processing and internal reasoning strategies, with implications for our understanding of artificial intelligence systems.

Title: Video-based Surgical Tool-tip and Keypoint Tracking using Multi-frame Context-driven Deep Learning Models

• Authors: Bhargav Ghanekar, Lianne R. Johnson, Jacob L. Laughlin, Marcia K. O'Malley, Ashok Veeraraghavan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18361
• Pdf URL: https://arxiv.org/pdf/2501.18361
• Copy Paste: [[2501.18361]] Video-based Surgical Tool-tip and Keypoint Tracking using Multi-frame Context-driven Deep Learning Models(https://arxiv.org/abs/2501.18361)
• Keywords: segmentation
• Abstract: Automated tracking of surgical tool keypoints in robotic surgery videos is an essential task for various downstream use cases such as skill assessment, expertise assessment, and the delineation of safety zones. In recent years, the explosion of deep learning for vision applications has led to many works in surgical instrument segmentation, while lesser focus has been on tracking specific tool keypoints, such as tool tips. In this work, we propose a novel, multi-frame context-driven deep learning framework to localize and track tool keypoints in surgical videos. We train and test our models on the annotated frames from the 2015 EndoVis Challenge dataset, resulting in state-of-the-art performance. By leveraging sophisticated deep learning models and multi-frame context, we achieve 90\% keypoint detection accuracy and a localization RMS error of 5.27 pixels. Results on a self-annotated JIGSAWS dataset with more challenging scenarios also show that the proposed multi-frame models can accurately track tool-tip and tool-base keypoints, with ${<}4.2$-pixel RMS error overall. Such a framework paves the way for accurately tracking surgical instrument keypoints, enabling further downstream use cases. Project and dataset webpage: this https URL

Title: Robust Online Conformal Prediction under Uniform Label Noise

• Authors: Huajun Xi, Kangdao Liu, Hao Zeng, Wenguang Sun, Hongxin Wei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18363
• Pdf URL: https://arxiv.org/pdf/2501.18363
• Copy Paste: [[2501.18363]] Robust Online Conformal Prediction under Uniform Label Noise(https://arxiv.org/abs/2501.18363)
• Keywords: robust
• Abstract: Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Recent work develops online conformal prediction methods that adaptively construct prediction sets to accommodate distribution shifts. However, existing algorithms typically assume perfect label accuracy which rarely holds in practice. In this work, we investigate the robustness of online conformal prediction under uniform label noise with a known noise rate, in both constant and dynamic learning rate schedules. We show that label noise causes a persistent gap between the actual mis-coverage rate and the desired rate $\alpha$, leading to either overestimated or underestimated coverage guarantees. To address this issue, we propose Noise Robust Online Conformal Prediction (dubbed NR-OCP) by updating the threshold with a novel robust pinball los}, which provides an unbiased estimate of clean pinball loss without requiring ground-truth labels. Our theoretical analysis shows that NR-OCP eliminates the coverage gap in both constant and dynamic learning rate schedules, achieving a convergence rate of $\mathcal{O}(T^{-1/2})$ for both empirical and expected coverage errors under uniform label noise. Extensive experiments demonstrate the effectiveness of our method by achieving both precise coverage and improved efficiency.

Title: RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

• Authors: Yiteng Tu, Weihang Su, Yujia Zhou, Yiqun Liu, Qingyao Ai
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.18365
• Pdf URL: https://arxiv.org/pdf/2501.18365
• Copy Paste: [[2501.18365]] RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects(https://arxiv.org/abs/2501.18365)
• Keywords: robust, large language model
• Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved from a knowledge base. However, its effectiveness is fundamentally constrained by the reliability of both the retriever and the knowledge base. In real-world scenarios, imperfections in these components often lead to the retrieval of noisy, irrelevant, or misleading counterfactual information, ultimately undermining the trustworthiness of RAG systems. To address this challenge, we propose Robust Fine-Tuning (RbFT), a method designed to enhance the resilience of LLMs against retrieval defects through two targeted fine-tuning tasks. Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions, surpassing existing methods while maintaining high inference efficiency and compatibility with other robustness techniques.

Title: Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces

• Authors: Tyler Ingebrand, Adam J. Thorpe, Ufuk Topcu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18373
• Pdf URL: https://arxiv.org/pdf/2501.18373
• Copy Paste: [[2501.18373]] Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces(https://arxiv.org/abs/2501.18373)
• Keywords: transformer
• Abstract: A central challenge in transfer learning is designing algorithms that can quickly adapt and generalize to new tasks without retraining. Yet, the conditions of when and how algorithms can effectively transfer to new tasks is poorly characterized. We introduce a geometric characterization of transfer in Hilbert spaces and define three types of inductive transfer: interpolation within the convex hull, extrapolation to the linear span, and extrapolation outside the span. We propose a method grounded in the theory of function encoders to achieve all three types of transfer. Specifically, we introduce a novel training scheme for function encoders using least-squares optimization, prove a universal approximation theorem for function encoders, and provide a comprehensive comparison with existing approaches such as transformers and meta-learning on four diverse benchmarks. Our experiments demonstrate that the function encoder outperforms state-of-the-art methods on four benchmark tasks and on all three types of transfer.

Title: Cracks in concrete

• Authors: Tin Barisin, Christian Jung, Anna Nowacka, Claudia Redenbach, Katja Schladitz
• Subjects: cs.CV, eess.IV, stat.AP
• Abstract URL: https://arxiv.org/abs/2501.18376
• Pdf URL: https://arxiv.org/pdf/2501.18376
• Copy Paste: [[2501.18376]] Cracks in concrete(https://arxiv.org/abs/2501.18376)
• Keywords: segmentation
• Abstract: Finding and properly segmenting cracks in images of concrete is a challenging task. Cracks are thin and rough and being air filled do yield a very weak contrast in 3D images obtained by computed tomography. Enhancing and segmenting dark lower-dimensional structures is already demanding. The heterogeneous concrete matrix and the size of the images further increase the complexity. ML methods have proven to solve difficult segmentation problems when trained on enough and well annotated data. However, so far, there is not much 3D image data of cracks available at all, let alone annotated. Interactive annotation is error-prone as humans can easily tell cats from dogs or roads without from roads with cars but have a hard time deciding whether a thin and dark structure seen in a 2D slice continues in the next one. Training networks by synthetic, simulated images is an elegant way out, bears however its own challenges. In this contribution, we describe how to generate semi-synthetic image data to train CNN like the well known 3D U-Net or random forests for segmenting cracks in 3D images of concrete. The thickness of real cracks varies widely, both, within one crack as well as from crack to crack in the same sample. The segmentation method should therefore be invariant with respect to scale changes. We introduce the so-called RieszNet, designed for exactly this purpose. Finally, we discuss how to generalize the ML crack segmentation methods to other concrete types.

Title: AuthOr: Lower Cost Authenticity-Oriented Garbling for Arbitrary Boolean Circuits

• Authors: Osman Biçer, Ali Ajorian
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18387
• Pdf URL: https://arxiv.org/pdf/2501.18387
• Copy Paste: [[2501.18387]] AuthOr: Lower Cost Authenticity-Oriented Garbling for Arbitrary Boolean Circuits(https://arxiv.org/abs/2501.18387)
• Keywords: security, privacy
• Abstract: Authenticity-oriented (perviously named as privacy-free) garbling schemes (Frederiksen et al. Eurocrypt '15) are designed to satisfy only the authenticity criterion of (Bellare et al. ACM CCS '12), and to be more efficient compared to full-fledged garbling schemes. Here we report a novel authenticity-oriented garbling scheme ``AuthOr'' for general circuits, which has strictly lower communication cost than the state-of-the-art authenticity-oriented version of half gates garbling (Zahur et al. Crypto '15) without any further computation overhead or security assumption. Our solution successfully combines the ideas from information theoretical garbling (Kondi and Patra Crypto '17), half gates garbling, and a novel bandwidth-free AND gate garbling scheme (which we also propose here) while remaining compatible with FreeXOR (Kolesnikov and Schneider ICALP '08). While our scheme beats half gates garbling both at communication and computation costs for many circuits, the exact efficiency improvement can be seen empirically as it depends on the circuit.

Title: MatIR: A Hybrid Mamba-Transformer Image Restoration Model

• Authors: Juan Wen (1 and 2), Weiyan Hou (1), Luc Van Gool (2 and 3 and 4), Radu Timofte (5) ((1) Zhengzhou University, (2) ETH Zurich, (3) KU Leuven, (4) INSAIT, Sofia University, (5) University of Wurzburg)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18401
• Pdf URL: https://arxiv.org/pdf/2501.18401
• Copy Paste: [[2501.18401]] MatIR: A Hybrid Mamba-Transformer Image Restoration Model(https://arxiv.org/abs/2501.18401)
• Keywords: transformer
• Abstract: In recent years, Transformers-based models have made significant progress in the field of image restoration by leveraging their inherent ability to capture complex contextual features. Recently, Mamba models have made a splash in the field of computer vision due to their ability to handle long-range dependencies and their significant computational efficiency compared to Transformers. However, Mamba currently lags behind Transformers in contextual learning capabilities. To overcome the limitations of these two models, we propose a Mamba-Transformer hybrid image restoration model called MatIR. Specifically, MatIR cross-cycles the blocks of the Transformer layer and the Mamba layer to extract features, thereby taking full advantage of the advantages of the two architectures. In the Mamba module, we introduce the Image Inpainting State Space (IRSS) module, which traverses along four scan paths to achieve efficient processing of long sequence data. In the Transformer module, we combine triangular window-based local attention with channel-based global attention to effectively activate the attention mechanism over a wider range of image pixels. Extensive experimental results and ablation studies demonstrate the effectiveness of our approach.

Title: Efficient Transformer for High Resolution Image Motion Deblurring

• Authors: Amanturdieva Akmaral, Muhammad Hamza Zafar
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18403
• Pdf URL: https://arxiv.org/pdf/2501.18403
• Copy Paste: [[2501.18403]] Efficient Transformer for High Resolution Image Motion Deblurring(https://arxiv.org/abs/2501.18403)
• Keywords: robust, transformer
• Abstract: This paper presents a comprehensive study and improvement of the Restormer architecture for high-resolution image motion deblurring. We introduce architectural modifications that reduce model complexity by 18.4% while maintaining or improving performance through optimized attention mechanisms. Our enhanced training pipeline incorporates additional transformations including color jitter, Gaussian blur, and perspective transforms to improve model robustness as well as a new frequency loss term. Extensive experiments on the RealBlur-R, RealBlur-J, and Ultra-High-Definition Motion blurred (UHDM) datasets demonstrate the effectiveness of our approach. The improved architecture shows better convergence behavior and reduced training time while maintaining competitive performance across challenging scenarios. We also provide detailed ablation studies analyzing the impact of our modifications on model behavior and performance. Our results suggest that thoughtful architectural simplification combined with enhanced training strategies can yield more efficient yet equally capable models for motion deblurring tasks. Code and Data Available at: this https URL

Title: Segmentation of cracks in 3d images of fiber reinforced concrete using deep learning

• Authors: Anna Nowacka, Katja Schladitz, Szymon Grzesiak, Matthias Pahn
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18405
• Pdf URL: https://arxiv.org/pdf/2501.18405
• Copy Paste: [[2501.18405]] Segmentation of cracks in 3d images of fiber reinforced concrete using deep learning(https://arxiv.org/abs/2501.18405)
• Keywords: segmentation
• Abstract: Cracks in concrete structures are very common and are an integral part of this heterogeneous material. Characteristics of cracks induced by standardized tests yield valuable information about the tested concrete formulation and its mechanical properties. Observing cracks on the surface of the concrete structure leaves a wealth of structural information unused. Computed tomography enables looking into the sample without interfering or destroying the microstructure. The reconstructed tomographic images are 3d images, consisting of voxels whose gray values represent local X-ray absorption. In order to identify voxels belonging to the crack, so to segment the crack structure in the images, appropriate algorithms need to be developed. Convolutional neural networks are known to solve this type of task very well given enough and consistent training data. We adapted a 3d version of the well-known U-Net and trained it on semi-synthetic 3d images of real concrete samples equipped with simulated crack structures. Here, we explain the general approach. Moreover, we show how to teach the network to detect also real crack systems in 3d images of varying types of real concrete, in particular of fiber reinforced concrete.

Title: Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation

• Authors: Youngjoon Lee, Taehyun Park, Yunho Lee, Jinu Gong, Joonhyuk Kang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18416
• Pdf URL: https://arxiv.org/pdf/2501.18416
• Copy Paste: [[2501.18416]] Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation(https://arxiv.org/abs/2501.18416)
• Keywords: security, attack, federate, large language model
• Abstract: Federated Learning (FL) is increasingly being adopted in military collaborations to develop Large Language Models (LLMs) while preserving data sovereignty. However, prompt injection attacks-malicious manipulations of input prompts-pose new threats that may undermine operational security, disrupt decision-making, and erode trust among allies. This perspective paper highlights four potential vulnerabilities in federated military LLMs: secret data leakage, free-rider exploitation, system disruption, and misinformation spread. To address these potential risks, we propose a human-AI collaborative framework that introduces both technical and policy countermeasures. On the technical side, our framework uses red/blue team wargaming and quality assurance to detect and mitigate adversarial behaviors of shared LLM weights. On the policy side, it promotes joint AI-human policy development and verification of security protocols. Our findings will guide future research and emphasize proactive strategies for emerging military contexts.

Title: Causal Inference Real-Time Anomaly Detection with Synthetic Anomaly Monitoring (SAM)

• Authors: Emanuele Luzio, Moacir Antonelli Ponti
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18417
• Pdf URL: https://arxiv.org/pdf/2501.18417
• Copy Paste: [[2501.18417]] Causal Inference Real-Time Anomaly Detection with Synthetic Anomaly Monitoring (SAM)(https://arxiv.org/abs/2501.18417)
• Keywords: security, robust, interpretability
• Abstract: Anomaly detection is essential for identifying rare and significant events across diverse domains such as finance, cybersecurity, and network monitoring. This paper presents Synthetic Anomaly Monitoring (SAM), an innovative approach that applies synthetic control methods from causal inference to improve both the accuracy and interpretability of anomaly detection processes. By modeling normal behavior through the treatment of each feature as a control unit, SAM identifies anomalies as deviations within this causal framework. We conducted extensive experiments comparing SAM with established benchmark models, including Isolation Forest, Local Outlier Factor (LOF), k-Nearest Neighbors (kNN), and One-Class Support Vector Machine (SVM), across five diverse datasets, including Credit Card Fraud, HTTP Dataset CSIC 2010, and KDD Cup 1999, among others. Our results demonstrate that SAM consistently delivers robust performance, highlighting its potential as a powerful tool for real-time anomaly detection in dynamic and complex environments.

Title: SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer

• Authors: Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, Bingchen Liu, Daquan Zhou, Song Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18427
• Pdf URL: https://arxiv.org/pdf/2501.18427
• Copy Paste: [[2501.18427]] SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer(https://arxiv.org/abs/2501.18427)
• Keywords: diffusion, transformer
• Abstract: This paper presents SANA-1.5, a linear Diffusion Transformer for efficient scaling in text-to-image generation. Building upon SANA-1.0, we introduce three key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that enables scaling from 1.6B to 4.8B parameters with significantly reduced computational resources, combined with a memory-efficient 8-bit optimizer. (2) Model Depth Pruning: A block importance analysis technique for efficient model compression to arbitrary sizes with minimal quality loss. (3) Inference-time Scaling: A repeated sampling strategy that trades computation for model capacity, enabling smaller models to match larger model quality at inference time. Through these strategies, SANA-1.5 achieves a text-image alignment score of 0.72 on GenEval, which can be further improved to 0.80 through inference scaling, establishing a new SoTA on GenEval benchmark. These innovations enable efficient model scaling across different compute budgets while maintaining high quality, making high-quality image generation more accessible.

Title: GENIE: Generative Note Information Extraction model for structuring EHR data

• Authors: Huaiyuan Ying, Hongyi Yuan, Jinsen Lu, Zitian Qu, Yang Zhao, Zhengyun Zhao, Isaac Kohane, Tianxi Cai, Sheng Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18435
• Pdf URL: https://arxiv.org/pdf/2501.18435
• Copy Paste: [[2501.18435]] GENIE: Generative Note Information Extraction model for structuring EHR data(https://arxiv.org/abs/2501.18435)
• Keywords: robust, extraction, generative, large language model
• Abstract: Electronic Health Records (EHRs) hold immense potential for advancing healthcare, offering rich, longitudinal data that combines structured information with valuable insights from unstructured clinical notes. However, the unstructured nature of clinical text poses significant challenges for secondary applications. Traditional methods for structuring EHR free-text data, such as rule-based systems and multi-stage pipelines, are often limited by their time-consuming configurations and inability to adapt across clinical notes from diverse healthcare settings. Few systems provide a comprehensive attribute extraction for terminologies. While giant large language models (LLMs) like GPT-4 and LLaMA 405B excel at structuring tasks, they are slow, costly, and impractical for large-scale use. To overcome these limitations, we introduce GENIE, a Generative Note Information Extraction system that leverages LLMs to streamline the structuring of unstructured clinical text into usable data with standardized format. GENIE processes entire paragraphs in a single pass, extracting entities, assertion statuses, locations, modifiers, values, and purposes with high accuracy. Its unified, end-to-end approach simplifies workflows, reduces errors, and eliminates the need for extensive manual intervention. Using a robust data preparation pipeline and fine-tuned small scale LLMs, GENIE achieves competitive performance across multiple information extraction tasks, outperforming traditional tools like cTAKES and MetaMap and can handle extra attributes to be extracted. GENIE strongly enhances real-world applicability and scalability in healthcare systems. By open-sourcing the model and test data, we aim to encourage collaboration and drive further advancements in EHR structurization.

Title: MolGraph-xLSTM: A graph-based dual-level xLSTM framework with multi-head mixture-of-experts for enhanced molecular representation and interpretability

• Authors: Yan Sun, Yutong Lu, Yan Yi Li, Zihao Jing, Carson K. Leung, Pingzhao Hu
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2501.18439
• Pdf URL: https://arxiv.org/pdf/2501.18439
• Copy Paste: [[2501.18439]] MolGraph-xLSTM: A graph-based dual-level xLSTM framework with multi-head mixture-of-experts for enhanced molecular representation and interpretability(https://arxiv.org/abs/2501.18439)
• Keywords: extraction, interpretability
• Abstract: Predicting molecular properties is essential for drug discovery, and computational methods can greatly enhance this process. Molecular graphs have become a focus for representation learning, with Graph Neural Networks (GNNs) widely used. However, GNNs often struggle with capturing long-range dependencies. To address this, we propose MolGraph-xLSTM, a novel graph-based xLSTM model that enhances feature extraction and effectively models molecule long-range interactions. Our approach processes molecular graphs at two scales: atom-level and motif-level. For atom-level graphs, a GNN-based xLSTM framework with jumping knowledge extracts local features and aggregates multilayer information to capture both local and global patterns effectively. Motif-level graphs provide complementary structural information for a broader molecular view. Embeddings from both scales are refined via a multi-head mixture of experts (MHMoE), further enhancing expressiveness and performance. We validate MolGraph-xLSTM on 10 molecular property prediction datasets, covering both classification and regression tasks. Our model demonstrates consistent performance across all datasets, with improvements of up to 7.03% on the BBBP dataset for classification and 7.54% on the ESOL dataset for regression compared to baselines. On average, MolGraph-xLSTM achieves an AUROC improvement of 3.18\% for classification tasks and an RMSE reduction of 3.83\% across regression datasets compared to the baseline methods. These results confirm the effectiveness of our model, offering a promising solution for molecular representation learning for drug discovery.

Title: CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

• Authors: Yumeng Wang, Zhiyuan Fan, Qingyun Wang, May Fung, Heng Ji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18457
• Pdf URL: https://arxiv.org/pdf/2501.18457
• Copy Paste: [[2501.18457]] CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering(https://arxiv.org/abs/2501.18457)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) are pretrained on extensive multilingual corpora to acquire both language-specific cultural knowledge and general knowledge. Ideally, while LLMs should provide consistent responses to culture-independent questions across languages, we observe significant performance disparities. To address this, we explore the Cross-Lingual Self-Aligning ability of Language Models (CALM) to align knowledge across languages. Specifically, for a given question, we sample multiple responses across different languages, and select the most self-consistent response as the target, leaving the remaining responses as negative examples. We then employ direct preference optimization (DPO) to align the model's knowledge across different languages. Evaluations on the MEDQA and X-CSQA datasets demonstrate CALM's effectiveness in enhancing cross-lingual knowledge question answering, both in zero-shot and retrieval augmented settings. We also found that increasing the number of languages involved in CALM training leads to even higher accuracy and consistency. We offer a qualitative analysis of how cross-lingual consistency can enhance knowledge alignment and explore the method's generalizability. The source code and data of this paper are available on GitHub.

Title: A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models

• Authors: Shiho Noda, Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu Aizawa
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18463
• Pdf URL: https://arxiv.org/pdf/2501.18463
• Copy Paste: [[2501.18463]] A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models(https://arxiv.org/abs/2501.18463)
• Keywords: robust
• Abstract: Out-of-distribution (OOD) detection is a task that detects OOD samples during inference to ensure the safety of deployed models. However, conventional benchmarks have reached performance saturation, making it difficult to compare recent OOD detection methods. To address this challenge, we introduce three novel OOD detection benchmarks that enable a deeper understanding of method characteristics and reflect real-world conditions. First, we present ImageNet-X, designed to evaluate performance under challenging semantic shifts. Second, we propose ImageNet-FS-X for full-spectrum OOD detection, assessing robustness to covariate shifts (feature distribution shifts). Finally, we propose Wilds-FS-X, which extends these evaluations to real-world datasets, offering a more comprehensive testbed. Our experiments reveal that recent CLIP-based OOD detection methods struggle to varying degrees across the three proposed benchmarks, and none of them consistently outperforms the others. We hope the community goes beyond specific benchmarks and includes more challenging conditions reflecting real-world scenarios. The code is this https URL.

Title: Tuning Vision Foundation Model via Test-Time Prompt-Guided Training for VFSS Segmentations

• Authors: Chengxi Zeng, David Smithard, Alberto M Gambaruto, Tilo Burghardt
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18474
• Pdf URL: https://arxiv.org/pdf/2501.18474
• Copy Paste: [[2501.18474]] Tuning Vision Foundation Model via Test-Time Prompt-Guided Training for VFSS Segmentations(https://arxiv.org/abs/2501.18474)
• Keywords: segmentation
• Abstract: Vision foundation models have demonstrated exceptional generalization capabilities in segmentation tasks for both generic and specialized images. However, a performance gap persists between foundation models and task-specific, specialized models. Fine-tuning foundation models on downstream datasets is often necessary to bridge this gap. Unfortunately, obtaining fully annotated ground truth for downstream datasets is both challenging and costly. To address this limitation, we propose a novel test-time training paradigm that enhances the performance of foundation models on downstream datasets without requiring full annotations. Specifically, our method employs simple point prompts to guide a test-time semi-self-supervised training task. The model learns by resolving the ambiguity of the point prompt through various augmentations. This approach directly tackles challenges in the medical imaging field, where acquiring annotations is both time-intensive and expensive. We conducted extensive experiments on our new Videofluoroscopy dataset (VFSS-5k) for the instance segmentation task, achieving an average Dice coefficient of 0.868 across 12 anatomies with a single model.

Title: CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization

• Authors: Yanxia Deng, Aozhong Zhang, Naigang Wang, Selcuk Gurses, Zi Yang, Penghang Yin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18475
• Pdf URL: https://arxiv.org/pdf/2501.18475
• Copy Paste: [[2501.18475]] CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization(https://arxiv.org/abs/2501.18475)
• Keywords: large language model
• Abstract: Fine-tuning large language models (LLMs) using low-rank adaptation (LoRA) has become a highly efficient approach for downstream tasks, particularly in scenarios with limited computational resources. However, applying LoRA techniques to quantized LLMs poses unique challenges due to the reduced representational precision of quantized weights. In this paper, we introduce CLoQ (Calibrated LoRA initialization for Quantized LLMs), a simplistic initialization strategy designed to overcome these challenges. Our approach focuses on minimizing the layer-wise discrepancy between the original LLM and its quantized counterpart with LoRA components during initialization. By leveraging a small calibration dataset, CLoQ quantizes a pre-trained LLM and determines the optimal LoRA components for each layer, ensuring a strong foundation for subsequent fine-tuning. A key contribution of this work is a novel theoretical result that enables the accurate and closed-form construction of these optimal LoRA components. We validate the efficacy of CLoQ across multiple tasks such as language generation, arithmetic reasoning, and commonsense reasoning, demonstrating that it consistently outperforms existing LoRA fine-tuning methods for quantized LLMs, especially at ultra low-bit widths.

Title: Track-On: Transformer-based Online Point Tracking with Memory

• Authors: Görkay Aydemir, Xiongyi Cai, Weidi Xie, Fatma Güney
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.18487
• Pdf URL: https://arxiv.org/pdf/2501.18487
• Copy Paste: [[2501.18487]] Track-On: Transformer-based Online Point Tracking with Memory(https://arxiv.org/abs/2501.18487)
• Keywords: robust, transformer
• Abstract: In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across multiple frames in a video, despite changes in appearance, lighting, perspective, and occlusions. We target online tracking on a frame-by-frame basis, making it suitable for real-world, streaming scenarios. Specifically, we introduce Track-On, a simple transformer-based model designed for online long-term point tracking. Unlike prior methods that depend on full temporal modeling, our model processes video frames causally without access to future frames, leveraging two memory modules -- spatial memory and context memory -- to capture temporal information and maintain reliable point tracking over long time horizons. At inference time, it employs patch classification and refinement to identify correspondences and track points with high accuracy. Through extensive experiments, we demonstrate that Track-On sets a new state-of-the-art for online models and delivers superior or competitive results compared to offline approaches on seven datasets, including the TAP-Vid benchmark. Our method offers a robust and scalable solution for real-time tracking in diverse applications. Project page: this https URL

Title: GuardReasoner: Towards Reasoning-based LLM Safeguards

• Authors: Yue Liu, Hongcheng Gao, Shengfang Zhai, Jun Xia, Tianyi Wu, Zhiwei Xue, Yulin Chen, Kenji Kawaguchi, Jiaheng Zhang, Bryan Hooi
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18492
• Pdf URL: https://arxiv.org/pdf/2501.18492
• Copy Paste: [[2501.18492]] GuardReasoner: Towards Reasoning-based LLM Safeguards(https://arxiv.org/abs/2501.18492)
• Keywords: explainability
• Abstract: As LLMs increasingly impact safety-critical applications, ensuring their safety using guardrails remains a key challenge. This paper proposes GuardReasoner, a new safeguard for LLMs, by guiding the guard model to learn to reason. Concretely, we first create the GuardReasonerTrain dataset, which consists of 127K samples with 460K detailed reasoning steps. Then, we introduce reasoning SFT to unlock the reasoning capability of guard models. In addition, we present hard sample DPO to further strengthen their reasoning ability. In this manner, GuardReasoner achieves better performance, explainability, and generalizability. Extensive experiments and analyses on 13 benchmarks of 3 guardrail tasks demonstrate its superiority. Remarkably, GuardReasoner 8B surpasses GPT-4o+CoT by 5.74% and LLaMA Guard 3 8B by 20.84% F1 score on average. We release the training data, code, and models with different scales (1B, 3B, 8B) of GuardReasoner : this https URL.

Title: Runway vs. Taxiway: Challenges in Automated Line Identification and Notation Approaches

• Authors: Parth Ganeriwala, Amy Alvarez, Abdullah AlQahtani, Siddhartha Bhattacharyya, Mohammed Abdul Hafeez Khan, Natasha Neogi
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18494
• Pdf URL: https://arxiv.org/pdf/2501.18494
• Copy Paste: [[2501.18494]] Runway vs. Taxiway: Challenges in Automated Line Identification and Notation Approaches(https://arxiv.org/abs/2501.18494)
• Keywords: robust
• Abstract: The increasing complexity of autonomous systems has amplified the need for accurate and reliable labeling of runway and taxiway markings to ensure operational safety. Precise detection and labeling of these markings are critical for tasks such as navigation, landing assistance, and ground control automation. Existing labeling algorithms, like the Automated Line Identification and Notation Algorithm (ALINA), have demonstrated success in identifying taxiway markings but encounter significant challenges when applied to runway markings. This limitation arises due to notable differences in line characteristics, environmental context, and interference from elements such as shadows, tire marks, and varying surface conditions. To address these challenges, we modified ALINA by adjusting color thresholds and refining region of interest (ROI) selection to better suit runway-specific contexts. While these modifications yielded limited improvements, the algorithm still struggled with consistent runway identification, often mislabeling elements such as the horizon or non-relevant background features. This highlighted the need for a more robust solution capable of adapting to diverse visual interferences. In this paper, we propose integrating a classification step using a Convolutional Neural Network (CNN) named AssistNet. By incorporating this classification step, the detection pipeline becomes more resilient to environmental variations and misclassifications. This work not only identifies the challenges but also outlines solutions, paving the way for improved automated labeling techniques essential for autonomous aviation systems.

Title: CLEAR: Cue Learning using Evolution for Accurate Recognition Applied to Sustainability Data Extraction

• Authors: Peter J. Bentley, Soo Ling Lim, Fuyuki Ishikawa
• Subjects: cs.CV, cs.AI, cs.NE
• Abstract URL: https://arxiv.org/abs/2501.18504
• Pdf URL: https://arxiv.org/pdf/2501.18504
• Copy Paste: [[2501.18504]] CLEAR: Cue Learning using Evolution for Accurate Recognition Applied to Sustainability Data Extraction(https://arxiv.org/abs/2501.18504)
• Keywords: extraction, large language model
• Abstract: Large Language Model (LLM) image recognition is a powerful tool for extracting data from images, but accuracy depends on providing sufficient cues in the prompt - requiring a domain expert for specialized tasks. We introduce Cue Learning using Evolution for Accurate Recognition (CLEAR), which uses a combination of LLMs and evolutionary computation to generate and optimize cues such that recognition of specialized features in images is improved. It achieves this by auto-generating a novel domain-specific representation and then using it to optimize suitable textual cues with a genetic algorithm. We apply CLEAR to the real-world task of identifying sustainability data from interior and exterior images of buildings. We investigate the effects of using a variable-length representation compared to fixed-length and show how LLM consistency can be improved by refactoring from categorical to real-valued estimates. We show that CLEAR enables higher accuracy compared to expert human recognition and human-authored prompts in every task with error rates improved by up to two orders of magnitude and an ablation study evincing solution concision.

Title: Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch

• Authors: Arthur Douillard, Yanislav Donchev, Keith Rush, Satyen Kale, Zachary Charles, Zachary Garrett, Gabriel Teston, Dave Lacey, Ross McIlroy, Jiajun Shen, Alexandre Ramé, Arthur Szlam, Marc'Aurelio Ranzato, Paul Barham
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18512
• Pdf URL: https://arxiv.org/pdf/2501.18512
• Copy Paste: [[2501.18512]] Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch(https://arxiv.org/abs/2501.18512)
• Keywords: large language model
• Abstract: Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need to be exchanged at each and every single gradient step, all devices need to be co-located using low-latency high-bandwidth communication links to support the required high volume of exchanged bits. Recently, distributed algorithms like DiLoCo have relaxed such co-location constraint: accelerators can be grouped into ``workers'', where synchronizations between workers only occur infrequently. This in turn means that workers can afford being connected by lower bandwidth communication links without affecting learning quality. However, in these methods, communication across workers still requires the same peak bandwidth as before, as the synchronizations require all parameters to be exchanged across all workers. In this paper, we improve DiLoCo in three ways. First, we synchronize only subsets of parameters in sequence, rather than all at once, which greatly reduces peak bandwidth. Second, we allow workers to continue training while synchronizing, which decreases wall clock time. Third, we quantize the data exchanged by workers, which further reduces bandwidth across workers. By properly combining these modifications, we show experimentally that we can distribute training of billion-scale parameters and reach similar quality as before, but reducing required bandwidth by two orders of magnitude.

Title: Differentially Private Steering for Large Language Model Alignment

• Authors: Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18532
• Pdf URL: https://arxiv.org/pdf/2501.18532
• Copy Paste: [[2501.18532]] Differentially Private Steering for Large Language Model Alignment(https://arxiv.org/abs/2501.18532)
• Keywords: privacy, attack, membership infer, large language model
• Abstract: Aligning Large Language Models (LLMs) with human values and away from undesirable behaviors (such as hallucination) has become increasingly important. Recently, steering LLMs towards a desired behavior via activation editing has emerged as an effective method to mitigate harmful generations at inference-time. Activation editing modifies LLM representations by preserving information from positive demonstrations (e.g., truthful) and minimising information from negative demonstrations (e.g., hallucinations). When these demonstrations come from a private dataset, the aligned LLM may leak private information contained in those private samples. In this work, we present the first study of aligning LLM behavior with private datasets. Our work proposes the \textit{\underline{P}rivate \underline{S}teering for LLM \underline{A}lignment (PSA)} algorithm to edit LLM activations with differential privacy (DP) guarantees. We conduct extensive experiments on seven different benchmarks with open-source LLMs of different sizes (0.5B to 7B) and model families (LlaMa, Qwen, Mistral and Gemma). Our results show that PSA achieves DP guarantees for LLM alignment with minimal loss in performance, including alignment metrics, open-ended text generation quality, and general-purpose reasoning. We also develop the first Membership Inference Attack (MIA) for evaluating and auditing the empirical privacy for the problem of LLM steering via activation editing. Our attack is tailored for activation editing and relies solely on the generated texts without their associated probabilities. Our experiments support the theoretical guarantees by showing improved guarantees for our \textit{PSA} algorithm compared to several existing non-private techniques.

Title: Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models

• Authors: Yi Ding, Lijun Li, Bing Cao, Jing Shao
• Subjects: cs.CV, cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18533
• Pdf URL: https://arxiv.org/pdf/2501.18533
• Copy Paste: [[2501.18533]] Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models(https://arxiv.org/abs/2501.18533)
• Keywords: attack
• Abstract: Large Vision-Language Models (VLMs) have achieved remarkable performance across a wide range of tasks. However, their deployment in safety-critical domains poses significant challenges. Existing safety fine-tuning methods, which focus on textual or multimodal content, fall short in addressing challenging cases or disrupt the balance between helpfulness and harmlessness. Our evaluation highlights a safety reasoning gap: these methods lack safety visual reasoning ability, leading to such bottlenecks. To address this limitation and enhance both visual perception and reasoning in safety-critical contexts, we propose a novel dataset that integrates multi-image inputs with safety Chain-of-Thought (CoT) labels as fine-grained reasoning logic to improve model performance. Specifically, we introduce the Multi-Image Safety (MIS) dataset, an instruction-following dataset tailored for multi-image safety scenarios, consisting of training and test splits. Our experiments demonstrate that fine-tuning InternVL2.5-8B with MIS significantly outperforms both powerful open-source models and API-based models in challenging multi-image tasks requiring safety-related visual reasoning. This approach not only delivers exceptional safety performance but also preserves general capabilities without any trade-offs. Specifically, fine-tuning with MIS increases average accuracy by 0.83% across five general benchmarks and reduces the Attack Success Rate (ASR) on multiple safety benchmarks by a large margin. Data and Models are released under: \href{this https URL}{\texttt{this https URL}}

Title: A Hybrid Data-Driven Approach For Analyzing And Predicting Inpatient Length Of Stay In Health Centre

• Authors: Tasfia Noor Chowdhury, Sanjida Afrin Mou, Kazi Naimur Rahman
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18535
• Pdf URL: https://arxiv.org/pdf/2501.18535
• Copy Paste: [[2501.18535]] A Hybrid Data-Driven Approach For Analyzing And Predicting Inpatient Length Of Stay In Health Centre(https://arxiv.org/abs/2501.18535)
• Keywords: robust
• Abstract: Patient length of stay (LoS) is a critical metric for evaluating the efficacy of hospital management. The primary objectives encompass to improve efficiency and reduce costs while enhancing patient outcomes and hospital capacity within the patient journey. By seamlessly merging data-driven techniques with simulation methodologies, the study proposes an all-encompassing framework for the optimization of patient flow. Using a comprehensive dataset of 2.3 million de-identified patient records, we analyzed demographics, diagnoses, treatments, services, costs, and charges with machine learning models (Decision Tree, Logistic Regression, Random Forest, Adaboost, LightGBM) and Python tools (Spark, AWS clusters, dimensionality reduction). Our model predicts patient length of stay (LoS) upon admission using supervised learning algorithms. This hybrid approach enables the identification of key factors influencing LoS, offering a robust framework for hospitals to streamline patient flow and resource utilization. The research focuses on patient flow, corroborating the efficacy of the approach, illustrating decreased patient length of stay within a real healthcare environment. The findings underscore the potential of hybrid data-driven models in transforming hospital management practices. This innovative methodology provides generally flexible decision-making, training, and patient flow enhancement; such a system could have huge implications for healthcare administration and overall satisfaction with healthcare.

Title: Learning Priors of Human Motion With Vision Transformers

• Authors: Placido Falqueto, Alberto Sanfeliu, Luigi Palopoli, Daniele Fontanelli
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.18543
• Pdf URL: https://arxiv.org/pdf/2501.18543
• Copy Paste: [[2501.18543]] Learning Priors of Human Motion With Vision Transformers(https://arxiv.org/abs/2501.18543)
• Keywords: transformer
• Abstract: A clear understanding of where humans move in a scenario, their usual paths and speeds, and where they stop, is very important for different applications, such as mobility studies in urban areas or robot navigation tasks within human-populated environments. We propose in this article, a neural architecture based on Vision Transformers (ViTs) to provide this information. This solution can arguably capture spatial correlations more effectively than Convolutional Neural Networks (CNNs). In the paper, we describe the methodology and proposed neural architecture and show the experiments' results with a standard dataset. We show that the proposed ViT architecture improves the metrics compared to a method based on a CNN.

Title: CryptoDNA: A Machine Learning Paradigm for DDoS Detection in Healthcare IoT, Inspired by crypto jacking prevention Models

• Authors: Zag ElSayed, Ahmed Abdelgawad, Nelly Elsayed
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.18549
• Pdf URL: https://arxiv.org/pdf/2501.18549
• Copy Paste: [[2501.18549]] CryptoDNA: A Machine Learning Paradigm for DDoS Detection in Healthcare IoT, Inspired by crypto jacking prevention Models(https://arxiv.org/abs/2501.18549)
• Keywords: security, defense, attack, robust
• Abstract: The rapid integration of the Internet of Things (IoT) and Internet of Medical (IoM) devices in the healthcare industry has markedly improved patient care and hospital operations but has concurrently brought substantial risks. Distributed Denial-of-Service (DDoS) attacks present significant dangers, jeopardizing operational stability and patient safety. This study introduces CryptoDNA, an innovative machine learning detection framework influenced by cryptojacking detection methods, designed to identify and alleviate DDoS attacks in healthcare IoT settings. The proposed approach relies on behavioral analytics, including atypical resource usage and network activity patterns. Key features derived from cryptojacking-inspired methodologies include entropy-based analysis of traffic, time-series monitoring of device performance, and dynamic anomaly detection. A lightweight architecture ensures inter-compatibility with resource-constrained IoT devices while maintaining high detection accuracy. The proposed architecture and model were tested in real-world and synthetic datasets to demonstrate the model's superior performance, achieving over 96% accuracy with minimal computational overhead. Comparative analysis reveals its resilience against emerging attack vectors and scalability across diverse device ecosystems. By bridging principles from cryptojacking and DDoS detection, CryptoDNA offers a robust, innovative solution to fortify the healthcare IoT landscape against evolving cyber threats and highlights the potential of interdisciplinary approaches in adaptive cybersecurity defense mechanisms for critical healthcare infrastructures.

Title: BounTCHA: A CAPTCHA Utilizing Boundary Identification in AI-extended Videos

• Authors: Lehao Lin, Ke Wang, Maha Abdallah, Wei Cai
• Subjects: cs.CR, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2501.18565
• Pdf URL: https://arxiv.org/pdf/2501.18565
• Copy Paste: [[2501.18565]] BounTCHA: A CAPTCHA Utilizing Boundary Identification in AI-extended Videos(https://arxiv.org/abs/2501.18565)
• Keywords: security, defense, attack, robust, large language model
• Abstract: In recent years, the rapid development of artificial intelligence (AI) especially multi-modal Large Language Models (MLLMs), has enabled it to understand text, images, videos, and other multimedia data, allowing AI systems to execute various tasks based on human-provided prompts. However, AI-powered bots have increasingly been able to bypass most existing CAPTCHA systems, posing significant security threats to web applications. This makes the design of new CAPTCHA mechanisms an urgent priority. We observe that humans are highly sensitive to shifts and abrupt changes in videos, while current AI systems still struggle to comprehend and respond to such situations effectively. Based on this observation, we design and implement BounTCHA, a CAPTCHA mechanism that leverages human perception of boundaries in video transitions and disruptions. By utilizing AI's capability to expand original videos with prompts, we introduce unexpected twists and changes to create a pipeline for generating short videos for CAPTCHA purposes. We develop a prototype and conduct experiments to collect data on humans' time biases in boundary identification. This data serves as a basis for distinguishing between human users and bots. Additionally, we perform a detailed security analysis of BounTCHA, demonstrating its resilience against various types of attacks. We hope that BounTCHA will act as a robust defense, safeguarding millions of web applications in the AI-driven era.

Title: Token-Hungry, Yet Precise: DeepSeek R1 Highlights the Need for Multi-Step Reasoning Over Speed in MATH

• Authors: Evgenii Evstafev
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18576
• Pdf URL: https://arxiv.org/pdf/2501.18576
• Copy Paste: [[2501.18576]] Token-Hungry, Yet Precise: DeepSeek R1 Highlights the Need for Multi-Step Reasoning Over Speed in MATH(https://arxiv.org/abs/2501.18576)
• Keywords: large language model
• Abstract: This study investigates the performance of the DeepSeek R1 language model on 30 challenging mathematical problems derived from the MATH dataset, problems that previously proved unsolvable by other models under time constraints. Unlike prior work, this research removes time limitations to explore whether DeepSeek R1's architecture, known for its reliance on token-based reasoning, can achieve accurate solutions through a multi-step process. The study compares DeepSeek R1 with four other models (gemini-1.5-flash-8b, gpt-4o-mini-2024-07-18, llama3.1:8b, and mistral-8b-latest) across 11 temperature settings. Results demonstrate that DeepSeek R1 achieves superior accuracy on these complex problems but generates significantly more tokens than other models, confirming its token-intensive approach. The findings highlight a trade-off between accuracy and efficiency in mathematical problem-solving with large language models: while DeepSeek R1 excels in accuracy, its reliance on extensive token generation may not be optimal for applications requiring rapid responses. The study underscores the importance of considering task-specific requirements when selecting an LLM and emphasizes the role of temperature settings in optimizing performance.

Title: Accuracy and Robustness of Weight-Balancing Methods for Training PINNs

• Authors: Matthieu Barreau, Haoming Shen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18582
• Pdf URL: https://arxiv.org/pdf/2501.18582
• Copy Paste: [[2501.18582]] Accuracy and Robustness of Weight-Balancing Methods for Training PINNs(https://arxiv.org/abs/2501.18582)
• Keywords: robust
• Abstract: Physics-Informed Neural Networks (PINNs) have emerged as powerful tools for integrating physics-based models with data by minimizing both data and physics losses. However, this multi-objective optimization problem is notoriously challenging, with some benchmark problems leading to unfeasible solutions. To address these issues, various strategies have been proposed, including adaptive weight adjustments in the loss function. In this work, we introduce clear definitions of accuracy and robustness in the context of PINNs and propose a novel training algorithm based on the Primal-Dual (PD) optimization framework. Our approach enhances the robustness of PINNs while maintaining comparable performance to existing weight-balancing methods. Numerical experiments demonstrate that the PD method consistently achieves reliable solutions across all investigated cases and can be easily implemented, facilitating its practical adoption. The code is available at this https URL.

Title: Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs

• Authors: Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, Dong Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18585
• Pdf URL: https://arxiv.org/pdf/2501.18585
• Copy Paste: [[2501.18585]] Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs(https://arxiv.org/abs/2501.18585)
• Keywords: large language model
• Abstract: Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable abilities in complex reasoning tasks by scaling test-time compute and exhibiting human-like deep thinking. However, we identify a phenomenon we term underthinking, where o1-like LLMs frequently switch between different reasoning thoughts without sufficiently exploring promising paths to reach a correct solution. This behavior leads to inadequate depth of reasoning and decreased performance, particularly on challenging mathematical problems. To systematically analyze this issue, we conduct experiments on three challenging test sets and two representative open-source o1-like models, revealing that frequent thought switching correlates with incorrect responses. We introduce a novel metric to quantify underthinking by measuring token efficiency in incorrect answers. To address underthinking, we propose a decoding strategy with thought switching penalty TIP that discourages premature transitions between thoughts, encouraging deeper exploration of each reasoning path. Experimental results demonstrate that our approach improves accuracy across challenging datasets without requiring model fine-tuning. Our findings contribute to understanding reasoning inefficiencies in o1-like LLMs and offer a practical solution to enhance their problem-solving capabilities.

Title: DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models

• Authors: Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, Zian Wang
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2501.18590
• Pdf URL: https://arxiv.org/pdf/2501.18590
• Copy Paste: [[2501.18590]] DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models(https://arxiv.org/abs/2501.18590)
• Keywords: diffusion
• Abstract: Understanding and modeling lighting effects are fundamental tasks in computer vision and graphics. Classic physically-based rendering (PBR) accurately simulates the light transport, but relies on precise scene representations--explicit 3D geometry, high-quality material properties, and lighting conditions--that are often impractical to obtain in real-world scenarios. Therefore, we introduce DiffusionRenderer, a neural approach that addresses the dual problem of inverse and forward rendering within a holistic framework. Leveraging powerful video diffusion model priors, the inverse rendering model accurately estimates G-buffers from real-world videos, providing an interface for image editing tasks, and training data for the rendering model. Conversely, our rendering model generates photorealistic images from G-buffers without explicit light transport simulation. Experiments demonstrate that DiffusionRenderer effectively approximates inverse and forwards rendering, consistently outperforming the state-of-the-art. Our model enables practical applications from a single video input--including relighting, material editing, and realistic object insertion.

Title: Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models

• Authors: Hao Dong, Moru Liu, Kaiyang Zhou, Eleni Chatzi, Juho Kannala, Cyrill Stachniss, Olga Fink
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.18592
• Pdf URL: https://arxiv.org/pdf/2501.18592
• Copy Paste: [[2501.18592]] Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models(https://arxiv.org/abs/2501.18592)
• Keywords: segmentation
• Abstract: In real-world scenarios, achieving domain adaptation and generalization poses significant challenges, as models must adapt to or generalize across unknown target distributions. Extending these capabilities to unseen multimodal distributions, i.e., multimodal domain adaptation and generalization, is even more challenging due to the distinct characteristics of different modalities. Significant progress has been made over the years, with applications ranging from action recognition to semantic segmentation. Besides, the recent advent of large-scale pre-trained multimodal foundation models, such as CLIP, has inspired works leveraging these models to enhance adaptation and generalization performances or adapting them to downstream tasks. This survey provides the first comprehensive review of recent advances from traditional approaches to foundation models, covering: (1) Multimodal domain adaptation; (2) Multimodal test-time adaptation; (3) Multimodal domain generalization; (4) Domain adaptation and generalization with the help of multimodal foundation models; and (5) Adaptation of multimodal foundation models. For each topic, we formally define the problem and thoroughly review existing methods. Additionally, we analyze relevant datasets and applications, highlighting open challenges and potential future research directions. We maintain an active repository that contains up-to-date literature at this https URL.

Title: Diffusion Autoencoders are Scalable Image Tokenizers

• Authors: Yinbo Chen, Rohit Girdhar, Xiaolong Wang, Sai Saketh Rambhatla, Ishan Misra
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18593
• Pdf URL: https://arxiv.org/pdf/2501.18593
• Copy Paste: [[2501.18593]] Diffusion Autoencoders are Scalable Image Tokenizers(https://arxiv.org/abs/2501.18593)
• Keywords: diffusion, generative
• Abstract: Tokenizing images into compact visual representations is a key step in learning efficient and high-quality image generative models. We present a simple diffusion tokenizer (DiTo) that learns compact visual representations for image generation models. Our key insight is that a single learning objective, diffusion L2 loss, can be used for training scalable image tokenizers. Since diffusion is already widely used for image generation, our insight greatly simplifies training such tokenizers. In contrast, current state-of-the-art tokenizers rely on an empirically found combination of heuristics and losses, thus requiring a complex training recipe that relies on non-trivially balancing different losses and pretrained supervised models. We show design decisions, along with theoretical grounding, that enable us to scale DiTo for learning competitive image representations. Our results show that DiTo is a simpler, scalable, and self-supervised alternative to the current state-of-the-art image tokenizer which is supervised. DiTo achieves competitive or better quality than state-of-the-art in image reconstruction and downstream image generation tasks.

Title: DeltaLLM: Compress LLMs with Low-Rank Deltas between Shared Weights

• Authors: Liana Mikaelyan, Ayyoob Imani, Mathew Salvaris, Parth Pathak, Mohsen Fayyaz
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18596
• Pdf URL: https://arxiv.org/pdf/2501.18596
• Copy Paste: [[2501.18596]] DeltaLLM: Compress LLMs with Low-Rank Deltas between Shared Weights(https://arxiv.org/abs/2501.18596)
• Keywords: transformer
• Abstract: We introduce DeltaLLM, a new post-training compression technique to reduce the memory footprint of LLMs. We propose an alternative way of structuring LLMs with weight sharing between layers in subsequent Transformer blocks, along with additional low-rank difference matrices between them. For training, we adopt the progressing module replacement method and show that the lightweight training of the low-rank modules with approximately 30M-40M tokens is sufficient to achieve performance on par with LLMs of comparable sizes trained from scratch. We release the resultant models, DeltaLLAMA and DeltaPHI, with a 12% parameter reduction, retaining 90% of the performance of the base Llama and Phi models on common knowledge and reasoning benchmarks. Our method also outperforms compression techniques JointDrop, LaCo, ShortGPT and SliceGPT with the same number of parameters removed. For example, DeltaPhi 2.9B with a 24% reduction achieves similar average zero-shot accuracies as recovery fine-tuned SlicedPhi 3.3B with a 12% reduction, despite being approximately 400M parameters smaller with no fine-tuning applied. This work provides new insights into LLM architecture design and compression methods when storage space is critical.