2025-01-27

Title: GaussMark: A Practical Approach for Structural Watermarking of Language Models

• Authors: Adam Block, Ayush Sekhari, Alexander Rakhlin
• Subjects: cs.CR, cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13941
• Pdf URL: https://arxiv.org/pdf/2501.13941
• Copy Paste: [[2501.13941]] GaussMark: A Practical Approach for Structural Watermarking of Language Models(https://arxiv.org/abs/2501.13941)
• Keywords: robust, watermark, large language model
• Abstract: Recent advances in Large Language Models (LLMs) have led to significant improvements in natural language processing tasks, but their ability to generate human-quality text raises significant ethical and operational concerns in settings where it is important to recognize whether or not a given text was generated by a human. Thus, recent work has focused on developing techniques for watermarking LLM-generated text, i.e., introducing an almost imperceptible signal that allows a provider equipped with a secret key to determine if given text was generated by their model. Current watermarking techniques are often not practical due to concerns with generation latency, detection time, degradation in text quality, or robustness. Many of these drawbacks come from the focus on token-level watermarking, which ignores the inherent structure of text. In this work, we introduce a new scheme, GaussMark, that is simple and efficient to implement, has formal statistical guarantees on its efficacy, comes at no cost in generation latency, and embeds the watermark into the weights of the model itself, providing a structural watermark. Our approach is based on Gaussian independence testing and is motivated by recent empirical observations that minor additive corruptions to LLM weights can result in models of identical (or even improved) quality. We show that by adding a small amount of Gaussian noise to the weights of a given LLM, we can watermark the model in a way that is statistically detectable by a provider who retains the secret key. We provide formal statistical bounds on the validity and power of our procedure. Through an extensive suite of experiments, we demonstrate that GaussMark is reliable, efficient, and relatively robust to corruptions such as insertions, deletions, substitutions, and roundtrip translations and can be instantiated with essentially no loss in model quality.

Title: Fanar: An Arabic-Centric Multimodal Generative AI Platform

• Authors: Fanar Team: Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus'ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, Chaoyi Ruan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13944
• Pdf URL: https://arxiv.org/pdf/2501.13944
• Copy Paste: [[2501.13944]] Fanar: An Arabic-Centric Multimodal Generative AI Platform(https://arxiv.org/abs/2501.13944)
• Keywords: generative, large language model
• Abstract: We present Fanar, a platform for Arabic-centric multimodal generative AI systems, that supports language, speech and image generation tasks. At the heart of Fanar are Fanar Star and Fanar Prime, two highly capable Arabic Large Language Models (LLMs) that are best in the class on well established benchmarks for similar sized models. Fanar Star is a 7B (billion) parameter model that was trained from scratch on nearly 1 trillion clean and deduplicated Arabic, English and Code tokens. Fanar Prime is a 9B parameter model continually trained on the Gemma-2 9B base model on the same 1 trillion token set. Both models are concurrently deployed and designed to address different types of prompts transparently routed through a custom-built orchestrator. The Fanar platform provides many other capabilities including a customized Islamic Retrieval Augmented Generation (RAG) system for handling religious prompts, a Recency RAG for summarizing information about current or recent events that have occurred after the pre-training data cut-off date. The platform provides additional cognitive capabilities including in-house bilingual speech recognition that supports multiple Arabic dialects, voice and image generation that is fine-tuned to better reflect regional characteristics. Finally, Fanar provides an attribution service that can be used to verify the authenticity of fact based generated content. The design, development, and implementation of Fanar was entirely undertaken at Hamad Bin Khalifa University's Qatar Computing Research Institute (QCRI) and was sponsored by Qatar's Ministry of Communications and Information Technology to enable sovereign AI technology development.

Title: Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks

• Authors: Diego Gosmar, Deborah A. Dahl
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.13946
• Pdf URL: https://arxiv.org/pdf/2501.13946
• Copy Paste: [[2501.13946]] Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks(https://arxiv.org/abs/2501.13946)
• Keywords: generative, large language model
• Abstract: Hallucinations remain a significant challenge in current Generative AI models, undermining trust in AI systems and their reliability. This study investigates how orchestrating multiple specialized Artificial Intelligent Agents can help mitigate such hallucinations, with a focus on systems leveraging Natural Language Processing (NLP) to facilitate seamless agent interactions. To achieve this, we design a pipeline that introduces over three hundred prompts, purposefully crafted to induce hallucinations, into a front-end agent. The outputs are then systematically reviewed and refined by second- and third-level agents, each employing distinct large language models and tailored strategies to detect unverified claims, incorporate explicit disclaimers, and clarify speculative content. Additionally, we introduce a set of novel Key Performance Indicators (KPIs) specifically designed to evaluate hallucination score levels. A dedicated fourth-level AI agent is employed to evaluate these KPIs, providing detailed assessments and ensuring accurate quantification of shifts in hallucination-related behaviors. A core component of this investigation is the use of the OVON (Open Voice Network) framework, which relies on universal NLP-based interfaces to transfer contextual information among agents. Through structured JSON messages, each agent communicates its assessment of the hallucination likelihood and the reasons underlying questionable content, thereby enabling the subsequent stage to refine the text without losing context. The results demonstrate that employing multiple specialized agents capable of interoperating with each other through NLP-based agentic frameworks can yield promising outcomes in hallucination mitigation, ultimately bolstering trust within the AI community.

Title: A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods

• Authors: Lilian Some, Wenli Yang, Michael Bain, Byeong Kang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13947
• Pdf URL: https://arxiv.org/pdf/2501.13947
• Copy Paste: [[2501.13947]] A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods(https://arxiv.org/abs/2501.13947)
• Keywords: generative, large language model
• Abstract: The rapid development of artificial intelligence has brought about substantial advancements in the field. One promising direction is the integration of Large Language Models (LLMs) with structured knowledge-based systems. This approach aims to enhance AI capabilities by combining the generative language understanding of LLMs with the precise knowledge representation of structured systems. This survey explores the synergy between LLMs and knowledge bases, focusing on real-world applications and addressing associated technical, operational, and ethical challenges. Through a comprehensive literature review, the study identifies critical issues and evaluates existing solutions. The paper highlights the benefits of integrating generative AI with knowledge bases, including improved data contextualization, enhanced model accuracy, and better utilization of knowledge resources. The findings provide a detailed overview of the current state of research, identify key gaps, and offer actionable recommendations. These insights contribute to advancing AI technologies and support their practical deployment across various sectors.

Title: Longitudinal Abuse and Sentiment Analysis of Hollywood Movie Dialogues using LLMs

• Authors: Rohitash Chandra, Guoxiang Ren, Group-H
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13948
• Pdf URL: https://arxiv.org/pdf/2501.13948
• Copy Paste: [[2501.13948]] Longitudinal Abuse and Sentiment Analysis of Hollywood Movie Dialogues using LLMs(https://arxiv.org/abs/2501.13948)
• Keywords: large language model
• Abstract: Over the past decades, there has been an increasing concern about the prevalence of abusive and violent content in Hollywood movies. This study uses Large Language Models (LLMs) to explore the longitudinal abuse and sentiment analysis of Hollywood Oscar and blockbuster movie dialogues from 1950 to 2024. By employing fine-tuned LLMs, we analyze subtitles for over a thousand movies categorised into four genres to examine the trends and shifts in emotional and abusive content over the past seven decades. Our findings reveal significant temporal changes in movie dialogues, which reflect broader social and cultural influences. Overall, the emotional tendencies in the films are diverse, and the detection of abusive content also exhibits significant fluctuations. The results show a gradual rise in abusive content in recent decades, reflecting social norms and regulatory policy changes. Genres such as thrillers still present a higher frequency of abusive content that emphasises the ongoing narrative role of violence and conflict. At the same time, underlying positive emotions such as humour and optimism remain prevalent in most of the movies. Furthermore, the gradual increase of abusive content in movie dialogues has been significant over the last two decades, where Oscar-nominated movies overtook the top ten blockbusters.

Title: Can OpenAI o1 Reason Well in Ophthalmology? A 6,990-Question Head-to-Head Evaluation Study

• Authors: Sahana Srinivasan, Xuguang Ai, Minjie Zou, Ke Zou, Hyunjae Kim, Thaddaeus Wai Soon Lo, Krithi Pushpanathan, Yiming Kong, Anran Li, Maxwell Singer, Kai Jin, Fares Antaki, David Ziyou Chen, Dianbo Liu, Ron A. Adelman, Qingyu Chen, Yih Chung Tham
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13949
• Pdf URL: https://arxiv.org/pdf/2501.13949
• Copy Paste: [[2501.13949]] Can OpenAI o1 Reason Well in Ophthalmology? A 6,990-Question Head-to-Head Evaluation Study(https://arxiv.org/abs/2501.13949)
• Keywords: large language model
• Abstract: Question: What is the performance and reasoning ability of OpenAI o1 compared to other large language models in addressing ophthalmology-specific questions? Findings: This study evaluated OpenAI o1 and five LLMs using 6,990 ophthalmological questions from MedMCQA. O1 achieved the highest accuracy (0.88) and macro-F1 score but ranked third in reasoning capabilities based on text-generation metrics. Across subtopics, o1 ranked first in ``Lens'' and ``Glaucoma'' but second to GPT-4o in ``Corneal and External Diseases'', ``Vitreous and Retina'' and ``Oculoplastic and Orbital Diseases''. Subgroup analyses showed o1 performed better on queries with longer ground truth explanations. Meaning: O1's reasoning enhancements may not fully extend to ophthalmology, underscoring the need for domain-specific refinements to optimize performance in specialized fields like ophthalmology.

Title: DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention

• Authors: Naga VS Raviteja Chappa, Matthew Shepard, Connor McCurtain, Charlotte McCormick, Page Daniel Dobbs, Khoa Luu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.13950
• Pdf URL: https://arxiv.org/pdf/2501.13950
• Copy Paste: [[2501.13950]] DEFEND: A Large-scale 1M Dataset and Foundation Model for Tobacco Addiction Prevention(https://arxiv.org/abs/2501.13950)
• Keywords: robust
• Abstract: While tobacco advertising innovates at unprecedented speed, traditional surveillance methods remain frozen in time, especially in the context of social media. The lack of large-scale, comprehensive datasets and sophisticated monitoring systems has created a widening gap between industry advancement and public health oversight. This paper addresses this critical challenge by introducing Tobacco-1M, a comprehensive dataset of one million tobacco product images with hierarchical labels spanning 75 product categories, and DEFEND, a novel foundation model for tobacco product understanding. Our approach integrates a Feature Enhancement Module for rich multimodal representation learning, a Local-Global Visual Coherence mechanism for detailed feature discrimination, and an Enhanced Image-Text Alignment strategy for precise product characterization. Experimental results demonstrate DEFEND's superior performance, achieving 83.1% accuracy in product classification and 73.8% in visual question-answering tasks, outperforming existing methods by significant margins. Moreover, the model exhibits robust zero-shot learning capabilities with 45.6% accuracy on novel product categories. This work provides regulatory bodies and public health researchers with powerful tools for monitoring emerging tobacco products and marketing strategies, potentially revolutionizing approaches to tobacco control and public health surveillance.

Title: A Layered Multi-Expert Framework for Long-Context Mental Health Assessments

• Authors: Jinwen Tang, Qiming Guo, Wenbo Sun, Yi Shang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13951
• Pdf URL: https://arxiv.org/pdf/2501.13951
• Copy Paste: [[2501.13951]] A Layered Multi-Expert Framework for Long-Context Mental Health Assessments(https://arxiv.org/abs/2501.13951)
• Keywords: large language model
• Abstract: Long-form mental health assessments pose unique challenges for large language models (LLMs), which often exhibit hallucinations or inconsistent reasoning when handling extended, domain-specific contexts. We introduce Stacked Multi-Model Reasoning (SMMR), a layered framework that leverages multiple LLMs and specialized smaller models as coequal 'experts'. Early layers isolate short, discrete subtasks, while later layers integrate and refine these partial outputs through more advanced long-context models. We evaluate SMMR on the DAIC-WOZ depression-screening dataset and 48 curated case studies with psychiatric diagnoses, demonstrating consistent improvements over single-model baselines in terms of accuracy, F1-score, and PHQ-8 error reduction. By harnessing diverse 'second opinions', SMMR mitigates hallucinations, captures subtle clinical nuances, and enhances reliability in high-stakes mental health assessments. Our findings underscore the value of multi-expert frameworks for more trustworthy AI-driven screening.

Title: The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?

• Authors: Yiyi Zhang, Xingyu Chen, Kexin Chen, Yuyang Du, Xilin Dang, Pheng-Ann Heng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13952
• Pdf URL: https://arxiv.org/pdf/2501.13952
• Copy Paste: [[2501.13952]] The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?(https://arxiv.org/abs/2501.13952)
• Keywords: large language model
• Abstract: Recent years have witnessed extensive efforts to enhance Large Language Models (LLMs) across various domains, alongside growing attention to their ethical implications. However, a critical challenge remains largely overlooked: LLMs must balance between rejecting harmful requests for safety and accommodating legitimate ones for utility. This paper presents a Direct Preference Optimization (DPO) based alignment framework that achieves better overall performance by addressing this ethical-utility trade-off, using chemical domain applications as a proof-of-concept. Our alignment pipeline starts with a GPT-assisted three-phase data generation scheme, in which we create LibraChemQA, a chemical question-answering dataset comprising 31.6k triplet instances. By incorporating an innovative balanced seed in the data generation process, our framework systematically considers both legitimate and illegitimate requests. The framework also introduces a rephrasing mechanism for efficient data augmentation that enhances the model's chemical comprehension. We further develop a novel hybrid evaluation scheme with LLM judges for precise assessment of both safety and utility. Experimental results demonstrate our model's substantial improvements in overall performance where both safety and utility are considered - our resulting model, LibraChem, outperforms leading LLMs including Claude-3, GPT-4o, and LLaMA-3 by margins of 13.44%, 7.16%, and 7.10% respectively on our released benchmark.

Title: Redundancy Principles for MLLMs Benchmarks

• Authors: Zicheng Zhang, Xiangyu Zhao, Xinyu Fang, Chunyi Li, Xiaohong Liu, Xiongkuo Min, Haodong Duan, Kai Chen, Guangtao Zhai
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13953
• Pdf URL: https://arxiv.org/pdf/2501.13953
• Copy Paste: [[2501.13953]] Redundancy Principles for MLLMs Benchmarks(https://arxiv.org/abs/2501.13953)
• Keywords: large language model
• Abstract: With the rapid iteration of Multi-modality Large Language Models (MLLMs) and the evolving demands of the field, the number of benchmarks produced annually has surged into the hundreds. The rapid growth has inevitably led to significant redundancy among benchmarks. Therefore, it is crucial to take a step back and critically assess the current state of redundancy and propose targeted principles for constructing effective MLLM benchmarks. In this paper, we focus on redundancy from three key perspectives: 1) Redundancy of benchmark capability dimensions, 2) Redundancy in the number of test questions, and 3) Cross-benchmark redundancy within specific domains. Through the comprehensive analysis over hundreds of MLLMs' performance across more than 20 benchmarks, we aim to quantitatively measure the level of redundancy lies in existing MLLM evaluations, provide valuable insights to guide the future development of MLLM benchmarks, and offer strategies to refine and address redundancy issues effectively.

Title: Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documents

• Authors: Long Huang, Ming Zhao, Limin Xiao, Xiujun Zhang, Jungang Hu
• Subjects: cs.CL, cs.AI, cs.DC, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.13954
• Pdf URL: https://arxiv.org/pdf/2501.13954
• Copy Paste: [[2501.13954]] Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documents(https://arxiv.org/abs/2501.13954)
• Keywords: large language model
• Abstract: The 3rd Generation Partnership Project (3GPP) documents is key standards in global telecommunications, while posing significant challenges for engineers and researchers in the telecommunications field due to the large volume and complexity of their contents as well as the frequent updates. Large language models (LLMs) have shown promise in natural language processing tasks, but their general-purpose nature limits their effectiveness in specific domains like telecommunications. To address this, we propose Chat3GPP, an open-source retrieval-augmented generation (RAG) framework tailored for 3GPP specifications. By combining chunking strategies, hybrid retrieval and efficient indexing methods, Chat3GPP can efficiently retrieve relevant information and generate accurate responses to user queries without requiring domain-specific fine-tuning, which is both flexible and scalable, offering significant potential for adapting to other technical standards beyond 3GPP. We evaluate Chat3GPP on two telecom-specific datasets and demonstrate its superior performance compared to existing methods, showcasing its potential for downstream tasks like protocol generation and code automation.

Title: Guided Persona-based AI Surveys: Can we replicate personal mobility preferences at scale using LLMs?

• Authors: Ioannis Tzachristas, Santhanakrishnan Narayanan, Constantinos Antoniou
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.13955
• Pdf URL: https://arxiv.org/pdf/2501.13955
• Copy Paste: [[2501.13955]] Guided Persona-based AI Surveys: Can we replicate personal mobility preferences at scale using LLMs?(https://arxiv.org/abs/2501.13955)
• Keywords: privacy, large language model
• Abstract: This study explores the potential of Large Language Models (LLMs) to generate artificial surveys, with a focus on personal mobility preferences in Germany. By leveraging LLMs for synthetic data creation, we aim to address the limitations of traditional survey methods, such as high costs, inefficiency and scalability challenges. A novel approach incorporating "Personas" - combinations of demographic and behavioural attributes - is introduced and compared to five other synthetic survey methods, which vary in their use of real-world data and methodological complexity. The MiD 2017 dataset, a comprehensive mobility survey in Germany, serves as a benchmark to assess the alignment of synthetic data with real-world patterns. The results demonstrate that LLMs can effectively capture complex dependencies between demographic attributes and preferences while offering flexibility to explore hypothetical scenarios. This approach presents valuable opportunities for transportation planning and social science research, enabling scalable, cost-efficient and privacy-preserving data generation.

Title: Zep: A Temporal Knowledge Graph Architecture for Agent Memory

• Authors: Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.13956
• Pdf URL: https://arxiv.org/pdf/2501.13956
• Copy Paste: [[2501.13956]] Zep: A Temporal Knowledge Graph Architecture for Agent Memory(https://arxiv.org/abs/2501.13956)
• Keywords: large language model
• Abstract: We introduce Zep, a novel memory layer service for AI agents that outperforms the current state-of-the-art system, MemGPT, in the Deep Memory Retrieval (DMR) benchmark. Additionally, Zep excels in more comprehensive and challenging evaluations than DMR that better reflect real-world enterprise use cases. While existing retrieval-augmented generation (RAG) frameworks for large language model (LLM)-based agents are limited to static document retrieval, enterprise applications demand dynamic knowledge integration from diverse sources including ongoing conversations and business data. Zep addresses this fundamental limitation through its core component Graphiti -- a temporally-aware knowledge graph engine that dynamically synthesizes both unstructured conversational data and structured business data while maintaining historical relationships. In the DMR benchmark, which the MemGPT team established as their primary evaluation metric, Zep demonstrates superior performance (94.8% vs 93.4%). Beyond DMR, Zep's capabilities are further validated through the more challenging LongMemEval benchmark, which better reflects enterprise use cases through complex temporal reasoning tasks. In this evaluation, Zep achieves substantial results with accuracy improvements of up to 18.5% while simultaneously reducing response latency by 90% compared to baseline implementations. These results are particularly pronounced in enterprise-critical tasks such as cross-session information synthesis and long-term context maintenance, demonstrating Zep's effectiveness for deployment in real-world applications.

Title: Benchmarking Generative AI for Scoring Medical Student Interviews in Objective Structured Clinical Examinations (OSCEs)

• Authors: Jadon Geathers, Yann Hicke, Colleen Chan, Niroop Rajashekar, Justin Sewell, Susannah Cornes, Rene Kizilcec, Dennis Shung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13957
• Pdf URL: https://arxiv.org/pdf/2501.13957
• Copy Paste: [[2501.13957]] Benchmarking Generative AI for Scoring Medical Student Interviews in Objective Structured Clinical Examinations (OSCEs)(https://arxiv.org/abs/2501.13957)
• Keywords: generative, large language model
• Abstract: Introduction. Objective Structured Clinical Examinations (OSCEs) are widely used to assess medical students' communication skills, but scoring interview-based assessments is time-consuming and potentially subject to human bias. This study explored the potential of large language models (LLMs) to automate OSCE evaluations using the Master Interview Rating Scale (MIRS). Methods. We compared the performance of four state-of-the-art LLMs (GPT-4o, Claude 3.5, Llama 3.1, and Gemini 1.5 Pro) in evaluating OSCE transcripts across all 28 items of the MIRS under the conditions of zero-shot, chain-of-thought (CoT), few-shot, and multi-step prompting. The models were benchmarked against a dataset of 10 OSCE cases with 174 expert consensus scores available. Model performance was measured using three accuracy metrics (exact, off-by-one, thresholded). Results. Averaging across all MIRS items and OSCE cases, LLMs performed with low exact accuracy (0.27 to 0.44), and moderate to high off-by-one accuracy (0.67 to 0.87) and thresholded accuracy (0.75 to 0.88). A zero temperature parameter ensured high intra-rater reliability ($\alpha = 0.98$ for GPT-4o). CoT, few-shot, and multi-step techniques proved valuable when tailored to specific assessment items. The performance was consistent across MIRS items independent of encounter phases and communication domains. Conclusion. We demonstrated the feasibility of AI-assisted OSCE evaluation and provided benchmarking of multiple LLMs across multiple prompt techniques. Our work provides a baseline performance assessment for LLMs that lays a foundation for future research in automated assessment of clinical communication skills.

Title: A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models

• Authors: Qinggang Zhang, Shengyuan Chen, Yuanchen Bei, Zheng Yuan, Huachi Zhou, Zijin Hong, Junnan Dong, Hao Chen, Yi Chang, Xiao Huang
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.13958
• Pdf URL: https://arxiv.org/pdf/2501.13958
• Copy Paste: [[2501.13958]] A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models(https://arxiv.org/abs/2501.13958)
• Keywords: large language model
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in a wide range of tasks, yet their application to specialized domains remains challenging due to the need for deep expertise. Retrieval-augmented generation (RAG) has emerged as a promising solution to customize LLMs for professional fields by seamlessly integrating external knowledge bases, enabling real-time access to domain-specific expertise during inference. Despite its potential, traditional RAG systems, based on flat text retrieval, face three critical challenges: (i) complex query understanding in professional contexts, (ii) difficulties in knowledge integration across distributed sources, and (iii) system efficiency bottlenecks at scale. This survey presents a systematic analysis of Graph-based Retrieval-Augmented Generation (GraphRAG), a new paradigm that revolutionizes domain-specific LLM applications. GraphRAG addresses traditional RAG limitations through three key innovations: (i) graph-structured knowledge representation that explicitly captures entity relationships and domain hierarchies, (ii) efficient graph-based retrieval techniques that enable context-preserving knowledge retrieval with multihop reasoning ability, and (iii) structure-aware knowledge integration algorithms that leverage retrieved knowledge for accurate and logical coherent generation of LLMs. In this survey, we systematically analyze the technical foundations of GraphRAG and examine current implementations across various professional domains, identifying key technical challenges and promising research directions. All the related resources of GraphRAG, including research papers, open-source data, and projects, are collected for the community in \textcolor{blue}{\url{this https URL}}.

Title: A Fast, Scalable, and Robust Deep Learning-based Iterative Reconstruction Framework for Accelerated Industrial Cone-beam X-ray Computed Tomography

• Authors: Aniket Pramanik, Obaidullah Rahman, Singanallur V. Venkatakrishnan, Amirkoushyar Ziabari
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13961
• Pdf URL: https://arxiv.org/pdf/2501.13961
• Copy Paste: [[2501.13961]] A Fast, Scalable, and Robust Deep Learning-based Iterative Reconstruction Framework for Accelerated Industrial Cone-beam X-ray Computed Tomography(https://arxiv.org/abs/2501.13961)
• Keywords: robust
• Abstract: Cone-beam X-ray Computed Tomography (XCT) with large detectors and corresponding large-scale 3D reconstruction plays a pivotal role in micron-scale characterization of materials and parts across various industries. In this work, we present a novel deep neural network-based iterative algorithm that integrates an artifact reduction-trained CNN as a prior model with automated regularization parameter selection, tailored for large-scale industrial cone-beam XCT data. Our method achieves high-quality 3D reconstructions even for extremely dense thick metal parts - which traditionally pose challenges to industrial CT images - in just a few iterations. Furthermore, we show the generalizability of our approach to out-of-distribution scans obtained under diverse scanning conditions. Our method effectively handles significant noise and streak artifacts, surpassing state-of-the-art supervised learning methods trained on the same data.

Title: Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models

• Authors: Afrah Gueriani, Hamza Kheddar, Ahmed Cherif Mazari
• Subjects: cs.CR, cs.AI, cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.13962
• Pdf URL: https://arxiv.org/pdf/2501.13962
• Copy Paste: [[2501.13962]] Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models(https://arxiv.org/abs/2501.13962)
• Keywords: attack
• Abstract: The rapid expansion of the industrial Internet of things (IIoT) has introduced new challenges in securing critical infrastructures against sophisticated cyberthreats. This study presents the development and evaluation of an advanced Intrusion detection (IDS) based on a hybrid LSTM-convolution neural network (CNN)-Attention architecture, specifically designed to detect and classify cyberattacks in IIoT environments. The research focuses on two key classification tasks: binary and multi-class classification. The proposed models was rigorously tested using the Edge-IIoTset dataset. To mitigate the class imbalance in the dataset, the synthetic minority over-sampling technique (SMOTE) was employed to generate synthetic samples for the underrepresented classes. This ensured that the model could learn effectively from all classes, thereby improving the overall classification performance. Through systematic experimentation, various deep learning (DL) models were compared, ultimately demonstrating that the LSTM-CNN-Attention model consistently outperformed others across key performance metrics. In binary classification, the model achieved near-perfect accuracy, while in multi-class classification, it maintained a high accuracy level (99.04%), effectively categorizing different attack types with a loss value of 0.0220%.

Title: Procedural Generation of 3D Maize Plant Architecture from LIDAR Data

• Authors: Mozhgan Hadadi, Mehdi Saraeian, Jackson Godbersen, Talukder Jubery, Yawei Li, Lakshmi Attigala, Aditya Balu, Soumik Sarkar, Patrick S. Schnable, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13963
• Pdf URL: https://arxiv.org/pdf/2501.13963
• Copy Paste: [[2501.13963]] Procedural Generation of 3D Maize Plant Architecture from LIDAR Data(https://arxiv.org/abs/2501.13963)
• Keywords: robust, extraction
• Abstract: This study introduces a robust framework for generating procedural 3D models of maize (Zea mays) plants from LiDAR point cloud data, offering a scalable alternative to traditional field-based phenotyping. Our framework leverages Non-Uniform Rational B-Spline (NURBS) surfaces to model the leaves of maize plants, combining Particle Swarm Optimization (PSO) for an initial approximation of the surface and a differentiable programming framework for precise refinement of the surface to fit the point cloud data. In the first optimization phase, PSO generates an approximate NURBS surface by optimizing its control points, aligning the surface with the LiDAR data, and providing a reliable starting point for refinement. The second phase uses NURBS-Diff, a differentiable programming framework, to enhance the accuracy of the initial fit by refining the surface geometry and capturing intricate leaf details. Our results demonstrate that, while PSO establishes a robust initial fit, the integration of differentiable NURBS significantly improves the overall quality and fidelity of the reconstructed surface. This hierarchical optimization strategy enables accurate 3D reconstruction of maize leaves across diverse genotypes, facilitating the subsequent extraction of complex traits like phyllotaxy. We demonstrate our approach on diverse genotypes of field-grown maize plants. All our codes are open-source to democratize these phenotyping approaches.

Title: ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification

• Authors: Bidhan Roy, Peter Potash, Marcos Villagra
• Subjects: cs.CR, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13965
• Pdf URL: https://arxiv.org/pdf/2501.13965
• Copy Paste: [[2501.13965]] ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification(https://arxiv.org/abs/2501.13965)
• Keywords: secure, large language model
• Abstract: Low-Rank Adaptation (LoRA) is a widely adopted method for customizing large-scale language models. In distributed, untrusted training environments, an open source base model user may want to use LoRA weights created by an external contributor, leading to two requirements: (1) the base model user must confirm that the LoRA weights are effective when paired with the intended base model, and (2) the LoRA contributor must keep their proprietary weights private until compensation is assured. We present ZKLoRA, a zero-knowledge verification protocol that relies on succinct proofs and our novel Multi-Party Inference procedure to verify LoRA-base model compatibility without exposing LoRA weights. ZKLoRA produces deterministic correctness guarantees and validates each LoRA module in only 1-2 seconds on state-of-the-art large language models. This low-latency approach enables nearly real-time verification and promotes secure collaboration among geographically decentralized teams and contract-based training pipelines. The protocol ensures that the delivered LoRA module works as claimed, safeguarding the contributor's intellectual property while providing the base model user with verification of compatibility and lineage.

Title: FedDAG: Federated Domain Adversarial Generation Towards Generalizable Medical Image Analysis

• Authors: Haoxuan Che, Yifei Wu, Haibo Jin, Yong Xia, Hao Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13967
• Pdf URL: https://arxiv.org/pdf/2501.13967
• Copy Paste: [[2501.13967]] FedDAG: Federated Domain Adversarial Generation Towards Generalizable Medical Image Analysis(https://arxiv.org/abs/2501.13967)
• Keywords: federate
• Abstract: Federated domain generalization aims to train a global model from multiple source domains and ensure its generalization ability to unseen target domains. {Due to the target domain being with unknown domain shifts, attempting to approximate these gaps by source domains may be the key to improving model generalization capability.} Existing works mainly focus on sharing and recombining local domain-specific attributes to increase data diversity and simulate potential domain shifts. {However, these methods may be insufficient since only the local attribute recombination can be hard to touch the out-of-distribution of global data.} In this paper, we propose a simple-yet-efficient framework named Federated Domain Adversarial Generation (FedDAG). {It aims to simulate the domain shift and improve the model generalization by adversarially generating novel domains different from local and global source domains.} Specifically, it generates novel-style images by maximizing the instance-level feature discrepancy between original and generated images and trains a generalizable task model by minimizing their feature discrepancy. {Further, we observed that FedDAG could cause different performance improvements for local models. It may be due to inherent data isolation and heterogeneity among clients, exacerbating the imbalance in their generalization contributions to the global model.} {Ignoring this imbalance can lead the global model's generalization ability to be sub-optimal, further limiting the novel domain generation procedure. } Thus, to mitigate this imbalance, FedDAG hierarchically aggregates local models at the within-client and across-client levels by using the sharpness concept to evaluate client model generalization contributions. {Extensive experiments across four medical benchmarks demonstrate FedDAG's ability to enhance generalization in federated medical scenarios.}

Title: InsTex: Indoor Scenes Stylized Texture Synthesis

• Authors: Yunfan Zhang, Zhiwei Xiong, Zhiqi Shen, Guosheng Lin, Hao Wang, Nicolas Vun
• Subjects: cs.CV, cs.GR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13969
• Pdf URL: https://arxiv.org/pdf/2501.13969
• Copy Paste: [[2501.13969]] InsTex: Indoor Scenes Stylized Texture Synthesis(https://arxiv.org/abs/2501.13969)
• Keywords: diffusion, generative
• Abstract: Generating high-quality textures for 3D scenes is crucial for applications in interior design, gaming, and augmented/virtual reality (AR/VR). Although recent advancements in 3D generative models have enhanced content creation, significant challenges remain in achieving broad generalization and maintaining style consistency across multiple viewpoints. Current methods, such as 2D diffusion models adapted for 3D texturing, suffer from lengthy processing times and visual artifacts, while approaches driven by 3D data often fail to generalize effectively. To overcome these challenges, we introduce InsTex, a two-stage architecture designed to generate high-quality, style-consistent textures for 3D indoor scenes. InsTex utilizes depth-to-image diffusion priors in a coarse-to-fine pipeline, first generating multi-view images with a pre-trained 2D diffusion model and subsequently refining the textures for consistency. Our method supports both textual and visual prompts, achieving state-of-the-art results in visual quality and quantitative metrics, and demonstrates its effectiveness across various 3D texturing applications.

Title: Absolute Governance: A Framework for Synchronization and Certification of the Corporate Contractual State

• Authors: Antonio Hoffert
• Subjects: cs.CR, cs.IT
• Abstract URL: https://arxiv.org/abs/2501.13974
• Pdf URL: https://arxiv.org/pdf/2501.13974
• Copy Paste: [[2501.13974]] Absolute Governance: A Framework for Synchronization and Certification of the Corporate Contractual State(https://arxiv.org/abs/2501.13974)
• Keywords: robust
• Abstract: This dissertation addresses the challenge of ensuring transactional integrity and reducing costs in corporate governance through blockchain technology. We propose an on-chain methodology for certifying, registering, and querying institutional transactional status. Our decentralized governance approach utilizes consensus mechanisms and smart contracts to automate and enforce business rules. The framework aims to reduce the transaction costs associated with contractual measurement reports and enhance overall transactional integrity. We provide a detailed exploration of how blockchain technology can be effectively harnessed to offer a robust solution to these challenges, setting the stage for our proposed solution and its potential impact on corporate governance. The application of the methodology resulted in as average of 2% overbilling reduction.

Title: Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models

• Authors: Akash Bonagiri, Lucen Li, Rajvardhan Oak, Zeerak Babar, Magdalena Wojcieszak, Anshuman Chhabra
• Subjects: cs.CL, cs.AI, cs.CY, cs.SI
• Abstract URL: https://arxiv.org/abs/2501.13976
• Pdf URL: https://arxiv.org/pdf/2501.13976
• Copy Paste: [[2501.13976]] Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models(https://arxiv.org/abs/2501.13976)
• Keywords: large language model
• Abstract: The prevalence of harmful content on social media platforms poses significant risks to users and society, necessitating more effective and scalable content moderation strategies. Current approaches rely on human moderators, supervised classifiers, and large volumes of training data, and often struggle with scalability, subjectivity, and the dynamic nature of harmful content (e.g., violent content, dangerous challenge trends, etc.). To bridge these gaps, we utilize Large Language Models (LLMs) to undertake few-shot dynamic content moderation via in-context learning. Through extensive experiments on multiple LLMs, we demonstrate that our few-shot approaches can outperform existing proprietary baselines (Perspective and OpenAI Moderation) as well as prior state-of-the-art few-shot learning methods, in identifying harm. We also incorporate visual information (video thumbnails) and assess if different multimodal techniques improve model performance. Our results underscore the significant benefits of employing LLM based methods for scalable and dynamic harmful content moderation online.

Title: Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms

• Authors: Rajvardhan Oak, Muhammad Haroon, Claire Jo, Magdalena Wojcieszak, Anshuman Chhabra
• Subjects: cs.CL, cs.AI, cs.CY, cs.SI
• Abstract URL: https://arxiv.org/abs/2501.13977
• Pdf URL: https://arxiv.org/pdf/2501.13977
• Copy Paste: [[2501.13977]] Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms(https://arxiv.org/abs/2501.13977)
• Keywords: large language model
• Abstract: Social media platforms utilize Machine Learning (ML) and Artificial Intelligence (AI) powered recommendation algorithms to maximize user engagement, which can result in inadvertent exposure to harmful content. Current moderation efforts, reliant on classifiers trained with extensive human-annotated data, struggle with scalability and adapting to new forms of harm. To address these challenges, we propose a novel re-ranking approach using Large Language Models (LLMs) in zero-shot and few-shot settings. Our method dynamically assesses and re-ranks content sequences, effectively mitigating harmful content exposure without requiring extensive labeled data. Alongside traditional ranking metrics, we also introduce two new metrics to evaluate the effectiveness of re-ranking in reducing exposure to harmful content. Through experiments on three datasets, three models and across three configurations, we demonstrate that our LLM-based approach significantly outperforms existing proprietary moderation approaches, offering a scalable and adaptable solution for harm mitigation.

Title: Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation

• Authors: Sangyeop Yeo, Seung-won Hwang, Yu-Seung Ma
• Subjects: cs.CL, cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.13978
• Pdf URL: https://arxiv.org/pdf/2501.13978
• Copy Paste: [[2501.13978]] Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation(https://arxiv.org/abs/2501.13978)
• Keywords: large language model
• Abstract: The use of Large Language Models (LLMs) for code generation has gained significant attention in recent years. Existing methods often aim to improve the quality of generated code by incorporating additional contextual information or guidance into input prompts. Many of these approaches adopt sequential reasoning strategies, mimicking human-like step-by-step thinking. However, such strategies may constrain flexibility, as they do not always align with the structured characteristics of programming languages. This paper introduces the Chain of Grounded Objectives (CGO), a method that embeds functional objectives into input prompts to enhance code generation. By leveraging appropriately structured objectives as input and avoiding explicit sequential procedures, CGO adapts effectively to the structured nature of programming tasks. Empirical evaluations demonstrate that CGO effectively enhances code generation, addressing limitations of existing approaches.

Title: Enhanced PEC-YOLO for Detecting Improper Safety Gear Wearing Among Power Line Workers

• Authors: Chen Zuguo, Kuang Aowei, Huang Yi, Jin Jie
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.13981
• Pdf URL: https://arxiv.org/pdf/2501.13981
• Copy Paste: [[2501.13981]] Enhanced PEC-YOLO for Detecting Improper Safety Gear Wearing Among Power Line Workers(https://arxiv.org/abs/2501.13981)
• Keywords: extraction
• Abstract: To address the high risks associated with improper use of safety gear in complex power line environments, where target occlusion and large variance are prevalent, this paper proposes an enhanced PEC-YOLO object detection algorithm. The method integrates deep perception with multi-scale feature fusion, utilizing PConv and EMA attention mechanisms to enhance feature extraction efficiency and minimize model complexity. The CPCA attention mechanism is incorporated into the SPPF module, improving the model's ability to focus on critical information and enhance detection accuracy, particularly in challenging conditions. Furthermore, the introduction of the BiFPN neck architecture optimizes the utilization of low-level and high-level features, enhancing feature representation through adaptive fusion and context-aware mechanism. Experimental results demonstrate that the proposed PEC-YOLO achieves a 2.7% improvement in detection accuracy compared to YOLOv8s, while reducing model parameters by 42.58%. Under identical conditions, PEC-YOLO outperforms other models in detection speed, meeting the stringent accuracy requirements for safety gear detection in construction sites. This study contributes to the development of efficient and accurate intelligent monitoring systems for ensuring worker safety in hazardous environments.

Title: AdEval: Alignment-based Dynamic Evaluation to Mitigate Data Contamination in Large Language Models

• Authors: Yang Fan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13983
• Pdf URL: https://arxiv.org/pdf/2501.13983
• Copy Paste: [[2501.13983]] AdEval: Alignment-based Dynamic Evaluation to Mitigate Data Contamination in Large Language Models(https://arxiv.org/abs/2501.13983)
• Keywords: robust, fair, large language model
• Abstract: As Large Language Models (LLMs) are pretrained on massive-scale corpora, the issue of data contamination has become increasingly severe, leading to potential overestimation of model performance during evaluation. To address this, we propose AdEval (Alignment-based Dynamic Evaluation), a dynamic data evaluation method aimed at mitigating the impact of data contamination on evaluation reliability. AdEval extracts key knowledge points and main ideas to align dynamically generated questions with static data's core concepts. It also leverages online search to provide detailed explanations of related knowledge points, thereby creating high-quality evaluation samples with robust knowledge support. Furthermore, AdEval incorporates mechanisms to control the number and complexity of questions, enabling dynamic alignment and flexible adjustment. This ensures that the generated questions align with the complexity of static data while supporting varied complexity levels. Based on Bloom's taxonomy, AdEval conducts a multi-dimensional evaluation of LLMs across six cognitive levels: remembering, understanding, applying, analyzing, evaluating, and creating. Experimental results on multiple datasets demonstrate that AdEval effectively reduces the impact of data contamination on evaluation outcomes, enhancing both the fairness and reliability of the evaluation process.

Title: Comprehensive Modeling and Question Answering of Cancer Clinical Practice Guidelines using LLMs

• Authors: Bhumika Gupta, Pralaypati Ta, Keerthi Ram, Mohanasankar Sivaprakasam
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.13984
• Pdf URL: https://arxiv.org/pdf/2501.13984
• Copy Paste: [[2501.13984]] Comprehensive Modeling and Question Answering of Cancer Clinical Practice Guidelines using LLMs(https://arxiv.org/abs/2501.13984)
• Keywords: extraction, large language model
• Abstract: The updated recommendations on diagnostic procedures and treatment pathways for a medical condition are documented as graphical flows in Clinical Practice Guidelines (CPGs). For effective use of the CPGs in helping medical professionals in the treatment decision process, it is necessary to fully capture the guideline knowledge, particularly the contexts and their relationships in the graph. While several existing works have utilized these guidelines to create rule bases for Clinical Decision Support Systems, limited work has been done toward directly capturing the full medical knowledge contained in CPGs. This work proposes an approach to create a contextually enriched, faithful digital representation of National Comprehensive Cancer Network (NCCN) Cancer CPGs in the form of graphs using automated extraction and node & relationship classification. We also implement semantic enrichment of the model by using Large Language Models (LLMs) for node classification, achieving an accuracy of 80.86% and 88.47% with zero-shot learning and few-shot learning, respectively. Additionally, we introduce a methodology for answering natural language questions with constraints to guideline text by leveraging LLMs to extract the relevant subgraph from the guideline knowledge base. By generating natural language answers based on subgraph paths and semantic information, we mitigate the risk of incorrect answers and hallucination associated with LLMs, ensuring factual accuracy in medical domain Question Answering.

Title: Pilot: Building the Federated Multimodal Instruction Tuning Framework

• Authors: Baochen Xiong, Xiaoshan Yang, Yaguang Song, Yaowei Wang, Changsheng Xu
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.13985
• Pdf URL: https://arxiv.org/pdf/2501.13985
• Copy Paste: [[2501.13985]] Pilot: Building the Federated Multimodal Instruction Tuning Framework(https://arxiv.org/abs/2501.13985)
• Keywords: federate
• Abstract: In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices. To solve the new task, we propose a federated multimodal instruction tuning framework(Pilot). Our framework integrates two stages of "adapter on adapter" into the connector of the vision encoder and the LLM. In stage 1, we extract task-specific features and client-specific features from visual information. In stage 2, we build the cross-task Mixture-of-Adapters(CT-MoA) module to perform cross-task interaction. Each client can not only capture personalized information of local data and learn task-related multimodal information, but also learn general knowledge from other tasks. In addition, we introduce an adaptive parameter aggregation strategy for text training parameters, which optimizes parameter aggregation by calculating weights based on the euclidean distance between parameters, so that parameter aggregation can benefit from positive effects to the greatest extent while effectively reducing negative effects. Our framework can collaboratively exploit distributed data from different local clients to learn cross-task knowledge without being affected by the task heterogeneity during instruction tuning. The effectiveness of our method is verified in two different cross-task scenarios.

Title: OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting

• Authors: Xing Hu, Yuan Cheng, Dawei Yang, Zukang Xu, Zhihang Yuan, Jiangyong Yu, Chen Xu, Zhe Jiang, Sifan Zhou
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13987
• Pdf URL: https://arxiv.org/pdf/2501.13987
• Copy Paste: [[2501.13987]] OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting(https://arxiv.org/abs/2501.13987)
• Keywords: large language model
• Abstract: Post-training quantization (PTQ) has emerged as a widely adopted technique for compressing and accelerating Large Language Models (LLMs). The major challenge in LLM quantization is that uneven and heavy-tailed data distributions can expand the quantization range, thereby reducing bit precision for most values. Recent methods attempt to eliminate outliers and balance inter-channel differences by employing linear transformations; however, they remain heuristic and are often overlook optimizing the data distribution across the entire quantization this http URL this paper, we introduce Quantization Space Utilization Rate (QSUR), a novel metric that effectively assesses the quantizability of transformed data by measuring the space utilization of the data in the quantization space. We complement QSUR with mathematical derivations that examine the effects and limitations of various transformations, guiding our development of Orthogonal and Scaling Transformation-based Quantization (OSTQuant). OSQuant employs a learnable equivalent transformation, consisting of an orthogonal transformation and a scaling transformation, to optimize the distributions of weights and activations across the entire quantization space. Futhermore, we propose the KL-Top loss function, designed to mitigate noise during optimization while retaining richer semantic information within the limited calibration data imposed by PTQ. OSTQuant outperforms existing work on various LLMs and benchmarks. In the W4-only setting, it retains 99.5\% of the floating-point accuracy. In the more challenging W4A4KV4 configuration, OSTQuant reduces the performance gap by 32\% on the LLaMA-3-8B model compared to state-of-the-art methods. \href{this https URL}{this https URL}.

Title: FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting

• Authors: Wenzhen Yue, Yong Liu, Xianghua Ying, Bowei Xing, Ruohao Guo, Ji Shi
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13989
• Pdf URL: https://arxiv.org/pdf/2501.13989
• Copy Paste: [[2501.13989]] FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting(https://arxiv.org/abs/2501.13989)
• Keywords: robust, transformer
• Abstract: This paper presents \textbf{FreEformer}, a simple yet effective model that leverages a \textbf{Fre}quency \textbf{E}nhanced Trans\textbf{former} for multivariate time series forecasting. Our work is based on the assumption that the frequency spectrum provides a global perspective on the composition of series across various frequencies and is highly suitable for robust representation learning. Specifically, we first convert time series into the complex frequency domain using the Discrete Fourier Transform (DFT). The Transformer architecture is then applied to the frequency spectra to capture cross-variate dependencies, with the real and imaginary parts processed independently. However, we observe that the vanilla attention matrix exhibits a low-rank characteristic, thus limiting representation diversity. This could be attributed to the inherent sparsity of the frequency domain and the strong-value-focused nature of Softmax in vanilla attention. To address this, we enhance the vanilla attention mechanism by introducing an additional learnable matrix to the original attention matrix, followed by row-wise L1 normalization. Theoretical analysis~demonstrates that this enhanced attention mechanism improves both feature diversity and gradient flow. Extensive experiments demonstrate that FreEformer consistently outperforms state-of-the-art models on eighteen real-world benchmarks covering electricity, traffic, weather, healthcare and finance. Notably, the enhanced attention mechanism also consistently improves the performance of state-of-the-art Transformer-based forecasters.

Title: CGI: Identifying Conditional Generative Models with Example Images

• Authors: Zhi Zhou, Hao-Zhe Tan, Peng-Xiao Song, Lan-Zhe Guo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13991
• Pdf URL: https://arxiv.org/pdf/2501.13991
• Copy Paste: [[2501.13991]] CGI: Identifying Conditional Generative Models with Example Images(https://arxiv.org/abs/2501.13991)
• Keywords: generative
• Abstract: Generative models have achieved remarkable performance recently, and thus model hubs have emerged. Existing model hubs typically assume basic text matching is sufficient to search for models. However, in reality, due to different abstractions and the large number of models in model hubs, it is not easy for users to review model descriptions and example images, choosing which model best meets their needs. Therefore, it is necessary to describe model functionality wisely so that future users can efficiently search for the most suitable model for their needs. Efforts to address this issue remain limited. In this paper, we propose Conditional Generative Model Identification (CGI), which aims to provide an effective way to identify the most suitable model using user-provided example images rather than requiring users to manually review a large number of models with example images. To address this problem, we propose the PromptBased Model Identification (PMI) , which can adequately describe model functionality and precisely match requirements with specifications. To evaluate PMI approach and promote related research, we provide a benchmark comprising 65 models and 9100 identification tasks. Extensive experimental and human evaluation results demonstrate that PMI is effective. For instance, 92% of models are correctly identified with significantly better FID scores when four example images are provided.

Title: CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generation

• Authors: Hamza Landolsi, Kais Letaief, Nizar Taghouti, Ines Abdeljaoued-Tej
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.13993
• Pdf URL: https://arxiv.org/pdf/2501.13993
• Copy Paste: [[2501.13993]] CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generation(https://arxiv.org/abs/2501.13993)
• Keywords: large language model
• Abstract: The introduction of new features and services in the banking sector often overwhelms customers, creating an opportunity for banks to enhance user experience through financial chatbots powered by large language models (LLMs). We initiated an AI agent designed to provide customers with relevant information about banking services and insights from annual reports. We proposed a hybrid Customer Analysis Pipeline Retrieval-Augmented Generation (CAPRAG) that effectively addresses both relationship-based and contextual queries, thereby improving customer engagement in the digital banking landscape. To implement this, we developed a processing pipeline to refine text data, which we utilized in two main frameworks: Vector RAG and Graph RAG. This dual approach enables us to populate both vector and graph databases with processed data for efficient retrieval. The Cypher query component is employed to effectively query the graph database. When a user submits a query, it is first expanded by a query expansion module before being routed to construct a final query from the hybrid Knowledge Base (KB). This final query is then sent to an open-source LLM for response generation. Overall, our innovative, designed to international banks, serves bank's customers in an increasingly complex digital environment, enhancing clarity and accessibility of information.

Title: CSAOT: Cooperative Multi-Agent System for Active Object Tracking

• Authors: Hy Nguyen, Bao Pham, Hung Du, Srikanth Thudumu, Rajesh Vasa, Kon Mouzakis
• Subjects: cs.CV, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.13994
• Pdf URL: https://arxiv.org/pdf/2501.13994
• Copy Paste: [[2501.13994]] CSAOT: Cooperative Multi-Agent System for Active Object Tracking(https://arxiv.org/abs/2501.13994)
• Keywords: robust
• Abstract: Object Tracking is essential for many computer vision applications, such as autonomous navigation, surveillance, and robotics. Unlike Passive Object Tracking (POT), which relies on static camera viewpoints to detect and track objects across consecutive frames, Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios due to limited information gathering and processing capabilities, often resulting in suboptimal decision-making. Alleviating these limitations necessitates the development of a multi-agent system where different agents perform distinct roles and collaborate to enhance learning and robustness in dynamic and complex environments. Although some multi-agent approaches exist for AOT, they typically rely on external auxiliary agents, which require additional devices, making them costly. In contrast, we introduce the Collaborative System for Active Object Tracking (CSAOT), a method that leverages multi-agent deep reinforcement learning (MADRL) and a Mixture of Experts (MoE) framework to enable multiple agents to operate on a single device, thereby improving tracking performance and reducing costs. Our approach enhances robustness against occlusions and rapid motion while optimizing camera movements to extend tracking duration. We validated the effectiveness of CSAOT on various interactive maps with dynamic and stationary obstacles.

Title: Framework for Progressive Knowledge Fusion in Large Language Models Through Structured Conceptual Redundancy Analysis

• Authors: Joseph Sakau, Evander Kozlowski, Roderick Thistledown, Basil Steinberger
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.13999
• Pdf URL: https://arxiv.org/pdf/2501.13999
• Copy Paste: [[2501.13999]] Framework for Progressive Knowledge Fusion in Large Language Models Through Structured Conceptual Redundancy Analysis(https://arxiv.org/abs/2501.13999)
• Keywords: robust, interpretability, large language model
• Abstract: The organization of latent knowledge within large-scale models poses unique challenges when addressing overlapping representations and optimizing contextual accuracy. Conceptual redundancies embedded across layers often result in inefficiencies that affect both computational demands and task-specific outcomes. A framework was proposed to restructure these redundancies through advanced clustering techniques and dynamic thresholding, ensuring that critical semantic relationships are preserved while removing unnecessary overlaps. Evaluations revealed improved memory efficiency and faster inference times, alongside better alignment in latent knowledge clusters that enhanced interpretability. Improvements in error rates and adversarial robustness suggest that restructuring redundancies has broader implications for increasing model reliability across diverse applications. Comparative analyses highlighted reductions in resource consumption and notable gains in performance, particularly in translation and summarization tasks. Energy metrics demonstrated significant savings during training phases, further validating the practicality of the approach for real-world deployments. Representational fidelity was also enhanced, with latent space evaluations indicating better cluster alignment and higher semantic consistency. The methodology bridges a key gap in model optimization through directly addressing redundancies at the structural level. Its application opens avenues for scalable, efficient, and contextually aware systems that can adapt to complex, domain-specific tasks without compromising on performance.

Title: Enhancing kelp forest detection in remote sensing images using crowdsourced labels with Mixed Vision Transformers and ConvNeXt segmentation models

• Authors: Ioannis Nasios
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14001
• Pdf URL: https://arxiv.org/pdf/2501.14001
• Copy Paste: [[2501.14001]] Enhancing kelp forest detection in remote sensing images using crowdsourced labels with Mixed Vision Transformers and ConvNeXt segmentation models(https://arxiv.org/abs/2501.14001)
• Keywords: transformer, segmentation
• Abstract: Kelp forests, as foundation species, are vital to marine ecosystems, providing essential food and habitat for numerous organisms. This study explores the integration of crowdsourced labels with advanced artificial intelligence models to develop a fast and accurate kelp canopy detection pipeline using Landsat images. Building on the success of a machine learning competition, where this approach ranked third and performed consistently well on both local validation and public and private leaderboards, the research highlights the effectiveness of combining Mixed Vision Transformers (MIT) with ConvNeXt models. Training these models on various image sizes significantly enhanced the accuracy of the ensemble results. U-Net emerged as the best segmentation architecture, with UpperNet also contributing to the final ensemble. Key Landsat bands, such as ShortWave InfraRed (SWIR1) and Near-InfraRed (NIR), were crucial while altitude data was used in postprocessing to eliminate false positives on land. The methodology achieved a high detection rate, accurately identifying about three out of four pixels containing kelp canopy while keeping false positives low. Despite the medium resolution of Landsat satellites, their extensive historical coverage makes them effective for studying kelp forests. This work also underscores the potential of combining machine learning models with crowdsourced data for effective and scalable environmental monitoring. All running code for training all models and inference can be found at this https URL.

Title: ME-CPT: Multi-Task Enhanced Cross-Temporal Point Transformer for Urban 3D Change Detection

• Authors: Luqi Zhang, Haiping Wang, Chong Liu, Zhen Dong, Bisheng Yang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14004
• Pdf URL: https://arxiv.org/pdf/2501.14004
• Copy Paste: [[2501.14004]] ME-CPT: Multi-Task Enhanced Cross-Temporal Point Transformer for Urban 3D Change Detection(https://arxiv.org/abs/2501.14004)
• Keywords: extraction, transformer, segmentation
• Abstract: The point clouds collected by the Airborne Laser Scanning (ALS) system provide accurate 3D information of urban land covers. By utilizing multi-temporal ALS point clouds, semantic changes in urban area can be captured, demonstrating significant potential in urban planning, emergency management, and infrastructure maintenance. Existing 3D change detection methods struggle to efficiently extract multi-class semantic information and change features, still facing the following challenges: (1) the difficulty of accurately modeling cross-temporal point clouds spatial relationships for effective change feature extraction; (2) class imbalance of change samples which hinders distinguishability of semantic features; (3) the lack of real-world datasets for 3D semantic change detection. To resolve these challenges, we propose the Multi-task Enhanced Cross-temporal Point Transformer (ME-CPT) network. ME-CPT establishes spatiotemporal correspondences between point cloud across different epochs and employs attention mechanisms to jointly extract semantic change features, facilitating information exchange and change comparison. Additionally, we incorporate a semantic segmentation task and through the multi-task training strategy, further enhance the distinguishability of semantic features, reducing the impact of class imbalance in change types. Moreover, we release a 22.5 $km^2$ 3D semantic change detection dataset, offering diverse scenes for comprehensive evaluation. Experiments on multiple datasets show that the proposed MT-CPT achieves superior performance compared to existing state-of-the-art methods. The source code and dataset will be released upon acceptance at \url{this https URL}.

Title: Device-aware Optical Adversarial Attack for a Portable Projector-camera System

• Authors: Ning Jiang (1 and 2), Yanhong Liu (2), Dingheng Zeng (2), Yue Feng (2), Weihong Deng (2), Ying Li (1) ((1) School of Software & Microelectronics, Peking University, Beijing, China (2) Mashang Consumer Finance Co., Ltd., Chongqing, China)
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14005
• Pdf URL: https://arxiv.org/pdf/2501.14005
• Copy Paste: [[2501.14005]] Device-aware Optical Adversarial Attack for a Portable Projector-camera System(https://arxiv.org/abs/2501.14005)
• Keywords: attack
• Abstract: Deep-learning-based face recognition (FR) systems are susceptible to adversarial examples in both digital and physical domains. Physical attacks present a greater threat to deployed systems as adversaries can easily access the input channel, allowing them to provide malicious inputs to impersonate a victim. This paper addresses the limitations of existing projector-camera-based adversarial light attacks in practical FR setups. By incorporating device-aware adaptations into the digital attack algorithm, such as resolution-aware and color-aware adjustments, we mitigate the degradation from digital to physical domains. Experimental validation showcases the efficacy of our proposed algorithm against real and spoof adversaries, achieving high physical similarity scores in FR models and state-of-the-art commercial systems. On average, there is only a 14% reduction in scores from digital to physical attacks, with high attack success rate in both white- and black-box scenarios.

Title: WAFBOOSTER: Automatic Boosting of WAF Security Against Mutated Malicious Payloads

• Authors: Cong Wu, Jing Chen, Simeng Zhu, Wenqi Feng, Ruiying Du, Yang Xiang
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14008
• Pdf URL: https://arxiv.org/pdf/2501.14008
• Copy Paste: [[2501.14008]] WAFBOOSTER: Automatic Boosting of WAF Security Against Mutated Malicious Payloads(https://arxiv.org/abs/2501.14008)
• Keywords: security, defense, attack
• Abstract: Web application firewall (WAF) examines malicious traffic to and from a web application via a set of security rules. It plays a significant role in securing Web applications against web attacks. However, as web attacks grow in sophistication, it is becoming increasingly difficult for WAFs to block the mutated malicious payloads designed to bypass their defenses. In response to this critical security issue, we have developed a novel learning-based framework called WAFBOOSTER, designed to unveil potential bypasses in WAF detections and suggest rules to fortify their security. Using a combination of shadow models and payload generation techniques, we can identify malicious payloads and remove or modify them as needed. WAFBOOSTER generates signatures for these malicious payloads using advanced clustering and regular expression matching techniques to repair any security gaps we uncover. In our comprehensive evaluation of eight real-world WAFs, WAFBOOSTER improved the true rejection rate of mutated malicious payloads from 21% to 96%, with no false rejections. WAFBOOSTER achieves a false acceptance rate 3X lower than state-of-the-art methods for generating malicious payloads. With WAFBOOSTER, we have taken a step forward in securing web applications against the ever-evolving threats.

Title: Scalable and Explainable Verification of Image-based Neural Network Controllers for Autonomous Vehicles

• Authors: Aditya Parameshwaran, Yue Wang
• Subjects: cs.LG, cs.AI, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.14009
• Pdf URL: https://arxiv.org/pdf/2501.14009
• Copy Paste: [[2501.14009]] Scalable and Explainable Verification of Image-based Neural Network Controllers for Autonomous Vehicles(https://arxiv.org/abs/2501.14009)
• Keywords: robust, explainability
• Abstract: Existing formal verification methods for image-based neural network controllers in autonomous vehicles often struggle with high-dimensional inputs, computational inefficiency, and a lack of explainability. These challenges make it difficult to ensure safety and reliability, as processing high-dimensional image data is computationally intensive and neural networks are typically treated as black boxes. To address these issues, we propose \textbf{SEVIN} (Scalable and Explainable Verification of Image-Based Neural Network Controllers), a framework that leverages a Variational Autoencoders (VAE) to encode high-dimensional images into a lower-dimensional, explainable latent space. By annotating latent variables with corresponding control actions, we generate convex polytopes that serve as structured input spaces for verification, significantly reducing computational complexity and enhancing scalability. Integrating the VAE's decoder with the neural network controller allows for formal and robustness verification using these explainable polytopes. Our approach also incorporates robustness verification under real-world perturbations by augmenting the dataset and retraining the VAE to capture environmental variations. Experimental results demonstrate that SEVIN achieves efficient and scalable verification while providing explainable insights into controller behavior, bridging the gap between formal verification techniques and practical applications in safety-critical systems.

Title: INDIGO+: A Unified INN-Guided Probabilistic Diffusion Algorithm for Blind and Non-Blind Image Restoration

• Authors: Di You, Pier Luigi Dragotti
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.14014
• Pdf URL: https://arxiv.org/pdf/2501.14014
• Copy Paste: [[2501.14014]] INDIGO+: A Unified INN-Guided Probabilistic Diffusion Algorithm for Blind and Non-Blind Image Restoration(https://arxiv.org/abs/2501.14014)
• Keywords: diffusion, generative
• Abstract: Generative diffusion models are becoming one of the most popular prior in image restoration (IR) tasks due to their remarkable ability to generate realistic natural images. Despite achieving satisfactory results, IR methods based on diffusion models present several limitations. First of all, most non-blind approaches require an analytical expression of the degradation model to guide the sampling process. Secondly, most existing blind approaches rely on families of pre-defined degradation models for training their deep networks. The above issues limit the flexibility of these approaches and so their ability to handle real-world degradation tasks. In this paper, we propose a novel INN-guided probabilistic diffusion algorithm for non-blind and blind image restoration, namely INDIGO and BlindINDIGO, which combines the merits of the perfect reconstruction property of invertible neural networks (INN) with the strong generative capabilities of pre-trained diffusion models. Specifically, we train the forward process of the INN to simulate an arbitrary degradation process and use the inverse to obtain an intermediate image that we use to guide the reverse diffusion sampling process through a gradient step. We also introduce an initialization strategy, to further improve the performance and inference speed of our algorithm. Experiments demonstrate that our algorithm obtains competitive results compared with recently leading methods both quantitatively and visually on synthetic and real-world low-quality images.

Title: Efficient Precision Control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting

• Authors: Vincent Blot, Alexandra Lorenzo de Brionne, Ines Sellami, Olivier Trassard, Isabelle Beau, Charlotte Sonigo, Nicolas J-B. Brunel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14036
• Pdf URL: https://arxiv.org/pdf/2501.14036
• Copy Paste: [[2501.14036]] Efficient Precision Control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting(https://arxiv.org/abs/2501.14036)
• Keywords: robust
• Abstract: Image analysis is a key tool for describing the detailed mechanisms of folliculogenesis, such as evaluating the quantity of mouse Primordial ovarian Follicles (PMF) in the ovarian reserve. The development of high-resolution virtual slide scanners offers the possibility of quantifying, robustifying and accelerating the histopathological procedure. A major challenge for machine learning is to control the precision of predictions while enabling a high recall, in order to provide reproducibility. We use a multiple testing procedure that gives an overperforming way to solve the standard Precision-Recall trade-off that gives probabilistic guarantees on the precision. In addition, we significantly improve the overall performance of the models (increase of F1-score) by selecting the decision threshold using contextual biological information or using an auxiliary model. As it is model-agnostic, this contextual selection procedure paves the way to the development of a strategy that can improve the performance of any model without the need of retraining it.

Title: Leveraging Large Language Models to Analyze Emotional and Contextual Drivers of Teen Substance Use in Online Discussions

• Authors: Jianfeng Zhu, Ruoming Jin, Hailong Jiang, Yulan Wang, Xinyu Zhang, Karin G. Coifman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14037
• Pdf URL: https://arxiv.org/pdf/2501.14037
• Copy Paste: [[2501.14037]] Leveraging Large Language Models to Analyze Emotional and Contextual Drivers of Teen Substance Use in Online Discussions(https://arxiv.org/abs/2501.14037)
• Keywords: protect, large language model
• Abstract: Adolescence is a critical stage often linked to risky behaviors, including substance use, with significant developmental and public health implications. Social media provides a lens into adolescent self-expression, but interpreting emotional and contextual signals remains complex. This study applies Large Language Models (LLMs) to analyze adolescents' social media posts, uncovering emotional patterns (e.g., sadness, guilt, fear, joy) and contextual factors (e.g., family, peers, school) related to substance use. Heatmap and machine learning analyses identified key predictors of substance use-related posts. Negative emotions like sadness and guilt were significantly more frequent in substance use contexts, with guilt acting as a protective factor, while shame and peer influence heightened substance use risk. Joy was more common in non-substance use discussions. Peer influence correlated strongly with sadness, fear, and disgust, while family and school environments aligned with non-substance use. Findings underscore the importance of addressing emotional vulnerabilities and contextual influences, suggesting that collaborative interventions involving families, schools, and communities can reduce risk factors and foster healthier adolescent development.

Title: LLM-guided Instance-level Image Manipulation with Diffusion U-Net Cross-Attention Maps

• Authors: Andrey Palaev, Adil Khan, Syed M. Ahsan Kazmi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14046
• Pdf URL: https://arxiv.org/pdf/2501.14046
• Copy Paste: [[2501.14046]] LLM-guided Instance-level Image Manipulation with Diffusion U-Net Cross-Attention Maps(https://arxiv.org/abs/2501.14046)
• Keywords: diffusion, generative, large language model
• Abstract: The advancement of text-to-image synthesis has introduced powerful generative models capable of creating realistic images from textual prompts. However, precise control over image attributes remains challenging, especially at the instance level. While existing methods offer some control through fine-tuning or auxiliary information, they often face limitations in flexibility and accuracy. To address these challenges, we propose a pipeline leveraging Large Language Models (LLMs), open-vocabulary detectors, cross-attention maps and intermediate activations of diffusion U-Net for instance-level image manipulation. Our method detects objects mentioned in the prompt and present in the generated image, enabling precise manipulation without extensive training or input masks. By incorporating cross-attention maps, our approach ensures coherence in manipulated images while controlling object positions. Our method enables precise manipulations at the instance level without fine-tuning or auxiliary information such as masks or bounding boxes. Code is available at this https URL

Title: GraphRAG under Fire

• Authors: Jiacheng Liang, Yuhui Wang, Changjiang Li, Rongyi Zhu, Tanqiu Jiang, Neil Gong, Ting Wang
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14050
• Pdf URL: https://arxiv.org/pdf/2501.14050
• Copy Paste: [[2501.14050]] GraphRAG under Fire(https://arxiv.org/abs/2501.14050)
• Keywords: security, attack
• Abstract: GraphRAG advances retrieval-augmented generation (RAG) by structuring external knowledge as multi-scale knowledge graphs, enabling language models to integrate both broad context and granular details in their reasoning. While GraphRAG has demonstrated success across domains, its security implications remain largely unexplored. To bridge this gap, this work examines GraphRAG's vulnerability to poisoning attacks, uncovering an intriguing security paradox: compared to conventional RAG, GraphRAG's graph-based indexing and retrieval enhance resilience against simple poisoning attacks; meanwhile, the same features also create new attack surfaces. We present GRAGPoison, a novel attack that exploits shared relations in the knowledge graph to craft poisoning text capable of compromising multiple queries simultaneously. GRAGPoison employs three key strategies: i) relation injection to introduce false knowledge, ii) relation enhancement to amplify poisoning influence, and iii) narrative generation to embed malicious content within coherent text. Empirical evaluation across diverse datasets and models shows that GRAGPoison substantially outperforms existing attacks in terms of effectiveness (up to 98% success rate) and scalability (using less than 68% poisoning text). We also explore potential defensive measures and their limitations, identifying promising directions for future research.

Title: Prior Knowledge Injection into Deep Learning Models Predicting Gene Expression from Whole Slide Images

• Authors: Max Hallemeesch, Marija Pizurica, Paloma Rabaey, Olivier Gevaert, Thomas Demeester, Kathleen Marchal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14056
• Pdf URL: https://arxiv.org/pdf/2501.14056
• Copy Paste: [[2501.14056]] Prior Knowledge Injection into Deep Learning Models Predicting Gene Expression from Whole Slide Images(https://arxiv.org/abs/2501.14056)
• Keywords: robust
• Abstract: Cancer diagnosis and prognosis primarily depend on clinical parameters such as age and tumor grade, and are increasingly complemented by molecular data, such as gene expression, from tumor sequencing. However, sequencing is costly and delays oncology workflows. Recent advances in Deep Learning allow to predict molecular information from morphological features within Whole Slide Images (WSIs), offering a cost-effective proxy of the molecular markers. While promising, current methods lack the robustness to fully replace direct sequencing. Here we aim to improve existing methods by introducing a model-agnostic framework that allows to inject prior knowledge on gene-gene interactions into Deep Learning architectures, thereby increasing accuracy and robustness. We design the framework to be generic and flexibly adaptable to a wide range of architectures. In a case study on breast cancer, our strategy leads to an average increase of 983 significant genes (out of 25,761) across all 18 experiments, with 14 generalizing to an increase on an independent dataset. Our findings reveal a high potential for injection of prior knowledge to increase gene expression prediction performance from WSIs across a wide range of architectures.

Title: Expanding on the BRIAR Dataset: A Comprehensive Whole Body Biometric Recognition Resource at Extreme Distances and Real-World Scenarios (Collections 1-4)

• Authors: Gavin Jager, David Cornett III, Gavin Glenn, Deniz Aykac, Christi Johnson, Robert Zhang, Ryan Shivers, David Bolme, Laura Davies, Scott Dolvin, Nell Barber, Joel Brogan, Nick Burchfield, Carl Dukes, Andrew Duncan, Regina Ferrell, Austin Garrett, Jim Goddard, Jairus Hines, Bart Murphy, Sean Pharris, Brandon Stockwell, Leanne Thompson, Matthew Yohe
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14070
• Pdf URL: https://arxiv.org/pdf/2501.14070
• Copy Paste: [[2501.14070]] Expanding on the BRIAR Dataset: A Comprehensive Whole Body Biometric Recognition Resource at Extreme Distances and Real-World Scenarios (Collections 1-4)(https://arxiv.org/abs/2501.14070)
• Keywords: robust, biometric
• Abstract: The state-of-the-art in biometric recognition algorithms and operational systems has advanced quickly in recent years providing high accuracy and robustness in more challenging collection environments and consumer applications. However, the technology still suffers greatly when applied to non-conventional settings such as those seen when performing identification at extreme distances or from elevated cameras on buildings or mounted to UAVs. This paper summarizes an extension to the largest dataset currently focused on addressing these operational challenges, and describes its composition as well as methodologies of collection, curation, and annotation.

Title: LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

• Authors: Yubin Ge, Neeraja Kirtane, Hao Peng, Dilek Hakkani-Tür
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14073
• Pdf URL: https://arxiv.org/pdf/2501.14073
• Copy Paste: [[2501.14073]] LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language(https://arxiv.org/abs/2501.14073)
• Keywords: large language model
• Abstract: As large language models (LLMs) have been deployed in various real-world settings, concerns about the harm they may propagate have grown. Various jailbreaking techniques have been developed to expose the vulnerabilities of these models and improve their safety. This work reveals that many state-of-the-art proprietary and open-source LLMs are vulnerable to malicious requests hidden behind scientific language. Specifically, our experiments with GPT4o, GPT4o-mini, GPT-4, LLama3-405B-Instruct, Llama3-70B-Instruct, Cohere, Gemini models on the StereoSet data demonstrate that, the models' biases and toxicity substantially increase when prompted with requests that deliberately misinterpret social science and psychological studies as evidence supporting the benefits of stereotypical biases. Alarmingly, these models can also be manipulated to generate fabricated scientific arguments claiming that biases are beneficial, which can be used by ill-intended actors to systematically jailbreak even the strongest models like GPT. Our analysis studies various factors that contribute to the models' vulnerabilities to malicious requests in academic language. Mentioning author names and venues enhances the persuasiveness of some models, and the bias scores can increase as dialogues progress. Our findings call for a more careful investigation on the use of scientific data in the training of LLMs.

Title: Enhancing Biomedical Relation Extraction with Directionality

• Authors: Po-Ting Lai, Chih-Hsuan Wei, Shubo Tian, Robert Leaman, Zhiyong Lu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14079
• Pdf URL: https://arxiv.org/pdf/2501.14079
• Copy Paste: [[2501.14079]] Enhancing Biomedical Relation Extraction with Directionality(https://arxiv.org/abs/2501.14079)
• Keywords: extraction, large language model
• Abstract: Biological relation networks contain rich information for understanding the biological mechanisms behind the relationship of entities such as genes, proteins, diseases, and chemicals. The vast growth of biomedical literature poses significant challenges updating the network knowledge. The recent Biomedical Relation Extraction Dataset (BioRED) provides valuable manual annotations, facilitating the develop-ment of machine-learning and pre-trained language model approaches for automatically identifying novel document-level (inter-sentence context) relationships. Nonetheless, its annotations lack directionality (subject/object) for the entity roles, essential for studying complex biological networks. Herein we annotate the entity roles of the relationships in the BioRED corpus and subsequently propose a novel multi-task language model with soft-prompt learning to jointly identify the relationship, novel findings, and entity roles. Our results in-clude an enriched BioRED corpus with 10,864 directionality annotations. Moreover, our proposed method outperforms existing large language models such as the state-of-the-art GPT-4 and Llama-3 on two benchmarking tasks. Our source code and dataset are available at this https URL.

Title: Communicating Activations Between Language Model Agents

• Authors: Vignav Ramesh, Kenneth Li
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14082
• Pdf URL: https://arxiv.org/pdf/2501.14082
• Copy Paste: [[2501.14082]] Communicating Activations Between Language Model Agents(https://arxiv.org/abs/2501.14082)
• Keywords: robust
• Abstract: Communication between multiple language model (LM) agents has been shown to scale up the reasoning ability of LMs. While natural language has been the dominant medium for inter-LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be otherwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via activations; concretely, we pause an LM $\textit{B}$'s computation at an intermediate layer, combine its current activation with another LM $\textit{A}$'s intermediate activation via some function $\textit{f}$, then pass $\textit{f}$'s output into the next layer of $\textit{B}$ and continue the forward pass till decoding is complete. This approach scales up LMs on new tasks with zero additional parameters and data, and saves a substantial amount of compute over natural language communication. We test our method with various functional forms $\textit{f}$ on two experimental setups--multi-player coordination games and reasoning benchmarks--and find that it achieves up to $27.0\%$ improvement over natural language communication across datasets with $<$$1/4$ the compute, illustrating the superiority and robustness of activations as an alternative "language" for communication between LMs.

Title: StreamingRAG: Real-time Contextual Retrieval and Generation Framework

• Authors: Murugan Sankaradas, Ravi K.Rajendran, Srimat T.Chakradhar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14101
• Pdf URL: https://arxiv.org/pdf/2501.14101
• Copy Paste: [[2501.14101]] StreamingRAG: Real-time Contextual Retrieval and Generation Framework(https://arxiv.org/abs/2501.14101)
• Keywords: large language model
• Abstract: Extracting real-time insights from multi-modal data streams from various domains such as healthcare, intelligent transportation, and satellite remote sensing remains a challenge. High computational demands and limited knowledge scope restrict the applicability of Multi-Modal Large Language Models (MM-LLMs) on these data streams. Traditional Retrieval-Augmented Generation (RAG) systems address knowledge limitations of these models, but suffer from slow preprocessing, making them unsuitable for real-time analysis. We propose StreamingRAG, a novel RAG framework designed for streaming data. StreamingRAG constructs evolving knowledge graphs capturing scene-object-entity relationships in real-time. The knowledge graph achieves temporal-aware scene representations using MM-LLMs and enables timely responses for specific events or user queries. StreamingRAG addresses limitations in existing methods, achieving significant improvements in real-time analysis (5-6x faster throughput), contextual accuracy (through a temporal knowledge graph), and reduced resource consumption (using lightweight models by 2-3x).

Title: 5G LDPC Linear Transformer for Channel Decoding

• Authors: Mario Hernandez, Fernando Pinero
• Subjects: cs.LG, cs.IT
• Abstract URL: https://arxiv.org/abs/2501.14102
• Pdf URL: https://arxiv.org/pdf/2501.14102
• Copy Paste: [[2501.14102]] 5G LDPC Linear Transformer for Channel Decoding(https://arxiv.org/abs/2501.14102)
• Keywords: transformer
• Abstract: This work introduces a novel, fully differentiable linear-time complexity transformer decoder and a transformer decoder to correct 5G New Radio (NR) LDPC. We propose a scalable approach to decode linear block codes with $O(n)$ complexity rather than $O(n^2)$ for regular transformers. The architectures' performances are compared to Belief Propagation (BP), the production-level decoding algorithm used for 5G New Radio (NR) LDPC codes. We achieve bit error rate performance that matches a regular Transformer decoder and surpases one iteration BP, also achieving competitive time performance against BP, even for larger block codes. We utilize Sionna, Nvidia's 5G & 6G physical layer research software, for reproducible results.

Title: MedSlice: Fine-Tuned Large Language Models for Secure Clinical Note Sectioning

• Authors: Joshua Davis, Thomas Sounack, Kate Sciacca, Jessie M Brain, Brigitte N Durieux, Nicole D Agaronnik, Charlotta Lindvall
• Subjects: cs.CL, cs.AI, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14105
• Pdf URL: https://arxiv.org/pdf/2501.14105
• Copy Paste: [[2501.14105]] MedSlice: Fine-Tuned Large Language Models for Secure Clinical Note Sectioning(https://arxiv.org/abs/2501.14105)
• Keywords: secure, privacy, large language model
• Abstract: Extracting sections from clinical notes is crucial for downstream analysis but is challenging due to variability in formatting and labor-intensive nature of manual sectioning. While proprietary large language models (LLMs) have shown promise, privacy concerns limit their accessibility. This study develops a pipeline for automated note sectioning using open-source LLMs, focusing on three sections: History of Present Illness, Interval History, and Assessment and Plan. We fine-tuned three open-source LLMs to extract sections using a curated dataset of 487 progress notes, comparing results relative to proprietary models (GPT-4o, GPT-4o mini). Internal and external validity were assessed via precision, recall and F1 score. Fine-tuned Llama 3.1 8B outperformed GPT-4o (F1=0.92). On the external validity test set, performance remained high (F1= 0.85). Fine-tuned open-source LLMs can surpass proprietary models in clinical note sectioning, offering advantages in cost, performance, and accessibility.

Title: LeCoPCR: Legal Concept-guided Prior Case Retrieval for European Court of Human Rights cases

• Authors: T.Y.S.S. Santosh, Isaac Misael Olguín Nolasco, Matthias Grabmair
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14114
• Pdf URL: https://arxiv.org/pdf/2501.14114
• Copy Paste: [[2501.14114]] LeCoPCR: Legal Concept-guided Prior Case Retrieval for European Court of Human Rights cases(https://arxiv.org/abs/2501.14114)
• Keywords: extraction
• Abstract: Prior case retrieval (PCR) is crucial for legal practitioners to find relevant precedent cases given the facts of a query case. Existing approaches often overlook the underlying semantic intent in determining relevance with respect to the query case. In this work, we propose LeCoPCR, a novel approach that explicitly generate intents in the form of legal concepts from a given query case facts and then augments the query with these concepts to enhance models understanding of semantic intent that dictates relavance. To overcome the unavailability of annotated legal concepts, we employ a weak supervision approach to extract key legal concepts from the reasoning section using Determinantal Point Process (DPP) to balance quality and diversity. Experimental results on the ECtHR-PCR dataset demonstrate the effectiveness of leveraging legal concepts and DPP-based key concept extraction.

Title: Autonomous Structural Memory Manipulation for Large Language Models Using Hierarchical Embedding Augmentation

• Authors: Derek Yotheringhay, Alistair Kirkland, Humphrey Kirkbride, Josiah Whitesteeple
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14119
• Pdf URL: https://arxiv.org/pdf/2501.14119
• Copy Paste: [[2501.14119]] Autonomous Structural Memory Manipulation for Large Language Models Using Hierarchical Embedding Augmentation(https://arxiv.org/abs/2501.14119)
• Keywords: robust, interpretability, large language model
• Abstract: Transformative innovations in model architectures have introduced hierarchical embedding augmentation as a means to redefine the representation of tokens through multi-level semantic structures, offering enhanced adaptability to complex linguistic inputs. Autonomous structural memory manipulation further advances this paradigm through dynamic memory reallocation mechanisms that prioritize critical contextual features while suppressing less relevant information, enabling scalable and efficient performance across diverse tasks. Experimental results reveal substantial improvements in computational efficiency, with marked reductions in processing overhead for longer input sequences, achieved through memory reorganization strategies that adapt to evolving contextual requirements. Hierarchical embeddings not only improved contextual alignment but also facilitated task generalization by capturing relationships at varying semantic granularities, ensuring coherence across layers without introducing significant computational redundancies. Comparative analysis against baseline models demonstrated unique advantages in accuracy, efficiency, and interpretability, particularly in tasks requiring complex contextual understanding or domain-specific adaptability. The ability to dynamically adjust token representations and memory configurations contributed to the model's robustness under varied and unpredictable input conditions. Applications benefiting from these advancements include multi-domain generalization, interactive systems, and scenarios involving real-time decision-making, where traditional static memory architectures often face limitations. The proposed methodology combines advanced embedding and memory management strategies into a cohesive framework that addresses scalability challenges while preserving task-specific relevance.

Title: Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filters

• Authors: Soumyendu Sarkar, Ashwin Ramesh Babu, Sajad Mousavi, Vineet Gundecha, Sahand Ghorbanpour, Avisek Naug, Ricardo Luna Gutierrez, Antonio Guillen
• Subjects: cs.LG, cs.AI, cs.CR, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14122
• Pdf URL: https://arxiv.org/pdf/2501.14122
• Copy Paste: [[2501.14122]] Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filters(https://arxiv.org/abs/2501.14122)
• Keywords: attack, robust
• Abstract: We present a Reinforcement Learning Platform for Adversarial Black-box untargeted and targeted attacks, RLAB, that allows users to select from various distortion filters to create adversarial examples. The platform uses a Reinforcement Learning agent to add minimum distortion to input images while still causing misclassification by the target model. The agent uses a novel dual-action method to explore the input image at each step to identify sensitive regions for adding distortions while removing noises that have less impact on the target model. This dual action leads to faster and more efficient convergence of the attack. The platform can also be used to measure the robustness of image classification models against specific distortion types. Also, retraining the model with adversarial samples significantly improved robustness when evaluated on benchmark datasets. The proposed platform outperforms state-of-the-art methods in terms of the average number of queries required to cause misclassification. This advances trustworthiness with a positive social impact.

Title: An Extensive and Methodical Review of Smart Grids for Sustainable Energy Management-Addressing Challenges with AI, Renewable Energy Integration and Leading-edge Technologies

• Authors: Parag Biswas, Abdur Rashid, abdullah al masum, MD Abdullah Al Nasim, A.S.M Anas Ferdous, Kishor Datta Gupta, Angona Biswas
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.14143
• Pdf URL: https://arxiv.org/pdf/2501.14143
• Copy Paste: [[2501.14143]] An Extensive and Methodical Review of Smart Grids for Sustainable Energy Management-Addressing Challenges with AI, Renewable Energy Integration and Leading-edge Technologies(https://arxiv.org/abs/2501.14143)
• Keywords: security, privacy, attack
• Abstract: Energy management decreases energy expenditures and consumption while simultaneously increasing energy efficiency, reducing carbon emissions, and enhancing operational performance. Smart grids are a type of sophisticated energy infrastructure that increase the generation and distribution of electricity's sustainability, dependability, and efficiency by utilizing digital communication technologies. They combine a number of cutting-edge techniques and technology to improve energy resource management. A large amount of research study on the topic of smart grids for energy management has been completed in the last several years. The authors of the present study want to cover a number of topics, including smart grid benefits and components, technical developments, integrating renewable energy sources, using artificial intelligence and data analytics, cybersecurity, and privacy. Smart Grids for Energy Management are an innovative field of study aiming at tackling various difficulties and magnifying the efficiency, dependability, and sustainability of energy systems, including: 1) Renewable sources of power like solar and wind are intermittent and unpredictable 2) Defending smart grid system from various cyber-attacks 3) Incorporating an increasing number of electric vehicles into the system of power grid without overwhelming it. Additionally, it is proposed to use AI and data analytics for better performance on the grid, reliability, and energy management. It also looks into how AI and data analytics can be used to optimize grid performance, enhance reliability, and improve energy management. The authors will explore these significant challenges and ongoing research. Lastly, significant issues in this field are noted, and recommendations for further work are provided.

Title: Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction

• Authors: Dongming Sheng, Kexin Han, Hao Li, Yan Zhang, Yucheng Huang, Jun Lang, Wenqiang Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14144
• Pdf URL: https://arxiv.org/pdf/2501.14144
• Copy Paste: [[2501.14144]] Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction(https://arxiv.org/abs/2501.14144)
• Keywords: extraction, generative
• Abstract: Aspect Sentiment Triplet Extraction (ASTE) is a thriving research area with impressive outcomes being achieved on high-resource languages. However, the application of cross-lingual transfer to the ASTE task has been relatively unexplored, and current code-switching methods still suffer from term boundary detection issues and out-of-dictionary problems. In this study, we introduce a novel Test-Time Code-SWitching (TT-CSW) framework, which bridges the gap between the bilingual training phase and the monolingual test-time prediction. During training, a generative model is developed based on bilingual code-switched training data and can produce bilingual ASTE triplets for bilingual inputs. In the testing stage, we employ an alignment-based code-switching technique for test-time augmentation. Extensive experiments on cross-lingual ASTE datasets validate the effectiveness of our proposed method. We achieve an average improvement of 3.7% in terms of weighted-averaged F1 in four datasets with different languages. Additionally, we set a benchmark using ChatGPT and GPT-4, and demonstrate that even smaller generative models fine-tuned with our proposed TT-CSW framework surpass ChatGPT and GPT-4 by 14.2% and 5.0% respectively.

Title: SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation

• Authors: Shuvendu Roy, Ali Etemad
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14148
• Pdf URL: https://arxiv.org/pdf/2501.14148
• Copy Paste: [[2501.14148]] SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation(https://arxiv.org/abs/2501.14148)
• Keywords: robust
• Abstract: We present SelfPrompt, a novel prompt-tuning approach for vision-language models (VLMs) in a semi-supervised learning setup. Existing methods for tuning VLMs in semi-supervised setups struggle with the negative impact of the miscalibrated VLMs on pseudo-labelling, and the accumulation of noisy pseudo-labels. SelfPrompt addresses these challenges by introducing a cluster-guided pseudo-labelling method that improves pseudo-label accuracy, and a confidence-aware semi-supervised learning module that maximizes the utilization of unlabelled data by combining supervised learning and weakly-supervised learning. Additionally, we investigate our method in an active semi-supervised learning setup, where the labelled set is strategically selected to ensure the best utilization of a limited labelling budget. To this end, we propose a weakly-supervised sampling technique that selects a diverse and representative labelled set, which can be seamlessly integrated into existing methods to enhance their performance. We conduct extensive evaluations across 13 datasets, significantly surpassing state-of-the-art performances with average improvements of 6.23% in standard semi-supervised learning, 6.25% in active semi-supervised learning, and 4.9% in base-to-novel generalization, using a 2-shot setup. Furthermore, SelfPrompt shows excellent generalization in single-shot settings, achieving an average improvement of 11.78%.

Title: Effective Defect Detection Using Instance Segmentation for NDI

• Authors: Ashiqur Rahman, Venkata Devesh Reddy Seethi, Austin Yunker, Zachary Kral, Rajkumar Kettimuthu, Hamed Alhoori
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14149
• Pdf URL: https://arxiv.org/pdf/2501.14149
• Copy Paste: [[2501.14149]] Effective Defect Detection Using Instance Segmentation for NDI(https://arxiv.org/abs/2501.14149)
• Keywords: segmentation
• Abstract: Ultrasonic testing is a common Non-Destructive Inspection (NDI) method used in aerospace manufacturing. However, the complexity and size of the ultrasonic scans make it challenging to identify defects through visual inspection or machine learning models. Using computer vision techniques to identify defects from ultrasonic scans is an evolving research area. In this study, we used instance segmentation to identify the presence of defects in the ultrasonic scan images of composite panels that are representative of real components manufactured in aerospace. We used two models based on Mask-RCNN (Detectron 2) and YOLO 11 respectively. Additionally, we implemented a simple statistical pre-processing technique that reduces the burden of requiring custom-tailored pre-processing techniques. Our study demonstrates the feasibility and effectiveness of using instance segmentation in the NDI pipeline by significantly reducing data pre-processing time, inspection time, and overall costs.

Title: Multimodal Prescriptive Deep Learning

• Authors: Dimitris Bertsimas, Lisa Everest, Vasiliki Stoumpou
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.14152
• Pdf URL: https://arxiv.org/pdf/2501.14152
• Copy Paste: [[2501.14152]] Multimodal Prescriptive Deep Learning(https://arxiv.org/abs/2501.14152)
• Keywords: interpretability
• Abstract: We introduce a multimodal deep learning framework, Prescriptive Neural Networks (PNNs), that combines ideas from optimization and machine learning, and is, to the best of our knowledge, the first prescriptive method to handle multimodal data. The PNN is a feedforward neural network trained on embeddings to output an outcome-optimizing prescription. In two real-world multimodal datasets, we demonstrate that PNNs prescribe treatments that are able to significantly improve estimated outcomes in transcatheter aortic valve replacement (TAVR) procedures by reducing estimated postoperative complication rates by 32% and in liver trauma injuries by reducing estimated mortality rates by over 40%. In four real-world, unimodal tabular datasets, we demonstrate that PNNs outperform or perform comparably to other well-known, state-of-the-art prescriptive models; importantly, on tabular datasets, we also recover interpretability through knowledge distillation, fitting interpretable Optimal Classification Tree models onto the PNN prescriptions as classification targets, which is critical for many real-world applications. Finally, we demonstrate that our multimodal PNN models achieve stability across randomized data splits comparable to other prescriptive methods and produce realistic prescriptions across the different datasets.

Title: Advancing MRI Reconstruction: A Systematic Review of Deep Learning and Compressed Sensing Integration

• Authors: Mojtaba Safari, Zach Eidex, Chih-Wei Chang, Richard L.J. Qiu, Xiaofeng Yang
• Subjects: cs.CV, cs.AI, physics.med-ph
• Abstract URL: https://arxiv.org/abs/2501.14158
• Pdf URL: https://arxiv.org/pdf/2501.14158
• Copy Paste: [[2501.14158]] Advancing MRI Reconstruction: A Systematic Review of Deep Learning and Compressed Sensing Integration(https://arxiv.org/abs/2501.14158)
• Keywords: federate
• Abstract: Magnetic resonance imaging (MRI) is a non-invasive imaging modality and provides comprehensive anatomical and functional insights into the human body. However, its long acquisition times can lead to patient discomfort, motion artifacts, and limiting real-time applications. To address these challenges, strategies such as parallel imaging have been applied, which utilize multiple receiver coils to speed up the data acquisition process. Additionally, compressed sensing (CS) is a method that facilitates image reconstruction from sparse data, significantly reducing image acquisition time by minimizing the amount of data collection needed. Recently, deep learning (DL) has emerged as a powerful tool for improving MRI reconstruction. It has been integrated with parallel imaging and CS principles to achieve faster and more accurate MRI reconstructions. This review comprehensively examines DL-based techniques for MRI reconstruction. We categorize and discuss various DL-based methods, including end-to-end approaches, unrolled optimization, and federated learning, highlighting their potential benefits. Our systematic review highlights significant contributions and underscores the potential of DL in MRI reconstruction. Additionally, we summarize key results and trends in DL-based MRI reconstruction, including quantitative metrics, the dataset, acceleration factors, and the progress of and research interest in DL techniques over time. Finally, we discuss potential future directions and the importance of DL-based MRI reconstruction in advancing medical imaging. To facilitate further research in this area, we provide a GitHub repository that includes up-to-date DL-based MRI reconstruction publications and public datasets-this https URL.

Title: Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation

• Authors: Cong-Duy Nguyen, Xiaobao Wu, Thong Nguyen, Shuai Zhao, Khoi Le, Viet-Anh Nguyen, Feng Yichao, Anh Tuan Luu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14166
• Pdf URL: https://arxiv.org/pdf/2501.14166
• Copy Paste: [[2501.14166]] Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation(https://arxiv.org/abs/2501.14166)
• Keywords: robust
• Abstract: Previous research on multimodal entity linking (MEL) has primarily employed contrastive learning as the primary objective. However, using the rest of the batch as negative samples without careful consideration, these studies risk leveraging easy features and potentially overlook essential details that make entities unique. In this work, we propose JD-CCL (Jaccard Distance-based Conditional Contrastive Learning), a novel approach designed to enhance the ability to match multimodal entity linking models. JD-CCL leverages meta-information to select negative samples with similar attributes, making the linking task more challenging and robust. Additionally, to address the limitations caused by the variations within the visual modality among mentions and entities, we introduce a novel method, CVaCPT (Contextual Visual-aid Controllable Patch Transform). It enhances visual representations by incorporating multi-view synthetic images and contextual textual representations to scale and shift patch representations. Experimental results on benchmark MEL datasets demonstrate the strong effectiveness of our approach.

Title: Argos: Agentic Time-Series Anomaly Detection with Autonomous Rule Generation via Large Language Models

• Authors: Yile Gu, Yifan Xiong, Jonathan Mace, Yuting Jiang, Yigong Hu, Baris Kasikci, Peng Cheng
• Subjects: cs.LG, cs.DC, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.14170
• Pdf URL: https://arxiv.org/pdf/2501.14170
• Copy Paste: [[2501.14170]] Argos: Agentic Time-Series Anomaly Detection with Autonomous Rule Generation via Large Language Models(https://arxiv.org/abs/2501.14170)
• Keywords: explainability, large language model
• Abstract: Observability in cloud infrastructure is critical for service providers, driving the widespread adoption of anomaly detection systems for monitoring metrics. However, existing systems often struggle to simultaneously achieve explainability, reproducibility, and autonomy, which are three indispensable properties for production use. We introduce Argos, an agentic system for detecting time-series anomalies in cloud infrastructure by leveraging large language models (LLMs). Argos proposes to use explainable and reproducible anomaly rules as intermediate representation and employs LLMs to autonomously generate such rules. The system will efficiently train error-free and accuracy-guaranteed anomaly rules through multiple collaborative agents and deploy the trained rules for low-cost online anomaly detection. Through evaluation results, we demonstrate that Argos outperforms state-of-the-art methods, increasing $F_1$ scores by up to $9.5\%$ and $28.3\%$ on public anomaly detection datasets and an internal dataset collected from Microsoft, respectively.

Title: Cybersecurity Assessment of Smart Grid Exposure Using a Machine Learning Based Approach

• Authors: Mofe O. Jeje
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14175
• Pdf URL: https://arxiv.org/pdf/2501.14175
• Copy Paste: [[2501.14175]] Cybersecurity Assessment of Smart Grid Exposure Using a Machine Learning Based Approach(https://arxiv.org/abs/2501.14175)
• Keywords: security, attack
• Abstract: Given that disturbances to the stable and normal operation of power systems have grown phenomenally, particularly in terms of unauthorized access to confidential and critical data, injection of malicious software, and exploitation of security vulnerabilities in a poorly patched software among others; then developing, as a countermeasure, an assessment solutions with machine learning capabilities to match up in real-time, with the growth and fast pace of these cyber-attacks, is not only critical to the security, reliability and safe operation of power system, but also germane to guaranteeing advanced monitoring and efficient threat detection. Using the Mississippi State University and Oak Ridge National Laboratory dataset, the study used an XGB Classifier modeling approach in machine learning to diagnose and assess power system disturbances, in terms of Attack Events, Natural Events and No-Events. As test results show, the model, in all the three sub-datasets, generally demonstrates good performance on all metrics, as it relates to accurately identifying and classifying all the three power system events.

Title: RL + Transformer = A General-Purpose Problem Solver

• Authors: Micah Rentschler, Jesse Roberts
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14176
• Pdf URL: https://arxiv.org/pdf/2501.14176
• Copy Paste: [[2501.14176]] RL + Transformer = A General-Purpose Problem Solver(https://arxiv.org/abs/2501.14176)
• Keywords: robust, transformer
• Abstract: What if artificial intelligence could not only solve problems for which it was trained but also learn to teach itself to solve new problems (i.e., meta-learn)? In this study, we demonstrate that a pre-trained transformer fine-tuned with reinforcement learning over multiple episodes develops the ability to solve problems that it has never encountered before - an emergent ability called In-Context Reinforcement Learning (ICRL). This powerful meta-learner not only excels in solving unseen in-distribution environments with remarkable sample efficiency, but also shows strong performance in out-of-distribution environments. In addition, we show that it exhibits robustness to the quality of its training data, seamlessly stitches together behaviors from its context, and adapts to non-stationary environments. These behaviors demonstrate that an RL-trained transformer can iteratively improve upon its own solutions, making it an excellent general-purpose problem solver.

Title: High-Precision Fabric Defect Detection via Adaptive Shape Convolutions and Large Kernel Spatial Modeling

• Authors: Shuai Wang, Yang Xu, Hui Zheng, Baotian Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14190
• Pdf URL: https://arxiv.org/pdf/2501.14190
• Copy Paste: [[2501.14190]] High-Precision Fabric Defect Detection via Adaptive Shape Convolutions and Large Kernel Spatial Modeling(https://arxiv.org/abs/2501.14190)
• Keywords: extraction
• Abstract: Detecting fabric defects in the textile industry remains a challenging task due to the diverse and complex nature of defect patterns. Traditional methods often suffer from slow inference speeds, limited accuracy, and inadequate recognition rates, particularly in scenarios involving intricate or subtle defects. To overcome these limitations, we introduce Fab-ASLKS, an advanced fabric defect detection framework built upon the YOLOv8s architecture. Fab-ASLKS incorporates two key modules: (1) the Adaptive Shape Convolution Module (ASCM), which leverages adaptive shape convolution within the Neck to enhance feature fusion and improve efficiency by extending the capabilities of the standard C2f structure, and (2) the Large Kernel Shift Convolution Module (LKSCM), designed to emulate large kernel effects within the Backbone, enabling superior spatial information extraction. These modules collaboratively optimize feature extraction and information integration across the network. Extensive experiments conducted on the Tianchi fabric defect detection dataset demonstrate that Fab-ASLKS achieves a 5% improvement in mAP@50 over the baseline, showcasing its capability to deliver high precision and efficiency.

Title: ENTER: Event Based Interpretable Reasoning for VideoQA

• Authors: Hammad Ayyubi, Junzhang Liu, Ali Asgarov, Zaber Ibn Abdul Hakim, Najibul Haque Sarker, Zhecan Wang, Chia-Wei Tang, Hani Alomari, Md. Atabuzzaman, Xudong Lin, Naveen Reddy Dyava, Shih-Fu Chang, Chris Thomas
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14194
• Pdf URL: https://arxiv.org/pdf/2501.14194
• Copy Paste: [[2501.14194]] ENTER: Event Based Interpretable Reasoning for VideoQA(https://arxiv.org/abs/2501.14194)
• Keywords: robust, interpretability, explainability
• Abstract: In this paper, we present ENTER, an interpretable Video Question Answering (VideoQA) system based on event graphs. Event graphs convert videos into graphical representations, where video events form the nodes and event-event relationships (temporal/causal/hierarchical) form the edges. This structured representation offers many benefits: 1) Interpretable VideoQA via generated code that parses event-graph; 2) Incorporation of contextual visual information in the reasoning process (code generation) via event graphs; 3) Robust VideoQA via Hierarchical Iterative Update of the event graphs. Existing interpretable VideoQA systems are often top-down, disregarding low-level visual information in the reasoning plan generation, and are brittle. While bottom-up approaches produce responses from visual data, they lack interpretability. Experimental results on NExT-QA, IntentQA, and EgoSchema demonstrate that not only does our method outperform existing top-down approaches while obtaining competitive performance against bottom-up approaches, but more importantly, offers superior interpretability and explainability in the reasoning process.

Title: VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking

• Authors: Runyi Hu, Jie Zhang, Yiming Li, Jiwei Li, Qing Guo, Han Qiu, Tianwei Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14195
• Pdf URL: https://arxiv.org/pdf/2501.14195
• Copy Paste: [[2501.14195]] VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking(https://arxiv.org/abs/2501.14195)
• Keywords: robust, extraction, watermark, diffusion
• Abstract: Artificial Intelligence Generated Content (AIGC) has advanced significantly, particularly with the development of video generation models such as text-to-video (T2V) models and image-to-video (I2V) models. However, like other AIGC types, video generation requires robust content control. A common approach is to embed watermarks, but most research has focused on images, with limited attention given to videos. Traditional methods, which embed watermarks frame-by-frame in a post-processing manner, often degrade video quality. In this paper, we propose VideoShield, a novel watermarking framework specifically designed for popular diffusion-based video generation models. Unlike post-processing methods, VideoShield embeds watermarks directly during video generation, eliminating the need for additional training. To ensure video integrity, we introduce a tamper localization feature that can detect changes both temporally (across frames) and spatially (within individual frames). Our method maps watermark bits to template bits, which are then used to generate watermarked noise during the denoising process. Using DDIM Inversion, we can reverse the video to its original watermarked noise, enabling straightforward watermark extraction. Additionally, template bits allow precise detection for potential temporal and spatial modification. Extensive experiments across various video models (both T2V and I2V models) demonstrate that our method effectively extracts watermarks and detects tamper without compromising video quality. Furthermore, we show that this approach is applicable to image generation models, enabling tamper detection in generated images as well. Codes and models are available at \href{this https URL}{this https URL}.

Title: PuzzleGPT: Emulating Human Puzzle-Solving Ability for Time and Location Prediction

• Authors: Hammad Ayyubi, Xuande Feng, Junzhang Liu, Xudong Lin, Zhecan Wang, Shih-Fu Chang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14210
• Pdf URL: https://arxiv.org/pdf/2501.14210
• Copy Paste: [[2501.14210]] PuzzleGPT: Emulating Human Puzzle-Solving Ability for Time and Location Prediction(https://arxiv.org/abs/2501.14210)
• Keywords: robust
• Abstract: The task of predicting time and location from images is challenging and requires complex human-like puzzle-solving ability over different clues. In this work, we formalize this ability into core skills and implement them using different modules in an expert pipeline called PuzzleGPT. PuzzleGPT consists of a perceiver to identify visual clues, a reasoner to deduce prediction candidates, a combiner to combinatorially combine information from different clues, a web retriever to get external knowledge if the task can't be solved locally, and a noise filter for robustness. This results in a zero-shot, interpretable, and robust approach that records state-of-the-art performance on two datasets -- TARA and WikiTilo. PuzzleGPT outperforms large VLMs such as BLIP-2, InstructBLIP, LLaVA, and even GPT-4V, as well as automatically generated reasoning pipelines like VisProg, by at least 32% and 38%, respectively. It even rivals or surpasses finetuned models.

Title: TFG-Flow: Training-free Guidance in Multimodal Generative Flow

• Authors: Haowei Lin, Shanda Li, Haotian Ye, Yiming Yang, Stefano Ermon, Yitao Liang, Jianzhu Ma
• Subjects: cs.LG, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2501.14216
• Pdf URL: https://arxiv.org/pdf/2501.14216
• Copy Paste: [[2501.14216]] TFG-Flow: Training-free Guidance in Multimodal Generative Flow(https://arxiv.org/abs/2501.14216)
• Keywords: diffusion, generative
• Abstract: Given an unconditional generative model and a predictor for a target property (e.g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training. As a highly efficient technique for steering generative models toward flexible outcomes, training-free guidance has gained increasing attention in diffusion models. However, existing methods only handle data in continuous spaces, while many scientific applications involve both continuous and discrete data (referred to as multimodality). Another emerging trend is the growing use of the simple and general flow matching framework in building generative foundation models, where guided generation remains under-explored. To address this, we introduce TFG-Flow, a novel training-free guidance method for multimodal generative flow. TFG-Flow addresses the curse-of-dimensionality while maintaining the property of unbiased sampling in guiding discrete variables. We validate TFG-Flow on four molecular design tasks and show that TFG-Flow has great potential in drug design by generating molecules with desired properties.

Title: Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game

• Authors: Rong Ye, Yongxin Zhang, Yikai Zhang, Haoyu Kuang, Zhongyu Wei, Peng Sun
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2501.14225
• Pdf URL: https://arxiv.org/pdf/2501.14225
• Copy Paste: [[2501.14225]] Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game(https://arxiv.org/abs/2501.14225)
• Keywords: large language model
• Abstract: Achieving Artificial General Intelligence (AGI) requires AI agents that can not only make stratigic decisions but also engage in flexible and meaningful communication. Inspired by Wittgenstein's language game theory in Philosophical Investigations, we propose that language agents can learn through in-context interaction rather than traditional multi-stage frameworks that separate decision-making from language expression. Using Werewolf, a social deduction game that tests language understanding, strategic interaction, and adaptability, we develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO). MaKTO engages diverse models in extensive gameplay to generate unpaired desirable and unacceptable responses, then employs KTO to refine the model's decision-making process. In 9-player Werewolf games, MaKTO achieves a 61% average win rate across various models, outperforming GPT-4o and two-stage RL agents by relative improvements of 23.0% and 10.9%, respectively. Notably, MaKTO also demonstrates human-like performance, winning 60% against expert players and showing only 49% detectability in Turing-style blind tests. These results showcase MaKTO's superior decision-making, strategic adaptation, and natural language generation in complex social deduction games.

Title: GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithm

• Authors: Hanrui Wang, Ching-Chun Chang, Chun-Shien Lu, Christopher Leckie, Isao Echizen
• Subjects: cs.CV, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14230
• Pdf URL: https://arxiv.org/pdf/2501.14230
• Copy Paste: [[2501.14230]] GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithm(https://arxiv.org/abs/2501.14230)
• Keywords: attack, robust
• Abstract: A critical requirement for deep learning models is ensuring their robustness against adversarial attacks. These attacks commonly introduce noticeable perturbations, compromising the visual fidelity of adversarial examples. Another key challenge is that while white-box algorithms can generate effective adversarial perturbations, they require access to the model gradients, limiting their practicality in many real-world scenarios. Existing attack mechanisms struggle to achieve similar efficacy without access to these gradients. In this paper, we introduce GreedyPixel, a novel pixel-wise greedy algorithm designed to generate high-quality adversarial examples using only query-based feedback from the target model. GreedyPixel improves computational efficiency in what is typically a brute-force process by perturbing individual pixels in sequence, guided by a pixel-wise priority map. This priority map is constructed by ranking gradients obtained from a surrogate model, providing a structured path for perturbation. Our results demonstrate that GreedyPixel achieves attack success rates comparable to white-box methods without the need for gradient information, and surpasses existing algorithms in black-box settings, offering higher success rates, reduced computational time, and imperceptible perturbations. These findings underscore the advantages of GreedyPixel in terms of attack efficacy, time efficiency, and visual quality.

Title: A Data-driven Dynamic Temporal Correlation Modeling Framework for Renewable Energy Scenario Generation

• Authors: Xiaochong Dong, Yilin Liu, Xuemin Zhang, Shengwei Mei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14233
• Pdf URL: https://arxiv.org/pdf/2501.14233
• Copy Paste: [[2501.14233]] A Data-driven Dynamic Temporal Correlation Modeling Framework for Renewable Energy Scenario Generation(https://arxiv.org/abs/2501.14233)
• Keywords: interpretability
• Abstract: Renewable energy power is influenced by the atmospheric system, which exhibits nonlinear and time-varying features. To address this, a dynamic temporal correlation modeling framework is proposed for renewable energy scenario generation. A novel decoupled mapping path is employed for joint probability distribution modeling, formulating regression tasks for both marginal distributions and the correlation structure using proper scoring rules to ensure the rationality of the modeling process. The scenario generation process is divided into two stages. Firstly, the dynamic correlation network models temporal correlations based on a dynamic covariance matrix, capturing the time-varying features of renewable energy while enhancing the interpretability of the black-box model. Secondly, the implicit quantile network models the marginal quantile function in a nonparametric, continuous manner, enabling scenario generation through marginal inverse sampling. Experimental results demonstrate that the proposed dynamic correlation quantile network outperforms state-of-the-art methods in quantifying uncertainty and capturing dynamic correlation for short-term renewable energy scenario generation.

Title: Point-LN: A Lightweight Framework for Efficient Point Cloud Classification Using Non-Parametric Positional Encoding

• Authors: Marzieh Mohammadi, Amir Salarpour, Pedram MohajerAnsari
• Subjects: cs.CV, cs.AI, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.14238
• Pdf URL: https://arxiv.org/pdf/2501.14238
• Copy Paste: [[2501.14238]] Point-LN: A Lightweight Framework for Efficient Point Cloud Classification Using Non-Parametric Positional Encoding(https://arxiv.org/abs/2501.14238)
• Keywords: robust
• Abstract: We introduce Point-LN, a novel lightweight framework engineered for efficient 3D point cloud classification. Point-LN integrates essential non-parametric components-such as Farthest Point Sampling (FPS), k-Nearest Neighbors (k-NN), and non-learnable positional encoding-with a streamlined learnable classifier that significantly enhances classification accuracy while maintaining a minimal parameter footprint. This hybrid architecture ensures low computational costs and rapid inference speeds, making Point-LN ideal for real-time and resource-constrained applications. Comprehensive evaluations on benchmark datasets, including ModelNet40 and ScanObjectNN, demonstrate that Point-LN achieves competitive performance compared to state-of-the-art methods, all while offering exceptional efficiency. These results establish Point-LN as a robust and scalable solution for diverse point cloud classification tasks, highlighting its potential for widespread adoption in various computer vision applications.

Title: Humanity's Last Exam

• Authors: Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Daron Anderson, Tung Nguyen, Mobeen Mahmood, Fiona Feng, Steven Y. Feng, Haoran Zhao, Michael Yu, Varun Gangal, Chelsea Zou, Zihan Wang, Jessica P. Wang, Pawan Kumar, Oleksandr Pokutnyi, Robert Gerbicz, Serguei Popov, John-Clark Levin, Mstyslav Kazakov, Johannes Schmitt, Geoff Galgon, Alvaro Sanchez, Yongki Lee, Will Yeadon, Scott Sauers, Marc Roth, Chidozie Agu, Søren Riis, Fabian Giska, Saiteja Utpala, Zachary Giboney, Gashaw M. Goshu, Joan of Arc Xavier, Sarah-Jane Crowson, Mohinder Maheshbhai Naiya, Noah Burns, Lennart Finke, Zerui Cheng, Hyunwoo Park, Francesco Fournier-Facio, John Wydallis, Mark Nandor, Ankit Singh, Tim Gehrunger, Jiaqi Cai, Ben McCarty, Darling Duclosel, Jungbae Nam, Jennifer Zampese, Ryan G. Hoerr, Aras Bacho, Gautier Abou Loume, Abdallah Galal, Hangrui Cao, Alexis C Garretson, Damien Sileo, Qiuyu Ren, Doru Cojoc, Pavel Arkhipov, Usman Qazi, Lianghui Li, Sumeet Motwani, Christian Schroeder de Witt, Edwin Taylor, Johannes Veith, Eric Singer, Taylor D. Hartman, Paolo Rissone, Jaehyeok Jin, Jack Wei Lun Shi, Chris G. Willcocks, Joshua Robinson, Aleksandar Mikov, Ameya Prabhu, Longke Tang, Xavier Alapont, Justine Leon Uro, Kevin Zhou, Emily de Oliveira Santos, Andrey Pupasov Maksimov, Edward Vendrow, Kengo Zenitani, Julien Guillod, Yuqi Li, Joshua Vendrow, Vladyslav Kuchkin, Ng Ze-An
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14249
• Pdf URL: https://arxiv.org/pdf/2501.14249
• Copy Paste: [[2501.14249]] Humanity's Last Exam(https://arxiv.org/abs/2501.14249)
• Keywords: large language model
• Abstract: Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. HLE consists of 3,000 questions across dozens of subjects, including mathematics, humanities, and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer questions suitable for automated grading. Each question has a known solution that is unambiguous and easily verifiable, but cannot be quickly answered via internet retrieval. State-of-the-art LLMs demonstrate low accuracy and calibration on HLE, highlighting a significant gap between current LLM capabilities and the expert human frontier on closed-ended academic questions. To inform research and policymaking upon a clear understanding of model capabilities, we publicly release HLE at this https URL.

Title: Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors

• Authors: Yi Zhao, Youzhi Zhang
• Subjects: cs.CL, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14250
• Pdf URL: https://arxiv.org/pdf/2501.14250
• Copy Paste: [[2501.14250]] Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors(https://arxiv.org/abs/2501.14250)
• Keywords: defense, attack, large language model
• Abstract: Large language models (LLMs) are widely used in real-world applications, raising concerns about their safety and trustworthiness. While red-teaming with jailbreak prompts exposes the vulnerabilities of LLMs, current efforts focus primarily on single-turn attacks, overlooking the multi-turn strategies used by real-world adversaries. Existing multi-turn methods rely on static patterns or predefined logical chains, failing to account for the dynamic strategies during attacks. We propose Siren, a learning-based multi-turn attack framework designed to simulate real-world human jailbreak behaviors. Siren consists of three stages: (1) training set construction utilizing Turn-Level LLM feedback (Turn-MF), (2) post-training attackers with supervised fine-tuning (SFT) and direct preference optimization (DPO), and (3) interactions between the attacking and target LLMs. Experiments demonstrate that Siren achieves an attack success rate (ASR) of 90% with LLaMA-3-8B as the attacker against Gemini-1.5-Pro as the target model, and 70% with Mistral-7B against GPT-4o, significantly outperforming single-turn baselines. Moreover, Siren with a 7B-scale model achieves performance comparable to a multi-turn baseline that leverages GPT-4o as the attacker, while requiring fewer turns and employing decomposition strategies that are better semantically aligned with attack goals. We hope Siren inspires the development of stronger defenses against advanced multi-turn jailbreak attacks under realistic scenarios. Code is available at this https URL. Warning: This paper contains potentially harmful text.

Title: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data

• Authors: Yiyun Zhou, Wenkang Han, Jingyuan Chen
• Subjects: cs.LG, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.14256
• Pdf URL: https://arxiv.org/pdf/2501.14256
• Copy Paste: [[2501.14256]] Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data(https://arxiv.org/abs/2501.14256)
• Keywords: interpretability
• Abstract: Knowledge Tracing (KT) is a fundamental component of Intelligent Tutoring Systems (ITS), enabling the modeling of students' knowledge states to predict future performance. The introduction of Deep Knowledge Tracing (DKT), the first deep learning-based KT (DLKT) model, has brought significant advantages in terms of applicability and comprehensiveness. However, recent DLKT models, such as Attentive Knowledge Tracing (AKT), have often prioritized predictive performance at the expense of these benefits. While deep sequential models like DKT have shown potential, they face challenges related to parallel computing, storage decision modification, and limited storage capacity. To address these limitations, we propose DKT2, a novel KT model that leverages the recently developed xLSTM architecture. DKT2 enhances input representation using the Rasch model and incorporates Item Response Theory (IRT) for interpretability, allowing for the decomposition of learned knowledge into familiar and unfamiliar knowledge. By integrating this knowledge with predicted questions, DKT2 generates comprehensive knowledge states. Extensive experiments conducted across three large-scale datasets demonstrate that DKT2 consistently outperforms 17 baseline models in various prediction tasks, underscoring its potential for real-world educational applications. This work bridges the gap between theoretical advancements and practical implementation in this http URL code and datasets will be available at this https URL.

Title: TrajFlow: A Generative Framework for Occupancy Density Estimation Using Normalizing Flows

• Authors: Mitch Kosieradzki, Seongjin Choi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14266
• Pdf URL: https://arxiv.org/pdf/2501.14266
• Copy Paste: [[2501.14266]] TrajFlow: A Generative Framework for Occupancy Density Estimation Using Normalizing Flows(https://arxiv.org/abs/2501.14266)
• Keywords: generative
• Abstract: In transportation systems and autonomous vehicles, intelligent agents must understand the future motion of traffic participants to effectively plan motion trajectories. At the same time, the motion of traffic participants is inherently uncertain. In this paper, we propose TrajFlow, a generative framework for estimating the occupancy density of traffic participants. Our framework utilizes a causal encoder to extract semantically meaningful embeddings of the observed trajectory, as well as a normalizing flow to decode these embeddings and determine the most likely future location of traffic participants at some time point in the future. Our formulation differs from existing approaches because we model the marginal distribution of spatial locations instead of the joint distribution of unobserved trajectories. The advantages of a marginal formulation are numerous. First, we demonstrate that the marginal formulation produces higher accuracy on challenging trajectory forecasting benchmarks. Second, the marginal formulation allows for a fully continuous sampling of future locations. Finally, marginal densities are better suited for downstream tasks as they allow for the computation of per-agent motion trajectories and occupancy grids, the two most commonly used representations for motion forecasting. We present a novel architecture based entirely on neural differential equations as an implementation of this framework and provide ablations to demonstrate the advantages of a continuous implementation over a more traditional discrete neural network based approach. The code is available at this https URL .

Title: TLXML: Task-Level Explanation of Meta-Learning via Influence Functions

• Authors: Yoshihiro Mitsuka, Shadan Golestan, Zahin Sufiyan, Sheila Schoepp, Shotaro Miwa, Osmar R. Zaïane
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14271
• Pdf URL: https://arxiv.org/pdf/2501.14271
• Copy Paste: [[2501.14271]] TLXML: Task-Level Explanation of Meta-Learning via Influence Functions(https://arxiv.org/abs/2501.14271)
• Keywords: explainability
• Abstract: The scheme of adaptation via meta-learning is seen as an ingredient for solving the problem of data shortage or distribution shift in real-world applications, but it also brings the new risk of inappropriate updates of the model in the user environment, which increases the demand for explainability. Among the various types of XAI methods, establishing a method of explanation based on past experience in meta-learning requires special consideration due to its bi-level structure of training, which has been left unexplored. In this work, we propose influence functions for explaining meta-learning that measure the sensitivities of training tasks to adaptation and inference. We also argue that the approximation of the Hessian using the Gauss-Newton matrix resolves computational barriers peculiar to meta-learning. We demonstrate the adequacy of the method through experiments on task distinction and task distribution distinction using image classification tasks with MAML and Prototypical Network.

Title: Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation

• Authors: Sadegh Mahdavi, Muchen Li, Kaiwen Liu, Christos Thrampoulidis, Leonid Sigal, Renjie Liao
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14275
• Pdf URL: https://arxiv.org/pdf/2501.14275
• Copy Paste: [[2501.14275]] Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation(https://arxiv.org/abs/2501.14275)
• Keywords: large language model
• Abstract: Advances in Large Language Models (LLMs) have sparked interest in their ability to solve Olympiad-level math problems. However, the training and evaluation of these models are constrained by the limited size and quality of available datasets, as creating large-scale data for such advanced problems requires extensive effort from human experts. In addition, current benchmarks are prone to contamination, leading to unreliable evaluations. In this paper, we present an automated pipeline that leverages the rich resources of the Art of Problem Solving (AoPS) forum, which predominantly features Olympiad-level problems and community-driven solutions. Using open-source LLMs, we develop a method to extract question-answer pairs from the forum, resulting in AoPS-Instruct, a dataset of more than 600,000 high-quality QA pairs. Our experiments demonstrate that fine-tuning LLMs on AoPS-Instruct improves their reasoning abilities across various benchmarks. Moreover, we build an automatic pipeline that introduces LiveAoPSBench, an evolving evaluation set with timestamps, derived from the latest forum data, providing a contamination-resistant benchmark for assessing LLM performance. Notably, we observe a significant decline in LLM performance over time, suggesting their success on older examples may stem from pre-training exposure rather than true reasoning ability. Our work presents a scalable approach to creating and maintaining large-scale, high-quality datasets for advanced math reasoning, offering valuable insights into the capabilities and limitations of LLMs in this domain. Our benchmark and code is available at this https URL

Title: Dense-SfM: Structure from Motion with Dense Consistent Matching

• Authors: JongMin Lee, Sungjoo Yoo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14277
• Pdf URL: https://arxiv.org/pdf/2501.14277
• Copy Paste: [[2501.14277]] Dense-SfM: Structure from Motion with Dense Consistent Matching(https://arxiv.org/abs/2501.14277)
• Keywords: robust, transformer
• Abstract: We present Dense-SfM, a novel Structure from Motion (SfM) framework designed for dense and accurate 3D reconstruction from multi-view images. Sparse keypoint matching, which traditional SfM methods often rely on, limits both accuracy and point density, especially in texture-less areas. Dense-SfM addresses this limitation by integrating dense matching with a Gaussian Splatting (GS) based track extension which gives more consistent, longer feature tracks. To further improve reconstruction accuracy, Dense-SfM is equipped with a multi-view kernelized matching module leveraging transformer and Gaussian Process architectures, for robust track refinement across multi-views. Evaluations on the ETH3D and Texture-Poor SfM datasets show that Dense-SfM offers significant improvements in accuracy and density over state-of-the-art methods.

Title: A Comprehensive Framework for Semantic Similarity Detection Using Transformer Architectures and Enhanced Ensemble Techniques

• Authors: Lifu Gao, Qi Zhang, Ziwei Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14288
• Pdf URL: https://arxiv.org/pdf/2501.14288
• Copy Paste: [[2501.14288]] A Comprehensive Framework for Semantic Similarity Detection Using Transformer Architectures and Enhanced Ensemble Techniques(https://arxiv.org/abs/2501.14288)
• Keywords: robust, transformer
• Abstract: Detecting AI-generated text, especially in short-context documents, is difficult because there is not enough context for accurate classification. This paper presents a new teacher-student model that uses domain adaptation and data augmentation to solve these problems. The teacher model, which combines DeBERTa-v3-large and Mamba-790m, learns semantic knowledge through domain-specific fine-tuning. The student model handles short-context text more efficiently. The system uses a Mean Squared Error (MSE) loss function to guide the student's learning, improving both accuracy and efficiency. Also, data augmentation methods like spelling correction and error injection make the model more robust. Experimental results show that this approach works better than baseline methods, proving its usefulness for real-time AI-generated text detection and other text classification tasks.

Title: Advances in Temporal Point Processes: Bayesian, Deep, and LLM Approaches

• Authors: Feng Zhou, Quyu Kong, Yixuan Zhang
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.14291
• Pdf URL: https://arxiv.org/pdf/2501.14291
• Copy Paste: [[2501.14291]] Advances in Temporal Point Processes: Bayesian, Deep, and LLM Approaches(https://arxiv.org/abs/2501.14291)
• Keywords: large language model
• Abstract: Temporal point processes (TPPs) are stochastic process models used to characterize event sequences occurring in continuous time. Traditional statistical TPPs have a long-standing history, with numerous models proposed and successfully applied across diverse domains. In recent years, advances in deep learning have spurred the development of neural TPPs, enabling greater flexibility and expressiveness in capturing complex temporal dynamics. The emergence of large language models (LLMs) has further sparked excitement, offering new possibilities for modeling and analyzing event sequences by leveraging their rich contextual understanding. This survey presents a comprehensive review of recent research on TPPs from three perspectives: Bayesian, deep learning, and LLM approaches. We begin with a review of the fundamental concepts of TPPs, followed by an in-depth discussion of model design and parameter estimation techniques in these three frameworks. We also revisit classic application areas of TPPs to highlight their practical relevance. Finally, we outline challenges and promising directions for future research.

Title: Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes

• Authors: Sullam Jeoung, Yubin Ge, Haohan Wang, Jana Diesner
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14294
• Pdf URL: https://arxiv.org/pdf/2501.14294
• Copy Paste: [[2501.14294]] Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes(https://arxiv.org/abs/2501.14294)
• Keywords: large language model
• Abstract: Examining the alignment of large language models (LLMs) has become increasingly important, particularly when these systems fail to operate as intended. This study explores the challenge of aligning LLMs with human intentions and values, with specific focus on their political inclinations. Previous research has highlighted LLMs' propensity to display political leanings, and their ability to mimic certain political parties' stances on various issues. However, the extent and conditions under which LLMs deviate from empirical positions have not been thoroughly examined. To address this gap, our study systematically investigates the factors contributing to LLMs' deviations from empirical positions on political issues, aiming to quantify these deviations and identify the conditions that cause them. Drawing on cognitive science findings related to representativeness heuristics -- where individuals readily recall the representative attribute of a target group in a way that leads to exaggerated beliefs -- we scrutinize LLM responses through this heuristics lens. We conduct experiments to determine how LLMs exhibit stereotypes by inflating judgments in favor of specific political parties. Our results indicate that while LLMs can mimic certain political parties' positions, they often exaggerate these positions more than human respondents do. Notably, LLMs tend to overemphasize representativeness to a greater extent than humans. This study highlights the susceptibility of LLMs to representativeness heuristics, suggeseting potential vulnerabilities to political stereotypes. We propose prompt-based mitigation strategies that demonstrate effectiveness in reducing the influence of representativeness in LLM responses.

Title: Additive Manufacturing Processes Protocol Prediction by Artificial Intelligence using X-ray Computed Tomography data

• Authors: Sunita Khod, Akshay Dvivedi, Mayank Goswami
• Subjects: cs.CV, physics.app-ph
• Abstract URL: https://arxiv.org/abs/2501.14306
• Pdf URL: https://arxiv.org/pdf/2501.14306
• Copy Paste: [[2501.14306]] Additive Manufacturing Processes Protocol Prediction by Artificial Intelligence using X-ray Computed Tomography data(https://arxiv.org/abs/2501.14306)
• Keywords: segmentation
• Abstract: The quality of the part fabricated from the Additive Manufacturing (AM) process depends upon the process parameters used, and therefore, optimization is required for apt quality. A methodology is proposed to set these parameters non-iteratively without human intervention. It utilizes Artificial Intelligence (AI) to fully automate the process, with the capability to self-train any apt AI model by further assimilating the training this http URL study includes three commercially available 3D printers for soft material printing based on the Material Extrusion (MEX) AM process. The samples are 3D printed for six different AM process parameters obtained by varying layer height and nozzle speed. The novelty part of the methodology is incorporating an AI-based image segmentation step in the decision-making stage that uses quality inspected training data from the Non-Destructive Testing (NDT) method. The performance of the trained AI model is compared with the two software tools based on the classical thresholding method. The AI-based Artificial Neural Network (ANN) model is trained from NDT-assessed and AI-segmented data to automate the selection of optimized process parameters. The AI-based model is 99.3 % accurate, while the best available commercial classical image method is 83.44 % accurate. The best value of overall R for training ANN is 0.82. The MEX process gives a 22.06 % porosity error relative to the design. The NDT-data trained two AI models integrated into a series pipeline for optimal process parameters are proposed and verified by classical optimization and mechanical testing methods.

Title: BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activities

• Authors: Zhibo Tian, Ruijie Quan, Fan Ma, Kun Zhan, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14309
• Pdf URL: https://arxiv.org/pdf/2501.14309
• Copy Paste: [[2501.14309]] BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activities(https://arxiv.org/abs/2501.14309)
• Keywords: secure, privacy, protect
• Abstract: Reconstructing perceived images from human brain activity forms a crucial link between human and machine learning through Brain-Computer Interfaces. Early methods primarily focused on training separate models for each individual to account for individual variability in brain activity, overlooking valuable cross-subject commonalities. Recent advancements have explored multisubject methods, but these approaches face significant challenges, particularly in data privacy and effectively managing individual variability. To overcome these challenges, we introduce BrainGuard, a privacy-preserving collaborative training framework designed to enhance image reconstruction from multisubject fMRI data while safeguarding individual privacy. BrainGuard employs a collaborative global-local architecture where individual models are trained on each subject's local data and operate in conjunction with a shared global model that captures and leverages cross-subject patterns. This architecture eliminates the need to aggregate fMRI data across subjects, thereby ensuring privacy preservation. To tackle the complexity of fMRI data, BrainGuard integrates a hybrid synchronization strategy, enabling individual models to dynamically incorporate parameters from the global model. By establishing a secure and collaborative training environment, BrainGuard not only protects sensitive brain data but also improves the image reconstructions accuracy. Extensive experiments demonstrate that BrainGuard sets a new benchmark in both high-level and low-level metrics, advancing the state-of-the-art in brain decoding through its innovative design.

Title: Permutation-based multi-objective evolutionary feature selection for high-dimensional data

• Authors: Raquel Espinosa, Gracia Sánchez, José Palma, Fernando Jiménez
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14310
• Pdf URL: https://arxiv.org/pdf/2501.14310
• Copy Paste: [[2501.14310]] Permutation-based multi-objective evolutionary feature selection for high-dimensional data(https://arxiv.org/abs/2501.14310)
• Keywords: interpretability
• Abstract: Feature selection is a critical step in the analysis of high-dimensional data, where the number of features often vastly exceeds the number of samples. Effective feature selection not only improves model performance and interpretability but also reduces computational costs and mitigates the risk of overfitting. In this context, we propose a novel feature selection method for high-dimensional data, based on the well-known permutation feature importance approach, but extending it to evaluate subsets of attributes rather than individual features. This extension more effectively captures how interactions among features influence model performance. The proposed method employs a multi-objective evolutionary algorithm to search for candidate feature subsets, with the objectives of maximizing the degradation in model performance when the selected features are shuffled, and minimizing the cardinality of the feature subset. The effectiveness of our method has been validated on a set of 24 publicly available high-dimensional datasets for classification and regression tasks, and compared against 9 well-established feature selection methods designed for high-dimensional problems, including the conventional permutation feature importance method. The results demonstrate the ability of our approach in balancing accuracy and computational efficiency, providing a powerful tool for feature selection in complex, high-dimensional datasets.

Title: An Efficient Real Time DDoS Detection Model Using Machine Learning Algorithms

• Authors: Debashis Kar Suvra
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14311
• Pdf URL: https://arxiv.org/pdf/2501.14311
• Copy Paste: [[2501.14311]] An Efficient Real Time DDoS Detection Model Using Machine Learning Algorithms(https://arxiv.org/abs/2501.14311)
• Keywords: attack
• Abstract: Distributed Denial of Service attacks have become a significant threat to industries and governments leading to substantial financial losses. With the growing reliance on internet services, DDoS attacks can disrupt services by overwhelming servers with false traffic causing downtime and data breaches. Although various detection techniques exist, selecting an effective method remains challenging due to trade-offs between time efficiency and accuracy. This research focuses on developing an efficient real-time DDoS detection system using machine learning algorithms leveraging the UNB CICDDoS2019 dataset including various traffic features. The study aims to classify DDoS and non-DDoS traffic through various ML classifiers including Logistic Regression, K-Nearest Neighbors, Random Forest, Support Vector Machine, Naive Bayes. The dataset is preprocessed through data cleaning, standardization and feature selection techniques using Principal Component Analysis. The research explores the performance of these algorithms in terms of precision, recall and F1-score as well as time complexity to create a reliable system capable of real-time detection and mitigation of DDoS attacks. The findings indicate that RF, AdaBoost and XGBoost outperform other algorithms in accuracy and efficiency, making them ideal candidates for real-time applications.

Title: Clear Minds Think Alike: What Makes LLM Fine-tuning Robust? A Study of Token Perplexity

• Authors: Chao-Chung Wu, Zhi Rui Tam, Chieh-Yen Lin, Hung-yi Lee, Yun-Nung Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14315
• Pdf URL: https://arxiv.org/pdf/2501.14315
• Copy Paste: [[2501.14315]] Clear Minds Think Alike: What Makes LLM Fine-tuning Robust? A Study of Token Perplexity(https://arxiv.org/abs/2501.14315)
• Keywords: robust
• Abstract: Maintaining consistent model performance across domains is a fundamental challenge in machine learning. While recent work has explored using LLM-generated data for fine-tuning, its impact on cross-domain generalization remains poorly understood. In this paper, we present a systematic analysis revealing that fine-tuning with LLM-generated data not only improves target task performance but also reduces out-of-domain (OOD) degradation compared to fine-tuning with ground truth data. Through analyzing the data sequence in tasks of various domains, we demonstrate that this enhanced OOD robustness stems from a reduced prevalence of high perplexity tokens in LLM-generated sequences. Following this hypothesis we showed that masking high perplexity tokens in ground truth training data also achieves similar OOD preservation comparable to using LLM-generated data. Extensive experiments across diverse model architectures and scales, including Gemma2-2B, Mistral-7B and Llama3-8B, corroborate the consistency of our findings. To the best of our knowledge, this work provides the first mechanistic explanation for the superior OOD robustness conferred by LLM-generated training data, offering valuable insights for developing more robust fine-tuning strategies.

Title: Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video

• Authors: Xiaohao Xu, Tianyi Zhang, Shibo Zhao, Xiang Li, Sibo Wang, Yongqi Chen, Ye Li, Bhiksha Raj, Matthew Johnson-Roberson, Sebastian Scherer, Xiaonan Huang
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.14319
• Pdf URL: https://arxiv.org/pdf/2501.14319
• Copy Paste: [[2501.14319]] Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video(https://arxiv.org/abs/2501.14319)
• Keywords: robust
• Abstract: We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: the reliance on noise-free data in existing models. While such sanitized conditions simplify evaluation, they fail to capture the unpredictable, noisy complexities of real-world environments. Dynamic motion, sensor imperfections, and synchronization perturbations lead to sharp performance declines when these models are deployed in practice, revealing an urgent need for frameworks that embrace and excel under real-world noise. To bridge this gap, we tackle three core challenges: scalable data generation, comprehensive benchmarking, and model robustness enhancement. First, we introduce a scalable noisy data synthesis pipeline that generates diverse datasets simulating complex motion, sensor imperfections, and synchronization errors. Second, we leverage this pipeline to create Robust-Ego3D, a benchmark rigorously designed to expose noise-induced performance degradation, highlighting the limitations of current learning-based methods in ego-motion accuracy and 3D reconstruction quality. Third, we propose Correspondence-guided Gaussian Splatting (CorrGS), a novel test-time adaptation method that progressively refines an internal clean 3D representation by aligning noisy observations with rendered RGB-D frames from clean 3D map, enhancing geometric alignment and appearance restoration through visual correspondence. Extensive experiments on synthetic and real-world data demonstrate that CorrGS consistently outperforms prior state-of-the-art methods, particularly in scenarios involving rapid motion and dynamic illumination.

Title: Relative Layer-Wise Relevance Propagation: a more Robust Neural Networks eXplaination

• Authors: Eric Nyiri, Olivier Gibaru
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14322
• Pdf URL: https://arxiv.org/pdf/2501.14322
• Copy Paste: [[2501.14322]] Relative Layer-Wise Relevance Propagation: a more Robust Neural Networks eXplaination(https://arxiv.org/abs/2501.14322)
• Keywords: robust
• Abstract: Machine learning methods are solving very successfully a plethora of tasks, but they have the disadvantage of not providing any information about their decision. Consequently, estimating the reasoning of the system provides additional information. For this, Layer-Wise Relevance Propagation (LRP) is one of the methods in eXplainable Machine Learning (XML). Its purpose is to provide contributions of any neural network output in the domain of its input. The main drawback of current methods is mainly due to division by small values. To overcome this problem, we provide a new definition called Relative LRP where the classical conservation law is satisfied up to a multiplicative factor but without divisions by small values except for Resnet skip connection. In this article, we will focus on image classification. This allows us to visualize the contributions of a pixel to the predictions of a multi-layer neural network. Pixel contributions provide a focus to further analysis on regions of potential interest. R-LRP can be applied for any dense, CNN or residual neural networks. Moreover, R-LRP doesn't need any hyperparameters to tune contrary to other LRP methods. We then compare the R-LRP method on different datasets with simple CNN, VGG16, VGG19 and Resnet50 networks.

Title: Securing DRAM at Scale: ARFM-Driven Row Hammer Defense with Unveiling the Threat of Short tRC Patterns

• Authors: Nogeun Joo, Donghyuk Kim, Hyunjun Cho, Junseok Noh, Dongha Jung, Joo-Young Kim
• Subjects: cs.CR, cs.AR
• Abstract URL: https://arxiv.org/abs/2501.14328
• Pdf URL: https://arxiv.org/pdf/2501.14328
• Copy Paste: [[2501.14328]] Securing DRAM at Scale: ARFM-Driven Row Hammer Defense with Unveiling the Threat of Short tRC Patterns(https://arxiv.org/abs/2501.14328)
• Keywords: security, defense, attack
• Abstract: To address the issue of powerful row hammer (RH) attacks, our study involved an extensive analysis of the prevalent attack patterns in the field. We discovered a strong correlation between the timing and density of the active-to-active command period, ${tRC}$, and the likelihood of RH attacks. In this paper, we introduce MARC, an innovative ARFM-driven RH mitigation IP that significantly reinforces existing RH mitigation IPs. MARC dynamically adjusts the frequency of RFM in response to the severity of the RH attack environment, offering a tailored security solution that not only detects the threats but also adapts to varying threat levels. MARC's detection mechanism has demonstrated remarkable efficiency, identifying over 99\% of attack patterns. Moreover, MARC is designed as a compact hardware module, facilitating tight integration either on the memory controller-side or DRAM-side within the memory system. It only occupies a negligible hardware area of 3363~\textit{$\mu m^2$}. By activating ARFM based on MARC's detection, the additional energy overhead is also negligible in normal workloads. We conduct experiments to compare the highest row count throughout the patterns, defined as max exposure, between the vanilla RH mitigation IPs and the MARC-enhanced versions of the same IPs, focusing on both DRAM-side and memory controller-side. On the DRAM-side, MARC + probabilistic scheme and MARC + counter-based tracking scheme achieve 8.1$\times$ and 1.5$\times$ improvement in max exposure ratio compared to the vanilla IPs, respectively. On the memory controller-side, the MARC + PARA and MARC + Graphene achieve 50$\times$ and 5.7$\times$ improvement in max exposure ratio compared to the vanilla IPs, respectively. MARC ensures optimal security without sacrificing system performance, making MARC a pioneering solution in the realm of RH attack mitigation.

Title: Online Authentication Habits of Indian Users

• Authors: Pratyush Choudhary, Subhrajit Das, Mukul Paras Potta, Prasuj Das, Abhishek Bichhawat
• Subjects: cs.CR, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.14330
• Pdf URL: https://arxiv.org/pdf/2501.14330
• Copy Paste: [[2501.14330]] Online Authentication Habits of Indian Users(https://arxiv.org/abs/2501.14330)
• Keywords: secure, security
• Abstract: Passwords have been long used as the primary authentication method for web services. Weak passwords used by the users have prompted the use of password management tools and two-factor authentication to ensure better account security. While prior studies have studied their adoption individually, none of these studies focuses particularly on the Indian setting, which is culturally and economically different from the countries in which these studies have been done in the past. To this end, we conducted a survey with 90 participants residing in India to better understand the mindset of people on using password managers and two-factor authentication (2FA). Our findings suggest that a majority of the participants have used 2FA and password managers in some form, although they are sometimes unaware of their formal names. While many participants used some form of 2FA across all their accounts, browser-integrated and device-default password managers are predominantly utilized for less sensitive platforms such as e-commerce and social media rather than for more critical accounts like banking. The primary motivation for using password managers is the convenience of auto-filling. However, some participants avoid using password managers due to a lack of trust in these tools. Notably, dedicated third-party applications show low adoption for both password manager and 2FA. Despite acknowledging the importance of secure password practices, many participants still reuse passwords across multiple accounts, prefer shorter passwords, and use commonly predictable password patterns. Overall, the study suggests that Indians are more inclined to choose default settings, underscoring the need for tailored strategies to improve user awareness and strengthen password security practices.

Title: Online Inverse Linear Optimization: Improved Regret Bound, Robustness to Suboptimality, and Toward Tight Regret Analysis

• Authors: Shinsaku Sakaue, Taira Tsuchiya, Han Bao, Taihei Oki
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14349
• Pdf URL: https://arxiv.org/pdf/2501.14349
• Copy Paste: [[2501.14349]] Online Inverse Linear Optimization: Improved Regret Bound, Robustness to Suboptimality, and Toward Tight Regret Analysis(https://arxiv.org/abs/2501.14349)
• Keywords: robust
• Abstract: We study an online learning problem where, over $T$ rounds, a learner observes both time-varying sets of feasible actions and an agent's optimal actions, selected by solving linear optimization over the feasible actions. The learner sequentially makes predictions of the agent's underlying linear objective function, and their quality is measured by the regret, the cumulative gap between optimal objective values and those achieved by following the learner's predictions. A seminal work by Bärmann et al. (ICML 2017) showed that online learning methods can be applied to this problem to achieve regret bounds of $O(\sqrt{T})$. Recently, Besbes et al. (COLT 2021, Oper. Res. 2023) significantly improved the result by achieving an $O(n^4\ln T)$ regret bound, where $n$ is the dimension of the ambient space of objective vectors. Their method, based on the ellipsoid method, runs in polynomial time but is inefficient for large $n$ and $T$. In this paper, we obtain an $O(n\ln T)$ regret bound, improving upon the previous bound of $O(n^4\ln T)$ by a factor of $n^3$. Our method is simple and efficient: we apply the online Newton step (ONS) to appropriate exp-concave loss functions. Moreover, for the case where the agent's actions are possibly suboptimal, we establish an $O(n\ln T+\sqrt{\Delta_Tn\ln T})$ regret bound, where $\Delta_T$ is the cumulative suboptimality of the agent's actions. This bound is achieved by using MetaGrad, which runs ONS with $\Theta(\ln T)$ different learning rates in parallel. We also provide a simple instance that implies an $\Omega(n)$ lower bound, showing that our $O(n\ln T)$ bound is tight up to an $O(\ln T)$ factor. This gives rise to a natural question: can the $O(\ln T)$ factor in the upper bound be removed? For the special case of $n=2$, we show that an $O(1)$ regret bound is possible, while we delineate challenges in extending this result to higher dimensions.

Title: Causal-Inspired Multitask Learning for Video-Based Human Pose Estimation

• Authors: Haipeng Chen, Sifan Wu, Zhigang Wang, Yifang Yin, Yingying Jiao, Yingda Lyu, Zhenguang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14356
• Pdf URL: https://arxiv.org/pdf/2501.14356
• Copy Paste: [[2501.14356]] Causal-Inspired Multitask Learning for Video-Based Human Pose Estimation(https://arxiv.org/abs/2501.14356)
• Keywords: robust, interpretability
• Abstract: Video-based human pose estimation has long been a fundamental yet challenging problem in computer vision. Previous studies focus on spatio-temporal modeling through the enhancement of architecture design and optimization strategies. However, they overlook the causal relationships in the joints, leading to models that may be overly tailored and thus estimate poorly to challenging scenes. Therefore, adequate causal reasoning capability, coupled with good interpretability of model, are both indispensable and prerequisite for achieving reliable results. In this paper, we pioneer a causal perspective on pose estimation and introduce a causal-inspired multitask learning framework, consisting of two stages. \textit{In the first stage}, we try to endow the model with causal spatio-temporal modeling ability by introducing two self-supervision auxiliary tasks. Specifically, these auxiliary tasks enable the network to infer challenging keypoints based on observed keypoint information, thereby imbuing causal reasoning capabilities into the model and making it robust to challenging scenes. \textit{In the second stage}, we argue that not all feature tokens contribute equally to pose estimation. Prioritizing causal (keypoint-relevant) tokens is crucial to achieve reliable results, which could improve the interpretability of the model. To this end, we propose a Token Causal Importance Selection module to identify the causal tokens and non-causal tokens (\textit{e.g.}, background and objects). Additionally, non-causal tokens could provide potentially beneficial cues but may be redundant. We further introduce a non-causal tokens clustering module to merge the similar non-causal tokens. Extensive experiments show that our method outperforms state-of-the-art methods on three large-scale benchmark datasets.

Title: Low-rank Prompt Interaction for Continual Vision-Language Retrieval

• Authors: Weicai Yan, Ye Wang, Wang Lin, Zirun Guo, Zhou Zhao, Tao Jin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14369
• Pdf URL: https://arxiv.org/pdf/2501.14369
• Copy Paste: [[2501.14369]] Low-rank Prompt Interaction for Continual Vision-Language Retrieval(https://arxiv.org/abs/2501.14369)
• Keywords: robust, transformer
• Abstract: Research on continual learning in multi-modal tasks has been receiving increasing attention. However, most existing work overlooks the explicit cross-modal and cross-task interactions. In this paper, we innovatively propose the Low-rank Prompt Interaction (LPI) to address this general problem of multi-modal understanding, which considers both cross-modal and cross-task interactions. Specifically, as for the former, we employ multi-modal correlation modules for corresponding Transformer layers. Considering that the training parameters scale to the number of layers and tasks, we propose low-rank interaction-augmented decomposition to avoid memory explosion while enhancing the cross-modal association through sharing and separating common-specific low-rank factors. In addition, due to the multi-modal semantic differences carried by the low-rank initialization, we adopt hierarchical low-rank contrastive learning to ensure training robustness. As for the latter, we initially employ a visual analysis and identify that different tasks have clear distinctions in proximity. Therefore, we introduce explicit task contrastive constraints in the prompt learning process based on task semantic distances. Experiments on two retrieval tasks show performance improvements with the introduction of a minimal number of parameters, demonstrating the effectiveness of our method. Code is available at this https URL.

Title: DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

• Authors: Xinyu Ma, Yifeng Xu, Yang Lin, Tianlong Wang, Xu Chu, Xin Gao, Junfeng Zhao, Yasha Wang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14371
• Pdf URL: https://arxiv.org/pdf/2501.14371
• Copy Paste: [[2501.14371]] DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing(https://arxiv.org/abs/2501.14371)
• Keywords: large language model
• Abstract: We introduce DRESS, a novel approach for generating stylized large language model (LLM) responses through representation editing. Existing methods like prompting and fine-tuning are either insufficient for complex style adaptation or computationally expensive, particularly in tasks like NPC creation or character role-playing. Our approach leverages the over-parameterized nature of LLMs to disentangle a style-relevant subspace within the model's representation space to conduct representation editing, ensuring a minimal impact on the original semantics. By applying adaptive editing strengths, we dynamically adjust the steering vectors in the style subspace to maintain both stylistic fidelity and semantic integrity. We develop two stylized QA benchmark datasets to validate the effectiveness of DRESS, and the results demonstrate significant improvements compared to baseline methods such as prompting and ITI. In short, DRESS is a lightweight, train-free solution for enhancing LLMs with flexible and effective style control, making it particularly useful for developing stylized conversational agents. Codes and benchmark datasets are available at this https URL.

Title: Distinguishing Parkinson's Patients Using Voice-Based Feature Extraction and Classification

• Authors: Burak Çelik, Ayhan Akbal
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14390
• Pdf URL: https://arxiv.org/pdf/2501.14390
• Copy Paste: [[2501.14390]] Distinguishing Parkinson's Patients Using Voice-Based Feature Extraction and Classification(https://arxiv.org/abs/2501.14390)
• Keywords: extraction, generative
• Abstract: Parkinson's disease (PD) is a progressive neurodegenerative disorder that impacts motor functions and speech characteristics This study focuses on differentiating individuals with Parkinson's disease from healthy controls through the extraction and classification of speech features. Patients were further divided into 2 groups. Med On represents the patient with medication, while Med Off represents the patient without medication. The dataset consisted of patients and healthy individuals who read a predefined text using the H1N Zoom microphone in a suitable recording environment at Fırat University Neurology Department. Speech recordings from PD patients and healthy controls were analyzed, and 19 key features were extracted, including jitter, luminance, zero-crossing rate (ZCR), root mean square (RMS) energy, entropy, skewness, and this http URL features were visualized in graphs and statistically evaluated to identify distinctive patterns in PD patients. Using MATLAB's Classification Learner toolbox, several machine learning classification algorithm models were applied to classify groups and significant accuracy rates were achieved. The accuracy of our 3-layer artificial neural network architecture was also compared with classical machine learning algorithms. This study highlights the potential of noninvasive voice analysis combined with machine learning for early detection and monitoring of PD patients. Future research can improve diagnostic accuracy by optimizing feature selection and exploring advanced classification techniques.

Title: Context-CrackNet: A Context-Aware Framework for Precise Segmentation of Tiny Cracks in Pavement images

• Authors: Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14413
• Pdf URL: https://arxiv.org/pdf/2501.14413
• Copy Paste: [[2501.14413]] Context-CrackNet: A Context-Aware Framework for Precise Segmentation of Tiny Cracks in Pavement images(https://arxiv.org/abs/2501.14413)
• Keywords: segmentation
• Abstract: The accurate detection and segmentation of pavement distresses, particularly tiny and small cracks, are critical for early intervention and preventive maintenance in transportation infrastructure. Traditional manual inspection methods are labor-intensive and inconsistent, while existing deep learning models struggle with fine-grained segmentation and computational efficiency. To address these challenges, this study proposes Context-CrackNet, a novel encoder-decoder architecture featuring the Region-Focused Enhancement Module (RFEM) and Context-Aware Global Module (CAGM). These innovations enhance the model's ability to capture fine-grained local details and global contextual dependencies, respectively. Context-CrackNet was rigorously evaluated on ten publicly available crack segmentation datasets, covering diverse pavement distress scenarios. The model consistently outperformed 9 state-of-the-art segmentation frameworks, achieving superior performance metrics such as mIoU and Dice score, while maintaining competitive inference efficiency. Ablation studies confirmed the complementary roles of RFEM and CAGM, with notable improvements in mIoU and Dice score when both modules were integrated. Additionally, the model's balance of precision and computational efficiency highlights its potential for real-time deployment in large-scale pavement monitoring systems.

Title: SoK: What Makes Private Learning Unfair?

• Authors: Kai Yao, Marc Juarez
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14414
• Pdf URL: https://arxiv.org/pdf/2501.14414
• Copy Paste: [[2501.14414]] SoK: What Makes Private Learning Unfair?(https://arxiv.org/abs/2501.14414)
• Keywords: privacy, fair
• Abstract: Differential privacy has emerged as the most studied framework for privacy-preserving machine learning. However, recent studies show that enforcing differential privacy guarantees can not only significantly degrade the utility of the model, but also amplify existing disparities in its predictive performance across demographic groups. Although there is extensive research on the identification of factors that contribute to this phenomenon, we still lack a complete understanding of the mechanisms through which differential privacy exacerbates disparities. The literature on this problem is muddled by varying definitions of fairness, differential privacy mechanisms, and inconsistent experimental settings, often leading to seemingly contradictory results. This survey provides the first comprehensive overview of the factors that contribute to the disparate effect of training models with differential privacy guarantees. We discuss their impact and analyze their causal role in such a disparate effect. Our analysis is guided by a taxonomy that categorizes these factors by their position within the machine learning pipeline, allowing us to draw conclusions about their interaction and the feasibility of potential mitigation strategies. We find that factors related to the training dataset and the underlying distribution play a decisive role in the occurrence of disparate impact, highlighting the need for research on these factors to address the issue.

Title: Timelock-Free Rationally-Secure Virtual Channels

• Authors: Zeta Avarikioti, Yuheng Wang, Yuyi Wang
• Subjects: cs.CR, cs.DC, cs.GT
• Abstract URL: https://arxiv.org/abs/2501.14418
• Pdf URL: https://arxiv.org/pdf/2501.14418
• Copy Paste: [[2501.14418]] Timelock-Free Rationally-Secure Virtual Channels(https://arxiv.org/abs/2501.14418)
• Keywords: secure, security, attack
• Abstract: Payment channel networks (PCNs) offer a promising solution to address the limited transaction throughput of deployed blockchains. However, several attacks have recently been proposed that stress the vulnerability of PCNs to timelock and censoring attacks. To address such attacks, we introduce Thunderdome, the first timelock-free PCN. Instead, Thunderdome leverages the design rationale of virtual channels to extend a timelock-free payment channel primitive, thereby enabling multi-hop transactions without timelocks. Previous works either utilize timelocks or do not accommodate transactions between parties that do not share a channel. At its core, Thunderdome relies on a committee of non-trusted watchtowers, known as wardens, who ensure that no honest party loses funds, even when offline, during the channel closure process. We introduce tailored incentive mechanisms to ensure that all participants follow the protocol's correct execution. Besides a traditional security proof that assumes an honest majority of the committee, we conduct a formal game-theoretic analysis to demonstrate the security of Thunderdome when all participants, including wardens, act rationally. We implement a proof of concept of Thunderdome on Ethereum to validate its feasibility and evaluate its costs. Our evaluation shows that deploying Thunderdome, including opening the underlying payment channel, costs approximately \$15 (0.0089 ETH), while the worst-case cost for closing a channel is about \$7 (0.004 ETH).

Title: CENTS: Generating synthetic electricity consumption time series for rare and unseen scenarios

• Authors: Michael Fuest, Alfredo Cuesta, Kalyan Veeramachaneni
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14426
• Pdf URL: https://arxiv.org/pdf/2501.14426
• Copy Paste: [[2501.14426]] CENTS: Generating synthetic electricity consumption time series for rare and unseen scenarios(https://arxiv.org/abs/2501.14426)
• Keywords: generative
• Abstract: Recent breakthroughs in large-scale generative modeling have demonstrated the potential of foundation models in domains such as natural language, computer vision, and protein structure prediction. However, their application in the energy and smart grid sector remains limited due to the scarcity and heterogeneity of high-quality data. In this work, we propose a method for creating high-fidelity electricity consumption time series data for rare and unseen context variables (e.g. location, building type, photovoltaics). Our approach, Context Encoding and Normalizing Time Series Generation, or CENTS, includes three key innovations: (i) A context normalization approach that enables inverse transformation for time series context variables unseen during training, (ii) a novel context encoder to condition any state-of-the-art time-series generator on arbitrary numbers and combinations of context variables, (iii) a framework for training this context encoder jointly with a time-series generator using an auxiliary context classification loss designed to increase expressivity of context embeddings and improve model performance. We further provide a comprehensive overview of different evaluation metrics for generative time series models. Our results highlight the efficacy of the proposed method in generating realistic household-level electricity consumption data, paving the way for training larger foundation models in the energy domain on synthetic as well as real-world data.

Title: GraphBC: Improving LLMs for Better Graph Data Processing

• Authors: Xu Chu, Hanlin Xue, Zhijie Tan, Bingce Wang, Tong Mo, Weiping Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14427
• Pdf URL: https://arxiv.org/pdf/2501.14427
• Copy Paste: [[2501.14427]] GraphBC: Improving LLMs for Better Graph Data Processing(https://arxiv.org/abs/2501.14427)
• Keywords: large language model
• Abstract: The success of Large Language Models (LLMs) in various domains has led researchers to apply them to graph-related problems by converting graph data into natural language text. However, unlike graph data, natural language inherently has sequential order. We observe that when the order of nodes or edges in the natural language description of a graph is shuffled, despite describing the same graph, model performance fluctuates between high performance and random guessing. Additionally, due to the limited input context length of LLMs, current methods typically randomly sample neighbors of target nodes as representatives of their neighborhood, which may not always be effective for accurate reasoning. To address these gaps, we introduce GraphBC. This novel model framework features an Order Selector Module to ensure proper serialization order of the graph and a Subgraph Sampling Module to sample subgraphs with better structure for better reasoning. Furthermore, we propose Graph CoT obtained through distillation, and enhance LLM's reasoning and zero-shot learning capabilities for graph tasks through instruction tuning. Experiments on multiple datasets for node classification and graph question-answering demonstrate that GraphBC improves LLMs' performance and generalization ability on graph tasks.

Title: Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains

• Authors: Xu Chu, Zhijie Tan, Hanlin Xue, Guanyu Wang, Tong Mo, Weiping Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14431
• Pdf URL: https://arxiv.org/pdf/2501.14431
• Copy Paste: [[2501.14431]] Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains(https://arxiv.org/abs/2501.14431)
• Keywords: explainability, large language model
• Abstract: Large Language Models (LLMs) are widely applied to downstream domains. However, current LLMs for high-stakes domain tasks, such as financial investment and legal QA, typically generate brief answers without reasoning processes and explanations. This limits users' confidence in making decisions based on their responses. While original CoT shows promise, it lacks self-correction mechanisms during reasoning. This work introduces Domain$o1$s, which enhances LLMs' reasoning capabilities on domain tasks through supervised fine-tuning and tree search. We construct CoT-stock-2k and CoT-legal-2k datasets for fine-tuning models that activate domain-specific reasoning steps based on their judgment. Additionally, we propose Selective Tree Exploration to spontaneously explore solution spaces and sample optimal reasoning paths to improve performance. We also introduce PROOF-Score, a new metric for evaluating domain models' explainability, complementing traditional accuracy metrics with richer assessment dimensions. Extensive experiments on stock investment recommendation and legal reasoning QA tasks demonstrate Domaino1s's leading performance and explainability. Our code is available at this https URL.

Title: Optimizing Human Pose Estimation Through Focused Human and Joint Regions

• Authors: Yingying Jiao, Zhigang Wang, Zhenguang Liu, Shaojing Fan, Sifan Wu, Zheqi Wu, Zhuoyue Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14439
• Pdf URL: https://arxiv.org/pdf/2501.14439
• Copy Paste: [[2501.14439]] Optimizing Human Pose Estimation Through Focused Human and Joint Regions(https://arxiv.org/abs/2501.14439)
• Keywords: transformer
• Abstract: Human pose estimation has given rise to a broad spectrum of novel and compelling applications, including action recognition, sports analysis, as well as surveillance. However, accurate video pose estimation remains an open challenge. One aspect that has been overlooked so far is that existing methods learn motion clues from all pixels rather than focusing on the target human body, making them easily misled and disrupted by unimportant information such as background changes or movements of other people. Additionally, while the current Transformer-based pose estimation methods has demonstrated impressive performance with global modeling, they struggle with local context perception and precise positional identification. In this paper, we try to tackle these challenges from three aspects: (1) We propose a bilayer Human-Keypoint Mask module that performs coarse-to-fine visual token refinement, which gradually zooms in on the target human body and keypoints while masking out unimportant figure regions. (2) We further introduce a novel deformable cross attention mechanism and a bidirectional separation strategy to adaptively aggregate spatial and temporal motion clues from constrained surrounding contexts. (3) We mathematically formulate the deformable cross attention, constraining that the model focuses solely on the regions centered at the target person body. Empirically, our method achieves state-of-the-art performance on three large-scale benchmark datasets. A remarkable highlight is that our method achieves an 84.8 mean Average Precision (mAP) on the challenging wrist joint, which significantly outperforms the 81.5 mAP achieved by the current state-of-the-art method on the PoseTrack2017 dataset.

Title: Optimal Strategies for Federated Learning Maintaining Client Privacy

• Authors: Uday Bhaskar, Varul Srivastava, Avyukta Manjunatha Vummintala, Naresh Manwani, Sujit Gujar
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14453
• Pdf URL: https://arxiv.org/pdf/2501.14453
• Copy Paste: [[2501.14453]] Optimal Strategies for Federated Learning Maintaining Client Privacy(https://arxiv.org/abs/2501.14453)
• Keywords: privacy, protect, attack, federate
• Abstract: Federated Learning (FL) emerged as a learning method to enable the server to train models over data distributed among various clients. These clients are protective about their data being leaked to the server, any other client, or an external adversary, and hence, locally train the model and share it with the server rather than sharing the data. The introduction of sophisticated inferencing attacks enabled the leakage of information about data through access to model parameters. To tackle this challenge, privacy-preserving federated learning aims to achieve differential privacy through learning algorithms like DP-SGD. However, such methods involve adding noise to the model, data, or gradients, reducing the model's performance. This work provides a theoretical analysis of the tradeoff between model performance and communication complexity of the FL system. We formally prove that training for one local epoch per global round of training gives optimal performance while preserving the same privacy budget. We also investigate the change of utility (tied to privacy) of FL models with a change in the number of clients and observe that when clients are training using DP-SGD and argue that for the same privacy budget, the utility improved with increased clients. We validate our findings through experiments on real-world datasets. The results from this paper aim to improve the performance of privacy-preserving federated learning systems.

Title: Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing

• Authors: Zeping Yu, Sophia Ananiadou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14457
• Pdf URL: https://arxiv.org/pdf/2501.14457
• Copy Paste: [[2501.14457]] Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing(https://arxiv.org/abs/2501.14457)
• Keywords: large language model
• Abstract: Large language models (LLMs) often exhibit gender bias, posing challenges for their safe deployment. Existing methods to mitigate bias lack a comprehensive understanding of its mechanisms or compromise the model's core capabilities. To address these issues, we propose the CommonWords dataset, to systematically evaluate gender bias in LLMs. Our analysis reveals pervasive bias across models and identifies specific neuron circuits, including gender neurons and general neurons, responsible for this behavior. Notably, editing even a small number of general neurons can disrupt the model's overall capabilities due to hierarchical neuron interactions. Based on these insights, we propose an interpretable neuron editing method that combines logit-based and causal-based strategies to selectively target biased neurons. Experiments on five LLMs demonstrate that our method effectively reduces gender bias while preserving the model's original capabilities, outperforming existing fine-tuning and editing approaches. Our findings contribute a novel dataset, a detailed analysis of bias mechanisms, and a practical solution for mitigating gender bias in LLMs.

Title: Pesti-Gen: Unleashing a Generative Molecule Approach for Toxicity Aware Pesticide Design

• Authors: Taehan Kim, Wonduk Seo
• Subjects: cs.LG, cs.AI, q-bio.BM, q-bio.MN
• Abstract URL: https://arxiv.org/abs/2501.14469
• Pdf URL: https://arxiv.org/pdf/2501.14469
• Copy Paste: [[2501.14469]] Pesti-Gen: Unleashing a Generative Molecule Approach for Toxicity Aware Pesticide Design(https://arxiv.org/abs/2501.14469)
• Keywords: generative
• Abstract: Global climate change has reduced crop resilience and pesticide efficacy, making reliance on synthetic pesticides inevitable, even though their widespread use poses significant health and environmental risks. While these pesticides remain a key tool in pest management, previous machine-learning applications in pesticide and agriculture have focused on classification or regression, leaving the fundamental challenge of generating new molecular structures or designing novel candidates unaddressed. In this paper, we propose Pesti-Gen, a novel generative model based on variational auto-encoders, designed to create pesticide candidates with optimized properties for the first time. Specifically, Pesti-Gen leverages a two-stage learning process: an initial pre-training phase that captures a generalized chemical structure representation, followed by a fine-tuning stage that incorporates toxicity-specific information. The model simultaneously optimizes over multiple toxicity metrics, such as (1) livestock toxicity and (2) aqua toxicity to generate environmentally friendly pesticide candidates. Notably, Pesti-Gen achieves approximately 68\% structural validity in generating new molecular structures, demonstrating the model's effectiveness in producing optimized and feasible pesticide candidates, thereby providing a new way for safer and more sustainable pest management solutions.

Title: RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques

• Authors: Zhengyang Tang, Ziniu Li, Zhenyang Xiao, Tian Ding, Ruoyu Sun, Benyou Wang, Dayiheng Liu, Fei Huang, Tianyu Liu, Bowen Yu, Junyang Lin
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14492
• Pdf URL: https://arxiv.org/pdf/2501.14492
• Copy Paste: [[2501.14492]] RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques(https://arxiv.org/abs/2501.14492)
• Keywords: large language model
• Abstract: Critiques are important for enhancing the performance of Large Language Models (LLMs), enabling both self-improvement and constructive feedback for others by identifying flaws and suggesting improvements. However, evaluating the critique capabilities of LLMs presents a significant challenge due to the open-ended nature of the task. In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs. Unlike existing benchmarks, which typically function in an open-loop fashion, our approach employs a closed-loop methodology that evaluates the quality of corrections generated from critiques. Moreover, the benchmark incorporates features such as self-critique, cross-critique, and iterative critique, which are crucial for distinguishing the abilities of advanced reasoning models from more classical ones. We implement this benchmark using eight challenging reasoning tasks. We have several interesting findings. First, despite demonstrating comparable performance in direct chain-of-thought generation, classical LLMs significantly lag behind the advanced reasoning-based model o1-mini across all critique scenarios. Second, in self-critique and iterative critique settings, classical LLMs may even underperform relative to their baseline capabilities. We hope that this benchmark will serve as a valuable resource to guide future advancements. The code and data are available at \url{this https URL}.

Title: A Note on Implementation Errors in Recent Adaptive Attacks Against Multi-Resolution Self-Ensembles

• Authors: Stanislav Fort
• Subjects: cs.CR, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14496
• Pdf URL: https://arxiv.org/pdf/2501.14496
• Copy Paste: [[2501.14496]] A Note on Implementation Errors in Recent Adaptive Attacks Against Multi-Resolution Self-Ensembles(https://arxiv.org/abs/2501.14496)
• Keywords: defense, attack, robust
• Abstract: This note documents an implementation issue in recent adaptive attacks (Zhang et al. [2024]) against the multi-resolution self-ensemble defense (Fort and Lakshminarayanan [2024]). The implementation allowed adversarial perturbations to exceed the standard $L_\infty = 8/255$ bound by up to a factor of 20$\times$, reaching magnitudes of up to $L_\infty = 160/255$. When attacks are properly constrained within the intended bounds, the defense maintains non-trivial robustness. Beyond highlighting the importance of careful validation in adversarial machine learning research, our analysis reveals an intriguing finding: properly bounded adaptive attacks against strong multi-resolution self-ensembles often align with human perception, suggesting the need to reconsider how we measure adversarial robustness.

Title: Evaluating and Improving Graph to Text Generation with Large Language Models

• Authors: Jie He, Yijun Yang, Wanqiu Long, Deyi Xiong, Victor Gutierrez Basulto, Jeff Z. Pan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14497
• Pdf URL: https://arxiv.org/pdf/2501.14497
• Copy Paste: [[2501.14497]] Evaluating and Improving Graph to Text Generation with Large Language Models(https://arxiv.org/abs/2501.14497)
• Keywords: large language model
• Abstract: Large language models (LLMs) have demonstrated immense potential across various tasks. However, research for exploring and improving the capabilities of LLMs in interpreting graph structures remains limited. To address this gap, we conduct a comprehensive evaluation of prompting current open-source LLMs on graph-to-text generation tasks. Although we explored the optimal prompting strategies and proposed a novel and effective diversity-difficulty-based few-shot sample selection method, we found that the improvements from tuning-free approaches were incremental, as LLMs struggle with planning on complex graphs, particularly those with a larger number of triplets. To further improve LLMs in planning with graph sequences and grounding in truth, we introduce a new graph-to-text dataset, PlanGTG, annotated with two sub-tasks: reordering and attribution. Through extensive automatic and human evaluations, we demonstrate significant improvements in the quality of generated text from both few-shot learning and fine-tuning perspectives using the PlanGTG dataset. Our study paves the way for new research directions in graph-to-text generation. PlanGTG datasets can be found in this https URL.

Title: Automated Assignment Grading with Large Language Models: Insights From a Bioinformatics Course

• Authors: Pavlin G. Poličar, Martin Špendl, Tomaž Curk, Blaž Zupan
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2501.14499
• Pdf URL: https://arxiv.org/pdf/2501.14499
• Copy Paste: [[2501.14499]] Automated Assignment Grading with Large Language Models: Insights From a Bioinformatics Course(https://arxiv.org/abs/2501.14499)
• Keywords: privacy, large language model
• Abstract: Providing students with individualized feedback through assignments is a cornerstone of education that supports their learning and development. Studies have shown that timely, high-quality feedback plays a critical role in improving learning outcomes. However, providing personalized feedback on a large scale in classes with large numbers of students is often impractical due to the significant time and effort required. Recent advances in natural language processing and large language models (LLMs) offer a promising solution by enabling the efficient delivery of personalized feedback. These technologies can reduce the workload of course staff while improving student satisfaction and learning outcomes. Their successful implementation, however, requires thorough evaluation and validation in real classrooms. We present the results of a practical evaluation of LLM-based graders for written assignments in the 2024/25 iteration of the Introduction to Bioinformatics course at the University of Ljubljana. Over the course of the semester, more than 100 students answered 36 text-based questions, most of which were automatically graded using LLMs. In a blind study, students received feedback from both LLMs and human teaching assistants without knowing the source, and later rated the quality of the feedback. We conducted a systematic evaluation of six commercial and open-source LLMs and compared their grading performance with human teaching assistants. Our results show that with well-designed prompts, LLMs can achieve grading accuracy and feedback quality comparable to human graders. Our results also suggest that open-source LLMs perform as well as commercial LLMs, allowing schools to implement their own grading systems while maintaining privacy.

Title: NIFuzz: Estimating Quantified Information Flow with a Fuzzer

• Authors: Daniel Blackwell, Ingolf Becker, David Clark
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2501.14500
• Pdf URL: https://arxiv.org/pdf/2501.14500
• Copy Paste: [[2501.14500]] NIFuzz: Estimating Quantified Information Flow with a Fuzzer(https://arxiv.org/abs/2501.14500)
• Keywords: security
• Abstract: This paper presents a scalable, practical approach to quantifying information leaks in software; these errors are often overlooked and downplayed, but can seriously compromise security mechanisms such as address space layout randomisation (ASLR) and Pointer Authentication (PAC). We introduce approaches for three different metrics to estimate the size of information leaks, including a new derivation for the calculation of conditional mutual information. Together, these metrics can inform of the relative safety of the target program against different threat models and provide useful details for finding the source of any leaks. We provide an implementation of a fuzzer, NIFuzz, which is capable of dynamically computing these metrics with little overhead and has several strategies to optimise for the detection and quantification of information leaks. We evaluate NIFuzz on a set of 14 programs -- including 8 real-world CVEs and ranging up to 278k lines of code in size -- where we find that it is capable of detecting and providing good estimates for all of the known information leaks.

Title: WanJuanSiLu: A High-Quality Open-Source Webtext Dataset for Low-Resource Languages

• Authors: Jia Yu, Fei Yuan, Rui Min, Jing Yu, Pei Chu, Jiayang Li, Wei Li, Ruijie Zhang, Zhenxiang Li, Zhifei Ren, Dong Zheng, Wenjian Zhang, Yan Teng, Lingyu Meng, ZhenJiang Jin, Jiantao Qiu, ShaSha Wang, Zhongying Tu, Dahua Lin, Yu Wang, Yu Qiao, Yanfeng Wang, Conghui He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14506
• Pdf URL: https://arxiv.org/pdf/2501.14506
• Copy Paste: [[2501.14506]] WanJuanSiLu: A High-Quality Open-Source Webtext Dataset for Low-Resource Languages(https://arxiv.org/abs/2501.14506)
• Keywords: security, extraction
• Abstract: This paper introduces the open-source dataset WanJuanSiLu, designed to provide high-quality training corpora for low-resource languages, thereby advancing the research and development of multilingual models. To achieve this, we have developed a systematic data processing framework tailored for low-resource languages. This framework encompasses key stages such as data extraction, corpus cleaning, content deduplication, security filtering, quality evaluation, and theme classification. Through the implementation of this framework, we have significantly improved both the quality and security of the dataset, while maintaining its linguistic diversity. As of now, data for all five languages have been fully open-sourced. The dataset can be accessed at this https URL, and GitHub repository is available at this https URL

Title: Deep-BrownConrady: Prediction of Camera Calibration and Distortion Parameters Using Deep Learning and Synthetic Data

• Authors: Faiz Muhammad Chaudhry, Jarno Ralli, Jerome Leudet, Fahad Sohrab, Farhad Pakdaman, Pierre Corbani, Moncef Gabbouj
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14510
• Pdf URL: https://arxiv.org/pdf/2501.14510
• Copy Paste: [[2501.14510]] Deep-BrownConrady: Prediction of Camera Calibration and Distortion Parameters Using Deep Learning and Synthetic Data(https://arxiv.org/abs/2501.14510)
• Keywords: robust
• Abstract: This research addresses the challenge of camera calibration and distortion parameter prediction from a single image using deep learning models. The main contributions of this work are: (1) demonstrating that a deep learning model, trained on a mix of real and synthetic images, can accurately predict camera and lens parameters from a single image, and (2) developing a comprehensive synthetic dataset using the AILiveSim simulation platform. This dataset includes variations in focal length and lens distortion parameters, providing a robust foundation for model training and testing. The training process predominantly relied on these synthetic images, complemented by a small subset of real images, to explore how well models trained on synthetic data can perform calibration tasks on real-world images. Traditional calibration methods require multiple images of a calibration object from various orientations, which is often not feasible due to the lack of such images in publicly available datasets. A deep learning network based on the ResNet architecture was trained on this synthetic dataset to predict camera calibration parameters following the Brown-Conrady lens model. The ResNet architecture, adapted for regression tasks, is capable of predicting continuous values essential for accurate camera calibration in applications such as autonomous driving, robotics, and augmented reality. Keywords: Camera calibration, distortion, synthetic data, deep learning, residual networks (ResNet), AILiveSim, horizontal field-of-view, principal point, Brown-Conrady Model.

Title: Real-world Edge Neural Network Implementations Leak Private Interactions Through Physical Side Channel

• Authors: Zhuoran Liu, Senna van Hoek, Péter Horváth, Dirk Lauret, Xiaoyun Xu, Lejla Batina
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.14512
• Pdf URL: https://arxiv.org/pdf/2501.14512
• Copy Paste: [[2501.14512]] Real-world Edge Neural Network Implementations Leak Private Interactions Through Physical Side Channel(https://arxiv.org/abs/2501.14512)
• Keywords: privacy, protect, attack, membership infer, federate, large language model
• Abstract: Neural networks have become a fundamental component of numerous practical applications, and their implementations, which are often accelerated by hardware, are integrated into all types of real-world physical devices. User interactions with neural networks on hardware accelerators are commonly considered privacy-sensitive. Substantial efforts have been made to uncover vulnerabilities and enhance privacy protection at the level of machine learning algorithms, including membership inference attacks, differential privacy, and federated learning. However, neural networks are ultimately implemented and deployed on physical devices, and current research pays comparatively less attention to privacy protection at the implementation level. In this paper, we introduce a generic physical side-channel attack, ScaAR, that extracts user interactions with neural networks by leveraging electromagnetic (EM) emissions of physical devices. Our proposed attack is implementation-agnostic, meaning it does not require the adversary to possess detailed knowledge of the hardware or software implementations, thanks to the capabilities of deep learning-based side-channel analysis (DLSCA). Experimental results demonstrate that, through the EM side channel, ScaAR can effectively extract the class label of user interactions with neural classifiers, including inputs and outputs, on the AMD-Xilinx MPSoC ZCU104 FPGA and Raspberry Pi 3 B. In addition, for the first time, we provide side-channel analysis on edge Large Language Model (LLM) implementations on the Raspberry Pi 5, showing that EM side channel leaks interaction data, and different LLM tokens can be distinguishable from the EM traces.

Title: PARASIDE: An Automatic Paranasal Sinus Segmentation and Structure Analysis Tool for MRI

• Authors: Hendrik Möller, Lukas Krautschick, Matan Atad, Robert Graf, Chia-Jung Busch, Achim Beule, Christian Scharf, Lars Kaderali, Bjoern Menze, Daniel Rueckert, Jan Kirschke, Fabian Schwitzing
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14514
• Pdf URL: https://arxiv.org/pdf/2501.14514
• Copy Paste: [[2501.14514]] PARASIDE: An Automatic Paranasal Sinus Segmentation and Structure Analysis Tool for MRI(https://arxiv.org/abs/2501.14514)
• Keywords: segmentation
• Abstract: Chronic rhinosinusitis (CRS) is a common and persistent sinus imflammation that affects 5 - 12\% of the general population. It significantly impacts quality of life and is often difficult to assess due to its subjective nature in clinical evaluation. We introduce PARASIDE, an automatic tool for segmenting air and soft tissue volumes of the structures of the sinus maxillaris, frontalis, sphenodalis and ethmoidalis in T1 MRI. By utilizing that segmentation, we can quantify feature relations that have been observed only manually and subjectively before. We performed an exemplary study and showed both volume and intensity relations between structures and radiology reports. While the soft tissue segmentation is good, the automated annotations of the air volumes are excellent. The average intensity over air structures are consistently below those of the soft tissues, close to perfect separability. Healthy subjects exhibit lower soft tissue volumes and lower intensities. Our developed system is the first automated whole nasal segmentation of 16 structures, and capable of calculating medical relevant features such as the Lund-Mackay score.

Title: Training-Free Style and Content Transfer by Leveraging U-Net Skip Connections in Stable Diffusion 2.*

• Authors: Ludovica Schaerf, Andrea Alfarano, Fabrizio Silvestri, Leonardo Impett
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14524
• Pdf URL: https://arxiv.org/pdf/2501.14524
• Copy Paste: [[2501.14524]] Training-Free Style and Content Transfer by Leveraging U-Net Skip Connections in Stable Diffusion 2.*(https://arxiv.org/abs/2501.14524)
• Keywords: diffusion
• Abstract: Despite significant recent advances in image generation with diffusion models, their internal latent representations remain poorly understood. Existing works focus on the bottleneck layer (h-space) of Stable Diffusion's U-Net or leverage the cross-attention, self-attention, or decoding layers. Our model, SkipInject takes advantage of U-Net's skip connections. We conduct thorough analyses on the role of the skip connections and find that the residual connections passed by the third encoder block carry most of the spatial information of the reconstructed image, splitting the content from the style. We show that injecting the representations from this block can be used for text-based editing, precise modifications, and style transfer. We compare our methods state-of-the-art style transfer and image editing methods and demonstrate that our method obtains the best content alignment and optimal structural preservation tradeoff.

Title: Idiom Detection in Sorani Kurdish Texts

• Authors: Skala Kamaran Omer, Hossein Hassani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14528
• Pdf URL: https://arxiv.org/pdf/2501.14528
• Copy Paste: [[2501.14528]] Idiom Detection in Sorani Kurdish Texts(https://arxiv.org/abs/2501.14528)
• Keywords: transformer
• Abstract: Idiom detection using Natural Language Processing (NLP) is the computerized process of recognizing figurative expressions within a text that convey meanings beyond the literal interpretation of the words. While idiom detection has seen significant progress across various languages, the Kurdish language faces a considerable research gap in this area despite the importance of idioms in tasks like machine translation and sentiment analysis. This study addresses idiom detection in Sorani Kurdish by approaching it as a text classification task using deep learning techniques. To tackle this, we developed a dataset containing 10,580 sentences embedding 101 Sorani Kurdish idioms across diverse contexts. Using this dataset, we developed and evaluated three deep learning models: KuBERT-based transformer sequence classification, a Recurrent Convolutional Neural Network (RCNN), and a BiLSTM model with an attention mechanism. The evaluations revealed that the transformer model, the fine-tuned BERT, consistently outperformed the others, achieving nearly 99% accuracy while the RCNN achieved 96.5% and the BiLSTM 80%. These results highlight the effectiveness of Transformer-based architectures in low-resource languages like Kurdish. This research provides a dataset, three optimized models, and insights into idiom detection, laying a foundation for advancing Kurdish NLP.

Title: On Hardening DNNs against Noisy Computations

• Authors: Xiao Wang, Hendrik Borras, Bernhard Klein, Holger Fröning
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14531
• Pdf URL: https://arxiv.org/pdf/2501.14531
• Copy Paste: [[2501.14531]] On Hardening DNNs against Noisy Computations(https://arxiv.org/abs/2501.14531)
• Keywords: robust
• Abstract: The success of deep learning has sparked significant interest in designing computer hardware optimized for the high computational demands of neural network inference. As further miniaturization of digital CMOS processors becomes increasingly challenging, alternative computing paradigms, such as analog computing, are gaining consideration. Particularly for compute-intensive tasks such as matrix multiplication, analog computing presents a promising alternative due to its potential for significantly higher energy efficiency compared to conventional digital technology. However, analog computations are inherently noisy, which makes it challenging to maintain high accuracy on deep neural networks. This work investigates the effectiveness of training neural networks with quantization to increase the robustness against noise. Experimental results across various network architectures show that quantization-aware training with constant scaling factors enhances robustness. We compare these methods with noisy training, which incorporates a noise injection during training that mimics the noise encountered during inference. While both two methods increase tolerance against noise, noisy training emerges as the superior approach for achieving robust neural network performance, especially in complex neural architectures.

Title: Rethinking Encoder-Decoder Flow Through Shared Structures

• Authors: Frederik Laboyrie, Mehmet Kerim Yucel, Albert Saa-Garriga
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14535
• Pdf URL: https://arxiv.org/pdf/2501.14535
• Copy Paste: [[2501.14535]] Rethinking Encoder-Decoder Flow Through Shared Structures(https://arxiv.org/abs/2501.14535)
• Keywords: transformer
• Abstract: Dense prediction tasks have enjoyed a growing complexity of encoder architectures, decoders, however, have remained largely the same. They rely on individual blocks decoding intermediate feature maps sequentially. We introduce banks, shared structures that are used by each decoding block to provide additional context in the decoding process. These structures, through applying them via resampling and feature fusion, improve performance on depth estimation for state-of-the-art transformer-based architectures on natural and synthetic images whilst training on large-scale datasets.

Title: Distributed Conformal Prediction via Message Passing

• Authors: Haifeng Wen, Hong Xing, Osvaldo Simeone
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.14544
• Pdf URL: https://arxiv.org/pdf/2501.14544
• Copy Paste: [[2501.14544]] Distributed Conformal Prediction via Message Passing(https://arxiv.org/abs/2501.14544)
• Keywords: robust
• Abstract: Post-hoc calibration of pre-trained models is critical for ensuring reliable inference, especially in safety-critical domains such as healthcare. Conformal Prediction (CP) offers a robust post-hoc calibration framework, providing distribution-free statistical coverage guarantees for prediction sets by leveraging held-out datasets. In this work, we address a decentralized setting where each device has limited calibration data and can communicate only with its neighbors over an arbitrary graph topology. We propose two message-passing-based approaches for achieving reliable inference via CP: quantile-based distributed conformal prediction (Q-DCP) and histogram-based distributed conformal prediction (H-DCP). Q-DCP employs distributed quantile regression enhanced with tailored smoothing and regularization terms to accelerate convergence, while H-DCP uses a consensus-based histogram estimation approach. Through extensive experiments, we investigate the trade-offs between hyperparameter tuning requirements, communication overhead, coverage guarantees, and prediction set sizes across different network topologies.

Title: Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research

• Authors: Hamid Sarmadi, Ola Hall, Thorsteinn Rögnvaldsson, Mattias Ohlsson
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14546
• Pdf URL: https://arxiv.org/pdf/2501.14546
• Copy Paste: [[2501.14546]] Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research(https://arxiv.org/abs/2501.14546)
• Keywords: large language model
• Abstract: This paper investigates the novel application of Large Language Models (LLMs) with vision capabilities to analyze satellite imagery for village-level poverty prediction. Although LLMs were originally designed for natural language understanding, their adaptability to multimodal tasks, including geospatial analysis, has opened new frontiers in data-driven research. By leveraging advancements in vision-enabled LLMs, we assess their ability to provide interpretable, scalable, and reliable insights into human poverty from satellite images. Using a pairwise comparison approach, we demonstrate that ChatGPT can rank satellite images based on poverty levels with accuracy comparable to domain experts. These findings highlight both the promise and the limitations of LLMs in socioeconomic research, providing a foundation for their integration into poverty assessment workflows. This study contributes to the ongoing exploration of unconventional data sources for welfare analysis and opens pathways for cost-effective, large-scale poverty monitoring.

Title: Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation

• Authors: Estanislao Claucich, Sara Hooker, Diego H. Milone, Enzo Ferrante, Rodrigo Echeveste
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14551
• Pdf URL: https://arxiv.org/pdf/2501.14551
• Copy Paste: [[2501.14551]] Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation(https://arxiv.org/abs/2501.14551)
• Keywords: protect, robust, fair
• Abstract: Ensembling is commonly regarded as an effective way to improve the general performance of models in machine learning, while also increasing the robustness of predictions. When it comes to algorithmic fairness, heterogeneous ensembles, composed of multiple model types, have been employed to mitigate biases in terms of demographic attributes such as sex, age or ethnicity. Moreover, recent work has shown how in multi-class problems even simple homogeneous ensembles may favor performance of the worst-performing target classes. While homogeneous ensembles are simpler to implement in practice, it is not yet clear whether their benefits translate to groups defined not in terms of their target class, but in terms of demographic or protected attributes, hence improving fairness. In this work we show how this simple and straightforward method is indeed able to mitigate disparities, particularly benefiting under-performing subgroups. Interestingly, this can be achieved without sacrificing overall performance, which is a common trade-off observed in bias mitigation strategies. Moreover, we analyzed the interplay between two factors which may result in biases: sub-group under-representation and the inherent difficulty of the task for each group. These results revealed that, contrary to popular assumptions, having balanced datasets may be suboptimal if the task difficulty varies between subgroups. Indeed, we found that a perfectly balanced dataset may hurt both the overall performance and the gap between groups. This highlights the importance of considering the interaction between multiple forces at play in fairness.

Title: Exploring Answer Set Programming for Provenance Graph-Based Cyber Threat Detection: A Novel Approach

• Authors: Fang Li, Fei Zuo, Gopal Gupta
• Subjects: cs.CR, cs.PL
• Abstract URL: https://arxiv.org/abs/2501.14555
• Pdf URL: https://arxiv.org/pdf/2501.14555
• Copy Paste: [[2501.14555]] Exploring Answer Set Programming for Provenance Graph-Based Cyber Threat Detection: A Novel Approach(https://arxiv.org/abs/2501.14555)
• Keywords: security, attack
• Abstract: Provenance graphs are useful and powerful tools for representing system-level activities in cybersecurity; however, existing approaches often struggle with complex queries and flexible reasoning. This paper presents a novel approach using Answer Set Programming (ASP) to model and analyze provenance graphs. We introduce an ASP-based representation that captures intricate relationships between system entities, including temporal and causal dependencies. Our model enables sophisticated analysis capabilities such as attack path tracing, data exfiltration detection, and anomaly identification. The declarative nature of ASP allows for concise expression of complex security patterns and policies, facilitating both real-time threat detection and forensic analysis. We demonstrate our approach's effectiveness through case studies showcasing its threat detection capabilities. Experimental results illustrate the model's ability to handle large-scale provenance graphs while providing expressive querying. The model's extensibility allows for incorporation of new system behaviors and security rules, adapting to evolving cyber threats. This work contributes a powerful, flexible, and explainable framework for reasoning about system behaviors and security incidents, advancing the development of effective threat detection and forensic investigation tools.

Title: A sandbox study proposal for private and distributed health data analysis

• Authors: Rickard Brännvall, Hanna Svensson, Kannaki Kaliyaperumal, Håkan Burden, Susanne Stenberg
• Subjects: cs.CR, cs.CY, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.14556
• Pdf URL: https://arxiv.org/pdf/2501.14556
• Copy Paste: [[2501.14556]] A sandbox study proposal for private and distributed health data analysis(https://arxiv.org/abs/2501.14556)
• Keywords: secure, privacy, federate
• Abstract: This paper presents a sandbox study proposal focused on the distributed processing of personal health data within the Vinnova-funded SARDIN project. The project aims to develop the Health Data Bank (Hälsodatabanken in Swedish), a secure platform for research and innovation that complies with the European Health Data Space (EHDS) legislation. By minimizing the sharing and storage of personal data, the platform sends analysis tasks directly to the original data locations, avoiding centralization. This approach raises questions about data controller responsibilities in distributed environments and the anonymization status of aggregated statistical results. The study explores federated analysis, secure multi-party aggregation, and differential privacy techniques, informed by real-world examples from clinical research on Parkinson's disease, stroke rehabilitation, and wound analysis. To validate the proposed study, numerical experiments were conducted using four open-source datasets to assess the feasibility and effectiveness of the proposed methods. The results support the methods for the proposed sandbox study by demonstrating that differential privacy in combination with secure aggregation techniques significantly improves the privacy-utility trade-off.

Title: ZETA: Leveraging Z-order Curves for Efficient Top-k Attention

• Authors: Qiuhao Zeng, Jerry Huang, Peng Lu, Gezheng Xu, Boxing Chen, Charles Ling, Boyu Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14577
• Pdf URL: https://arxiv.org/pdf/2501.14577
• Copy Paste: [[2501.14577]] ZETA: Leveraging Z-order Curves for Efficient Top-k Attention(https://arxiv.org/abs/2501.14577)
• Keywords: transformer
• Abstract: Over recent years, the Transformer has become a fundamental building block for sequence modeling architectures. Yet at its core is the use of self-attention, whose memory and computational cost grow quadratically with the sequence length $N$, rendering it prohibitively expensive for long sequences. A promising approach is top-$k$ attention, which selects only the $k$ most relevant tokens and achieves performance comparable to vanilla self-attention while significantly reducing space and computational demands. However, causal masks require the current query token to only attend to past tokens, preventing the existing top-$k$ attention method from efficiently searching for the most relevant tokens in parallel, thereby limiting training efficiency. In this work, we propose ZETA, leveraging \textbf{Z}-Order Curves for \textbf{E}fficient \textbf{T}op-$k$ \textbf{A}ttention, to enable parallel querying of past tokens for entire sequences. % in both space and time complexity of $\mathcal{O}(N \log N)$. We first theoretically show that the choice of key and query dimensions involves a trade-off between the curse of dimensionality and the preservation of relative distances after projection. In light of this insight, we propose reducing the dimensionality of keys and queries in contrast to values and further leverage $Z$-order curves to map low-dimensional keys and queries into \emph{one}-dimensional space, which permits parallel sorting, thereby largely improving the efficiency for top-$k$ token selection. Experimental results demonstrate that ZETA matches the performance of standard attention on the synthetic \textsc{Multi-Query Associative Recall} task and outperforms attention and its variants on \textsc{Long Range Arena} and \textsc{WikiText-103} language modeling.

Title: Visual Localization via Semantic Structures in Autonomous Photovoltaic Power Plant Inspection

• Authors: Viktor Kozák, Karel Košnar, Jan Chudoba, Miroslav Kulich, Libor Přeučil
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.14587
• Pdf URL: https://arxiv.org/pdf/2501.14587
• Copy Paste: [[2501.14587]] Visual Localization via Semantic Structures in Autonomous Photovoltaic Power Plant Inspection(https://arxiv.org/abs/2501.14587)
• Keywords: robust, segmentation
• Abstract: Inspection systems utilizing unmanned aerial vehicles (UAVs) equipped with thermal cameras are increasingly popular for the maintenance of photovoltaic (PV) power plants. However, automation of the inspection task is a challenging problem as it requires precise navigation to capture images from optimal distances and viewing angles. This paper presents a novel localization pipeline that directly integrates PV module detection with UAV navigation, allowing precise positioning during inspection. Detections are used to identify the power plant structures in the image and associate these with the power plant model. We define visually recognizable anchor points for the initial association and use object tracking to discern global associations. We present three distinct methods for visual segmentation of PV modules based on traditional computer vision, deep learning, and their fusion, and we evaluate their performance in relation to the proposed localization pipeline. The presented methods were verified and evaluated using custom aerial inspection data sets, demonstrating their robustness and applicability for real-time navigation. Additionally, we evaluate the influence of the power plant model's precision on the localization methods.

Title: Data Assetization via Resources-decoupled Federated Learning

• Authors: Jianzhe Zhao, Feida Zhu, Lingyan He, Zixin Tang, Mingce Gao, Shiyu Yang, Guibing Guo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14588
• Pdf URL: https://arxiv.org/pdf/2501.14588
• Copy Paste: [[2501.14588]] Data Assetization via Resources-decoupled Federated Learning(https://arxiv.org/abs/2501.14588)
• Keywords: privacy, federate
• Abstract: With the development of the digital economy, data is increasingly recognized as an essential resource for both work and life. However, due to privacy concerns, data owners tend to maximize the value of data through information flow rather than direct data transfer. Federated learning (FL) provides an effective approach to collaborative training models while preserving privacy. However, different data owners not only have variations in the quantity and quality of their data resources but also face mismatches between data and computing resources as model parameters and training data grow. These challenges hinder data owners' willingness to participate and reduce the effectiveness of data assetization. In this work, we first identify the resource-decoupled FL environment, which includes model owners, data owners, and computing centers. We design a Tripartite Stackelberg Model and theoretically analyze the Stackelberg-Nash Equilibrium (SNE) for participants to optimize global utility. We propose the Quality-aware Dynamic Resources-decoupled FL algorithm (QD-RDFL), in which we derive and solve the optimal strategies of all parties to achieve SHE using backward induction, and a dynamic optimization mechanism is designed to improve the optimal strategy profile by evaluating the contribution of data quality from data owners to the global model during real training. Our comprehensive experiments demonstrate that our method effectively encourages the linkage of the three parties involved, maximizing global utility and data asset value.

Title: Inverse Evolution Data Augmentation for Neural PDE Solvers

• Authors: Chaoyu Liu, Chris Budd, Carola-Bibiane Schönlieb
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14604
• Pdf URL: https://arxiv.org/pdf/2501.14604
• Copy Paste: [[2501.14604]] Inverse Evolution Data Augmentation for Neural PDE Solvers(https://arxiv.org/abs/2501.14604)
• Keywords: robust
• Abstract: Neural networks have emerged as promising tools for solving partial differential equations (PDEs), particularly through the application of neural operators. Training neural operators typically requires a large amount of training data to ensure accuracy and generalization. In this paper, we propose a novel data augmentation method specifically designed for training neural operators on evolution equations. Our approach utilizes insights from inverse processes of these equations to efficiently generate data from random initialization that are combined with original data. To further enhance the accuracy of the augmented data, we introduce high-order inverse evolution schemes. These schemes consist of only a few explicit computation steps, yet the resulting data pairs can be proven to satisfy the corresponding implicit numerical schemes. In contrast to traditional PDE solvers that require small time steps or implicit schemes to guarantee accuracy, our data augmentation method employs explicit schemes with relatively large time steps, thereby significantly reducing computational costs. Accuracy and efficacy experiments confirm the effectiveness of our approach. Additionally, we validate our approach through experiments with the Fourier Neural Operator and UNet on three common evolution equations that are Burgers' equation, the Allen-Cahn equation and the Navier-Stokes equation. The results demonstrate a significant improvement in the performance and robustness of the Fourier Neural Operator when coupled with our inverse evolution data augmentation method.

Title: 3DLabelProp: Geometric-Driven Domain Generalization for LiDAR Semantic Segmentation in Autonomous Driving

• Authors: Jules Sanchez, Jean-Emmanuel Deschaud, François Goulette
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14605
• Pdf URL: https://arxiv.org/pdf/2501.14605
• Copy Paste: [[2501.14605]] 3DLabelProp: Geometric-Driven Domain Generalization for LiDAR Semantic Segmentation in Autonomous Driving(https://arxiv.org/abs/2501.14605)
• Keywords: robust, segmentation
• Abstract: Domain generalization aims to find ways for deep learning models to maintain their performance despite significant domain shifts between training and inference datasets. This is particularly important for models that need to be robust or are costly to train. LiDAR perception in autonomous driving is impacted by both of these concerns, leading to the emergence of various approaches. This work addresses the challenge by proposing a geometry-based approach, leveraging the sequential structure of LiDAR sensors, which sets it apart from the learning-based methods commonly found in the literature. The proposed method, called 3DLabelProp, is applied on the task of LiDAR Semantic Segmentation (LSS). Through extensive experimentation on seven datasets, it is demonstrated to be a state-of-the-art approach, outperforming both naive and other domain generalization methods.

Title: ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations

• Authors: Tianming Liang, Kun-Yu Lin, Chaolei Tan, Jianguo Zhang, Wei-Shi Zheng, Jian-Fang Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14607
• Pdf URL: https://arxiv.org/pdf/2501.14607
• Copy Paste: [[2501.14607]] ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations(https://arxiv.org/abs/2501.14607)
• Keywords: segmentation
• Abstract: Referring video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description. Despite notable progress in recent years, current RVOS models remain struggle to handle complicated object descriptions due to their limited video-language understanding. To address this limitation, we present \textbf{ReferDINO}, an end-to-end RVOS model that inherits strong vision-language understanding from the pretrained visual grounding foundation models, and is further endowed with effective temporal understanding and object segmentation capabilities. In ReferDINO, we contribute three technical innovations for effectively adapting the foundation models to RVOS: 1) an object-consistent temporal enhancer that capitalizes on the pretrained object-text representations to enhance temporal understanding and object consistency; 2) a grounding-guided deformable mask decoder that integrates text and grounding conditions to generate accurate object masks; 3) a confidence-aware query pruning strategy that significantly improves the object decoding efficiency without compromising performance. We conduct extensive experiments on five public RVOS benchmarks to demonstrate that our proposed ReferDINO outperforms state-of-the-art methods significantly. Project page: \url{this https URL}

Title: ACT-JEPA: Joint-Embedding Predictive Architecture Improves Policy Representation Learning

• Authors: Aleksandar Vujinovic, Aleksandar Kovacevic
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14622
• Pdf URL: https://arxiv.org/pdf/2501.14622
• Copy Paste: [[2501.14622]] ACT-JEPA: Joint-Embedding Predictive Architecture Improves Policy Representation Learning(https://arxiv.org/abs/2501.14622)
• Keywords: robust
• Abstract: Learning efficient representations for decision-making policies is a challenge in imitation learning (IL). Current IL methods require expert demonstrations, which are expensive to collect. Consequently, they often have underdeveloped world models. Self-supervised learning (SSL) offers an alternative by allowing models to learn from diverse, unlabeled data, including failures. However, SSL methods often operate in raw input space, making them inefficient. In this work, we propose ACT-JEPA, a novel architecture that integrates IL and SSL to enhance policy representations. We train a policy to predict (1) action sequences and (2) abstract observation sequences. The first objective uses action chunking to improve action prediction and reduce compounding errors. The second objective extends this idea of chunking by predicting abstract observation sequences. We utilize Joint-Embedding Predictive Architecture to predict in abstract representation space, allowing the model to filter out irrelevant details, improve efficiency, and develop a robust world model. Our experiments show that ACT-JEPA improves the quality of representations by learning temporal environment dynamics. Additionally, the model's ability to predict abstract observation sequences results in representations that effectively generalize to action sequence prediction. ACT-JEPA performs on par with established baselines across a range of decision-making tasks.

Title: Towards Scalable Topological Regularizers

• Authors: Hiu-Tung Wong, Darrick Lee, Hong Yan
• Subjects: cs.LG, math.AT
• Abstract URL: https://arxiv.org/abs/2501.14641
• Pdf URL: https://arxiv.org/pdf/2501.14641
• Copy Paste: [[2501.14641]] Towards Scalable Topological Regularizers(https://arxiv.org/abs/2501.14641)
• Keywords: defense, attack, generative
• Abstract: Latent space matching, which consists of matching distributions of features in latent space, is a crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling. Metrics for probability measures, such as Wasserstein and maximum mean discrepancy, are commonly used to quantify the differences between such distributions. However, these are often costly to compute, or do not appropriately take the geometric and topological features of the distributions into consideration. Persistent homology is a tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks. However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. We propose the use of principal persistence measures, based on computing the persistent homology of a large number of small subsamples, as a topological regularizer. We provide a parallelized GPU implementation of this regularizer, and prove that gradients are continuous for smooth densities. Furthermore, we demonstrate the efficacy of this regularizer on shape matching, image generation, and semi-supervised learning tasks, opening the door towards a scalable regularizer for topological features.

Title: Whisper D-SGD: Correlated Noise Across Agents for Differentially Private Decentralized Learning

• Authors: Angelo Rodio, Zheng Chen, Erik G. Larsson
• Subjects: cs.LG, cs.AI, cs.CR, cs.DC
• Abstract URL: https://arxiv.org/abs/2501.14644
• Pdf URL: https://arxiv.org/pdf/2501.14644
• Copy Paste: [[2501.14644]] Whisper D-SGD: Correlated Noise Across Agents for Differentially Private Decentralized Learning(https://arxiv.org/abs/2501.14644)
• Keywords: privacy
• Abstract: Decentralized learning enables distributed agents to train a shared machine learning model through local computation and peer-to-peer communication. Although each agent retains its dataset locally, the communication of local models can still expose private information to adversaries. To mitigate these threats, local differential privacy (LDP) injects independent noise per agent, but it suffers a larger utility gap than central differential privacy (CDP). We introduce Whisper D-SGD, a novel covariance-based approach that generates correlated privacy noise across agents, unifying several state-of-the-art methods as special cases. By leveraging network topology and mixing weights, Whisper D-SGD optimizes the noise covariance to achieve network-wide noise cancellation. Experimental results show that Whisper D-SGD cancels more noise than existing pairwise-correlation schemes, substantially narrowing the CDP-LDP gap and improving model performance under the same privacy guarantees.

Title: Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion

• Authors: Ziyao Xu, Houfeng Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14649
• Pdf URL: https://arxiv.org/pdf/2501.14649
• Copy Paste: [[2501.14649]] Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion(https://arxiv.org/abs/2501.14649)
• Keywords: robust, large language model
• Abstract: To achieve generalized and robust natural-to-formal language conversion (N2F), large language models (LLMs) need to have strong capabilities of decomposition and composition in N2F when faced with an unfamiliar formal language and be able to cope with compositional gaps and counter-intuitive symbolic names. To investigate whether LLMs have this set of basic capabilities in N2F, we propose the DEDC framework. This framework semi-automatically performs sample and task construction, allowing decoupled evaluation of the set of decomposition and composition capabilities of LLMs in N2F. Based on this framework, we evaluate and analyze the most advanced LLMs, and the main findings include that: (1) the LLMs are deficient in both decomposition and composition; (2) the LLMs show a wide coverage of error types that can be attributed to deficiencies in natural language understanding and the learning and use of symbolic systems; (3) compositional gaps and counter-intuitive symbolic names both affect the decomposition and composition of the LLMs. Our work provides a new perspective for investigating the basic capabilities of decomposition and composition of LLMs in N2F. The detailed analysis of deficiencies and attributions can help subsequent improvements of LLMs.

Title: Decoupled SGDA for Games with Intermittent Strategy Communication

• Authors: Ali Zindari, Parham Yazdkhasti, Anton Rodomanov, Tatjana Chavdarova, Sebastian U. Stich
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14652
• Pdf URL: https://arxiv.org/pdf/2501.14652
• Copy Paste: [[2501.14652]] Decoupled SGDA for Games with Intermittent Strategy Communication(https://arxiv.org/abs/2501.14652)
• Keywords: federate
• Abstract: We focus on reducing communication overhead in multiplayer games, where frequently exchanging strategies between players is not feasible and players have noisy or outdated strategies of the other players. We introduce Decoupled SGDA, a novel adaptation of Stochastic Gradient Descent Ascent (SGDA). In this approach, players independently update their strategies based on outdated opponent strategies, with periodic synchronization to align strategies. For Strongly-Convex-Strongly-Concave (SCSC) games, we demonstrate that Decoupled SGDA achieves near-optimal communication complexity comparable to the best-known GDA rates. For weakly coupled games where the interaction between players is lower relative to the non-interactive part of the game, Decoupled SGDA significantly reduces communication costs compared to standard SGDA. Our findings extend to multi-player games. To provide insights into the effect of communication frequency and convergence, we extensively study the convergence of Decoupled SGDA for quadratic minimax problems. Lastly, in settings where the noise over the players is imbalanced, Decoupled SGDA significantly outperforms federated minimax methods.

Title: Federated Domain Generalization with Data-free On-server Gradient Matching

• Authors: Trong-Binh Nguyen, Minh-Duong Nguyen, Jinsun Park, Quoc-Viet Pham, Won Joo Hwang
• Subjects: cs.LG, cs.AI, cs.DC, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.14653
• Pdf URL: https://arxiv.org/pdf/2501.14653
• Copy Paste: [[2501.14653]] Federated Domain Generalization with Data-free On-server Gradient Matching(https://arxiv.org/abs/2501.14653)
• Keywords: robust, federate, data-free
• Abstract: Domain Generalization (DG) aims to learn from multiple known source domains a model that can generalize well to unknown target domains. One of the key approaches in DG is training an encoder which generates domain-invariant representations. However, this approach is not applicable in Federated Domain Generalization (FDG), where data from various domains are distributed across different clients. In this paper, we introduce a novel approach, dubbed Federated Learning via On-server Matching Gradient (FedOMG), which can \emph{efficiently leverage domain information from distributed domains}. Specifically, we utilize the local gradients as information about the distributed models to find an invariant gradient direction across all domains through gradient inner product maximization. The advantages are two-fold: 1) FedOMG can aggregate the characteristics of distributed models on the centralized server without incurring any additional communication cost, and 2) FedOMG is orthogonal to many existing FL/FDG methods, allowing for additional performance improvements by being seamlessly integrated with them. Extensive experimental evaluations on various settings to demonstrate the robustness of FedOMG compared to other FL/FDG baselines. Our method outperforms recent SOTA baselines on four FL benchmark datasets (MNIST, EMNIST, CIFAR-10, and CIFAR-100), and three FDG benchmark datasets (PACS, VLCS, and OfficeHome).

Title: MedAgentBench: Dataset for Benchmarking LLMs as Agents in Medical Applications

• Authors: Yixing Jiang, Kameron C. Black, Gloria Geng, Danny Park, Andrew Y. Ng, Jonathan H. Chen
• Subjects: cs.LG, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2501.14654
• Pdf URL: https://arxiv.org/pdf/2501.14654
• Copy Paste: [[2501.14654]] MedAgentBench: Dataset for Benchmarking LLMs as Agents in Medical Applications(https://arxiv.org/abs/2501.14654)
• Keywords: large language model
• Abstract: Recent large language models (LLMs) have demonstrated significant advancements, particularly in their ability to serve as agents thereby surpassing their traditional role as chatbots. These agents can leverage their planning and tool utilization capabilities to address tasks specified at a high level. However, a standardized dataset to benchmark the agent capabilities of LLMs in medical applications is currently lacking, making the evaluation of LLMs on complex tasks in interactive healthcare environments challenging. To address this gap, we introduce MedAgentBench, a broad evaluation suite designed to assess the agent capabilities of large language models within medical records contexts. MedAgentBench encompasses 100 patient-specific clinically-derived tasks from 10 categories written by human physicians, realistic profiles of 100 patients with over 700,000 data elements, a FHIR-compliant interactive environment, and an accompanying codebase. The environment uses the standard APIs and communication infrastructure used in modern EMR systems, so it can be easily migrated into live EMR systems. MedAgentBench presents an unsaturated agent-oriented benchmark that current state-of-the-art LLMs exhibit some ability to succeed at. The best model (GPT-4o) achieves a success rate of 72%. However, there is still substantial space for improvement to give the community a next direction to optimize. Furthermore, there is significant variation in performance across task categories. MedAgentBench establishes this and is publicly available at this https URL , offering a valuable framework for model developers to track progress and drive continuous improvements in the agent capabilities of large language models within the medical domain.

Title: Neural-Symbolic Message Passing with Dynamic Pruning

• Authors: Chongzhi Zhang, Junhao Zheng, Zhiping Peng, Qianli Ma
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14661
• Pdf URL: https://arxiv.org/pdf/2501.14661
• Copy Paste: [[2501.14661]] Neural-Symbolic Message Passing with Dynamic Pruning(https://arxiv.org/abs/2501.14661)
• Keywords: interpretability
• Abstract: Complex Query Answering (CQA) over incomplete Knowledge Graphs (KGs) is a challenging task. Recently, a line of message-passing-based research has been proposed to solve CQA. However, they perform unsatisfactorily on negative queries and fail to address the noisy messages between variable nodes in the query graph. Moreover, they offer little interpretability and require complex query data and resource-intensive training. In this paper, we propose a Neural-Symbolic Message Passing (NSMP) framework based on pre-trained neural link predictors. By introducing symbolic reasoning and fuzzy logic, NSMP can generalize to arbitrary existential first order logic queries without requiring training while providing interpretable answers. Furthermore, we introduce a dynamic pruning strategy to filter out noisy messages between variable nodes. Experimental results show that NSMP achieves a strong performance. Additionally, through complexity analysis and empirical verification, we demonstrate the superiority of NSMP in inference time over the current state-of-the-art neural-symbolic method. Compared to this approach, NSMP demonstrates faster inference times across all query types on benchmark datasets, with speedup ranging from 2$\times$ to over 150$\times$.

Title: MatAnyone: Stable Video Matting with Consistent Memory Propagation

• Authors: Peiqing Yang, Shangchen Zhou, Jixin Zhao, Qingyi Tao, Chen Change Loy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14677
• Pdf URL: https://arxiv.org/pdf/2501.14677
• Copy Paste: [[2501.14677]] MatAnyone: Stable Video Matting with Consistent Memory Propagation(https://arxiv.org/abs/2501.14677)
• Keywords: robust, segmentation
• Abstract: Auxiliary-free human video matting methods, which rely solely on input frames, often struggle with complex or ambiguous backgrounds. To address this, we propose MatAnyone, a robust framework tailored for target-assigned video matting. Specifically, building on a memory-based paradigm, we introduce a consistent memory propagation module via region-adaptive memory fusion, which adaptively integrates memory from the previous frame. This ensures semantic stability in core regions while preserving fine-grained details along object boundaries. For robust training, we present a larger, high-quality, and diverse dataset for video matting. Additionally, we incorporate a novel training strategy that efficiently leverages large-scale segmentation data, boosting matting stability. With this new network design, dataset, and training strategy, MatAnyone delivers robust and accurate video matting results in diverse real-world scenarios, outperforming existing methods.

Title: Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation

• Authors: Rongzhao He, Weihao Zheng
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14679
• Pdf URL: https://arxiv.org/pdf/2501.14679
• Copy Paste: [[2501.14679]] Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation(https://arxiv.org/abs/2501.14679)
• Keywords: transformer
• Abstract: Attention-based methods have demonstrated exceptional performance in modelling long-range dependencies on spherical cortical surfaces, surpassing traditional Geometric Deep Learning (GDL) models. However, their extensive inference time and high memory demands pose challenges for application to large datasets with limited computing resources. Inspired by the state space model in computer vision, we introduce the attention-free Vision Mamba (Vim) to spherical surfaces, presenting a domain-agnostic architecture for analyzing data on spherical manifolds. Our method achieves surface patching by representing spherical data as a sequence of triangular patches derived from a subdivided icosphere. The proposed Surface Vision Mamba (SiM) is evaluated on multiple neurodevelopmental phenotype regression tasks using cortical surface metrics from neonatal brains. Experimental results demonstrate that SiM outperforms both attention- and GDL-based methods, delivering 4.8 times faster inference and achieving 91.7% lower memory consumption compared to the Surface Vision Transformer (SiT) under the Ico-4 grid partitioning. Sensitivity analysis further underscores the potential of SiM to identify subtle cognitive developmental patterns. The code is available at this https URL.

Title: Rethinking Table Instruction Tuning

• Authors: Naihao Deng, Rada Mihalcea
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.14693
• Pdf URL: https://arxiv.org/pdf/2501.14693
• Copy Paste: [[2501.14693]] Rethinking Table Instruction Tuning(https://arxiv.org/abs/2501.14693)
• Keywords: large language model
• Abstract: Recent advances in table understanding have focused on instruction-tuning large language models (LLMs) for table-related tasks. However, existing research has overlooked the impact of hyperparameter choices and lacks a comprehensive evaluation of the out-of-domain table understanding ability and the general capabilities of these table LLMs. In this paper, we evaluate these abilities in existing table LLMs, and reveal significant declines in both out-of-domain table understanding and general capabilities compared to their base models. Through systematic analysis, we show that hyperparameters, such as learning rate, can significantly influence both table-specific and general capabilities. Contrary to the existing table instruction-tuning works, we demonstrate that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities. Based on our findings, we introduce TAMA, a TAble LLM instruction-tuned from LLaMA 3.1 8B Instruct, which achieves performance on par with, or surpassing GPT-3.5 and GPT-4 on table tasks, while maintaining strong out-of-domain generalization and general capabilities. Our findings highlight the potential for reduced data annotation costs and more efficient model development through careful hyperparameter selection.

Title: An Attentive Graph Agent for Topology-Adaptive Cyber Defence

• Authors: Ilya Orson Sandoval, Isaac Symes Thompson, Vasilios Mavroudis, Chris Hicks
• Subjects: cs.LG, cs.AI, cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.14700
• Pdf URL: https://arxiv.org/pdf/2501.14700
• Copy Paste: [[2501.14700]] An Attentive Graph Agent for Topology-Adaptive Cyber Defence(https://arxiv.org/abs/2501.14700)
• Keywords: security, attack, robust, interpretability
• Abstract: As cyber threats grow increasingly sophisticated, reinforcement learning is emerging as a promising technique to create intelligent, self-improving defensive systems. However, most existing autonomous defensive agents have overlooked the inherent graph structure of computer networks subject to cyber attacks, potentially missing critical information. To address this gap, we developed a custom version of the Cyber Operations Research Gym (CybORG) environment that encodes the observable network state as a directed graph, utilizing realistic and interpretable low-level features. %, like number of open ports and unexpected detected connections. We leverage a Graph Attention Network (GAT) architecture to process node, edge, and global features, and modify its output to be compatible with policy gradient methods in reinforcement learning. GAT policies offer several advantages over standard approaches based on simplistic flattened state observations. They can handle the changes in network topology that occur at runtime when dynamic connections between hosts appear. Policies can be deployed to networks that differ in size to the ones seen during training, enabling a degree of generalisation inaccessible with alternative approaches. Furthermore, the graph neural network policies outputs are explainable in terms of tangible network properties, providing enhanced interpretability of defensive actions. We verify that our low-level graph observations are meaningful enough to train GAT defensive policies that are able to adapt to changing topologies. We evaluate how our trained policies perform when deployed on networks of varying sizes with the same subnetwork structure, comparing them against policies specifically trained for each network configuration. Our study contributes to the development of robust cyber defence systems that can better adapt to real-world network security challenges.

Title: NLP-based assessment of prescription appropriateness from Italian referrals

• Authors: Vittorio Torri, Annamaria Bottelli, Michele Ercolanoni, Olivia Leoni, Francesca Ieva
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14701
• Pdf URL: https://arxiv.org/pdf/2501.14701
• Copy Paste: [[2501.14701]] NLP-based assessment of prescription appropriateness from Italian referrals(https://arxiv.org/abs/2501.14701)
• Keywords: transformer
• Abstract: Objective: This study proposes a Natural Language Processing pipeline to evaluate prescription appropriateness in Italian referrals, where reasons for prescriptions are recorded only as free text, complicating automated comparisons with guidelines. The pipeline aims to derive, for the first time, a comprehensive summary of the reasons behind these referrals and a quantification of their appropriateness. While demonstrated in a specific case study, the approach is designed to generalize to other types of examinations. Methods: Leveraging embeddings from a transformer-based model, the proposed approach clusters referral texts, maps clusters to labels, and aligns these labels with existing guidelines. We present a case study on a dataset of 496,971 referrals, consisting of all referrals for venous echocolordopplers of the lower limbs between 2019 and 2021 in the Lombardy Region. A sample of 1,000 referrals was manually annotated to validate the results. Results: The pipeline exhibited high performance for referrals' reasons (Prec=92.43%, Rec=83.28%) and excellent results for referrals' appropriateness (Prec=93.58%, Rec=91.52%) on the annotated subset. Analysis of the entire dataset identified clusters matching guideline-defined reasons - both appropriate and inappropriate - as well as clusters not addressed in the guidelines. Overall, 34.32% of referrals were marked as appropriate, 34.07% inappropriate, 14.37% likely inappropriate, and 17.24% could not be mapped to guidelines. Conclusions: The proposed pipeline effectively assessed prescription appropriateness across a large dataset, serving as a valuable tool for health authorities. Findings have informed the Lombardy Region's efforts to strengthen recommendations and reduce the burden of inappropriate referrals.

Title: The Karp Dataset

• Authors: Mason DiCicco, Eamon Worden, Conner Olsen, Nikhil Gangaram, Daniel Reichman, Neil Heffernan
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14705
• Pdf URL: https://arxiv.org/pdf/2501.14705
• Copy Paste: [[2501.14705]] The Karp Dataset(https://arxiv.org/abs/2501.14705)
• Keywords: large language model
• Abstract: Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training and benchmarking the performance of LLMs. To this end, we introduce the Karp dataset: The first dataset composed of detailed proofs of NP-completeness reductions. The reductions vary in difficulty, ranging from simple exercises of undergraduate courses to more challenging reductions from academic papers. We compare the performance of state-of-the-art models on this task and demonstrate the effect of fine-tuning with the Karp dataset on reasoning capacity.

Title: FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing

• Authors: James Seale Smith, Chi-Heng Lin, Shikhar Tuli, Haris Jeelani, Shangqian Gao, Yilin Shen, Hongxia Jin, Yen-Chang Hsu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.14713
• Pdf URL: https://arxiv.org/pdf/2501.14713
• Copy Paste: [[2501.14713]] FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing(https://arxiv.org/abs/2501.14713)
• Keywords: large language model
• Abstract: The rapid proliferation of large language models (LLMs) in natural language processing (NLP) has created a critical need for techniques that enable efficient deployment on memory-constrained devices without compromising performance. We present a method to prune LLMs that selectively prunes model blocks based on an importance score and replaces them with a low-parameter replacement strategy. Specifically, we propose a principled metric to replace each pruned block using a weight-sharing mechanism that leverages unpruned counterparts from the model and block-specific low-rank adapters. Furthermore, we facilitate the learning of these replacement blocks with output feature normalization and an adapter initialization scheme built on low-rank SVD reconstructions. Empirical evaluations demonstrate substantial performance gains over existing methods, achieving state-of-the-art performance on 5/6 benchmarks for a compression rate of 30% and 6/6 benchmarks for a compression rate of 40%. We also demonstrate that our approach can extend smaller models, boosting performance on 6/6 benchmarks using only ~0.3% tokens of extended training with minimal additional parameter costs.

Title: Towards Better Understanding Table Instruction Tuning: Decoupling the Effects from Data versus Models

• Authors: Naihao Deng, Sheng Zhang, Henghui Zhu, Shuaichen Chang, Jiani Zhang, Alexander Hanbo Li, Chung-Wei Hang, Hideo Kobayashi, Yiqun Hu, Patrick Ng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.14717
• Pdf URL: https://arxiv.org/pdf/2501.14717
• Copy Paste: [[2501.14717]] Towards Better Understanding Table Instruction Tuning: Decoupling the Effects from Data versus Models(https://arxiv.org/abs/2501.14717)
• Keywords: large language model
• Abstract: Recent advances in natural language processing have leveraged instruction tuning to enhance Large Language Models (LLMs) for table-related tasks. However, previous works train different base models with different training data, lacking an apples-to-apples comparison across the result table LLMs. To address this, we fine-tune base models from the Mistral, OLMo, and Phi families on existing public training datasets. Our replication achieves performance on par with or surpassing existing table LLMs, establishing new state-of-the-art performance on Hitab, a table question-answering dataset. More importantly, through systematic out-of-domain evaluation, we decouple the contributions of training data and the base model, providing insight into their individual impacts. In addition, we assess the effects of table-specific instruction tuning on general-purpose benchmarks, revealing trade-offs between specialization and generalization.

Title: Do LLMs Provide Consistent Answers to Health-Related Questions across Languages?

• Authors: Ipek Baris Schlicht, Zhixue Zhao, Burcu Sayin, Lucie Flek, Paolo Rosso
• Subjects: cs.CL, cs.AI, cs.HC, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.14719
• Pdf URL: https://arxiv.org/pdf/2501.14719
• Copy Paste: [[2501.14719]] Do LLMs Provide Consistent Answers to Health-Related Questions across Languages?(https://arxiv.org/abs/2501.14719)
• Keywords: large language model
• Abstract: Equitable access to reliable health information is vital for public health, but the quality of online health resources varies by language, raising concerns about inconsistencies in Large Language Models (LLMs) for healthcare. In this study, we examine the consistency of responses provided by LLMs to health-related questions across English, German, Turkish, and Chinese. We largely expand the HealthFC dataset by categorizing health-related questions by disease type and broadening its multilingual scope with Turkish and Chinese translations. We reveal significant inconsistencies in responses that could spread healthcare misinformation. Our main contributions are 1) a multilingual health-related inquiry dataset with meta-information on disease categories, and 2) a novel prompt-based evaluation workflow that enables sub-dimensional comparisons between two languages through parsing. Our findings highlight key challenges in deploying LLM-based tools in multilingual contexts and emphasize the need for improved cross-lingual alignment to ensure accurate and equitable healthcare information.

Title: HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation

• Authors: Xin Zhou, Dingkang Liang, Sifan Tu, Xiwu Chen, Yikang Ding, Dingyuan Zhang, Feiyang Tan, Hengshuang Zhao, Xiang Bai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.14729
• Pdf URL: https://arxiv.org/pdf/2501.14729
• Copy Paste: [[2501.14729]] HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation(https://arxiv.org/abs/2501.14729)
• Keywords: large language model
• Abstract: Driving World Models (DWMs) have become essential for autonomous driving by enabling future scene prediction. However, existing DWMs are limited to scene generation and fail to incorporate scene understanding, which involves interpreting and reasoning about the driving environment. In this paper, we present a unified Driving World Model named HERMES. We seamlessly integrate 3D scene understanding and future scene evolution (generation) through a unified framework in driving scenarios. Specifically, HERMES leverages a Bird's-Eye View (BEV) representation to consolidate multi-view spatial information while preserving geometric relationships and interactions. We also introduce world queries, which incorporate world knowledge into BEV features via causal attention in the Large Language Model (LLM), enabling contextual enrichment for understanding and generation tasks. We conduct comprehensive studies on nuScenes and OmniDrive-nuScenes datasets to validate the effectiveness of our method. HERMES achieves state-of-the-art performance, reducing generation error by 32.4% and improving understanding metrics such as CIDEr by 8.0%. The model and code will be publicly released at this https URL.