2025-01-20

Title: Enhancing the De-identification of Personally Identifiable Information in Educational Data

• Authors: Y. Shen, Z. Ji, J. Lin, K. R. Koedginer
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09765
• Pdf URL: https://arxiv.org/pdf/2501.09765
• Copy Paste: [[2501.09765]] Enhancing the De-identification of Personally Identifiable Information in Educational Data(https://arxiv.org/abs/2501.09765)
• Keywords: privacy, protect, robust
• Abstract: Protecting Personally Identifiable Information (PII), such as names, is a critical requirement in learning technologies to safeguard student and teacher privacy and maintain trust. Accurate PII detection is an essential step toward anonymizing sensitive information while preserving the utility of educational data. Motivated by recent advancements in artificial intelligence, our study investigates the GPT-4o-mini model as a cost-effective and efficient solution for PII detection tasks. We explore both prompting and fine-tuning approaches and compare GPT-4o-mini's performance against established frameworks, including Microsoft Presidio and Azure AI Language. Our evaluation on two public datasets, CRAPII and TSCC, demonstrates that the fine-tuned GPT-4o-mini model achieves superior performance, with a recall of 0.9589 on CRAPII. Additionally, fine-tuned GPT-4o-mini significantly improves precision scores (a threefold increase) while reducing computational costs to nearly one-tenth of those associated with Azure AI Language. Furthermore, our bias analysis reveals that the fine-tuned GPT-4o-mini model consistently delivers accurate results across diverse cultural backgrounds and genders. The generalizability analysis using the TSCC dataset further highlights its robustness, achieving a recall of 0.9895 with minimal additional training data from TSCC. These results emphasize the potential of fine-tuned GPT-4o-mini as an accurate and cost-effective tool for PII detection in educational data. It offers robust privacy protection while preserving the data's utility for research and pedagogical analysis. Our code is available on GitHub: this https URL

Title: Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning

• Authors: Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Wu Ning, Yutai Hou, Xu Huang, Bing Qin, Ting Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.09766
• Pdf URL: https://arxiv.org/pdf/2501.09766
• Copy Paste: [[2501.09766]] Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning(https://arxiv.org/abs/2501.09766)
• Keywords: large language model
• Abstract: Augmenting large language models (LLMs) with external tools is a promising approach to enhance their capabilities. Effectively leveraging this potential for complex tasks hinges crucially on improving their ability to use tools. Synthesizing tool use data by simulating the real world is an effective approach. Nevertheless, our investigation reveals that training gains significantly decay as the scale of these data increases. The primary factor is the model's poor performance (a.k.a deficiency) in complex scenarios, which hinders learning from data using SFT. Driven by this objective, we propose an iterative reinforced fine-tuning strategy to continually guide the model to alleviate it. Specifically, we first identify deficiency-related data based on feedback from the policy model, then perform a Monte Carlo Tree Search to collect fine-grained preference pairs to pinpoint deficiencies. Subsequently, we update the policy model using preference optimization to align with ground truth and misalign with deficiencies. This process can be iterated. Moreover, before the iteration, we propose an easy-to-hard warm-up SFT strategy to facilitate learning from challenging data. The experiments demonstrate our models go beyond the same parametric models, outperforming many larger open-source and closed-source models. Additionally, it has achieved notable training gains in complex tool use scenarios.

Title: Can Large Language Models Predict the Outcome of Judicial Decisions?

• Authors: Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Amani Al-Ghraibah
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09768
• Pdf URL: https://arxiv.org/pdf/2501.09768
• Copy Paste: [[2501.09768]] Can Large Language Models Predict the Outcome of Judicial Decisions?(https://arxiv.org/abs/2501.09768)
• Keywords: robust, large language model
• Abstract: Large Language Models (LLMs) have shown exceptional capabilities in Natural Language Processing (NLP) across diverse domains. However, their application in specialized tasks such as Legal Judgment Prediction (LJP) for low-resource languages like Arabic remains underexplored. In this work, we address this gap by developing an Arabic LJP dataset, collected and preprocessed from Saudi commercial court judgments. We benchmark state-of-the-art open-source LLMs, including LLaMA-3.2-3B and LLaMA-3.1-8B, under varying configurations such as zero-shot, one-shot, and fine-tuning using QLoRA. Additionally, we used a comprehensive evaluation framework combining quantitative metrics (BLEU and ROUGE) and qualitative assessments (Coherence, legal language, clarity). Our results demonstrate that fine-tuned smaller models achieve comparable performance to larger models in task-specific contexts while offering significant resource efficiency. Furthermore, we investigate the effects of prompt engineering and fine-tuning on model outputs, providing insights into performance variability and instruction sensitivity. By making the dataset, implementation code, and models publicly available, we establish a robust foundation for future research in Arabic legal NLP.

Title: EVAL: EigenVector-based Average-reward Learning

• Authors: Jacob Adamczyk, Volodymyr Makarenko, Stas Tiomkin, Rahul V. Kulkarni
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09770
• Pdf URL: https://arxiv.org/pdf/2501.09770
• Copy Paste: [[2501.09770]] EVAL: EigenVector-based Average-reward Learning(https://arxiv.org/abs/2501.09770)
• Keywords: robust
• Abstract: In reinforcement learning, two objective functions have been developed extensively in the literature: discounted and averaged rewards. The generalization to an entropy-regularized setting has led to improved robustness and exploration for both of these objectives. Recently, the entropy-regularized average-reward problem was addressed using tools from large deviation theory in the tabular setting. This method has the advantage of linearity, providing access to both the optimal policy and average reward-rate through properties of a single matrix. In this paper, we extend that framework to more general settings by developing approaches based on function approximation by neural networks. This formulation reveals new theoretical insights into the relationship between different objectives used in RL. Additionally, we combine our algorithm with a posterior policy iteration scheme, showing how our approach can also solve the average-reward RL problem without entropy-regularization. Using classic control benchmarks, we experimentally find that our method compares favorably with other algorithms in terms of stability and rate of convergence.

Title: Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident Even When They Are Wrong

• Authors: Tairan Fu, Javier Conde, Gonzalo Martínez, María Grandury, Pedro Reviriego
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09775
• Pdf URL: https://arxiv.org/pdf/2501.09775
• Copy Paste: [[2501.09775]] Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident Even When They Are Wrong(https://arxiv.org/abs/2501.09775)
• Keywords: large language model
• Abstract: One of the most widely used methods to evaluate LLMs are Multiple Choice Question (MCQ) tests. MCQ benchmarks enable the testing of LLM knowledge on almost any topic at scale as the results can be processed automatically. To help the LLM answer, a few examples called few shots can be included in the prompt. Moreover, the LLM can be asked to answer the question directly with the selected option or to first provide the reasoning and then the selected answer, which is known as chain of thought. In addition to checking whether the selected answer is correct, the evaluation can look at the LLM-estimated probability of its response as an indication of the confidence of the LLM in the response. In this paper, we study how the LLM confidence in its answer depends on whether the model has been asked to answer directly or to provide the reasoning before answering. The results of the evaluation of questions on a wide range of topics in seven different models show that LLMs are more confident in their answers when they provide reasoning before the answer. This occurs regardless of whether the selected answer is correct. Our hypothesis is that this behavior is due to the reasoning that modifies the probability of the selected answer, as the LLM predicts the answer based on the input question and the reasoning that supports the selection made. Therefore, LLM estimated probabilities seem to have intrinsic limitations that should be understood in order to use them in evaluation procedures. Interestingly, the same behavior has been observed in humans, for whom explaining an answer increases confidence in its correctness.

Title: Sentiment Analysis in Twitter Social Network Centered on Cryptocurrencies Using Machine Learning

• Authors: Vahid Amiri, Mahmood Ahmadi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09777
• Pdf URL: https://arxiv.org/pdf/2501.09777
• Copy Paste: [[2501.09777]] Sentiment Analysis in Twitter Social Network Centered on Cryptocurrencies Using Machine Learning(https://arxiv.org/abs/2501.09777)
• Keywords: secure
• Abstract: Cryptocurrency is a digital currency that uses blockchain technology with secure encryption. Due to the decentralization of these currencies, traditional monetary systems and the capital market of each they, can influence a society. Therefore, due to the importance of the issue, the need to understand public opinion and analyze people's opinions in this regard increases. To understand the opinions and views of people about different topics, you can take help from social networks because they are a rich source of opinions. The Twitter social network is one of the main platforms where users discuss various topics, therefore, in the shortest time and with the lowest cost, the opinion of the community can be measured on this social network. Twitter Sentiment Analysis (TSA) is a field that analyzes the sentiment expressed in tweets. Considering that most of TSA's research efforts on cryptocurrencies are focused on English language, the purpose of this paper is to investigate the opinions of Iranian users on the Twitter social network about cryptocurrencies and provide the best model for classifying tweets based on sentiment. In the case of automatic analysis of tweets, managers and officials in the field of economy can gain knowledge from the general public's point of view about this issue and use the information obtained in order to properly manage this phenomenon. For this purpose, in this paper, in order to build emotion classification models, natural language processing techniques such as bag of words (BOW) and FastText for text vectorization and classical machine learning algorithms including KNN, SVM and Adaboost learning methods Deep including LSTM and BERT model were used for classification, and finally BERT linguistic model had the best accuracy with 83.50%.

Title: VideoWorld: Exploring Knowledge Learning from Unlabeled Videos

• Authors: Zhongwei Ren, Yunchao Wei, Xun Guo, Yao Zhao, Bingyi Kang, Jiashi Feng, Xiaojie Jin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09781
• Pdf URL: https://arxiv.org/pdf/2501.09781
• Copy Paste: [[2501.09781]] VideoWorld: Exploring Knowledge Learning from Unlabeled Videos(https://arxiv.org/abs/2501.09781)
• Keywords: generative, large language model
• Abstract: This work explores whether a deep generative model can learn complex knowledge solely from visual input, in contrast to the prevalent focus on text-based models like large language models (LLMs). We develop VideoWorld, an auto-regressive video generation model trained on unlabeled video data, and test its knowledge acquisition abilities in video-based Go and robotic control tasks. Our experiments reveal two key findings: (1) video-only training provides sufficient information for learning knowledge, including rules, reasoning and planning capabilities, and (2) the representation of visual change is crucial for knowledge acquisition. To improve both the efficiency and efficacy of this process, we introduce the Latent Dynamics Model (LDM) as a key component of VideoWorld. Remarkably, VideoWorld reaches a 5-dan professional level in the Video-GoBench with just a 300-million-parameter model, without relying on search algorithms or reward mechanisms typical in reinforcement learning. In robotic tasks, VideoWorld effectively learns diverse control operations and generalizes across environments, approaching the performance of oracle models in CALVIN and RLBench. This study opens new avenues for knowledge acquisition from visual data, with all code, data, and models open-sourced for further research.

Title: SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation

• Authors: Wanqi Yin, Zhongang Cai, Ruisi Wang, Ailing Zeng, Chen Wei, Qingping Sun, Haiyi Mei, Yanjun Wang, Hui En Pang, Mingyuan Zhang, Lei Zhang, Chen Change Loy, Atsushi Yamashita, Lei Yang, Ziwei Liu
• Subjects: cs.CV, cs.GR, cs.HC, cs.MM, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.09782
• Pdf URL: https://arxiv.org/pdf/2501.09782
• Copy Paste: [[2501.09782]] SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation(https://arxiv.org/abs/2501.09782)
• Keywords: transformer
• Abstract: Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications. Despite encouraging progress, current state-of-the-art methods focus on training innovative architectural designs on confined datasets. In this work, we investigate the impact of scaling up EHPS towards a family of generalist foundation models. 1) For data scaling, we perform a systematic investigation on 40 EHPS datasets, encompassing a wide range of scenarios that a model trained on any single dataset cannot handle. More importantly, capitalizing on insights obtained from the extensive benchmarking process, we optimize our training scheme and select datasets that lead to a significant leap in EHPS capabilities. Ultimately, we achieve diminishing returns at 10M training instances from diverse data sources. 2) For model scaling, we take advantage of vision transformers (up to ViT-Huge as the backbone) to study the scaling law of model sizes in EHPS. To exclude the influence of algorithmic design, we base our experiments on two minimalist architectures: SMPLer-X, which consists of an intermediate step for hand and face localization, and SMPLest-X, an even simpler version that reduces the network to its bare essentials and highlights significant advances in the capture of articulated hands. With big data and the large model, the foundation models exhibit strong performance across diverse test benchmarks and excellent transferability to even unseen environments. Moreover, our finetuning strategy turns the generalist into specialist models, allowing them to achieve further performance boosts. Notably, our foundation models consistently deliver state-of-the-art results on seven benchmarks such as AGORA, UBody, EgoBody, and our proposed SynHand dataset for comprehensive hand evaluation. (Code is available at: this https URL).

Title: Computing Optimization-Based Prompt Injections Against Closed-Weights Models By Misusing a Fine-Tuning API

• Authors: Andrey Labunets, Nishit V. Pandya, Ashish Hooda, Xiaohan Fu, Earlence Fernandes
• Subjects: cs.CR, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09798
• Pdf URL: https://arxiv.org/pdf/2501.09798
• Copy Paste: [[2501.09798]] Computing Optimization-Based Prompt Injections Against Closed-Weights Models By Misusing a Fine-Tuning API(https://arxiv.org/abs/2501.09798)
• Keywords: security, attack, large language model
• Abstract: We surface a new threat to closed-weight Large Language Models (LLMs) that enables an attacker to compute optimization-based prompt injections. Specifically, we characterize how an attacker can leverage the loss-like information returned from the remote fine-tuning interface to guide the search for adversarial prompts. The fine-tuning interface is hosted by an LLM vendor and allows developers to fine-tune LLMs for their tasks, thus providing utility, but also exposes enough information for an attacker to compute adversarial prompts. Through an experimental analysis, we characterize the loss-like values returned by the Gemini fine-tuning API and demonstrate that they provide a useful signal for discrete optimization of adversarial prompts using a greedy search algorithm. Using the PurpleLlama prompt injection benchmark, we demonstrate attack success rates between 65% and 82% on Google's Gemini family of LLMs. These attacks exploit the classic utility-security tradeoff - the fine-tuning interface provides a useful feature for developers but also exposes the LLMs to powerful attacks.

Title: W3ID: A Quantum Computing-Secure Digital Identity System Redefining Standards for Web3 and Digital Twins

• Authors: Joseph Yun (1 and 2), Eli Lifton (1), Eunseo Lee (1), Yohan Yun (1), Abigail Song (1), Joshua Lee (1), Cristian Jimenez-Bert (1), Benedict Song (1), Yejun Lee (1), Alex Seo (1), Sijung Yun (1 and 2) ((1) Predictiv Care, Inc., (2) Johns Hopkins University)
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.09802
• Pdf URL: https://arxiv.org/pdf/2501.09802
• Copy Paste: [[2501.09802]] W3ID: A Quantum Computing-Secure Digital Identity System Redefining Standards for Web3 and Digital Twins(https://arxiv.org/abs/2501.09802)
• Keywords: secure, security, attack
• Abstract: The rapid advancements in quantum computing present significant threats to existing encryption standards and internet security. Simultaneously, the advent of Web 3.0 marks a transformative era in internet history, emphasizing enhanced data security, decentralization, and user ownership. This white paper introduces the W3ID, an abbreviation of Web3 standard meeting universal digital ID, which is a Universal Digital Identity (UDI) model designed to meet Web3 standards while addressing vulnerabilities posed by quantum computing. W3ID innovatively generates secure Digital Object Identifiers (DOIs) tailored for the decentralized Web 3.0 ecosystem. Additionally, W3ID employs a dual-key system for secure authentication, enhancing both public and private verification mechanisms. To further enhance encryption strength and authentication integrity in the quantum computing era, W3ID incorporates an advanced security mechanism. By requiring quadruple application of SHA-256, with consecutive matches for validation, the system expands the number of possibilities to 256^4, which is approximately 4.3 billion times the current SHA-256 capacity. This dramatic increase in computational complexity ensures that even advanced quantum computing systems would face significant challenges in executing brute-force attacks. W3ID redefines digital identity standards for Web 3.0 and the quantum computing era, setting a new benchmark for security, scalability, and decentralization in the global digital twin ecosystem.

Title: Enhancing Generalization in Chain of Thought Reasoning for Smaller Models

• Authors: Maxwell J. Yin, Dingyi Jiang, Yongbing Chen, Boyu Wang, Charles Ling
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09804
• Pdf URL: https://arxiv.org/pdf/2501.09804
• Copy Paste: [[2501.09804]] Enhancing Generalization in Chain of Thought Reasoning for Smaller Models(https://arxiv.org/abs/2501.09804)
• Keywords: robust, explainability
• Abstract: Chain-of-Thought (CoT) reasoning in smaller language models is a challenging natural language process problem yet highly desirable in many real-life applications. Existing CoT knowledge distillation methods often suffer from overly conservative memorization in smaller LLMs, leading to low generalization confidence. As fully preserving the CoT ability of teacher model is impossible, we hypothesize that adversarial CoT fine-tuning is crucial for developing smaller LLM with robust CoT generalization. To this end, we propose \textit{PRompt-Assisted Domain-Adversarial fine-tuning} (PRADA), a principled fine-tuning framework that integrates diverse CoT domains. Specifically, PRADA pioneers two CoT improvements in smaller LLM: (1) Recovering the domain-invariant feature insight which typically lost during distillation with domain adversarial fine-tuning; (2) Enhancing the domain adaptability of CoT prompt engineering by employing domain-adversarial approaches. We theoretically demonstrate the effectiveness of our approach and empirically show that it significantly outperforms the state of the arts in a wide range of tasks. Moreover, our empirical findings reveal that the smaller LLM, when leveraging PRADA, aligns closely with domain knowledge, thereby improving the explainability of our approach.

Title: Ruling the Unruly: Designing Effective, Low-Noise Network Intrusion Detection Rules for Security Operations Centers

• Authors: Koen T. W. Teuwen, Tom Mulders, Emmanuele Zambon, Luca Allodi
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.09808
• Pdf URL: https://arxiv.org/pdf/2501.09808
• Copy Paste: [[2501.09808]] Ruling the Unruly: Designing Effective, Low-Noise Network Intrusion Detection Rules for Security Operations Centers(https://arxiv.org/abs/2501.09808)
• Keywords: security
• Abstract: Many Security Operations Centers (SOCs) today still heavily rely on signature-based Network Intrusion Detection Systems (NIDS) such as Suricata. The specificity of intrusion detection rules and the coverage provided by rulesets are common concerns within the professional community surrounding SOCs, which impact the effectiveness of automated alert post-processing approaches. We postulate a better understanding of factors influencing the quality of rules can help address current SOC issues. In this paper, we characterize the rules in use at a collaborating commercial (managed) SOC serving customers in sectors including education and IT management. During this process, we discover six relevant design principles, which we consolidate through interviews with experienced rule designers at the this http URL then validate our design principles by quantitatively assessing their effect on rule specificity. We find that several of these design considerations significantly impact unnecessary workload caused by rules. For instance, rules that leverage proxies for detection, and rules that do not employ alert throttling or do not distinguish (un)successful malicious actions, cause significantly more workload for SOC analysts. Moreover, rules that match a generalized characteristic to detect malicious behavior, which is believed to increase coverage, also significantly increase workload, suggesting a tradeoff must be struck between rule specificity and coverage. We show that these design principles can be applied successfully at a SOC to reduce workload whilst maintaining coverage despite the prevalence of violations of the principles.

Title: Lossy Compression with Pretrained Diffusion Models

• Authors: Jeremy Vonderfecht, Feng Liu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09815
• Pdf URL: https://arxiv.org/pdf/2501.09815
• Copy Paste: [[2501.09815]] Lossy Compression with Pretrained Diffusion Models(https://arxiv.org/abs/2501.09815)
• Keywords: diffusion, generative
• Abstract: We apply the DiffC algorithm (Theis et al. 2022) to Stable Diffusion 1.5, 2.1, XL, and Flux-dev, and demonstrate that these pretrained models are remarkably capable lossy image compressors. A principled algorithm for lossy compression using pretrained diffusion models has been understood since at least Ho et al. 2020, but challenges in reverse-channel coding have prevented such algorithms from ever being fully implemented. We introduce simple workarounds that lead to the first complete implementation of DiffC, which is capable of compressing and decompressing images using Stable Diffusion in under 10 seconds. Despite requiring no additional training, our method is competitive with other state-of-the-art generative compression methods at low ultra-low bitrates.

Title: Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer

• Authors: Haoyu Zhang, Raghavendra Ramachandra, Kiran Raja, Christoph Busch
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09817
• Pdf URL: https://arxiv.org/pdf/2501.09817
• Copy Paste: [[2501.09817]] Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer(https://arxiv.org/abs/2501.09817)
• Keywords: attack, robust, transformer
• Abstract: Face morphing attacks have posed severe threats to Face Recognition Systems (FRS), which are operated in border control and passport issuance use cases. Correspondingly, morphing attack detection algorithms (MAD) are needed to defend against such attacks. MAD approaches must be robust enough to handle unknown attacks in an open-set scenario where attacks can originate from various morphing generation algorithms, post-processing and the diversity of printers/scanners. The problem of generalization is further pronounced when the detection has to be made on a single suspected image. In this paper, we propose a generalized single-image-based MAD (S-MAD) algorithm by learning the encoding from Vision Transformer (ViT) architecture. Compared to CNN-based architectures, ViT model has the advantage on integrating local and global information and hence can be suitable to detect the morphing traces widely distributed among the face region. Extensive experiments are carried out on face morphing datasets generated using publicly available FRGC face datasets. Several state-of-the-art (SOTA) MAD algorithms, including representative ones that have been publicly evaluated, have been selected and benchmarked with our ViT-based approach. Obtained results demonstrate the improved detection performance of the proposed S-MAD method on inter-dataset testing (when different data is used for training and testing) and comparable performance on intra-dataset testing (when the same data is used for training and testing) experimental protocol.

Title: BN-Pool: a Bayesian Nonparametric Approach to Graph Pooling

• Authors: Daniele Castellana, Filippo Maria Bianchi
• Subjects: cs.LG, math.PR
• Abstract URL: https://arxiv.org/abs/2501.09821
• Pdf URL: https://arxiv.org/pdf/2501.09821
• Copy Paste: [[2501.09821]] BN-Pool: a Bayesian Nonparametric Approach to Graph Pooling(https://arxiv.org/abs/2501.09821)
• Keywords: generative
• Abstract: We introduce BN-Pool, the first clustering-based pooling method for Graph Neural Networks (GNNs) that adaptively determines the number of supernodes in a coarsened graph. By leveraging a Bayesian non-parametric framework, BN-Pool employs a generative model capable of partitioning graph nodes into an unbounded number of clusters. During training, we learn the node-to-cluster assignments by combining the supervised loss of the downstream task with an unsupervised auxiliary term, which encourages the reconstruction of the original graph topology while penalizing unnecessary proliferation of clusters. This adaptive strategy allows BN-Pool to automatically discover an optimal coarsening level, offering enhanced flexibility and removing the need to specify sensitive pooling ratios. We show that BN-Pool achieves superior performance across diverse benchmarks.

Title: pFedWN: A Personalized Federated Learning Framework for D2D Wireless Networks with Heterogeneous Data

• Authors: Zhou Ni, Masoud Ghazikor, Morteza Hashemi
• Subjects: cs.LG, cs.NI
• Abstract URL: https://arxiv.org/abs/2501.09822
• Pdf URL: https://arxiv.org/pdf/2501.09822
• Copy Paste: [[2501.09822]] pFedWN: A Personalized Federated Learning Framework for D2D Wireless Networks with Heterogeneous Data(https://arxiv.org/abs/2501.09822)
• Keywords: robust, federate
• Abstract: Traditional Federated Learning (FL) approaches often struggle with data heterogeneity across clients, leading to suboptimal model performance for individual clients. To address this issue, Personalized Federated Learning (PFL) emerges as a solution to the challenges posed by non-independent and identically distributed (non-IID) and unbalanced data across clients. Furthermore, in most existing decentralized machine learning works, a perfect communication channel is considered for model parameter transmission between clients and servers. However, decentralized PFL over wireless links introduces new challenges, such as resource allocation and interference management. To overcome these challenges, we formulate a joint optimization problem that incorporates the underlying device-to-device (D2D) wireless channel conditions into a server-free PFL approach. The proposed method, dubbed pFedWN, optimizes the learning performance for each client while accounting for the variability in D2D wireless channels. To tackle the formulated problem, we divide it into two sub-problems: PFL neighbor selection and PFL weight assignment. The PFL neighbor selection is addressed through channel-aware neighbor selection within unlicensed spectrum bands such as ISM bands. Next, to assign PFL weights, we utilize the Expectation-Maximization (EM) method to evaluate the similarity between clients' data and obtain optimal weight distribution among the chosen PFL neighbors. Empirical results show that pFedWN provides efficient and personalized learning performance with non-IID and unbalanced datasets. Furthermore, it outperforms the existing FL and PFL methods in terms of learning efficacy and robustness, particularly under dynamic and unpredictable wireless channel conditions.

Title: Bridging Language Barriers in Healthcare: A Study on Arabic LLMs

• Authors: Nada Saadi, Tathagata Raha, Clément Christophe, Marco AF Pimentel, Ronnie Rajan, Praveen K Kanithi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09825
• Pdf URL: https://arxiv.org/pdf/2501.09825
• Copy Paste: [[2501.09825]] Bridging Language Barriers in Healthcare: A Study on Arabic LLMs(https://arxiv.org/abs/2501.09825)
• Keywords: large language model
• Abstract: This paper investigates the challenges of developing large language models (LLMs) proficient in both multilingual understanding and medical knowledge. We demonstrate that simply translating medical data does not guarantee strong performance on clinical tasks in the target language. Our experiments reveal that the optimal language mix in training data varies significantly across different medical tasks. We find that larger models with carefully calibrated language ratios achieve superior performance on native-language clinical tasks. Furthermore, our results suggest that relying solely on fine-tuning may not be the most effective approach for incorporating new language knowledge into LLMs. Instead, data and computationally intensive pretraining methods may still be necessary to achieve optimal performance in multilingual medical settings. These findings provide valuable guidance for building effective and inclusive medical AI systems for diverse linguistic communities.

Title: PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery

• Authors: Shristi Das Biswas, Matthew Shreve, Xuelu Li, Prateek Singhal, Kaushik Roy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09826
• Pdf URL: https://arxiv.org/pdf/2501.09826
• Copy Paste: [[2501.09826]] PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery(https://arxiv.org/abs/2501.09826)
• Keywords: diffusion
• Abstract: Recent advancements in language-guided diffusion models for image editing are often bottle-necked by cumbersome prompt engineering to precisely articulate desired changes. An intuitive alternative calls on guidance from in-the-wild image exemplars to help users bring their imagined edits to life. Contemporary exemplar-based editing methods shy away from leveraging the rich latent space learnt by pre-existing large text-to-image (TTI) models and fall back on training with curated objective functions to achieve the task. Though somewhat effective, this demands significant computational resources and lacks compatibility with diverse base models and arbitrary exemplar count. On further investigation, we also find that these techniques restrict user control to only applying uniform global changes over the entire edited region. In this paper, we introduce a novel framework for progressive exemplar-driven editing with off-the-shelf diffusion models, dubbed PIXELS, to enable customization by providing granular control over edits, allowing adjustments at the pixel or region level. Our method operates solely during inference to facilitate imitative editing, enabling users to draw inspiration from a dynamic number of reference images, or multimodal prompts, and progressively incorporate all the desired changes without retraining or fine-tuning existing TTI models. This capability of fine-grained control opens up a range of new possibilities, including selective modification of individual objects and specifying gradual spatial changes. We demonstrate that PIXELS delivers high-quality edits efficiently, leading to a notable improvement in quantitative metrics as well as human evaluation. By making high-quality image editing more accessible, PIXELS has the potential to enable professional-grade edits to a wider audience with the ease of using any open-source image generation model.

Title: EraseBench: Understanding The Ripple Effects of Concept Erasure Techniques

• Authors: Ibtihel Amara, Ahmed Imtiaz Humayun, Ivana Kajic, Zarana Parekh, Natalie Harris, Sarah Young, Chirag Nagpal, Najoung Kim, Junfeng He, Cristina Nader Vasconcelos, Deepak Ramachandran, Goolnoosh Farnadi, Katherine Heller, Mohammad Havaei, Negar Rostamzadeh
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09833
• Pdf URL: https://arxiv.org/pdf/2501.09833
• Copy Paste: [[2501.09833]] EraseBench: Understanding The Ripple Effects of Concept Erasure Techniques(https://arxiv.org/abs/2501.09833)
• Keywords: robust
• Abstract: Concept erasure techniques have recently gained significant attention for their potential to remove unwanted concepts from text-to-image models. While these methods often demonstrate success in controlled scenarios, their robustness in real-world applications and readiness for deployment remain uncertain. In this work, we identify a critical gap in evaluating sanitized models, particularly in terms of their performance across various concept dimensions. We systematically investigate the failure modes of current concept erasure techniques, with a focus on visually similar, binomial, and semantically related concepts. We propose that these interconnected relationships give rise to a phenomenon of concept entanglement resulting in ripple effects and degradation in image quality. To facilitate more comprehensive evaluation, we introduce EraseBENCH, a multi-dimensional benchmark designed to assess concept erasure methods with greater depth. Our dataset includes over 100 diverse concepts and more than 1,000 tailored prompts, paired with a comprehensive suite of metrics that together offer a holistic view of erasure efficacy. Our findings reveal that even state-of-the-art techniques struggle with maintaining quality post-erasure, indicating that these approaches are not yet ready for real-world deployment. This highlights the gap in reliability of the concept erasure techniques.

Title: CrossModalityDiffusion: Multi-Modal Novel View Synthesis with Unified Intermediate Representation

• Authors: Alex Berian, Daniel Brignac, JhihYang Wu, Natnael Daba, Abhijit Mahalanobis
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09838
• Pdf URL: https://arxiv.org/pdf/2501.09838
• Copy Paste: [[2501.09838]] CrossModalityDiffusion: Multi-Modal Novel View Synthesis with Unified Intermediate Representation(https://arxiv.org/abs/2501.09838)
• Keywords: diffusion
• Abstract: Geospatial imaging leverages data from diverse sensing modalities-such as EO, SAR, and LiDAR, ranging from ground-level drones to satellite views. These heterogeneous inputs offer significant opportunities for scene understanding but present challenges in interpreting geometry accurately, particularly in the absence of precise ground truth data. To address this, we propose CrossModalityDiffusion, a modular framework designed to generate images across different modalities and viewpoints without prior knowledge of scene geometry. CrossModalityDiffusion employs modality-specific encoders that take multiple input images and produce geometry-aware feature volumes that encode scene structure relative to their input camera positions. The space where the feature volumes are placed acts as a common ground for unifying input modalities. These feature volumes are overlapped and rendered into feature images from novel perspectives using volumetric rendering techniques. The rendered feature images are used as conditioning inputs for a modality-specific diffusion model, enabling the synthesis of novel images for the desired output modality. In this paper, we show that jointly training different modules ensures consistent geometric understanding across all modalities within the framework. We validate CrossModalityDiffusion's capabilities on the synthetic ShapeNet cars dataset, demonstrating its effectiveness in generating accurate and consistent novel views across multiple imaging modalities and perspectives.

Title: From Explainability to Interpretability: Interpretable Policies in Reinforcement Learning Via Model Explanation

• Authors: Peilang Li, Umer Siddique, Yongcan Cao
• Subjects: cs.LG, cs.AI, eess.SY
• Abstract URL: https://arxiv.org/abs/2501.09858
• Pdf URL: https://arxiv.org/pdf/2501.09858
• Copy Paste: [[2501.09858]] From Explainability to Interpretability: Interpretable Policies in Reinforcement Learning Via Model Explanation(https://arxiv.org/abs/2501.09858)
• Keywords: interpretability, explainability
• Abstract: Deep reinforcement learning (RL) has shown remarkable success in complex domains, however, the inherent black box nature of deep neural network policies raises significant challenges in understanding and trusting the decision-making processes. While existing explainable RL methods provide local insights, they fail to deliver a global understanding of the model, particularly in high-stakes applications. To overcome this limitation, we propose a novel model-agnostic approach that bridges the gap between explainability and interpretability by leveraging Shapley values to transform complex deep RL policies into transparent representations. The proposed approach offers two key contributions: a novel approach employing Shapley values to policy interpretation beyond local explanations and a general framework applicable to off-policy and on-policy algorithms. We evaluate our approach with three existing deep RL algorithms and validate its performance in two classic control environments. The results demonstrate that our approach not only preserves the original models' performance but also generates more stable interpretable policies.

Title: An LLM-Guided Tutoring System for Social Skills Training

• Authors: Michael Guevarra (1), Indronil Bhattacharjee (2), Srijita Das (3), Christabel Wayllace (2), Carrie Demmans Epp (4), Matthew E. Taylor (4 and 5), Alan Tay (1) ((1) Illumia Labs, (2) New Mexico State University, (3) University of Michigan-Dearborn, (4) University of Alberta, (5) Alberta Machine Intelligence Institute)
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.09870
• Pdf URL: https://arxiv.org/pdf/2501.09870
• Copy Paste: [[2501.09870]] An LLM-Guided Tutoring System for Social Skills Training(https://arxiv.org/abs/2501.09870)
• Keywords: large language model
• Abstract: Social skills training targets behaviors necessary for success in social interactions. However, traditional classroom training for such skills is often insufficient to teach effective communication -- one-to-one interaction in real-world scenarios is preferred to lecture-style information delivery. This paper introduces a framework that allows instructors to collaborate with large language models to dynamically design realistic scenarios for students to communicate. Our framework uses these scenarios to enable student rehearsal, provide immediate feedback, and visualize performance for both students and instructors. Unlike traditional intelligent tutoring systems, instructors can easily co-create scenarios with a large language model without technical skills. Additionally, the system generates new scenario branches in real time when existing options do not fit the student's response.

Title: ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction

• Authors: Izzeddin Teeti, Aniket Thomas, Munish Monga, Sachin Kumar, Uddeshya Singh, Andrew Bradley, Biplab Banerjee, Fabio Cuzzolin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09878
• Pdf URL: https://arxiv.org/pdf/2501.09878
• Copy Paste: [[2501.09878]] ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction(https://arxiv.org/abs/2501.09878)
• Keywords: transformer, generative
• Abstract: We present ASTRA (A} Scene-aware TRAnsformer-based model for trajectory prediction), a light-weight pedestrian trajectory forecasting model that integrates the scene context, spatial dynamics, social inter-agent interactions and temporal progressions for precise forecasting. We utilised a U-Net-based feature extractor, via its latent vector representation, to capture scene representations and a graph-aware transformer encoder for capturing social interactions. These components are integrated to learn an agent-scene aware embedding, enabling the model to learn spatial dynamics and forecast the future trajectory of pedestrians. The model is designed to produce both deterministic and stochastic outcomes, with the stochastic predictions being generated by incorporating a Conditional Variational Auto-Encoder (CVAE). ASTRA also proposes a simple yet effective weighted penalty loss function, which helps to yield predictions that outperform a wide array of state-of-the-art deterministic and generative models. ASTRA demonstrates an average improvement of 27%/10% in deterministic/stochastic settings on the ETH-UCY dataset, and 26% improvement on the PIE dataset, respectively, along with seven times fewer parameters than the existing state-of-the-art model (see Figure 1). Additionally, the model's versatility allows it to generalize across different perspectives, such as Bird's Eye View (BEV) and Ego-Vehicle View (EVV).

Title: Semi-Supervised Image-Based Narrative Extraction: A Case Study with Historical Photographic Records

• Authors: Fausto German, Brian Keith, Mauricio Matus, Diego Urrutia, Claudio Meneses
• Subjects: cs.CV, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.09884
• Pdf URL: https://arxiv.org/pdf/2501.09884
• Copy Paste: [[2501.09884]] Semi-Supervised Image-Based Narrative Extraction: A Case Study with Historical Photographic Records(https://arxiv.org/abs/2501.09884)
• Keywords: extraction
• Abstract: This paper presents a semi-supervised approach to extracting narratives from historical photographic records using an adaptation of the narrative maps algorithm. We extend the original unsupervised text-based method to work with image data, leveraging deep learning techniques for visual feature extraction and similarity computation. Our method is applied to the ROGER dataset, a collection of photographs from the 1928 Sacambaya Expedition in Bolivia captured by Robert Gerstmann. We compare our algorithmically extracted visual narratives with expert-curated timelines of varying lengths (5 to 30 images) to evaluate the effectiveness of our approach. In particular, we use the Dynamic Time Warping (DTW) algorithm to match the extracted narratives with the expert-curated baseline. In addition, we asked an expert on the topic to qualitatively evaluate a representative example of the resulting narratives. Our findings show that the narrative maps approach generally outperforms random sampling for longer timelines (10+ images, p < 0.05), with expert evaluation confirming the historical accuracy and coherence of the extracted narratives. This research contributes to the field of computational analysis of visual cultural heritage, offering new tools for historians, archivists, and digital humanities scholars to explore and understand large-scale image collections. The method's ability to generate meaningful narratives from visual data opens up new possibilities for the study and interpretation of historical events through photographic evidence.

Title: FLORA: Formal Language Model Enables Robust Training-free Zero-shot Object Referring Analysis

• Authors: Zhe Chen, Zijing Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09887
• Pdf URL: https://arxiv.org/pdf/2501.09887
• Copy Paste: [[2501.09887]] FLORA: Formal Language Model Enables Robust Training-free Zero-shot Object Referring Analysis(https://arxiv.org/abs/2501.09887)
• Keywords: robust, interpretability, large language model, segmentation
• Abstract: Object Referring Analysis (ORA), commonly known as referring expression comprehension, requires the identification and localization of specific objects in an image based on natural descriptions. Unlike generic object detection, ORA requires both accurate language understanding and precise visual localization, making it inherently more complex. Although recent pre-trained large visual grounding detectors have achieved significant progress, they heavily rely on extensively labeled data and time-consuming learning. To address these, we introduce a novel, training-free framework for zero-shot ORA, termed FLORA (Formal Language for Object Referring and Analysis). FLORA harnesses the inherent reasoning capabilities of large language models (LLMs) and integrates a formal language model - a logical framework that regulates language within structured, rule-based descriptions - to provide effective zero-shot ORA. More specifically, our formal language model (FLM) enables an effective, logic-driven interpretation of object descriptions without necessitating any training processes. Built upon FLM-regulated LLM outputs, we further devise a Bayesian inference framework and employ appropriate off-the-shelf interpretive models to finalize the reasoning, delivering favorable robustness against LLM hallucinations and compelling ORA performance in a training-free manner. In practice, our FLORA boosts the zero-shot performance of existing pretrained grounding detectors by up to around 45%. Our comprehensive evaluation across different challenging datasets also confirms that FLORA consistently surpasses current state-of-the-art zero-shot methods in both detection and segmentation tasks associated with zero-shot ORA. We believe our probabilistic parsing and reasoning of the LLM outputs elevate the reliability and interpretability of zero-shot ORA. We shall release codes upon publication.

Title: Advancing Image Security with Quantum Key Distribution and Multi-Layer Chaotic Encryption for Quantum Resilient Transmission

• Authors: Tasmin Karim, Md. Shazzad Hossain Shaon, Md. Fahim Sultan, Mst Shapna Akter
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.09895
• Pdf URL: https://arxiv.org/pdf/2501.09895
• Copy Paste: [[2501.09895]] Advancing Image Security with Quantum Key Distribution and Multi-Layer Chaotic Encryption for Quantum Resilient Transmission(https://arxiv.org/abs/2501.09895)
• Keywords: secure, security, attack, robust
• Abstract: Quantum security improves cryptographic protocols by applying quantum mechanics principles, assuring resistance to both quantum and conventional computer attacks. This work addresses these issues by integrating Quantum Key Distribution (QKD) utilizing the E91 method with Multi-Layer Chaotic Encryption, which employs a variety of patterns to detect eavesdropping, resulting in a highly secure image-transmission architecture. The method leverages entropy calculations to determine the unpredictability and integrity of encrypted and decrypted pictures, guaranteeing strong security. Extensive statistical scenarios illustrate the framework's effectiveness in image encryption while preserving high entropy and sensitivity to the original visuals. The findings indicate significant improvement in encryption and decryption performance, demonstrating the framework's potential as a robust response to weaknesses introduced by advances in quantum computing. Several metrics, such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Normalized Cross-Correlation (NCC), Bit Error Rate (BER), entropy values for original, encrypted, and decrypted images, and the correlation between original and decrypted images, validate the framework's effectiveness. The combination of QKD with Multi-Layer Chaotic Encryption provides a scalable and resilient technique to secure image communication. As quantum computing advances, this framework offers a future-proof approach for defining secure communication protocols in crucial sectors such as medical treatment, forensic computing, and national security, where information confidentiality is valuable.

Title: FoundationStereo: Zero-Shot Stereo Matching

• Authors: Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield
• Subjects: cs.CV, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2501.09898
• Pdf URL: https://arxiv.org/pdf/2501.09898
• Copy Paste: [[2501.09898]] FoundationStereo: Zero-Shot Stereo Matching(https://arxiv.org/abs/2501.09898)
• Keywords: robust
• Abstract: Tremendous progress has been made in deep stereo matching to excel on benchmark datasets through per-domain fine-tuning. However, achieving strong zero-shot generalization - a hallmark of foundation models in other computer vision tasks - remains challenging for stereo matching. We introduce FoundationStereo, a foundation model for stereo depth estimation designed to achieve strong zero-shot generalization. To this end, we first construct a large-scale (1M stereo pairs) synthetic training dataset featuring large diversity and high photorealism, followed by an automatic self-curation pipeline to remove ambiguous samples. We then design a number of network architecture components to enhance scalability, including a side-tuning feature backbone that adapts rich monocular priors from vision foundation models to mitigate the sim-to-real gap, and long-range context reasoning for effective cost volume filtering. Together, these components lead to strong robustness and accuracy across domains, establishing a new standard in zero-shot stereo depth estimation.

Title: Steering Large Language Models with Feature Guided Activation Additions

• Authors: Samuel Soo, Wesley Teng, Chandrasekaran Balaganesh
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09929
• Pdf URL: https://arxiv.org/pdf/2501.09929
• Copy Paste: [[2501.09929]] Steering Large Language Models with Feature Guided Activation Additions(https://arxiv.org/abs/2501.09929)
• Keywords: interpretability, large language model
• Abstract: Effective and reliable control over large language model (LLM) behavior is a significant challenge. While activation steering methods, which add steering vectors to a model's hidden states, are a promising approach, existing techniques often lack precision and interpretability in how they influence model outputs. We introduce Feature Guided Activation Additions (FGAA), a novel activation steering method that leverages insights from Contrastive Activation Addition (CAA) and Sparse Autoencoder-Targeted Steering (SAE-TS). By operating in the latent space of a Sparse Autoencoder (SAE) and employing optimization techniques to select desired SAE features, FGAA constructs precise steering vectors that provide better steering effects while maintaining coherence of steered model outputs. In this regard, evaluations on Gemma-2-2B and Gemma-2-9B models across various steering tasks demonstrate that FGAA outperforms existing steering methods of CAA, SAE decoder steering, and SAE-TS. Our results also highlight important trade-offs between steering scale and general model capabilities that are consistent across all tested steering methods.

Title: HEART: Achieving Timely Multi-Model Training for Vehicle-Edge-Cloud-Integrated Hierarchical Federated Learning

• Authors: Xiaohong Yang, Minghui Liwang, Xianbin Wang, Zhipeng Cheng, Seyyedali Hosseinalipour, Huaiyu Dai, Zhenzhen Jiao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09934
• Pdf URL: https://arxiv.org/pdf/2501.09934
• Copy Paste: [[2501.09934]] HEART: Achieving Timely Multi-Model Training for Vehicle-Edge-Cloud-Integrated Hierarchical Federated Learning(https://arxiv.org/abs/2501.09934)
• Keywords: federate
• Abstract: The rapid growth of AI-enabled Internet of Vehicles (IoV) calls for efficient machine learning (ML) solutions that can handle high vehicular mobility and decentralized data. This has motivated the emergence of Hierarchical Federated Learning over vehicle-edge-cloud architectures (VEC-HFL). Nevertheless, one aspect which is underexplored in the literature on VEC-HFL is that vehicles often need to execute multiple ML tasks simultaneously, where this multi-model training environment introduces crucial challenges. First, improper aggregation rules can lead to model obsolescence and prolonged training times. Second, vehicular mobility may result in inefficient data utilization by preventing the vehicles from returning their models to the network edge. Third, achieving a balanced resource allocation across diverse tasks becomes of paramount importance as it majorly affects the effectiveness of collaborative training. We take one of the first steps towards addressing these challenges via proposing a framework for multi-model training in dynamic VEC-HFL with the goal of minimizing global training latency while ensuring balanced training across various tasks-a problem that turns out to be NP-hard. To facilitate timely model training, we introduce a hybrid synchronous-asynchronous aggregation rule. Building on this, we present a novel method called Hybrid Evolutionary And gReedy allocaTion (HEART). The framework operates in two stages: first, it achieves balanced task scheduling through a hybrid heuristic approach that combines improved Particle Swarm Optimization (PSO) and Genetic Algorithms (GA); second, it employs a low-complexity greedy algorithm to determine the training priority of assigned tasks on vehicles. Experiments on real-world datasets demonstrate the superiority of HEART over existing methods.

Title: Threat-Specific Risk Assessment for IP Multimedia Subsystem Networks Based on Hierarchical Models

• Authors: Abdullah Ehsan Shaikh, Simon Yusuf Enoch
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.09936
• Pdf URL: https://arxiv.org/pdf/2501.09936
• Copy Paste: [[2501.09936]] Threat-Specific Risk Assessment for IP Multimedia Subsystem Networks Based on Hierarchical Models(https://arxiv.org/abs/2501.09936)
• Keywords: security, defense, attack
• Abstract: Over the years, IP Multimedia Subsystems (IMS) networks have become increasingly critical as they form the backbone of modern telecommunications, enabling the integration of multimedia services such as voice, video, and messaging over IP-based infrastructures and next-generation networks. However, this integration has led to an increase in the attack surface of the IMS network, making it more prone to various forms of cyber threats and attacks, including Denial of Service (DoS) attacks, SIP-based attacks, unauthorized access, etc. As a result, it is important to find a way to manage and assess the security of IMS networks, but there is a lack of a systematic approach to managing the identification of vulnerabilities and threats. In this paper, we propose a model and a threat-specific risk security modeling and assessment approach to model and assess the threats of the IMS network. This model will provide a structured methodology for representing and analyzing threats and attack scenarios in layers within a hierarchical model. The proposed model aims to enhance the security posture of IMS networks by improving vulnerability management, risk evaluation, and defense evaluation against cyber threats. We perform a preliminary evaluation based on vulnerability collected from the National Vulnerability Database for devices in the IMS network. The results showed that we can model and assess the threats of IMS networks. IMS network defenders can use this model to understand their security postures taking into account the threat and risk posed by each vulnerability.

Title: A Multi-Scale Feature Extraction and Fusion Deep Learning Method for Classification of Wheat Diseases

• Authors: Sajjad Saleem, Adil Hussain, Nabila Majeed, Zahid Akhtar, Kamran Siddique
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.09938
• Pdf URL: https://arxiv.org/pdf/2501.09938
• Copy Paste: [[2501.09938]] A Multi-Scale Feature Extraction and Fusion Deep Learning Method for Classification of Wheat Diseases(https://arxiv.org/abs/2501.09938)
• Keywords: extraction, segmentation
• Abstract: Wheat is an important source of dietary fiber and protein that is negatively impacted by a number of risks to its growth. The difficulty of identifying and classifying wheat diseases is discussed with an emphasis on wheat loose smut, leaf rust, and crown and root rot. Addressing conditions like crown and root rot, this study introduces an innovative approach that integrates multi-scale feature extraction with advanced image segmentation techniques to enhance classification accuracy. The proposed method uses neural network models Xception, Inception V3, and ResNet 50 to train on a large wheat disease classification dataset 2020 in conjunction with an ensemble of machine vision classifiers, including voting and stacking. The study shows that the suggested methodology has a superior accuracy of 99.75% in the classification of wheat diseases when compared to current state-of-the-art approaches. A deep learning ensemble model Xception showed the highest accuracy.

Title: Passage Segmentation of Documents for Extractive Question Answering

• Authors: Zuhong Liu, Charles-Elie Simon, Fabien Caspani
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.09940
• Pdf URL: https://arxiv.org/pdf/2501.09940
• Copy Paste: [[2501.09940]] Passage Segmentation of Documents for Extractive Question Answering(https://arxiv.org/abs/2501.09940)
• Keywords: segmentation
• Abstract: Retrieval-Augmented Generation (RAG) has proven effective in open-domain question answering. However, the chunking process, which is essential to this pipeline, often receives insufficient attention relative to retrieval and synthesis components. This study emphasizes the critical role of chunking in improving the performance of both dense passage retrieval and the end-to-end RAG pipeline. We then introduce the Logits-Guided Multi-Granular Chunker (LGMGC), a novel framework that splits long documents into contextualized, self-contained chunks of varied granularity. Our experimental results, evaluated on two benchmark datasets, demonstrate that LGMGC not only improves the retrieval step but also outperforms existing chunking methods when integrated into a RAG pipeline.

Title: Client-Centric Federated Adaptive Optimization

• Authors: Jianhui Sun, Xidong Wu, Heng Huang, Aidong Zhang
• Subjects: cs.LG, cs.AI, math.OC
• Abstract URL: https://arxiv.org/abs/2501.09946
• Pdf URL: https://arxiv.org/pdf/2501.09946
• Copy Paste: [[2501.09946]] Client-Centric Federated Adaptive Optimization(https://arxiv.org/abs/2501.09946)
• Keywords: federate
• Abstract: Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. With an increasing scale of clients and models, FL encounters two key challenges, client drift due to a high degree of statistical/system heterogeneity, and lack of adaptivity. However, most existing FL research is based on unrealistic assumptions that virtually ignore system heterogeneity. In this paper, we propose Client-Centric Federated Adaptive Optimization, which is a class of novel federated adaptive optimization approaches. We enable several features in this framework such as arbitrary client participation, asynchronous server aggregation, and heterogeneous local computing, which are ubiquitous in real-world FL systems but are missed in most existing works. We provide a rigorous convergence analysis of our proposed framework for general nonconvex objectives, which is shown to converge with the best-known rate. Extensive experiments show that our approaches consistently outperform the baseline by a large margin across benchmarks.

Title: Surface-SOS: Self-Supervised Object Segmentation via Neural Surface Representation

• Authors: Xiaoyun Zheng, Liwei Liao, Jianbo Jiao, Feng Gao, Ronggang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09947
• Pdf URL: https://arxiv.org/pdf/2501.09947
• Copy Paste: [[2501.09947]] Surface-SOS: Self-Supervised Object Segmentation via Neural Surface Representation(https://arxiv.org/abs/2501.09947)
• Keywords: segmentation
• Abstract: Self-supervised Object Segmentation (SOS) aims to segment objects without any annotations. Under conditions of multi-camera inputs, the structural, textural and geometrical consistency among each view can be leveraged to achieve fine-grained object segmentation. To make better use of the above information, we propose Surface representation based Self-supervised Object Segmentation (Surface-SOS), a new framework to segment objects for each view by 3D surface representation from multi-view images of a scene. To model high-quality geometry surfaces for complex scenes, we design a novel scene representation scheme, which decomposes the scene into two complementary neural representation modules respectively with a Signed Distance Function (SDF). Moreover, Surface-SOS is able to refine single-view segmentation with multi-view unlabeled images, by introducing coarse segmentation masks as additional input. To the best of our knowledge, Surface-SOS is the first self-supervised approach that leverages neural surface representation to break the dependence on large amounts of annotated data and strong constraints. These constraints typically involve observing target objects against a static background or relying on temporal supervision in videos. Extensive experiments on standard benchmarks including LLFF, CO3D, BlendedMVS, TUM and several real-world scenes show that Surface-SOS always yields finer object masks than its NeRF-based counterparts and surpasses supervised single-view baselines remarkably. Code is available at: this https URL.

Title: MultiPruner: Balanced Structure Removal in Foundation Models

• Authors: J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09949
• Pdf URL: https://arxiv.org/pdf/2501.09949
• Copy Paste: [[2501.09949]] MultiPruner: Balanced Structure Removal in Foundation Models(https://arxiv.org/abs/2501.09949)
• Keywords: transformer
• Abstract: Recently, state-of-the-art approaches for pruning large pre-trained models (LPMs) have demonstrated that the training-free removal of non-critical residual blocks in Transformers is viable for reducing model size, achieving results that outperform previous training-free pruning approaches. Motivated by these findings, we extend BlockPruner (Zhong et al., 2024) and propose MultiPruner, a pruning approach that surpasses recent training-free pruning methods by adopting a multidimensional, iterative, fine-grained pruning strategy. In MultiPruner, multidimensional pruning reinstates the structural balance in block-pruned models by sequentially compressing along three dimensions: i) residual blocks, ii) channels of multilayer perceptrons (MLP), and iii) attention heads. This solution enhances zero-shot accuracy on downstream tasks compared to other techniques while improving model compression ratios, producing compressed models with fewer computing and memory requirements. Extensive experiments demonstrate the advantages of the proposed method across various large pre-trained models. The code and pruning configurations are available at this https URL.

Title: AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representations

• Authors: Jamin Seo, Akshat Ramachandran, Yu-Chuan Chuang, Anirudh Itagi, Tushar Krishna
• Subjects: cs.LG, cs.AI, cs.AR
• Abstract URL: https://arxiv.org/abs/2501.09954
• Pdf URL: https://arxiv.org/pdf/2501.09954
• Copy Paste: [[2501.09954]] AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representations(https://arxiv.org/abs/2501.09954)
• Keywords: transformer
• Abstract: Design space exploration (DSE) plays a crucial role in enabling custom hardware architectures, particularly for emerging applications like AI, where optimized and specialized designs are essential. With the growing complexity of deep neural networks (DNNs) and the introduction of advanced foundational models (FMs), the design space for DNN accelerators is expanding at an exponential rate. Additionally, this space is highly non-uniform and non-convex, making it increasingly difficult to navigate and optimize. Traditional DSE techniques rely on search-based methods, which involve iterative sampling of the design space to find the optimal solution. However, this process is both time-consuming and often fails to converge to the global optima for such design spaces. Recently, AIrchitect v1, the first attempt to address the limitations of search-based techniques, transformed DSE into a constant-time classification problem using recommendation networks. In this work, we propose AIrchitect v2, a more accurate and generalizable learning-based DSE technique applicable to large-scale design spaces that overcomes the shortcomings of earlier approaches. Specifically, we devise an encoder-decoder transformer model that (a) encodes the complex design space into a uniform intermediate representation using contrastive learning and (b) leverages a novel unified representation blending the advantages of classification and regression to effectively explore the large DSE space without sacrificing accuracy. Experimental results evaluated on 10^5 real DNN workloads demonstrate that, on average, AIrchitect v2 outperforms existing techniques by 15% in identifying optimal design points. Furthermore, to demonstrate the generalizability of our method, we evaluate performance on unseen model workloads (LLMs) and attain a 1.7x improvement in inference latency on the identified hardware architecture.

Title: FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs

• Authors: Zengyi Gao, Yukun Cao, Hairu Wang, Ao Ke, Yuan Feng, Xike Xie, S Kevin Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09957
• Pdf URL: https://arxiv.org/pdf/2501.09957
• Copy Paste: [[2501.09957]] FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs(https://arxiv.org/abs/2501.09957)
• Keywords: large language model
• Abstract: To mitigate the hallucination and knowledge deficiency in large language models (LLMs), Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) has shown promising potential by utilizing KGs as external resource to enhance LLMs this http URL, existing KG-RAG approaches struggle with a trade-off between flexibility and retrieval this http URL methods prioritize flexibility by avoiding the use of KG-fine-tuned models during retrieval, leading to fixed retrieval strategies and suboptimal retrieval this http URL, coupled methods embed KG information within models to improve retrieval quality, but at the expense of this http URL this paper, we propose a novel flexible modular KG-RAG framework, termed FRAG, which synergizes the advantages of both this http URL estimates the hop range of reasoning paths based solely on the query and classify it as either simple or this http URL match the complexity of the query, tailored pipelines are applied to ensure efficient and accurate reasoning path retrieval, thus fostering the final reasoning this http URL using the query text instead of the KG to infer the structural information of reasoning paths and employing adaptable retrieval strategies, FRAG improves retrieval quality while maintaining this http URL, FRAG does not require extra LLMs fine-tuning or calls, significantly boosting efficiency and conserving this http URL experiments show that FRAG achieves state-of-the-art performance with high efficiency and low resource consumption.

Title: A Survey on Multi-Turn Interaction Capabilities of Large Language Models

• Authors: Chen Zhang, Xinyi Dai, Yaxiong Wu, Qu Yang, Yasheng Wang, Ruiming Tang, Yong Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09959
• Pdf URL: https://arxiv.org/pdf/2501.09959
• Copy Paste: [[2501.09959]] A Survey on Multi-Turn Interaction Capabilities of Large Language Models(https://arxiv.org/abs/2501.09959)
• Keywords: large language model
• Abstract: Multi-turn interaction in the dialogue system research refers to a system's ability to maintain context across multiple dialogue turns, enabling it to generate coherent and contextually relevant responses. Recent advancements in large language models (LLMs) have significantly expanded the scope of multi-turn interaction, moving beyond chatbots to enable more dynamic agentic interactions with users or environments. In this paper, we provide a focused review of the multi-turn capabilities of LLMs, which are critical for a wide range of downstream applications, including conversational search and recommendation, consultation services, and interactive tutoring. This survey explores four key aspects: (1) the core model capabilities that contribute to effective multi-turn interaction, (2) how multi-turn interaction is evaluated in current practice, (3) the general algorithms used to enhance multi-turn interaction, and (4) potential future directions for research in this field.

Title: Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration

• Authors: Lianxin Xie, Bingbing Zheng, Wen Xue, Yunfei Zhang, Le Jiang, Ruotao Xu, Si Wu, Hau-San Wong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09960
• Pdf URL: https://arxiv.org/pdf/2501.09960
• Copy Paste: [[2501.09960]] Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration(https://arxiv.org/abs/2501.09960)
• Keywords: transformer
• Abstract: Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration.

Title: Explainable artificial intelligence (XAI): from inherent explainability to large language models

• Authors: Fuseini Mumuni, Alhassan Mumuni
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2501.09967
• Pdf URL: https://arxiv.org/pdf/2501.09967
• Copy Paste: [[2501.09967]] Explainable artificial intelligence (XAI): from inherent explainability to large language models(https://arxiv.org/abs/2501.09967)
• Keywords: interpretability, explainability, large language model
• Abstract: Artificial Intelligence (AI) has continued to achieve tremendous success in recent times. However, the decision logic of these frameworks is often not transparent, making it difficult for stakeholders to understand, interpret or explain their behavior. This limitation hinders trust in machine learning systems and causes a general reluctance towards their adoption in practical applications, particularly in mission-critical domains like healthcare and autonomous driving. Explainable AI (XAI) techniques facilitate the explainability or interpretability of machine learning models, enabling users to discern the basis of the decision and possibly avert undesirable behavior. This comprehensive survey details the advancements of explainable AI methods, from inherently interpretable models to modern approaches for achieving interpretability of various black box models, including large language models (LLMs). Additionally, we review explainable AI techniques that leverage LLM and vision-language model (VLM) frameworks to automate or improve the explainability of other machine learning models. The use of LLM and VLM as interpretability methods particularly enables high-level, semantically meaningful explanations of model decisions and behavior. Throughout the paper, we highlight the scientific principles, strengths and weaknesses of state-of-the-art methods and outline different areas of improvement. Where appropriate, we also present qualitative and quantitative comparison results of various methods to show how they compare. Finally, we discuss the key challenges of XAI and directions for future research.

Title: Aneumo: A Large-Scale Comprehensive Synthetic Dataset of Aneurysm Hemodynamics

• Authors: Xigui Li, Yuanye Zhou, Feiyang Xiao, Xin Guo, Yichi Zhang, Chen Jiang, Jianchao Ge, Xiansheng Wang, Qimeng Wang, Taiwei Zhang, Chensen Lin, Yuan Cheng, Yuan Qi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.09980
• Pdf URL: https://arxiv.org/pdf/2501.09980
• Copy Paste: [[2501.09980]] Aneumo: A Large-Scale Comprehensive Synthetic Dataset of Aneurysm Hemodynamics(https://arxiv.org/abs/2501.09980)
• Keywords: segmentation
• Abstract: Intracranial aneurysm (IA) is a common cerebrovascular disease that is usually asymptomatic but may cause severe subarachnoid hemorrhage (SAH) if ruptured. Although clinical practice is usually based on individual factors and morphological features of the aneurysm, its pathophysiology and hemodynamic mechanisms remain controversial. To address the limitations of current research, this study constructed a comprehensive hemodynamic dataset of intracranial aneurysms. The dataset is based on 466 real aneurysm models, and 10,000 synthetic models were generated by resection and deformation operations, including 466 aneurysm-free models and 9,534 deformed aneurysm models. The dataset also provides medical image-like segmentation mask files to support insightful analysis. In addition, the dataset contains hemodynamic data measured at eight steady-state flow rates (0.001 to 0.004 kg/s), including critical parameters such as flow velocity, pressure, and wall shear stress, providing a valuable resource for investigating aneurysm pathogenesis and clinical prediction. This dataset will help advance the understanding of the pathologic features and hemodynamic mechanisms of intracranial aneurysms and support in-depth research in related fields. Dataset hosted at this https URL.

Title: Agent-as-Judge for Factual Summarization of Long Narratives

• Authors: Yeonseok Jeong, Minsoo Kim, Seung-won Hwang, Byung-Hak Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.09993
• Pdf URL: https://arxiv.org/pdf/2501.09993
• Copy Paste: [[2501.09993]] Agent-as-Judge for Factual Summarization of Long Narratives(https://arxiv.org/abs/2501.09993)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated near-human performance in summarization tasks based on traditional metrics such as ROUGE and BERTScore. However, these metrics do not adequately capture critical aspects of summarization quality, such as factual accuracy, particularly for long narratives (>100K tokens). Recent advances, such as LLM-as-a-Judge, address the limitations of metrics based on lexical similarity but still exhibit factual inconsistencies, especially in understanding character relationships and states. In this work, we introduce NarrativeFactScore, a novel "Agent-as-a-Judge" framework for evaluating and refining summaries. By leveraging a Character Knowledge Graph (CKG) extracted from input and generated summaries, NarrativeFactScore assesses the factual consistency and provides actionable guidance for refinement, such as identifying missing or erroneous facts. We demonstrate the effectiveness of NarrativeFactScore through a detailed workflow illustration and extensive validation on widely adopted benchmarks, achieving superior performance compared to competitive methods. Our results highlight the potential of agent-driven evaluation systems to improve the factual reliability of LLM-generated summaries.

Title: Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography

• Authors: Mohammed Salah, Naoufel Werghi, Davor Svetinovic, Yusra Abdulrahman
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09994
• Pdf URL: https://arxiv.org/pdf/2501.09994
• Copy Paste: [[2501.09994]] Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography(https://arxiv.org/abs/2501.09994)
• Keywords: segmentation
• Abstract: AI-driven pulse thermography (PT) has become a crucial tool in non-destructive testing (NDT), enabling automatic detection of hidden anomalies in various industrial components. Current state-of-the-art techniques feed segmentation and depth estimation networks compressed PT sequences using either Principal Component Analysis (PCA) or Thermographic Signal Reconstruction (TSR). However, treating these two modalities independently constrains the performance of PT inspection models as these representations possess complementary semantic features. To address this limitation, this work proposes PT-Fusion, a multi-modal attention-based fusion network that fuses both PCA and TSR modalities for defect segmentation and depth estimation of subsurface defects in PT setups. PT-Fusion introduces novel feature fusion modules, Encoder Attention Fusion Gate (EAFG) and Attention Enhanced Decoding Block (AEDB), to fuse PCA and TSR features for enhanced segmentation and depth estimation of subsurface defects. In addition, a novel data augmentation technique is proposed based on random data sampling from thermographic sequences to alleviate the scarcity of PT datasets. The proposed method is benchmarked against state-of-the-art PT inspection models, including U-Net, attention U-Net, and 3D-CNN on the Université Laval IRT-PVC dataset. The results demonstrate that PT-Fusion outperforms the aforementioned models in defect segmentation and depth estimation accuracies with a margin of 10%.

Title: Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models

• Authors: Qiang Liu, Xinlong Chen, Yue Ding, Shizhen Xu, Shu Wu, Liang Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.09997
• Pdf URL: https://arxiv.org/pdf/2501.09997
• Copy Paste: [[2501.09997]] Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models(https://arxiv.org/abs/2501.09997)
• Keywords: large language model
• Abstract: Hallucination has emerged as a significant barrier to the effective application of Large Language Models (LLMs). In this work, we introduce a novel Attention-Guided SElf-Reflection (AGSER) approach for zero-shot hallucination detection in LLMs. The AGSER method utilizes attention contributions to categorize the input query into attentive and non-attentive queries. Each query is then processed separately through the LLMs, allowing us to compute consistency scores between the generated responses and the original answer. The difference between the two consistency scores serves as a hallucination estimator. In addition to its efficacy in detecting hallucinations, AGSER notably reduces computational complexity, requiring only three passes through the LLM and utilizing two sets of tokens. We have conducted extensive experiments with four widely-used LLMs across three different hallucination benchmarks, demonstrating that our approach significantly outperforms existing methods in zero-shot hallucination detection.

Title: Deep Learning for Early Alzheimer Disease Detection with MRI Scans

• Authors: Mohammad Rafsan, Tamer Oraby, Upal Roy, Sanjeev Kumar, Hansapani Rodrigo
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.09999
• Pdf URL: https://arxiv.org/pdf/2501.09999
• Copy Paste: [[2501.09999]] Deep Learning for Early Alzheimer Disease Detection with MRI Scans(https://arxiv.org/abs/2501.09999)
• Keywords: robust, generative
• Abstract: Alzheimer's Disease is a neurodegenerative condition characterized by dementia and impairment in neurological function. The study primarily focuses on the individuals above age 40, affecting their memory, behavior, and cognitive processes of the brain. Alzheimer's disease requires diagnosis by a detailed assessment of MRI scans and neuropsychological tests of the patients. This project compares existing deep learning models in the pursuit of enhancing the accuracy and efficiency of AD diagnosis, specifically focusing on the Convolutional Neural Network, Bayesian Convolutional Neural Network, and the U-net model with the Open Access Series of Imaging Studies brain MRI dataset. Besides, to ensure robustness and reliability in the model evaluations, we address the challenge of imbalance in data. We then perform rigorous evaluation to determine strengths and weaknesses for each model by considering sensitivity, specificity, and computational efficiency. This comparative analysis would shed light on the future role of AI in revolutionizing AD diagnostics but also paved ways for future innovation in medical imaging and the management of neurodegenerative diseases.

Title: SyzParam: Introducing Runtime Parameters into Kernel Driver Fuzzing

• Authors: Yue Sun, Yan Kang, Chenggang Wu, Kangjie Lu, Jiming Wang, Xingwei Li, Yuhao Hu, Jikai Ren, Yuanming Lai, Mengyao Xie, Zhe Wang
• Subjects: cs.CR, cs.OS
• Abstract URL: https://arxiv.org/abs/2501.10002
• Pdf URL: https://arxiv.org/pdf/2501.10002
• Copy Paste: [[2501.10002]] SyzParam: Introducing Runtime Parameters into Kernel Driver Fuzzing(https://arxiv.org/abs/2501.10002)
• Keywords: extraction
• Abstract: This paper introduces a novel fuzzing framework, SyzParam which incorporates runtime parameters into the fuzzing process. Achieving this objective requires addressing several key challenges, including valid value extraction, inter-device relation construction, and fuzz engine integration. By inspecting the data structures and functions associated with the LKDM, our tool can extract runtime parameters across various drivers through static analysis. Additionally, SyzParam collects inter-device relations and identifies associations between runtime parameters and drivers. Furthermore, SyzParam proposes a novel mutation strategy, which leverages these relations and prioritizes parameter modification during related driver execution. Our evaluation demonstrates that SyzParam outperforms existing fuzzing works in driver code coverage and bug-detection capabilities. To date, we have identified 30 unique bugs in the latest kernel upstreams, with 20 confirmed and 14 patched into the mainline kernel, including 9 CVEs.

Title: Adaptive Spatiotemporal Augmentation for Improving Dynamic Graph Learning

• Authors: Xu Chu, Hanlin Xue, Bingce Wang, Xiaoyang Liu, Weiping Li, Tong Mo, Tuoyu Feng, Zhijie Tan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10010
• Pdf URL: https://arxiv.org/pdf/2501.10010
• Copy Paste: [[2501.10010]] Adaptive Spatiotemporal Augmentation for Improving Dynamic Graph Learning(https://arxiv.org/abs/2501.10010)
• Keywords: diffusion
• Abstract: Dynamic graph augmentation is used to improve the performance of dynamic GNNs. Most methods assume temporal locality, meaning that recent edges are more influential than earlier edges. However, for temporal changes in edges caused by random noise, overemphasizing recent edges while neglecting earlier ones may lead to the model capturing noise. To address this issue, we propose STAA (SpatioTemporal Activity-Aware Random Walk Diffusion). STAA identifies nodes likely to have noisy edges in spatiotemporal dimensions. Spatially, it analyzes critical topological positions through graph wavelet coefficients. Temporally, it analyzes edge evolution through graph wavelet coefficient change rates. Then, random walks are used to reduce the weights of noisy edges, deriving a diffusion matrix containing spatiotemporal information as an augmented adjacency matrix for dynamic GNN learning. Experiments on multiple datasets show that STAA outperforms other dynamic graph augmentation methods in node classification and link prediction tasks.

Title: Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions

• Authors: Zhijie Tan, Yuzhi Li, Shengwei Meng, Xiang Yuan, Weiping Li, Tong Mo, Bingce Wang, Xu Chu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10011
• Pdf URL: https://arxiv.org/pdf/2501.10011
• Copy Paste: [[2501.10011]] Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions(https://arxiv.org/abs/2501.10011)
• Keywords: large language model
• Abstract: Current popular Large Vision-Language Models (LVLMs) are suffering from Hallucinations on Object Attributes (HoOA), leading to incorrect determination of fine-grained attributes in the input images. Leveraging significant advancements in 3D generation from a single image, this paper proposes a novel method to mitigate HoOA in LVLMs. This method utilizes multiview images sampled from generated 3D representations as visual prompts for LVLMs, thereby providing more visual information from other viewpoints. Furthermore, we observe the input order of multiple multiview images significantly affects the performance of LVLMs. Consequently, we have devised Multiview Image Augmented VLM (MIAVLM), incorporating a Multiview Attributes Perceiver (MAP) submodule capable of simultaneously eliminating the influence of input image order and aligning visual information from multiview images with Large Language Models (LLMs). Besides, we designed and employed negative instructions to mitigate LVLMs' bias towards ``Yes" responses. Comprehensive experiments demonstrate the effectiveness of our method.

Title: CaFA: Cost-aware, Feasible Attacks With Database Constraints Against Neural Tabular Classifiers

• Authors: Matan Ben-Tov, Daniel Deutch, Nave Frost, Mahmood Sharif
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.10013
• Pdf URL: https://arxiv.org/pdf/2501.10013
• Copy Paste: [[2501.10013]] CaFA: Cost-aware, Feasible Attacks With Database Constraints Against Neural Tabular Classifiers(https://arxiv.org/abs/2501.10013)
• Keywords: attack, robust
• Abstract: This work presents CaFA, a system for Cost-aware Feasible Attacks for assessing the robustness of neural tabular classifiers against adversarial examples realizable in the problem space, while minimizing adversaries' effort. To this end, CaFA leverages TabPGD$-$an algorithm we set forth to generate adversarial perturbations suitable for tabular data$-$ and incorporates integrity constraints automatically mined by state-of-the-art database methods. After producing adversarial examples in the feature space via TabPGD, CaFA projects them on the mined constraints, leading, in turn, to better attack realizability. We tested CaFA with three datasets and two architectures and found, among others, that the constraints we use are of higher quality (measured via soundness and completeness) than ones employed in prior work. Moreover, CaFA achieves higher feasible success rates$-$i.e., it generates adversarial examples that are often misclassified while satisfying constraints$-$than prior attacks while simultaneously perturbing few features with lower magnitudes, thus saving effort and improving inconspicuousness. We open-source CaFA, hoping it will serve as a generic system enabling machine-learning engineers to assess their models' robustness against realizable attacks, thus advancing deployed models' trustworthiness.

Title: DiffuEraser: A Diffusion Model for Video Inpainting

• Authors: Xiaowen Li, Haolan Xue, Peiran Ren, Liefeng Bo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10018
• Pdf URL: https://arxiv.org/pdf/2501.10018
• Copy Paste: [[2501.10018]] DiffuEraser: A Diffusion Model for Video Inpainting(https://arxiv.org/abs/2501.10018)
• Keywords: diffusion, transformer, generative
• Abstract: Recent video inpainting algorithms integrate flow-based pixel propagation with transformer-based generation to leverage optical flow for restoring textures and objects using information from neighboring frames, while completing masked regions through visual Transformers. However, these approaches often encounter blurring and temporal inconsistencies when dealing with large masks, highlighting the need for models with enhanced generative capabilities. Recently, diffusion models have emerged as a prominent technique in image and video generation due to their impressive performance. In this paper, we introduce DiffuEraser, a video inpainting model based on stable diffusion, designed to fill masked regions with greater details and more coherent structures. We incorporate prior information to provide initialization and weak conditioning,which helps mitigate noisy artifacts and suppress hallucinations. Additionally, to improve temporal consistency during long-sequence inference, we expand the temporal receptive fields of both the prior model and DiffuEraser, and further enhance consistency by leveraging the temporal smoothing property of Video Diffusion Models. Experimental results demonstrate that our proposed method outperforms state-of-the-art techniques in both content completeness and temporal consistency while maintaining acceptable efficiency.

Title: X-Dyna: Expressive Dynamic Human Image Animation

• Authors: Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guoxian Song, Chao Wang, Yichun Shi, Zeyuan Chen, Shijie Zhou, Linjie Luo, Gordon Wetzstein, Mohammad Soleymani
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10021
• Pdf URL: https://arxiv.org/pdf/2501.10021
• Copy Paste: [[2501.10021]] X-Dyna: Expressive Dynamic Human Image Animation(https://arxiv.org/abs/2501.10021)
• Keywords: diffusion
• Abstract: We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at this https URL.

Title: Automatic Speech Recognition for Sanskrit with Transfer Learning

• Authors: Bidit Sadhukhan, Swami Punyeshwarananda
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10024
• Pdf URL: https://arxiv.org/pdf/2501.10024
• Copy Paste: [[2501.10024]] Automatic Speech Recognition for Sanskrit with Transfer Learning(https://arxiv.org/abs/2501.10024)
• Keywords: robust
• Abstract: Sanskrit, one of humanity's most ancient languages, has a vast collection of books and manuscripts on diverse topics that have been accumulated over millennia. However, its digital content (audio and text), which is vital for the training of AI systems, is profoundly limited. Furthermore, its intricate linguistics make it hard to develop robust NLP tools for wider accessibility. Given these constraints, we have developed an automatic speech recognition model for Sanskrit by employing transfer learning mechanism on OpenAI's Whisper model. After carefully optimising the hyper-parameters, we obtained promising results with our transfer-learned model achieving a word error rate of 15.42% on Vaksancayah dataset. An online demo of our model is made available for the use of public and to evaluate its performance firsthand thereby paving the way for improved accessibility and technological support for Sanskrit learning in the modern era.

Title: LWGANet: A Lightweight Group Attention Backbone for Remote Sensing Visual Tasks

• Authors: Wei Lu, Si-Bao Chen, Chris H. Q. Ding, Jin Tang, Bin Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10040
• Pdf URL: https://arxiv.org/pdf/2501.10040
• Copy Paste: [[2501.10040]] LWGANet: A Lightweight Group Attention Backbone for Remote Sensing Visual Tasks(https://arxiv.org/abs/2501.10040)
• Keywords: robust, extraction, segmentation
• Abstract: Remote sensing (RS) visual tasks have gained significant academic and practical importance. However, they encounter numerous challenges that hinder effective feature extraction, including the detection and recognition of multiple objects exhibiting substantial variations in scale within a single image. While prior dual-branch or multi-branch architectural strategies have been effective in managing these object variances, they have concurrently resulted in considerable increases in computational demands and parameter counts. Consequently, these architectures are rendered less viable for deployment on resource-constrained devices. Contemporary lightweight backbone networks, designed primarily for natural images, frequently encounter difficulties in effectively extracting features from multi-scale objects, which compromises their efficacy in RS visual tasks. This article introduces LWGANet, a specialized lightweight backbone network tailored for RS visual tasks, incorporating a novel lightweight group attention (LWGA) module designed to address these specific challenges. LWGA module, tailored for RS imagery, adeptly harnesses redundant features to extract a wide range of spatial information, from local to global scales, without introducing additional complexity or computational overhead. This facilitates precise feature extraction across multiple scales within an efficient this http URL was rigorously evaluated across twelve datasets, which span four crucial RS visual tasks: scene classification, oriented object detection, semantic segmentation, and change detection. The results confirm LWGANet's widespread applicability and its ability to maintain an optimal balance between high performance and low complexity, achieving SOTA results across diverse datasets. LWGANet emerged as a novel solution for resource-limited scenarios requiring robust RS image processing capabilities.

Title: Virtual Nodes Improve Long-term Traffic Prediction

• Authors: Xiaoyang Cao, Dingyi Zhuang, Jinhua Zhao, Shenhao Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10048
• Pdf URL: https://arxiv.org/pdf/2501.10048
• Copy Paste: [[2501.10048]] Virtual Nodes Improve Long-term Traffic Prediction(https://arxiv.org/abs/2501.10048)
• Keywords: explainability
• Abstract: Effective traffic prediction is a cornerstone of intelligent transportation systems, enabling precise forecasts of traffic flow, speed, and congestion. While traditional spatio-temporal graph neural networks (ST-GNNs) have achieved notable success in short-term traffic forecasting, their performance in long-term predictions remains limited. This challenge arises from over-squashing problem, where bottlenecks and limited receptive fields restrict information flow and hinder the modeling of global dependencies. To address these challenges, this study introduces a novel framework that incorporates virtual nodes, which are additional nodes added to the graph and connected to existing nodes, in order to aggregate information across the entire graph within a single GNN layer. Our proposed model incorporates virtual nodes by constructing a semi-adaptive adjacency matrix. This matrix integrates distance-based and adaptive adjacency matrices, allowing the model to leverage geographical information while also learning task-specific features from data. Experimental results demonstrate that the inclusion of virtual nodes significantly enhances long-term prediction accuracy while also improving layer-wise sensitivity to mitigate the over-squashing problem. Virtual nodes also offer enhanced explainability by focusing on key intersections and high-traffic areas, as shown by the visualization of their adjacency matrix weights on road network heat maps. Our advanced approach enhances the understanding and management of urban traffic systems, making it particularly well-suited for real-world applications.

Title: PandaSkill -- Player Performance and Skill Rating in Esports: Application to League of Legends

• Authors: Maxime De Bois, Flora Parmentier, Raphaël Puget, Matthew Tanti, Jordan Peltier
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10049
• Pdf URL: https://arxiv.org/pdf/2501.10049
• Copy Paste: [[2501.10049]] PandaSkill -- Player Performance and Skill Rating in Esports: Application to League of Legends(https://arxiv.org/abs/2501.10049)
• Keywords: fair
• Abstract: To take the esports scene to the next level, we introduce PandaSkill, a framework for assessing player performance and skill rating. Traditional rating systems like Elo and TrueSkill often overlook individual contributions and face challenges in professional esports due to limited game data and fragmented competitive scenes. PandaSkill leverages machine learning to estimate in-game player performance from individual player statistics. Each in-game role is modeled independently, ensuring a fair comparison between them. Then, using these performance scores, PandaSkill updates the player skill ratings using the Bayesian framework OpenSkill in a free-for-all setting. In this setting, skill ratings are updated solely based on performance scores rather than game outcomes, hightlighting individual contributions. To address the challenge of isolated rating pools that hinder cross-regional comparisons, PandaSkill introduces a dual-rating system that combines players' regional ratings with a meta-rating representing each region's overall skill level. Applying PandaSkill to five years of professional League of Legends matches worldwide, we show that our method produces skill ratings that better predict game outcomes and align more closely with expert opinions compared to existing methods.

Title: Accelerating Large Language Models through Partially Linear Feed-Forward Network

• Authors: Gansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, Haibo Chen
• Subjects: cs.LG, cs.AI, cs.PL
• Abstract URL: https://arxiv.org/abs/2501.10054
• Pdf URL: https://arxiv.org/pdf/2501.10054
• Copy Paste: [[2501.10054]] Accelerating Large Language Models through Partially Linear Feed-Forward Network(https://arxiv.org/abs/2501.10054)
• Keywords: large language model
• Abstract: Large language models (LLMs) demonstrate remarkable capabilities but face deployment challenges due to their massive parameter counts. While existing compression techniques like pruning can reduce model size, it leads to significant accuracy degradation under high compression ratios. We present a novel perspective inspired by constant folding in compiler optimization. Our approach enables parameter reduction by treating activation functions in LLMs as linear functions. However, recent LLMs use complex non-linear activations like GELU that prevent direct application of this technique. We propose TARDIS, which enables optimization of LLMs with non-linear activations by partially approximating them with linear functions in frequently occurring input ranges. For outlier inputs, TARDIS employs an online predictor to dynamically fall back to original computations. Our experiments demonstrate that TARDIS achieves 80% parameter reduction in feed-forward networks, while significantly outperforming state-of-the-art pruning methods Wanda and RIA with up to 65% higher accuracy. In practical deployments for a 7B model, TARDIS achieves 1.6x end-to-end inference speedup when integrated with the vLLM serving system, and 1.4x speedup with the widely adopted HuggingFace implementation, while incurring only a 10.9% accuracy trade-off.

Title: Introducing Post-Quantum algorithms in Open RAN interfaces

• Authors: Pedro Otero-García, Ana Fernández-Vilas, Manuel Fernández-Veiga
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.10060
• Pdf URL: https://arxiv.org/pdf/2501.10060
• Copy Paste: [[2501.10060]] Introducing Post-Quantum algorithms in Open RAN interfaces(https://arxiv.org/abs/2501.10060)
• Keywords: security, protect, attack
• Abstract: Nowadays, 5G architecture is characterized by the use of monolithic hardware, where the configuration of its elements is completely proprietary for each manufacturer. In recent years, as an alternative to this centralized architecture, a new model has emerged: the Open Radio Access Network (Open RAN). One of its main features has been the split of the Base Band Unit (BBU) into new simpler hardware with more specific functions approaching to a more modular model. As a consequence of this split, new interfaces appeared to connect these components that need to be protected. With the developments in the field of quantum computing, traditional protection mechanisms for this kind of interfaces may be deprecated in the near future. This security issue motivates this paper, which aims to study how to integrate post-quantum cryptography (PQC) mechanisms to current security standards, such as IPsec and MACsec. In addition, the proposal is also put into practice to compare the performance of traditional mechanisms with PQC implementations. This research shows that the new implementation does not reduce the performance of the aforementioned standards, while the security is reinforced against quantum attacks.

Title: One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression

• Authors: Keita Miwa, Kento Sasaki, Hidehisa Arai, Tsubasa Takahashi, Yu Yamaguchi
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10064
• Pdf URL: https://arxiv.org/pdf/2501.10064
• Copy Paste: [[2501.10064]] One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression(https://arxiv.org/abs/2501.10064)
• Keywords: segmentation
• Abstract: Current image tokenization methods require a large number of tokens to capture the information contained within images. Although the amount of information varies across images, most image tokenizers only support fixed-length tokenization, leading to inefficiency in token allocation. In this study, we introduce One-D-Piece, a discrete image tokenizer designed for variable-length tokenization, achieving quality-controllable mechanism. To enable variable compression rate, we introduce a simple but effective regularization mechanism named "Tail Token Drop" into discrete one-dimensional image tokenizers. This method encourages critical information to concentrate at the head of the token sequence, enabling support of variadic tokenization, while preserving state-of-the-art reconstruction quality. We evaluate our tokenizer across multiple reconstruction quality metrics and find that it delivers significantly better perceptual quality than existing quality-controllable compression methods, including JPEG and WebP, at smaller byte sizes. Furthermore, we assess our tokenizer on various downstream computer vision tasks, including image classification, object detection, semantic segmentation, and depth estimation, confirming its adaptability to numerous applications compared to other variable-rate methods. Our approach demonstrates the versatility of variable-length discrete image tokenization, establishing a new paradigm in both compression efficiency and reconstruction performance. Finally, we validate the effectiveness of tail token drop via detailed analysis of tokenizers.

Title: FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localization

• Authors: Zhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen, Ming Tang, Jinqiao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10067
• Pdf URL: https://arxiv.org/pdf/2501.10067
• Copy Paste: [[2501.10067]] FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localization(https://arxiv.org/abs/2501.10067)
• Keywords: large language model
• Abstract: Anomaly detection methods typically require extensive normal samples from the target class for training, limiting their applicability in scenarios that require rapid adaptation, such as cold start. Zero-shot and few-shot anomaly detection do not require labeled samples from the target class in advance, making them a promising research direction. Existing zero-shot and few-shot approaches often leverage powerful multimodal models to detect and localize anomalies by comparing image-text similarity. However, their handcrafted generic descriptions fail to capture the diverse range of anomalies that may emerge in different objects, and simple patch-level image-text matching often struggles to localize anomalous regions of varying shapes and sizes. To address these issues, this paper proposes the FiLo++ method, which consists of two key components. The first component, Fused Fine-Grained Descriptions (FusDes), utilizes large language models to generate anomaly descriptions for each object category, combines both fixed and learnable prompt templates and applies a runtime prompt filtering method, producing more accurate and task-specific textual descriptions. The second component, Deformable Localization (DefLoc), integrates the vision foundation model Grounding DINO with position-enhanced text descriptions and a Multi-scale Deformable Cross-modal Interaction (MDCI) module, enabling accurate localization of anomalies with various shapes and sizes. In addition, we design a position-enhanced patch matching approach to improve few-shot anomaly detection performance. Experiments on multiple datasets demonstrate that FiLo++ achieves significant performance improvements compared with existing methods. Code will be available at this https URL.

Title: CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment

• Authors: Yating Liu, Yujie Zhang, Ziyu Shan, Yiling Xu
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2501.10071
• Pdf URL: https://arxiv.org/pdf/2501.10071
• Copy Paste: [[2501.10071]] CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment(https://arxiv.org/abs/2501.10071)
• Keywords: extraction
• Abstract: In recent years, No-Reference Point Cloud Quality Assessment (NR-PCQA) research has achieved significant progress. However, existing methods mostly seek a direct mapping function from visual data to the Mean Opinion Score (MOS), which is contradictory to the mechanism of practical subjective evaluation. To address this, we propose a novel language-driven PCQA method named CLIP-PCQA. Considering that human beings prefer to describe visual quality using discrete quality descriptions (e.g., "excellent" and "poor") rather than specific scores, we adopt a retrieval-based mapping strategy to simulate the process of subjective assessment. More specifically, based on the philosophy of CLIP, we calculate the cosine similarity between the visual features and multiple textual features corresponding to different quality descriptions, in which process an effective contrastive loss and learnable prompts are introduced to enhance the feature extraction. Meanwhile, given the personal limitations and bias in subjective experiments, we further covert the feature similarities into probabilities and consider the Opinion Score Distribution (OSD) rather than a single MOS as the final target. Experimental results show that our CLIP-PCQA outperforms other State-Of-The-Art (SOTA) approaches.

Title: Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework

• Authors: Ali Can Karaca, M. Enes Ozelbas, Saadettin Berber, Orkhan Karimli, Turabi Yildirim, M. Fatih Amasyali
• Subjects: cs.CV, cs.AI, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2501.10075
• Pdf URL: https://arxiv.org/pdf/2501.10075
• Copy Paste: [[2501.10075]] Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework(https://arxiv.org/abs/2501.10075)
• Keywords: robust, segmentation
• Abstract: Remote sensing change captioning (RSICC) aims to describe changes between bitemporal images in natural language. Existing methods often fail under challenges like illumination differences, viewpoint changes, blur effects, leading to inaccuracies, especially in no-change regions. Moreover, the images acquired at different spatial resolutions and have registration errors tend to affect the captions. To address these issues, we introduce SECOND-CC, a novel RSICC dataset featuring high-resolution RGB image pairs, semantic segmentation maps, and diverse real-world scenarios. SECOND-CC which contains 6,041 pairs of bitemporal RS images and 30,205 sentences describing the differences between images. Additionally, we propose MModalCC, a multimodal framework that integrates semantic and visual data using advanced attention mechanisms, including Cross-Modal Cross Attention (CMCA) and Multimodal Gated Cross Attention (MGCA). Detailed ablation studies and attention visualizations further demonstrate its effectiveness and ability to address RSICC challenges. Comprehensive experiments show that MModalCC outperforms state-of-the-art RSICC methods, including RSICCformer, Chg2Cap, and PSNet with +4.6% improvement on BLEU4 score and +9.6% improvement on CIDEr score. We will make our dataset and codebase publicly available to facilitate future research at this https URL

Title: Few-shot Structure-Informed Machinery Part Segmentation with Foundation Models and Graph Neural Networks

• Authors: Michael Schwingshackl, Fabio Francisco Oberweger, Markus Murschitz
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10080
• Pdf URL: https://arxiv.org/pdf/2501.10080
• Copy Paste: [[2501.10080]] Few-shot Structure-Informed Machinery Part Segmentation with Foundation Models and Graph Neural Networks(https://arxiv.org/abs/2501.10080)
• Keywords: robust, segmentation
• Abstract: This paper proposes a novel approach to few-shot semantic segmentation for machinery with multiple parts that exhibit spatial and hierarchical relationships. Our method integrates the foundation models CLIPSeg and Segment Anything Model (SAM) with the interest point detector SuperPoint and a graph convolutional network (GCN) to accurately segment machinery parts. By providing 1 to 25 annotated samples, our model, evaluated on a purely synthetic dataset depicting a truck-mounted loading crane, achieves effective segmentation across various levels of detail. Training times are kept under five minutes on consumer GPUs. The model demonstrates robust generalization to real data, achieving a qualitative synthetic-to-real generalization with a $J\&F$ score of 92.2 on real data using 10 synthetic support samples. When benchmarked on the DAVIS 2017 dataset, it achieves a $J\&F$ score of 71.5 in semi-supervised video segmentation with three support samples. This method's fast training times and effective generalization to real data make it a valuable tool for autonomous systems interacting with machinery and infrastructure, and illustrate the potential of combined and orchestrated foundation models for few-shot segmentation tasks.

Title: Two-level Solar Irradiance Clustering with Season Identification: A Comparative Analysis

• Authors: Roshni Agrawal, Sivakumar Subramanian, Venkataramana Runkana
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10084
• Pdf URL: https://arxiv.org/pdf/2501.10084
• Copy Paste: [[2501.10084]] Two-level Solar Irradiance Clustering with Season Identification: A Comparative Analysis(https://arxiv.org/abs/2501.10084)
• Keywords: robust
• Abstract: Solar irradiance clustering can enhance solar power capacity planning and help improve forecasting models by identifying similar irradiance patterns influenced by seasonal and weather changes. In this study, we adopt an efficient two-level clustering approach to automatically identify seasons using the clear sky irradiance in first level and subsequently to identify daily cloud level as clear, cloudy and partly cloudy within each season in second level. In the second level of clustering, three methods are compared, namely, Daily Irradiance Index (DII or $\beta$), Euclidean Distance (ED), and Dynamic Time Warping (DTW) distance. The DII is computed as the ratio of time integral of measured irradiance to time integral of the clear sky irradiance. The identified clusters were compared quantitatively using established clustering metrics and qualitatively by comparing the mean irradiance profiles. The results clearly establish the superiority of the $\beta$-based clustering approach as the leader, setting a new benchmark for solar irradiance clustering studies. Moreover, $\beta$-based clustering remains effective even for annual data unlike the time-series methods which suffer significant performance degradation. Interestingly, contrary to expectations, ED-based clustering outperforms the more compute-intensive DTW distance-based clustering. The method has been rigorously validated using data from two distinct US locations, demonstrating robust scalability for larger datasets and potential applicability for other locations.

Title: A recursive Bayesian neural network for constitutive modeling of sands under monotonic loading

• Authors: Toiba Noor, Soban Nasir Lone, G. V. Ramana, Rajdip Nayek
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10088
• Pdf URL: https://arxiv.org/pdf/2501.10088
• Copy Paste: [[2501.10088]] A recursive Bayesian neural network for constitutive modeling of sands under monotonic loading(https://arxiv.org/abs/2501.10088)
• Keywords: robust
• Abstract: In geotechnical engineering, constitutive models play a crucial role in describing soil behavior under varying loading conditions. Data-driven deep learning (DL) models offer a promising alternative for developing predictive constitutive models. When prediction is the primary focus, quantifying the predictive uncertainty of a trained DL model and communicating this uncertainty to end users is crucial for informed decision-making. This study proposes a recursive Bayesian neural network (rBNN) framework, which builds upon recursive feedforward neural networks (rFFNNs) by introducing generalized Bayesian inference for uncertainty quantification. A significant contribution of this work is the incorporation of a sliding window approach in rFFNNs, allowing the models to effectively capture temporal dependencies across load steps. The rBNN extends this framework by treating model parameters as random variables, with their posterior distributions inferred using generalized variational inference. The proposed framework is validated on two datasets: (i) a numerically simulated consolidated drained (CD) triaxial dataset employing a hardening soil model and (ii) an experimental dataset comprising 28 CD triaxial tests on Baskarp sand. Comparative analyses with LSTM, Bi-LSTM, and GRU models demonstrate that the deterministic rFFNN achieves superior predictive accuracy, attributed to its transparent structure and sliding window design. While the rBNN marginally trails in accuracy for the experimental case, it provides robust confidence intervals, addressing data sparsity and measurement noise in experimental conditions. The study underscores the trade-offs between deterministic and probabilistic approaches and the potential of rBNNs for uncertainty-aware constitutive modeling.

Title: DiffVSR: Enhancing Real-World Video Super-Resolution with Diffusion Models for Advanced Visual Quality and Temporal Consistency

• Authors: Xiaohui Li, Yihao Liu, Shuo Cao, Ziyan Chen, Shaobin Zhuang, Xiangyu Chen, Yinan He, Yi Wang, Yu Qiao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10110
• Pdf URL: https://arxiv.org/pdf/2501.10110
• Copy Paste: [[2501.10110]] DiffVSR: Enhancing Real-World Video Super-Resolution with Diffusion Models for Advanced Visual Quality and Temporal Consistency(https://arxiv.org/abs/2501.10110)
• Keywords: robust, diffusion
• Abstract: Diffusion models have demonstrated exceptional capabilities in image generation and restoration, yet their application to video super-resolution faces significant challenges in maintaining both high fidelity and temporal consistency. We present DiffVSR, a diffusion-based framework for real-world video super-resolution that effectively addresses these challenges through key innovations. For intra-sequence coherence, we develop a multi-scale temporal attention module and temporal-enhanced VAE decoder that capture fine-grained motion details. To ensure inter-sequence stability, we introduce a noise rescheduling mechanism with an interweaved latent transition approach, which enhances temporal consistency without additional training overhead. We propose a progressive learning strategy that transitions from simple to complex degradations, enabling robust optimization despite limited high-quality video data. Extensive experiments demonstrate that DiffVSR delivers superior results in both visual quality and temporal consistency, setting a new performance standard in real-world video super-resolution.

Title: Spatio-temporal Graph Learning on Adaptive Mined Key Frames for High-performance Multi-Object Tracking

• Authors: Futian Wang, Fengxiang Liu, Xiao Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10129
• Pdf URL: https://arxiv.org/pdf/2501.10129
• Copy Paste: [[2501.10129]] Spatio-temporal Graph Learning on Adaptive Mined Key Frames for High-performance Multi-Object Tracking(https://arxiv.org/abs/2501.10129)
• Keywords: extraction
• Abstract: In the realm of multi-object tracking, the challenge of accurately capturing the spatial and temporal relationships between objects in video sequences remains a significant hurdle. This is further complicated by frequent occurrences of mutual occlusions among objects, which can lead to tracking errors and reduced performance in existing methods. Motivated by these challenges, we propose a novel adaptive key frame mining strategy that addresses the limitations of current tracking approaches. Specifically, we introduce a Key Frame Extraction (KFE) module that leverages reinforcement learning to adaptively segment videos, thereby guiding the tracker to exploit the intrinsic logic of the video content. This approach allows us to capture structured spatial relationships between different objects as well as the temporal relationships of objects across frames. To tackle the issue of object occlusions, we have developed an Intra-Frame Feature Fusion (IFF) module. Unlike traditional graph-based methods that primarily focus on inter-frame feature fusion, our IFF module uses a Graph Convolutional Network (GCN) to facilitate information exchange between the target and surrounding objects within a frame. This innovation significantly enhances target distinguishability and mitigates tracking loss and appearance similarity due to occlusions. By combining the strengths of both long and short trajectories and considering the spatial relationships between objects, our proposed tracker achieves impressive results on the MOT17 dataset, i.e., 68.6 HOTA, 81.0 IDF1, 66.6 AssA, and 893 IDS, proving its effectiveness and accuracy.

Title: ACE: Anatomically Consistent Embeddings in Composition and Decomposition

• Authors: Ziyu Zhou, Haozhe Luo, Mohammad Reza Hosseinzadeh Taher, Jiaxuan Pang, Xiaowei Ding, Michael Gotway, Jianming Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10131
• Pdf URL: https://arxiv.org/pdf/2501.10131
• Copy Paste: [[2501.10131]] ACE: Anatomically Consistent Embeddings in Composition and Decomposition(https://arxiv.org/abs/2501.10131)
• Keywords: robust
• Abstract: Medical images acquired from standardized protocols show consistent macroscopic or microscopic anatomical structures, and these structures consist of composable/decomposable organs and tissues, but existing self-supervised learning (SSL) methods do not appreciate such composable/decomposable structure attributes inherent to medical images. To overcome this limitation, this paper introduces a novel SSL approach called ACE to learn anatomically consistent embedding via composition and decomposition with two key branches: (1) global consistency, capturing discriminative macro-structures via extracting global features; (2) local consistency, learning fine-grained anatomical details from composable/decomposable patch features via corresponding matrix matching. Experimental results across 6 datasets 2 backbones, evaluated in few-shot learning, fine-tuning, and property analysis, show ACE's superior robustness, transferability, and clinical potential. The innovations of our ACE lie in grid-wise image cropping, leveraging the intrinsic properties of compositionality and decompositionality of medical images, bridging the semantic gap from high-level pathologies to low-level tissue anomalies, and providing a new SSL method for medical imaging.

Title: ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario

• Authors: Lucen Zhong, Zhengxiao Du, Xiaohan Zhang, Haiyi Hu, Jie Tang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.10132
• Pdf URL: https://arxiv.org/pdf/2501.10132
• Copy Paste: [[2501.10132]] ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario(https://arxiv.org/abs/2501.10132)
• Keywords: large language model
• Abstract: Enhancing large language models (LLMs) with real-time APIs can help generate more accurate and up-to-date responses. However, evaluating the function calling abilities of LLMs in real-world scenarios remains under-explored due to the complexity of data collection and evaluation. In this work, we introduce ComplexFuncBench, a benchmark for complex function calling across five real-world scenarios. Compared to existing benchmarks, ComplexFuncBench encompasses multi-step and constrained function calling, which requires long-parameter filing, parameter value reasoning, and 128k long context. Additionally, we propose an automatic framework, ComplexEval, for quantitatively evaluating complex function calling tasks. Through comprehensive experiments, we demonstrate the deficiencies of state-of-the-art LLMs in function calling and suggest future directions for optimizing these capabilities. The data and code are available at \url{this https URL}.

Title: Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation

• Authors: Tomasz Limisiewicz, David Mareček, Tomáš Musil
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10150
• Pdf URL: https://arxiv.org/pdf/2501.10150
• Copy Paste: [[2501.10150]] Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation(https://arxiv.org/abs/2501.10150)
• Keywords: robust, fair
• Abstract: Mitigation of biases, such as language models' reliance on gender stereotypes, is a crucial endeavor required for the creation of reliable and useful language technology. The crucial aspect of debiasing is to ensure that the models preserve their versatile capabilities, including their ability to solve language tasks and equitably represent various genders. To address this issue, we introduce a streamlined Dual Dabiasing Algorithm through Model Adaptation (2DAMA). Novel Dual Debiasing enables robust reduction of stereotypical bias while preserving desired factual gender information encoded by language models. We show that 2DAMA effectively reduces gender bias in English and is one of the first approaches facilitating the mitigation of stereotypical tendencies in translation. The proposed method's key advantage is the preservation of factual gender cues, which are useful in a wide range of natural language processing tasks.

Title: Region-wise stacking ensembles for estimating brain-age using MRI

• Authors: Georgios Antonopoulos, Shammi More, Simon B. Eickhoff, Federico Raimondo, Kaustubh R. Patil
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10153
• Pdf URL: https://arxiv.org/pdf/2501.10153
• Copy Paste: [[2501.10153]] Region-wise stacking ensembles for estimating brain-age using MRI(https://arxiv.org/abs/2501.10153)
• Keywords: privacy, robust, extraction, interpretability, generative
• Abstract: Predictive modeling using structural magnetic resonance imaging (MRI) data is a prominent approach to study brain-aging. Machine learning algorithms and feature extraction methods have been employed to improve predictions and explore healthy and accelerated aging e.g. neurodegenerative and psychiatric disorders. The high-dimensional MRI data pose challenges to building generalizable and interpretable models as well as for data privacy. Common practices are resampling or averaging voxels within predefined parcels, which reduces anatomical specificity and biological interpretability as voxels within a region may differently relate to aging. Effectively, naive fusion by averaging can result in information loss and reduced accuracy. We present a conceptually novel two-level stacking ensemble (SE) approach. The first level comprises regional models for predicting individuals' age based on voxel-wise information, fused by a second-level model yielding final predictions. Eight data fusion scenarios were explored using as input Gray matter volume (GMV) estimates from four datasets covering the adult lifespan. Performance, measured using mean absolute error (MAE), R2, correlation and prediction bias, showed that SE outperformed the region-wise averages. The best performance was obtained when first-level regional predictions were obtained as out-of-sample predictions on the application site with second-level models trained on independent and site-specific data (MAE=4.75 vs baseline regional mean GMV MAE=5.68). Performance improved as more datasets were used for training. First-level predictions showed improved and more robust aging signal providing new biological insights and enhanced data privacy. Overall, the SE improves accuracy compared to the baseline while preserving or enhancing data privacy.

Title: Michscan: Black-Box Neural Network Integrity Checking at Runtime Through Power Analysis

• Authors: Robi Paul, Michael Zuzak
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2501.10174
• Pdf URL: https://arxiv.org/pdf/2501.10174
• Copy Paste: [[2501.10174]] Michscan: Black-Box Neural Network Integrity Checking at Runtime Through Power Analysis(https://arxiv.org/abs/2501.10174)
• Keywords: security, protect, attack
• Abstract: As neural networks are increasingly used for critical decision-making tasks, the threat of integrity attacks, where an adversary maliciously alters a model, has become a significant security and safety concern. These concerns are compounded by the use of licensed models, where end-users purchase third-party models with only black-box access to protect model intellectual property (IP). In such scenarios, conventional approaches to verify model integrity require knowledge of model parameters or cooperative model owners. To address this challenge, we propose Michscan, a methodology leveraging power analysis to verify the integrity of black-box TinyML neural networks designed for resource-constrained devices. Michscan is based on the observation that modifications to model parameters impact the instantaneous power consumption of the device. We leverage this observation to develop a runtime model integrity-checking methodology that employs correlational power analysis using a golden template or signature to mathematically quantify the likelihood of model integrity violations at runtime through the Mann-Whitney U-Test. Michscan operates in a black-box environment and does not require a cooperative or trustworthy model owner. We evaluated Michscan using an STM32F303RC microcontroller with an ARM Cortex-M4 running four TinyML models in the presence of three model integrity violations. Michscan successfully detected all integrity violations at runtime using power data from five inferences. All detected violations had a negligible probability P < 10^(-5) of being produced from an unmodified model (i.e., false positive).

Title: Multi-stage Training of Bilingual Islamic LLM for Neural Passage Retrieval

• Authors: Vera Pavlova
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.10175
• Pdf URL: https://arxiv.org/pdf/2501.10175
• Copy Paste: [[2501.10175]] Multi-stage Training of Bilingual Islamic LLM for Neural Passage Retrieval(https://arxiv.org/abs/2501.10175)
• Keywords: robust, large language model
• Abstract: This study examines the use of Natural Language Processing (NLP) technology within the Islamic domain, focusing on developing an Islamic neural retrieval model. By leveraging the robust XLM-R model, the research employs a language reduction technique to create a lightweight bilingual large language model (LLM). Our approach for domain adaptation addresses the unique challenges faced in the Islamic domain, where substantial in-domain corpora exist only in Arabic while limited in other languages, including English. The work utilizes a multi-stage training process for retrieval models, incorporating large retrieval datasets, such as MS MARCO, and smaller, in-domain datasets to improve retrieval performance. Additionally, we have curated an in-domain retrieval dataset in English by employing data augmentation techniques and involving a reliable Islamic source. This approach enhances the domain-specific dataset for retrieval, leading to further performance gains. The findings suggest that combining domain adaptation and a multi-stage training method for the bilingual Islamic neural retrieval model enables it to outperform monolingual models on downstream retrieval tasks.

Title: Secure Semantic Communication With Homomorphic Encryption

• Authors: Rui Meng, Dayu Fan, Haixiao Gao, Yifan Yuan, Bizhu Wang, Xiaodong Xu, Mengying Sun, Chen Dong, Xiaofeng Tao, Ping Zhang, Dusit Niyato
• Subjects: cs.CR, eess.SP
• Abstract URL: https://arxiv.org/abs/2501.10182
• Pdf URL: https://arxiv.org/pdf/2501.10182
• Copy Paste: [[2501.10182]] Secure Semantic Communication With Homomorphic Encryption(https://arxiv.org/abs/2501.10182)
• Keywords: secure, security, privacy
• Abstract: In recent years, Semantic Communication (SemCom), which aims to achieve efficient and reliable transmission of meaning between agents, has garnered significant attention from both academia and industry. To ensure the security of communication systems, encryption techniques are employed to safeguard confidentiality and integrity. However, traditional cryptography-based encryption algorithms encounter obstacles when applied to SemCom. Motivated by this, this paper explores the feasibility of applying homomorphic encryption to SemCom. Initially, we review the encryption algorithms utilized in mobile communication systems and analyze the challenges associated with their application to SemCom. Subsequently, we employ scale-invariant feature transform to demonstrate that semantic features can be preserved in homomorphic encrypted ciphertext. Based on this finding, we propose a task-oriented SemCom scheme secured through homomorphic encryption. We design the privacy preserved deep joint source-channel coding (JSCC) encoder and decoder, and the frequency of key updates can be adjusted according to service requirements without compromising transmission performance. Simulation results validate that, when compared to plaintext images, the proposed scheme can achieve almost the same classification accuracy performance when dealing with homomorphic ciphertext images. Furthermore, we provide potential future research directions for homomorphic encrypted SemCom.

Title: CSHNet: A Novel Information Asymmetric Image Translation Method

• Authors: Xi Yang, Haoyuan Shi, Zihan Wang, Nannan Wang, Xinbo Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10197
• Pdf URL: https://arxiv.org/pdf/2501.10197
• Copy Paste: [[2501.10197]] CSHNet: A Novel Information Asymmetric Image Translation Method(https://arxiv.org/abs/2501.10197)
• Keywords: extraction, transformer
• Abstract: Despite advancements in cross-domain image translation, challenges persist in asymmetric tasks such as SAR-to-Optical and Sketch-to-Instance conversions, which involve transforming data from a less detailed domain into one with richer content. Traditional CNN-based methods are effective at capturing fine details but struggle with global structure, leading to unwanted merging of image regions. To address this, we propose the CNN-Swin Hybrid Network (CSHNet), which combines two key modules: Swin Embedded CNN (SEC) and CNN Embedded Swin (CES), forming the SEC-CES-Bottleneck (SCB). SEC leverages CNN's detailed feature extraction while integrating the Swin Transformer's structural bias. CES, in turn, preserves the Swin Transformer's global integrity, compensating for CNN's lack of focus on structure. Additionally, CSHNet includes two components designed to enhance cross-domain information retention: the Interactive Guided Connection (IGC), which enables dynamic information exchange between SEC and CES, and Adaptive Edge Perception Loss (AEPL), which maintains structural boundaries during translation. Experimental results show that CSHNet outperforms existing methods in both visual quality and performance metrics across scene-level and instance-level datasets. Our code is available at: this https URL.

Title: Adaptive Clustering for Efficient Phenotype Segmentation of UAV Hyperspectral Data

• Authors: Ciem Cornelissen, Sam Leroux, Pieter Simoens
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2501.10199
• Pdf URL: https://arxiv.org/pdf/2501.10199
• Copy Paste: [[2501.10199]] Adaptive Clustering for Efficient Phenotype Segmentation of UAV Hyperspectral Data(https://arxiv.org/abs/2501.10199)
• Keywords: segmentation
• Abstract: Unmanned Aerial Vehicles (UAVs) combined with Hyperspectral imaging (HSI) offer potential for environmental and agricultural applications by capturing detailed spectral information that enables the prediction of invisible features like biochemical leaf properties. However, the data-intensive nature of HSI poses challenges for remote devices, which have limited computational resources and storage. This paper introduces an Online Hyperspectral Simple Linear Iterative Clustering algorithm (OHSLIC) framework for real-time tree phenotype segmentation. OHSLIC reduces inherent noise and computational demands through adaptive incremental clustering and a lightweight neural network, which phenotypes trees using leaf contents such as chlorophyll, carotenoids, and anthocyanins. A hyperspectral dataset is created using a custom simulator that incorporates realistic leaf parameters, and light interactions. Results demonstrate that OHSLIC achieves superior regression accuracy and segmentation performance compared to pixel- or window-based methods while significantly reducing inference time. The method`s adaptive clustering enables dynamic trade-offs between computational efficiency and accuracy, paving the way for scalable edge-device deployment in HSI applications.

Title: Disharmony: Forensics using Reverse Lighting Harmonization

• Authors: Philip Wootaek Shin, Jack Sampson, Vijaykrishnan Narayanan, Andres Marquez, Mahantesh Halappanavar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10212
• Pdf URL: https://arxiv.org/pdf/2501.10212
• Copy Paste: [[2501.10212]] Disharmony: Forensics using Reverse Lighting Harmonization(https://arxiv.org/abs/2501.10212)
• Keywords: segmentation
• Abstract: Content generation and manipulation approaches based on deep learning methods have seen significant advancements, leading to an increased need for techniques to detect whether an image has been generated or edited. Another area of research focuses on the insertion and harmonization of objects within images. In this study, we explore the potential of using harmonization data in conjunction with a segmentation model to enhance the detection of edited image regions. These edits can be either manually crafted or generated using deep learning methods. Our findings demonstrate that this approach can effectively identify such edits. Existing forensic models often overlook the detection of harmonized objects in relation to the background, but our proposed Disharmony Network addresses this gap. By utilizing an aggregated dataset of harmonization techniques, our model outperforms existing forensic networks in identifying harmonized objects integrated into their backgrounds, and shows potential for detecting various forms of edits, including virtual try-on tasks.

Title: Temporal Graph MLP Mixer for Spatio-Temporal Forecasting

• Authors: Muhammad Bilal, Luis Carretero Lopez
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10214
• Pdf URL: https://arxiv.org/pdf/2501.10214
• Copy Paste: [[2501.10214]] Temporal Graph MLP Mixer for Spatio-Temporal Forecasting(https://arxiv.org/abs/2501.10214)
• Keywords: robust
• Abstract: Spatiotemporal forecasting is critical in applications such as traffic prediction, climate modeling, and environmental monitoring. However, the prevalence of missing data in real-world sensor networks significantly complicates this task. In this paper, we introduce the Temporal Graph MLP-Mixer (T-GMM), a novel architecture designed to address these challenges. The model combines node-level processing with patch-level subgraph encoding to capture localized spatial dependencies while leveraging a three-dimensional MLP-Mixer to handle temporal, spatial, and feature-based dependencies. Experiments on the AQI, ENGRAD, PV-US and METR-LA datasets demonstrate the model's ability to effectively forecast even in the presence of significant missing data. While not surpassing state-of-the-art models in all scenarios, the T-GMM exhibits strong learning capabilities, particularly in capturing long-range dependencies. These results highlight its potential for robust, scalable spatiotemporal forecasting.

Title: The Relevance of AWS Chronos: An Evaluation of Standard Methods for Time Series Forecasting with Limited Tuning

• Authors: Matthew Baron, Alex Karpinski
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10216
• Pdf URL: https://arxiv.org/pdf/2501.10216
• Copy Paste: [[2501.10216]] The Relevance of AWS Chronos: An Evaluation of Standard Methods for Time Series Forecasting with Limited Tuning(https://arxiv.org/abs/2501.10216)
• Keywords: transformer
• Abstract: A systematic comparison of Chronos, a transformer-based time series forecasting framework, against traditional approaches including ARIMA and Prophet. We evaluate these models across multiple time horizons and user categories, with a focus on the impact of historical context length. Our analysis reveals that while Chronos demonstrates superior performance for longer-term predictions and maintains accuracy with increased context, traditional models show significant degradation as context length increases. We find that prediction quality varies systematically between user classes, suggesting that underlying behavior patterns always influence model performance. This study provides a case for deploying Chronos in real-world applications where limited model tuning is feasible, especially in scenarios requiring longer prediction.

Title: Modelling Activity Scheduling Behaviour with Deep Generative Machine Learning

• Authors: Fred Shone, Tim Hillel
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10221
• Pdf URL: https://arxiv.org/pdf/2501.10221
• Copy Paste: [[2501.10221]] Modelling Activity Scheduling Behaviour with Deep Generative Machine Learning(https://arxiv.org/abs/2501.10221)
• Keywords: generative
• Abstract: We model human activity scheduling behaviour using a deep generative machine learning approach. Activity schedules, which represent the activities and associated travel behaviours of individuals, are a core component of many applied models in the transport, energy and epidemiology domains. Our data driven approach learns human preferences and scheduling logic without the need for complex interacting combinations of sub-models and custom-rules, this makes our approach significantly faster and simpler to operate that existing approaches. We find activity schedule data combines aspects of both continuous image data and also discrete text data, requiring novel approaches. We additionally contribute a novel schedule representation and comprehensive evaluation framework for generated schedules. Evaluation shows our approach is able to rapidly generate large, diverse and realistic synthetic samples of activity schedules.

Title: Challenges and recommendations for Electronic Health Records data extraction and preparation for dynamic prediction modelling in hospitalized patients -- a practical guide

• Authors: Elena Albu, Shan Gao, Pieter Stijnen, Frank E. Rademakers, Bas C T van Bussel, Taya Collyer, Tina Hernandez-Boussard, Laure Wynants, Ben Van Calster
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.10240
• Pdf URL: https://arxiv.org/pdf/2501.10240
• Copy Paste: [[2501.10240]] Challenges and recommendations for Electronic Health Records data extraction and preparation for dynamic prediction modelling in hospitalized patients -- a practical guide(https://arxiv.org/abs/2501.10240)
• Keywords: extraction
• Abstract: Dynamic predictive modeling using electronic health record (EHR) data has gained significant attention in recent years. The reliability and trustworthiness of such models depend heavily on the quality of the underlying data, which is largely determined by the stages preceding the model development: data extraction from EHR systems and data preparation. We list over forty challenges encountered during these stages and provide actionable recommendations for addressing them. These challenges are organized into four categories: cohort definition, outcome definition, feature engineering, and data cleaning. This list is designed to serve as a practical guide for data extraction engineers and researchers, supporting better practices and improving the quality and real-world applicability of dynamic prediction models in clinical settings.

Title: MutualForce: Mutual-Aware Enhancement for 4D Radar-LiDAR 3D Object Detection

• Authors: Xiangyuan Peng, Huawei Sun, Kay Bierzynski, Anton Fischbacher, Lorenzo Servadei, Robert Wille
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10266
• Pdf URL: https://arxiv.org/pdf/2501.10266
• Copy Paste: [[2501.10266]] MutualForce: Mutual-Aware Enhancement for 4D Radar-LiDAR 3D Object Detection(https://arxiv.org/abs/2501.10266)
• Keywords: robust, extraction
• Abstract: Radar and LiDAR have been widely used in autonomous driving as LiDAR provides rich structure information, and radar demonstrates high robustness under adverse weather. Recent studies highlight the effectiveness of fusing radar and LiDAR point clouds. However, challenges remain due to the modality misalignment and information loss during feature extractions. To address these issues, we propose a 4D radar-LiDAR framework to mutually enhance their representations. Initially, the indicative features from radar are utilized to guide both radar and LiDAR geometric feature learning. Subsequently, to mitigate their sparsity gap, the shape information from LiDAR is used to enrich radar BEV features. Extensive experiments on the View-of-Delft (VoD) dataset demonstrate our approach's superiority over existing methods, achieving the highest mAP of 71.76% across the entire area and 86.36\% within the driving corridor. Especially for cars, we improve the AP by 4.17% and 4.20% due to the strong indicative features and symmetric shapes.

Title: GSTAR: Gaussian Surface Tracking and Reconstruction

• Authors: Chengwei Zheng, Lixin Xue, Juan Zarate, Jie Song
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10283
• Pdf URL: https://arxiv.org/pdf/2501.10283
• Copy Paste: [[2501.10283]] GSTAR: Gaussian Surface Tracking and Reconstruction(https://arxiv.org/abs/2501.10283)
• Keywords: robust
• Abstract: 3D Gaussian Splatting techniques have enabled efficient photo-realistic rendering of static scenes. Recent works have extended these approaches to support surface reconstruction and tracking. However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians. In regions where topology changes, GSTAR adaptively unbinds Gaussians from the mesh, enabling accurate registration and the generation of new surfaces based on these optimized Gaussians. Additionally, we introduce a surface-based scene flow method that provides robust initialization for tracking between frames. Experiments demonstrate that our method effectively tracks and reconstructs dynamic surfaces, enabling a range of applications. Our project page with the code release is available at this https URL.

Title: HiMix: Reducing Computational Complexity in Large Vision-Language Models

• Authors: Xuange Zhang, Dengjie Li, Bo Liu, Zenghao Bao, Yao Zhou, Baisong Yang, Zhongying Liu, Yujie Zhong, Zheng Zhao, Tongtong Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10318
• Pdf URL: https://arxiv.org/pdf/2501.10318
• Copy Paste: [[2501.10318]] HiMix: Reducing Computational Complexity in Large Vision-Language Models(https://arxiv.org/abs/2501.10318)
• Keywords: large language model
• Abstract: Benefiting from recent advancements in large language models and modality alignment techniques, existing Large Vision-Language Models(LVLMs) have achieved prominent performance across a wide range of scenarios. However, the excessive computational complexity limits the widespread use of these models in practical applications. We argue that one main bottleneck in computational complexity is caused by the involvement of redundant vision sequences in model computation. This is inspired by a reassessment of the efficiency of vision and language information transmission in the language decoder of LVLMs. Then, we propose a novel hierarchical vision-language interaction mechanism called Hierarchical Vision injection for Mixture Attention (HiMix). In HiMix, only the language sequence undergoes full forward propagation, while the vision sequence interacts with the language at specific stages within each language decoder layer. It is striking that our approach significantly reduces computational complexity with minimal performance loss. Specifically, HiMix achieves a 10x reduction in the computational cost of the language decoder across multiple LVLM models while maintaining comparable performance. This highlights the advantages of our method, and we hope our research brings new perspectives to the field of vision-language understanding. Project Page: this https URL

Title: Natural Language Processing of Privacy Policies: A Survey

• Authors: Andrick Adhikari, Sanchari Das, Rinku Dewri
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.10319
• Pdf URL: https://arxiv.org/pdf/2501.10319
• Copy Paste: [[2501.10319]] Natural Language Processing of Privacy Policies: A Survey(https://arxiv.org/abs/2501.10319)
• Keywords: privacy, robust
• Abstract: Natural Language Processing (NLP) is an essential subset of artificial intelligence. It has become effective in several domains, such as healthcare, finance, and media, to identify perceptions, opinions, and misuse, among others. Privacy is no exception, and initiatives have been taken to address the challenges of usable privacy notifications to users with the help of NLP. To this aid, we conduct a literature review by analyzing 109 papers at the intersection of NLP and privacy policies. First, we provide a brief introduction to privacy policies and discuss various facets of associated problems, which necessitate the application of NLP to elevate the current state of privacy notices and disclosures to users. Subsequently, we a) provide an overview of the implementation and effectiveness of NLP approaches for better privacy policy communication; b) identify the methodologies that can be further enhanced to provide robust privacy policies; and c) identify the gaps in the current state-of-the-art research. Our systematic analysis reveals that several research papers focus on annotating and classifying privacy texts for analysis but need to adequately dwell on other aspects of NLP applications, such as summarization. More specifically, ample research opportunities exist in this domain, covering aspects such as corpus generation, summarization vectors, contextualized word embedding, identification of privacy-relevant statement categories, fine-grained classification, and domain-specific model tuning.

Title: Towards Human-Guided, Data-Centric LLM Co-Pilots

• Authors: Evgeny Saveliev, Jiashuo Liu, Nabeel Seedat, Anders Boyd, Mihaela van der Schaar
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2501.10321
• Pdf URL: https://arxiv.org/pdf/2501.10321
• Copy Paste: [[2501.10321]] Towards Human-Guided, Data-Centric LLM Co-Pilots(https://arxiv.org/abs/2501.10321)
• Keywords: robust
• Abstract: Machine learning (ML) has the potential to revolutionize healthcare, but its adoption is often hindered by the disconnect between the needs of domain experts and translating these needs into robust and valid ML tools. Despite recent advances in LLM-based co-pilots to democratize ML for non-technical domain experts, these systems remain predominantly focused on model-centric aspects while overlooking critical data-centric challenges. This limitation is problematic in complex real-world settings where raw data often contains complex issues, such as missing values, label noise, and domain-specific nuances requiring tailored handling. To address this we introduce CliMB-DC, a human-guided, data-centric framework for LLM co-pilots that combines advanced data-centric tools with LLM-driven reasoning to enable robust, context-aware data processing. At its core, CliMB-DC introduces a novel, multi-agent reasoning system that combines a strategic coordinator for dynamic planning and adaptation with a specialized worker agent for precise execution. Domain expertise is then systematically incorporated to guide the reasoning process using a human-in-the-loop approach. To guide development, we formalize a taxonomy of key data-centric challenges that co-pilots must address. Thereafter, to address the dimensions of the taxonomy, we integrate state-of-the-art data-centric tools into an extensible, open-source architecture, facilitating the addition of new tools from the research community. Empirically, using real-world healthcare datasets we demonstrate CliMB-DC's ability to transform uncurated datasets into ML-ready formats, significantly outperforming existing co-pilot baselines for handling data-centric challenges. CliMB-DC promises to empower domain experts from diverse domains -- healthcare, finance, social sciences and more -- to actively participate in driving real-world impact using ML.

Title: Hierarchical Autoregressive Transformers: Combining Byte-~and Word-Level Processing for Robust, Adaptable Language Models

• Authors: Pit Neitemeier, Björn Deiseroth, Constantin Eichenberg, Lukas Balles
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10322
• Pdf URL: https://arxiv.org/pdf/2501.10322
• Copy Paste: [[2501.10322]] Hierarchical Autoregressive Transformers: Combining Byte-~and Word-Level Processing for Robust, Adaptable Language Models(https://arxiv.org/abs/2501.10322)
• Keywords: robust, transformer
• Abstract: Tokenization is a fundamental step in natural language processing, breaking text into units that computational models can process. While learned subword tokenizers have become the de-facto standard, they present challenges such as large vocabularies, limited adaptability to new domains or languages, and sensitivity to spelling errors and variations. To overcome these limitations, we investigate a hierarchical architecture for autoregressive language modelling that combines character-level and word-level processing. It employs a lightweight character-level encoder to convert character sequences into word embeddings, which are then processed by a word-level backbone model and decoded back into characters via a compact character-level decoder. This method retains the sequence compression benefits of word-level tokenization without relying on a rigid, predefined vocabulary. We demonstrate, at scales up to 7 billion parameters, that hierarchical transformers match the downstream task performance of subword-tokenizer-based models while exhibiting significantly greater robustness to input perturbations. Additionally, during continued pretraining on an out-of-domain language, our model trains almost twice as fast, achieves superior performance on the target language, and retains more of its previously learned knowledge. Hierarchical transformers pave the way for NLP systems that are more robust, flexible, and generalizable across languages and domains.

Title: DiffStereo: High-Frequency Aware Diffusion Model for Stereo Image Restoration

• Authors: Huiyun Cao, Yuan Shi, Bin Xia, Xiaoyu Jin, Wenming Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10325
• Pdf URL: https://arxiv.org/pdf/2501.10325
• Copy Paste: [[2501.10325]] DiffStereo: High-Frequency Aware Diffusion Model for Stereo Image Restoration(https://arxiv.org/abs/2501.10325)
• Keywords: diffusion, transformer, generative
• Abstract: Diffusion models (DMs) have achieved promising performance in image restoration but haven't been explored for stereo images. The application of DM in stereo image restoration is confronted with a series of challenges. The need to reconstruct two images exacerbates DM's computational cost. Additionally, existing latent DMs usually focus on semantic information and remove high-frequency details as redundancy during latent compression, which is precisely what matters for image restoration. To address the above problems, we propose a high-frequency aware diffusion model, DiffStereo for stereo image restoration as the first attempt at DM in this domain. Specifically, DiffStereo first learns latent high-frequency representations (LHFR) of HQ images. DM is then trained in the learned space to estimate LHFR for stereo images, which are fused into a transformer-based stereo image restoration network providing beneficial high-frequency information of corresponding HQ images. The resolution of LHFR is kept the same as input images, which preserves the inherent texture from distortion. And the compression in channels alleviates the computational burden of DM. Furthermore, we devise a position encoding scheme when integrating the LHFR into the restoration network, enabling distinctive guidance in different depths of the restoration network. Comprehensive experiments verify that by combining generative DM and transformer, DiffStereo achieves both higher reconstruction accuracy and better perceptual quality on stereo super-resolution, deblurring, and low-light enhancement compared with state-of-the-art methods.

Title: BoK: Introducing Bag-of-Keywords Loss for Interpretable Dialogue Response Generation

• Authors: Suvodip Dey, Maunendra Sankar Desarkar
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.10328
• Pdf URL: https://arxiv.org/pdf/2501.10328
• Copy Paste: [[2501.10328]] BoK: Introducing Bag-of-Keywords Loss for Interpretable Dialogue Response Generation(https://arxiv.org/abs/2501.10328)
• Keywords: interpretability
• Abstract: The standard language modeling (LM) loss by itself has been shown to be inadequate for effective dialogue modeling. As a result, various training approaches, such as auxiliary loss functions and leveraging human feedback, are being adopted to enrich open-domain dialogue systems. One such auxiliary loss function is Bag-of-Words (BoW) loss, defined as the cross-entropy loss for predicting all the words/tokens of the next utterance. In this work, we propose a novel auxiliary loss named Bag-of-Keywords (BoK) loss to capture the central thought of the response through keyword prediction and leverage it to enhance the generation of meaningful and interpretable responses in open-domain dialogue systems. BoK loss upgrades the BoW loss by predicting only the keywords or critical words/tokens of the next utterance, intending to estimate the core idea rather than the entire response. We incorporate BoK loss in both encoder-decoder (T5) and decoder-only (DialoGPT) architecture and train the models to minimize the weighted sum of BoK and LM (BoK-LM) loss. We perform our experiments on two popular open-domain dialogue datasets, DailyDialog and Persona-Chat. We show that the inclusion of BoK loss improves the dialogue generation of backbone models while also enabling post-hoc interpretability. We also study the effectiveness of BoK-LM loss as a reference-free metric and observe comparable performance to the state-of-the-art metrics on various dialogue evaluation datasets.

Title: ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems

• Authors: Chao Feng, Nicolas Fazli Kohler, Alberto Huertas Celdran, Gerome Bovet, Burkhard Stiller
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10347
• Pdf URL: https://arxiv.org/pdf/2501.10347
• Copy Paste: [[2501.10347]] ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems(https://arxiv.org/abs/2501.10347)
• Keywords: federate
• Abstract: The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most existing research focuses on data heterogeneity (e.g., addressing non-IID data) rather than task heterogeneity, where clients solve fundamentally different tasks. Additionally, much of the work relies on centralized settings with a server managing the federation, leaving the more challenging domain of decentralized FMTL largely unexplored. Thus, this work bridges this gap by proposing ColNet, a framework designed for heterogeneous tasks in decentralized federated environments. ColNet divides models into the backbone and task-specific layers, forming groups of similar clients, with group leaders performing conflict-averse cross-group aggregation. A pool of experiments with different federations demonstrated ColNet outperforms the compared aggregation schemes in decentralized settings with label and task heterogeneity scenarios.

Title: Credit Risk Identification in Supply Chains Using Generative Adversarial Networks

• Authors: Zizhou Zhang, Xinshi Li, Yu Cheng, Zhenrui Chen, Qianying Liu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2501.10348
• Pdf URL: https://arxiv.org/pdf/2501.10348
• Copy Paste: [[2501.10348]] Credit Risk Identification in Supply Chains Using Generative Adversarial Networks(https://arxiv.org/abs/2501.10348)
• Keywords: robust, generative
• Abstract: Credit risk management within supply chains has emerged as a critical research area due to its significant implications for operational stability and financial sustainability. The intricate interdependencies among supply chain participants mean that credit risks can propagate across networks, with impacts varying by industry. This study explores the application of Generative Adversarial Networks (GANs) to enhance credit risk identification in supply chains. GANs enable the generation of synthetic credit risk scenarios, addressing challenges related to data scarcity and imbalanced datasets. By leveraging GAN-generated data, the model improves predictive accuracy while effectively capturing dynamic and temporal dependencies in supply chain data. The research focuses on three representative industries-manufacturing (steel), distribution (pharmaceuticals), and services (e-commerce) to assess industry-specific credit risk contagion. Experimental results demonstrate that the GAN-based model outperforms traditional methods, including logistic regression, decision trees, and neural networks, achieving superior accuracy, recall, and F1 scores. The findings underscore the potential of GANs in proactive risk management, offering robust tools for mitigating financial disruptions in supply chains. Future research could expand the model by incorporating external market factors and supplier relationships to further enhance predictive capabilities. Keywords- Generative Adversarial Networks (GANs); Supply Chain Risk; Credit Risk Identification; Machine Learning; Data Augmentation

Title: FaceXBench: Evaluating Multimodal LLMs on Face Understanding

• Authors: Kartik Narayan, Vibashan VS, Vishal M. Patel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2501.10360
• Pdf URL: https://arxiv.org/pdf/2501.10360
• Copy Paste: [[2501.10360]] FaceXBench: Evaluating Multimodal LLMs on Face Understanding(https://arxiv.org/abs/2501.10360)
• Keywords: fair, large language model
• Abstract: Multimodal Large Language Models (MLLMs) demonstrate impressive problem-solving abilities across a wide range of tasks and domains. However, their capacity for face understanding has not been systematically studied. To address this gap, we introduce FaceXBench, a comprehensive benchmark designed to evaluate MLLMs on complex face understanding tasks. FaceXBench includes 5,000 multimodal multiple-choice questions derived from 25 public datasets and a newly created dataset, FaceXAPI. These questions cover 14 tasks across 6 broad categories, assessing MLLMs' face understanding abilities in bias and fairness, face authentication, recognition, analysis, localization and tool retrieval. Using FaceXBench, we conduct an extensive evaluation of 26 open-source MLLMs alongside 2 proprietary models, revealing the unique challenges in complex face understanding tasks. We analyze the models across three evaluation settings: zero-shot, in-context task description, and chain-of-thought prompting. Our detailed analysis reveals that current MLLMs, including advanced models like GPT-4o, and GeminiPro 1.5, show significant room for improvement. We believe FaceXBench will be a crucial resource for developing MLLMs equipped to perform sophisticated face understanding. Code: this https URL