2025-04-09

Title: Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models

• Authors: Shiran Dudy, Thulasi Tholeti, Resmi Ramachandranpillai, Muhammad Ali, Toby Jia-Jun Li, Ricardo Baeza-Yates
• Subjects: cs.CL, cs.AI, cs.CY, cs.HC, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.05325
• Pdf URL: https://arxiv.org/pdf/2504.05325
• Copy Paste: [[2504.05325]] Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models(https://arxiv.org/abs/2504.05325)
• Keywords: large language model
• Abstract: Recent advancements in Large Language Models (LLMs) have made them a popular information-seeking tool among end users. However, the statistical training methods for LLMs have raised concerns about their representation of under-represented topics, potentially leading to biases that could influence real-world decisions and opportunities. These biases could have significant economic, social, and cultural impacts as LLMs become more prevalent, whether through direct interactions--such as when users engage with chatbots or automated assistants--or through their integration into third-party applications (as agents), where the models influence decision-making processes and functionalities behind the scenes. Our study examines the biases present in LLMs recommendations of U.S. cities and towns across three domains: relocation, tourism, and starting a business. We explore two key research questions: (i) How similar LLMs responses are, and (ii) How this similarity might favor areas with certain characteristics over others, introducing biases. We focus on the consistency of LLMs responses and their tendency to over-represent or under-represent specific locations. Our findings point to consistent demographic biases in these recommendations, which could perpetuate a ``rich-get-richer'' effect that widens existing economic disparities.

Title: Level Generation with Constrained Expressive Range

• Authors: Mahsa Bazzaz, Seth Cooper
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05334
• Pdf URL: https://arxiv.org/pdf/2504.05334
• Copy Paste: [[2504.05334]] Level Generation with Constrained Expressive Range(https://arxiv.org/abs/2504.05334)
• Keywords: generative
• Abstract: Expressive range analysis is a visualization-based technique used to evaluate the performance of generative models, particularly in game level generation. It typically employs two quantifiable metrics to position generated artifacts on a 2D plot, offering insight into how content is distributed within a defined metric space. In this work, we use the expressive range of a generator as the conceptual space of possible creations. Inspired by the quality diversity paradigm, we explore this space to generate levels. To do so, we use a constraint-based generator that systematically traverses and generates levels in this space. To train the constraint-based generator we use different tile patterns to learn from the initial example levels. We analyze how different patterns influence the exploration of the expressive range. Specifically, we compare the exploration process based on time, the number of successful and failed sample generations, and the overall interestingness of the generated levels. Unlike typical quality diversity approaches that rely on random generation and hope to get good coverage of the expressive range, this approach systematically traverses the grid ensuring more coverage. This helps create unique and interesting game levels while also improving our understanding of the generator's strengths and limitations.

Title: Impact of Price Inflation on Algorithmic Collusion Through Reinforcement Learning Agents

• Authors: Sebastián Tinoco, Andrés Abeliuk, Javier Ruiz del Solar
• Subjects: cs.LG, cs.GT
• Abstract URL: https://arxiv.org/abs/2504.05335
• Pdf URL: https://arxiv.org/pdf/2504.05335
• Copy Paste: [[2504.05335]] Impact of Price Inflation on Algorithmic Collusion Through Reinforcement Learning Agents(https://arxiv.org/abs/2504.05335)
• Keywords: robust
• Abstract: Algorithmic pricing is increasingly shaping market competition, raising concerns about its potential to compromise competitive dynamics. While prior work has shown that reinforcement learning (RL)-based pricing algorithms can lead to tacit collusion, less attention has been given to the role of macroeconomic factors in shaping these dynamics. This study examines the role of inflation in influencing algorithmic collusion within competitive markets. By incorporating inflation shocks into a RL-based pricing model, we analyze whether agents adapt their strategies to sustain supra-competitive profits. Our findings indicate that inflation reduces market competitiveness by fostering implicit coordination among agents, even without direct collusion. However, despite achieving sustained higher profitability, agents fail to develop robust punishment mechanisms to deter deviations from equilibrium strategies. The results suggest that inflation amplifies non-competitive dynamics in algorithmic pricing, emphasizing the need for regulatory oversight in markets where AI-driven pricing is prevalent.

Title: MASS: MoErging through Adaptive Subspace Selection

• Authors: Donato Crisostomi, Alessandro Zirilli, Antonio Andrea Gargiulo, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, Iacopo Masi, Emanuele Rodolà
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05342
• Pdf URL: https://arxiv.org/pdf/2504.05342
• Copy Paste: [[2504.05342]] MASS: MoErging through Adaptive Subspace Selection(https://arxiv.org/abs/2504.05342)
• Keywords: data-free
• Abstract: Model merging has recently emerged as a lightweight alternative to ensembling, combining multiple fine-tuned models into a single set of parameters with no additional training overhead. Yet, existing merging methods fall short of matching the full accuracy of separately fine-tuned endpoints. We present MASS (MoErging through Adaptive Subspace Selection), a new approach that closes this gap by unifying multiple fine-tuned models while retaining near state-of-the-art performance across tasks. Building on the low-rank decomposition of per-task updates, MASS stores only the most salient singular components for each task and merges them into a shared model. At inference time, a non-parametric, data-free router identifies which subspace (or combination thereof) best explains an input's intermediate features and activates the corresponding task-specific block. This procedure is fully training-free and introduces only a two-pass inference overhead plus a ~2 storage factor compared to a single pretrained model, irrespective of the number of tasks. We evaluate MASS on CLIP-based image classification using ViT-B-16, ViT-B-32 and ViT-L-14 for benchmarks of 8, 14 and 20 tasks respectively, establishing a new state-of-the-art. Most notably, MASS recovers up to ~98% of the average accuracy of individual fine-tuned models, making it a practical alternative to ensembling at a fraction of the storage cost.

Title: AROMA: Autonomous Rank-one Matrix Adaptation

• Authors: Hao Nan Sheng, Zhi-yong Wang, Mingrui Yang, Hing Cheung So
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05343
• Pdf URL: https://arxiv.org/pdf/2504.05343
• Copy Paste: [[2504.05343]] AROMA: Autonomous Rank-one Matrix Adaptation(https://arxiv.org/abs/2504.05343)
• Keywords: large language model
• Abstract: As large language models continue to grow in size, parameter-efficient fine-tuning has become increasingly crucial. While low-rank adaptation (LoRA) offers a solution through low-rank updates, its static rank allocation may yield suboptimal results. Adaptive low-rank adaptation (AdaLoRA) improves this with dynamic allocation but remains sensitive to initial and target rank configurations. We introduce AROMA, a framework that automatically constructs layer-specific updates by iteratively building up rank-one components with very few trainable parameters that gradually diminish to zero. Unlike existing methods that employ rank reduction mechanisms, AROMA introduces a dual-loop architecture for rank growth. The inner loop extracts information from each rank-one subspace, while the outer loop determines the number of rank-one subspaces, i.e., the optimal rank. We reset optimizer states to maintain subspace independence. AROMA significantly reduces parameters compared to LoRA and AdaLoRA while achieving superior performance on natural language understanding and commonsense reasoning tasks, offering new insights into adaptive parameter-efficient fine-tuning. The code is available at \href{this https URL}{AROMA}.

Title: ZeroED: Hybrid Zero-shot Error Detection through Large Language Model Reasoning

• Authors: Wei Ni, Kaihang Zhang, Xiaoye Miao, Xiangyu Zhao, Yangyang Wu, Yaoshu Wang, Jianwei Yin
• Subjects: cs.LG, cs.DB
• Abstract URL: https://arxiv.org/abs/2504.05345
• Pdf URL: https://arxiv.org/pdf/2504.05345
• Copy Paste: [[2504.05345]] ZeroED: Hybrid Zero-shot Error Detection through Large Language Model Reasoning(https://arxiv.org/abs/2504.05345)
• Keywords: large language model
• Abstract: Error detection (ED) in tabular data is crucial yet challenging due to diverse error types and the need for contextual understanding. Traditional ED methods often rely heavily on manual criteria and labels, making them labor-intensive. Large language models (LLM) can minimize human effort but struggle with errors requiring a comprehensive understanding of data context. In this paper, we propose ZeroED, a novel hybrid zero-shot error detection framework, which combines LLM reasoning ability with the manual label-based ED pipeline. ZeroED operates in four steps, i.e., feature representation, error labeling, training data construction, and detector training. Initially, to enhance error distinction, ZeroED generates rich data representations using error reason-aware binary features, pre-trained embeddings, and statistical features. Then, ZeroED employs LLM to label errors holistically through in-context learning, guided by a two-step reasoning process for detailed error detection guidelines. To reduce token costs, LLMs are applied only to representative data selected via clustering-based sampling. High-quality training data is constructed through in-cluster label propagation and LLM augmentation with verification. Finally, a classifier is trained to detect all errors. Extensive experiments on seven public datasets demonstrate that, ZeroED substantially outperforms state-of-the-art methods by a maximum 30% improvement in F1 score and up to 90% token cost reduction.

Title: Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression

• Authors: Ivan Ilin, Peter Richtarik
• Subjects: cs.LG, cs.AI, cs.CL, cs.PF
• Abstract URL: https://arxiv.org/abs/2504.05346
• Pdf URL: https://arxiv.org/pdf/2504.05346
• Copy Paste: [[2504.05346]] Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression(https://arxiv.org/abs/2504.05346)
• Keywords: large language model
• Abstract: This paper presents Thanos, a novel weight-pruning algorithm designed to reduce the memory footprint and enhance the computational efficiency of large language models (LLMs) by removing redundant weights while maintaining accuracy. Thanos introduces a block-wise pruning strategy with adaptive masks that dynamically adjust to weight importance, enabling flexible sparsity patterns and structured formats, such as $n:m$ sparsity, optimized for hardware acceleration. Experimental evaluations demonstrate that Thanos achieves state-of-the-art performance in structured pruning and outperforms existing methods in unstructured pruning. By providing an efficient and adaptable approach to model compression, Thanos offers a practical solution for deploying large models in resource-constrained environments.

Title: Achieving binary weight and activation for LLMs using Post-Training Quantization

• Authors: Siqing Song, Chuang Wang, Ruiqi Wang, Yi Yang, Xuyao Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05352
• Pdf URL: https://arxiv.org/pdf/2504.05352
• Copy Paste: [[2504.05352]] Achieving binary weight and activation for LLMs using Post-Training Quantization(https://arxiv.org/abs/2504.05352)
• Keywords: large language model
• Abstract: Quantizing large language models (LLMs) to 1-bit precision significantly reduces computational costs, but existing quantization techniques suffer from noticeable performance degradation when using weight and activation precisions below 4 bits (W4A4). In this paper, we propose a post-training quantization framework with W(1+1)A(1*4) configuration, where weights are quantized to 1 bit with an additional 1 bit for fine-grain grouping and activations are quantized to 1 bit with a 4-fold increase in the number of channels. For weight quantization, we propose utilizing Hessian-aware fine-grained grouping along with an EM-based quantization scheme. For activation quantization, we decompose INT4-quantized activations into a 4 * INT1 format equivalently and simultaneously smooth the scaling factors based on quantization errors, which further reduces the quantization errors in activations. Our method surpasses state-of-the-art (SOTA) LLM quantization baselines on W2A4 across multiple tasks, pushing the boundaries of existing LLM quantization methods toward fully binarized models.

Title: Deep Learning for Double Auction

• Authors: Jiayin Liu, Chenglong Zhang
• Subjects: cs.LG, cs.GT, econ.TH
• Abstract URL: https://arxiv.org/abs/2504.05355
• Pdf URL: https://arxiv.org/pdf/2504.05355
• Copy Paste: [[2504.05355]] Deep Learning for Double Auction(https://arxiv.org/abs/2504.05355)
• Keywords: transformer
• Abstract: Auctions are important mechanisms extensively implemented in various markets, e.g., search engines' keyword auctions, antique auctions, etc. Finding an optimal auction mechanism is extremely difficult due to the constraints of imperfect information, incentive compatibility (IC), and individual rationality (IR). In addition to the traditional economic methods, some recently attempted to find the optimal (single) auction using deep learning methods. Unlike those attempts focusing on single auctions, we develop deep learning methods for double auctions, where imperfect information exists on both the demand and supply sides. The previous attempts on single auction cannot directly apply to our contexts and those attempts additionally suffer from limited generalizability, inefficiency in ensuring the constraints, and learning fluctuations. We innovate in designing deep learning models for solving the more complex problem and additionally addressing the previous models' three limitations. Specifically, we achieve generalizability by leveraging a transformer-based architecture to model market participants as sequences for varying market sizes; we utilize the numerical features of the constraints and pre-treat them for a higher learning efficiency; we develop a gradient-conflict-elimination scheme to address the problem of learning fluctuation. Extensive experimental evaluations demonstrate the superiority of our approach to classical and machine learning baselines.

Title: DyTTP: Trajectory Prediction with Normalization-Free Transformers

• Authors: Yunxiang Liu, Hongkuo Niu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05356
• Pdf URL: https://arxiv.org/pdf/2504.05356
• Copy Paste: [[2504.05356]] DyTTP: Trajectory Prediction with Normalization-Free Transformers(https://arxiv.org/abs/2504.05356)
• Keywords: robust, transformer
• Abstract: Accurate trajectory prediction is a cornerstone for the safe operation of autonomous driving systems, where understanding the dynamic behavior of surrounding agents is crucial. Transformer-based architectures have demonstrated significant promise in capturing complex spatio-temporality dependencies. However, their reliance on normalization layers can lead to computation overhead and training instabilities. In this work, we present a two-fold approach to address these challenges. First, we integrate DynamicTanh (DyT), which is the latest method to promote transformers, into the backbone, replacing traditional layer normalization. This modification simplifies the network architecture and improves the stability of the inference. We are the first work to deploy the DyT to the trajectory prediction task. Complementing this, we employ a snapshot ensemble strategy to further boost trajectory prediction performance. Using cyclical learning rate scheduling, multiple model snapshots are captured during a single training run. These snapshots are then aggregated via simple averaging at inference time, allowing the model to benefit from diverse hypotheses without incurring substantial additional computational cost. Extensive experiments on Argoverse datasets demonstrate that our combined approach significantly improves prediction accuracy, inference speed and robustness in diverse driving scenarios. This work underscores the potential of normalization-free transformer designs augmented with lightweight ensemble techniques in advancing trajectory forecasting for autonomous vehicles.

Title: Handling Weather Uncertainty in Air Traffic Prediction through an Inverse Approach

• Authors: G. Lancia, D. Falanga, S. Alam, G. Lulli
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05366
• Pdf URL: https://arxiv.org/pdf/2504.05366
• Copy Paste: [[2504.05366]] Handling Weather Uncertainty in Air Traffic Prediction through an Inverse Approach(https://arxiv.org/abs/2504.05366)
• Keywords: robust, explainability
• Abstract: Adverse weather conditions, particularly convective phenomena, pose significant challenges to Air Traffic Management, often requiring real-time rerouting decisions that impact efficiency and safety. This study introduces a 3-D Gaussian Mixture Model to predict long lead-time flight trajectory changes, incorporating comprehensive weather and traffic data. Utilizing high-resolution meteorological datasets, including convective weather maps and wind data, alongside traffic records, the model demonstrates robust performance in forecasting reroutes up to 60 minutes. The novel 3-D Gaussian Mixture Model framework employs a probabilistic approach to capture uncertainty while providing accurate forecasts of altitude, latitude, and longitude. Extensive evaluation revealed a Mean Absolute Percentage Error below 0.02 across varying lead times, highlighting the model's accuracy and scalability. By integrating explainability techniques such as the Vanilla Gradient algorithm, the study provides insights into feature contributions, showing that they contribute to improving Air Traffic Management strategies to mitigate weather-induced disruptions.

Title: GARF: Learning Generalizable 3D Reassembly for Real-World Fractures

• Authors: Sihang Li, Zeyu Jiang, Grace Chen, Chenyang Xu, Siqi Tan, Xue Wang, Irving Fang, Kristof Zyskowski, Shannon P. McPherron, Radu Iovita, Chen Feng, Jing Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05400
• Pdf URL: https://arxiv.org/pdf/2504.05400
• Copy Paste: [[2504.05400]] GARF: Learning Generalizable 3D Reassembly for Real-World Fractures(https://arxiv.org/abs/2504.05400)
• Keywords: robust
• Abstract: 3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains. While large-scale synthetic datasets have fueled promising learning-based approaches, their generalizability to different domains is limited. Critically, it remains uncertain whether models trained on synthetic datasets can generalize to real-world fractures where breakage patterns are more complex. To bridge this gap, we propose GARF, a generalizable 3D reassembly framework for real-world fractures. GARF leverages fracture-aware pretraining to learn fracture features from individual fragments, with flow matching enabling precise 6-DoF alignments. At inference time, we introduce one-step preassembly, improving robustness to unseen objects and varying numbers of fractures. In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities, featuring real-world fracture types across ceramics, bones, eggshells, and lithics. Comprehensive experiments have shown our approach consistently outperforms state-of-the-art methods on both synthetic and real-world datasets, achieving 82.87\% lower rotation error and 25.15\% higher part accuracy. This sheds light on training on synthetic data to advance real-world 3D puzzle solving, demonstrating its strong generalization across unseen object shapes and diverse fracture types.

Title: Time-adaptive Video Frame Interpolation based on Residual Diffusion

• Authors: Victor Fonte Chavez, Claudia Esteves, Jean-Bernard Hayet
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05402
• Pdf URL: https://arxiv.org/pdf/2504.05402
• Copy Paste: [[2504.05402]] Time-adaptive Video Frame Interpolation based on Residual Diffusion(https://arxiv.org/abs/2504.05402)
• Keywords: diffusion
• Abstract: In this work, we propose a new diffusion-based method for video frame interpolation (VFI), in the context of traditional hand-made animation. We introduce three main contributions: The first is that we explicitly handle the interpolation time in our model, which we also re-estimate during the training process, to cope with the particularly large variations observed in the animation domain, compared to natural videos; The second is that we adapt and generalize a diffusion scheme called ResShift recently proposed in the super-resolution community to VFI, which allows us to perform a very low number of diffusion steps (in the order of 10) to produce our estimates; The third is that we leverage the stochastic nature of the diffusion process to provide a pixel-wise estimate of the uncertainty on the interpolated frame, which could be useful to anticipate where the model may be wrong. We provide extensive comparisons with respect to state-of-the-art models and show that our model outperforms these models on animation videos.

Title: SoK: Frontier AI's Impact on the Cybersecurity Landscape

• Authors: Wenbo Guo, Yujin Potter, Tianneng Shi, Zhun Wang, Andy Zhang, Dawn Song
• Subjects: cs.CR, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2504.05408
• Pdf URL: https://arxiv.org/pdf/2504.05408
• Copy Paste: [[2504.05408]] SoK: Frontier AI's Impact on the Cybersecurity Landscape(https://arxiv.org/abs/2504.05408)
• Keywords: secure, security, defense, attack
• Abstract: As frontier AI advances rapidly, understanding its impact on cybersecurity and inherent risks is essential to ensuring safe AI evolution (e.g., guiding risk mitigation and informing policymakers). While some studies review AI applications in cybersecurity, none of them comprehensively discuss AI's future impacts or provide concrete recommendations for navigating its safe and secure usage. This paper presents an in-depth analysis of frontier AI's impact on cybersecurity and establishes a systematic framework for risk assessment and mitigation. To this end, we first define and categorize the marginal risks of frontier AI in cybersecurity and then systemically analyze the current and future impacts of frontier AI in cybersecurity, qualitatively and quantitatively. We also discuss why frontier AI likely benefits attackers more than defenders in the short term from equivalence classes, asymmetry, and economic impact. Next, we explore frontier AI's impact on future software system development, including enabling complex hybrid systems while introducing new risks. Based on our findings, we provide security recommendations, including constructing fine-grained benchmarks for risk assessment, designing AI agents for defenses, building security mechanisms and provable defenses for hybrid systems, enhancing pre-deployment security testing and transparency, and strengthening defenses for users. Finally, we present long-term research questions essential for understanding AI's future impacts and unleashing its defensive capabilities.

Title: Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection

• Authors: Lingzhi Shen, Yunfei Long, Xiaohao Cai, Guanming Chen, Imran Razzak, Shoaib Jameel
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05411
• Pdf URL: https://arxiv.org/pdf/2504.05411
• Copy Paste: [[2504.05411]] Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection(https://arxiv.org/abs/2504.05411)
• Keywords: large language model
• Abstract: Personality detection automatically identifies an individual's personality from various data sources, such as social media texts. However, as the parameter scale of language models continues to grow, the computational cost becomes increasingly difficult to manage. Fine-tuning also grows more complex, making it harder to justify the effort and reliably predict outcomes. We introduce a novel parameter-efficient fine-tuning framework, PersLLM, to address these challenges. In PersLLM, a large language model (LLM) extracts high-dimensional representations from raw data and stores them in a dynamic memory layer. PersLLM then updates the downstream layers with a replaceable output network, enabling flexible adaptation to various personality detection scenarios. By storing the features in the memory layer, we eliminate the need for repeated complex computations by the LLM. Meanwhile, the lightweight output network serves as a proxy for evaluating the overall effectiveness of the framework, improving the predictability of results. Experimental results on key benchmark datasets like Kaggle and Pandora show that PersLLM significantly reduces computational cost while maintaining competitive performance and strong adaptability.

Title: EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representations

• Authors: Yue Yao, Mohamed-Khalil Bouzidi, Daniel Goehring, Joerg Reichardt
• Subjects: cs.CV, cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.05422
• Pdf URL: https://arxiv.org/pdf/2504.05422
• Copy Paste: [[2504.05422]] EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representations(https://arxiv.org/abs/2504.05422)
• Keywords: robust, diffusion, generative
• Abstract: As the prediction horizon increases, predicting the future evolution of traffic scenes becomes increasingly difficult due to the multi-modal nature of agent motion. Most state-of-the-art (SotA) prediction models primarily focus on forecasting the most likely future. However, for the safe operation of autonomous vehicles, it is equally important to cover the distribution for plausible motion alternatives. To address this, we introduce EP-Diffuser, a novel parameter-efficient diffusion-based generative model designed to capture the distribution of possible traffic scene evolutions. Conditioned on road layout and agent history, our model acts as a predictor and generates diverse, plausible scene continuations. We benchmark EP-Diffuser against two SotA models in terms of accuracy and plausibility of predictions on the Argoverse 2 dataset. Despite its significantly smaller model size, our approach achieves both highly accurate and plausible traffic scene predictions. We further evaluate model generalization ability in an out-of-distribution (OoD) test setting using Waymo Open dataset and show superior robustness of our approach. The code and model checkpoints can be found here: this https URL.

Title: GraphPINE: Graph Importance Propagation for Interpretable Drug Response Prediction

• Authors: Yoshitaka Inoue, Tianfan Fu, Augustin Luna
• Subjects: cs.LG, cs.AI, cs.CE, q-bio.GN, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2504.05454
• Pdf URL: https://arxiv.org/pdf/2504.05454
• Copy Paste: [[2504.05454]] GraphPINE: Graph Importance Propagation for Interpretable Drug Response Prediction(https://arxiv.org/abs/2504.05454)
• Keywords: explainability
• Abstract: Explainability is necessary for many tasks in biomedical research. Recent explainability methods have focused on attention, gradient, and Shapley value. These do not handle data with strong associated prior knowledge and fail to constrain explainability results based on known relationships between predictive features. We propose GraphPINE, a graph neural network (GNN) architecture leveraging domain-specific prior knowledge to initialize node importance optimized during training for drug response prediction. Typically, a manual post-prediction step examines literature (i.e., prior knowledge) to understand returned predictive features. While node importance can be obtained for gradient and attention after prediction, node importance from these methods lacks complementary prior knowledge; GraphPINE seeks to overcome this limitation. GraphPINE differs from other GNN gating methods by utilizing an LSTM-like sequential format. We introduce an importance propagation layer that unifies 1) updates for feature matrix and node importance and 2) uses GNN-based graph propagation of feature values. This initialization and updating mechanism allows for informed feature learning and improved graph representation. We apply GraphPINE to cancer drug response prediction using drug screening and gene data collected for over 5,000 gene nodes included in a gene-gene graph with a drug-target interaction (DTI) graph for initial importance. The gene-gene graph and DTIs were obtained from curated sources and weighted by article count discussing relationships between drugs and genes. GraphPINE achieves a PR-AUC of 0.894 and ROC-AUC of 0.796 across 952 drugs. Code is available at this https URL.

Title: Generative Adversarial Networks with Limited Data: A Survey and Benchmarking

• Authors: Omar De Mitri, Ruyu Wang, Marco F. Huber
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05456
• Pdf URL: https://arxiv.org/pdf/2504.05456
• Copy Paste: [[2504.05456]] Generative Adversarial Networks with Limited Data: A Survey and Benchmarking(https://arxiv.org/abs/2504.05456)
• Keywords: generative
• Abstract: Generative Adversarial Networks (GANs) have shown impressive results in various image synthesis tasks. Vast studies have demonstrated that GANs are more powerful in feature and expression learning compared to other generative models and their latent space encodes rich semantic information. However, the tremendous performance of GANs heavily relies on the access to large-scale training data and deteriorates rapidly when the amount of data is limited. This paper aims to provide an overview of GANs, its variants and applications in various vision tasks, focusing on addressing the limited data issue. We analyze state-of-the-art GANs in limited data regime with designed experiments, along with presenting various methods attempt to tackle this problem from different perspectives. Finally, we further elaborate on remaining challenges and trends for future research.

Title: Studying Image Diffusion Features for Zero-Shot Video Object Segmentation

• Authors: Thanos Delatolas, Vicky Kalogeiton, Dim P. Papadopoulos
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05468
• Pdf URL: https://arxiv.org/pdf/2504.05468
• Copy Paste: [[2504.05468]] Studying Image Diffusion Features for Zero-Shot Video Object Segmentation(https://arxiv.org/abs/2504.05468)
• Keywords: extraction, diffusion, segmentation
• Abstract: This paper investigates the use of large-scale diffusion models for Zero-Shot Video Object Segmentation (ZS-VOS) without fine-tuning on video data or training on any image segmentation data. While diffusion models have demonstrated strong visual representations across various tasks, their direct application to ZS-VOS remains underexplored. Our goal is to find the optimal feature extraction process for ZS-VOS by identifying the most suitable time step and layer from which to extract features. We further analyze the affinity of these features and observe a strong correlation with point correspondences. Through extensive experiments on DAVIS-17 and MOSE, we find that diffusion models trained on ImageNet outperform those trained on larger, more diverse datasets for ZS-VOS. Additionally, we highlight the importance of point correspondences in achieving high segmentation accuracy, and we yield state-of-the-art results in ZS-VOS. Finally, our approach performs on par with models trained on expensive image segmentation datasets.

Title: GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases

• Authors: Alfred Clemedtson, Borun Shi
• Subjects: cs.LG, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2504.05478
• Pdf URL: https://arxiv.org/pdf/2504.05478
• Copy Paste: [[2504.05478]] GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases(https://arxiv.org/abs/2504.05478)
• Keywords: large language model
• Abstract: Large language models have shown remarkable language processing and reasoning ability but are prone to hallucinate when asked about private data. Retrieval-augmented generation (RAG) retrieves relevant data that fit into an LLM's context window and prompts the LLM for an answer. GraphRAG extends this approach to structured Knowledge Graphs (KGs) and questions regarding entities multiple hops away. The majority of recent GraphRAG methods either overlook the retrieval step or have ad hoc retrieval processes that are abstract or inefficient. This prevents them from being adopted when the KGs are stored in graph databases supporting graph query languages. In this work, we present GraphRAFT, a retrieve-and-reason framework that finetunes LLMs to generate provably correct Cypher queries to retrieve high-quality subgraph contexts and produce accurate answers. Our method is the first such solution that can be taken off-the-shelf and used on KGs stored in native graph DBs. Benchmarks suggest that our method is sample-efficient and scales with the availability of training data. Our method achieves significantly better results than all state-of-the-art models across all four standard metrics on two challenging Q\&As on large text-attributed KGs.

Title: Secure Diagnostics: Adversarial Robustness Meets Clinical Interpretability

• Authors: Mohammad Hossein Najafi, Mohammad Morsali, Mohammadreza Pashanejad, Saman Soleimani Roudi, Mohammad Norouzi, Saeed Bagheri Shouraki
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05483
• Pdf URL: https://arxiv.org/pdf/2504.05483
• Copy Paste: [[2504.05483]] Secure Diagnostics: Adversarial Robustness Meets Clinical Interpretability(https://arxiv.org/abs/2504.05483)
• Keywords: secure, attack, robust, interpretability
• Abstract: Deep neural networks for medical image classification often fail to generalize consistently in clinical practice due to violations of the i.i.d. assumption and opaque decision-making. This paper examines interpretability in deep neural networks fine-tuned for fracture detection by evaluating model performance against adversarial attack and comparing interpretability methods to fracture regions annotated by an orthopedic surgeon. Our findings prove that robust models yield explanations more aligned with clinically meaningful areas, indicating that robustness encourages anatomically relevant feature prioritization. We emphasize the value of interpretability for facilitating human-AI collaboration, in which models serve as assistants under a human-in-the-loop paradigm: clinically plausible explanations foster trust, enable error correction, and discourage reliance on AI for high-stakes decisions. This paper investigates robustness and interpretability as complementary benchmarks for bridging the gap between benchmark performance and safe, actionable clinical deployment.

Title: Towards Zero Trust Security in Connected Vehicles: A Comprehensive Survey

• Authors: Malak Annabi, Abdelhafid Zeroual, Nadhir Messai
• Subjects: cs.CR, eess.SY
• Abstract URL: https://arxiv.org/abs/2504.05485
• Pdf URL: https://arxiv.org/pdf/2504.05485
• Copy Paste: [[2504.05485]] Towards Zero Trust Security in Connected Vehicles: A Comprehensive Survey(https://arxiv.org/abs/2504.05485)
• Keywords: security, attack
• Abstract: Zero Trust is the new cybersecurity model that challenges the traditional one by promoting continuous verification of users, devices, and applications, whatever their position or origin. This model is critical for reducing the attack surface and preventing lateral movement without relying on implicit trust. Adopting the zero trust principle in Intelligent Transportation Systems (ITS), especially in the context of connected vehicles (CVs), presents an adequate solution in the face of increasing cyber threats, thereby strengthening the ITS environment. This paper offers an understanding of Zero Trust security through a comprehensive review of existing literature, principles, and challenges. It specifically examines its applications in emerging technologies, particularly within connected vehicles, addressing potential issues and cyber threats faced by CVs. Inclusion/exclusion criteria for the systematic literature review were planned alongside a bibliometric analysis. Moreover, keyword co-occurrence analysis was done, which indicates trends and general themes for the Zero Trust model, Zero Trust implementation, and Zero Trust application. Furthermore, the paper explores various ZT models proposed in the literature for connected vehicles, shedding light on the challenges associated with their integration into CV systems. Future directions of this research will focus on incorporating Zero Trust principles within Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communication paradigms. This initiative intends to enhance the security posture and safety protocols within interconnected vehicular networks. The proposed research seeks to address the unique cybersecurity vulnerabilities inherent in the highly dynamic nature of vehicular communication systems.

Title: REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding

• Authors: Sakib Reza, Xiyun Song, Heather Yu, Zongfang Lin, Mohsen Moghaddam, Octavia Camps
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05491
• Pdf URL: https://arxiv.org/pdf/2504.05491
• Copy Paste: [[2504.05491]] REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding(https://arxiv.org/abs/2504.05491)
• Keywords: large language model
• Abstract: Integrating vision models into large language models (LLMs) has sparked significant interest in creating vision-language foundation models, especially for video understanding. Recent methods often utilize memory banks to handle untrimmed videos for video-level understanding. However, they typically compress visual memory using similarity-based greedy approaches, which can overlook the contextual importance of individual tokens. To address this, we introduce an efficient LLM adapter designed for video-level understanding of untrimmed videos that prioritizes the contextual relevance of spatio-temporal tokens. Our framework leverages scorer networks to selectively compress the visual memory bank and filter spatial tokens based on relevance, using a differentiable Top-K operator for end-to-end training. Across three key video-level understanding tasks$\unicode{x2013}$ untrimmed video classification, video question answering, and video captioning$\unicode{x2013}$our method achieves competitive or superior results on four large-scale datasets while reducing computational overhead by up to 34%. The code will be available soon on GitHub.

Title: A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models

• Authors: Atilla Kaan Alkan, Shashwat Sourav, Maja Jablonska, Simone Astarita, Rishabh Chakrabarty, Nikhil Garuda, Pranav Khetarpal, Maciej Pióro, Dimitrios Tanoglidis, Kartheik G. Iyer, Mugdha S. Polimera, Michael J. Smith, Tirthankar Ghosal, Marc Huertas-Company, Sandor Kruk, Kevin Schawinski, Ioana Ciucă
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05496
• Pdf URL: https://arxiv.org/pdf/2504.05496
• Copy Paste: [[2504.05496]] A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models(https://arxiv.org/abs/2504.05496)
• Keywords: large language model
• Abstract: Hypothesis generation is a fundamental step in scientific discovery, yet it is increasingly challenged by information overload and disciplinary fragmentation. Recent advances in Large Language Models (LLMs) have sparked growing interest in their potential to enhance and automate this process. This paper presents a comprehensive survey of hypothesis generation with LLMs by (i) reviewing existing methods, from simple prompting techniques to more complex frameworks, and proposing a taxonomy that categorizes these approaches; (ii) analyzing techniques for improving hypothesis quality, such as novelty boosting and structured reasoning; (iii) providing an overview of evaluation strategies; and (iv) discussing key challenges and future directions, including multimodal integration and human-AI collaboration. Our survey aims to serve as a reference for researchers exploring LLMs for hypothesis generation.

Title: SelfMAD: Enhancing Generalization and Robustness in Morphing Attack Detection via Self-Supervised Learning

• Authors: Marija Ivanovska, Leon Todorov, Naser Damer, Deepak Kumar Jain, Peter Peer, Vitomir Štruc
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05504
• Pdf URL: https://arxiv.org/pdf/2504.05504
• Copy Paste: [[2504.05504]] SelfMAD: Enhancing Generalization and Robustness in Morphing Attack Detection via Self-Supervised Learning(https://arxiv.org/abs/2504.05504)
• Keywords: attack, robust, generative
• Abstract: With the continuous advancement of generative models, face morphing attacks have become a significant challenge for existing face verification systems due to their potential use in identity fraud and other malicious activities. Contemporary Morphing Attack Detection (MAD) approaches frequently rely on supervised, discriminative models trained on examples of bona fide and morphed images. These models typically perform well with morphs generated with techniques seen during training, but often lead to sub-optimal performance when subjected to novel unseen morphing techniques. While unsupervised models have been shown to perform better in terms of generalizability, they typically result in higher error rates, as they struggle to effectively capture features of subtle artifacts. To address these shortcomings, we present SelfMAD, a novel self-supervised approach that simulates general morphing attack artifacts, allowing classifiers to learn generic and robust decision boundaries without overfitting to the specific artifacts induced by particular face morphing methods. Through extensive experiments on widely used datasets, we demonstrate that SelfMAD significantly outperforms current state-of-the-art MADs, reducing the detection error by more than 64% in terms of EER when compared to the strongest unsupervised competitor, and by more than 66%, when compared to the best performing discriminative MAD model, tested in cross-morph settings. The source code for SelfMAD is available at this https URL.

Title: Secure Smart Contract with Control Flow Integrity

• Authors: Zhiyang Chen, Sidi Mohamed Beillahi, Pasha Barahimi, Cyrus Minwalla, Han Du, Andreas Veneris, Fan Long
• Subjects: cs.CR, cs.SE
• Abstract URL: https://arxiv.org/abs/2504.05509
• Pdf URL: https://arxiv.org/pdf/2504.05509
• Copy Paste: [[2504.05509]] Secure Smart Contract with Control Flow Integrity(https://arxiv.org/abs/2504.05509)
• Keywords: secure, security, protect, attack
• Abstract: Smart contracts power decentralized financial (DeFi) services but are vulnerable to complex security exploits that can lead to significant financial losses. Existing security measures often fail to adequately protect these contracts due to the composability of DeFi protocols and the increasing sophistication of attacks. Through a large-scale empirical study of historical transactions from the 30 hacked DeFi protocols, we discovered that while benign transactions typically exhibit a limited number of unique control flows, in stark contrast, attack transactions consistently introduce novel, previously unobserved control flows. Building on these insights, we developed CrossGuard, a novel framework that enforces control flow integrity in real-time to secure smart contracts. Crucially, CrossGuard does not require prior knowledge of specific hacks; instead, it dynamically enforces control flow whitelisting policies and applies simplification heuristics at runtime. This approach monitors and prevents potential attacks by reverting all transactions that do not adhere to the established control flow whitelisting rules. Our evaluation demonstrates that CrossGuard effectively blocks 28 of the 30 analyzed attacks when configured only once prior to contract deployment, maintaining a low false positive rate of 0.28% and minimal additional gas costs. These results underscore the efficacy of applying control flow integrity to smart contracts, significantly enhancing security beyond traditional methods and addressing the evolving threat landscape in the DeFi ecosystem.

Title: Efficient Reinforcement Finetuning via Adaptive Curriculum Learning

• Authors: Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, Jieyu Zhao
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05520
• Pdf URL: https://arxiv.org/pdf/2504.05520
• Copy Paste: [[2504.05520]] Efficient Reinforcement Finetuning via Adaptive Curriculum Learning(https://arxiv.org/abs/2504.05520)
• Keywords: large language model
• Abstract: Reinforcement finetuning (RFT) has shown great potential for enhancing the mathematical reasoning capabilities of large language models (LLMs), but it is often sample- and compute-inefficient, requiring extensive training. In this work, we introduce AdaRFT (Adaptive Curriculum Reinforcement Finetuning), a method that significantly improves both the efficiency and final accuracy of RFT through adaptive curriculum learning. AdaRFT dynamically adjusts the difficulty of training problems based on the model's recent reward signals, ensuring that the model consistently trains on tasks that are challenging but solvable. This adaptive sampling strategy accelerates learning by maintaining an optimal difficulty range, avoiding wasted computation on problems that are too easy or too hard. AdaRFT requires only a lightweight extension to standard RFT algorithms like Proximal Policy Optimization (PPO), without modifying the reward function or model architecture. Experiments on competition-level math datasets-including AMC, AIME, and IMO-style problems-demonstrate that AdaRFT significantly improves both training efficiency and reasoning performance. We evaluate AdaRFT across multiple data distributions and model sizes, showing that it reduces the number of training steps by up to 2x and improves accuracy by a considerable margin, offering a more scalable and effective RFT framework.

Title: Pretraining Language Models for Diachronic Linguistic Change Discovery

• Authors: Elisabeth Fittschen, Sabrina Li, Tom Lippincott, Leshem Choshsem, Craig Messner
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05523
• Pdf URL: https://arxiv.org/pdf/2504.05523
• Copy Paste: [[2504.05523]] Pretraining Language Models for Diachronic Linguistic Change Discovery(https://arxiv.org/abs/2504.05523)
• Keywords: large language model
• Abstract: Large language models (LLMs) have shown potential as tools for scientific discovery. This has engendered growing interest in their use in humanistic disciplines, such as historical linguistics and literary studies. These fields often construct arguments on the basis of delineations like genre, or more inflexibly, time period. Although efforts have been made to restrict inference to specific domains via fine-tuning or model editing, we posit that the only true guarantee is domain-restricted pretraining -- typically, a data- and compute-expensive proposition. We show that efficient pretraining techniques can produce useful models over corpora too large for easy manual inspection but too small for "typical" LLM approaches. We employ a novel date-attribution pipeline in order to obtain a temporally-segmented dataset of five 10-million-word slices. We train two corresponding five-model batteries over these corpus segments, efficient pretraining and Llama3-8B parameter efficiently finetuned. We find that the pretrained models are faster to train than the finetuned baselines and that they better respect the historical divisions of our corpus. Emphasizing speed and precision over a-historical comprehensiveness enables a number of novel approaches to hypothesis discovery and testing in our target fields. Taking up diachronic linguistics as a testbed, we show that our method enables the detection of a diverse set of phenomena, including en masse lexical change, non-lexical (grammatical and morphological) change, and word sense introduction/obsolescence. We provide a ready-to-use pipeline that allows extension of our approach to other target fields with only minimal adaptation.

Title: Bridging Industrial Expertise and XR with LLM-Powered Conversational Agents

• Authors: Despina Tomkou, George Fatouros, Andreas Andreou, Georgios Makridis, Fotis Liarokapis, Dimitrios Dardanis, Athanasios Kiourtis, John Soldatos, Dimosthenis Kyriazis
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05527
• Pdf URL: https://arxiv.org/pdf/2504.05527
• Copy Paste: [[2504.05527]] Bridging Industrial Expertise and XR with LLM-Powered Conversational Agents(https://arxiv.org/abs/2504.05527)
• Keywords: large language model
• Abstract: This paper introduces a novel integration of Retrieval-Augmented Generation (RAG) enhanced Large Language Models (LLMs) with Extended Reality (XR) technologies to address knowledge transfer challenges in industrial environments. The proposed system embeds domain-specific industrial knowledge into XR environments through a natural language interface, enabling hands-free, context-aware expert guidance for workers. We present the architecture of the proposed system consisting of an LLM Chat Engine with dynamic tool orchestration and an XR application featuring voice-driven interaction. Performance evaluation of various chunking strategies, embedding models, and vector databases reveals that semantic chunking, balanced embedding models, and efficient vector stores deliver optimal performance for industrial knowledge retrieval. The system's potential is demonstrated through early implementation in multiple industrial use cases, including robotic assembly, smart infrastructure maintenance, and aerospace component servicing. Results indicate potential for enhancing training efficiency, remote assistance capabilities, and operational guidance in alignment with Industry 5.0's human-centric and resilient approach to industrial development.

Title: COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values

• Authors: M-A-P Team, Siwei Wu, Jincheng Ren, Xinrun Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zenith Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05535
• Pdf URL: https://arxiv.org/pdf/2504.05535
• Copy Paste: [[2504.05535]] COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values(https://arxiv.org/abs/2504.05535)
• Keywords: robust, large language model
• Abstract: Aligning large language models (LLMs) with human preferences has achieved remarkable success. However, existing Chinese preference datasets are limited by small scale, narrow domain coverage, and lack of rigorous data validation. Additionally, the reliance on human annotators for instruction and response labeling significantly constrains the scalability of human preference datasets. To address these challenges, we design an LLM-based Chinese preference dataset annotation pipeline with no human intervention. Specifically, we crawled and carefully filtered 92k high-quality Chinese queries and employed 15 mainstream LLMs to generate and score chosen-rejected response pairs. Based on it, we introduce COIG-P (Chinese Open Instruction Generalist - Preference), a high-quality, large-scale Chinese preference dataset, comprises 1,009k Chinese preference pairs spanning 6 diverse domains: Chat, Code, Math, Logic, Novel, and Role. Building upon COIG-P, to reduce the overhead of using LLMs for scoring, we trained a 8B-sized Chinese Reward Model (CRM) and meticulously constructed a Chinese Reward Benchmark (CRBench). Evaluation results based on AlignBench \citep{liu2024alignbenchbenchmarkingchinesealignment} show that that COIG-P significantly outperforms other Chinese preference datasets, and it brings significant performance improvements ranging from 2% to 12% for the Qwen2/2.5 and Infinity-Instruct-3M-0625 model series, respectively. The results on CRBench demonstrate that our CRM has a strong and robust scoring ability. We apply it to filter chosen-rejected response pairs in a test split of COIG-P, and our experiments show that it is comparable to GPT-4o in identifying low-quality samples while maintaining efficiency and cost-effectiveness. Our codes and data are released in this https URL.

Title: Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling

• Authors: Tasmiah Haque, Md. Asif Bin Syed, Byungheon Jeong, Xue Bai, Sumit Mohan, Somdyuti Paul, Imtiaz Ahmed, Srinjoy Das
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05537
• Pdf URL: https://arxiv.org/pdf/2504.05537
• Copy Paste: [[2504.05537]] Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling(https://arxiv.org/abs/2504.05537)
• Keywords: generative
• Abstract: We propose a deep learning framework designed to significantly optimize bandwidth for motion-transfer-enabled video applications, including video conferencing, virtual reality interactions, health monitoring systems, and vision-based real-time anomaly detection. To capture complex motion effectively, we utilize the First Order Motion Model (FOMM), which encodes dynamic objects by detecting keypoints and their associated local affine transformations. These keypoints are identified using a self-supervised keypoint detector and arranged into a time series corresponding to the successive frames. Forecasting is performed on these keypoints by integrating two advanced generative time series models into the motion transfer pipeline, namely the Variational Recurrent Neural Network (VRNN) and the Gated Recurrent Unit with Normalizing Flow (GRU-NF). The predicted keypoints are subsequently synthesized into realistic video frames using an optical flow estimator paired with a generator network, thereby facilitating accurate video forecasting and enabling efficient, low-frame-rate video transmission. We validate our results across three datasets for video animation and reconstruction using the following metrics: Mean Absolute Error, Joint Embedding Predictive Architecture Embedding Distance, Structural Similarity Index, and Average Pair-wise Displacement. Our results confirm that by utilizing the superior reconstruction property of the Variational Autoencoder, the VRNN integrated FOMM excels in applications involving multi-step ahead forecasts such as video conferencing. On the other hand, by leveraging the Normalizing Flow architecture for exact likelihood estimation, and enabling efficient latent space sampling, the GRU-NF based FOMM exhibits superior capabilities for producing diverse future samples while maintaining high visual quality for tasks like real-time video-based anomaly detection.

Title: Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting

• Authors: Yunlong Tang, Jing Bi, Chao Huang, Susan Liang, Daiki Shimada, Hang Hua, Yunzhong Xiao, Yizhi Song, Pinxin Liu, Mingqian Feng, Junjia Guo, Zhuo Liu, Luchuan Song, Ali Vosoughi, Jinxi He, Liu He, Zeliang Zhang, Jiebo Luo, Chenliang Xu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05541
• Pdf URL: https://arxiv.org/pdf/2504.05541
• Copy Paste: [[2504.05541]] Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting(https://arxiv.org/abs/2504.05541)
• Keywords: segmentation
• Abstract: We present CAT-V (Caption AnyThing in Video), a training-free framework for fine-grained object-centric video captioning that enables detailed descriptions of user-selected objects through time. CAT-V integrates three key components: a Segmenter based on SAMURAI for precise object segmentation across frames, a Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection and temporal analysis, and a Captioner using InternVL-2.5 for generating detailed object-centric descriptions. Through spatiotemporal visual prompts and chain-of-thought reasoning, our framework generates detailed, temporally-aware descriptions of objects' attributes, actions, statuses, interactions, and environmental contexts without requiring additional training data. CAT-V supports flexible user interactions through various visual prompts (points, bounding boxes, and irregular regions) and maintains temporal sensitivity by tracking object states and interactions across different time segments. Our approach addresses limitations of existing video captioning methods, which either produce overly abstract descriptions or lack object-level precision, enabling fine-grained, object-specific descriptions while maintaining temporal coherence and spatial accuracy. The GitHub repository for this project is available at this https URL

Title: Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control

• Authors: Yongjie Fu, Lingyun Zhong, Zifan Li, Xuan Di
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05553
• Pdf URL: https://arxiv.org/pdf/2504.05553
• Copy Paste: [[2504.05553]] Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control(https://arxiv.org/abs/2504.05553)
• Keywords: robust, federate
• Abstract: Multi-agent reinforcement learning (MARL) has shown promise for adaptive traffic signal control (ATSC), enabling multiple intersections to coordinate signal timings in real time. However, in large-scale settings, MARL faces constraints due to extensive data sharing and communication requirements. Federated learning (FL) mitigates these challenges by training shared models without directly exchanging raw data, yet traditional FL methods such as FedAvg struggle with highly heterogeneous intersections. Different intersections exhibit varying traffic patterns, demands, and road structures, so performing FedAvg across all agents is inefficient. To address this gap, we propose Hierarchical Federated Reinforcement Learning (HFRL) for ATSC. HFRL employs clustering-based or optimization-based techniques to dynamically group intersections and perform FedAvg independently within groups of intersections with similar characteristics, enabling more effective coordination and scalability than standard FedAvg. Our experiments on synthetic and real-world traffic networks demonstrate that HFRL not only outperforms both decentralized and standard federated RL approaches but also identifies suitable grouping patterns based on network structure or traffic demand, resulting in a more robust framework for distributed, heterogeneous systems.

Title: Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study

• Authors: Conrad Borchers, Tianze Shou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05570
• Pdf URL: https://arxiv.org/pdf/2504.05570
• Copy Paste: [[2504.05570]] Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study(https://arxiv.org/abs/2504.05570)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) hold promise as dynamic instructional aids. Yet, it remains unclear whether LLMs can replicate the adaptivity of intelligent tutoring systems (ITS)--where student knowledge and pedagogical strategies are explicitly modeled. We propose a prompt variation framework to assess LLM-generated instructional moves' adaptivity and pedagogical soundness across 75 real-world tutoring scenarios from an ITS. We systematically remove key context components (e.g., student errors and knowledge components) from prompts to create variations of each scenario. Three representative LLMs (Llama3-8B, Llama3-70B, and GPT-4o) generate 1,350 instructional moves. We use text embeddings and randomization tests to measure how the omission of each context feature impacts the LLMs' outputs (adaptivity) and a validated tutor-training classifier to evaluate response quality (pedagogical soundness). Surprisingly, even the best-performing model only marginally mimics the adaptivity of ITS. Specifically, Llama3-70B demonstrates statistically significant adaptivity to student errors. Although Llama3-8B's recommendations receive higher pedagogical soundness scores than the other models, it struggles with instruction-following behaviors, including output formatting. By contrast, GPT-4o reliably adheres to instructions but tends to provide overly direct feedback that diverges from effective tutoring, prompting learners with open-ended questions to gauge knowledge. Given these results, we discuss how current LLM-based tutoring is unlikely to produce learning benefits rivaling known-to-be-effective ITS tutoring. Through our open-source benchmarking code, we contribute a reproducible method for evaluating LLMs' instructional adaptivity and fidelity.

Title: Knowledge-Instruct: Effective Continual Pre-training from Limited Data using Instructions

• Authors: Oded Ovadia, Meni Brief, Rachel Lemberg, Eitam Sheetrit
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05571
• Pdf URL: https://arxiv.org/pdf/2504.05571
• Copy Paste: [[2504.05571]] Knowledge-Instruct: Effective Continual Pre-training from Limited Data using Instructions(https://arxiv.org/abs/2504.05571)
• Keywords: large language model
• Abstract: While Large Language Models (LLMs) acquire vast knowledge during pre-training, they often lack domain-specific, new, or niche information. Continual pre-training (CPT) attempts to address this gap but suffers from catastrophic forgetting and inefficiencies in low-data regimes. We introduce Knowledge-Instruct, a novel approach to efficiently inject knowledge from limited corpora through pure instruction-tuning. By generating information-dense synthetic instruction data, it effectively integrates new knowledge while preserving general reasoning and instruction-following abilities. Knowledge-Instruct demonstrates superior factual memorization, minimizes catastrophic forgetting, and remains scalable by leveraging synthetic data from relatively small language models. Additionally, it enhances contextual understanding, including complex multi-hop reasoning, facilitating integration with retrieval systems. We validate its effectiveness across diverse benchmarks, including Companies, a new dataset that we release to measure knowledge injection capabilities.

Title: A Lightweight Large Vision-language Model for Multimodal Medical Images

• Authors: Belal Alsinglawi, Chris McCarthy, Sara Webb, Christopher Fluke, Navid Toosy Saidy
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05575
• Pdf URL: https://arxiv.org/pdf/2504.05575
• Copy Paste: [[2504.05575]] A Lightweight Large Vision-language Model for Multimodal Medical Images(https://arxiv.org/abs/2504.05575)
• Keywords: extraction
• Abstract: Medical Visual Question Answering (VQA) enhances clinical decision-making by enabling systems to interpret medical images and answer clinical queries. However, developing efficient, high-performance VQA models is challenging due to the complexity of medical imagery and diverse modalities. In this paper, we introduce a lightweight, multimodal VQA model integrating BiomedCLIP for image feature extraction and LLaMA-3 for text processing. Designed for medical VQA tasks, our model achieves state-of-the-art performance on the OmniMedVQA dataset. With approximately 8 billion parameters, it requires only two NVIDIA 40 GB A100 GPUs, demonstrating superior efficiency over larger models. Our results show 73.4% accuracy for open-end questions, surpassing existing models and validating its potential for real-world medical applications. Key contributions include a specialized multimodal VQA model, a resource-efficient architecture, and strong performance in answering open-ended clinical questions.

Title: Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification

• Authors: Jiahang Li, Shibo Xue, Yong Su
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05583
• Pdf URL: https://arxiv.org/pdf/2504.05583
• Copy Paste: [[2504.05583]] Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification(https://arxiv.org/abs/2504.05583)
• Keywords: transformer
• Abstract: Inspired by human visual attention, deep neural networks have widely adopted attention mechanisms to learn locally discriminative attributes for challenging visual classification tasks. However, existing approaches primarily emphasize the representation of such features while neglecting their precise localization, which often leads to misclassification caused by shortcut biases. This limitation becomes even more pronounced when models are evaluated on transfer or out-of-distribution datasets. In contrast, humans are capable of leveraging prior object knowledge to quickly localize and compare fine-grained attributes, a capability that is especially crucial in complex and high-variance classification scenarios. Motivated by this, we introduce Gaze-CIFAR-10, a human gaze time-series dataset, along with a dual-sequence gaze encoder that models the precise sequential localization of human attention on distinct local attributes. In parallel, a Vision Transformer (ViT) is employed to learn the sequential representation of image content. Through cross-modal fusion, our framework integrates human gaze priors with machine-derived visual sequences, effectively correcting inaccurate localization in image feature representations. Extensive qualitative and quantitative experiments demonstrate that gaze-guided cognitive cues significantly enhance classification accuracy.

Title: TW-CRL: Time-Weighted Contrastive Reward Learning for Efficient Inverse Reinforcement Learning

• Authors: Yuxuan Li, Ning Yang, Stephen Xia
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05585
• Pdf URL: https://arxiv.org/pdf/2504.05585
• Copy Paste: [[2504.05585]] TW-CRL: Time-Weighted Contrastive Reward Learning for Efficient Inverse Reinforcement Learning(https://arxiv.org/abs/2504.05585)
• Keywords: robust
• Abstract: Episodic tasks in Reinforcement Learning (RL) often pose challenges due to sparse reward signals and high-dimensional state spaces, which hinder efficient learning. Additionally, these tasks often feature hidden "trap states" -- irreversible failures that prevent task completion but do not provide explicit negative rewards to guide agents away from repeated errors. To address these issues, we propose Time-Weighted Contrastive Reward Learning (TW-CRL), an Inverse Reinforcement Learning (IRL) framework that leverages both successful and failed demonstrations. By incorporating temporal information, TW-CRL learns a dense reward function that identifies critical states associated with success or failure. This approach not only enables agents to avoid trap states but also encourages meaningful exploration beyond simple imitation of expert trajectories. Empirical evaluations on navigation tasks and robotic manipulation benchmarks demonstrate that TW-CRL surpasses state-of-the-art methods, achieving improved efficiency and robustness.

Title: Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations

• Authors: Ajay Jaiswal, Jianyu Wang, Yixiao Li, Pingzhi Li, Tianlong Chen, Zhangyang Wang, Chong Wang, Ruoming Pang, Xianzhi Du
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05586
• Pdf URL: https://arxiv.org/pdf/2504.05586
• Copy Paste: [[2504.05586]] Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations(https://arxiv.org/abs/2504.05586)
• Keywords: robust
• Abstract: Sparsely activated Mixture-of-Experts (SMoE) has shown promise in scaling up the learning capacity of neural networks. However, vanilla SMoEs have issues such as expert redundancy and heavy memory requirements, making them inefficient and non-scalable, especially for resource-constrained scenarios. Expert-level sparsification of SMoEs involves pruning the least important experts to address these limitations. In this work, we aim to address three questions: (1) What is the best recipe to identify the least knowledgeable subset of experts that can be dropped with minimal impact on performance? (2) How should we perform expert dropping (one-shot or iterative), and what correction measures can we undertake to minimize its drastic impact on SMoE subnetwork capabilities? (3) What capabilities of full-SMoEs are severely impacted by the removal of the least dominant experts, and how can we recover them? Firstly, we propose MoE Experts Compression Suite (MC-Suite), which is a collection of some previously explored and multiple novel recipes to provide a comprehensive benchmark for estimating expert importance from diverse perspectives, as well as unveil numerous valuable insights for SMoE experts. Secondly, unlike prior works with a one-shot expert pruning approach, we explore the benefits of iterative pruning with the re-estimation of the MC-Suite criterion. Moreover, we introduce the benefits of task-agnostic fine-tuning as a correction mechanism during iterative expert dropping, which we term MoE Lottery Subnetworks. Lastly, we present an experimentally validated conjecture that, during expert dropping, SMoEs' instruction-following capabilities are predominantly hurt, which can be restored to a robust level subject to external augmentation of instruction-following capabilities using k-shot examples and supervised fine-tuning.

Title: Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model

• Authors: Qi Mao, Lan Chen, Yuchao Gu, Mike Zheng Shou, Ming-Hsuan Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05594
• Pdf URL: https://arxiv.org/pdf/2504.05594
• Copy Paste: [[2504.05594]] Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model(https://arxiv.org/abs/2504.05594)
• Keywords: robust, diffusion
• Abstract: Balancing fidelity and editability is essential in text-based image editing (TIE), where failures commonly lead to over- or under-editing issues. Existing methods typically rely on attention injections for structure preservation and leverage the inherent text alignment capabilities of pre-trained text-to-image (T2I) models for editability, but they lack explicit and unified mechanisms to properly balance these two objectives. In this work, we introduce UnifyEdit, a tuning-free method that performs diffusion latent optimization to enable a balanced integration of fidelity and editability within a unified framework. Unlike direct attention injections, we develop two attention-based constraints: a self-attention (SA) preservation constraint for structural fidelity, and a cross-attention (CA) alignment constraint to enhance text alignment for improved editability. However, simultaneously applying both constraints can lead to gradient conflicts, where the dominance of one constraint results in over- or under-editing. To address this challenge, we introduce an adaptive time-step scheduler that dynamically adjusts the influence of these constraints, guiding the diffusion latent toward an optimal balance. Extensive quantitative and qualitative experiments validate the effectiveness of our approach, demonstrating its superiority in achieving a robust balance between structure preservation and text alignment across various editing tasks, outperforming other state-of-the-art methods. The source code will be available at this https URL.

Title: DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding

• Authors: Hossein Entezari Zarch, Lei Gao, Chaoyi Jiang, Murali Annavaram
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05598
• Pdf URL: https://arxiv.org/pdf/2504.05598
• Copy Paste: [[2504.05598]] DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding(https://arxiv.org/abs/2504.05598)
• Keywords: large language model
• Abstract: Speculative Decoding (SD) is a widely used approach to accelerate the inference of large language models (LLMs) without reducing generation quality. It operates by first using a compact model to draft multiple tokens efficiently, followed by parallel verification using the target LLM. This approach leads to faster inference compared to auto-regressive decoding. While there are multiple approaches to create a draft model, one promising approach is to use early-exit methods. These methods draft candidate tokens by using a subset of layers of the primary model and applying the remaining layers for verification, allowing a single model to handle both drafting and verification. While this technique reduces memory usage and computational cost, its performance relies on the choice of the exit layer for drafting and the number of tokens drafted (speculation length) in each SD round. Prior works use hyperparameter exploration to statically select these values. However, our evaluations show that these hyperparameter values are task-specific, and even within a task they are dependent on the current sequence context. We introduce DEL, a plug-and-play method that adaptively selects the exit layer and speculation length during inference. DEL dynamically tracks the token acceptance rate if the tokens are drafted at each layer of an LLM and uses that knowledge to heuristically select the optimal exit layer and speculation length. Our experiments across a broad range of models and downstream tasks show that DEL achieves overall speedups of $2.16\times$$\sim$$2.50\times$ over vanilla auto-regressive decoding and improves upon the state-of-the-art SD methods by up to $0.27\times$.

Title: Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

• Authors: Yi Peng, Chris, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu, Yahui Zhou
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05599
• Pdf URL: https://arxiv.org/pdf/2504.05599
• Copy Paste: [[2504.05599]] Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought(https://arxiv.org/abs/2504.05599)
• Keywords: robust, large language model
• Abstract: We introduce Skywork R1V, a multimodal reasoning model extending the an R1-series Large language models (LLM) to visual modalities via an efficient multimodal transfer method. Leveraging a lightweight visual projector, Skywork R1V facilitates seamless multimodal adaptation without necessitating retraining of either the foundational language model or the vision encoder. To strengthen visual-text alignment, we propose a hybrid optimization strategy that combines Iterative Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO), significantly enhancing cross-modal integration efficiency. Additionally, we introduce an adaptive-length Chain-of-Thought distillation approach for reasoning data generation. This approach dynamically optimizes reasoning chain lengths, thereby enhancing inference efficiency and preventing excessive reasoning overthinking. Empirical evaluations demonstrate that Skywork R1V, with only 38B parameters, delivers competitive performance, achieving a score of 69.0 on the MMMU benchmark and 67.5 on MathVista. Meanwhile, it maintains robust textual reasoning performance, evidenced by impressive scores of 72.0 on AIME and 94.0 on MATH500. The Skywork R1V model weights have been publicly released to promote openness and reproducibility.

Title: On the Impact of Language Nuances on Sentiment Analysis with Large Language Models: Paraphrasing, Sarcasm, and Emojis

• Authors: Naman Bhargava, Mohammed I. Radaideh, O Hwang Kwon, Aditi Verma, Majdi I. Radaideh
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05603
• Pdf URL: https://arxiv.org/pdf/2504.05603
• Copy Paste: [[2504.05603]] On the Impact of Language Nuances on Sentiment Analysis with Large Language Models: Paraphrasing, Sarcasm, and Emojis(https://arxiv.org/abs/2504.05603)
• Keywords: robust, large language model
• Abstract: Large Language Models (LLMs) have demonstrated impressive performance across various tasks, including sentiment analysis. However, data quality--particularly when sourced from social media--can significantly impact their accuracy. This research explores how textual nuances, including emojis and sarcasm, affect sentiment analysis, with a particular focus on improving data quality through text paraphrasing techniques. To address the lack of labeled sarcasm data, the authors created a human-labeled dataset of 5929 tweets that enabled the assessment of LLM in various sarcasm contexts. The results show that when topic-specific datasets, such as those related to nuclear power, are used to finetune LLMs these models are not able to comprehend accurate sentiment in presence of sarcasm due to less diverse text, requiring external interventions like sarcasm removal to boost model accuracy. Sarcasm removal led to up to 21% improvement in sentiment accuracy, as LLMs trained on nuclear power-related content struggled with sarcastic tweets, achieving only 30% accuracy. In contrast, LLMs trained on general tweet datasets, covering a broader range of topics, showed considerable improvements in predicting sentiment for sarcastic tweets (60% accuracy), indicating that incorporating general text data can enhance sarcasm detection. The study also utilized adversarial text augmentation, showing that creating synthetic text variants by making minor changes significantly increased model robustness and accuracy for sarcastic tweets (approximately 85%). Additionally, text paraphrasing of tweets with fragmented language transformed around 40% of the tweets with low-confidence labels into high-confidence ones, improving LLMs sentiment analysis accuracy by 6%.

Title: ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs

• Authors: Gejian Zhao, Hanzhou Wu, Xinpeng Zhang, Athanasios V. Vasilakos
• Subjects: cs.CR, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05605
• Pdf URL: https://arxiv.org/pdf/2504.05605
• Copy Paste: [[2504.05605]] ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs(https://arxiv.org/abs/2504.05605)
• Keywords: security, defense, attack, steal
• Abstract: Chain-of-Thought (CoT) enhances an LLM's ability to perform complex reasoning tasks, but it also introduces new security issues. In this work, we present ShadowCoT, a novel backdoor attack framework that targets the internal reasoning mechanism of LLMs. Unlike prior token-level or prompt-based attacks, ShadowCoT directly manipulates the model's cognitive reasoning path, enabling it to hijack multi-step reasoning chains and produce logically coherent but adversarial outcomes. By conditioning on internal reasoning states, ShadowCoT learns to recognize and selectively disrupt key reasoning steps, effectively mounting a self-reflective cognitive attack within the target model. Our approach introduces a lightweight yet effective multi-stage injection pipeline, which selectively rewires attention pathways and perturbs intermediate representations with minimal parameter overhead (only 0.15% updated). ShadowCoT further leverages reinforcement learning and reasoning chain pollution (RCP) to autonomously synthesize stealthy adversarial CoTs that remain undetectable to advanced defenses. Extensive experiments across diverse reasoning benchmarks and LLMs show that ShadowCoT consistently achieves high Attack Success Rate (94.4%) and Hijacking Success Rate (88.4%) while preserving benign performance. These results reveal an emergent class of cognition-level threats and highlight the urgent need for defenses beyond shallow surface-level consistency.

Title: FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extraction

• Authors: Qian-Wen Zhang, Fang Li, Jie Wang, Lingfeng Qiao, Yifei Yu, Di Yin, Xing Sun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05607
• Pdf URL: https://arxiv.org/pdf/2504.05607
• Copy Paste: [[2504.05607]] FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extraction(https://arxiv.org/abs/2504.05607)
• Keywords: extraction, large language model
• Abstract: Extractive reading comprehension systems are designed to locate the correct answer to a question within a given text. However, a persistent challenge lies in ensuring these models maintain high accuracy in answering questions while reliably recognizing unanswerable queries. Despite significant advances in large language models (LLMs) for reading comprehension, this issue remains critical, particularly as the length of supported contexts continues to expand. To address this challenge, we propose an innovative data augmentation methodology grounded in a multi-agent collaborative framework. Unlike traditional methods, such as the costly human annotation process required for datasets like SQuAD 2.0, our method autonomously generates evidence-based question-answer pairs and systematically constructs unanswerable questions. Using this methodology, we developed the FactGuard-Bench dataset, which comprises 25,220 examples of both answerable and unanswerable question scenarios, with context lengths ranging from 8K to 128K. Experimental evaluations conducted on seven popular LLMs reveal that even the most advanced models achieve only 61.79% overall accuracy. Furthermore, we emphasize the importance of a model's ability to reason about unanswerable questions to avoid generating plausible but incorrect answers. By implementing efficient data selection and generation within the multi-agent collaborative framework, our method significantly reduces the traditionally high costs associated with manual annotation and provides valuable insights for the training and optimization of LLMs.

Title: Fairness in Machine Learning-based Hand Load Estimation: A Case Study on Load Carriage Tasks

• Authors: Arafat Rahman, Sol Lim, Seokhyun Chung
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05610
• Pdf URL: https://arxiv.org/pdf/2504.05610
• Copy Paste: [[2504.05610]] Fairness in Machine Learning-based Hand Load Estimation: A Case Study on Load Carriage Tasks(https://arxiv.org/abs/2504.05610)
• Keywords: fair
• Abstract: Predicting external hand load from sensor data is essential for ergonomic exposure assessments, as obtaining this information typically requires direct observation or supplementary data. While machine learning methods have been used to estimate external hand load from worker postures or force exertion data, our findings reveal systematic bias in these predictions due to individual differences such as age and biological sex. To explore this issue, we examined bias in hand load prediction by varying the sex ratio in the training dataset. We found substantial sex disparity in predictive performance, especially when the training dataset is more sex-imbalanced. To address this bias, we developed and evaluated a fair predictive model for hand load estimation that leverages a Variational Autoencoder (VAE) with feature disentanglement. This approach is designed to separate sex-agnostic and sex-specific latent features, minimizing feature overlap. The disentanglement capability enables the model to make predictions based solely on sex-agnostic features of motion patterns, ensuring fair prediction for both biological sexes. Our proposed fair algorithm outperformed conventional machine learning methods (e.g., Random Forests) in both fairness and predictive accuracy, achieving a lower mean absolute error (MAE) difference across male and female sets and improved fairness metrics such as statistical parity (SP) and positive and negative residual differences (PRD and NRD), even when trained on imbalanced sex datasets. These findings emphasize the importance of fairness-aware machine learning algorithms to prevent potential disadvantages in workplace health and safety for certain worker populations.

Title: Falcon: Fractional Alternating Cut with Overcoming Minima in Unsupervised Segmentation

• Authors: Xiao Zhang, Xiangyu Han, Xiwen Lai, Yao Sun, Pei Zhang, Konrad Kording
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05613
• Pdf URL: https://arxiv.org/pdf/2504.05613
• Copy Paste: [[2504.05613]] Falcon: Fractional Alternating Cut with Overcoming Minima in Unsupervised Segmentation(https://arxiv.org/abs/2504.05613)
• Keywords: transformer, segmentation
• Abstract: Today's unsupervised image segmentation algorithms often segment suboptimally. Modern graph-cut based approaches rely on high-dimensional attention maps from Transformer-based foundation models, typically employing a relaxed Normalized Cut solved recursively via the Fiedler vector (the eigenvector of the second smallest eigenvalue). Consequently, they still lag behind supervised methods in both mask generation speed and segmentation accuracy. We present a regularized fractional alternating cut (Falcon), an optimization-based K-way Normalized Cut without relying on recursive eigenvector computations, achieving substantially improved speed and accuracy. Falcon operates in two stages: (1) a fast K-way Normalized Cut solved by extending into a fractional quadratic transformation, with an alternating iterative procedure and regularization to avoid local minima; and (2) refinement of the resulting masks using complementary low-level information, producing high-quality pixel-level segmentations. Experiments show that Falcon not only surpasses existing state-of-the-art methods by an average of 2.5% across six widely recognized benchmarks (reaching up to 4.3\% improvement on Cityscapes), but also reduces runtime by around 30% compared to prior graph-based approaches. These findings demonstrate that the semantic information within foundation-model attention can be effectively harnessed by a highly parallelizable graph cut framework. Consequently, Falcon can narrow the gap between unsupervised and supervised segmentation, enhancing scalability in real-world applications and paving the way for dense prediction-based vision pre-training in various downstream tasks. The code is released in this https URL.

Title: Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement

• Authors: Yichen Dong, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, Hao Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05614
• Pdf URL: https://arxiv.org/pdf/2504.05614
• Copy Paste: [[2504.05614]] Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement(https://arxiv.org/abs/2504.05614)
• Keywords: large language model
• Abstract: Recent research has shown that large language models (LLMs) can enhance translation quality through self-refinement. In this paper, we build on this idea by extending the refinement from sentence-level to document-level translation, specifically focusing on document-to-document (Doc2Doc) translation refinement. Since sentence-to-sentence (Sent2Sent) and Doc2Doc translation address different aspects of the translation process, we propose fine-tuning LLMs for translation refinement using two intermediate translations, combining the strengths of both Sent2Sent and Doc2Doc. Additionally, recognizing that the quality of intermediate translations varies, we introduce an enhanced fine-tuning method with quality awareness that assigns lower weights to easier translations and higher weights to more difficult ones, enabling the model to focus on challenging translation cases. Experimental results across ten translation tasks with LLaMA-3-8B-Instruct and Mistral-Nemo-Instruct demonstrate the effectiveness of our approach.

Title: FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels

• Authors: Seunghun Yu, Jin-Hyun Ahn, Joonhyuk Kang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05615
• Pdf URL: https://arxiv.org/pdf/2504.05615
• Copy Paste: [[2504.05615]] FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels(https://arxiv.org/abs/2504.05615)
• Keywords: privacy, federate
• Abstract: Federated Learning (FL) is a powerful framework for privacy-preserving distributed learning. It enables multiple clients to collaboratively train a global model without sharing raw data. However, handling noisy labels in FL remains a major challenge due to heterogeneous data distributions and communication constraints, which can severely degrade model performance. To address this issue, we propose FedEFC, a novel method designed to tackle the impact of noisy labels in FL. FedEFC mitigates this issue through two key techniques: (1) prestopping, which prevents overfitting to mislabeled data by dynamically halting training at an optimal point, and (2) loss correction, which adjusts model updates to account for label noise. In particular, we develop an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training. Furthermore, we provide a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution. Extensive experimental results validate the effectiveness of our approach, showing that it consistently outperforms existing FL techniques in mitigating the impact of noisy labels, particularly under heterogeneous data settings (e.g., achieving up to 41.64% relative performance improvement over the existing loss correction method).

Title: Technical Report: Full Version of Analyzing and Optimizing Perturbation of DP-SGD Geometrically

• Authors: Jiawei Duan, Haibo Hu, Qingqing Ye, Xinyue Sun
• Subjects: cs.LG, cs.AI, cs.CV, cs.DB
• Abstract URL: https://arxiv.org/abs/2504.05618
• Pdf URL: https://arxiv.org/pdf/2504.05618
• Copy Paste: [[2504.05618]] Technical Report: Full Version of Analyzing and Optimizing Perturbation of DP-SGD Geometrically(https://arxiv.org/abs/2504.05618)
• Keywords: privacy
• Abstract: Differential privacy (DP) has become a prevalent privacy model in a wide range of machine learning tasks, especially after the debut of DP-SGD. However, DP-SGD, which directly perturbs gradients in the training iterations, fails to mitigate the negative impacts of noise on gradient direction. As a result, DP-SGD is often inefficient. Although various solutions (e.g., clipping to reduce the sensitivity of gradients and amplifying privacy bounds to save privacy budgets) are proposed to trade privacy for model efficiency, the root cause of its inefficiency is yet unveiled. In this work, we first generalize DP-SGD and theoretically derive the impact of DP noise on the training process. Our analysis reveals that, in terms of a perturbed gradient, only the noise on direction has eminent impact on the model efficiency while that on magnitude can be mitigated by optimization techniques, i.e., fine-tuning gradient clipping and learning rate. Besides, we confirm that traditional DP introduces biased noise on the direction when adding unbiased noise to the gradient itself. Overall, the perturbation of DP-SGD is actually sub-optimal from a geometric perspective. Motivated by this, we design a geometric perturbation strategy GeoDP within the DP framework, which perturbs the direction and the magnitude of a gradient, respectively. By directly reducing the noise on the direction, GeoDP mitigates the negative impact of DP noise on model efficiency with the same DP guarantee. Extensive experiments on two public datasets (i.e., MNIST and CIFAR-10), one synthetic dataset and three prevalent models (i.e., Logistic Regression, CNN and ResNet) confirm the effectiveness and generality of our strategy.

Title: Model-Agnostic Policy Explanations with Large Language Models

• Authors: Zhang Xi-Jia, Yue Guo, Shufei Chen, Simon Stepputtis, Matthew Gombolay, Katia Sycara, Joseph Campbell
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05625
• Pdf URL: https://arxiv.org/pdf/2504.05625
• Copy Paste: [[2504.05625]] Model-Agnostic Policy Explanations with Large Language Models(https://arxiv.org/abs/2504.05625)
• Keywords: interpretability, large language model
• Abstract: Intelligent agents, such as robots, are increasingly deployed in real-world, human-centric environments. To foster appropriate human trust and meet legal and ethical standards, these agents must be able to explain their behavior. However, state-of-the-art agents are typically driven by black-box models like deep neural networks, limiting their interpretability. We propose a method for generating natural language explanations of agent behavior based only on observed states and actions -- without access to the agent's underlying model. Our approach learns a locally interpretable surrogate model of the agent's behavior from observations, which then guides a large language model to generate plausible explanations with minimal hallucination. Empirical results show that our method produces explanations that are more comprehensible and correct than those from baselines, as judged by both language models and human evaluators. Furthermore, we find that participants in a user study more accurately predicted the agent's future actions when given our explanations, suggesting improved understanding of agent behavior.

Title: Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning

• Authors: Sanchit Kabra, Akshita Jha, Chandan Reddy
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05632
• Pdf URL: https://arxiv.org/pdf/2504.05632
• Copy Paste: [[2504.05632]] Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning(https://arxiv.org/abs/2504.05632)
• Keywords: fair, generative
• Abstract: Recent advances in large-scale generative language models have shown that reasoning capabilities can significantly improve model performance across a variety of tasks. However, the impact of reasoning on a model's ability to mitigate stereotypical responses remains largely underexplored. In this work, we investigate the crucial relationship between a model's reasoning ability and fairness, and ask whether improved reasoning capabilities can mitigate harmful stereotypical responses, especially those arising due to shallow or flawed reasoning. We conduct a comprehensive evaluation of multiple open-source LLMs, and find that larger models with stronger reasoning abilities exhibit substantially lower stereotypical bias on existing fairness benchmarks. Building on this insight, we introduce ReGiFT -- Reasoning Guided Fine-Tuning, a novel approach that extracts structured reasoning traces from advanced reasoning models and infuses them into models that lack such capabilities. We use only general-purpose reasoning and do not require any fairness-specific supervision for bias mitigation. Notably, we see that models fine-tuned using ReGiFT not only improve fairness relative to their non-reasoning counterparts but also outperform advanced reasoning models on fairness benchmarks. We also analyze how variations in the correctness of the reasoning traces and their length influence model fairness and their overall performance. Our findings highlight that enhancing reasoning capabilities is an effective, fairness-agnostic strategy for mitigating stereotypical bias caused by reasoning flaws.

Title: TAGC: Optimizing Gradient Communication in Distributed Transformer Training

• Authors: Igor Polyakov, Alexey Dukhanov, Egor Spirin
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2504.05638
• Pdf URL: https://arxiv.org/pdf/2504.05638
• Copy Paste: [[2504.05638]] TAGC: Optimizing Gradient Communication in Distributed Transformer Training(https://arxiv.org/abs/2504.05638)
• Keywords: transformer, large language model
• Abstract: The increasing complexity of large language models (LLMs) necessitates efficient training strategies to mitigate the high computational costs associated with distributed training. A significant bottleneck in this process is gradient synchronization across multiple GPUs, particularly in the zero-redundancy parallelism mode. In this paper, we introduce Transformer-Aware Gradient Compression (TAGC), an optimized gradient compression algorithm designed specifically for transformer-based models. TAGC extends the lossless homomorphic compression method by adapting it for sharded models and incorporating transformer-specific optimizations, such as layer-selective compression and dynamic sparsification. Our experimental results demonstrate that TAGC accelerates training by up to 15% compared to the standard Fully Sharded Data Parallel (FSDP) approach, with minimal impact on model quality. We integrate TAGC into the PyTorch FSDP framework, the implementation is publicly available at this https URL.

Title: DBOT: Artificial Intelligence for Systematic Long-Term Investing

• Authors: Vasant Dhar, João Sedoc
• Subjects: cs.CL, cs.AI, q-fin.PR
• Abstract URL: https://arxiv.org/abs/2504.05639
• Pdf URL: https://arxiv.org/pdf/2504.05639
• Copy Paste: [[2504.05639]] DBOT: Artificial Intelligence for Systematic Long-Term Investing(https://arxiv.org/abs/2504.05639)
• Keywords: generative
• Abstract: Long-term investing was previously seen as requiring human judgment. With the advent of generative artificial intelligence (AI) systems, automated systematic long-term investing is now feasible. In this paper, we present DBOT, a system whose goal is to reason about valuation like Aswath Damodaran, who is a unique expert in the investment arena in terms of having published thousands of valuations on companies in addition to his numerous writings on the topic, which provide ready training data for an AI system. DBOT can value any publicly traded company. DBOT can also be back-tested, making its behavior and performance amenable to scientific inquiry. We compare DBOT to its analytic parent, Damodaran, and highlight the research challenges involved in raising its current capability to that of Damodaran's. Finally, we examine the implications of DBOT-like AI agents for the financial industry, especially how they will impact the role of human analysts in valuation.

Title: CTI-Unet: Cascaded Threshold Integration for Improved U-Net Segmentation of Pathology Images

• Authors: Mingyang Zhu, Yuqiu Liang, Jiacheng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05640
• Pdf URL: https://arxiv.org/pdf/2504.05640
• Copy Paste: [[2504.05640]] CTI-Unet: Cascaded Threshold Integration for Improved U-Net Segmentation of Pathology Images(https://arxiv.org/abs/2504.05640)
• Keywords: robust, segmentation
• Abstract: Chronic kidney disease (CKD) is a growing global health concern, necessitating precise and efficient image analysis to aid diagnosis and treatment planning. Automated segmentation of kidney pathology images plays a central role in facilitating clinical workflows, yet conventional segmentation models often require delicate threshold tuning. This paper proposes a novel \textit{Cascaded Threshold-Integrated U-Net (CTI-Unet)} to overcome the limitations of single-threshold segmentation. By sequentially integrating multiple thresholded outputs, our approach can reconcile noise suppression with the preservation of finer structural details. Experiments on the challenging KPIs2024 dataset demonstrate that CTI-Unet outperforms state-of-the-art architectures such as nnU-Net, Swin-Unet, and CE-Net, offering a robust and flexible framework for kidney pathology image segmentation.

Title: Leveraging Prompt-Tuning for Bengali Grammatical Error Explanation Using Large Language Models

• Authors: Subhankar Maity, Aniket Deroy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05642
• Pdf URL: https://arxiv.org/pdf/2504.05642
• Copy Paste: [[2504.05642]] Leveraging Prompt-Tuning for Bengali Grammatical Error Explanation Using Large Language Models(https://arxiv.org/abs/2504.05642)
• Keywords: large language model
• Abstract: We propose a novel three-step prompt-tuning method for Bengali Grammatical Error Explanation (BGEE) using state-of-the-art large language models (LLMs) such as GPT-4, GPT-3.5 Turbo, and Llama-2-70b. Our approach involves identifying and categorizing grammatical errors in Bengali sentences, generating corrected versions of the sentences, and providing natural language explanations for each identified error. We evaluate the performance of our BGEE system using both automated evaluation metrics and human evaluation conducted by experienced Bengali language experts. Our proposed prompt-tuning approach shows that GPT-4, the best performing LLM, surpasses the baseline model in automated evaluation metrics, with a 5.26% improvement in F1 score and a 6.95% improvement in exact match. Furthermore, compared to the previous baseline, GPT-4 demonstrates a decrease of 25.51% in wrong error type and a decrease of 26.27% in wrong error explanation. However, the results still lag behind the human baseline.

Title: Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking

• Authors: Yu-Hang Wu, Yu-Jie Xiong, Jie-Zhang
• Subjects: cs.CR, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05652
• Pdf URL: https://arxiv.org/pdf/2504.05652
• Copy Paste: [[2504.05652]] Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking(https://arxiv.org/abs/2504.05652)
• Keywords: defense, attack, large language model
• Abstract: Large Language Models (LLMs) have become increasingly integral to a wide range of applications. However, they still remain the threat of jailbreak attacks, where attackers manipulate designed prompts to make the models elicit malicious outputs. Analyzing jailbreak methods can help us delve into the weakness of LLMs and improve it. In this paper, We reveal a vulnerability in large language models (LLMs), which we term Defense Threshold Decay (DTD), by analyzing the attention weights of the model's output on input and subsequent output on prior output: as the model generates substantial benign content, its attention weights shift from the input to prior output, making it more susceptible to jailbreak attacks. To demonstrate the exploitability of DTD, we propose a novel jailbreak attack method, Sugar-Coated Poison (SCP), which induces the model to generate substantial benign content through benign input and adversarial reasoning, subsequently producing malicious content. To mitigate such attacks, we introduce a simple yet effective defense strategy, POSD, which significantly reduces jailbreak success rates while preserving the model's generalization capabilities.

Title: Reconstruction-Free Anomaly Detection with Diffusion Models via Direct Latent Likelihood Evaluation

• Authors: Shunsuke Sakai, Tatsuhito Hasegawa
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05662
• Pdf URL: https://arxiv.org/pdf/2504.05662
• Copy Paste: [[2504.05662]] Reconstruction-Free Anomaly Detection with Diffusion Models via Direct Latent Likelihood Evaluation(https://arxiv.org/abs/2504.05662)
• Keywords: robust, diffusion
• Abstract: Diffusion models, with their robust distribution approximation capabilities, have demonstrated excellent performance in anomaly detection. However, conventional reconstruction-based approaches rely on computing the reconstruction error between the original and denoised images, which requires careful noise-strength tuning and over ten network evaluations per input-leading to significantly slower detection speeds. To address these limitations, we propose a novel diffusion-based anomaly detection method that circumvents the need for resource-intensive reconstruction. Instead of reconstructing the input image, we directly infer its corresponding latent variables and measure their density under the Gaussian prior distribution. Remarkably, the prior density proves effective as an anomaly score even when using a short partial diffusion process of only 2-5 steps. We evaluate our method on the MVTecAD dataset, achieving an AUC of 0.991 at 15 FPS, thereby setting a new state-of-the-art speed-AUC anomaly detection trade-off.

Title: VC-LLM: Automated Advertisement Video Creation from Raw Footage using Multi-modal LLMs

• Authors: Dongjun Qian, Kai Su, Yiming Tan, Qishuai Diao, Xian Wu, Chang Liu, Bingyue Peng, Zehuan Yuan
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05673
• Pdf URL: https://arxiv.org/pdf/2504.05673
• Copy Paste: [[2504.05673]] VC-LLM: Automated Advertisement Video Creation from Raw Footage using Multi-modal LLMs(https://arxiv.org/abs/2504.05673)
• Keywords: large language model
• Abstract: As short videos have risen in popularity, the role of video content in advertising has become increasingly significant. Typically, advertisers record a large amount of raw footage about the product and then create numerous different short-form advertisement videos based on this raw footage. Creating such videos mainly involves editing raw footage and writing advertisement scripts, which requires a certain level of creative ability. It is usually challenging to create many different video contents for the same product, and manual efficiency is often low. In this paper, we present VC-LLM, a framework powered by Large Language Models for the automatic creation of high-quality short-form advertisement videos. Our approach leverages high-resolution spatial input and low-resolution temporal input to represent video clips more effectively, capturing both fine-grained visual details and broader temporal dynamics. In addition, during training, we incorporate supplementary information generated by rewriting the ground truth text, ensuring that all key output information can be directly traced back to the input, thereby reducing model hallucinations. We also designed a benchmark to evaluate the quality of the created videos. Experiments show that VC-LLM based on GPT-4o can produce videos comparable to those created by humans. Furthermore, we collected numerous high-quality short advertisement videos to create a pre-training dataset and manually cleaned a portion of the data to construct a high-quality fine-tuning dataset. Experiments indicate that, on the benchmark, the VC-LLM based on fine-tuned LLM can produce videos with superior narrative logic compared to those created by the VC-LLM based on GPT-4o.

Title: Event-based Civil Infrastructure Visual Defect Detection: ev-CIVIL Dataset and Benchmark

• Authors: Udayanga G.W.K.N. Gamage, Xuanni Huo, Luca Zanatta, T Delbruck, Cesar Cadena, Matteo Fumagalli, Silvia Tolu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05679
• Pdf URL: https://arxiv.org/pdf/2504.05679
• Copy Paste: [[2504.05679]] Event-based Civil Infrastructure Visual Defect Detection: ev-CIVIL Dataset and Benchmark(https://arxiv.org/abs/2504.05679)
• Keywords: robust
• Abstract: Small Unmanned Aerial Vehicle (UAV) based visual inspections are a more efficient alternative to manual methods for examining civil structural defects, offering safe access to hazardous areas and significant cost savings by reducing labor requirements. However, traditional frame-based cameras, widely used in UAV-based inspections, often struggle to capture defects under low or dynamic lighting conditions. In contrast, Dynamic Vision Sensors (DVS), or event-based cameras, excel in such scenarios by minimizing motion blur, enhancing power efficiency, and maintaining high-quality imaging across diverse lighting conditions without saturation or information loss. Despite these advantages, existing research lacks studies exploring the feasibility of using DVS for detecting civil structural this http URL, there is no dedicated event-based dataset tailored for this purpose. Addressing this gap, this study introduces the first event-based civil infrastructure defect detection dataset, capturing defective surfaces as a spatio-temporal event stream using this http URL addition to event-based data, the dataset includes grayscale intensity image frames captured simultaneously using an Active Pixel Sensor (APS). Both data types were collected using the DAVIS346 camera, which integrates DVS and APS this http URL dataset focuses on two types of defects: cracks and spalling, and includes data from both field and laboratory environments. The field dataset comprises 318 recording sequences,documenting 458 distinct cracks and 121 distinct spalling this http URL laboratory dataset includes 362 recording sequences, covering 220 distinct cracks and 308 spalling this http URL realtime object detection models were evaluated on it to validate the dataset this http URL results demonstrate the dataset robustness in enabling accurate defect detection and classification,even under challenging lighting conditions.

Title: Towards Smarter Hiring: Are Zero-Shot and Few-Shot Pre-trained LLMs Ready for HR Spoken Interview Transcript Analysis?

• Authors: Subhankar Maity, Aniket Deroy, Sudeshna Sarkar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05683
• Pdf URL: https://arxiv.org/pdf/2504.05683
• Copy Paste: [[2504.05683]] Towards Smarter Hiring: Are Zero-Shot and Few-Shot Pre-trained LLMs Ready for HR Spoken Interview Transcript Analysis?(https://arxiv.org/abs/2504.05683)
• Keywords: large language model
• Abstract: This research paper presents a comprehensive analysis of the performance of prominent pre-trained large language models (LLMs), including GPT-4 Turbo, GPT-3.5 Turbo, text-davinci-003, text-babbage-001, text-curie-001, text-ada-001, llama-2-7b-chat, llama-2-13b-chat, and llama-2-70b-chat, in comparison to expert human evaluators in providing scores, identifying errors, and offering feedback and improvement suggestions to candidates during mock HR (Human Resources) interviews. We introduce a dataset called HURIT (Human Resource Interview Transcripts), which comprises 3,890 HR interview transcripts sourced from real-world HR interview scenarios. Our findings reveal that pre-trained LLMs, particularly GPT-4 Turbo and GPT-3.5 Turbo, exhibit commendable performance and are capable of producing evaluations comparable to those of expert human evaluators. Although these LLMs demonstrate proficiency in providing scores comparable to human experts in terms of human evaluation metrics, they frequently fail to identify errors and offer specific actionable advice for candidate performance improvement in HR interviews. Our research suggests that the current state-of-the-art pre-trained LLMs are not fully conducive for automatic deployment in an HR interview assessment. Instead, our findings advocate for a human-in-the-loop approach, to incorporate manual checks for inconsistencies and provisions for improving feedback quality as a more suitable strategy.

Title: Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators

• Authors: Xitao Li, Haijun Wang, Jiang Wu, Ting Liu
• Subjects: cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.05689
• Pdf URL: https://arxiv.org/pdf/2504.05689
• Copy Paste: [[2504.05689]] Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators(https://arxiv.org/abs/2504.05689)
• Keywords: security, attack, large language model
• Abstract: Conversational large language models (LLMs) have gained widespread attention due to their instruction-following capabilities. To ensure conversational LLMs follow instructions, role separators are employed to distinguish between different participants in a conversation. However, incorporating role separators introduces potential vulnerabilities. Misusing roles can lead to prompt injection attacks, which can easily misalign the model's behavior with the user's intentions, raising significant security concerns. Although various prompt injection attacks have been proposed, recent research has largely overlooked the impact of role separators on safety. This highlights the critical need to thoroughly understand the systemic weaknesses in dialogue systems caused by role separators. This paper identifies modeling weaknesses caused by role separators. Specifically, we observe a strong positional bias associated with role separators, which is inherent in the format of dialogue modeling and can be triggered by the insertion of role separators. We further develop the Separators Injection Attack (SIA), a new orthometric attack based on role separators. The experiment results show that SIA is efficient and extensive in manipulating model behavior with an average gain of 18.2% for manual methods and enhances the attack success rate to 100% with automatic methods.

Title: STRIVE: A Think & Improve Approach with Iterative Refinement for Enhancing Question Quality Estimation

• Authors: Aniket Deroy, Subhankar Maity
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05693
• Pdf URL: https://arxiv.org/pdf/2504.05693
• Copy Paste: [[2504.05693]] STRIVE: A Think & Improve Approach with Iterative Refinement for Enhancing Question Quality Estimation(https://arxiv.org/abs/2504.05693)
• Keywords: large language model
• Abstract: Automatically assessing question quality is crucial for educators as it saves time, ensures consistency, and provides immediate feedback for refining teaching materials. We propose a novel methodology called STRIVE (Structured Thinking and Refinement with multiLLMs for Improving Verified Question Estimation) using a series of Large Language Models (LLMs) for automatic question evaluation. This approach aims to improve the accuracy and depth of question quality assessment, ultimately supporting diverse learners and enhancing educational practices. The method estimates question quality in an automated manner by generating multiple evaluations based on the strengths and weaknesses of the provided question and then choosing the best solution generated by the LLM. Then the process is improved by iterative review and response with another LLM until the evaluation metric values converge. This sophisticated method of evaluating question quality improves the estimation of question quality by automating the task of question quality evaluation. Correlation scores show that using this proposed method helps to improve correlation with human judgments compared to the baseline method. Error analysis shows that metrics like relevance and appropriateness improve significantly relative to human judgments by using STRIVE.

Title: Point-based Instance Completion with Scene Constraints

• Authors: Wesley Khademi, Li Fuxin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05698
• Pdf URL: https://arxiv.org/pdf/2504.05698
• Copy Paste: [[2504.05698]] Point-based Instance Completion with Scene Constraints(https://arxiv.org/abs/2504.05698)
• Keywords: robust
• Abstract: Recent point-based object completion methods have demonstrated the ability to accurately recover the missing geometry of partially observed objects. However, these approaches are not well-suited for completing objects within a scene, as they do not consider known scene constraints (e.g., other observed surfaces) in their completions and further expect the partial input to be in a canonical coordinate system, which does not hold for objects within scenes. While instance scene completion methods have been proposed for completing objects within a scene, they lag behind point-based object completion methods in terms of object completion quality and still do not consider known scene constraints during completion. To overcome these limitations, we propose a point cloud-based instance completion model that can robustly complete objects at arbitrary scales and pose in the scene. To enable reasoning at the scene level, we introduce a sparse set of scene constraints represented as point clouds and integrate them into our completion model via a cross-attention mechanism. To evaluate the instance scene completion task on indoor scenes, we further build a new dataset called ScanWCF, which contains labeled partial scans as well as aligned ground truth scene completions that are watertight and collision-free. Through several experiments, we demonstrate that our method achieves improved fidelity to partial scans, higher completion quality, and greater plausibility over existing state-of-the-art methods.

Title: Pose-Aware Weakly-Supervised Action Segmentation

• Authors: Seth Z. Zhao, Reza Ghoddoosian, Isht Dwivedi, Nakul Agarwal, Behzad Dariush
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05700
• Pdf URL: https://arxiv.org/pdf/2504.05700
• Copy Paste: [[2504.05700]] Pose-Aware Weakly-Supervised Action Segmentation(https://arxiv.org/abs/2504.05700)
• Keywords: segmentation
• Abstract: Understanding human behavior is an important problem in the pursuit of visual intelligence. A challenge in this endeavor is the extensive and costly effort required to accurately label action segments. To address this issue, we consider learning methods that demand minimal supervision for segmentation of human actions in long instructional videos. Specifically, we introduce a weakly-supervised framework that uniquely incorporates pose knowledge during training while omitting its use during inference, thereby distilling pose knowledge pertinent to each action component. We propose a pose-inspired contrastive loss as a part of the whole weakly-supervised framework which is trained to distinguish action boundaries more effectively. Our approach, validated through extensive experiments on representative datasets, outperforms previous state-of-the-art (SOTA) in segmenting long instructional videos under both online and offline settings. Additionally, we demonstrate the framework's adaptability to various segmentation backbones and pose extractors across different datasets.

Title: SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning

• Authors: Fida Mohammad Thoker, Letian Jiang, Chen Zhao, Piyush Bagad, Hazel Doughty, Bernard Ghanem, Cees G. M. Snoek
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05706
• Pdf URL: https://arxiv.org/pdf/2504.05706
• Copy Paste: [[2504.05706]] SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning(https://arxiv.org/abs/2504.05706)
• Keywords: transformer
• Abstract: Continued advances in self-supervised learning have led to significant progress in video representation learning, offering a scalable alternative to supervised approaches by removing the need for manual annotations. Despite strong performance on standard action recognition benchmarks, video self-supervised learning methods are largely evaluated under narrow protocols, typically pretraining on Kinetics-400 and fine-tuning on similar datasets, limiting our understanding of their generalization in real world scenarios. In this work, we present a comprehensive evaluation of modern video self-supervised models, focusing on generalization across four key downstream factors: domain shift, sample efficiency, action granularity, and task diversity. Building on our prior work analyzing benchmark sensitivity in CNN-based contrastive learning, we extend the study to cover state-of-the-art transformer-based video-only and video-text models. Specifically, we benchmark 12 transformer-based methods (7 video-only, 5 video-text) and compare them to 10 CNN-based methods, totaling over 1100 experiments across 8 datasets and 7 downstream tasks. Our analysis shows that, despite architectural advances, transformer-based models remain sensitive to downstream conditions. No method generalizes consistently across all factors, video-only transformers perform better under domain shifts, CNNs outperform for fine-grained tasks, and video-text models often underperform despite large scale pretraining. We also find that recent transformer models do not consistently outperform earlier approaches. Our findings provide a detailed view of the strengths and limitations of current video SSL methods and offer a unified benchmark for evaluating generalization in video representation learning.

Title: Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment

• Authors: Gen Li, Li Chen, Cheng Tang, Valdemar Švábenský, Daisuke Deguchi, Takayoshi Yamashita, Atsushi Shimada
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2504.05716
• Pdf URL: https://arxiv.org/pdf/2504.05716
• Copy Paste: [[2504.05716]] Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment(https://arxiv.org/abs/2504.05716)
• Keywords: generative, large language model
• Abstract: We explore the use of Large Language Models (LLMs) for automated assessment of open-text student reflections and prediction of academic performance. Traditional methods for evaluating reflections are time-consuming and may not scale effectively in educational settings. In this work, we employ LLMs to transform student reflections into quantitative scores using two assessment strategies (single-agent and multi-agent) and two prompting techniques (zero-shot and few-shot). Our experiments, conducted on a dataset of 5,278 reflections from 377 students over three academic terms, demonstrate that the single-agent with few-shot strategy achieves the highest match rate with human evaluations. Furthermore, models utilizing LLM-assessed reflection scores outperform baselines in both at-risk student identification and grade prediction tasks. These findings suggest that LLMs can effectively automate reflection assessment, reduce educators' workload, and enable timely support for students who may need additional assistance. Our work emphasizes the potential of integrating advanced generative AI technologies into educational practices to enhance student engagement and academic success.

Title: QEMesh: Employing A Quadric Error Metrics-Based Representation for Mesh Generation

• Authors: Jiaqi Li, Ruowei Wang, Yu Liu, Qijun Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05720
• Pdf URL: https://arxiv.org/pdf/2504.05720
• Copy Paste: [[2504.05720]] QEMesh: Employing A Quadric Error Metrics-Based Representation for Mesh Generation(https://arxiv.org/abs/2504.05720)
• Keywords: diffusion, generative
• Abstract: Mesh generation plays a crucial role in 3D content creation, as mesh is widely used in various industrial applications. Recent works have achieved impressive results but still face several issues, such as unrealistic patterns or pits on surfaces, thin parts missing, and incomplete structures. Most of these problems stem from the choice of shape representation or the capabilities of the generative network. To alleviate these, we extend PoNQ, a Quadric Error Metrics (QEM)-based representation, and propose a novel model, QEMesh, for high-quality mesh generation. PoNQ divides the shape surface into tiny patches, each represented by a point with its normal and QEM matrix, which preserves fine local geometry information. In our QEMesh, we regard these elements as generable parameters and design a unique latent diffusion model containing a novel multi-decoder VAE for PoNQ parameters generation. Given the latent code generated by the diffusion model, three parameter decoders produce several PoNQ parameters within each voxel cell, and an occupancy decoder predicts which voxel cells containing parameters to form the final shape. Extensive evaluations demonstrate that our method generates results with watertight surfaces and is comparable to state-of-the-art methods in several main metrics.

Title: LLM$\times$MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources

• Authors: Haoyu Wang, Yujia Fu, Zhu Zhang, Shuo Wang, Zirui Ren, Xiaorong Wang, Zhili Li, Chaoqun He, Bo An, Zhiyuan Liu, Maosong Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05732
• Pdf URL: https://arxiv.org/pdf/2504.05732
• Copy Paste: [[2504.05732]] LLM$\times$MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources(https://arxiv.org/abs/2504.05732)
• Keywords: large language model
• Abstract: Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation. While short-to-long generations have received considerable attention, generating long texts from extremely long resources remains relatively underexplored. The primary challenge in long-to-long generation lies in effectively integrating and analyzing relevant information from extensive inputs, which remains difficult for current large language models (LLMs). In this paper, we propose LLM$\times$MapReduce-V2, a novel test-time scaling strategy designed to enhance the ability of LLMs to process extremely long inputs. Drawing inspiration from convolutional neural networks, which iteratively integrate local features into higher-level global representations, LLM$\times$MapReduce-V2 utilizes stacked convolutional scaling layers to progressively expand the understanding of input materials. Both quantitative and qualitative experimental results demonstrate that our approach substantially enhances the ability of LLMs to process long inputs and generate coherent, informative long-form articles, outperforming several representative baselines.

Title: Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring

• Authors: Yida Cai, Kun Liang, Sanwoo Lee, Qinghan Wang, Yunfang Wu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05736
• Pdf URL: https://arxiv.org/pdf/2504.05736
• Copy Paste: [[2504.05736]] Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring(https://arxiv.org/abs/2504.05736)
• Keywords: large language model
• Abstract: In recent years, large language models (LLMs) achieve remarkable success across a variety of tasks. However, their potential in the domain of Automated Essay Scoring (AES) remains largely underexplored. Moreover, compared to English data, the methods for Chinese AES is not well developed. In this paper, we propose Rank-Then-Score (RTS), a fine-tuning framework based on large language models to enhance their essay scoring capabilities. Specifically, we fine-tune the ranking model (Ranker) with feature-enriched data, and then feed the output of the ranking model, in the form of a candidate score set, with the essay content into the scoring model (Scorer) to produce the final score. Experimental results on two benchmark datasets, HSK and ASAP, demonstrate that RTS consistently outperforms the direct prompting (Vanilla) method in terms of average QWK across all LLMs and datasets, and achieves the best performance on Chinese essay scoring using the HSK dataset.

Title: DDT: Decoupled Diffusion Transformer

• Authors: Shuai Wang, Zhi Tian, Weilin Huang, Limin Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05741
• Pdf URL: https://arxiv.org/pdf/2504.05741
• Copy Paste: [[2504.05741]] DDT: Decoupled Diffusion Transformer(https://arxiv.org/abs/2504.05741)
• Keywords: extraction, diffusion, transformer
• Abstract: Diffusion transformers have demonstrated remarkable generation quality, albeit requiring longer training iterations and numerous inference steps. In each denoising step, diffusion transformers encode the noisy inputs to extract the lower-frequency semantic component and then decode the higher frequency with identical modules. This scheme creates an inherent optimization dilemma: encoding low-frequency semantics necessitates reducing high-frequency components, creating tension between semantic encoding and high-frequency decoding. To resolve this challenge, we propose a new \textbf{\color{ddt}D}ecoupled \textbf{\color{ddt}D}iffusion \textbf{\color{ddt}T}ransformer~(\textbf{\color{ddt}DDT}), with a decoupled design of a dedicated condition encoder for semantic extraction alongside a specialized velocity decoder. Our experiments reveal that a more substantial encoder yields performance improvements as model size increases. For ImageNet $256\times256$, Our DDT-XL/2 achieves a new state-of-the-art performance of {1.31 FID}~(nearly $4\times$ faster training convergence compared to previous diffusion transformers). For ImageNet $512\times512$, Our DDT-XL/2 achieves a new state-of-the-art FID of 1.28. Additionally, as a beneficial by-product, our decoupled architecture enhances inference speed by enabling the sharing self-condition between adjacent denoising steps. To minimize performance degradation, we propose a novel statistical dynamic programming approach to identify optimal sharing strategies.

Title: SEA-LION: Southeast Asian Languages in One Network

• Authors: Raymond Ng, Thanh Ngan Nguyen, Yuli Huang, Ngee Chia Tai, Wai Yi Leong, Wei Qi Leong, Xianbin Yong, Jian Gang Ngui, Yosephine Susanto, Nicholas Cheng, Hamsawardhini Rengarajan, Peerat Limkonchotiwat, Adithya Venkatadri Hulagadri, Kok Wai Teng, Yeo Yeow Tong, Bryan Siow, Wei Yi Teo, Wayne Lau, Choon Meng Tan, Brandon Ong, Zhi Hao Ong, Jann Railey Montalan, Adwin Chan, Sajeban Antonyrex, Ren Lee, Esther Choa, David Ong Tat-Wee, Bing Jie Darius Liu, William Chandra Tjhi, Erik Cambria, Leslie Teo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05747
• Pdf URL: https://arxiv.org/pdf/2504.05747
• Copy Paste: [[2504.05747]] SEA-LION: Southeast Asian Languages in One Network(https://arxiv.org/abs/2504.05747)
• Keywords: large language model
• Abstract: Recently, Large Language Models (LLMs) have dominated much of the artificial intelligence scene with their ability to process and generate natural languages. However, the majority of LLM research and development remains English-centric, leaving low-resource languages such as those in the Southeast Asian (SEA) region under-represented. To address this representation gap, we introduce Llama-SEA-LION-v3-8B-IT and Gemma-SEA-LION-v3-9B-IT, two cutting-edge multilingual LLMs designed for SEA languages. The SEA-LION family of LLMs supports 11 SEA languages, namely English, Chinese, Indonesian, Vietnamese, Malay, Thai, Burmese, Lao, Filipino, Tamil, and Khmer. Our work leverages large-scale multilingual continued pre-training with a comprehensive post-training regime involving multiple stages of instruction fine-tuning, alignment, and model merging. Evaluation results on multilingual benchmarks indicate that our models achieve state-of-the-art performance across LLMs supporting SEA languages. We open-source the models to benefit the wider SEA community.

Title: InvNeRF-Seg: Fine-Tuning a Pre-Trained NeRF for 3D Object Segmentation

• Authors: Jiangsan Zhao, Jakob Geipel, Krzysztof Kusnierek, Xuean Cui
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05751
• Pdf URL: https://arxiv.org/pdf/2504.05751
• Copy Paste: [[2504.05751]] InvNeRF-Seg: Fine-Tuning a Pre-Trained NeRF for 3D Object Segmentation(https://arxiv.org/abs/2504.05751)
• Keywords: segmentation
• Abstract: Neural Radiance Fields (NeRF) have been widely adopted for reconstructing high quality 3D point clouds from 2D RGB images. However, the segmentation of these reconstructed 3D scenes is more essential for downstream tasks such as object counting, size estimation, and scene understanding. While segmentation on raw 3D point clouds using deep learning requires labor intensive and time-consuming manual annotation, directly training NeRF on binary masks also fails due to the absence of color and shading cues essential for geometry learning. We propose Invariant NeRF for Segmentation (InvNeRFSeg), a two step, zero change fine tuning strategy for 3D segmentation. We first train a standard NeRF on RGB images and then fine tune it using 2D segmentation masks without altering either the model architecture or loss function. This approach produces higher quality, cleaner segmented point clouds directly from the refined radiance field with minimal computational overhead or complexity. Field density analysis reveals consistent semantic refinement: densities of object regions increase while background densities are suppressed, ensuring clean and interpretable segmentations. We demonstrate InvNeRFSegs superior performance over both SA3D and FruitNeRF on both synthetic fruit and real world soybean datasets. This approach effectively extends 2D segmentation to high quality 3D segmentation.

Title: Interpretable Non-linear Survival Analysis with Evolutionary Symbolic Regression

• Authors: Luigi Rovito, Marco Virgolin
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2504.05756
• Pdf URL: https://arxiv.org/pdf/2504.05756
• Copy Paste: [[2504.05756]] Interpretable Non-linear Survival Analysis with Evolutionary Symbolic Regression(https://arxiv.org/abs/2504.05756)
• Keywords: interpretability
• Abstract: Survival Regression (SuR) is a key technique for modeling time to event in important applications such as clinical trials and semiconductor manufacturing. Currently, SuR algorithms belong to one of three classes: non-linear black-box -- allowing adaptability to many datasets but offering limited interpretability (e.g., tree ensembles); linear glass-box -- being easier to interpret but limited to modeling only linear interactions (e.g., Cox proportional hazards); and non-linear glass-box -- allowing adaptability and interpretability, but empirically found to have several limitations (e.g., explainable boosting machines, survival trees). In this work, we investigate whether Symbolic Regression (SR), i.e., the automated search of mathematical expressions from data, can lead to non-linear glass-box survival models that are interpretable and accurate. We propose an evolutionary, multi-objective, and multi-expression implementation of SR adapted to SuR. Our empirical results on five real-world datasets show that SR consistently outperforms traditional glass-box methods for SuR in terms of accuracy per number of dimensions in the model, while exhibiting comparable accuracy with black-box methods. Furthermore, we offer qualitative examples to assess the interpretability potential of SR models for SuR. Code at: this https URL.

Title: AiGAS-dEVL-RC: An Adaptive Growing Neural Gas Model for Recurrently Drifting Unsupervised Data Streams

• Authors: Maria Arostegi, Miren Nekane Bilbao, Jesus L. Lobo, Javier Del Ser
• Subjects: cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2504.05761
• Pdf URL: https://arxiv.org/pdf/2504.05761
• Copy Paste: [[2504.05761]] AiGAS-dEVL-RC: An Adaptive Growing Neural Gas Model for Recurrently Drifting Unsupervised Data Streams(https://arxiv.org/abs/2504.05761)
• Keywords: robust
• Abstract: Concept drift and extreme verification latency pose significant challenges in data stream learning, particularly when dealing with recurring concept changes in dynamic environments. This work introduces a novel method based on the Growing Neural Gas (GNG) algorithm, designed to effectively handle abrupt recurrent drifts while adapting to incrementally evolving data distributions (incremental drifts). Leveraging the self-organizing and topological adaptability of GNG, the proposed approach maintains a compact yet informative memory structure, allowing it to efficiently store and retrieve knowledge of past or recurring concepts, even under conditions of delayed or sparse stream supervision. Our experiments highlight the superiority of our approach over existing data stream learning methods designed to cope with incremental non-stationarities and verification latency, demonstrating its ability to quickly adapt to new drifts, robustly manage recurring patterns, and maintain high predictive accuracy with a minimal memory footprint. Unlike other techniques that fail to leverage recurring knowledge, our proposed approach is proven to be a robust and efficient online learning solution for unsupervised drifting data flows.

Title: Cross-Document Contextual Coreference Resolution in Knowledge Graphs

• Authors: Zhang Dong, Mingbang Wang, Songhang deng, Le Dai, Jiyuan Li, Xingzu Liu, Ruilin Nong
• Subjects: cs.CL, cs.MA
• Abstract URL: https://arxiv.org/abs/2504.05767
• Pdf URL: https://arxiv.org/pdf/2504.05767
• Copy Paste: [[2504.05767]] Cross-Document Contextual Coreference Resolution in Knowledge Graphs(https://arxiv.org/abs/2504.05767)
• Keywords: extraction
• Abstract: Coreference resolution across multiple documents poses a significant challenge in natural language processing, particularly within the domain of knowledge graphs. This study introduces an innovative method aimed at identifying and resolving references to the same entities that appear across differing texts, thus enhancing the coherence and collaboration of information. Our method employs a dynamic linking mechanism that associates entities in the knowledge graph with their corresponding textual mentions. By utilizing contextual embeddings along with graph-based inference strategies, we effectively capture the relationships and interactions among entities, thereby improving the accuracy of coreference resolution. Rigorous evaluations on various benchmark datasets highlight notable advancements in our approach over traditional methodologies. The results showcase how the contextual information derived from knowledge graphs enhances the understanding of complex relationships across documents, leading to better entity linking and information extraction capabilities in applications driven by knowledge. Our technique demonstrates substantial improvements in both precision and recall, underscoring its effectiveness in the area of cross-document coreference resolution.

Title: A Lightweight Multi-Module Fusion Approach for Korean Character Recognition

• Authors: Inho Jake Park, Jaehoon Jay Jeong, Ho-Sang Jo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05770
• Pdf URL: https://arxiv.org/pdf/2504.05770
• Copy Paste: [[2504.05770]] A Lightweight Multi-Module Fusion Approach for Korean Character Recognition(https://arxiv.org/abs/2504.05770)
• Keywords: robust, extraction
• Abstract: Optical Character Recognition (OCR) is essential in applications such as document processing, license plate recognition, and intelligent surveillance. However, existing OCR models often underperform in real-world scenarios due to irregular text layouts, poor image quality, character variability, and high computational costs. This paper introduces SDA-Net (Stroke-Sensitive Attention and Dynamic Context Encoding Network), a lightweight and efficient architecture designed for robust single-character recognition. SDA-Net incorporates: (1) a Dual Attention Mechanism to enhance stroke-level and spatial feature extraction; (2) a Dynamic Context Encoding module that adaptively refines semantic information using a learnable gating mechanism; (3) a U-Net-inspired Feature Fusion Strategy for combining low-level and high-level features; and (4) a highly optimized lightweight backbone that reduces memory and computational demands. Experimental results show that SDA-Net achieves state-of-the-art accuracy on challenging OCR benchmarks, with significantly faster inference, making it well-suited for deployment in real-time and edge-based OCR systems.

Title: Transferable Mask Transformer: Cross-domain Semantic Segmentation with Region-adaptive Transferability Estimation

• Authors: Enming Zhang, Zhengyu Li, Yanru Wu, Jingge Wang, Yang Tan, Ruizhe Zhao, Guan Wang, Yang Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05774
• Pdf URL: https://arxiv.org/pdf/2504.05774
• Copy Paste: [[2504.05774]] Transferable Mask Transformer: Cross-domain Semantic Segmentation with Region-adaptive Transferability Estimation(https://arxiv.org/abs/2504.05774)
• Keywords: transformer, segmentation
• Abstract: Recent advances in Vision Transformers (ViTs) have set new benchmarks in semantic segmentation. However, when adapting pretrained ViTs to new target domains, significant performance degradation often occurs due to distribution shifts, resulting in suboptimal global attention. Since self-attention mechanisms are inherently data-driven, they may fail to effectively attend to key objects when source and target domains exhibit differences in texture, scale, or object co-occurrence patterns. While global and patch-level domain adaptation methods provide partial solutions, region-level adaptation with dynamically shaped regions is crucial due to spatial heterogeneity in transferability across different image areas. We present Transferable Mask Transformer (TMT), a novel region-level adaptation framework for semantic segmentation that aligns cross-domain representations through spatial transferability analysis. TMT consists of two key components: (1) An Adaptive Cluster-based Transferability Estimator (ACTE) that dynamically segments images into structurally and semantically coherent regions for localized transferability assessment, and (2) A Transferable Masked Attention (TMA) module that integrates region-specific transferability maps into ViTs' attention mechanisms, prioritizing adaptation in regions with low transferability and high semantic uncertainty. Comprehensive evaluations across 20 cross-domain pairs demonstrate TMT's superiority, achieving an average 2% MIoU improvement over vanilla fine-tuning and a 1.28% increase compared to state-of-the-art baselines. The source code will be publicly available.

Title: MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

• Authors: Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05782
• Pdf URL: https://arxiv.org/pdf/2504.05782
• Copy Paste: [[2504.05782]] MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models(https://arxiv.org/abs/2504.05782)
• Keywords: robust, large language model
• Abstract: Multimodal reasoning, which integrates language and visual cues into problem solving and decision making, is a fundamental aspect of human intelligence and a crucial step toward artificial general intelligence. However, the evaluation of multimodal reasoning capabilities in Multimodal Large Language Models (MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained by limited data size, narrow domain coverage, and unstructured knowledge distribution. To close these gaps, we introduce MDK12-Bench, a multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via real-world K-12 examinations. Spanning six disciplines (math, physics, chemistry, biology, geography, and information science), our benchmark comprises 140K reasoning instances across diverse difficulty levels from primary school to 12th grade. It features 6,827 instance-level knowledge point annotations based on a well-organized knowledge structure, detailed answer explanations, difficulty labels and cross-year partitions, providing a robust platform for comprehensive evaluation. Additionally, we present a novel dynamic evaluation framework to mitigate data contamination issues by bootstrapping question forms, question types, and image styles during evaluation. Extensive experiment on MDK12-Bench reveals the significant limitation of current MLLMs in multimodal reasoning. The findings on our benchmark provide insights into the development of the next-generation models. Our data and codes are available at this https URL.

Title: Video Flow as Time Series: Discovering Temporal Consistency and Variability for VideoQA

• Authors: Zijie Song, Zhenzhen Hu, Yixiao Ma, Jia Li, Richang Hong
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05783
• Pdf URL: https://arxiv.org/pdf/2504.05783
• Copy Paste: [[2504.05783]] Video Flow as Time Series: Discovering Temporal Consistency and Variability for VideoQA(https://arxiv.org/abs/2504.05783)
• Keywords: transformer
• Abstract: Video Question Answering (VideoQA) is a complex video-language task that demands a sophisticated understanding of both visual content and temporal dynamics. Traditional Transformer-style architectures, while effective in integrating multimodal data, often simplify temporal dynamics through positional encoding and fail to capture non-linear interactions within video sequences. In this paper, we introduce the Temporal Trio Transformer (T3T), a novel architecture that models time consistency and time variability. The T3T integrates three key components: Temporal Smoothing (TS), Temporal Difference (TD), and Temporal Fusion (TF). The TS module employs Brownian Bridge for capturing smooth, continuous temporal transitions, while the TD module identifies and encodes significant temporal variations and abrupt changes within the video content. Subsequently, the TF module synthesizes these temporal features with textual cues, facilitating a deeper contextual understanding and response accuracy. The efficacy of the T3T is demonstrated through extensive testing on multiple VideoQA benchmark datasets. Our results underscore the importance of a nuanced approach to temporal modeling in improving the accuracy and depth of video-based question answering.

Title: How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLM

• Authors: Jirong Zha, Yuxuan Fan, Xiao Yang, Chen Gao, Xinlei Chen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05786
• Pdf URL: https://arxiv.org/pdf/2504.05786
• Copy Paste: [[2504.05786]] How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLM(https://arxiv.org/abs/2504.05786)
• Keywords: large language model
• Abstract: 3D spatial understanding is essential in real-world applications such as robotics, autonomous vehicles, virtual reality, and medical imaging. Recently, Large Language Models (LLMs), having demonstrated remarkable success across various domains, have been leveraged to enhance 3D understanding tasks, showing potential to surpass traditional computer vision methods. In this survey, we present a comprehensive review of methods integrating LLMs with 3D spatial understanding. We propose a taxonomy that categorizes existing methods into three branches: image-based methods deriving 3D understanding from 2D visual data, point cloud-based methods working directly with 3D representations, and hybrid modality-based methods combining multiple data streams. We systematically review representative methods along these categories, covering data representations, architectural modifications, and training strategies that bridge textual and 3D modalities. Finally, we discuss current limitations, including dataset scarcity and computational challenges, while highlighting promising research directions in spatial perception, multi-modal fusion, and real-world applications.

Title: DefMamba: Deformable Visual State Space Model

• Authors: Leiye Liu, Miao Zhang, Jihao Yin, Tingwei Liu, Wei Ji, Yongri Piao, Huchuan Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05794
• Pdf URL: https://arxiv.org/pdf/2504.05794
• Copy Paste: [[2504.05794]] DefMamba: Deformable Visual State Space Model(https://arxiv.org/abs/2504.05794)
• Keywords: extraction, segmentation
• Abstract: Recently, state space models (SSM), particularly Mamba, have attracted significant attention from scholars due to their ability to effectively balance computational efficiency and performance. However, most existing visual Mamba methods flatten images into 1D sequences using predefined scan orders, which results the model being less capable of utilizing the spatial structural information of the image during the feature extraction process. To address this issue, we proposed a novel visual foundation model called DefMamba. This model includes a multi-scale backbone structure and deformable mamba (DM) blocks, which dynamically adjust the scanning path to prioritize important information, thus enhancing the capture and processing of relevant input features. By combining a deformable scanning(DS) strategy, this model significantly improves its ability to learn image structures and detects changes in object details. Numerous experiments have shown that DefMamba achieves state-of-the-art performance in various visual tasks, including image classification, object detection, instance segmentation, and semantic segmentation. The code is open source on DefMamba.

Title: Robust Fusion Controller: Degradation-aware Image Fusion with Fine-grained Language Instructions

• Authors: Hao Zhang, Yanping Zha, Qingwei Zhuang, Zhenfeng Shao, Jiayi Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05795
• Pdf URL: https://arxiv.org/pdf/2504.05795
• Copy Paste: [[2504.05795]] Robust Fusion Controller: Degradation-aware Image Fusion with Fine-grained Language Instructions(https://arxiv.org/abs/2504.05795)
• Keywords: robust
• Abstract: Current image fusion methods struggle to adapt to real-world environments encompassing diverse degradations with spatially varying characteristics. To address this challenge, we propose a robust fusion controller (RFC) capable of achieving degradation-aware image fusion through fine-grained language instructions, ensuring its reliable application in adverse environments. Specifically, RFC first parses language instructions to innovatively derive the functional condition and the spatial condition, where the former specifies the degradation type to remove, while the latter defines its spatial coverage. Then, a composite control priori is generated through a multi-condition coupling network, achieving a seamless transition from abstract language instructions to latent control variables. Subsequently, we design a hybrid attention-based fusion network to aggregate multi-modal information, in which the obtained composite control priori is deeply embedded to linearly modulate the intermediate fused features. To ensure the alignment between language instructions and control outcomes, we introduce a novel language-feature alignment loss, which constrains the consistency between feature-level gains and the composite control priori. Extensive experiments on publicly available datasets demonstrate that our RFC is robust against various composite degradations, particularly in highly challenging flare scenarios.

Title: Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling

• Authors: Jaskirat Singh, Junshen Kevin Chen, Jonas Kohler, Michael Cohen
• Subjects: cs.CV, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.05800
• Pdf URL: https://arxiv.org/pdf/2504.05800
• Copy Paste: [[2504.05800]] Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling(https://arxiv.org/abs/2504.05800)
• Keywords: diffusion
• Abstract: Training-free consistent text-to-image generation depicting the same subjects across different images is a topic of widespread recent interest. Existing works in this direction predominantly rely on cross-frame self-attention; which improves subject-consistency by allowing tokens in each frame to pay attention to tokens in other frames during self-attention computation. While useful for single subjects, we find that it struggles when scaling to multiple characters. In this work, we first analyze the reason for these limitations. Our exploration reveals that the primary-issue stems from self-attention-leakage, which is exacerbated when trying to ensure consistency across multiple-characters. This happens when tokens from one subject pay attention to other characters, causing them to appear like each other (e.g., a dog appearing like a duck). Motivated by these findings, we propose StoryBooth: a training-free approach for improving multi-character consistency. In particular, we first leverage multi-modal chain-of-thought reasoning and region-based generation to apriori localize the different subjects across the desired story outputs. The final outputs are then generated using a modified diffusion model which consists of two novel layers: 1) a bounded cross-frame self-attention layer for reducing inter-character attention leakage, and 2) token-merging layer for improving consistency of fine-grain subject details. Through both qualitative and quantitative results we find that the proposed approach surpasses prior state-of-the-art, exhibiting improved consistency across both multiple-characters and fine-grain subject details.

Title: PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning

• Authors: Xinpeng Ding, Kui Zhang, Jinahua Han, Lanqing Hong, Hang Xu, Xiaomeng Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05810
• Pdf URL: https://arxiv.org/pdf/2504.05810
• Copy Paste: [[2504.05810]] PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning(https://arxiv.org/abs/2504.05810)
• Keywords: large language model
• Abstract: Direct Preference Optimization (DPO) helps reduce hallucinations in Video Multimodal Large Language Models (VLLMs), but its reliance on offline preference data limits adaptability and fails to capture true video-response misalignment. We propose Video Direct Preference Optimization (VDPO), an online preference learning framework that eliminates the need for preference annotation by leveraging video augmentations to generate rejected samples while keeping responses fixed. However, selecting effective augmentations is non-trivial, as some clips may be semantically identical to the original under specific prompts, leading to false rejections and disrupting alignment. To address this, we introduce Prompt-aware Multi-instance Learning VDPO (PaMi-VDPO), which selects augmentations based on prompt context. Instead of a single rejection, we construct a candidate set of augmented clips and apply a close-to-far selection strategy, initially ensuring all clips are semantically relevant while then prioritizing the most prompt-aware distinct clip. This allows the model to better capture meaningful visual differences, mitigating hallucinations, while avoiding false rejections, and improving alignment. PaMi-VDPOseamlessly integrates into existing VLLMs without additional parameters, GPT-4/human supervision. With only 10k SFT data, it improves the base model by 5.3% on VideoHallucer, surpassing GPT-4o, while maintaining stable performance on general video benchmarks.

Title: Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization

• Authors: Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, Yatao Bian
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05812
• Pdf URL: https://arxiv.org/pdf/2504.05812
• Copy Paste: [[2504.05812]] Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization(https://arxiv.org/abs/2504.05812)
• Keywords: large language model
• Abstract: While large language models (LLMs) have demonstrated exceptional capabilities in challenging tasks such as mathematical reasoning, existing methods to enhance reasoning ability predominantly rely on supervised fine-tuning (SFT) followed by reinforcement learning (RL) on reasoning-specific data after pre-training. However, these approaches critically depend on external supervisions--such as human labelled reasoning traces, verified golden answers, or pre-trained reward models--which limits scalability and practical applicability. In this work, we propose Entropy Minimized Policy Optimization (EMPO), which makes an early attempt at fully unsupervised LLM reasoning incentivization. EMPO does not require any supervised information for incentivizing reasoning capabilities (i.e., neither verifiable reasoning traces, problems with golden answers, nor additional pre-trained reward models). By continuously minimizing the predictive entropy of LLMs on unlabeled user queries in a latent semantic space, EMPO enables purely self-supervised evolution of reasoning capabilities with strong flexibility and practicality. Our experiments demonstrate competitive performance of EMPO on both mathematical reasoning and free-form commonsense reasoning tasks. Specifically, without any supervised signals, EMPO boosts the accuracy of Qwen2.5-Math-7B Base from 30.7\% to 48.1\% on mathematical benchmarks and improves truthfulness accuracy of Qwen2.5-7B Instruct from 87.16\% to 97.25\% on TruthfulQA.

Title: Parasite: A Steganography-based Backdoor Attack Framework for Diffusion Models

• Authors: Jiahao Chen, Yu Pan, Yi Du, Chunkai Wu, Lin Wang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05815
• Pdf URL: https://arxiv.org/pdf/2504.05815
• Copy Paste: [[2504.05815]] Parasite: A Steganography-based Backdoor Attack Framework for Diffusion Models(https://arxiv.org/abs/2504.05815)
• Keywords: defense, attack, diffusion
• Abstract: Recently, the diffusion model has gained significant attention as one of the most successful image generation models, which can generate high-quality images by iteratively sampling noise. However, recent studies have shown that diffusion models are vulnerable to backdoor attacks, allowing attackers to enter input data containing triggers to activate the backdoor and generate their desired output. Existing backdoor attack methods primarily focused on target noise-to-image and text-to-image tasks, with limited work on backdoor attacks in image-to-image tasks. Furthermore, traditional backdoor attacks often rely on a single, conspicuous trigger to generate a fixed target image, lacking concealability and flexibility. To address these limitations, we propose a novel backdoor attack method called "Parasite" for image-to-image tasks in diffusion models, which not only is the first to leverage steganography for triggers hiding, but also allows attackers to embed the target content as a backdoor trigger to achieve a more flexible attack. "Parasite" as a novel attack method effectively bypasses existing detection frameworks to execute backdoor attacks. In our experiments, "Parasite" achieved a 0 percent backdoor detection rate against the mainstream defense frameworks. In addition, in the ablation study, we discuss the influence of different hiding coefficients on the attack results. You can find our code at this https URL.

Title: Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradients

• Authors: Alessio Mora, Carlo Mazzocca, Rebecca Montanari, Paolo Bellavista
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05822
• Pdf URL: https://arxiv.org/pdf/2504.05822
• Copy Paste: [[2504.05822]] Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradients(https://arxiv.org/abs/2504.05822)
• Keywords: privacy, federate, transformer, segmentation
• Abstract: The right to be forgotten is a fundamental principle of privacy-preserving regulations and extends to Machine Learning (ML) paradigms such as Federated Learning (FL). While FL enhances privacy by enabling collaborative model training without sharing private data, trained models still retain the influence of training data. Federated Unlearning (FU) methods recently proposed often rely on impractical assumptions for real-world FL deployments, such as storing client update histories or requiring access to a publicly available dataset. To address these constraints, this paper introduces a novel method that leverages negated Pseudo-gradients Updates for Federated Unlearning (PUF). Our approach only uses standard client model updates, anyway employed during regular FL rounds, and interprets them as pseudo-gradients. When a client needs to be forgotten, we apply the negated of their pseudo-gradients, appropriately scaled, to the global model. Unlike state-of-the-art mechanisms, PUF seamlessly integrates with FL workflows, incurs no additional computational and communication overhead beyond standard FL rounds, and supports concurrent unlearning requests. We extensively evaluated the proposed method on two well-known benchmark image classification datasets (CIFAR-10 and CIFAR-100) and a real-world medical imaging dataset for segmentation (ProstateMRI), using three different neural architectures: two residual networks and a vision transformer. The experimental results across various settings demonstrate that PUF achieves state-of-the-art forgetting effectiveness and recovery time, without relying on any additional assumptions, thus underscoring its practical applicability.

Title: Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset

• Authors: Shiao Wang, Xiao Wang, Bo Jiang, Lin Zhu, Guoqi Li, Yaowei Wang, Yonghong Tian, Jin Tang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05830
• Pdf URL: https://arxiv.org/pdf/2504.05830
• Copy Paste: [[2504.05830]] Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset(https://arxiv.org/abs/2504.05830)
• Keywords: robust
• Abstract: Human Activity Recognition (HAR) primarily relied on traditional RGB cameras to achieve high-performance activity recognition. However, the challenging factors in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras. To address these challenges, biologically inspired event cameras offer a promising solution to overcome the limitations of traditional RGB cameras. In this work, we rethink human activity recognition by combining the RGB and event cameras. The first contribution is the proposed large-scale multi-modal RGB-Event human activity recognition benchmark dataset, termed HARDVS 2.0, which bridges the dataset gaps. It contains 300 categories of everyday real-world actions with a total of 107,646 paired videos covering various challenging scenarios. Inspired by the physics-informed heat conduction model, we propose a novel multi-modal heat conduction operation framework for effective activity recognition, termed MMHCO-HAR. More in detail, given the RGB frames and event streams, we first extract the feature embeddings using a stem network. Then, multi-modal Heat Conduction blocks are designed to fuse the dual features, the key module of which is the multi-modal Heat Conduction Operation layer. We integrate RGB and event embeddings through a multi-modal DCT-IDCT layer while adaptively incorporating the thermal conductivity coefficient via FVEs into this module. After that, we propose an adaptive fusion module based on a policy routing strategy for high-performance classification. Comprehensive experiments demonstrate that our method consistently performs well, validating its effectiveness and robustness. The source code and benchmark dataset will be released on this https URL

Title: Leveraging Robust Optimization for LLM Alignment under Distribution Shifts

• Authors: Mingye Zhu, Yi Liu, Junbo Guo, Quan Wang, Yongdong Zhang, Zhendong Mao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05831
• Pdf URL: https://arxiv.org/pdf/2504.05831
• Copy Paste: [[2504.05831]] Leveraging Robust Optimization for LLM Alignment under Distribution Shifts(https://arxiv.org/abs/2504.05831)
• Keywords: robust, large language model
• Abstract: Large language models (LLMs) increasingly rely on preference alignment methods to steer outputs toward human values, yet these methods are often constrained by the scarcity of high-quality human-annotated data. To tackle this, recent approaches have turned to synthetic data generated by LLMs as a scalable alternative. However, synthetic data can introduce distribution shifts, compromising the nuanced human preferences that are essential for desirable outputs. In this paper, we propose a novel distribution-aware optimization framework that improves preference alignment in the presence of such shifts. Our approach first estimates the likelihood ratios between the target and training distributions leveraging a learned classifier, then it minimizes the worst-case loss over data regions that reflect the target human-preferred distribution. By explicitly prioritizing the target distribution during optimization, our method mitigates the adverse effects of distributional variation and enhances the generation of responses that faithfully reflect human values.

Title: Channel State Information Analysis for Jamming Attack Detection in Static and Dynamic UAV Networks -- An Experimental Study

• Authors: Pavlo Mykytyn, Ronald Chitauro, Zoya Dyka, Peter Langendoerfer
• Subjects: cs.CR, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.05832
• Pdf URL: https://arxiv.org/pdf/2504.05832
• Copy Paste: [[2504.05832]] Channel State Information Analysis for Jamming Attack Detection in Static and Dynamic UAV Networks -- An Experimental Study(https://arxiv.org/abs/2504.05832)
• Keywords: attack
• Abstract: Networks built on the IEEE 802.11 standard have experienced rapid growth in the last decade. Their field of application is vast, including smart home applications, Internet of Things (IoT), and short-range high throughput static and dynamic inter-vehicular communication networks. Within such networks, Channel State Information (CSI) provides a detailed view of the state of the communication channel and represents the combined effects of multipath propagation, scattering, phase shift, fading, and power decay. In this work, we investigate the problem of jamming attack detection in static and dynamic vehicular networks. We utilize ESP32-S3 modules to set up a communication network between an Unmanned Aerial Vehicle (UAV) and a Ground Control Station (GCS), to experimentally test the combined effects of a constant jammer on recorded CSI parameters, and the feasibility of jamming detection through CSI analysis in static and dynamic communication scenarios.

Title: Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking

• Authors: Junxi Chen, Junhao Dong, Xiaohua Xie
• Subjects: cs.CV, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2504.05838
• Pdf URL: https://arxiv.org/pdf/2504.05838
• Copy Paste: [[2504.05838]] Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking(https://arxiv.org/abs/2504.05838)
• Keywords: defense, attack, diffusion
• Abstract: Recently, the Image Prompt Adapter (IP-Adapter) has been increasingly integrated into text-to-image diffusion models (T2I-DMs) to improve controllability. However, in this paper, we reveal that T2I-DMs equipped with the IP-Adapter (T2I-IP-DMs) enable a new jailbreak attack named the hijacking attack. We demonstrate that, by uploading imperceptible image-space adversarial examples (AEs), the adversary can hijack massive benign users to jailbreak an Image Generation Service (IGS) driven by T2I-IP-DMs and mislead the public to discredit the service provider. Worse still, the IP-Adapter's dependency on open-source image encoders reduces the knowledge required to craft AEs. Extensive experiments verify the technical feasibility of the hijacking attack. In light of the revealed threat, we investigate several existing defenses and explore combining the IP-Adapter with adversarially trained models to overcome existing defenses' limitations. Our code is available at this https URL.

Title: Adaptive Substructure-Aware Expert Model for Molecular Property Prediction

• Authors: Tianyi Jiang, Zeyu Wang, Shanqing Yu, Qi Xuan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05844
• Pdf URL: https://arxiv.org/pdf/2504.05844
• Copy Paste: [[2504.05844]] Adaptive Substructure-Aware Expert Model for Molecular Property Prediction(https://arxiv.org/abs/2504.05844)
• Keywords: interpretability
• Abstract: Molecular property prediction is essential for applications such as drug discovery and toxicity assessment. While Graph Neural Networks (GNNs) have shown promising results by modeling molecules as molecular graphs, their reliance on data-driven learning limits their ability to generalize, particularly in the presence of data imbalance and diverse molecular substructures. Existing methods often overlook the varying contributions of different substructures to molecular properties, treating them uniformly. To address these challenges, we propose ASE-Mol, a novel GNN-based framework that leverages a Mixture-of-Experts (MoE) approach for molecular property prediction. ASE-Mol incorporates BRICS decomposition and significant substructure awareness to dynamically identify positive and negative substructures. By integrating a MoE architecture, it reduces the adverse impact of negative motifs while improving adaptability to positive motifs. Experimental results on eight benchmark datasets demonstrate that ASE-Mol achieves state-of-the-art performance, with significant improvements in both accuracy and interpretability.

Title: On the Importance of Conditioning for Privacy-Preserving Data Augmentation

• Authors: Julian Lorenz, Katja Ludwig, Valentin Haug, Rainer Lienhart
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05849
• Pdf URL: https://arxiv.org/pdf/2504.05849
• Copy Paste: [[2504.05849]] On the Importance of Conditioning for Privacy-Preserving Data Augmentation(https://arxiv.org/abs/2504.05849)
• Keywords: privacy, attack, diffusion
• Abstract: Latent diffusion models can be used as a powerful augmentation method to artificially extend datasets for enhanced training. To the human eye, these augmented images look very different to the originals. Previous work has suggested to use this data augmentation technique for data anonymization. However, we show that latent diffusion models that are conditioned on features like depth maps or edges to guide the diffusion process are not suitable as a privacy preserving method. We use a contrastive learning approach to train a model that can correctly identify people out of a pool of candidates. Moreover, we demonstrate that anonymization using conditioned diffusion models is susceptible to black box attacks. We attribute the success of the described methods to the conditioning of the latent diffusion model in the anonymization process. The diffusion model is instructed to produce similar edges for the anonymized images. Hence, a model can learn to recognize these patterns for identification.

Title: Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics

• Authors: Xingzu Liu, Songhang deng, Mingbang Wang, Zhang Dong, Le Dai, Jiyuan Li, Ruilin Nong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05855
• Pdf URL: https://arxiv.org/pdf/2504.05855
• Copy Paste: [[2504.05855]] Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics(https://arxiv.org/abs/2504.05855)
• Keywords: large language model
• Abstract: Large language models have made significant advancements in various natural language processing tasks, including coreference resolution. However, traditional methods often fall short in effectively distinguishing referential relationships due to a lack of integration between syntactic and semantic information. This study introduces an innovative framework aimed at enhancing coreference resolution by utilizing pretrained language models. Our approach combines syntax parsing with semantic role labeling to accurately capture finer distinctions in referential relationships. By employing state-of-the-art pretrained models to gather contextual embeddings and applying an attention mechanism for fine-tuning, we improve the performance of coreference tasks. Experimental results across diverse datasets show that our method surpasses conventional coreference resolution systems, achieving notable accuracy in disambiguating references. This development not only improves coreference resolution outcomes but also positively impacts other natural language processing tasks that depend on precise referential understanding.

Title: CTI-HAL: A Human-Annotated Dataset for Cyber Threat Intelligence Analysis

• Authors: Sofia Della Penna, Roberto Natella, Vittorio Orbinato, Lorenzo Parracino, Luciano Pianese
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2504.05866
• Pdf URL: https://arxiv.org/pdf/2504.05866
• Copy Paste: [[2504.05866]] CTI-HAL: A Human-Annotated Dataset for Cyber Threat Intelligence Analysis(https://arxiv.org/abs/2504.05866)
• Keywords: security, extraction, large language model
• Abstract: Organizations are increasingly targeted by Advanced Persistent Threats (APTs), which involve complex, multi-stage tactics and diverse techniques. Cyber Threat Intelligence (CTI) sources, such as incident reports and security blogs, provide valuable insights, but are often unstructured and in natural language, making it difficult to automatically extract information. Recent studies have explored the use of AI to perform automatic extraction from CTI data, leveraging existing CTI datasets for performance evaluation and fine-tuning. However, they present challenges and limitations that impact their effectiveness. To overcome these issues, we introduce a novel dataset manually constructed from CTI reports and structured according to the MITRE ATT&CK framework. To assess its quality, we conducted an inter-annotator agreement study using Krippendorff alpha, confirming its reliability. Furthermore, the dataset was used to evaluate a Large Language Model (LLM) in a real-world business context, showing promising generalizability.

Title: Turin3D: Evaluating Adaptation Strategies under Label Scarcity in Urban LiDAR Segmentation with Semi-Supervised Techniques

• Authors: Luca Barco, Giacomo Blanco, Gaetano Chiriaco, Alessia Intini, Luigi La Riccia, Vittorio Scolamiero, Piero Boccardo, Paolo Garza, Fabrizio Dominici
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05882
• Pdf URL: https://arxiv.org/pdf/2504.05882
• Copy Paste: [[2504.05882]] Turin3D: Evaluating Adaptation Strategies under Label Scarcity in Urban LiDAR Segmentation with Semi-Supervised Techniques(https://arxiv.org/abs/2504.05882)
• Keywords: segmentation
• Abstract: 3D semantic segmentation plays a critical role in urban modelling, enabling detailed understanding and mapping of city environments. In this paper, we introduce Turin3D: a new aerial LiDAR dataset for point cloud semantic segmentation covering an area of around 1.43 km2 in the city centre of Turin with almost 70M points. We describe the data collection process and compare Turin3D with others previously proposed in the literature. We did not fully annotate the dataset due to the complexity and time-consuming nature of the process; however, a manual annotation process was performed on the validation and test sets, to enable a reliable evaluation of the proposed techniques. We first benchmark the performances of several point cloud semantic segmentation models, trained on the existing datasets, when tested on Turin3D, and then improve their performances by applying a semi-supervised learning technique leveraging the unlabelled training set. The dataset will be publicly available to support research in outdoor point cloud segmentation, with particular relevance for self-supervised and semi-supervised learning approaches given the absence of ground truth annotations for the training set.

Title: HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference

• Authors: Shuzhang Zhong, Yanfan Sun, Ling Liang, Runsheng Wang, Ru Huang, Meng Li
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2504.05897
• Pdf URL: https://arxiv.org/pdf/2504.05897
• Copy Paste: [[2504.05897]] HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference(https://arxiv.org/abs/2504.05897)
• Keywords: transformer
• Abstract: The Mixture of Experts (MoE) architecture has demonstrated significant advantages as it enables to increase the model capacity without a proportional increase in computation. However, the large MoE model size still introduces substantial memory demands, which usually requires expert offloading on resource-constrained platforms and incurs significant overhead. Hybrid CPU-GPU inference has been proposed to leverage CPU computation to reduce expert loading overhead but faces major challenges: on one hand, the expert activation patterns of MoE models are highly unstable, rendering the fixed mapping strategies in existing works inefficient; on the other hand, the hybrid CPU-GPU schedule for MoE is inherently complex due to the diverse expert sizes, structures, uneven workload distribution, etc. To address these challenges, in this paper, we propose HybriMoE, a hybrid CPU-GPU inference framework that improves resource utilization through a novel CPU-GPU scheduling and cache management system. HybriMoE introduces (i) a dynamic intra-layer scheduling strategy to balance workloads across CPU and GPU, (ii) an impact-driven inter-layer prefetching algorithm, and (iii) a score-based caching algorithm to mitigate expert activation instability. We implement HybriMoE on top of the kTransformers framework and evaluate it on three widely used MoE-based LLMs. Experimental results demonstrate that HybriMoE achieves an average speedup of 1.33$\times$ in the prefill stage and 1.70$\times$ in the decode stage compared to state-of-the-art hybrid MoE inference framework. Our code is available at: this https URL.

Title: Assessing Thai Dialect Performance in LLMs with Automatic Benchmarks and Human Evaluation

• Authors: Peerat Limkonchotiwat, Kanruethai Masuk, Surapon Nonesung, Chalermpun Mai-On, Sarana Nutanong, Wuttikorn Ponwitayarat, Potsawee Manakul
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05898
• Pdf URL: https://arxiv.org/pdf/2504.05898
• Copy Paste: [[2504.05898]] Assessing Thai Dialect Performance in LLMs with Automatic Benchmarks and Human Evaluation(https://arxiv.org/abs/2504.05898)
• Keywords: robust, large language model
• Abstract: Large language models show promising results in various NLP tasks. Despite these successes, the robustness and consistency of LLMs in underrepresented languages remain largely unexplored, especially concerning local dialects. Existing benchmarks also focus on main dialects, neglecting LLMs' ability on local dialect texts. In this paper, we introduce a Thai local dialect benchmark covering Northern (Lanna), Northeastern (Isan), and Southern (Dambro) Thai, evaluating LLMs on five NLP tasks: summarization, question answering, translation, conversation, and food-related tasks. Furthermore, we propose a human evaluation guideline and metric for Thai local dialects to assess generation fluency and dialect-specific accuracy. Results show that LLM performance declines significantly in local Thai dialects compared to standard Thai, with only proprietary models like GPT-4o and Gemini2 demonstrating some fluency

Title: Defending Deep Neural Networks against Backdoor Attacks via Module Switching

• Authors: Weijun Li, Ansh Arora, Xuanli He, Mark Dras, Qiongkai Xu
• Subjects: cs.CR, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05902
• Pdf URL: https://arxiv.org/pdf/2504.05902
• Copy Paste: [[2504.05902]] Defending Deep Neural Networks against Backdoor Attacks via Module Switching(https://arxiv.org/abs/2504.05902)
• Keywords: security, defense, attack
• Abstract: The exponential increase in the parameters of Deep Neural Networks (DNNs) has significantly raised the cost of independent training, particularly for resource-constrained entities. As a result, there is a growing reliance on open-source models. However, the opacity of training processes exacerbates security risks, making these models more vulnerable to malicious threats, such as backdoor attacks, while simultaneously complicating defense mechanisms. Merging homogeneous models has gained attention as a cost-effective post-training defense. However, we notice that existing strategies, such as weight averaging, only partially mitigate the influence of poisoned parameters and remain ineffective in disrupting the pervasive spurious correlations embedded across model parameters. We propose a novel module-switching strategy to break such spurious correlations within the model's propagation path. By leveraging evolutionary algorithms to optimize fusion strategies, we validate our approach against backdoor attacks targeting text and vision domains. Our method achieves effective backdoor mitigation even when incorporating a couple of compromised models, e.g., reducing the average attack success rate (ASR) to 22% compared to 31.9% with the best-performing baseline on SST-2.

Title: Intrinsic Saliency Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation

• Authors: Xiangyu Zheng, Wanyun Li, Songcheng He, Xiaoqiang Li, We Zhang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05904
• Pdf URL: https://arxiv.org/pdf/2504.05904
• Copy Paste: [[2504.05904]] Intrinsic Saliency Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation(https://arxiv.org/abs/2504.05904)
• Keywords: segmentation
• Abstract: Recent unsupervised video object segmentation (UVOS) methods predominantly adopt the motion-appearance paradigm. Mainstream motion-appearance approaches use either the two-encoder structure to separately encode motion and appearance features, or the single-encoder structure for joint encoding. However, these methods fail to properly balance the motion-appearance relationship. Consequently, even with complex fusion modules for motion-appearance integration, the extracted suboptimal features degrade the models' overall performance. Moreover, the quality of optical flow varies across scenarios, making it insufficient to rely solely on optical flow to achieve high-quality segmentation results. To address these challenges, we propose the Intrinsic Saliency guided Trunk-Collateral Net}work (ISTC-Net), which better balances the motion-appearance relationship and incorporates model's intrinsic saliency information to enhance segmentation performance. Specifically, considering that optical flow maps are derived from RGB images, they share both commonalities and differences. We propose a novel Trunk-Collateral structure. The shared trunk backbone captures the motion-appearance commonality, while the collateral branch learns the uniqueness of motion features. Furthermore, an Intrinsic Saliency guided Refinement Module (ISRM) is devised to efficiently leverage the model's intrinsic saliency information to refine high-level features, and provide pixel-level guidance for motion-appearance fusion, thereby enhancing performance without additional input. Experimental results show that ISTC-Net achieved state-of-the-art performance on three UVOS datasets (89.2% J&F on DAVIS-16, 76% J on YouTube-Objects, 86.4% J on FBMS) and four standard video salient object detection (VSOD) benchmarks with the notable increase, demonstrating its effectiveness and superiority over previous methods.

Title: Balancing long- and short-term dynamics for the modeling of saliency in videos

• Authors: Theodor Wulff, Fares Abawi, Philipp Allgeuer, Stefan Wermter
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05913
• Pdf URL: https://arxiv.org/pdf/2504.05913
• Copy Paste: [[2504.05913]] Balancing long- and short-term dynamics for the modeling of saliency in videos(https://arxiv.org/abs/2504.05913)
• Keywords: transformer
• Abstract: The role of long- and short-term dynamics towards salient object detection in videos is under-researched. We present a Transformer-based approach to learn a joint representation of video frames and past saliency information. Our model embeds long- and short-term information to detect dynamically shifting saliency in video. We provide our model with a stream of video frames and past saliency maps, which acts as a prior for the next prediction, and extract spatiotemporal tokens from both modalities. The decomposition of the frame sequence into tokens lets the model incorporate short-term information from within the token, while being able to make long-term connections between tokens throughout the sequence. The core of the system consists of a dual-stream Transformer architecture to process the extracted sequences independently before fusing the two modalities. Additionally, we apply a saliency-based masking scheme to the input frames to learn an embedding that facilitates the recognition of deviations from previous outputs. We observe that the additional prior information aids in the first detection of the salient location. Our findings indicate that the ratio of spatiotemporal long- and short-term features directly impacts the model's performance. While increasing the short-term context is beneficial up to a certain threshold, the model's performance greatly benefits from an expansion of the long-term context.

Title: High-Resource Translation:Turning Abundance into Accessibility

• Authors: Abhiram Reddy Yanampally
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.05914
• Pdf URL: https://arxiv.org/pdf/2504.05914
• Copy Paste: [[2504.05914]] High-Resource Translation:Turning Abundance into Accessibility(https://arxiv.org/abs/2504.05914)
• Keywords: robust
• Abstract: This paper presents a novel approach to constructing an English-to-Telugu translation model by leveraging transfer learning techniques and addressing the challenges associated with low-resource languages. Utilizing the Bharat Parallel Corpus Collection (BPCC) as the primary dataset, the model incorporates iterative backtranslation to generate synthetic parallel data, effectively augmenting the training dataset and enhancing the model's translation capabilities. The research focuses on a comprehensive strategy for improving model performance through data augmentation, optimization of training parameters, and the effective use of pre-trained models. These methodologies aim to create a robust translation system that can handle diverse sentence structures and linguistic nuances in both English and Telugu. This work highlights the significance of innovative data handling techniques and the potential of transfer learning in overcoming limitations posed by sparse datasets in low-resource languages. The study contributes to the field of machine translation and seeks to improve communication between English and Telugu speakers in practical contexts.

Title: Uncovering Fairness through Data Complexity as an Early Indicator

• Authors: Juliett Suárez Ferreira, Marija Slavkovik, Jorge Casillas
• Subjects: cs.LG, cs.AI, cs.DS
• Abstract URL: https://arxiv.org/abs/2504.05923
• Pdf URL: https://arxiv.org/pdf/2504.05923
• Copy Paste: [[2504.05923]] Uncovering Fairness through Data Complexity as an Early Indicator(https://arxiv.org/abs/2504.05923)
• Keywords: fair
• Abstract: Fairness constitutes a concern within machine learning (ML) applications. Currently, there is no study on how disparities in classification complexity between privileged and unprivileged groups could influence the fairness of solutions, which serves as a preliminary indicator of potential unfairness. In this work, we investigate this gap, specifically, we focus on synthetic datasets designed to capture a variety of biases ranging from historical bias to measurement and representational bias to evaluate how various complexity metrics differences correlate with group fairness metrics. We then apply association rule mining to identify patterns that link disproportionate complexity differences between groups with fairness-related outcomes, offering data-centric indicators to guide bias mitigation. Our findings are also validated by their application in real-world problems, providing evidence that quantifying group-wise classification complexity can uncover early indicators of potential fairness challenges. This investigation helps practitioners to proactively address bias in classification tasks.

Title: SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation

• Authors: Hao Du, Bo Wu, Yan Lu, Zhendong Mao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05925
• Pdf URL: https://arxiv.org/pdf/2504.05925
• Copy Paste: [[2504.05925]] SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation(https://arxiv.org/abs/2504.05925)
• Keywords: fair
• Abstract: Vision-language temporal alignment is a crucial capability for human dynamic recognition and cognition in real-world scenarios. While existing research focuses on capturing vision-language relevance, it faces limitations due to biased temporal distributions, imprecise annotations, and insufficient compositionally. To achieve fair evaluation and comprehensive exploration, our objective is to investigate and evaluate the ability of models to achieve alignment from a temporal perspective, specifically focusing on their capacity to synchronize visual scenarios with linguistic context in a temporally coherent manner. As a preliminary step, we present the statistical analysis of existing benchmarks and reveal the existing challenges from a decomposed perspective. To this end, we introduce SVLTA, the Synthetic Vision-Language Temporal Alignment derived via a well-designed and feasible control generation method within a simulation environment. The approach considers commonsense knowledge, manipulable action, and constrained filtering, which generates reasonable, diverse, and balanced data distributions for diagnostic evaluations. Our experiments reveal diagnostic insights through the evaluations in temporal question answering, distributional shift sensitiveness, and temporal alignment adaptation.

Title: CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics

• Authors: Kuntian Zhang, Simin Yu, Yaoshu Wang, Makoto Onizuka, Chuan Xiao
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05945
• Pdf URL: https://arxiv.org/pdf/2504.05945
• Copy Paste: [[2504.05945]] CKGAN: Training Generative Adversarial Networks Using Characteristic Kernel Integral Probability Metrics(https://arxiv.org/abs/2504.05945)
• Keywords: generative
• Abstract: In this paper, we propose CKGAN, a novel generative adversarial network (GAN) variant based on an integral probability metrics framework with characteristic kernel (CKIPM). CKIPM, as a distance between two probability distributions, is designed to optimize the lowerbound of the maximum mean discrepancy (MMD) in a reproducing kernel Hilbert space, and thus can be used to train GANs. CKGAN mitigates the notorious problem of mode collapse by mapping the generated images back to random noise. To save the effort of selecting the kernel function manually, we propose a soft selection method to automatically learn a characteristic kernel function. The experimental evaluation conducted on a set of synthetic and real image benchmarks (MNIST, CelebA, etc.) demonstrates that CKGAN generally outperforms other MMD-based GANs. The results also show that at the cost of moderately more training time, the automatically selected kernel function delivers very close performance to the best of manually fine-tuned one on real image benchmarks and is able to improve the performances of other MMD-based GANs.

Title: Unsupervised Location Mapping for Narrative Corpora

• Authors: Eitan Wagner, Renana Keydar, Omri Abend
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.05954
• Pdf URL: https://arxiv.org/pdf/2504.05954
• Copy Paste: [[2504.05954]] Unsupervised Location Mapping for Narrative Corpora(https://arxiv.org/abs/2504.05954)
• Keywords: large language model
• Abstract: This work presents the task of unsupervised location mapping, which seeks to map the trajectory of an individual narrative on a spatial map of locations in which a large set of narratives take place. Despite the fundamentality and generality of the task, very little work addressed the spatial mapping of narrative texts. The task consists of two parts: (1) inducing a ``map'' with the locations mentioned in a set of texts, and (2) extracting a trajectory from a single narrative and positioning it on the map. Following recent advances in increasing the context length of large language models, we propose a pipeline for this task in a completely unsupervised manner without predefining the set of labels. We test our method on two different domains: (1) Holocaust testimonies and (2) Lake District writing, namely multi-century literature on travels in the English Lake District. We perform both intrinsic and extrinsic evaluations for the task, with encouraging results, thereby setting a benchmark and evaluation practices for the task, as well as highlighting challenges.

Title: Security Vulnerabilities in Ethereum Smart Contracts: A Systematic Analysis

• Authors: Jixuan Wu, Lei Xie, Xiaoqi Li
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2504.05968
• Pdf URL: https://arxiv.org/pdf/2504.05968
• Copy Paste: [[2504.05968]] Security Vulnerabilities in Ethereum Smart Contracts: A Systematic Analysis(https://arxiv.org/abs/2504.05968)
• Keywords: secure, security, attack
• Abstract: Smart contracts are a secure and trustworthy application that plays a vital role in decentralized applications in various fields such as insurance,the internet, and gaming. However, in recent years, smart contract security breaches have occurred frequently, and due to their financial properties, they have caused huge economic losses, such as the most famous security incident "The DAO" which caused a loss of over \$60 million in Ethereum. This has drawn a lot of attention from all sides. Writing a secure smart contract is now a critical this http URL paper focuses on Ether smart contracts and explains the main components of Ether, smart contract architecture and this http URL environment used in this paper is the Ethernet environment, using remix online compilation platform and Solidity language, according to the four security events of American Chain, The DAO, Parity and KotET, the principles of integer overflow attack, reentrant attack, access control attack and denial of service attack are studied and analyzed accordingly, and the scenarios of these vulnerabilities are reproduced, and the measures to prevent them are given. Finally, preventive measures are given. In addition, the principles of short address attack, early transaction attack and privileged function exposure attack are also introduced in detail, and security measures are this http URL vulnerabilities continue to emerge, their classification will also evolve. The analysis and research of the current vulnerabilities are also to lay a solid foundation for avoiding more vulnerabilities.

Title: Diffusion Based Ambiguous Image Segmentation

• Authors: Jakob Lønborg Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl, Vedrana Andersen Dahl
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05977
• Pdf URL: https://arxiv.org/pdf/2504.05977
• Copy Paste: [[2504.05977]] Diffusion Based Ambiguous Image Segmentation(https://arxiv.org/abs/2504.05977)
• Keywords: diffusion, generative, segmentation
• Abstract: Medical image segmentation often involves inherent uncertainty due to variations in expert annotations. Capturing this uncertainty is an important goal and previous works have used various generative image models for the purpose of representing the full distribution of plausible expert ground truths. In this work, we explore the design space of diffusion models for generative segmentation, investigating the impact of noise schedules, prediction types, and loss weightings. Notably, we find that making the noise schedule harder with input scaling significantly improves performance. We conclude that x- and v-prediction outperform epsilon-prediction, likely because the diffusion process is in the discrete segmentation domain. Many loss weightings achieve similar performance as long as they give enough weight to the end of the diffusion process. We base our experiments on the LIDC-IDRI lung lesion dataset and obtain state-of-the-art (SOTA) performance. Additionally, we introduce a randomly cropped variant of the LIDC-IDRI dataset that is better suited for uncertainty in image segmentation. Our model also achieves SOTA in this harder setting.

Title: An Empirical Study of GPT-4o Image Generation Capabilities

• Authors: Sixiang Chen, Jinbin Bai, Zhuoran Zhao, Tian Ye, Qingyu Shi, Donghao Zhou, Wenhao Chai, Xin Lin, Jianzong Wu, Chao Tang, Shilin Xu, Tao Zhang, Haobo Yuan, Yikang Zhou, Wei Chow, Linfeng Li, Xiangtai Li, Lei Zhu, Lu Qi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.05979
• Pdf URL: https://arxiv.org/pdf/2504.05979
• Copy Paste: [[2504.05979]] An Empirical Study of GPT-4o Image Generation Capabilities(https://arxiv.org/abs/2504.05979)
• Keywords: diffusion, generative
• Abstract: The landscape of image generation has rapidly evolved, from early GAN-based approaches to diffusion models and, most recently, to unified generative architectures that seek to bridge understanding and generation tasks. Recent advances, especially the GPT-4o, have demonstrated the feasibility of high-fidelity multimodal generation, their architectural design remains mysterious and unpublished. This prompts the question of whether image and text generation have already been successfully integrated into a unified framework for those methods. In this work, we conduct an empirical study of GPT-4o's image generation capabilities, benchmarking it against leading open-source and commercial models. Our evaluation covers four main categories, including text-to-image, image-to-image, image-to-3D, and image-to-X generation, with more than 20 tasks. Our analysis highlights the strengths and limitations of GPT-4o under various settings, and situates it within the broader evolution of generative modeling. Through this investigation, we identify promising directions for future unified generative models, emphasizing the role of architectural design and data scaling.

Title: NativQA Framework: Enabling LLMs with Native, Local, and Everyday Knowledge

• Authors: Firoj Alam, Md Arid Hasan, Sahinur Rahman Laskar, Mucahid Kutlu, Shammur Absar Chowdhury
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.05995
• Pdf URL: https://arxiv.org/pdf/2504.05995
• Copy Paste: [[2504.05995]] NativQA Framework: Enabling LLMs with Native, Local, and Everyday Knowledge(https://arxiv.org/abs/2504.05995)
• Keywords: fair, large language model
• Abstract: The rapid advancement of large language models (LLMs) has raised concerns about cultural bias, fairness, and their applicability in diverse linguistic and underrepresented regional contexts. To enhance and benchmark the capabilities of LLMs, there is a need to develop large-scale resources focused on multilingual, local, and cultural contexts. In this study, we propose a framework, NativQA, that can seamlessly construct large-scale, culturally and regionally aligned QA datasets in native languages. The framework utilizes user-defined seed queries and leverages search engines to collect location-specific, everyday information. It has been evaluated across 39 locations in 24 countries and in 7 languages, ranging from extremely low-resource to high-resource languages, which resulted over 300K Question Answer (QA) pairs. The developed resources can be used for LLM benchmarking and further fine-tuning. The framework has been made publicly available for the community (this https URL).

Title: econSG: Efficient and Multi-view Consistent Open-Vocabulary 3D Semantic Gaussians

• Authors: Can Zhang, Gim Hee Lee
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06003
• Pdf URL: https://arxiv.org/pdf/2504.06003
• Copy Paste: [[2504.06003]] econSG: Efficient and Multi-view Consistent Open-Vocabulary 3D Semantic Gaussians(https://arxiv.org/abs/2504.06003)
• Keywords: segmentation
• Abstract: The primary focus of most recent works on open-vocabulary neural fields is extracting precise semantic features from the VLMs and then consolidating them efficiently into a multi-view consistent 3D neural fields representation. However, most existing works over-trusted SAM to regularize image-level CLIP without any further refinement. Moreover, several existing works improved efficiency by dimensionality reduction of semantic features from 2D VLMs before fusing with 3DGS semantic fields, which inevitably leads to multi-view inconsistency. In this work, we propose econSG for open-vocabulary semantic segmentation with 3DGS. Our econSG consists of: 1) A Confidence-region Guided Regularization (CRR) that mutually refines SAM and CLIP to get the best of both worlds for precise semantic features with complete and precise boundaries. 2) A low dimensional contextual space to enforce 3D multi-view consistency while improving computational efficiency by fusing backprojected multi-view 2D features and follow by dimensional reduction directly on the fused 3D features instead of operating on each 2D view separately. Our econSG shows state-of-the-art performance on four benchmark datasets compared to the existing methods. Furthermore, we are also the most efficient training among all the methods.

Title: FedFeat+: A Robust Federated Learning Framework Through Federated Aggregation and Differentially Private Feature-Based Classifier Retraining

• Authors: Mrityunjoy Gain, Kitae Kim, Avi Deb Raha, Apurba Adhikary, Eui-Nam Huh, Zhu Han, Choong Seon Hong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06004
• Pdf URL: https://arxiv.org/pdf/2504.06004
• Copy Paste: [[2504.06004]] FedFeat+: A Robust Federated Learning Framework Through Federated Aggregation and Differentially Private Feature-Based Classifier Retraining(https://arxiv.org/abs/2504.06004)
• Keywords: privacy, robust, extraction, federate
• Abstract: In this paper, we propose the FedFeat+ framework, which distinctively separates feature extraction from classification. We develop a two-tiered model training process: following local training, clients transmit their weights and some features extracted from the feature extractor from the final local epochs to the server. The server aggregates these models using the FedAvg method and subsequently retrains the global classifier utilizing the shared features. The classifier retraining process enhances the model's understanding of the holistic view of the data distribution, ensuring better generalization across diverse datasets. This improved generalization enables the classifier to adaptively influence the feature extractor during subsequent local training epochs. We establish a balance between enhancing model accuracy and safeguarding individual privacy through the implementation of differential privacy mechanisms. By incorporating noise into the feature vectors shared with the server, we ensure that sensitive data remains confidential. We present a comprehensive convergence analysis, along with theoretical reasoning regarding performance enhancement and privacy preservation. We validate our approach through empirical evaluations conducted on benchmark datasets, including CIFAR-10, CIFAR-100, MNIST, and FMNIST, achieving high accuracy while adhering to stringent privacy guarantees. The experimental results demonstrate that the FedFeat+ framework, despite using only a lightweight two-layer CNN classifier, outperforms the FedAvg method in both IID and non-IID scenarios, achieving accuracy improvements ranging from 3.92 % to 12.34 % across CIFAR-10, CIFAR-100, and Fashion-MNIST datasets.

Title: Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?

• Authors: Roman Kochnev, Arash Torabi Goodarzi, Zofia Antonina Bentyn, Dmitry Ignatov, Radu Timofte
• Subjects: cs.LG, cs.AI, cs.NE
• Abstract URL: https://arxiv.org/abs/2504.06006
• Pdf URL: https://arxiv.org/pdf/2504.06006
• Copy Paste: [[2504.06006]] Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?(https://arxiv.org/abs/2504.06006)
• Keywords: large language model
• Abstract: Optimal hyperparameter selection is critical for maximizing neural network performance, especially as models grow in complexity. This work investigates the viability of using large language models (LLMs) for hyperparameter optimization by employing a fine-tuned version of Code Llama. Through parameter-efficient fine-tuning using LoRA, we adapt the LLM to generate accurate and efficient hyperparameter recommendations tailored to diverse neural network architectures. Unlike traditional methods such as Optuna, which rely on exhaustive trials, the proposed approach achieves competitive or superior results in terms of Root Mean Square Error (RMSE) while significantly reducing computational overhead. Our approach highlights that LLM-based optimization not only matches state-of-the-art methods like Tree-structured Parzen Estimators but also accelerates the tuning process. This positions LLMs as a promising alternative to conventional optimization techniques, particularly for rapid experimentation. Furthermore, the ability to generate hyperparameters in a single inference step makes this method particularly well-suited for resource-constrained environments such as edge devices and mobile applications, where computational efficiency is paramount. The results confirm that LLMs, beyond their efficiency, offer substantial time savings and comparable stability, underscoring their value in advancing machine learning workflows. All generated hyperparameters are included in the LEMUR Neural Network (NN) Dataset, which is publicly available and serves as an open-source benchmark for hyperparameter optimization research.

Title: Latent Multimodal Reconstruction for Misinformation Detection

• Authors: Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, Panagiotis C. Petrantonakis
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.06010
• Pdf URL: https://arxiv.org/pdf/2504.06010
• Copy Paste: [[2504.06010]] Latent Multimodal Reconstruction for Misinformation Detection(https://arxiv.org/abs/2504.06010)
• Keywords: robust
• Abstract: Multimodal misinformation, such as miscaptioned images, where captions misrepresent an image's origin, context, or meaning, poses a growing challenge in the digital age. To support fact-checkers, researchers have been focusing on creating datasets and developing methods for multimodal misinformation detection (MMD). Due to the scarcity of large-scale annotated MMD datasets, recent studies leverage synthetic training data via out-of-context image-caption pairs or named entity manipulations; altering names, dates, and locations. However, these approaches often produce simplistic misinformation that fails to reflect real-world complexity, limiting the robustness of detection models trained on them. Meanwhile, despite recent advancements, Large Vision-Language Models (LVLMs) remain underutilized for generating diverse, realistic synthetic training data for MMD. To address this gap, we introduce "MisCaption This!", a training dataset comprising LVLM-generated miscaptioned images. Additionally, we introduce "Latent Multimodal Reconstruction" (LAMAR), a network trained to reconstruct the embeddings of truthful captions, providing a strong auxiliary signal to the detection process. To optimize LAMAR, we explore different training strategies (end-to-end training and large-scale pre-training) and integration approaches (direct, mask, gate, and attention). Extensive experiments show that models trained on "MisCaption This!" generalize better on real-world misinformation, while LAMAR sets new state-of-the-art on both NewsCLIPpings and VERITE benchmarks; highlighting the potential of LVLM-generated data and reconstruction-based approaches for advancing MMD. We release our code at: this https URL

Title: Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi

• Authors: Monojit Choudhury, Shivam Chauhan, Rocktim Jyoti Das, Dhruv Sahnan, Xudong Han, Haonan Li, Aaryamonvikram Singh, Alok Anil Jadhav, Utkarsh Agarwal, Mukund Choudhary, Debopriyo Banerjee, Fajri Koto, Junaid Bhat, Awantika Shukla, Samujjwal Ghosh, Samta Kamboj, Onkar Pandit, Lalit Pradhan, Rahul Pal, Sunil Sahu, Soundar Doraiswamy, Parvez Mullah, Ali El Filali, Neha Sengupta, Gokul Ramakrishnan, Rituraj Joshi, Gurpreet Gosal, Avraham Sheinin, Natalia Vassilieva, Preslav Nakov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06011
• Pdf URL: https://arxiv.org/pdf/2504.06011
• Copy Paste: [[2504.06011]] Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi(https://arxiv.org/abs/2504.06011)
• Keywords: transformer, generative, large language model
• Abstract: Developing high-quality large language models (LLMs) for moderately resourced languages presents unique challenges in data availability, model adaptation, and evaluation. We introduce Llama-3-Nanda-10B-Chat, or Nanda for short, a state-of-the-art Hindi-centric instruction-tuned generative LLM, designed to push the boundaries of open-source Hindi language models. Built upon Llama-3-8B, Nanda incorporates continuous pre-training with expanded transformer blocks, leveraging the Llama Pro methodology. A key challenge was the limited availability of high-quality Hindi text data; we addressed this through rigorous data curation, augmentation, and strategic bilingual training, balancing Hindi and English corpora to optimize cross-linguistic knowledge transfer. With 10 billion parameters, Nanda stands among the top-performing open-source Hindi and multilingual models of similar scale, demonstrating significant advantages over many existing models. We provide an in-depth discussion of training strategies, fine-tuning techniques, safety alignment, and evaluation metrics, demonstrating how these approaches enabled Nanda to achieve state-of-the-art results. By open-sourcing Nanda, we aim to advance research in Hindi LLMs and support a wide range of real-world applications across academia, industry, and public services.

Title: CAI: An Open, Bug Bounty-Ready Cybersecurity AI

• Authors: Víctor Mayoral-Vilches, Luis Javier Navarrete-Lozano, María Sanz-Gómez, Lidia Salas Espejo, Martiño Crespo-Álvarez, Francisco Oca-Gonzalez, Francesco Balassone, Alfonso Glera-Picón, Unai Ayucar-Carbajo, Endika Gil-Uriarte
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2504.06017
• Pdf URL: https://arxiv.org/pdf/2504.06017
• Copy Paste: [[2504.06017]] CAI: An Open, Bug Bounty-Ready Cybersecurity AI(https://arxiv.org/abs/2504.06017)
• Keywords: secure, security
• Abstract: By 2028 most cybersecurity actions will be autonomous, with humans teleoperating. We present the first classification of autonomy levels in cybersecurity and introduce Cybersecurity AI (CAI), an open-source framework that democratizes advanced security testing through specialized AI agents. Through rigorous empirical evaluation, we demonstrate that CAI consistently outperforms state-of-the-art results in CTF benchmarks, solving challenges across diverse categories with significantly greater efficiency -up to 3,600x faster than humans in specific tasks and averaging 11x faster overall. CAI achieved first place among AI teams and secured a top-20 position worldwide in the "AI vs Human" CTF live Challenge, earning a monetary reward of $750. Based on our results, we argue against LLM-vendor claims about limited security capabilities. Beyond cybersecurity competitions, CAI demonstrates real-world effectiveness, reaching top-30 in Spain and top-500 worldwide on Hack The Box within a week, while dramatically reducing security testing costs by an average of 156x. Our framework transcends theoretical benchmarks by enabling non-professionals to discover significant security bugs (CVSS 4.3-7.5) at rates comparable to experts during bug bounty exercises. By combining modular agent design with seamless tool integration and human oversight (HITL), CAI addresses critical market gaps, offering organizations of all sizes access to AI-powered bug bounty security testing previously available only to well-resourced firms -thereby challenging the oligopolistic ecosystem currently dominated by major bug bounty platforms.

Title: Memory-Modular Classification: Learning to Generalize with Memory Replacement

• Authors: Dahyun Kang, Ahmet Iscen, Eunchan Jo, Sua Choi, Minsu Cho, Cordelia Schmid
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06021
• Pdf URL: https://arxiv.org/pdf/2504.06021
• Copy Paste: [[2504.06021]] Memory-Modular Classification: Learning to Generalize with Memory Replacement(https://arxiv.org/abs/2504.06021)
• Keywords: robust
• Abstract: We propose a novel memory-modular learner for image classification that separates knowledge memorization from reasoning. Our model enables effective generalization to new classes by simply replacing the memory contents, without the need for model retraining. Unlike traditional models that encode both world knowledge and task-specific skills into their weights during training, our model stores knowledge in the external memory of web-crawled image and text data. At inference time, the model dynamically selects relevant content from the memory based on the input image, allowing it to adapt to arbitrary classes by simply replacing the memory contents. The key differentiator that our learner meta-learns to perform classification tasks with noisy web data from unseen classes, resulting in robust performance across various classification scenarios. Experimental results demonstrate the promising performance and versatility of our approach in handling diverse classification tasks, including zero-shot/few-shot classification of unseen classes, fine-grained classification, and class-incremental classification.

Title: CamContextI2V: Context-aware Controllable Video Generation

• Authors: Luis Denninger, Sina Mokhtarzadeh Azar, Juergen Gall
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06022
• Pdf URL: https://arxiv.org/pdf/2504.06022
• Copy Paste: [[2504.06022]] CamContextI2V: Context-aware Controllable Video Generation(https://arxiv.org/abs/2504.06022)
• Keywords: diffusion, generative
• Abstract: Recently, image-to-video (I2V) diffusion models have demonstrated impressive scene understanding and generative quality, incorporating image conditions to guide generation. However, these models primarily animate static images without extending beyond their provided context. Introducing additional constraints, such as camera trajectories, can enhance diversity but often degrades visual quality, limiting their applicability for tasks requiring faithful scene representation. We propose CamContextI2V, an I2V model that integrates multiple image conditions with 3D constraints alongside camera control to enrich both global semantics and fine-grained visual details. This enables more coherent and context-aware video generation. Moreover, we motivate the necessity of temporal awareness for an effective context representation. Our comprehensive study on the RealEstate10K dataset demonstrates improvements in visual quality and camera controllability. We make our code and models publicly available at: this https URL.

Title: Multi-Sense Embeddings for Language Models and Knowledge Distillation

• Authors: Qitong Wang, Mohammed J. Zaki, Georgios Kollias, Vasileios Kalantzis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06036
• Pdf URL: https://arxiv.org/pdf/2504.06036
• Copy Paste: [[2504.06036]] Multi-Sense Embeddings for Language Models and Knowledge Distillation(https://arxiv.org/abs/2504.06036)
• Keywords: transformer, large language model
• Abstract: Transformer-based large language models (LLMs) rely on contextual embeddings which generate different (continuous) representations for the same token depending on its surrounding context. Nonetheless, words and tokens typically have a limited number of senses (or meanings). We propose multi-sense embeddings as a drop-in replacement for each token in order to capture the range of their uses in a language. To construct a sense embedding dictionary, we apply a clustering algorithm to embeddings generated by an LLM and consider the cluster centers as representative sense embeddings. In addition, we propose a novel knowledge distillation method that leverages the sense dictionary to learn a smaller student model that mimics the senses from the much larger base LLM model, offering significant space and inference time savings, while maintaining competitive performance. Via thorough experiments on various benchmarks, we showcase the effectiveness of our sense embeddings and knowledge distillation approach. We share our code at this https URL

Title: Enhanced Anomaly Detection for Capsule Endoscopy Using Ensemble Learning Strategies

• Authors: Julia Werner, Christoph Gerum, Jorg Nick, Maxime Le Floch, Franz Brinkmann, Jochen Hampe, Oliver Bringmann
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06039
• Pdf URL: https://arxiv.org/pdf/2504.06039
• Copy Paste: [[2504.06039]] Enhanced Anomaly Detection for Capsule Endoscopy Using Ensemble Learning Strategies(https://arxiv.org/abs/2504.06039)
• Keywords: robust
• Abstract: Capsule endoscopy is a method to capture images of the gastrointestinal tract and screen for diseases which might remain hidden if investigated with standard endoscopes. Due to the limited size of a video capsule, embedding AI models directly into the capsule demands careful consideration of the model size and thus complicates anomaly detection in this field. Furthermore, the scarcity of available data in this domain poses an ongoing challenge to achieving effective anomaly detection. Thus, this work introduces an ensemble strategy to address this challenge in anomaly detection tasks in video capsule endoscopies, requiring only a small number of individual neural networks during both the training and inference phases. Ensemble learning combines the predictions of multiple independently trained neural networks. This has shown to be highly effective in enhancing both the accuracy and robustness of machine learning models. However, this comes at the cost of higher memory usage and increased computational effort, which quickly becomes prohibitive in many real-world applications. Instead of applying the same training algorithm to each individual network, we propose using various loss functions, drawn from the anomaly detection field, to train each network. The methods are validated on the two largest publicly available datasets for video capsule endoscopy images, the Galar and the Kvasir-Capsule dataset. We achieve an AUC score of 76.86% on the Kvasir-Capsule and an AUC score of 76.98% on the Galar dataset. Our approach outperforms current baselines with significantly fewer parameters across all models, which is a crucial step towards incorporating artificial intelligence into capsule endoscopies.

Title: Explainable AI for building energy retrofitting under data scarcity

• Authors: Panagiota Rempi, Sotiris Pelekis, Alexandros Menelaos Tzortzis, Evangelos Karakolis, Christos Ntanos, Dimitris Askounis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06055
• Pdf URL: https://arxiv.org/pdf/2504.06055
• Copy Paste: [[2504.06055]] Explainable AI for building energy retrofitting under data scarcity(https://arxiv.org/abs/2504.06055)
• Keywords: generative
• Abstract: Enhancing energy efficiency in residential buildings is a crucial step toward mitigating climate change and reducing greenhouse gas emissions. Retrofitting existing buildings, which account for a significant portion of energy consumption, is critical particularly in regions with outdated and inefficient building stocks. This study presents an Artificial Intelligence (AI) and Machine Learning (ML)-based framework to recommend energy efficiency measures for residential buildings, leveraging accessible building characteristics to achieve energy class targets. Using Latvia as a case study, the methodology addresses challenges associated with limited datasets, class imbalance and data scarcity. The proposed approach integrates Conditional Tabular Generative Adversarial Networks (CTGAN) to generate synthetic data, enriching and balancing the dataset. A Multi-Layer Perceptron (MLP) model serves as the predictive model performing multi-label classification to predict appropriate retrofit strategies. Explainable Artificial Intelligence (XAI), specifically SHapley Additive exPlanations (SHAP), ensures transparency and trust by identifying key features that influence recommendations and guiding feature engineering choices for improved reliability and performance. The evaluation of the approach shows that it notably overcomes data limitations, achieving improvements up to 54% in precision, recall and F1 score. Although this study focuses on Latvia, the methodology is adaptable to other regions, underscoring the potential of AI in reducing the complexity and cost of building energy retrofitting overcoming data limitations. By facilitating decision-making processes and promoting stakeholders engagement, this work supports the global transition toward sustainable energy use in the residential building sector.

Title: PINP: Physics-Informed Neural Predictor with latent estimation of fluid flows

• Authors: Huaguan Chen, Yang Liu, Hao Sun
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06070
• Pdf URL: https://arxiv.org/pdf/2504.06070
• Copy Paste: [[2504.06070]] PINP: Physics-Informed Neural Predictor with latent estimation of fluid flows(https://arxiv.org/abs/2504.06070)
• Keywords: robust
• Abstract: Accurately predicting fluid dynamics and evolution has been a long-standing challenge in physical sciences. Conventional deep learning methods often rely on the nonlinear modeling capabilities of neural networks to establish mappings between past and future states, overlooking the fluid dynamics, or only modeling the velocity field, neglecting the coupling of multiple physical quantities. In this paper, we propose a new physics-informed learning approach that incorporates coupled physical quantities into the prediction process to assist with forecasting. Central to our method lies in the discretization of physical equations, which are directly integrated into the model architecture and loss function. This integration enables the model to provide robust, long-term future predictions. By incorporating physical equations, our model demonstrates temporal extrapolation and spatial generalization capabilities. Experimental results show that our approach achieves the state-of-the-art performance in spatiotemporal prediction across both numerical simulations and real-world extreme-precipitation nowcasting benchmarks.

Title: Security Analysis of Thumbnail-Preserving Image Encryption and a New Framework

• Authors: Dong Xie, Zhiyang Li, Shuangxi Guo, Fulong Chen, Peng Hu
• Subjects: cs.CR, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.06083
• Pdf URL: https://arxiv.org/pdf/2504.06083
• Copy Paste: [[2504.06083]] Security Analysis of Thumbnail-Preserving Image Encryption and a New Framework(https://arxiv.org/abs/2504.06083)
• Keywords: secure, security, privacy, attack, robust
• Abstract: As a primary encryption primitive balancing the privacy and searchability of cloud storage images, thumbnail preserving encryption (TPE) enables users to quickly identify the privacy personal image on the cloud and request this image from the owner through a secure channel. In this paper, we have found that two different plaintext images may produce the same thumbnail. It results in the failure of search strategy because the collision of thumbnail occurs. To address this serious security issues, we conduct an in-depth analysis on the collision probabilities of thumbnails, and then propose a new TPE framework, called multi-factor thumbnail preserving encryption (MFTPE). It starts from the collision probability of two blocks, extend to the probabilities of two images and ultimately to N images. Then, we in detail describe three specific MFTPE constructions preserving different combinations of factors, i.e., the sum and the geometric mean, the sum and the range, and the sum and the weighted mean. The theoretical and experimental results demonstrate that the proposed MFTPE reduces the probability of thumbnails, exhibits strong robustness, and also effectively resists face detection and noise attacks.

Title: MCAT: Visual Query-Based Localization of Standard Anatomical Clips in Fetal Ultrasound Videos Using Multi-Tier Class-Aware Token Transformer

• Authors: Divyanshu Mishra, Pramit Saha, He Zhao, Netzahualcoyotl Hernandez-Cruz, Olga Patey, Aris Papageorghiou, J. Alison Noble
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06088
• Pdf URL: https://arxiv.org/pdf/2504.06088
• Copy Paste: [[2504.06088]] MCAT: Visual Query-Based Localization of Standard Anatomical Clips in Fetal Ultrasound Videos Using Multi-Tier Class-Aware Token Transformer(https://arxiv.org/abs/2504.06088)
• Keywords: transformer
• Abstract: Accurate standard plane acquisition in fetal ultrasound (US) videos is crucial for fetal growth assessment, anomaly detection, and adherence to clinical guidelines. However, manually selecting standard frames is time-consuming and prone to intra- and inter-sonographer variability. Existing methods primarily rely on image-based approaches that capture standard frames and then classify the input frames across different anatomies. This ignores the dynamic nature of video acquisition and its interpretation. To address these challenges, we introduce Multi-Tier Class-Aware Token Transformer (MCAT), a visual query-based video clip localization (VQ-VCL) method, to assist sonographers by enabling them to capture a quick US sweep. By then providing a visual query of the anatomy they wish to analyze, MCAT returns the video clip containing the standard frames for that anatomy, facilitating thorough screening for potential anomalies. We evaluate MCAT on two ultrasound video datasets and a natural image VQ-VCL dataset based on Ego4D. Our model outperforms state-of-the-art methods by 10% and 13% mIoU on the ultrasound datasets and by 5.35% mIoU on the Ego4D dataset, using 96% fewer tokens. MCAT's efficiency and accuracy have significant potential implications for public health, especially in low- and middle-income countries (LMICs), where it may enhance prenatal care by streamlining standard plane acquisition, simplifying US-based screening, diagnosis and allowing sonographers to examine more patients.

Title: Towards Varroa destructor mite detection using a narrow spectra illumination

• Authors: Samuel Bielik, Simon Bilik
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06099
• Pdf URL: https://arxiv.org/pdf/2504.06099
• Copy Paste: [[2504.06099]] Towards Varroa destructor mite detection using a narrow spectra illumination(https://arxiv.org/abs/2504.06099)
• Keywords: segmentation
• Abstract: This paper focuses on the development and modification of a beehive monitoring device and Varroa destructor detection on the bees with the help of hyperspectral imagery while utilizing a U-net, semantic segmentation architecture, and conventional computer vision methods. The main objectives were to collect a dataset of bees and mites, and propose the computer vision model which can achieve the detection between bees and mites.

Title: Sherlock: A Dataset for Process-aware Intrusion Detection Research on Power Grid Networks

• Authors: Eric Wagner, Lennart Bader, Konrad Wolsing, Martin Serror
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2504.06102
• Pdf URL: https://arxiv.org/pdf/2504.06102
• Copy Paste: [[2504.06102]] Sherlock: A Dataset for Process-aware Intrusion Detection Research on Power Grid Networks(https://arxiv.org/abs/2504.06102)
• Keywords: security, protect, attack
• Abstract: Physically distributed components and legacy protocols make the protection of power grids against increasing cyberattack threats challenging. Infamously, the 2015 and 2016 blackouts in Ukraine were caused by cyberattacks, and the German Federal Office for Information Security (BSI) recorded over 200 cyber incidents against the German energy sector between 2023 and 2024. Intrusion detection promises to quickly detect such attacks and mitigate the worst consequences. However, public datasets of realistic scenarios are vital to evaluate these systems. This paper introduces Sherlock, a dataset generated with the co-simulator Wattson. In total, Sherlock covers three scenarios with various attacks manipulating the process state by injecting malicious commands or manipulating measurement values. We additionally test five recently-published intrusion detection systems on Sherlock, highlighting specific challenges for intrusion detection in power grids. Dataset and documentation are available at this https URL.

Title: Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testing

• Authors: Erik Hellsten, Carl Hvarfner, Leonard Papenmeier, Luigi Nardi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06111
• Pdf URL: https://arxiv.org/pdf/2504.06111
• Copy Paste: [[2504.06111]] Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testing(https://arxiv.org/abs/2504.06111)
• Keywords: interpretability
• Abstract: Bayesian optimization (BO ) is an effective method for optimizing expensive-to-evaluate black-box functions. While high-dimensional problems can be particularly challenging, due to the multitude of parameter choices and the potentially high number of data points required to fit the model, this limitation can be addressed if the problem satisfies simplifying assumptions. Axis-aligned subspace approaches, where few dimensions have a significant impact on the objective, motivated several algorithms for high-dimensional BO . However, the validity of this assumption is rarely verified, and the assumption is rarely exploited to its full extent. We propose a group testing ( GT) approach to identify active variables to facilitate efficient optimization in these domains. The proposed algorithm, Group Testing Bayesian Optimization (GTBO), first runs a testing phase where groups of variables are systematically selected and tested on whether they influence the objective, then terminates once active dimensions are identified. To that end, we extend the well-established GT theory to functions over continuous domains. In the second phase, GTBO guides optimization by placing more importance on the active dimensions. By leveraging the axis-aligned subspace assumption, GTBO outperforms state-of-the-art methods on benchmarks satisfying the assumption of axis-aligned subspaces, while offering improved interpretability.

Title: To Match or Not to Match: Revisiting Image Matching for Reliable Visual Place Recognition

• Authors: Davide Sferrazza, Gabriele Berton, Gabriele Trivigno, Carlo Masone
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06116
• Pdf URL: https://arxiv.org/pdf/2504.06116
• Copy Paste: [[2504.06116]] To Match or Not to Match: Revisiting Image Matching for Reliable Visual Place Recognition(https://arxiv.org/abs/2504.06116)
• Keywords: robust
• Abstract: Visual Place Recognition (VPR) is a critical task in computer vision, traditionally enhanced by re-ranking retrieval results with image matching. However, recent advancements in VPR methods have significantly improved performance, challenging the necessity of re-ranking. In this work, we show that modern retrieval systems often reach a point where re-ranking can degrade results, as current VPR datasets are largely saturated. We propose using image matching as a verification step to assess retrieval confidence, demonstrating that inlier counts can reliably predict when re-ranking is beneficial. Our findings shift the paradigm of retrieval pipelines, offering insights for more robust and adaptive VPR systems.

Title: A Robust Real-Time Lane Detection Method with Fog-Enhanced Feature Fusion for Foggy Conditions

• Authors: Ronghui Zhang, Yuhang Ma, Tengfei Li, Ziyu Lin, Yueying Wu, Junzhou Chen, Lin Zhang, Jia Hu, Tony Z. Qiu, Konghui Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06121
• Pdf URL: https://arxiv.org/pdf/2504.06121
• Copy Paste: [[2504.06121]] A Robust Real-Time Lane Detection Method with Fog-Enhanced Feature Fusion for Foggy Conditions(https://arxiv.org/abs/2504.06121)
• Keywords: robust
• Abstract: Lane detection is a critical component of Advanced Driver Assistance Systems (ADAS). Existing lane detection algorithms generally perform well under favorable weather conditions. However, their performance degrades significantly in adverse conditions, such as fog, which increases the risk of traffic accidents. This challenge is compounded by the lack of specialized datasets and methods designed for foggy environments. To address this, we introduce the FoggyLane dataset, captured in real-world foggy scenarios, and synthesize two additional datasets, FoggyCULane and FoggyTusimple, from existing popular lane detection datasets. Furthermore, we propose a robust Fog-Enhanced Network for lane detection, incorporating a Global Feature Fusion Module (GFFM) to capture global relationships in foggy images, a Kernel Feature Fusion Module (KFFM) to model the structural and positional relationships of lane instances, and a Low-level Edge Enhanced Module (LEEM) to address missing edge details in foggy conditions. Comprehensive experiments demonstrate that our method achieves state-of-the-art performance, with F1-scores of 95.04 on FoggyLane, 79.85 on FoggyCULane, and 96.95 on FoggyTusimple. Additionally, with TensorRT acceleration, the method reaches a processing speed of 38.4 FPS on the NVIDIA Jetson AGX Orin, confirming its real-time capabilities and robustness in foggy environments.

Title: FaceCloak: Learning to Protect Face Templates

• Authors: Sudipta Banerjee, Anubhav Jain, Chinmay Hegde, Nasir Memon
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06131
• Pdf URL: https://arxiv.org/pdf/2504.06131
• Copy Paste: [[2504.06131]] FaceCloak: Learning to Protect Face Templates(https://arxiv.org/abs/2504.06131)
• Keywords: security, privacy, protect, attack, biometric, extraction, generative
• Abstract: Generative models can reconstruct face images from encoded representations (templates) bearing remarkable likeness to the original face raising security and privacy concerns. We present FaceCloak, a neural network framework that protects face templates by generating smart, renewable binary cloaks. Our method proactively thwarts inversion attacks by cloaking face templates with unique disruptors synthesized from a single face template on the fly while provably retaining biometric utility and unlinkability. Our cloaked templates can suppress sensitive attributes while generalizing to novel feature extraction schemes and outperforms leading baselines in terms of biometric matching and resiliency to reconstruction attacks. FaceCloak-based matching is extremely fast (inference time cost=0.28ms) and light-weight (0.57MB).

Title: QGen Studio: An Adaptive Question-Answer Generation, Training and Evaluation Platform

• Authors: Movina Moses, Mohab Elkaref, James Barry, Shinnosuke Tanaka, Vishnudev Kuruvanthodi, Nathan Herr, Campbell D Watson, Geeth De Mel
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06136
• Pdf URL: https://arxiv.org/pdf/2504.06136
• Copy Paste: [[2504.06136]] QGen Studio: An Adaptive Question-Answer Generation, Training and Evaluation Platform(https://arxiv.org/abs/2504.06136)
• Keywords: large language model
• Abstract: We present QGen Studio: an adaptive question-answer generation, training, and evaluation platform. QGen Studio enables users to leverage large language models (LLMs) to create custom question-answer datasets and fine-tune models on this synthetic data. It features a dataset viewer and model explorer to streamline this process. The dataset viewer provides key metrics and visualizes the context from which the QA pairs are generated, offering insights into data quality. The model explorer supports model comparison, allowing users to contrast the performance of their trained LLMs against other models, supporting performance benchmarking and refinement. QGen Studio delivers an interactive, end-to-end solution for generating QA datasets and training scalable, domain-adaptable models. The studio will be open-sourced soon, allowing users to deploy it locally.

Title: Adversarial Training of Reward Models

• Authors: Alexander Bukharin, Haifeng Qian, Shengyang Sun, Adithya Renduchintala, Soumye Singhal, Zhilin Wang, Oleksii Kuchaiev, Olivier Delalleau, Tuo Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06141
• Pdf URL: https://arxiv.org/pdf/2504.06141
• Copy Paste: [[2504.06141]] Adversarial Training of Reward Models(https://arxiv.org/abs/2504.06141)
• Keywords: robust
• Abstract: Reward modeling has emerged as a promising approach for the scalable alignment of language models. However, contemporary reward models (RMs) often lack robustness, awarding high rewards to low-quality, out-of-distribution (OOD) samples. This can lead to reward hacking, where policies exploit unintended shortcuts to maximize rewards, undermining alignment. To address this challenge, we introduce Adv-RM, a novel adversarial training framework that automatically identifies adversarial examples -- responses that receive high rewards from the target RM but are OOD and of low quality. By leveraging reinforcement learning, Adv-RM trains a policy to generate adversarial examples that reliably expose vulnerabilities in large state-of-the-art reward models such as Nemotron 340B RM. Incorporating these adversarial examples into the reward training process improves the robustness of RMs, mitigating reward hacking and enhancing downstream performance in RLHF. We demonstrate that Adv-RM significantly outperforms conventional RM training, increasing stability and enabling more effective RLHF training in both synthetic and real-data settings.

Title: A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model

• Authors: Jihun Park, Jongmin Gim, Kyoungmin Lee, Minseok Oh, Minwoo Choi, Jaeyeul Kim, Woo Chool Park, Sunghoon Im
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06144
• Pdf URL: https://arxiv.org/pdf/2504.06144
• Copy Paste: [[2504.06144]] A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model(https://arxiv.org/abs/2504.06144)
• Keywords: diffusion
• Abstract: We present a training-free style-aligned image generation method that leverages a scale-wise autoregressive model. While large-scale text-to-image (T2I) models, particularly diffusion-based methods, have demonstrated impressive generation quality, they often suffer from style misalignment across generated image sets and slow inference speeds, limiting their practical usability. To address these issues, we propose three key components: initial feature replacement to ensure consistent background appearance, pivotal feature interpolation to align object placement, and dynamic style injection, which reinforces style consistency using a schedule function. Unlike previous methods requiring fine-tuning or additional training, our approach maintains fast inference while preserving individual content details. Extensive experiments show that our method achieves generation quality comparable to competing approaches, significantly improves style alignment, and delivers inference speeds over six times faster than the fastest model.

Title: V-MAGE: A Game Evaluation Framework for Assessing Visual-Centric Capabilities in Multimodal Large Language Models

• Authors: Xiangxi Zheng, Linjie Li, Zhengyuan Yang, Ping Yu, Alex Jinpeng Wang, Rui Yan, Yuan Yao, Lijuan Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06148
• Pdf URL: https://arxiv.org/pdf/2504.06148
• Copy Paste: [[2504.06148]] V-MAGE: A Game Evaluation Framework for Assessing Visual-Centric Capabilities in Multimodal Large Language Models(https://arxiv.org/abs/2504.06148)
• Keywords: large language model
• Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have led to significant improvements across various multimodal benchmarks. However, as evaluations shift from static datasets to open-world, dynamic environments, current game-based benchmarks remain inadequate because they lack visual-centric tasks and fail to assess the diverse reasoning skills required for real-world decision-making. To address this, we introduce Visual-centric Multiple Abilities Game Evaluation (V-MAGE), a game-based evaluation framework designed to assess visual reasoning capabilities of MLLMs. V-MAGE features five diverse games with 30+ handcrafted levels, testing models on core visual skills such as positioning, trajectory tracking, timing, and visual memory, alongside higher-level reasoning like long-term planning and deliberation. We use V-MAGE to evaluate leading MLLMs, revealing significant challenges in their visual perception and reasoning. In all game environments, the top-performing MLLMs, as determined by Elo rating comparisons, exhibit a substantial performance gap compared to humans. Our findings highlight critical limitations, including various types of perceptual errors made by the models, and suggest potential avenues for improvement from an agent-centric perspective, such as refining agent strategies and addressing perceptual inaccuracies. Code is available at this https URL.

Title: A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning

• Authors: Akash Kumar, Ashlesha Kumar, Vibhav Vineet, Yogesh S Rawat
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06153
• Pdf URL: https://arxiv.org/pdf/2504.06153
• Copy Paste: [[2504.06153]] A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning(https://arxiv.org/abs/2504.06153)
• Keywords: fair
• Abstract: Self-supervised learning has emerged as a powerful paradigm for label-free model pretraining, particularly in the video domain, where manual annotation is costly and time-intensive. However, existing self-supervised approaches employ diverse experimental setups, making direct comparisons challenging due to the absence of a standardized benchmark. In this work, we establish a unified benchmark that enables fair comparisons across different methods. Additionally, we systematically investigate five critical aspects of self-supervised learning in videos: (1) dataset size, (2) model complexity, (3) data distribution, (4) data noise, and (5) feature representations. To facilitate this study, we evaluate six self-supervised learning methods across six network architectures, conducting extensive experiments on five benchmark datasets and assessing performance on two distinct downstream tasks. Our analysis reveals key insights into the interplay between pretraining strategies, dataset characteristics, pretext tasks, and model architectures. Furthermore, we extend these findings to Video Foundation Models (ViFMs), demonstrating their relevance in large-scale video representation learning. Finally, leveraging these insights, we propose a novel approach that significantly reduces training data requirements while surpassing state-of-the-art methods that rely on 10% more pretraining data. We believe this work will guide future research toward a deeper understanding of self-supervised video representation learning and its broader implications.

Title: Rethinking the Nested U-Net Approach: Enhancing Biomarker Segmentation with Attention Mechanisms and Multiscale Feature Fusion

• Authors: Saad Wazir, Daeyoung Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06158
• Pdf URL: https://arxiv.org/pdf/2504.06158
• Copy Paste: [[2504.06158]] Rethinking the Nested U-Net Approach: Enhancing Biomarker Segmentation with Attention Mechanisms and Multiscale Feature Fusion(https://arxiv.org/abs/2504.06158)
• Keywords: extraction, transformer, segmentation
• Abstract: Identifying biomarkers in medical images is vital for a wide range of biotech applications. However, recent Transformer and CNN based methods often struggle with variations in morphology and staining, which limits their feature extraction capabilities. In medical image segmentation, where data samples are often limited, state-of-the-art (SOTA) methods improve accuracy by using pre-trained encoders, while end-to-end approaches typically fall short due to difficulties in transferring multiscale features effectively between encoders and decoders. To handle these challenges, we introduce a nested UNet architecture that captures both local and global context through Multiscale Feature Fusion and Attention Mechanisms. This design improves feature integration from encoders, highlights key channels and regions, and restores spatial details to enhance segmentation performance. Our method surpasses SOTA approaches, as evidenced by experiments across four datasets and detailed ablation studies. Code: this https URL

Title: Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups

• Authors: Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury
• Subjects: cs.CL, cs.AI, cs.CY, cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2504.06160
• Pdf URL: https://arxiv.org/pdf/2504.06160
• Copy Paste: [[2504.06160]] Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups(https://arxiv.org/abs/2504.06160)
• Keywords: attack, large language model
• Abstract: Large Language Models (LLMs) have been shown to demonstrate imbalanced biases against certain groups. However, the study of unprovoked targeted attacks by LLMs towards at-risk populations remains underexplored. Our paper presents three novel contributions: (1) the explicit evaluation of LLM-generated attacks on highly vulnerable mental health groups; (2) a network-based framework to study the propagation of relative biases; and (3) an assessment of the relative degree of stigmatization that emerges from these attacks. Our analysis of a recently released large-scale bias audit dataset reveals that mental health entities occupy central positions within attack narrative networks, as revealed by a significantly higher mean centrality of closeness (p-value = 4.06e-10) and dense clustering (Gini coefficient = 0.7). Drawing from sociological foundations of stigmatization theory, our stigmatization analysis indicates increased labeling components for mental health disorder-related targets relative to initial targets in generation chains. Taken together, these insights shed light on the structural predilections of large language models to heighten harmful discourse and highlight the need for suitable approaches for mitigation.

Title: Assessing how hyperparameters impact Large Language Models' sarcasm detection performance

• Authors: Montgomery Gole, Andriy Miranskyy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06166
• Pdf URL: https://arxiv.org/pdf/2504.06166
• Copy Paste: [[2504.06166]] Assessing how hyperparameters impact Large Language Models' sarcasm detection performance(https://arxiv.org/abs/2504.06166)
• Keywords: large language model
• Abstract: Sarcasm detection is challenging for both humans and machines. This work explores how model characteristics impact sarcasm detection in OpenAI's GPT, and Meta's Llama-2 models, given their strong natural language understanding, and popularity. We evaluate fine-tuned and zero-shot models across various sizes, releases, and hyperparameters. Experiments were conducted on the political and balanced (pol-bal) portion of the popular Self-Annotated Reddit Corpus (SARC2.0) sarcasm dataset. Fine-tuned performance improves monotonically with model size within a model family, while hyperparameter tuning also impacts performance. In the fine-tuning scenario, full precision Llama-2-13b achieves state-of-the-art accuracy and $F_1$-score, both measured at 0.83, comparable to average human performance. In the zero-shot setting, one GPT-4 model achieves competitive performance to prior attempts, yielding an accuracy of 0.70 and an $F_1$-score of 0.75. Furthermore, a model's performance may increase or decline with each release, highlighting the need to reassess performance after each release.

Title: Flash Sculptor: Modular 3D Worlds from Objects

• Authors: Yujia Hu, Songhua Liu, Xingyi Yang, Xinchao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06178
• Pdf URL: https://arxiv.org/pdf/2504.06178
• Copy Paste: [[2504.06178]] Flash Sculptor: Modular 3D Worlds from Objects(https://arxiv.org/abs/2504.06178)
• Keywords: robust
• Abstract: Existing text-to-3D and image-to-3D models often struggle with complex scenes involving multiple objects and intricate interactions. Although some recent attempts have explored such compositional scenarios, they still require an extensive process of optimizing the entire layout, which is highly cumbersome if not infeasible at all. To overcome these challenges, we propose Flash Sculptor in this paper, a simple yet effective framework for compositional 3D scene/object reconstruction from a single image. At the heart of Flash Sculptor lies a divide-and-conquer strategy, which decouples compositional scene reconstruction into a sequence of sub-tasks, including handling the appearance, rotation, scale, and translation of each individual instance. Specifically, for rotation, we introduce a coarse-to-fine scheme that brings the best of both worlds--efficiency and accuracy--while for translation, we develop an outlier-removal-based algorithm that ensures robust and precise parameters in a single step, without any iterative optimization. Extensive experiments demonstrate that Flash Sculptor achieves at least a 3 times speedup over existing compositional 3D methods, while setting new benchmarks in compositional 3D reconstruction performance. Codes are available at this https URL.

Title: Blockchain Oracles for Real Estate Rental

• Authors: Nuno Braz, João Santos, Tiago Dias, Miguel Correia
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/2504.06180
• Pdf URL: https://arxiv.org/pdf/2504.06180
• Copy Paste: [[2504.06180]] Blockchain Oracles for Real Estate Rental(https://arxiv.org/abs/2504.06180)
• Keywords: secure
• Abstract: Blockchain technology has seen adoption across various industries and the real estate sector is no exception. The traditional property leasing process guarantees no trust between parties, uses insecure communication channels, and forces participants who are not familiar with the process to perform contracts. Blockchain technology emerges as a solution to simplify the traditional property leasing process. This work proposes the use of two blockchain oracles to handle, respectively, maintenance issues and automate rent payments in the context of property rental. These two components are introduced in a blockchain-based property rental platform.

Title: WoundAmbit: Bridging State-of-the-Art Semantic Segmentation and Real-World Wound Care

• Authors: Vanessa Borst, Timo Dittus, Tassilo Dege, Astrid Schmieder, Samuel Kounev
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06185
• Pdf URL: https://arxiv.org/pdf/2504.06185
• Copy Paste: [[2504.06185]] WoundAmbit: Bridging State-of-the-Art Semantic Segmentation and Real-World Wound Care(https://arxiv.org/abs/2504.06185)
• Keywords: fair, interpretability, transformer, segmentation
• Abstract: Chronic wounds affect a large population, particularly the elderly and diabetic patients, who often exhibit limited mobility and co-existing health conditions. Automated wound monitoring via mobile image capture can reduce in-person physician visits by enabling remote tracking of wound size. Semantic segmentation is key to this process, yet wound segmentation remains underrepresented in medical imaging research. To address this, we benchmark state-of-the-art deep learning models from general-purpose vision, medical imaging, and top methods from public wound challenges. For fair comparison, we standardize training, data augmentation, and evaluation, conducting cross-validationto minimize partitioning bias. We also assess real-world deployment aspects, including generalization to an out-of-distribution wound dataset, computational efficiency, and interpretability. Additionally, we propose a reference object-based approach to convert AI-generated masks into clinically relevant wound size estimates, and evaluate this, along with mask quality, for the best models based on physician assessments. Overall, the transformer-based TransNeXt showed the highest levels of generalizability. Despite variations in inference times, all models processed at least one image per second on the CPU, which is deemed adequate for the intended application. Interpretability analysis typically revealed prominent activations in wound regions, emphasizing focus on clinically relevant features. Expert evaluation showed high mask approval for all analyzed models, with VWFormer and ConvNeXtS backbone performing the best. Size retrieval accuracy was similar across models, and predictions closely matched expert annotations. Finally, we demonstrate how our AI-driven wound size estimation framework, WoundAmbit, can be integrated into a custom telehealth system. Our code will be made available on GitHub upon publication.

Title: HRMedSeg: Unlocking High-resolution Medical Image segmentation via Memory-efficient Attention Modeling

• Authors: Qing Xu, Zhenye Lou, Chenxin Li, Xiangjian He, Rong Qu, Tesema Fiseha Berhanu, Yi Wang, Wenting Duan, Zhen Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06205
• Pdf URL: https://arxiv.org/pdf/2504.06205
• Copy Paste: [[2504.06205]] HRMedSeg: Unlocking High-resolution Medical Image segmentation via Memory-efficient Attention Modeling(https://arxiv.org/abs/2504.06205)
• Keywords: transformer, segmentation
• Abstract: High-resolution segmentation is critical for precise disease diagnosis by extracting micro-imaging information from medical images. Existing transformer-based encoder-decoder frameworks have demonstrated remarkable versatility and zero-shot performance in medical segmentation. While beneficial, they usually require huge memory costs when handling large-size segmentation mask predictions, which are expensive to apply to real-world scenarios. To address this limitation, we propose a memory-efficient framework for high-resolution medical image segmentation, called HRMedSeg. Specifically, we first devise a lightweight gated vision transformer (LGViT) as our image encoder to model long-range dependencies with linear complexity. Then, we design an efficient cross-multiscale decoder (ECM-Decoder) to generate high-resolution segmentation masks. Moreover, we utilize feature distillation during pretraining to unleash the potential of our proposed model. Extensive experiments reveal that HRMedSeg outperforms state-of-the-arts in diverse high-resolution medical image segmentation tasks. In particular, HRMedSeg uses only 0.59GB GPU memory per batch during fine-tuning, demonstrating low training costs. Besides, when HRMedSeg meets the Segment Anything Model (SAM), our HRMedSegSAM takes 0.61% parameters of SAM-H. The code is available at this https URL.

Title: NNN: Next-Generation Neural Networks for Marketing Mix Modeling

• Authors: Thomas Mulc, Mike Anderson, Paul Cubre, Huikun Zhang, Ivy Liu, Saket Kumar
• Subjects: cs.LG, stat.AP
• Abstract URL: https://arxiv.org/abs/2504.06212
• Pdf URL: https://arxiv.org/pdf/2504.06212
• Copy Paste: [[2504.06212]] NNN: Next-Generation Neural Networks for Marketing Mix Modeling(https://arxiv.org/abs/2504.06212)
• Keywords: interpretability, transformer
• Abstract: We present NNN, a Transformer-based neural network approach to Marketing Mix Modeling (MMM) designed to address key limitations of traditional methods. Unlike conventional MMMs which rely on scalar inputs and parametric decay functions, NNN uses rich embeddings to capture both quantitative and qualitative aspects of marketing and organic channels (e.g., search queries, ad creatives). This, combined with its attention mechanism, enables NNN to model complex interactions, capture long-term effects, and potentially improve sales attribution accuracy. We show that L1 regularization permits the use of such expressive models in typical data-constrained settings. Evaluating NNN on simulated and real-world data demonstrates its efficacy, particularly through considerable improvement in predictive power. Beyond attribution, NNN provides valuable, complementary insights through model probing, such as evaluating keyword or creative effectiveness, enhancing model interpretability.

Title: From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models

• Authors: Chejian Xu, Wei Ping, Peng Xu, Zihan Liu, Boxin Wang, Mohammad Shoeybi, Bo Li, Bryan Catanzaro
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06214
• Pdf URL: https://arxiv.org/pdf/2504.06214
• Copy Paste: [[2504.06214]] From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models(https://arxiv.org/abs/2504.06214)
• Keywords: robust, large language model
• Abstract: Long-context capabilities are essential for a wide range of applications, including document and video understanding, in-context learning, and inference-time scaling, all of which require models to process and reason over long sequences of text and multimodal data. In this work, we introduce a efficient training recipe for building ultra-long context LLMs from aligned instruct model, pushing the boundaries of context lengths from 128K to 1M, 2M, and 4M tokens. Our approach leverages efficient continued pretraining strategies to extend the context window and employs effective instruction tuning to maintain the instruction-following and reasoning abilities. Our UltraLong-8B, built on Llama3.1-Instruct with our recipe, achieves state-of-the-art performance across a diverse set of long-context benchmarks. Importantly, models trained with our approach maintain competitive performance on standard benchmarks, demonstrating balanced improvements for both long and short context tasks. We further provide an in-depth analysis of key design choices, highlighting the impacts of scaling strategies and data composition. Our findings establish a robust framework for efficiently scaling context lengths while preserving general model capabilities. We release all model weights at: this https URL.

Title: Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs

• Authors: Dongyang Fan, Vinko Sabolčec, Matin Ansaripour, Ayush Kumar Tarun, Martin Jaggi, Antoine Bosselut, Imanol Schlag
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06219
• Pdf URL: https://arxiv.org/pdf/2504.06219
• Copy Paste: [[2504.06219]] Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs(https://arxiv.org/abs/2504.06219)
• Keywords: large language model
• Abstract: The increasing adoption of web crawling opt-outs by copyright holders of online content raises critical questions about the impact of data compliance on large language model (LLM) performance. However, little is known about how these restrictions (and the resultant filtering of pretraining datasets) affect the capabilities of models trained using these corpora. In this work, we conceptualize this effect as the $\textit{data compliance gap}$ (DCG), which quantifies the performance difference between models trained on datasets that comply with web crawling opt-outs, and those that do not. We measure the data compliance gap in two settings: pretraining models from scratch and continual pretraining from existing compliant models (simulating a setting where copyrighted data could be integrated later in pretraining). Our experiments with 1.5B models show that, as of January 2025, compliance with web data opt-outs does not degrade general knowledge acquisition (close to 0\% DCG). However, in specialized domains such as biomedical research, excluding major publishers leads to performance declines. These findings suggest that while general-purpose LLMs can be trained to perform equally well using fully open data, performance in specialized domains may benefit from access to high-quality copyrighted sources later in training. Our study provides empirical insights into the long-debated trade-off between data compliance and downstream model performance, informing future discussions on AI training practices and policy decisions.

Title: Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation

• Authors: Xiaoxing Hu, Ziyang Gong, Yupei Wang, Yuru Jia, Gen Luo, Xue Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06220
• Pdf URL: https://arxiv.org/pdf/2504.06220
• Copy Paste: [[2504.06220]] Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation(https://arxiv.org/abs/2504.06220)
• Keywords: segmentation
• Abstract: Parameter-Efficient Fine-Tuning (PEFT) is a technique that allows us to adapt powerful Foundation Models (FMs) to diverse downstream tasks while preserving and unleashing their inherent capabilities. However, we have observed that existing PEFT methods, which are often designed with natural imagery in mind, struggle when applied to Remote Sensing (RS) scenarios. This is primarily due to their inability to handle artifact influences, a problem particularly severe in RS image features. To tackle this challenge, we introduce Earth-Adapter, the first PEFT method specifically designed for RS artifacts conquering. Earth-Adapter introduces a novel Mixture of Frequency Adaptation process that combines a Mixture of Adapter (MoA) with Discrete Fourier Transformation (DFT). By utilizing DFT, Earth-Adapter can decompose features into different frequency components, precisely separating artifacts from original features. The MoA then dynamically assigns weights to each adapter expert, allowing for the combination of features across various frequency domains. These simple-yet-effective approaches enable Earth-Adapter to more efficiently overcome the disturbances caused by artifacts than previous PEFT methods, significantly enhancing the FMs' performance on RS scenarios. Experiments on Domain Adaptation (DA), and Domain Generalization (DG) semantic segmentation benchmarks showcase the Earth-Adapter's effectiveness. Compared with baseline Rein, Earth-Adapter significantly improves 9.0% mIoU in DA and 3.1% mIoU in DG benchmarks. Our code will be released at this https URL.

Title: Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation

• Authors: Biao Zhang, Fedor Moiseev, Joshua Ainslie, Paul Suganthan, Min Ma, Surya Bhupatiraju, Fede Lebron, Orhan Firat, Armand Joulin, Zhe Dong
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.06225
• Pdf URL: https://arxiv.org/pdf/2504.06225
• Copy Paste: [[2504.06225]] Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation(https://arxiv.org/abs/2504.06225)
• Keywords: large language model
• Abstract: While decoder-only large language models (LLMs) have shown impressive results, encoder-decoder models are still widely adopted in real-world applications for their inference efficiency and richer encoder representation. In this paper, we study a novel problem: adapting pretrained decoder-only LLMs to encoder-decoder, with the goal of leveraging the strengths of both approaches to achieve a more favorable quality-efficiency trade-off. We argue that adaptation not only enables inheriting the capability of decoder-only LLMs but also reduces the demand for computation compared to pretraining from scratch. We rigorously explore different pretraining objectives and parameter initialization/optimization techniques. Through extensive experiments based on Gemma 2 (2B and 9B) and a suite of newly pretrained mT5-sized models (up to 1.6B), we demonstrate the effectiveness of adaptation and the advantage of encoder-decoder LLMs. Under similar inference budget, encoder-decoder LLMs achieve comparable (often better) pretraining performance but substantially better finetuning performance than their decoder-only counterpart. For example, Gemma 2B-2B outperforms Gemma 2B by $\sim$7\% after instruction tuning. Encoder-decoder adaptation also allows for flexible combination of different-sized models, where Gemma 9B-2B significantly surpasses Gemma 2B-2B by $>$3\%. The adapted encoder representation also yields better results on SuperGLUE. We will release our checkpoints to facilitate future research.

Title: LExT: Towards Evaluating Trustworthiness of Natural Language Explanations

• Authors: Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06227
• Pdf URL: https://arxiv.org/pdf/2504.06227
• Copy Paste: [[2504.06227]] LExT: Towards Evaluating Trustworthiness of Natural Language Explanations(https://arxiv.org/abs/2504.06227)
• Keywords: robust, interpretability, large language model
• Abstract: As Large Language Models (LLMs) become increasingly integrated into high-stakes domains, there have been several approaches proposed toward generating natural language explanations. These explanations are crucial for enhancing the interpretability of a model, especially in sensitive domains like healthcare, where transparency and reliability are key. In light of such explanations being generated by LLMs and its known concerns, there is a growing need for robust evaluation frameworks to assess model-generated explanations. Natural Language Generation metrics like BLEU and ROUGE capture syntactic and semantic accuracies but overlook other crucial aspects such as factual accuracy, consistency, and faithfulness. To address this gap, we propose a general framework for quantifying trustworthiness of natural language explanations, balancing Plausibility and Faithfulness, to derive a comprehensive Language Explanation Trustworthiness Score (LExT) (The code and set up to reproduce our experiments are publicly available at this https URL). Applying our domain-agnostic framework to the healthcare domain using public medical datasets, we evaluate six models, including domain-specific and general-purpose models. Our findings demonstrate significant differences in their ability to generate trustworthy explanations. On comparing these explanations, we make interesting observations such as inconsistencies in Faithfulness demonstrated by general-purpose models and their tendency to outperform domain-specific fine-tuned models. This work further highlights the importance of using a tailored evaluation framework to assess natural language explanations in sensitive fields, providing a foundation for improving the trustworthiness and transparency of language models in healthcare and beyond.

Title: HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance

• Authors: Jiazi Bu, Pengyang Ling, Yujie Zhou, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Dahua Lin, Jiaqi Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06232
• Pdf URL: https://arxiv.org/pdf/2504.06232
• Copy Paste: [[2504.06232]] HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance(https://arxiv.org/abs/2504.06232)
• Keywords: diffusion
• Abstract: Text-to-image (T2I) diffusion/flow models have drawn considerable attention recently due to their remarkable ability to deliver flexible visual creations. Still, high-resolution image synthesis presents formidable challenges due to the scarcity and complexity of high-resolution content. To this end, we present HiFlow, a training-free and model-agnostic framework to unlock the resolution potential of pre-trained flow models. Specifically, HiFlow establishes a virtual reference flow within the high-resolution space that effectively captures the characteristics of low-resolution flow information, offering guidance for high-resolution generation through three key aspects: initialization alignment for low-frequency consistency, direction alignment for structure preservation, and acceleration alignment for detail fidelity. By leveraging this flow-aligned guidance, HiFlow substantially elevates the quality of high-resolution image synthesis of T2I models and demonstrates versatility across their personalized variants. Extensive experiments validate HiFlow's superiority in achieving superior high-resolution image quality over current state-of-the-art methods.

Title: Decentralized Federated Domain Generalization with Style Sharing: A Formal Modeling and Convergence Analysis

• Authors: Shahryar Zehtabi, Dong-Jun Han, Seyyedali Hosseinalipour, Christopher G. Brinton
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06235
• Pdf URL: https://arxiv.org/pdf/2504.06235
• Copy Paste: [[2504.06235]] Decentralized Federated Domain Generalization with Style Sharing: A Formal Modeling and Convergence Analysis(https://arxiv.org/abs/2504.06235)
• Keywords: federate
• Abstract: Much of the federated learning (FL) literature focuses on settings where local dataset statistics remain the same between training and testing time. Recent advances in domain generalization (DG) aim to use data from source (training) domains to train a model that generalizes well to data from unseen target (testing) domains. In this paper, we are motivated by two major gaps in existing work on FL and DG: (1) the lack of formal mathematical analysis of DG objectives and training processes; and (2) DG research in FL being limited to the conventional star-topology architecture. Addressing the second gap, we develop $\textit{Decentralized Federated Domain Generalization with Style Sharing}$ ($\texttt{StyleDDG}$), a fully decentralized DG algorithm designed to allow devices in a peer-to-peer network to achieve DG based on sharing style information inferred from their datasets. Additionally, we fill the first gap by providing the first systematic approach to mathematically analyzing style-based DG training optimization. We cast existing centralized DG algorithms within our framework, and employ their formalisms to model $\texttt{StyleDDG}$. Based on this, we obtain analytical conditions under which a sub-linear convergence rate of $\texttt{StyleDDG}$ can be obtained. Through experiments on two popular DG datasets, we demonstrate that $\texttt{StyleDDG}$ can obtain significant improvements in accuracy across target domains with minimal added communication overhead compared to decentralized gradient methods that do not employ style sharing.

Title: A Case for Network-wide Orchestration of Host-based Intrusion Detection and Response

• Authors: Mark Timmons, Daniel Lukaszewski, Geoffrey Xie
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/2504.06241
• Pdf URL: https://arxiv.org/pdf/2504.06241
• Copy Paste: [[2504.06241]] A Case for Network-wide Orchestration of Host-based Intrusion Detection and Response(https://arxiv.org/abs/2504.06241)
• Keywords: security, attack
• Abstract: Recent cyber incidents and the push for zero trust security underscore the necessity of monitoring host-level events. However, current host-level intrusion detection systems (IDS) lack the ability to correlate alerts and coordinate a network-wide response in real time. Motivated by advances in system-level extensions free of rebooting and network-wide orchestration of host actions, we propose using a central IDS orchestrator to remotely program the logic of each host IDS and collect the alerts generated in real time. In this paper, we make arguments for such a system concept and provide a high level design of the main system components. Furthermore, we have developed a system prototype and evaluated it using two experimental scenarios rooted from real-world attacks. The evaluation results show that the host-based IDS orchestration system is able to defend against the attacks effectively.

Title: Transfer between Modalities with MetaQueries

• Authors: Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, Ji Hou, Saining Xie
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.06256
• Pdf URL: https://arxiv.org/pdf/2504.06256
• Copy Paste: [[2504.06256]] Transfer between Modalities with MetaQueries(https://arxiv.org/abs/2504.06256)
• Keywords: diffusion, generative
• Abstract: Unified multimodal models aim to integrate understanding (text output) and generation (pixel output), but aligning these different modalities within a single architecture often demands complex training recipes and careful data balancing. We introduce MetaQueries, a set of learnable queries that act as an efficient interface between autoregressive multimodal LLMs (MLLMs) and diffusion models. MetaQueries connects the MLLM's latents to the diffusion decoder, enabling knowledge-augmented image generation by leveraging the MLLM's deep understanding and reasoning capabilities. Our method simplifies training, requiring only paired image-caption data and standard diffusion objectives. Notably, this transfer is effective even when the MLLM backbone remains frozen, thereby preserving its state-of-the-art multimodal understanding capabilities while achieving strong generative performance. Additionally, our method is flexible and can be easily instruction-tuned for advanced applications such as image editing and subject-driven generation.

Title: Hogwild! Inference: Parallel LLM Generation via Concurrent Attention

• Authors: Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, Dan Alistarh
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.06261
• Pdf URL: https://arxiv.org/pdf/2504.06261
• Copy Paste: [[2504.06261]] Hogwild! Inference: Parallel LLM Generation via Concurrent Attention(https://arxiv.org/abs/2504.06261)
• Keywords: large language model
• Abstract: Large Language Models (LLMs) have demonstrated the ability to tackle increasingly complex tasks through advanced reasoning, long-form content generation, and tool use. Solving these tasks often involves long inference-time computations. In human problem solving, a common strategy to expedite work is collaboration: by dividing the problem into sub-tasks, exploring different strategies concurrently, etc. Recent research has shown that LLMs can also operate in parallel by implementing explicit cooperation frameworks, such as voting mechanisms or the explicit creation of independent sub-tasks that can be executed in parallel. However, each of these frameworks may not be suitable for all types of tasks, which can hinder their applicability. In this work, we propose a different design approach: we run LLM "workers" in parallel , allowing them to synchronize via a concurrently-updated attention cache and prompt these workers to decide how best to collaborate. Our approach allows the instances to come up with their own collaboration strategy for the problem at hand, all the while "seeing" each other's partial progress in the concurrent cache. We implement this approach via Hogwild! Inference: a parallel LLM inference engine where multiple instances of the same LLM run in parallel with the same attention cache, with "instant" access to each other's generated tokens. Hogwild! inference takes advantage of Rotary Position Embeddings (RoPE) to avoid recomputation while improving parallel hardware utilization. We find that modern reasoning-capable LLMs can perform inference with shared Key-Value cache out of the box, without additional fine-tuning.

Title: GOLLuM: Gaussian Process Optimized LLMs -- Reframing LLM Finetuning through Bayesian Optimization

• Authors: Bojana Ranković, Philippe Schwaller
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.06265
• Pdf URL: https://arxiv.org/pdf/2504.06265
• Copy Paste: [[2504.06265]] GOLLuM: Gaussian Process Optimized LLMs -- Reframing LLM Finetuning through Bayesian Optimization(https://arxiv.org/abs/2504.06265)
• Keywords: robust, large language model
• Abstract: Large Language Models (LLMs) can encode complex relationships in their latent spaces, yet harnessing them for optimization under uncertainty remains challenging. We address this gap with a novel architecture that reframes LLM finetuning as Gaussian process (GP) marginal likelihood optimization via deep kernel methods. We introduce LLM-based deep kernels, jointly optimized with GPs to preserve the benefits of both - LLMs to provide a rich and flexible input space for Bayesian optimization and - GPs to model this space with predictive uncertainty for more efficient sampling. Applied to Buchwald-Hartwig reaction optimization, our method nearly doubles the discovery rate of high-performing reactions compared to static LLM embeddings (from 24% to 43% coverage of the top 5% reactions in just 50 optimization iterations). We also observe a 14% improvement over domain-specific representations without requiring specialized features. Extensive empirical evaluation across 19 benchmarks - ranging from general chemistry to reaction and molecular property optimization - demonstrates our method's robustness, generality, and consistent improvements across: (1) tasks, (2) LLM architectures (encoder, decoder, encoder-decoder), (3) pretraining domains (chemistry-related or general-purpose) and (4) hyperparameter settings (tuned once on a single dataset). Finally, we explain these improvements: joint LLM-GP optimization through marginal likelihood implicitly performs contrastive learning, aligning representations to produce (1) better-structured embedding spaces, (2) improved uncertainty calibration, and (3) more efficient sampling - without requiring any external loss. This work provides both practical advances in sample-efficient optimization and insights into what makes effective Bayesian optimization.