2024-05-16

Title: Adversarial Machine Learning Threats to Spacecraft

• Authors: Rajiv Thummala, Shristi Sharma, Matteo Calabrese, Gregory Falco
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Adversarial Machine Learning Threats to Spacecraft(https://arxiv.org/abs/)
• Keywords: security, attack
• Abstract: Spacecraft are among the earliest autonomous systems. Their ability to function without a human in the loop have afforded some of humanity's grandest achievements. As reliance on autonomy grows, space vehicles will become increasingly vulnerable to attacks designed to disrupt autonomous processes-especially probabilistic ones based on machine learning. This paper aims to elucidate and demonstrate the threats that adversarial machine learning (AML) capabilities pose to spacecraft. First, an AML threat taxonomy for spacecraft is introduced. Next, we demonstrate the execution of AML attacks against spacecraft through experimental simulations using NASA's Core Flight System (cFS) and NASA's On-board Artificial Intelligence Research (OnAIR) Platform. Our findings highlight the imperative for incorporating AML-focused security measures in spacecraft that engage autonomy.

Title: A Click-Through Rate Prediction Method Based on Cross-Importance of Multi-Order Features

• Authors: Hao Wang, Nao Li
• Subjects: cs.LG, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] A Click-Through Rate Prediction Method Based on Cross-Importance of Multi-Order Features(https://arxiv.org/abs/)
• Keywords: interpretability
• Abstract: Most current click-through rate prediction(CTR)models create explicit or implicit high-order feature crosses through Hadamard product or inner product, with little attention to the importance of feature crossing; only few models are either limited to the second-order explicit feature crossing, implicitly to high-order feature crossing, or can learn the importance of high-order explicit feature crossing but fail to provide good interpretability for the model. This paper proposes a new model, FiiNet (Multiple Order Feature Interaction Importance Neural Networks). The model first uses the selective kernel network (SKNet) to explicitly construct multi-order feature crosses. It dynamically learns the importance of feature interaction combinations in a fine grained manner, increasing the attention weight of important feature cross combinations and reducing the weight of featureless crosses. To verify that the FiiNet model can dynamically learn the importance of feature interaction combinations in a fine-grained manner and improve the model's recommendation performance and interpretability, this paper compares it with many click-through rate prediction models on two real datasets, proving that the FiiNet model incorporating the selective kernel network can effectively improve the recommendation effect and provide better interpretability. FiiNet model implementations are available in PyTorch.

Title: The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks

• Authors: Ziquan Liu, Yufei Cui, Yan Yan, Yi Xu, Xiangyang Ji, Xue Liu, Antoni B. Chan
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks(https://arxiv.org/abs/)
• Keywords: protect, defense, attack, robust
• Abstract: In safety-critical applications such as medical imaging and autonomous driving, where decisions have profound implications for patient health and road safety, it is imperative to maintain both high adversarial robustness to protect against potential adversarial attacks and reliable uncertainty quantification in decision-making. With extensive research focused on enhancing adversarial robustness through various forms of adversarial training (AT), a notable knowledge gap remains concerning the uncertainty inherent in adversarially trained models. To address this gap, this study investigates the uncertainty of deep learning models by examining the performance of conformal prediction (CP) in the context of standard adversarial attacks within the adversarial defense community. It is first unveiled that existing CP methods do not produce informative prediction sets under the commonly used $l_{\infty}$-norm bounded attack if the model is not adversarially trained, which underpins the importance of adversarial training for CP. Our paper next demonstrates that the prediction set size (PSS) of CP using adversarially trained models with AT variants is often worse than using standard AT, inspiring us to research into CP-efficient AT for improved PSS. We propose to optimize a Beta-weighting loss with an entropy minimization regularizer during AT to improve CP-efficiency, where the Beta-weighting loss is shown to be an upper bound of PSS at the population level by our theoretical analysis. Moreover, our empirical study on four image classification datasets across three popular AT baselines validates the effectiveness of the proposed Uncertainty-Reducing AT (AT-UR).

Title: Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language

• Authors: Jan Kaiser, Annika Eichler, Anne Lauscher
• Subjects: cs.CL, cs.AI, cs.LG, physics.acc-ph
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Autonomous tuning of particle accelerators is an active and challenging field of research with the goal of enabling novel accelerator technologies cutting-edge high-impact applications, such as physics discovery, cancer research and material sciences. A key challenge with autonomous accelerator tuning remains that the most capable algorithms require an expert in optimisation, machine learning or a similar field to implement the algorithm for every new tuning task. In this work, we propose the use of large language models (LLMs) to tune particle accelerators. We demonstrate on a proof-of-principle example the ability of LLMs to successfully and autonomously tune a particle accelerator subsystem based on nothing more than a natural language prompt from the operator, and compare the performance of our LLM-based solution to state-of-the-art optimisation algorithms, such as Bayesian optimisation (BO) and reinforcement learning-trained optimisation (RLO). In doing so, we also show how LLMs can perform numerical optimisation of a highly non-linear real-world objective function. Ultimately, this work represents yet another complex task that LLMs are capable of solving and promises to help accelerate the deployment of autonomous tuning algorithms to the day-to-day operations of particle accelerators.

Title: Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video

• Authors: Tomoya Sugihara, Shuntaro Masuda, Ling Xiao, Toshihiko Yamasaki
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Current video summarization methods primarily depend on supervised computer vision techniques, which demands time-consuming manual annotations. Further, the annotations are always subjective which make this task more challenging. To address these issues, we analyzed the feasibility in transforming the video summarization into a text summary task and leverage Large Language Models (LLMs) to boost video summarization. This paper proposes a novel self-supervised framework for video summarization guided by LLMs. Our method begins by generating captions for video frames, which are then synthesized into text summaries by LLMs. Subsequently, we measure semantic distance between the frame captions and the text summary. It's worth noting that we propose a novel loss function to optimize our model according to the diversity of the video. Finally, the summarized video can be generated by selecting the frames whose captions are similar with the text summary. Our model achieves competitive results against other state-of-the-art methods and paves a novel pathway in video summarization.

Title: RS-Reg: Probabilistic and Robust Certified Regression Through Randomized Smoothing

• Authors: Aref Miri Rekavandi, Olga Ohrimenko, Benjamin I.P. Rubinstein
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] RS-Reg: Probabilistic and Robust Certified Regression Through Randomized Smoothing(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Randomized smoothing has shown promising certified robustness against adversaries in classification tasks. Despite such success with only zeroth-order access to base models, randomized smoothing has not been extended to a general form of regression. By defining robustness in regression tasks flexibly through probabilities, we demonstrate how to establish upper bounds on input data point perturbation (using the $\ell_2$ norm) for a user-specified probability of observing valid outputs. Furthermore, we showcase the asymptotic property of a basic averaging function in scenarios where the regression model operates without any constraint. We then derive a certified upper bound of the input perturbations when dealing with a family of regression models where the outputs are bounded. Our simulations verify the validity of the theoretical results and reveal the advantages and limitations of simple smoothing functions, i.e., averaging, in regression tasks. The code is publicly available at \url{this https URL}.

Title: CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

• Authors: Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Oncel Tuzel
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] CLIP with Quality Captions: A Strong Pretraining for Vision Tasks(https://arxiv.org/abs/)
• Keywords: segmentation
• Abstract: CLIP models perform remarkably well on zero-shot classification and retrieval tasks. But recent studies have shown that learnt representations in CLIP are not well suited for dense prediction tasks like object detection, semantic segmentation or depth estimation. More recently, multi-stage training methods for CLIP models was introduced to mitigate the weak performance of CLIP on downstream tasks. In this work, we find that simply improving the quality of captions in image-text datasets improves the quality of CLIP's visual representations, resulting in significant improvement on downstream dense prediction vision tasks. In fact, we find that CLIP pretraining with good quality captions can surpass recent supervised, self-supervised and weakly supervised pretraining methods. We show that when CLIP model with ViT-B/16 as image encoder is trained on well aligned image-text pairs it obtains 12.1% higher mIoU and 11.5% lower RMSE on semantic segmentation and depth estimation tasks over recent state-of-the-art Masked Image Modeling (MIM) pretraining methods like Masked Autoencoder (MAE). We find that mobile architectures also benefit significantly from CLIP pretraining. A recent mobile vision architecture, MCi2, with CLIP pretraining obtains similar performance as Swin-L, pretrained on ImageNet-22k for semantic segmentation task while being 6.1$\times$ smaller. Moreover, we show that improving caption quality results in $10\times$ data efficiency when finetuning for dense prediction tasks.

Title: Feature Importance and Explainability in Quantum Machine Learning

• Authors: Luke Power, Krishnendu Guha
• Subjects: cs.LG, math.QA, stat.ML
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Feature Importance and Explainability in Quantum Machine Learning(https://arxiv.org/abs/)
• Keywords: explainability
• Abstract: Many Machine Learning (ML) models are referred to as black box models, providing no real insights into why a prediction is made. Feature importance and explainability are important for increasing transparency and trust in ML models, particularly in settings such as healthcare and finance. With quantum computing's unique capabilities, such as leveraging quantum mechanical phenomena like superposition, which can be combined with ML techniques to create the field of Quantum Machine Learning (QML), and such techniques may be applied to QML models. This article explores feature importance and explainability insights in QML compared to Classical ML models. Utilizing the widely recognized Iris dataset, classical ML algorithms such as SVM and Random Forests, are compared against hybrid quantum counterparts, implemented via IBM's Qiskit platform: the Variational Quantum Classifier (VQC) and Quantum Support Vector Classifier (QSVC). This article aims to provide a comparison of the insights generated in ML by employing permutation and leave one out feature importance methods, alongside ALE (Accumulated Local Effects) and SHAP (SHapley Additive exPlanations) explainers.

Title: Neural Collapse Meets Differential Privacy: Curious Behaviors of NoisyGD with Near-perfect Representation Learning

• Authors: Chendi Wang, Yuqing Zhu, Weijie J. Su, Yu-Xiang Wang
• Subjects: cs.LG, cs.CR, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Neural Collapse Meets Differential Privacy: Curious Behaviors of NoisyGD with Near-perfect Representation Learning(https://arxiv.org/abs/)
• Keywords: privacy, robust, transformer
• Abstract: A recent study by De et al. (2022) has reported that large-scale representation learning through pre-training on a public dataset significantly enhances differentially private (DP) learning in downstream tasks, despite the high dimensionality of the feature space. To theoretically explain this phenomenon, we consider the setting of a layer-peeled model in representation learning, which results in interesting phenomena related to learned features in deep learning and transfer learning, known as Neural Collapse (NC). Within the framework of NC, we establish an error bound indicating that the misclassification error is independent of dimension when the distance between actual features and the ideal ones is smaller than a threshold. Additionally, the quality of the features in the last layer is empirically evaluated under different pre-trained models within the framework of NC, showing that a more powerful transformer leads to a better feature representation. Furthermore, we reveal that DP fine-tuning is less robust compared to fine-tuning without DP, particularly in the presence of perturbations. These observations are supported by both theoretical analyses and experimental evaluation. Moreover, to enhance the robustness of DP fine-tuning, we suggest several strategies, such as feature normalization or employing dimension reduction methods like Principal Component Analysis (PCA). Empirically, we demonstrate a significant improvement in testing accuracy by conducting PCA on the last-layer features.

Title: Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis

• Authors: Yao Fu
• Subjects: cs.LG, cs.AI, cs.CL, cs.DC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis(https://arxiv.org/abs/)
• Keywords: transformer, generative
• Abstract: Transformer-based long context generative models power emerging AI applications like hour-long video understanding and project-level coding agent. Deploying long context transformers (e.g., 100K to 10M tokens) is prohibitively expensive compared to short context (e.g., 4K tokens) model variants. Reducing the cost of long-context transformers is becoming a pressing research and engineering challenge starting from the year of 2024. This work describes a concurrent programming framework for quantitatively analyzing the efficiency challenges in serving multiple long-context requests under limited size of GPU high-bandwidth memory (HBM) regime. We give a detailed analysis of how all additional computational costs, compared to 4K context, trace back to \textit{one single source: the large size of the KV cache}. We use a 34B GPT-3.5 level model of 50K context on A100 NVLink as a running example, and describe how its large KV cache causes four types of deployment challenges: (1) prefilling long inputs takes much longer compute time and GPU memory than short inputs; (2) after prefilling, the large KV cache residing on the GPU HBM substantially restricts the number of concurrent users being served; (3) during decoding, repeatedly reading the KV cache from HBM to SM largely increases latency; (4) when KV cache memory overflows, swapping it from HBM to DDR causes significant context switching latency. We use this framework to analyze existing works and identify possibilities of combining them to build end-to-end systems. Overall, this work offers a foundational framework for analyzing long context transformer deployment and identifies directions towards reducing the inference cost of 1M context to be as cheap as 4K.

Title: An adaptive approach to Bayesian Optimization with switching costs

• Authors: Stefan Pricopie, Richard Allmendinger, Manuel Lopez-Ibanez, Clyde Fare, Matt Benatan, Joshua Knowles
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] An adaptive approach to Bayesian Optimization with switching costs(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: We investigate modifications to Bayesian Optimization for a resource-constrained setting of sequential experimental design where changes to certain design variables of the search space incur a switching cost. This models the scenario where there is a trade-off between evaluating more while maintaining the same setup, or switching and restricting the number of possible evaluations due to the incurred cost. We adapt two process-constrained batch algorithms to this sequential problem formulation, and propose two new methods: one cost-aware and one cost-ignorant. We validate and compare the algorithms using a set of 7 scalable test functions in different dimensionalities and switching-cost settings for 30 total configurations. Our proposed cost-aware hyperparameter-free algorithm yields comparable results to tuned process-constrained algorithms in all settings we considered, suggesting some degree of robustness to varying landscape features and cost trade-offs. This method starts to outperform the other algorithms with increasing switching-cost. Our work broadens out from other recent Bayesian Optimization studies in resource-constrained settings that consider a batch setting only. While the contributions of this work are relevant to the general class of resource-constrained problems, they are particularly relevant to problems where adaptability to varying resource availability is of high importance

Title: drGAT: Attention-Guided Gene Assessment of Drug Response Utilizing a Drug-Cell-Gene Heterogeneous Network

• Authors: Yoshitaka Inoue, Hunmin Lee, Tianfan Fu, Augustin Luna
• Subjects: cs.LG, q-bio.MN, q-bio.QM
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] drGAT: Attention-Guided Gene Assessment of Drug Response Utilizing a Drug-Cell-Gene Heterogeneous Network(https://arxiv.org/abs/)
• Keywords: interpretability
• Abstract: Drug development is a lengthy process with a high failure rate. Increasingly, machine learning is utilized to facilitate the drug development processes. These models aim to enhance our understanding of drug characteristics, including their activity in biological contexts. However, a major challenge in drug response (DR) prediction is model interpretability as it aids in the validation of findings. This is important in biomedicine, where models need to be understandable in comparison with established knowledge of drug interactions with proteins. drGAT, a graph deep learning model, leverages a heterogeneous graph composed of relationships between proteins, cell lines, and drugs. drGAT is designed with two objectives: DR prediction as a binary sensitivity prediction and elucidation of drug mechanism from attention coefficients. drGAT has demonstrated superior performance over existing models, achieving 78\% accuracy (and precision), and 76\% F1 score for 269 DNA-damaging compounds of the NCI60 drug response dataset. To assess the model's interpretability, we conducted a review of drug-gene co-occurrences in Pubmed abstracts in comparison to the top 5 genes with the highest attention coefficients for each drug. We also examined whether known relationships were retained in the model by inspecting the neighborhoods of topoisomerase-related drugs. For example, our model retained TOP1 as a highly weighted predictive feature for irinotecan and topotecan, in addition to other genes that could potentially be regulators of the drugs. Our method can be used to accurately predict sensitivity to drugs and may be useful in the identification of biomarkers relating to the treatment of cancer patients.

Title: Spatial Semantic Recurrent Mining for Referring Image Segmentation

• Authors: Jiaxing Yang, Lihe Zhang, Jiayu Sun, Huchuan Lu
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Spatial Semantic Recurrent Mining for Referring Image Segmentation(https://arxiv.org/abs/)
• Keywords: segmentation
• Abstract: Referring Image Segmentation (RIS) consistently requires language and appearance semantics to more understand each other. The need becomes acute especially under hard situations. To achieve, existing works tend to resort to various trans-representing mechanisms to directly feed forward language semantic along main RGB branch, which however will result in referent distribution weakly-mined in space and non-referent semantic contaminated along channel. In this paper, we propose Spatial Semantic Recurrent Mining (S\textsuperscript{2}RM) to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. During fusion, S\textsuperscript{2}RM will first generate a constraint-weak yet distribution-aware language feature, then bundle features of each row and column from rotated features of one modality context to recurrently correlate relevant semantic contained in feature from other modality context, and finally resort to self-distilled weights to weigh on the contributions of different parsed semantics. Via coparsing, S\textsuperscript{2}RM transports information from the near and remote slice layers of generator context to the current slice layer of parsed context, capable of better modeling global relationship bidirectional and structured. Besides, we also propose a Cross-scale Abstract Semantic Guided Decoder (CASG) to emphasize the foreground of the referent, finally integrating different grained features at a comparatively low cost. Extensive experimental results on four current challenging datasets show that our proposed method performs favorably against other state-of-the-art algorithms.

Title: Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy

• Authors: Feng Wang, M. Cenk Gursoy, Senem Velipasalar
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy(https://arxiv.org/abs/)
• Keywords: privacy, robust, federate
• Abstract: In this paper, we propose feature-based federated transfer learning as a novel approach to improve communication efficiency by reducing the uplink payload by multiple orders of magnitude compared to that of existing approaches in federated learning and federated transfer learning. Specifically, in the proposed feature-based federated learning, we design the extracted features and outputs to be uploaded instead of parameter updates. For this distributed learning model, we determine the required payload and provide comparisons with the existing schemes. Subsequently, we analyze the robustness of feature-based federated transfer learning against packet loss, data insufficiency, and quantization. Finally, we address privacy considerations by defining and analyzing label privacy leakage and feature privacy leakage, and investigating mitigating approaches. For all aforementioned analyses, we evaluate the performance of the proposed learning scheme via experiments on an image classification task and a natural language processing task to demonstrate its effectiveness.

Title: Deep Learning in Earthquake Engineering: A Comprehensive Review

• Authors: Yazhou Xie
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Deep Learning in Earthquake Engineering: A Comprehensive Review(https://arxiv.org/abs/)
• Keywords: extraction, generative
• Abstract: This article surveys the growing interest in utilizing Deep Learning (DL) as a powerful tool to address challenging problems in earthquake engineering. Despite decades of advancement in domain knowledge, issues such as uncertainty in earthquake occurrence, unpredictable seismic loads, nonlinear structural responses, and community engagement remain difficult to tackle using domain-specific methods. DL offers promising solutions by leveraging its data-driven capacity for nonlinear mapping, sequential data modeling, automatic feature extraction, dimensionality reduction, optimal decision-making, etc. However, the literature lacks a comprehensive review that systematically covers a consistent scope intersecting DL and earthquake engineering. To bridge the gap, the article first discusses methodological advances to elucidate various applicable DL techniques, such as multi-layer perceptron (MLP), convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), autoencoder (AE), transfer learning (TL), reinforcement learning (RL), and graph neural network (GNN). A thorough research landscape is then disclosed by exploring various DL applications across different research topics, including vision-based seismic damage assessment and structural characterization, seismic demand and damage state prediction, seismic response history prediction, regional seismic risk assessment and community resilience, ground motion (GM) for engineering use, seismic response control, and the inverse problem of system/damage identification. Suitable DL techniques for each research topic are identified, emphasizing the preeminence of CNN for vision-based tasks, RNN for sequential data, RL for community resilience, and unsupervised learning for GM analysis. The article also discusses opportunities and challenges for leveraging DL in earthquake engineering research and practice.

Title: Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labels

• Authors: Guozhang Liu, Ting Liu, Mengke Yuan, Tao Pang, Guangxing Yang, Hao Fu, Tao Wang, Tongkui Liao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labels(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: The ambiguous appearance, tiny scale, and fine-grained classes of objects in remote sensing imagery inevitably lead to the noisy annotations in category labels of detection dataset. However, the effects and treatments of the label noises are underexplored in modern oriented remote sensing object detectors. To address this issue, we propose a robust oriented remote sensing object detection method through dynamic loss decay (DLD) mechanism, inspired by the two phase ``early-learning'' and ``memorization'' learning dynamics of deep neural networks on clean and noisy samples. To be specific, we first observe the end point of early learning phase termed as EL, after which the models begin to memorize the false labels that significantly degrade the detection accuracy. Secondly, under the guidance of the training indicator, the losses of each sample are ranked in descending order, and we adaptively decay the losses of the top K largest ones (bad samples) in the following epochs. Because these large losses are of high confidence to be calculated with wrong labels. Experimental results show that the method achieves excellent noise resistance performance tested on multiple public datasets such as HRSC2016 and DOTA-v1.0/v2.0 with synthetic category label noise. Our solution also has won the 2st place in the "fine-grained object detection based on sub-meter remote sensing imagery" track with noisy labels of 2023 National Big Data and Computing Intelligence Challenge.

Title: Unmasking Efficiency: Learning Salient Sparse Models in Non-IID Federated Learning

• Authors: Riyasat Ohib, Bishal Thapaliya, Gintare Karolina Dziugaite, Jingyu Liu, Vince Calhoun, Sergey Plis
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Unmasking Efficiency: Learning Salient Sparse Models in Non-IID Federated Learning(https://arxiv.org/abs/)
• Keywords: federate
• Abstract: In this work, we propose Salient Sparse Federated Learning (SSFL), a streamlined approach for sparse federated learning with efficient communication. SSFL identifies a sparse subnetwork prior to training, leveraging parameter saliency scores computed separately on local client data in non-IID scenarios, and then aggregated, to determine a global mask. Only the sparse model weights are communicated each round between the clients and the server. We validate SSFL's effectiveness using standard non-IID benchmarks, noting marked improvements in the sparsity--accuracy trade-offs. Finally, we deploy our method in a real-world federated learning framework and report improvement in communication time.

Title: SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction

• Authors: Zhihao Yu, Xu Chu, Yujie Jin, Yasha Wang, Junfeng Zhao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Electronic health record (EHR) data has emerged as a valuable resource for analyzing patient health status. However, the prevalence of missing data in EHR poses significant challenges to existing methods, leading to spurious correlations and suboptimal predictions. While various imputation techniques have been developed to address this issue, they often obsess unnecessary details and may introduce additional noise when making clinical predictions. To tackle this problem, we propose SMART, a Self-Supervised Missing-Aware RepresenTation Learning approach for patient health status prediction, which encodes missing information via elaborated attentions and learns to impute missing values through a novel self-supervised pre-training approach that reconstructs missing data representations in the latent space. By adopting missing-aware attentions and focusing on learning higher-order representations, SMART promotes better generalization and robustness to missing data. We validate the effectiveness of SMART through extensive experiments on six EHR tasks, demonstrating its superiority over state-of-the-art methods.

Title: Learning from Partial Label Proportions for Whole Slide Image Segmentation

• Authors: Shinnosuke Matsuo, Daiki Suehiro, Seiichi Uchida, Hiroaki Ito, Kazuhiro Terada, Akihiko Yoshizawa, Ryoma Bise
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Learning from Partial Label Proportions for Whole Slide Image Segmentation(https://arxiv.org/abs/)
• Keywords: segmentation
• Abstract: In this paper, we address the segmentation of tumor subtypes in whole slide images (WSI) by utilizing incomplete label proportions. Specifically, we utilize `partial' label proportions, which give the proportions among tumor subtypes but do not give the proportion between tumor and non-tumor. Partial label proportions are recorded as the standard diagnostic information by pathologists, and we, therefore, want to use them for realizing the segmentation model that can classify each WSI patch into one of the tumor subtypes or non-tumor. We call this problem ``learning from partial label proportions (LPLP)'' and formulate the problem as a weakly supervised learning problem. Then, we propose an efficient algorithm for this challenging problem by decomposing it into two weakly supervised learning subproblems: multiple instance learning (MIL) and learning from label proportions (LLP). These subproblems are optimized efficiently in the end-to-end manner. The effectiveness of our algorithm is demonstrated through experiments conducted on two WSI datasets.

Title: AMSNet: Netlist Dataset for AMS Circuits

• Authors: Zhuofu Tao, Yichen Shi, Yiru Huo, Rui Ye, Zonghang Li, Li Huang, Chen Wu, Na Bai, Zhiping Yu, Ting-Jung Lin, Lei He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] AMSNet: Netlist Dataset for AMS Circuits(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Today's analog/mixed-signal (AMS) integrated circuit (IC) designs demand substantial manual intervention. The advent of multimodal large language models (MLLMs) has unveiled significant potential across various fields, suggesting their applicability in streamlining large-scale AMS IC design as well. A bottleneck in employing MLLMs for automatic AMS circuit generation is the absence of a comprehensive dataset delineating the schematic-netlist relationship. We therefore design an automatic technique for converting schematics into netlists, and create dataset AMSNet, encompassing transistor-level schematics and corresponding SPICE format netlists. With a growing size, AMSNet can significantly facilitate exploration of MLLM applications in AMS circuit design. We have made an initial set of netlists public, and will make both our netlist generation tool and the full dataset available upon publishing of this paper.

Title: Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving

• Authors: Ross Greer, Mohan Trivedi
• Subjects: cs.LG, cs.AI, cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: This study investigates the use of trajectory and dynamic state information for efficient data curation in autonomous driving machine learning tasks. We propose methods for clustering trajectory-states and sampling strategies in an active learning framework, aiming to reduce annotation and data costs while maintaining model performance. Our approach leverages trajectory information to guide data selection, promoting diversity in the training data. We demonstrate the effectiveness of our methods on the trajectory prediction task using the nuScenes dataset, showing consistent performance gains over random sampling across different data pool sizes, and even reaching sub-baseline displacement errors at just 50% of the data cost. Our results suggest that sampling typical data initially helps overcome the ''cold start problem,'' while introducing novelty becomes more beneficial as the training pool size increases. By integrating trajectory-state-informed active learning, we demonstrate that more efficient and robust autonomous driving systems are possible and practical using low-cost data curation strategies.

Title: Dim Small Target Detection and Tracking: A Novel Method Based on Temporal Energy Selective Scaling and Trajectory Association

• Authors: Weihua Gao, Wenlong Niu, Wenlong Lu, Pengcheng Wang, Zhaoyuan Qi, Xiaodong Peng, Zhen Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Dim Small Target Detection and Tracking: A Novel Method Based on Temporal Energy Selective Scaling and Trajectory Association(https://arxiv.org/abs/)
• Keywords: extraction, segmentation
• Abstract: The detection and tracking of small targets in passive optical remote sensing (PORS) has broad applications. However, most of the previously proposed methods seldom utilize the abundant temporal features formed by target motion, resulting in poor detection and tracking performance for low signal-to-clutter ratio (SCR) targets. In this article, we analyze the difficulty based on spatial features and the feasibility based on temporal features of realizing effective detection. According to this analysis, we use a multi-frame as a detection unit and propose a detection method based on temporal energy selective scaling (TESS). Specifically, we investigated the composition of intensity temporal profiles (ITPs) formed by pixels on a multi-frame detection unit. For the target-present pixel, the target passing through the pixel will bring a weak transient disturbance on the ITP and introduce a change in the statistical properties of ITP. We use a well-designed function to amplify the transient disturbance, suppress the background and noise components, and output the trajectory of the target on the multi-frame detection unit. Subsequently, to solve the contradiction between the detection rate and the false alarm rate brought by the traditional threshold segmentation, we associate the temporal and spatial features of the output trajectory and propose a trajectory extraction method based on the 3D Hough transform. Finally, we model the trajectory of the target and propose a trajectory-based multi-target tracking method. Compared with the various state-of-the-art detection and tracking methods, experiments in multiple scenarios prove the superiority of our proposed methods.

Title: A safety realignment framework via subspace-oriented model fusion for large language models

• Authors: Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, Liang He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] A safety realignment framework via subspace-oriented model fusion for large language models(https://arxiv.org/abs/)
• Keywords: attack, large language model
• Abstract: The current safeguard mechanisms for large language models (LLMs) are indeed susceptible to jailbreak attacks, making them inherently fragile. Even the process of fine-tuning on apparently benign data for downstream tasks can jeopardize safety. One potential solution is to conduct safety fine-tuning subsequent to downstream fine-tuning. However, there's a risk of catastrophic forgetting during safety fine-tuning, where LLMs may regain safety measures but lose the task-specific knowledge acquired during downstream fine-tuning. In this paper, we introduce a safety realignment framework through subspace-oriented model fusion (SOMF), aiming to combine the safeguard capabilities of initially aligned model and the current fine-tuned model into a realigned model. Our approach begins by disentangling all task vectors from the weights of each fine-tuned model. We then identify safety-related regions within these vectors by subspace masking techniques. Finally, we explore the fusion of the initial safely aligned LLM with all task vectors based on the identified safety subspace. We validate that our safety realignment framework satisfies the safety requirements of a single fine-tuned model as well as multiple models during their fusion. Our findings confirm that SOMF preserves safety without notably compromising performance on downstream tasks, including instruction following in Chinese, English, and Hindi, as well as problem-solving capabilities in Code and Math.

Title: CTS: A Consistency-Based Medical Image Segmentation Model

• Authors: Kejia Zhang, Lan Zhang, Haiwei Pan, Baolong Yu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] CTS: A Consistency-Based Medical Image Segmentation Model(https://arxiv.org/abs/)
• Keywords: diffusion, generative, segmentation
• Abstract: In medical image segmentation tasks, diffusion models have shown significant potential. However, mainstream diffusion models suffer from drawbacks such as multiple sampling times and slow prediction results. Recently, consistency models, as a standalone generative network, have resolved this issue. Compared to diffusion models, consistency models can reduce the sampling times to once, not only achieving similar generative effects but also significantly speeding up training and prediction. However, they are not suitable for image segmentation tasks, and their application in the medical imaging field has not yet been explored. Therefore, this paper applies the consistency model to medical image segmentation tasks, designing multi-scale feature signal supervision modes and loss function guidance to achieve model convergence. Experiments have verified that the CTS model can obtain better medical image segmentation results with a single sampling during the test phase.

Title: Response Matching for generating materials and molecules

• Authors: Bingqing Cheng
• Subjects: cs.LG, cond-mat.mtrl-sci, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Response Matching for generating materials and molecules(https://arxiv.org/abs/)
• Keywords: diffusion, generative
• Abstract: Machine learning has recently emerged as a powerful tool for generating new molecular and material structures. The success of state-of-the-art models stems from their ability to incorporate physical symmetries, such as translation, rotation, and periodicity. Here, we present a novel generative method called Response Matching (RM), which leverages the fact that each stable material or molecule exists at the minimum of its potential energy surface. Consequently, any perturbation induces a response in energy and stress, driving the structure back to equilibrium. Matching to such response is closely related to score matching in diffusion models. By employing the combination of a machine learning interatomic potential and random structure search as the denoising model, RM exploits the locality of atomic interactions, and inherently respects permutation, translation, rotation, and periodic invariances. RM is the first model to handle both molecules and bulk materials under the same framework. We demonstrate the efficiency and generalization of RM across three systems: a small organic molecular dataset, stable crystals from the Materials Project, and one-shot learning on a single diamond configuration.

Title: Improving Transformers using Faithful Positional Encoding

• Authors: Tsuyoshi Idé, Jokin Labaien, Pin-Yu Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Improving Transformers using Faithful Positional Encoding(https://arxiv.org/abs/)
• Keywords: transformer
• Abstract: We propose a new positional encoding method for a neural network architecture called the Transformer. Unlike the standard sinusoidal positional encoding, our approach is based on solid mathematical grounds and has a guarantee of not losing information about the positional order of the input sequence. We show that the new encoding approach systematically improves the prediction performance in the time-series classification task.

Title: RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing

• Authors: Jiamei Xiong, Xuefeng Yan, Yongzhen Wang, Wei Zhao, Xiao-Ping Zhang, Mingqiang Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing(https://arxiv.org/abs/)
• Keywords: diffusion
• Abstract: Haze severely degrades the visual quality of remote sensing images and hampers the performance of automotive navigation, intelligent monitoring, and urban management. The emerging denoising diffusion probabilistic model (DDPM) exhibits the significant potential for dense haze removal with its strong generation ability. Since remote sensing images contain extensive small-scale texture structures, it is important to effectively restore image details from hazy images. However, current wisdom of DDPM fails to preserve image details and color fidelity well, limiting its dehazing capacity for remote sensing images. In this paper, we propose a novel unified Fourier-aware diffusion model for remote sensing image dehazing, termed RSHazeDiff. From a new perspective, RSHazeDiff explores the conditional DDPM to improve image quality in dense hazy scenarios, and it makes three key contributions. First, RSHazeDiff refines the training phase of diffusion process by performing noise estimation and reconstruction constraints in a coarse-to-fine fashion. Thus, it remedies the unpleasing results caused by the simple noise estimation constraint in DDPM. Second, by taking the frequency information as important prior knowledge during iterative sampling steps, RSHazeDiff can preserve more texture details and color fidelity in dehazed images. Third, we design a global compensated learning module to utilize the Fourier transform to capture the global dependency features of input images, which can effectively mitigate the effects of boundary artifacts when processing fixed-size patches. Experiments on both synthetic and real-world benchmarks validate the favorable performance of RSHazeDiff over multiple state-of-the-art methods. Source code will be released at this https URL.

Title: Towards Next-Generation Steganalysis: LLMs Unleash the Power of Detecting Steganography

• Authors: Minhao Bai. Jinshuai Yang, Kaiyi Pang, Huili Wang, Yongfeng Huang
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Towards Next-Generation Steganalysis: LLMs Unleash the Power of Detecting Steganography(https://arxiv.org/abs/)
• Keywords: security, generative, large language model
• Abstract: Linguistic steganography provides convenient implementation to hide messages, particularly with the emergence of AI generation technology. The potential abuse of this technology raises security concerns within societies, calling for powerful linguistic steganalysis to detect carrier containing steganographic messages. Existing methods are limited to finding distribution differences between steganographic texts and normal texts from the aspect of symbolic statistics. However, the distribution differences of both kinds of texts are hard to build precisely, which heavily hurts the detection ability of the existing methods in realistic scenarios. To seek a feasible way to construct practical steganalysis in real world, this paper propose to employ human-like text processing abilities of large language models (LLMs) to realize the difference from the aspect of human perception, addition to traditional statistic aspect. Specifically, we systematically investigate the performance of LLMs in this task by modeling it as a generative paradigm, instead of traditional classification paradigm. Extensive experiment results reveal that generative LLMs exhibit significant advantages in linguistic steganalysis and demonstrate performance trends distinct from traditional approaches. Results also reveal that LLMs outperform existing baselines by a wide margin, and the domain-agnostic ability of LLMs makes it possible to train a generic steganalysis model (Both codes and trained models are openly available in this https URL).

Title: Optimizing Sensor Network Design for Multiple Coverage

• Authors: Lukas Taus, Yen-Hsi Richard Tsai
• Subjects: cs.LG, cs.RO, math.OC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Optimizing Sensor Network Design for Multiple Coverage(https://arxiv.org/abs/)
• Keywords: defense, attack, robust
• Abstract: Sensor placement optimization methods have been studied extensively. They can be applied to a wide range of applications, including surveillance of known environments, optimal locations for 5G towers, and placement of missile defense systems. However, few works explore the robustness and efficiency of the resulting sensor network concerning sensor failure or adversarial attacks. This paper addresses this issue by optimizing for the least number of sensors to achieve multiple coverage of non-simply connected domains by a prescribed number of sensors. We introduce a new objective function for the greedy (next-best-view) algorithm to design efficient and robust sensor networks and derive theoretical bounds on the network's optimality. We further introduce a Deep Learning model to accelerate the algorithm for near real-time computations. The Deep Learning model requires the generation of training examples. Correspondingly, we show that understanding the geometric properties of the training data set provides important insights into the performance and training process of deep learning techniques. Finally, we demonstrate that a simple parallel version of the greedy approach using a simpler objective can be highly competitive.

Title: Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

• Authors: Kai Hu, Weichen Yu, Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Yining Li, Kai Chen, Zhiqiang Shen, Matt Fredrikson
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization(https://arxiv.org/abs/)
• Keywords: attack, large language model
• Abstract: Recent research indicates that large language models (LLMs) are susceptible to jailbreaking attacks that can generate harmful content. This paper introduces a novel token-level attack method, Adaptive Dense-to-Sparse Constrained Optimization (ADC), which effectively jailbreaks several open-source LLMs. Our approach relaxes the discrete jailbreak optimization into a continuous optimization and progressively increases the sparsity of the optimizing vectors. Consequently, our method effectively bridges the gap between discrete and continuous space optimization. Experimental results demonstrate that our method is more effective and efficient than existing token-level methods. On Harmbench, our method achieves state of the art attack success rate on seven out of eight LLMs. Code will be made available. Trigger Warning: This paper contains model behavior that can be offensive in nature.

Title: SOEDiff: Efficient Distillation for Small Object Editing

• Authors: Qihe Pan, Zicheng Wang, Zhen Zhao, Yiming Wu, Sifan Long, Haoran Liang, Ronghua Liang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] SOEDiff: Efficient Distillation for Small Object Editing(https://arxiv.org/abs/)
• Keywords: diffusion
• Abstract: In this paper, we delve into a new task known as small object editing (SOE), which focuses on text-based image inpainting within a constrained, small-sized area. Despite the remarkable success have been achieved by current image inpainting approaches, their application to the SOE task generally results in failure cases such as Object Missing, Text-Image Mismatch, and Distortion. These failures stem from the limited use of small-sized objects in training datasets and the downsampling operations employed by U-Net models, which hinders accurate generation. To overcome these challenges, we introduce a novel training-based approach, SOEDiff, aimed at enhancing the capability of baseline models like StableDiffusion in editing small-sized objects while minimizing training costs. Specifically, our method involves two key components: SO-LoRA, which efficiently fine-tunes low-rank matrices, and Cross-Scale Score Distillation loss, which leverages high-resolution predictions from the pre-trained teacher diffusion model. Our method presents significant improvements on the test dataset collected from MSCOCO and OpenImage, validating the effectiveness of our proposed method in small object editing. In particular, when comparing SOEDiff with SD-I model on the OpenImage-f dataset, we observe a 0.99 improvement in CLIP-Score and a reduction of 2.87 in FID. Our project page can be found in this https URL.

Title: RobustMVS: Single Domain Generalized Deep Multi-view Stereo

• Authors: Hongbin Xu, Weitao Chen, Baigui Sun, Xuansong Xie, Wenxiong Kang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] RobustMVS: Single Domain Generalized Deep Multi-view Stereo(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Despite the impressive performance of Multi-view Stereo (MVS) approaches given plenty of training samples, the performance degradation when generalizing to unseen domains has not been clearly explored yet. In this work, we focus on the domain generalization problem in MVS. To evaluate the generalization results, we build a novel MVS domain generalization benchmark including synthetic and real-world datasets. In contrast to conventional domain generalization benchmarks, we consider a more realistic but challenging scenario, where only one source domain is available for training. The MVS problem can be analogized back to the feature matching task, and maintaining robust feature consistency among views is an important factor for improving generalization performance. To address the domain generalization problem in MVS, we propose a novel MVS framework, namely RobustMVS. A DepthClustering-guided Whitening (DCW) loss is further introduced to preserve the feature consistency among different views, which decorrelates multi-view features from viewpoint-specific style information based on geometric priors from depth maps. The experimental results further show that our method achieves superior performance on the domain generalization benchmark.

Title: OpenGait: A Comprehensive Benchmark Study for Gait Recognition towards Better Practicality

• Authors: Chao Fan, Saihui Hou, Junhao Liang, Chuanfu Shen, Jingzhe Ma, Dongyang Jin, Yongzhen Huang, Shiqi Yu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] OpenGait: A Comprehensive Benchmark Study for Gait Recognition towards Better Practicality(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Gait recognition, a rapidly advancing vision technology for person identification from a distance, has made significant strides in indoor settings. However, evidence suggests that existing methods often yield unsatisfactory results when applied to newly released real-world gait datasets. Furthermore, conclusions drawn from indoor gait datasets may not easily generalize to outdoor ones. Therefore, the primary goal of this work is to present a comprehensive benchmark study aimed at improving practicality rather than solely focusing on enhancing performance. To this end, we first develop OpenGait, a flexible and efficient gait recognition platform. Using OpenGait as a foundation, we conduct in-depth ablation experiments to revisit recent developments in gait recognition. Surprisingly, we detect some imperfect parts of certain prior methods thereby resulting in several critical yet undiscovered insights. Inspired by these findings, we develop three structurally simple yet empirically powerful and practically robust baseline models, i.e., DeepGaitV2, SkeletonGait, and SkeletonGait++, respectively representing the appearance-based, model-based, and multi-modal methodology for gait pattern description. Beyond achieving SoTA performances, more importantly, our careful exploration sheds new light on the modeling experience of deep gait models, the representational capacity of typical gait modalities, and so on. We hope this work can inspire further research and application of gait recognition towards better practicality. The code is available at this https URL.

Title: A Hierarchically Feature Reconstructed Autoencoder for Unsupervised Anomaly Detection

• Authors: Honghui Chen, Pingping Chen, Huan Mao, Mengxi Jiang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] A Hierarchically Feature Reconstructed Autoencoder for Unsupervised Anomaly Detection(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Anomaly detection and localization without any manual annotations and prior knowledge is a challenging task under the setting of unsupervised learning. The existing works achieve excellent performance in the anomaly detection, but with complex networks or cumbersome pipelines. To address this issue, this paper explores a simple but effective architecture in the anomaly detection. It consists of a well pre-trained encoder to extract hierarchical feature representations and a decoder to reconstruct these intermediate features from the encoder. In particular, it does not require any data augmentations and anomalous images for training. The anomalies can be detected when the decoder fails to reconstruct features well, and then errors of hierarchical feature reconstruction are aggregated into an anomaly map to achieve anomaly localization. The difference comparison between those features of encoder and decode lead to more accurate and robust localization results than the comparison in single feature or pixel-by-pixel comparison in the conventional works. Experiment results show that the proposed method outperforms the state-of-the-art methods on MNIST, Fashion-MNIST, CIFAR-10, and MVTec Anomaly Detection datasets on both anomaly detection and localization.

Title: Curriculum Dataset Distillation

• Authors: Zhiheng Ma, Anjia Cao, Funing Yang, Xing Wei
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Curriculum Dataset Distillation(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Most dataset distillation methods struggle to accommodate large-scale datasets due to their substantial computational and memory requirements. In this paper, we present a curriculum-based dataset distillation framework designed to harmonize scalability with efficiency. This framework strategically distills synthetic images, adhering to a curriculum that transitions from simple to complex. By incorporating curriculum evaluation, we address the issue of previous methods generating images that tend to be homogeneous and simplistic, doing so at a manageable computational cost. Furthermore, we introduce adversarial optimization towards synthetic images to further improve their representativeness and safeguard against their overfitting to the neural network involved in distilling. This enhances the generalization capability of the distilled images across various neural network architectures and also increases their robustness to noise. Extensive experiments demonstrate that our framework sets new benchmarks in large-scale dataset distillation, achieving substantial improvements of 11.1\% on Tiny-ImageNet, 9.0\% on ImageNet-1K, and 7.3\% on ImageNet-21K. The source code will be released to the community.

Title: Adapting Abstract Meaning Representation Parsing to the Clinical Narrative -- the SPRING THYME parser

• Authors: Jon Z. Cai, Kristin Wright-Bettner, Martha Palmer, Guergana K. Savova, James H. Martin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Adapting Abstract Meaning Representation Parsing to the Clinical Narrative -- the SPRING THYME parser(https://arxiv.org/abs/)
• Keywords: robust, interpretability
• Abstract: This paper is dedicated to the design and evaluation of the first AMR parser tailored for clinical notes. Our objective was to facilitate the precise transformation of the clinical notes into structured AMR expressions, thereby enhancing the interpretability and usability of clinical text data at scale. Leveraging the colon cancer dataset from the Temporal Histories of Your Medical Events (THYME) corpus, we adapted a state-of-the-art AMR parser utilizing continuous training. Our approach incorporates data augmentation techniques to enhance the accuracy of AMR structure predictions. Notably, through this learning strategy, our parser achieved an impressive F1 score of 88% on the THYME corpus's colon cancer dataset. Moreover, our research delved into the efficacy of data required for domain adaptation within the realm of clinical notes, presenting domain adaptation data requirements for AMR parsing. This exploration not only underscores the parser's robust performance but also highlights its potential in facilitating a deeper understanding of clinical narratives through structured semantic representations.

Title: Cross-Input Certified Training for Universal Perturbations

• Authors: Changming Xu, Gagandeep Singh
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Cross-Input Certified Training for Universal Perturbations(https://arxiv.org/abs/)
• Keywords: attack, robust
• Abstract: Existing work in trustworthy machine learning primarily focuses on single-input adversarial perturbations. In many real-world attack scenarios, input-agnostic adversarial attacks, e.g. universal adversarial perturbations (UAPs), are much more feasible. Current certified training methods train models robust to single-input perturbations but achieve suboptimal clean and UAP accuracy, thereby limiting their applicability in practical applications. We propose a novel method, CITRUS, for certified training of networks robust against UAP attackers. We show in an extensive evaluation across different datasets, architectures, and perturbation magnitudes that our method outperforms traditional certified training methods on standard accuracy (up to 10.3\%) and achieves SOTA performance on the more practical certified UAP accuracy metric.

Title: QMedShield: A Novel Quantum Chaos-based Image Encryption Scheme for Secure Medical Image Storage in the Cloud

• Authors: Arun Amaithi Rajan, Vetriselvi V
• Subjects: cs.CR, cs.MM
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] QMedShield: A Novel Quantum Chaos-based Image Encryption Scheme for Secure Medical Image Storage in the Cloud(https://arxiv.org/abs/)
• Keywords: secure, security, privacy, protect, attack, robust, diffusion
• Abstract: In the age of digital technology, medical images play a crucial role in the healthcare industry which aids surgeons in making precise decisions and reducing the diagnosis time. However, the storage of large amounts of these images in third-party cloud services raises privacy and security concerns. There are a lot of classical security mechanisms to protect them. Although, the advent of quantum computing entails the development of quantum-based encryption models for healthcare. Hence, we introduce a novel quantum chaos-based encryption scheme for medical images in this article. The model comprises bit-plane scrambling, quantum logistic map, quantum operations in the diffusion phase and hybrid chaotic map, DNA encoding, and computations in the confusion phase to transform the plain medical image into a cipher medical image. The proposed scheme has been evaluated using multiple statistical measures and validated against more attacks such as differential attacks with three different medical datasets. Hence the introduced encryption model has proved to be attack-resistant and robust than other existing image encryption schemes, ensuring the secure storage of medical images in cloud environments.

Title: Flexible image analysis for law enforcement agencies with deep neural networks to determine: where, who and what

• Authors: Henri Bouma, Bart Joosten, Maarten C Kruithof, Maaike H T de Boer, Alexandru Ginsca (LIST (CEA)), Benjamin Labbe (LIST (CEA)), Quoc T Vuong (LIST (CEA))
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Flexible image analysis for law enforcement agencies with deep neural networks to determine: where, who and what(https://arxiv.org/abs/)
• Keywords: security
• Abstract: Due to the increasing need for effective security measures and the integration of cameras in commercial products, a hugeamount of visual data is created today. Law enforcement agencies (LEAs) are inspecting images and videos to findradicalization, propaganda for terrorist organizations and illegal products on darknet markets. This is time consuming.Instead of an undirected search, LEAs would like to adapt to new crimes and threats, and focus only on data from specificlocations, persons or objects, which requires flexible interpretation of image content. Visual concept detection with deepconvolutional neural networks (CNNs) is a crucial component to understand the image content. This paper has fivecontributions. The first contribution allows image-based geo-localization to estimate the origin of an image. CNNs andgeotagged images are used to create a model that determines the location of an image by its pixel values. The secondcontribution enables analysis of fine-grained concepts to distinguish sub-categories in a generic concept. The proposedmethod encompasses data acquisition and cleaning and concept hierarchies. The third contribution is the recognition ofperson attributes (e.g., glasses or moustache) to enable query by textual description for a person. The person-attributeproblem is treated as a specific sub-task of concept classification. The fourth contribution is an intuitive image annotationtool based on active learning. Active learning allows users to define novel concepts flexibly and train CNNs with minimalannotation effort. The fifth contribution increases the flexibility for LEAs in the query definition by using query expansion.Query expansion maps user queries to known and detectable concepts. Therefore, no prior knowledge of the detectableconcepts is required for the users. The methods are validated on data with varying locations (popular and non-touristiclocations), varying person attributes (CelebA dataset), and varying number of annotations.

Title: A first look into Utiq: Next-generation cookies at the ISP level

• Authors: Ismael Castell-Uroz, Pere Barlet-Ros
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] A first look into Utiq: Next-generation cookies at the ISP level(https://arxiv.org/abs/)
• Keywords: privacy
• Abstract: Online privacy has become increasingly important in recent years. While third-party cookies have been widely used for years, they have also been criticized for their potential impact on user privacy. They can be used by advertisers to track users across multiple sites, allowing them to build detailed profiles of their behavior and interests. However, nowadays, many browsers allow users to block third-party cookies, which limits their usefulness for advertisers. In this paper, we take a first look at Utiq, a new way of user tracking performed directly by the ISP, to substitute the third-party cookies used until now. We study the main properties of this new identification methodology and their adoption on the 10K most popular websites. Our results show that, although still marginal due to the restrictions imposed by the system, between 0.7% and 1.2% of websites already include Utiq as one of their user identification methods.

Title: ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models

• Authors: Siwei Wang, Yifei Shen, Shi Feng, Haoran Sun, Shang-Hua Teng, Wei Chen
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models(https://arxiv.org/abs/)
• Keywords: transformer
• Abstract: In this paper, we present the findings of our Project ALPINE which stands for ``Autoregressive Learning for Planning In NEtworks." Project ALPINE initiates a theoretical investigation into the development of planning capabilities in Transformer-based language models through their autoregressive learning mechanisms, aiming to identify any potential limitations in their planning abilities. We abstract planning as a network path-finding task where the objective is to generate a valid path from a specified source node to a designated target node. In terms of expressiveness, we show that the Transformer is capable of executing path-finding by embedding the adjacency and reachability matrices within its weights. Our theoretical analysis of the gradient-based learning dynamic of the Transformer reveals that the Transformer is capable of learning both the adjacency matrix and a limited form of the reachability matrix. These theoretical insights are then validated through experiments, which demonstrate that the Transformer indeed learns the adjacency matrix and an incomplete reachability matrix, which aligns with the predictions made in our theoretical analysis. Additionally, when applying our methodology to a real-world planning benchmark, called Blocksworld, our observations remain consistent. Our theoretical and empirical analyses further unveil a potential limitation of Transformer in path-finding: it cannot identify reachability relationships through transitivity, and thus would fail when path concatenation is needed to generate a path. In summary, our findings shed new light on how the internal mechanisms of autoregressive learning enable planning in networks. This study may contribute to our understanding of the general planning capabilities in other related domains.

Title: Word Alignment as Preference for Machine Translation

• Authors: Qiyu Wu, Masaaki Nagata, Zhongtao Miao, Yoshimasa Tsuruoka
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Word Alignment as Preference for Machine Translation(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: The problem of hallucination and omission, a long-standing problem in machine translation (MT), is more pronounced when a large language model (LLM) is used in MT because an LLM itself is susceptible to these phenomena. In this work, we mitigate the problem in an LLM-based MT model by guiding it to better word alignment. We first study the correlation between word alignment and the phenomena of hallucination and omission in MT. Then we propose to utilize word alignment as preference to optimize the LLM-based MT model. The preference data are constructed by selecting chosen and rejected translations from multiple MT tools. Subsequently, direct preference optimization is used to optimize the LLM-based model towards the preference signal. Given the absence of evaluators specifically designed for hallucination and omission in MT, we further propose selecting hard instances and utilizing GPT-4 to directly evaluate the performance of the models in mitigating these issues. We verify the rationality of these designed evaluation methods by experiments, followed by extensive results demonstrating the effectiveness of word alignment-based preference optimization to mitigate hallucination and omission.

Title: Reduce to the MACs -- Privacy Friendly Generic Probe Requests

• Authors: Johanna Ansohn McDougall, Alessandro Brighente, Anne Kunstmann, Niklas Zapatka, Hannes Federrath
• Subjects: cs.CR, cs.NI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Reduce to the MACs -- Privacy Friendly Generic Probe Requests(https://arxiv.org/abs/)
• Keywords: privacy, robust
• Abstract: Abstract. Since the introduction of active discovery in Wi-Fi networks, users can be tracked via their probe requests. Although manufacturers typically try to conceal Media Access Control (MAC) addresses using MAC address randomisation, probe requests still contain Information Elements (IEs) that facilitate device identification. This paper introduces generic probe requests: By removing all unnecessary information from IEs, the requests become indistinguishable from one another, letting single devices disappear in the largest possible anonymity set. Conducting a comprehensive evaluation, we demonstrate that a large IE set contained within undirected probe requests does not necessarily imply fast connection establishment. Furthermore, we show that minimising IEs to nothing but Supported Rates would enable 82.55% of the devices to share the same anonymity set. Our contributions provide a significant advancement in the pursuit of robust privacy solutions for wireless networks, paving the way for more user anonymity and less surveillance in wireless communication ecosystems.

Title: Graph Neural Network based Handwritten Trajectories Recognition

• Authors: Anuj Sharma, Sukhdeep Singh, S Ratna
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Graph Neural Network based Handwritten Trajectories Recognition(https://arxiv.org/abs/)
• Keywords: extraction
• Abstract: The graph neural networks has been proved to be an efficient machine learning technique in real life applications. The handwritten recognition is one of the useful area in real life use where both offline and online handwriting recognition are required. The chain code as feature extraction technique has shown significant results in literature and we have been able to use chain codes with graph neural networks. To the best of our knowledge, this work presents first time a novel combination of handwritten trajectories features as chain codes and graph neural networks together. The handwritten trajectories for offline handwritten text has been evaluated using recovery of drawing order, whereas online handwritten trajectories are directly used with chain codes. Our results prove that present combination surpass previous results and minimize error rate in few epochs only.

Title: New Textual Corpora for Serbian Language Modeling

• Authors: Mihailo Škorić, Nikola Janković
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] New Textual Corpora for Serbian Language Modeling(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: This paper will present textual corpora for Serbian (and Serbo-Croatian), usable for the training of large language models and publicly available at one of the several notable online repositories. Each corpus will be classified using multiple methods and its characteristics will be detailed. Additionally, the paper will introduce three new corpora: a new umbrella web corpus of Serbo-Croatian, a new high-quality corpus based on the doctoral dissertations stored within National Repository of Doctoral Dissertations from all Universities in Serbia, and a parallel corpus of abstract translation from the same source. The uniqueness of both old and new corpora will be accessed via frequency-based stylometric methods, and the results will be briefly discussed.

Title: Does Machine Bring in Extra Bias in Learning? Approximating Fairness in Models Promptly

• Authors: Yijun Bian, Yujie Luo
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Does Machine Bring in Extra Bias in Learning? Approximating Fairness in Models Promptly(https://arxiv.org/abs/)
• Keywords: fair
• Abstract: Providing various machine learning (ML) applications in the real world, concerns about discrimination hidden in ML models are growing, particularly in high-stakes domains. Existing techniques for assessing the discrimination level of ML models include commonly used group and individual fairness measures. However, these two types of fairness measures are usually hard to be compatible with each other, and even two different group fairness measures might be incompatible as well. To address this issue, we investigate to evaluate the discrimination level of classifiers from a manifold perspective and propose a "harmonic fairness measure via manifolds (HFM)" based on distances between sets. Yet the direct calculation of distances might be too expensive to afford, reducing its practical applicability. Therefore, we devise an approximation algorithm named "Approximation of distance between sets (ApproxDist)" to facilitate accurate estimation of distances, and we further demonstrate its algorithmic effectiveness under certain reasonable assumptions. Empirical results indicate that the proposed fairness measure HFM is valid and that the proposed ApproxDist is effective and efficient.

Title: Dance Any Beat: Blending Beats with Visuals in Dance Video Generation

• Authors: Xuanchen Wang, Heng Wang, Dongnan Liu, Weidong Cai
• Subjects: cs.CV, cs.AI, cs.MM, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Dance Any Beat: Blending Beats with Visuals in Dance Video Generation(https://arxiv.org/abs/)
• Keywords: diffusion
• Abstract: The task of generating dance from music is crucial, yet current methods, which mainly produce joint sequences, lead to outputs that lack intuitiveness and complicate data collection due to the necessity for precise joint annotations. We introduce a Dance Any Beat Diffusion model, namely DabFusion, that employs music as a conditional input to directly create dance videos from still images, utilizing conditional image-to-video generation principles. This approach pioneers the use of music as a conditioning factor in image-to-video synthesis. Our method unfolds in two stages: training an auto-encoder to predict latent optical flow between reference and driving frames, eliminating the need for joint annotation, and training a U-Net-based diffusion model to produce these latent optical flows guided by music rhythm encoded by CLAP. Although capable of producing high-quality dance videos, the baseline model struggles with rhythm alignment. We enhance the model by adding beat information, improving synchronization. We introduce a 2D motion-music alignment score (2D-MM Align) for quantitative assessment. Evaluated on the AIST++ dataset, our enhanced model shows marked improvements in 2D-MM Align score and established metrics. Video results can be found on our project page: this https URL.

Title: Fair Generalized Linear Mixed Models

• Authors: Jan Pablo Burgard, João Vitor Pamplona
• Subjects: cs.LG, math.OC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Fair Generalized Linear Mixed Models(https://arxiv.org/abs/)
• Keywords: fair
• Abstract: When using machine learning for automated prediction, it is important to account for fairness in the prediction. Fairness in machine learning aims to ensure that biases in the data and model inaccuracies do not lead to discriminatory decisions. E.g., predictions from fair machine learning models should not discriminate against sensitive variables such as sexual orientation and ethnicity. The training data often in obtained from social surveys. In social surveys, oftentimes the data collection process is a strata sampling, e.g. due to cost restrictions. In strata samples, the assumption of independence between the observation is not fulfilled. Hence, if the machine learning models do not account for the strata correlations, the results may be biased. Especially high is the bias in cases where the strata assignment is correlated to the variable of interest. We present in this paper an algorithm that can handle both problems simultaneously, and we demonstrate the impact of stratified sampling on the quality of fair machine learning predictions in a reproducible simulation study.

Title: Dual-Segment Clustering Strategy for Federated Learning in Heterogeneous Environments

• Authors: Pengcheng Sun, Erwu Liu, Wei Ni, Kanglei Yu, Rui Wang, Abbas Jamalipour
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Dual-Segment Clustering Strategy for Federated Learning in Heterogeneous Environments(https://arxiv.org/abs/)
• Keywords: federate
• Abstract: Federated learning (FL) is a distributed machine learning paradigm with high efficiency and low communication load, only transmitting parameters or gradients of network. However, the non-independent and identically distributed (Non-IID) data characteristic has a negative impact on this paradigm. Furthermore, the heterogeneity of communication quality will significantly affect the accuracy of parameter transmission, causing a degradation in the performance of the FL system or even preventing its convergence. This letter proposes a dual-segment clustering (DSC) strategy, which first clusters the clients according to the heterogeneous communication conditions and then performs a second clustering by the sample size and label distribution, so as to solve the problem of data and communication heterogeneity. Experimental results show that the DSC strategy proposed in this letter can improve the convergence rate of FL, and has superiority on accuracy in a heterogeneous environment compared with the classical algorithm of cluster.

Title: Sign of the Times: Evaluating the use of Large Language Models for Idiomaticity Detection

• Authors: Dylan Phelps, Thomas Pickard, Maggie Mi, Edward Gow-Smith, Aline Villavicencio
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Sign of the Times: Evaluating the use of Large Language Models for Idiomaticity Detection(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Despite the recent ubiquity of large language models and their high zero-shot prompted performance across a wide range of tasks, it is still not known how well they perform on tasks which require processing of potentially idiomatic language. In particular, how well do such models perform in comparison to encoder-only models fine-tuned specifically for idiomaticity tasks? In this work, we attempt to answer this question by looking at the performance of a range of LLMs (both local and software-as-a-service models) on three idiomaticity datasets: SemEval 2022 Task 2a, FLUTE, and MAGPIE. Overall, we find that whilst these models do give competitive performance, they do not match the results of fine-tuned task-specific models, even at the largest scales (e.g. for GPT-4). Nevertheless, we do see consistent performance improvements across model scale. Additionally, we investigate prompting approaches to improve performance, and discuss the practicalities of using LLMs for these tasks.

Title: Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning

• Authors: Junfeng Chen, Kailiang Wu
• Subjects: cs.LG, math.NA
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning(https://arxiv.org/abs/)
• Keywords: interpretability, transformer
• Abstract: Operator learning for Partial Differential Equations (PDEs) is rapidly emerging as a promising approach for surrogate modeling of intricate systems. Transformers with the self-attention mechanism$\unicode{x2013}$a powerful tool originally designed for natural language processing$\unicode{x2013}$have recently been adapted for operator learning. However, they confront challenges, including high computational demands and limited interpretability. This raises a critical question: Is there a more efficient attention mechanism for Transformer-based operator learning? This paper proposes the Position-induced Transformer (PiT), built on an innovative position-attention mechanism, which demonstrates significant advantages over the classical self-attention in operator learning. Position-attention draws inspiration from numerical methods for PDEs. Different from self-attention, position-attention is induced by only the spatial interrelations of sampling positions for input functions of the operators, and does not rely on the input function values themselves, thereby greatly boosting efficiency. PiT exhibits superior performance over current state-of-the-art neural operators in a variety of complex operator learning tasks across diverse PDE benchmarks. Additionally, PiT possesses an enhanced discretization convergence feature, compared to the widely-used Fourier neural operator.

Title: DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

• Authors: Nima Fathi, Amar Kumar, Brennan Nichyporuk, Mohammad Havaei, Tal Arbel
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations(https://arxiv.org/abs/)
• Keywords: explainability, diffusion
• Abstract: Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at this https URL.

Title: Do language models capture implied discourse meanings? An investigation with exhaustivity implicatures of Korean morphology

• Authors: Hagyeong Shin, Sean Trott
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Do language models capture implied discourse meanings? An investigation with exhaustivity implicatures of Korean morphology(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Markedness in natural language is often associated with non-literal meanings in discourse. Differential Object Marking (DOM) in Korean is one instance of this phenomenon, where post-positional markers are selected based on both the semantic features of the noun phrases and the discourse features that are orthogonal to the semantic features. Previous work has shown that distributional models of language recover certain semantic features of words -- do these models capture implied discourse-level meanings as well? We evaluate whether a set of large language models are capable of associating discourse meanings with different object markings in Korean. Results suggest that discourse meanings of a grammatical marker can be more challenging to encode than that of a discourse marker.

Title: Comparing the Efficacy of GPT-4 and Chat-GPT in Mental Health Care: A Blind Assessment of Large Language Models for Psychological Support

• Authors: Birger Moell
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Comparing the Efficacy of GPT-4 and Chat-GPT in Mental Health Care: A Blind Assessment of Large Language Models for Psychological Support(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Background: Rapid advancements in natural language processing have led to the development of large language models with the potential to revolutionize mental health care. These models have shown promise in assisting clinicians and providing support to individuals experiencing various psychological challenges. Objective: This study aims to compare the performance of two large language models, GPT-4 and Chat-GPT, in responding to a set of 18 psychological prompts, to assess their potential applicability in mental health care settings. Methods: A blind methodology was employed, with a clinical psychologist evaluating the models' responses without knowledge of their origins. The prompts encompassed a diverse range of mental health topics, including depression, anxiety, and trauma, to ensure a comprehensive assessment. Results: The results demonstrated a significant difference in performance between the two models (p > 0.05). GPT-4 achieved an average rating of 8.29 out of 10, while Chat-GPT received an average rating of 6.52. The clinical psychologist's evaluation suggested that GPT-4 was more effective at generating clinically relevant and empathetic responses, thereby providing better support and guidance to potential users. Conclusions: This study contributes to the growing body of literature on the applicability of large language models in mental health care settings. The findings underscore the importance of continued research and development in the field to optimize these models for clinical use. Further investigation is necessary to understand the specific factors underlying the performance differences between the two models and to explore their generalizability across various populations and mental health conditions.

Title: TimeX++: Learning Time-Series Explanations with Information Bottleneck

• Authors: Zichuan Liu, Tianchun Wang, Jimeng Shi, Xu Zheng, Zhuomin Chen, Lei Song, Wenqian Dong, Jayantha Obeysekera, Farhad Shirani, Dongsheng Luo
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] TimeX++: Learning Time-Series Explanations with Information Bottleneck(https://arxiv.org/abs/)
• Keywords: explainability
• Abstract: Explaining deep learning models operating on time series data is crucial in various applications of interest which require interpretable and transparent insights from time series signals. In this work, we investigate this problem from an information theoretic perspective and show that most existing measures of explainability may suffer from trivial solutions and distributional shift issues. To address these issues, we introduce a simple yet practical objective function for time series explainable learning. The design of the objective function builds upon the principle of information bottleneck (IB), and modifies the IB objective function to avoid trivial solutions and distributional shift issues. We further present TimeX++, a novel explanation framework that leverages a parametric network to produce explanation-embedded instances that are both in-distributed and label-preserving. We evaluate TimeX++ on both synthetic and real-world datasets comparing its performance against leading baselines, and validate its practical efficacy through case studies in a real-world environmental application. Quantitative and qualitative evaluations show that TimeX++ outperforms baselines across all datasets, demonstrating a substantial improvement in explanation quality for time series data. The source code is available at \url{this https URL}.

Title: Agnostic Active Learning of Single Index Models with Linear Sample Complexity

• Authors: Aarshvi Gajjar, Wai Ming Tai, Xingyu Xu, Chinmay Hegde, Christopher Musco, Yi Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Agnostic Active Learning of Single Index Models with Linear Sample Complexity(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: We study active learning methods for single index models of the form $F({\mathbf x}) = f(\langle {\mathbf w}, {\mathbf x}\rangle)$, where $f:\mathbb{R} \to \mathbb{R}$ and ${\mathbf x,\mathbf w} \in \mathbb{R}^d$. In addition to their theoretical interest as simple examples of non-linear neural networks, single index models have received significant recent attention due to applications in scientific machine learning like surrogate modeling for partial differential equations (PDEs). Such applications require sample-efficient active learning methods that are robust to adversarial noise. I.e., that work even in the challenging agnostic learning setting. We provide two main results on agnostic active learning of single index models. First, when $f$ is known and Lipschitz, we show that $\tilde{O}(d)$ samples collected via {statistical leverage score sampling} are sufficient to learn a near-optimal single index model. Leverage score sampling is simple to implement, efficient, and already widely used for actively learning linear models. Our result requires no assumptions on the data distribution, is optimal up to log factors, and improves quadratically on a recent ${O}(d^{2})$ bound of \cite{gajjar2023active}. Second, we show that $\tilde{O}(d)$ samples suffice even in the more difficult setting when $f$ is \emph{unknown}. Our results leverage tools from high dimensional probability, including Dudley's inequality and dual Sudakov minoration, as well as a novel, distribution-aware discretization of the class of Lipschitz functions.

Title: Transfer Learning in Pre-Trained Large Language Models for Malware Detection Based on System Calls

• Authors: Pedro Miguel Sánchez Sánchez, Alberto Huertas Celdrán, Gérôme Bovet, Gregorio Martínez Pérez
• Subjects: cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Transfer Learning in Pre-Trained Large Language Models for Malware Detection Based on System Calls(https://arxiv.org/abs/)
• Keywords: security, protect, attack, robust, steal, large language model
• Abstract: In the current cybersecurity landscape, protecting military devices such as communication and battlefield management systems against sophisticated cyber attacks is crucial. Malware exploits vulnerabilities through stealth methods, often evading traditional detection mechanisms such as software signatures. The application of ML/DL in vulnerability detection has been extensively explored in the literature. However, current ML/DL vulnerability detection methods struggle with understanding the context and intent behind complex attacks. Integrating large language models (LLMs) with system call analysis offers a promising approach to enhance malware detection. This work presents a novel framework leveraging LLMs to classify malware based on system call data. The framework uses transfer learning to adapt pre-trained LLMs for malware detection. By retraining LLMs on a dataset of benign and malicious system calls, the models are refined to detect signs of malware activity. Experiments with a dataset of over 1TB of system calls demonstrate that models with larger context sizes, such as BigBird and Longformer, achieve superior accuracy and F1-Score of approximately 0.86. The results highlight the importance of context size in improving detection rates and underscore the trade-offs between computational complexity and performance. This approach shows significant potential for real-time detection in high-stakes environments, offering a robust solution to evolving cyber threats.

Title: Prompting-based Synthetic Data Generation for Few-Shot Question Answering

• Authors: Maximilian Schmidt, Andrea Bartezzaghi, Ngoc Thang Vu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Prompting-based Synthetic Data Generation for Few-Shot Question Answering(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Although language models (LMs) have boosted the performance of Question Answering, they still need plenty of data. Data annotation, in contrast, is a time-consuming process. This especially applies to Question Answering, where possibly large documents have to be parsed and annotated with questions and their corresponding answers. Furthermore, Question Answering models often only work well for the domain they were trained on. Since annotation is costly, we argue that domain-agnostic knowledge from LMs, such as linguistic understanding, is sufficient to create a well-curated dataset. With this motivation, we show that using large language models can improve Question Answering performance on various datasets in the few-shot setting compared to state-of-the-art approaches. For this, we perform data generation leveraging the Prompting framework, suggesting that language models contain valuable task-agnostic knowledge that can be used beyond the common pre-training/fine-tuning scheme. As a result, we consistently outperform previous approaches on few-shot Question Answering.

Title: Large Language Model Bias Mitigation from the Perspective of Knowledge Editing

• Authors: Ruizhe Chen, Yichen Li, Zikai Xiao, Zuozhu Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Large Language Model Bias Mitigation from the Perspective of Knowledge Editing(https://arxiv.org/abs/)
• Keywords: fair, large language model
• Abstract: Existing debiasing methods inevitably make unreasonable or undesired predictions as they are designated and evaluated to achieve parity across different social groups but leave aside individual facts, resulting in modified existing knowledge. In this paper, we first establish a new bias mitigation benchmark BiasKE leveraging existing and additional constructed datasets, which systematically assesses debiasing performance by complementary metrics on fairness, specificity, and generalization. Meanwhile, we propose a novel debiasing method, Fairness Stamp (FAST), which enables editable fairness through fine-grained calibration on individual biased knowledge. Comprehensive experiments demonstrate that FAST surpasses state-of-the-art baselines with remarkable debiasing performance while not hampering overall model capability for knowledge preservation, highlighting the prospect of fine-grained debiasing strategies for editable fairness in LLMs.

Title: The Unfairness of $\varepsilon$-Fairness

• Authors: Tolulope Fadina, Thorsten Schmidt
• Subjects: cs.LG, econ.TH, q-fin.MF, stat.ML
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] The Unfairness of $\varepsilon$-Fairness(https://arxiv.org/abs/)
• Keywords: fair
• Abstract: Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics may not fully capture the real-world consequences of unfairness. In this article, we adopt a utility-based approach to more accurately measure the real-world impacts of decision-making process. In particular, we show that if the concept of $\varepsilon$-fairness is employed, it can possibly lead to outcomes that are maximally unfair in the real-world context. Additionally, we address the common issue of unavailable data on false negatives by proposing a reduced setting that still captures essential fairness considerations. We illustrate our findings with two real-world examples: college admissions and credit risk assessment. Our analysis reveals that while traditional probability-based evaluations might suggest fairness, a utility-based approach uncovers the necessary actions to truly achieve equality. For instance, in the college admission case, we find that enhancing completion rates is crucial for ensuring fairness. Summarizing, this paper highlights the importance of considering the real-world context when evaluating fairness.

Title: PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Models

• Authors: Devansh Jain, Priyanshu Kumar, Samuel Gehman, Xuhui Zhou, Thomas Hartvigsen, Maarten Sap
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Models(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Recent advances in large language models (LLMs) have led to their extensive global deployment, and ensuring their safety calls for comprehensive and multilingual toxicity evaluations. However, existing toxicity benchmarks are overwhelmingly focused on English, posing serious risks to deploying LLMs in other languages. We address this by introducing PolygloToxicityPrompts (PTP), the first large-scale multilingual toxicity evaluation benchmark of 425K naturally occurring prompts spanning 17 languages. We overcome the scarcity of naturally occurring toxicity in web-text and ensure coverage across languages with varying resources by automatically scraping over 100M web-text documents. Using PTP, we investigate research questions to study the impact of model size, prompt language, and instruction and preference-tuning methods on toxicity by benchmarking over 60 LLMs. Notably, we find that toxicity increases as language resources decrease or model size increases. Although instruction- and preference-tuning reduce toxicity, the choice of preference-tuning method does not have any significant impact. Our findings shed light on crucial shortcomings of LLM safeguarding and highlight areas for future research.

Title: SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning

• Authors: Yuning Yang, Xiaohong Liu, Tianrun Gao, Xiaodong Xu, Guangyu Wang
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning(https://arxiv.org/abs/)
• Keywords: secure, federate
• Abstract: Fine-tuning large-scale pre-trained models via transfer learning is an emerging important paradigm for a wide range of downstream tasks, with performance heavily reliant on extensive data. Federated learning (FL), as a distributed framework, provides a secure solution to train models on local datasets while safeguarding raw sensitive data. However, FL networks encounter high communication costs due to the massive parameters of large-scale pre-trained models, necessitating parameter-efficient methods. Notably, parameter efficient fine tuning, such as Low-Rank Adaptation (LoRA), has shown remarkable success in fine-tuning pre-trained models. However, prior research indicates that the fixed parameter budget may be prone to the overfitting or slower convergence. To address this challenge, we propose a Simulated Annealing-based Federated Learning with LoRA tuning (SA-FedLoRA) approach by reducing trainable parameters. Specifically, SA-FedLoRA comprises two stages: initiating and annealing. (1) In the initiating stage, we implement a parameter regularization approach during the early rounds of aggregation, aiming to mitigate client drift and accelerate the convergence for the subsequent tuning. (2) In the annealing stage, we allocate higher parameter budget during the early 'heating' phase and then gradually shrink the budget until the 'cooling' phase. This strategy not only facilitates convergence to the global optimum but also reduces communication costs. Experimental results demonstrate that SA-FedLoRA is an efficient FL, achieving superior performance to FedAvg and significantly reducing communication parameters by up to 93.62%.

Title: Time-Equivariant Contrastive Learning for Degenerative Disease Progression in Retinal OCT

• Authors: Taha Emre, Arunava Chakravarty, Dmitrii Lachinov, Antoine Rivail, Ursula Schmidt-Erfurth, Hrvoje Bogunović
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Time-Equivariant Contrastive Learning for Degenerative Disease Progression in Retinal OCT(https://arxiv.org/abs/)
• Keywords: robust, generative
• Abstract: Contrastive pretraining provides robust representations by ensuring their invariance to different image transformations while simultaneously preventing representational collapse. Equivariant contrastive learning, on the other hand, provides representations sensitive to specific image transformations while remaining invariant to others. By introducing equivariance to time-induced transformations, such as disease-related anatomical changes in longitudinal imaging, the model can effectively capture such changes in the representation space. In this work, we pro-pose a Time-equivariant Contrastive Learning (TC) method. First, an encoder embeds two unlabeled scans from different time points of the same patient into the representation space. Next, a temporal equivariance module is trained to predict the representation of a later visit based on the representation from one of the previous visits and the corresponding time interval with a novel regularization loss term while preserving the invariance property to irrelevant image transformations. On a large longitudinal dataset, our model clearly outperforms existing equivariant contrastive methods in predicting progression from intermediate age-related macular degeneration (AMD) to advanced wet-AMD within a specified time-window.

Title: Real-World Federated Learning in Radiology: Hurdles to overcome and Benefits to gain

• Authors: Markus R. Bujotzek, Ünal Akünal, Stefan Denner, Peter Neher, Maximilian Zenk, Eric Frodl, Astha Jaiswal, Moon Kim, Nicolai R. Krekiehn, Manuel Nickel, Richard Ruppel, Marcus Both, Felix Döllinger, Marcel Opitz, Thorsten Persigehl, Jens Kleesiek, Tobias Penzkofer, Klaus Maier-Hein, Rickmer Braren, Andreas Bucher
• Subjects: cs.CV, cs.DC
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Real-World Federated Learning in Radiology: Hurdles to overcome and Benefits to gain(https://arxiv.org/abs/)
• Keywords: robust, federate, segmentation
• Abstract: Objective: Federated Learning (FL) enables collaborative model training while keeping data locally. Currently, most FL studies in radiology are conducted in simulated environments due to numerous hurdles impeding its translation into practice. The few existing real-world FL initiatives rarely communicate specific measures taken to overcome these hurdles, leaving behind a significant knowledge gap. Minding efforts to implement real-world FL, there is a notable lack of comprehensive assessment comparing FL to less complex alternatives. Materials & Methods: We extensively reviewed FL literature, categorizing insights along with our findings according to their nature and phase while establishing a FL initiative, summarized to a comprehensive guide. We developed our own FL infrastructure within the German Radiological Cooperative Network (RACOON) and demonstrated its functionality by training FL models on lung pathology segmentation tasks across six university hospitals. We extensively evaluated FL against less complex alternatives in three distinct evaluation scenarios. Results: The proposed guide outlines essential steps, identified hurdles, and proposed solutions for establishing successful FL initiatives conducting real-world experiments. Our experimental results show that FL outperforms less complex alternatives in all evaluation scenarios, justifying the effort required to translate FL into real-world applications. Discussion & Conclusion: Our proposed guide aims to aid future FL researchers in circumventing pitfalls and accelerating translation of FL into radiological applications. Our results underscore the value of efforts needed to translate FL into real-world applications by demonstrating advantageous performance over alternatives, and emphasize the importance of strategic organization, robust management of distributed data and infrastructure in real-world settings.

Title: Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images

• Authors: Memoona Aziz (Western University, Canada), Umair Rehman (Western University, Canada), Muhammad Umair Danish (Western University, Canada), Katarina Grolinger (Western University, Canada)
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images(https://arxiv.org/abs/)
• Keywords: interpretability, transformer, generative
• Abstract: This paper introduces the Global-Local Image Perceptual Score (GLIPS), an image metric designed to assess the photorealistic image quality of AI-generated images with a high degree of alignment to human visual perception. Traditional metrics such as FID and KID scores do not align closely with human evaluations. The proposed metric incorporates advanced transformer-based attention mechanisms to assess local similarity and Maximum Mean Discrepancy (MMD) to evaluate global distributional similarity. To evaluate the performance of GLIPS, we conducted a human study on photorealistic image quality. Comprehensive tests across various generative models demonstrate that GLIPS consistently outperforms existing metrics like FID, SSIM, and MS-SSIM in terms of correlation with human scores. Additionally, we introduce the Interpolative Binning Scale (IBS), a refined scaling method that enhances the interpretability of metric scores by aligning them more closely with human evaluative standards. The proposed metric and scaling approach not only provides more reliable assessments of AI-generated images but also suggest pathways for future enhancements in image generation technologies.

Title: A Survey On Text-to-3D Contents Generation In The Wild

• Authors: Chenhan Jiang
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] A Survey On Text-to-3D Contents Generation In The Wild(https://arxiv.org/abs/)
• Keywords: generative
• Abstract: 3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. However, the process is labor-intensive and time-consuming, requiring skilled designers to invest considerable effort in creating a single 3D asset. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation. Leveraging the success of large vision language models, these techniques aim to generate 3D content based on textual descriptions. Despite recent advancements in this area, existing solutions still face significant limitations in terms of generation quality and efficiency. In this survey, we conduct an in-depth investigation of the latest text-to-3D creation methods. We provide a comprehensive background on text-to-3D creation, including discussions on datasets employed in training and evaluation metrics used to assess the quality of generated 3D models. Then, we delve into the various 3D representations that serve as the foundation for the 3D generation process. Furthermore, we present a thorough comparison of the rapidly growing literature on generative pipelines, categorizing them into feedforward generators, optimization-based generation, and view reconstruction approaches. By examining the strengths and weaknesses of these methods, we aim to shed light on their respective capabilities and limitations. Lastly, we point out several promising avenues for future research. With this survey, we hope to inspire researchers further to explore the potential of open-vocabulary text-conditioned 3D content creation.

Title: Facilitating Opinion Diversity through Hybrid NLP Approaches

• Authors: Michiel van der Meer
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Facilitating Opinion Diversity through Hybrid NLP Approaches(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Modern democracies face a critical issue of declining citizen participation in decision-making. Online discussion forums are an important avenue for enhancing citizen participation. This thesis proposal 1) identifies the challenges involved in facilitating large-scale online discussions with Natural Language Processing (NLP), 2) suggests solutions to these challenges by incorporating hybrid human-AI technologies, and 3) investigates what these technologies can reveal about individual perspectives in online discussions. We propose a three-layered hierarchy for representing perspectives that can be obtained by a mixture of human intelligence and large language models. We illustrate how these representations can draw insights into the diversity of perspectives and allow us to investigate interactions in online discussions.

Title: Tell Me Why: Explainable Public Health Fact-Checking with Large Language Models

• Authors: Majid Zarharan, Pascal Wullschleger, Babak Behkam Kia, Mohammad Taher Pilehvar, Jennifer Foster
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Tell Me Why: Explainable Public Health Fact-Checking with Large Language Models(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: This paper presents a comprehensive analysis of explainable fact-checking through a series of experiments, focusing on the ability of large language models to verify public health claims and provide explanations or justifications for their veracity assessments. We examine the effectiveness of zero/few-shot prompting and parameter-efficient fine-tuning across various open and closed-source models, examining their performance in both isolated and joint tasks of veracity prediction and explanation generation. Importantly, we employ a dual evaluation approach comprising previously established automatic metrics and a novel set of criteria through human evaluation. Our automatic evaluation indicates that, within the zero-shot scenario, GPT-4 emerges as the standout performer, but in few-shot and parameter-efficient fine-tuning contexts, open-source models demonstrate their capacity to not only bridge the performance gap but, in some instances, surpass GPT-4. Human evaluation reveals yet more nuance as well as indicating potential problems with the gold explanations.

Title: Fourier Boundary Features Network with Wider Catchers for Glass Segmentation

• Authors: Xiaolin Qin, Jiacen Liu, Qianlei Wang, Shaolin Zhang, Fei Zhu, Zhang Yi
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Fourier Boundary Features Network with Wider Catchers for Glass Segmentation(https://arxiv.org/abs/)
• Keywords: robust, extraction, segmentation
• Abstract: Glass largely blurs the boundary between the real world and the reflection. The special transmittance and reflectance quality have confused the semantic tasks related to machine vision. Therefore, how to clear the boundary built by glass, and avoid over-capturing features as false positive information in deep structure, matters for constraining the segmentation of reflection surface and penetrating glass. We proposed the Fourier Boundary Features Network with Wider Catchers (FBWC), which might be the first attempt to utilize sufficiently wide horizontal shallow branches without vertical deepening for guiding the fine granularity segmentation boundary through primary glass semantic information. Specifically, we designed the Wider Coarse-Catchers (WCC) for anchoring large area segmentation and reducing excessive extraction from a structural perspective. We embed fine-grained features by Cross Transpose Attention (CTA), which is introduced to avoid the incomplete area within the boundary caused by reflection noise. For excavating glass features and balancing high-low layers context, a learnable Fourier Convolution Controller (FCC) is proposed to regulate information integration robustly. The proposed method has been validated on three different public glass segmentation datasets. Experimental results reveal that the proposed method yields better segmentation performance compared with the state-of-the-art (SOTA) methods in glass image segmentation.

Title: Flashback: Enhancing Proposer-Builder Design with Future-Block Auctions in Proof-of-Stake Ethereum

• Authors: Yifan Mao, Mengya Zhang, Shaileshh Bojja Venkatakrishnan, Zhiqiang Lin
• Subjects: cs.CR
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Flashback: Enhancing Proposer-Builder Design with Future-Block Auctions in Proof-of-Stake Ethereum(https://arxiv.org/abs/)
• Keywords: security
• Abstract: Maximal extractable value (MEV) in which block proposers unethically gain profits by manipulating the order in which transactions are included within a block, is a key challenge facing blockchains such as Ethereum today. Left unchecked, MEV can lead to a centralization of stake distribution thereby ultimately compromising the security of blockchain consensus. To preserve proposer decentralization (and hence security) of the blockchain, Ethereum has advocated for a proposer-builder separation (PBS) in which the functionality of transaction ordering is separated from proposers and assigned to separate entities called builders. Builders accept transaction bundles from searchers, who compete to find the most profitable bundles. Builders then bid completed blocks to proposers, who accept the most profitable blocks for publication. The auction mechanisms used between searchers, builders and proposers are crucial to the overall health of the blockchain. In this paper, we consider PBS design in Ethereum as a game between searchers, builders and proposers. A key novelty in our design is the inclusion of future block proposers, as all proposers of an epoch are decided ahead of time in proof-of-stake (PoS) Ethereum within the game model. Our analysis shows the existence of alternative auction mechanisms that result in a better (more profitable) equilibrium to players compared to state-of-the-art. Experimental evaluations based on synthetic and real-world data traces corroborate the analysis. Our results highlight that a rethinking of auction mechanism designs is necessary in PoS Ethereum to prevent disruption.

Title: Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts

• Authors: Donya Rooein, Paul Rottger, Anastassia Shaitarova, Dirk Hovy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts(https://arxiv.org/abs/)
• Keywords: large language model
• Abstract: Using large language models (LLMs) for educational applications like dialogue-based teaching is a hot topic. Effective teaching, however, requires teachers to adapt the difficulty of content and explanations to the education level of their students. Even the best LLMs today struggle to do this well. If we want to improve LLMs on this adaptation task, we need to be able to measure adaptation success reliably. However, current Static metrics for text difficulty, like the Flesch-Kincaid Reading Ease score, are known to be crude and brittle. We, therefore, introduce and evaluate a new set of Prompt-based metrics for text difficulty. Based on a user study, we create Prompt-based metrics as inputs for LLMs. They leverage LLM's general language understanding capabilities to capture more abstract and complex features than Static metrics. Regression experiments show that adding our Prompt-based metrics significantly improves text difficulty classification over Static metrics alone. Our results demonstrate the promise of using LLMs to evaluate text adaptation to different education levels.

Title: DemOpts: Fairness corrections in COVID-19 case prediction models

• Authors: Naman Awasthi, Saad Abrar, Daniel Smolyak, Vanessa Frias-Martinez
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] DemOpts: Fairness corrections in COVID-19 case prediction models(https://arxiv.org/abs/)
• Keywords: fair
• Abstract: COVID-19 forecasting models have been used to inform decision making around resource allocation and intervention decisions e.g., hospital beds or stay-at-home orders. State of the art deep learning models often use multimodal data such as mobility or socio-demographic data to enhance COVID-19 case prediction models. Nevertheless, related work has revealed under-reporting bias in COVID-19 cases as well as sampling bias in mobility data for certain minority racial and ethnic groups, which could in turn affect the fairness of the COVID-19 predictions along race labels. In this paper, we show that state of the art deep learning models output mean prediction errors that are significantly different across racial and ethnic groups; and which could, in turn, support unfair policy decisions. We also propose a novel de-biasing method, DemOpts, to increase the fairness of deep learning based forecasting models trained on potentially biased datasets. Our results show that DemOpts can achieve better error parity that other state of the art de-biasing approaches, thus effectively reducing the differences in the mean error distributions across more racial and ethnic groups.

Title: Color Space Learning for Cross-Color Person Re-Identification

• Authors: Jiahao Nie, Shan Lin, Alex C. Kot
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Color Space Learning for Cross-Color Person Re-Identification(https://arxiv.org/abs/)
• Keywords: privacy, robust
• Abstract: The primary color profile of the same identity is assumed to remain consistent in typical Person Re-identification (Person ReID) tasks. However, this assumption may be invalid in real-world situations and images hold variant color profiles, because of cross-modality cameras or identity with different clothing. To address this issue, we propose Color Space Learning (CSL) for those Cross-Color Person ReID problems. Specifically, CSL guides the model to be less color-sensitive with two modules: Image-level Color-Augmentation and Pixel-level Color-Transformation. The first module increases the color diversity of the inputs and guides the model to focus more on the non-color information. The second module projects every pixel of input images onto a new color space. In addition, we introduce a new Person ReID benchmark across RGB and Infrared modalities, NTU-Corridor, which is the first with privacy agreements from all participants. To evaluate the effectiveness and robustness of our proposed CSL, we evaluate it on several Cross-Color Person ReID benchmarks. Our method surpasses the state-of-the-art methods consistently. The code and benchmark are available at: this https URL

Title: QueryNER: Segmentation of E-commerce Queries

• Authors: Chester Palen-Michel, Lizzie Liang, Zhe Wu, Constantine Lignos
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] QueryNER: Segmentation of E-commerce Queries(https://arxiv.org/abs/)
• Keywords: robust, extraction, segmentation
• Abstract: We present QueryNER, a manually-annotated dataset and accompanying model for e-commerce query segmentation. Prior work in sequence labeling for e-commerce has largely addressed aspect-value extraction which focuses on extracting portions of a product title or query for narrowly defined aspects. Our work instead focuses on the goal of dividing a query into meaningful chunks with broadly applicable types. We report baseline tagging results and conduct experiments comparing token and entity dropping for null and low recall query recovery. Challenging test sets are created using automatic transformations and show how simple data augmentation techniques can make the models more robust to noise. We make the QueryNER dataset publicly available.

Title: Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Priming

• Authors: Bushi Xiao, Chao Gao, Demi Zhang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Priming(https://arxiv.org/abs/)
• Keywords: robust, transformer, large language model
• Abstract: This study evaluates the performance of Recurrent Neural Network (RNN) and Transformer in replicating cross-language structural priming: a key indicator of abstract grammatical representations in human language processing. Focusing on Chinese-English priming, which involves two typologically distinct languages, we examine how these models handle the robust phenomenon of structural priming, where exposure to a particular sentence structure increases the likelihood of selecting a similar structure subsequently. Additionally, we utilize large language models (LLM) to measure the cross-lingual structural priming effect. Our findings indicate that Transformer outperform RNN in generating primed sentence structures, challenging the conventional belief that human sentence processing primarily involves recurrent and immediate processing and suggesting a role for cue-based retrieval mechanisms. Overall, this work contributes to our understanding of how computational models may reflect human cognitive processes in multilingual contexts.

Title: Classifying geospatial objects from multiview aerial imagery using semantic meshes

• Authors: David Russell, Ben Weinstein, David Wettergreen, Derek Young
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] Classifying geospatial objects from multiview aerial imagery using semantic meshes(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: Aerial imagery is increasingly used in Earth science and natural resource management as a complement to labor-intensive ground-based surveys. Aerial systems can collect overlapping images that provide multiple views of each location from different perspectives. However, most prediction approaches (e.g. for tree species classification) use a single, synthesized top-down "orthomosaic" image as input that contains little to no information about the vertical aspects of objects and may include processing artifacts. We propose an alternate approach that generates predictions directly on the raw images and accurately maps these predictions into geospatial coordinates using semantic meshes. This method$\unicode{x2013}$released as a user-friendly open-source toolkit$\unicode{x2013}$enables analysts to use the highest quality data for predictions, capture information about the sides of objects, and leverage multiple viewpoints of each location for added robustness. We demonstrate the value of this approach on a new benchmark dataset of four forest sites in the western U.S. that consists of drone images, photogrammetry results, predicted tree locations, and species classification data derived from manual surveys. We show that our proposed multiview method improves classification accuracy from 53% to 75% relative to an orthomosaic baseline on a challenging cross-site tree species classification task.

Title: BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

• Authors: Yunhao Ge, Yihe Tang, Jiashu Xu, Cem Gokmen, Chengshu Li, Wensi Ai, Benjamin Jose Martinez, Arman Aydin, Mona Anvari, Ayush K Chakravarthy, Hong-Xing Yu, Josiah Wong, Sanjana Srivastava, Sharon Lee, Shengxin Zha, Laurent Itti, Yunzhu Li, Roberto Martín-Martín, Miao Liu, Pengchuan Zhang, Ruohan Zhang, Li Fei-Fei, Jiajun Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/
• Pdf URL: https://arxiv.org/pdf/
• Copy Paste: [[]] BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation(https://arxiv.org/abs/)
• Keywords: robust
• Abstract: The systematic evaluation and understanding of computer vision models under varying conditions require large amounts of data with comprehensive and customized labels, which real-world vision datasets rarely satisfy. While current synthetic data generators offer a promising alternative, particularly for embodied AI tasks, they often fall short for computer vision tasks due to low asset and rendering quality, limited diversity, and unrealistic physical properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models, based on the newly developed embodied AI benchmark, BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene level (e.g., lighting, object placement), the object level (e.g., joint configuration, attributes such as "filled" and "folded"), and the camera level (e.g., field of view, focal length). Researchers can arbitrarily vary these parameters during data generation to perform controlled experiments. We showcase three example application scenarios: systematically evaluating the robustness of models across different continuous axes of domain shift, evaluating scene understanding models on the same set of images, and training and evaluating simulation-to-real transfer for a novel vision task: unary and binary state prediction. Project website: this https URL