secure

Title: Turning Noises to Fingerprint-Free "Credentials": Secure and Usable Authentication for Drone Delivery. (arXiv:2302.09197v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09197
• Code URL: null
• Copy Paste: [[2302.09197] Turning Noises to Fingerprint-Free "Credentials": Secure and Usable Authentication for Drone Delivery](http://arxiv.org/abs/2302.09197) #secure
• Summary:

  Drone delivery is an emerging service that gains growing attention. Authentication is critical to ensure a package is picked up by a legitimate drone (rather than a malicious one) and delivered to the correct receiver (rather than an attacker). As delivery drones are expensive and may carry important packages, a drone should stay away from users until the authentication succeeds. Thus, authentication approaches that require physical contact of drones cannot be applied. Bluetooth can indicate proximity without physical contact but is vulnerable to radio relay attacks. Our work leverages drone noises for authentication. While using sounds for authentication is highly usable, how to handle various attacks that manipulate sounds is an unresolved challenge. It is also unclear whether such a system is robust under various environmental sounds. We address these challenges by exploiting unique characteristics of drone noises. We thereby build an authentication system that does not rely on any sound fingerprints, keeps resilient to attacks, and is robust under environmental sounds. An extensive evaluation demonstrates its security and usability.

security

Title: Web Photo Source Identification based on Neural Enhanced Camera Fingerprint. (arXiv:2302.09228v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09228
• Code URL: https://github.com/photonecf/photonecf
• Copy Paste: [[2302.09228] Web Photo Source Identification based on Neural Enhanced Camera Fingerprint](http://arxiv.org/abs/2302.09228) #security
• Summary:

  With the growing popularity of smartphone photography in recent years, web photos play an increasingly important role in all walks of life. Source camera identification of web photos aims to establish a reliable linkage from the captured images to their source cameras, and has a broad range of applications, such as image copyright protection, user authentication, investigated evidence verification, etc. This paper presents an innovative and practical source identification framework that employs neural-network enhanced sensor pattern noise to trace back web photos efficiently while ensuring security. Our proposed framework consists of three main stages: initial device fingerprint registration, fingerprint extraction and cryptographic connection establishment while taking photos, and connection verification between photos and source devices. By incorporating metric learning and frequency consistency into the deep network design, our proposed fingerprint extraction algorithm achieves state-of-the-art performance on modern smartphone photos for reliable source identification. Meanwhile, we also propose several optimization sub-modules to prevent fingerprint leakage and improve accuracy and efficiency. Finally for practical system design, two cryptographic schemes are introduced to reliably identify the correlation between registered fingerprint and verified photo fingerprint, i.e. fuzzy extractor and zero-knowledge proof (ZKP). The codes for fingerprint extraction network and benchmark dataset with modern smartphone cameras photos are all publicly available at https://github.com/PhotoNecf/PhotoNecf.

Title: Vulnerability analysis of captcha using Deep learning. (arXiv:2302.09389v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09389
• Code URL: null
• Copy Paste: [[2302.09389] Vulnerability analysis of captcha using Deep learning](http://arxiv.org/abs/2302.09389) #security
• Summary:

  Several websites improve their security and avoid dangerous Internet attacks by implementing CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart), a type of verification to identify whether the end-user is human or a robot. The most prevalent type of CAPTCHA is text-based, designed to be easily recognized by humans while being unsolvable towards machines or robots. However, as deep learning technology progresses, development of convolutional neural network (CNN) models that predict text-based CAPTCHAs becomes easier. The purpose of this research is to investigate the flaws and vulnerabilities in the CAPTCHA generating systems in order to design more resilient CAPTCHAs. To achieve this, we created CapNet, a Convolutional Neural Network. The proposed platform can evaluate both numerical and alphanumerical CAPTCHAs

Title: Reproducing Random Forest Efficacy in Detecting Port Scanning. (arXiv:2302.09317v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09317
• Code URL: https://github.com/jasonmpittman/rf-port-scans
• Copy Paste: [[2302.09317] Reproducing Random Forest Efficacy in Detecting Port Scanning](http://arxiv.org/abs/2302.09317) #security
• Summary:

  Port scanning is the process of attempting to connect to various network ports on a computing endpoint to determine which ports are open and which services are running on them. It is a common method used by hackers to identify vulnerabilities in a network or system. By determining which ports are open, an attacker can identify which services and applications are running on a device and potentially exploit any known vulnerabilities in those services. Consequently, it is important to detect port scanning because it is often the first step in a cyber attack. By identifying port scanning attempts, cybersecurity professionals can take proactive measures to protect the systems and networks before an attacker has a chance to exploit any vulnerabilities. Against this background, researchers have worked for over a decade to develop robust methods to detect port scanning. One such method revealed by a recent systematic review is the random forest supervised machine learning algorithm. The review revealed six existing studies using random forest since 2021. Unfortunately, those studies each exhibit different results, do not all use the same training and testing dataset, and only two include source code. Accordingly, the goal of this work was to reproduce the six random forest studies while addressing the apparent shortcomings. The outcomes are significant for researchers looking to explore random forest to detect port scanning and for practitioners interested in reliable technology to detect the early stages of cyber attack.

Title: Security of IT/OT Convergence: Design and Implementation Challenges. (arXiv:2302.09426v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09426
• Code URL: null
• Copy Paste: [[2302.09426] Security of IT/OT Convergence: Design and Implementation Challenges](http://arxiv.org/abs/2302.09426) #security
• Summary:

  IoT is undoubtedly considered the future of the Internet. Many sectors are moving towards the use of these devices to aid better monitoring, controlling of the surrounding environment, and manufacturing processes. The Industrial Internet of things is a sub-domain of IoT and serves as enablers of the industry. IIoT is providing valuable services to Industrial Control Systems such as logistics, manufacturing, healthcare, industrial surveillance, and others. Although IIoT service-offering to ICS is tempting, it comes with greater risk. ICS systems are protected by isolation and creating an air-gap to separate their network from the outside world. While IIoT by definition is a device that has connection ability. This creates multiple points of entry to a closed system. In this study, we examine the first automated risk assessment system designed specifically to deal with the automated risk assessment and defining potential threats associated with IT/OT convergence based on OCTAVE Allegro- ISO/IEC 27030 Frameworks.

Title: Comprehensive Evaluation of RSB and Spectre Vulnerability on Modern Processors. (arXiv:2302.09544v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09544
• Code URL: null
• Copy Paste: [[2302.09544] Comprehensive Evaluation of RSB and Spectre Vulnerability on Modern Processors](http://arxiv.org/abs/2302.09544) #security
• Summary:

  Performance-enhancing mechanisms such as branch prediction, out-of-order execution, and return stack buffer (RSB) have been widely employed in today's modern processing units. Although successful in increasing the CPU performance, exploiting the design flaws and security bugs in these components have set the background for various types of microarchitectural attacks such as Spectre and Meltdown. While many attacks such as Meltdown and Spectre have been numerously implemented and analyzed on Intel processors, few researches have been carried out to evaluate their impact on ARM processors. Moreover, SpectreRSB vulnerability, the newer variant of spectre attack based on RSB, has been neglected in recent studies. In this work, we first evaluate the SpectreRSB vulnerability by implementing this attack on ARM processors, which, to the best of our knowledge, has not been implemented and analyzed on ARM processors. We further present a security evaluation of ARM processors by implementing different variants of Spectre-family attacks. By analyzing the results obtained from various experiments, we evaluate ARM processors security regarding their diverse microarchitectural designs. We also introduce a high throughput and noise-free covert channel, based on the RSB structure. Based on our experiments, the throughput of the covert channel is 94.19KB/s with negligible error.

privacy

Title: Digital Privacy Under Attack: Challenges and Enablers. (arXiv:2302.09258v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09258
• Code URL: null
• Copy Paste: [[2302.09258] Digital Privacy Under Attack: Challenges and Enablers](http://arxiv.org/abs/2302.09258) #privacy
• Summary:

  Users have renewed interest in protecting their private data in the digital space. When they don't believe that their privacy is sufficiently covered by one platform, they will readily switch to another. Such an increasing level of privacy awareness has made privacy preservation an essential research topic. Nevertheless, new privacy attacks are emerging day by day. Therefore, a holistic survey to compare the discovered techniques on attacks over privacy preservation and their mitigation schemes is essential in the literature. We develop a study to fill this gap by assessing the resilience of privacy-preserving methods to various attacks and conducting a comprehensive review of countermeasures from a broader perspective. First, we introduce the fundamental concepts and critical components of privacy attacks. Second, we comprehensively cover major privacy attacks targeted at anonymous data, statistical aggregate data, and privacy-preserving models. We also summarize popular countermeasures to mitigate these attacks. Finally, some promising future research directions and related issues in the privacy community are envisaged. We believe this survey will successfully shed some light on privacy research and encourage researchers to entirely understand the resilience of different existing privacy-preserving approaches.

Title: Dynamic Private Task Assignment under Differential Privacy. (arXiv:2302.09511v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09511
• Code URL: null
• Copy Paste: [[2302.09511] Dynamic Private Task Assignment under Differential Privacy](http://arxiv.org/abs/2302.09511) #privacy
• Summary:

  Data collection is indispensable for spatial crowdsourcing services, such as resource allocation, policymaking, and scientific explorations. However, privacy issues make it challenging for users to share their information unless receiving sufficient compensation. Differential Privacy (DP) is a promising mechanism to release helpful information while protecting individuals' privacy. However, most DP mechanisms only consider a fixed compensation for each user's privacy loss. In this paper, we design a task assignment scheme that allows workers to dynamically improve their utility with dynamic distance privacy leakage. Specifically, we propose two solutions to improve the total utility of task assignment results, namely Private Utility Conflict-Elimination (PUCE) approach and Private Game Theory (PGT) approach, respectively. We prove that PUCE achieves higher utility than the state-of-the-art works. We demonstrate the efficiency and effectiveness of our PUCE and PGT approaches on both real and synthetic data sets compared with the recent distance-based approach, Private Distance Conflict-Elimination (PDCE). PUCE is always better than PDCE slightly. PGT is 50% to 63% faster than PDCE and can improve 16% utility on average when worker range is large enough.

Title: Learning with Impartiality to Walk on the Pareto Frontier of Fairness, Privacy, and Utility. (arXiv:2302.09183v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09183
• Code URL: null
• Copy Paste: [[2302.09183] Learning with Impartiality to Walk on the Pareto Frontier of Fairness, Privacy, and Utility](http://arxiv.org/abs/2302.09183) #privacy
• Summary:

  Deploying machine learning (ML) models often requires both fairness and privacy guarantees. Both of these objectives present unique trade-offs with the utility (e.g., accuracy) of the model. However, the mutual interactions between fairness, privacy, and utility are less well-understood. As a result, often only one objective is optimized, while the others are tuned as hyper-parameters. Because they implicitly prioritize certain objectives, such designs bias the model in pernicious, undetectable ways. To address this, we adopt impartiality as a principle: design of ML pipelines should not favor one objective over another. We propose impartially-specified models, which provide us with accurate Pareto frontiers that show the inherent trade-offs between the objectives. Extending two canonical ML frameworks for privacy-preserving learning, we provide two methods (FairDP-SGD and FairPATE) to train impartially-specified models and recover the Pareto frontier. Through theoretical privacy analysis and a comprehensive empirical study, we provide an answer to the question of where fairness mitigation should be integrated within a privacy-aware ML pipeline.

Title: Function Composition in Trustworthy Machine Learning: Implementation Choices, Insights, and Questions. (arXiv:2302.09190v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09190
• Code URL: null
• Copy Paste: [[2302.09190] Function Composition in Trustworthy Machine Learning: Implementation Choices, Insights, and Questions](http://arxiv.org/abs/2302.09190) #privacy
• Summary:

  Ensuring trustworthiness in machine learning (ML) models is a multi-dimensional task. In addition to the traditional notion of predictive performance, other notions such as privacy, fairness, robustness to distribution shift, adversarial robustness, interpretability, explainability, and uncertainty quantification are important considerations to evaluate and improve (if deficient). However, these sub-disciplines or 'pillars' of trustworthiness have largely developed independently, which has limited us from understanding their interactions in real-world ML pipelines. In this paper, focusing specifically on compositions of functions arising from the different pillars, we aim to reduce this gap, develop new insights for trustworthy ML, and answer questions such as the following. Does the composition of multiple fairness interventions result in a fairer model compared to a single intervention? How do bias mitigation algorithms for fairness affect local post-hoc explanations? Does a defense algorithm for untargeted adversarial attacks continue to be effective when composed with a privacy transformation? Toward this end, we report initial empirical results and new insights from 9 different compositions of functions (or pipelines) on 7 real-world datasets along two trustworthy dimensions - fairness and explainability. We also report progress, and implementation choices, on an extensible composer tool to encourage the combination of functionalities from multiple pillars. To-date, the tool supports bias mitigation algorithms for fairness and post-hoc explainability methods. We hope this line of work encourages the thoughtful consideration of multiple pillars when attempting to formulate and resolve a trustworthiness problem.

Title: On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge. (arXiv:2302.09310v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09310
• Code URL: null
• Copy Paste: [[2302.09310] On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge](http://arxiv.org/abs/2302.09310) #privacy
• Summary:

  Human activity recognition (HAR) has been a classic research problem. In particular, with recent machine learning (ML) techniques, the recognition task has been largely investigated by companies and integrated into their products for customers. However, most of them apply a predefined activity set and conduct the learning process on the cloud, hindering specific personalizations from end users (i.e., edge devices). Even though recent progress in Incremental Learning allows learning new-class data on the fly, the learning process is generally conducted on the cloud, requiring constant data exchange between cloud and edge devices, thus leading to data privacy issues. In this paper, we propose PILOTE, which pushes the incremental learning process to the extreme edge, while providing reliable data privacy and practical utility, e.g., low processing latency, personalization, etc. In particular, we consider the practical challenge of extremely limited data during the incremental learning process on edge, where catastrophic forgetting is required to be handled in a practical way. We validate PILOTE with extensive experiments on human activity data collected from mobile sensors. The results show PILOTE can work on edge devices with extremely limited resources while providing reliable performance.

Title: Why Is Public Pretraining Necessary for Private Model Training?. (arXiv:2302.09483v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09483
• Code URL: null
• Copy Paste: [[2302.09483] Why Is Public Pretraining Necessary for Private Model Training?](http://arxiv.org/abs/2302.09483) #privacy
• Summary:

  In the privacy-utility tradeoff of a model trained on benchmark language and vision tasks, remarkable improvements have been widely reported with the use of pretraining on publicly available data. This is in part due to the benefits of transfer learning, which is the standard motivation for pretraining in non-private settings. However, the stark contrast in the improvement achieved through pretraining under privacy compared to non-private settings suggests that there may be a deeper, distinct cause driving these gains. To explain this phenomenon, we hypothesize that the non-convex loss landscape of a model training necessitates an optimization algorithm to go through two phases. In the first, the algorithm needs to select a good "basin" in the loss landscape. In the second, the algorithm solves an easy optimization within that basin. The former is a harder problem to solve with private data, while the latter is harder to solve with public data due to a distribution shift or data scarcity. Guided by this intuition, we provide theoretical constructions that provably demonstrate the separation between private training with and without public pretraining. Further, systematic experiments on CIFAR10 and LibriSpeech provide supporting evidence for our hypothesis.

protect

Title: Designing Equitable Algorithms. (arXiv:2302.09157v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09157
• Code URL: https://github.com/madisoncoots/equitable-algorithms
• Copy Paste: [[2302.09157] Designing Equitable Algorithms](http://arxiv.org/abs/2302.09157) #protect
• Summary:

  Predictive algorithms are now used to help distribute a large share of our society's resources and sanctions, such as healthcare, loans, criminal detentions, and tax audits. Under the right circumstances, these algorithms can improve the efficiency and equity of decision-making. At the same time, there is a danger that the algorithms themselves could entrench and exacerbate disparities, particularly along racial, ethnic, and gender lines. To help ensure their fairness, many researchers suggest that algorithms be subject to at least one of three constraints: (1) no use of legally protected features, such as race, ethnicity, and gender; (2) equal rates of "positive" decisions across groups; and (3) equal error rates across groups. Here we show that these constraints, while intuitively appealing, often worsen outcomes for individuals in marginalized groups, and can even leave all groups worse off. The inherent trade-off we identify between formal fairness constraints and welfare improvements -- particularly for the marginalized -- highlights the need for a more robust discussion on what it means for an algorithm to be "fair". We illustrate these ideas with examples from healthcare and the criminal-legal system, and make several proposals to help practitioners design more equitable algorithms.

defense

Title: MorphGANFormer: Transformer-based Face Morphing and De-Morphing. (arXiv:2302.09404v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09404
• Code URL: null
• Copy Paste: [[2302.09404] MorphGANFormer: Transformer-based Face Morphing and De-Morphing](http://arxiv.org/abs/2302.09404) #defense
• Summary:

  Semantic face image manipulation has received increasing attention in recent years. StyleGAN-based approaches to face morphing are among the leading techniques; however, they often suffer from noticeable blurring and artifacts as a result of the uniform attention in the latent feature space. In this paper, we propose to develop a transformer-based alternative to face morphing and demonstrate its superiority to StyleGAN-based methods. Our contributions are threefold. First, inspired by GANformer, we introduce a bipartite structure to exploit long-range interactions in face images for iterative propagation of information from latent variables to salient facial features. Special loss functions are designed to support the optimization of face morphing. Second, we extend the study of transformer-based face morphing to demorphing by presenting an effective defense strategy with access to a reference image using the same generator of MorphGANFormer. Such demorphing is conceptually similar to unmixing of hyperspectral images but operates in the latent (instead of pixel) space. Third, for the first time, we address a fundamental issue of vulnerability-detectability trade-off for face morphing studies. It is argued that neither doppelganger norrandom pair selection is optimal, and a Lagrangian multiplier-based approach should be used to achieve an improved trade-off between recognition vulnerability and attack detectability.

attack

Title: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning. (arXiv:2302.09309v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09309
• Code URL: null
• Copy Paste: [[2302.09309] Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](http://arxiv.org/abs/2302.09309) #attack
• Summary:

  Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. It aims at transferring prior knowledge learned on the source dataset to novel target datasets. The CD-FSL task is especially challenged by the huge domain gap between different datasets. Critically, such a domain gap actually comes from the changes of visual styles, and wave-SAN empirically shows that spanning the style distribution of the source data helps alleviate this issue. However, wave-SAN simply swaps styles of two images. Such a vanilla operation makes the generated styles real'' andeasy'', which still fall into the original set of the source styles. Thus, inspired by vanilla adversarial learning, a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method is proposed for CD-FSL. Particularly, our style attack method synthesizes both virtual'' andhard'' adversarial styles for model training. This is achieved by perturbing the original style with the signed style gradients. By continually attacking styles and forcing the model to recognize these challenging adversarial styles, our model is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets. Besides the typical CNN-based backbone, we also employ our StyleAdv method on large-scale pretrained vision transformer. Extensive experiments conducted on eight various target datasets show the effectiveness of our method. Whether built upon ResNet or ViT, we achieve the new state of the art for CD-FSL. Codes and models will be released.

Title: Deep Neural Networks based Meta-Learning for Network Intrusion Detection. (arXiv:2302.09394v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09394
• Code URL: null
• Copy Paste: [[2302.09394] Deep Neural Networks based Meta-Learning for Network Intrusion Detection](http://arxiv.org/abs/2302.09394) #attack
• Summary:

  Designing an intrusion detection system is difficult as network traffic encompasses various attack types, including new and evolving ones with minor changes. The data used to construct a predictive model has a skewed class distribution and limited representation of attack types, which differ from real network traffic. These limitations result in dataset shift, negatively impacting the machine learning models' predictive abilities and reducing the detection rate against novel attacks. To address the challenge of dataset shift, we introduce the INformation FUsion and Stacking Ensemble (INFUSE) for network intrusion detection. This approach further improves its predictive power by employing a deep neural network-based Meta-Learner on top of INFUSE. First, a hybrid feature space is created by integrating decision and feature spaces. Five different classifiers are utilized to generate a pool of decision spaces. The feature space is then enriched through a deep sparse autoencoder that learns the semantic relationships between attacks. Finally, the deep Meta-Learner acts as an ensemble combiner to analyze the hybrid feature space and make a final decision. Our evaluation on stringent benchmark datasets and comparison to existing techniques showed the effectiveness of INFUSE with an F-Score of 0.91, Accuracy of 91.6%, and Recall of 0.94 on the Test+ dataset, and an F-Score of 0.91, Accuracy of 85.6%, and Recall of 0.87 on the stringent Test-21 dataset. These promising results indicate the proposed technique has strong generalization capability and the potential to detect network attacks.

Title: X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection. (arXiv:2302.09491v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09491
• Code URL: null
• Copy Paste: [[2302.09491] X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](http://arxiv.org/abs/2302.09491) #attack
• Summary:

  Adversarial attacks are valuable for evaluating the robustness of deep learning models. Existing attacks are primarily conducted on the visible light spectrum (e.g., pixel-wise texture perturbation). However, attacks targeting texture-free X-ray images remain underexplored, despite the widespread application of X-ray imaging in safety-critical scenarios such as the X-ray detection of prohibited items. In this paper, we take the first step toward the study of adversarial attacks targeted at X-ray prohibited item detection, and reveal the serious threats posed by such attacks in this safety-critical scenario. Specifically, we posit that successful physical adversarial attacks in this scenario should be specially designed to circumvent the challenges posed by color/texture fading and complex overlapping. To this end, we propose X-adv to generate physically printable metals that act as an adversarial agent capable of deceiving X-ray detectors when placed in luggage. To resolve the issues associated with color/texture fading, we develop a differentiable converter that facilitates the generation of 3D-printable objects with adversarial shapes, using the gradients of a surrogate model rather than directly generating adversarial textures. To place the printed 3D adversarial objects in luggage with complex overlapped instances, we design a policy-based reinforcement learning strategy to find locations eliciting strong attack performance in worst-case scenarios whereby the prohibited items are heavily occluded by other items. To verify the effectiveness of the proposed X-Adv, we conduct extensive experiments in both the digital and the physical world (employing a commercial X-ray security inspection system for the latter case). Furthermore, we present the physical-world X-ray adversarial attack dataset XAD.

Title: RetVec: Resilient and Efficient Text Vectorizer. (arXiv:2302.09207v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09207
• Code URL: null
• Copy Paste: [[2302.09207] RetVec: Resilient and Efficient Text Vectorizer](http://arxiv.org/abs/2302.09207) #attack
• Summary:

  This paper describes RetVec, a resilient multilingual embedding scheme designed for neural-based text processing, including small-text classification and large-language models. RetVec combines a novel character encoding with an optional small model to embed words into a 256-dimensional vector space. These embeddings enable training competitive multilingual text models resilient to typos and adversarial attacks. In this paper, we evaluate and compare RetVec to state-of-the-art tokenizers and word embeddings on common model architectures. These comparisons demonstrate that RetVec leads to competitive models that are significantly more resilient to text perturbations across a variety of common tasks. RetVec is available under Apache 2 license at \url{https://github.com/[anonymized]}.

Title: Differential Aggregation against General Colluding Attackers. (arXiv:2302.09315v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09315
• Code URL: null
• Copy Paste: [[2302.09315] Differential Aggregation against General Colluding Attackers](http://arxiv.org/abs/2302.09315) #attack
• Summary:

  Local Differential Privacy (LDP) is now widely adopted in large-scale systems to collect and analyze sensitive data while preserving users' privacy. However, almost all LDP protocols rely on a semi-trust model where users are curious-but-honest, which rarely holds in real-world scenarios. Recent works show poor estimation accuracy of many LDP protocols under malicious threat models. Although a few works have proposed some countermeasures to address these attacks, they all require prior knowledge of either the attacking pattern or the poison value distribution, which is impractical as they can be easily evaded by the attackers.

In this paper, we adopt a general opportunistic-and-colluding threat model and propose a multi-group Differential Aggregation Protocol (DAP) to improve the accuracy of mean estimation under LDP. Different from all existing works that detect poison values on individual basis, DAP mitigates the overall impact of poison values on the estimated mean. It relies on a new probing mechanism EMF (i.e., Expectation-Maximization Filter) to estimate features of the attackers. In addition to EMF, DAP also consists of two EMF post-processing procedures (EMF* and CEMF*), and a group-wise mean aggregation scheme to optimize the final estimated mean to achieve the smallest variance. Extensive experimental results on both synthetic and real-world datasets demonstrate the superior performance of DAP over state-of-the-art solutions.

Title: Backdoor Attacks to Pre-trained Unified Foundation Models. (arXiv:2302.09360v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09360
• Code URL: null
• Copy Paste: [[2302.09360] Backdoor Attacks to Pre-trained Unified Foundation Models](http://arxiv.org/abs/2302.09360) #attack
• Summary:

  The rise of pre-trained unified foundation models breaks down the barriers between different modalities and tasks, providing comprehensive support to users with unified architectures. However, the backdoor attack on pre-trained models poses a serious threat to their security. Previous research on backdoor attacks has been limited to uni-modal tasks or single tasks across modalities, making it inapplicable to unified foundation models. In this paper, we make proof-of-concept level research on the backdoor attack for pre-trained unified foundation models. Through preliminary experiments on NLP and CV classification tasks, we reveal the vulnerability of these models and suggest future research directions for enhancing the attack approach.

Title: RobustNLP: A Technique to Defend NLP Models Against Backdoor Attacks. (arXiv:2302.09420v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2302.09420
• Code URL: null
• Copy Paste: [[2302.09420] RobustNLP: A Technique to Defend NLP Models Against Backdoor Attacks](http://arxiv.org/abs/2302.09420) #attack
• Summary:

  As machine learning (ML) systems are being increasingly employed in the real world to handle sensitive tasks and make decisions in various fields, the security and privacy of those models have also become increasingly critical. In particular, Deep Neural Networks (DNN) have been shown to be vulnerable to backdoor attacks whereby adversaries have access to the training data and the opportunity to manipulate such data by inserting carefully developed samples into the training dataset. Although the NLP community has produced several studies on generating backdoor attacks proving the vulnerable state of language modes, to the best of our knowledge, there does not exist any work to combat such attacks. To bridge this gap, we present RobustEncoder: a novel clustering-based technique for detecting and removing backdoor attacks in the text domain. Extensive empirical results demonstrate the effectiveness of our technique in detecting and removing backdoor triggers. Our code is available at https://github.com/marwanomar1/Backdoor-Learning-for-NLP

Title: Adversarial Machine Learning: A Systematic Survey of Backdoor Attack, Weight Attack and Adversarial Example. (arXiv:2302.09457v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09457
• Code URL: null
• Copy Paste: [[2302.09457] Adversarial Machine Learning: A Systematic Survey of Backdoor Attack, Weight Attack and Adversarial Example](http://arxiv.org/abs/2302.09457) #attack
• Summary:

  Adversarial machine learning (AML) studies the adversarial phenomenon of machine learning, which may make inconsistent or unexpected predictions with humans. Some paradigms have been recently developed to explore this adversarial phenomenon occurring at different stages of a machine learning system, such as training-time adversarial attack (i.e., backdoor attack), deployment-time adversarial attack (i.e., weight attack), and inference-time adversarial attack (i.e., adversarial example). However, although these paradigms share a common goal, their developments are almost independent, and there is still no big picture of AML. In this work, we aim to provide a unified perspective to the AML community to systematically review the overall progress of this field. We firstly provide a general definition about AML, and then propose a unified mathematical framework to covering existing attack paradigms. According to the proposed unified framework, we can not only clearly figure out the connections and differences among these paradigms, but also systematically categorize and review existing works in each paradigm.

robust

Title: A Review on Generative Adversarial Networks for Data Augmentation in Person Re-Identification Systems. (arXiv:2302.09119v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09119
• Code URL: null
• Copy Paste: [[2302.09119] A Review on Generative Adversarial Networks for Data Augmentation in Person Re-Identification Systems](http://arxiv.org/abs/2302.09119) #robust
• Summary:

  Interest in automatic people re-identification systems has significantly grown in recent years, mainly for developing surveillance and smart shops software. Due to the variability in person posture, different lighting conditions, and occluded scenarios, together with the poor quality of the images obtained by different cameras, it is currently an unsolved problem. In machine learning-based computer vision applications with reduced data sets, one possibility to improve the performance of re-identification system is through the augmentation of the set of images or videos available for training the neural models. Currently, one of the most robust ways to generate synthetic information for data augmentation, whether it is video, images or text, are the generative adversarial networks. This article reviews the most relevant recent approaches to improve the performance of person re-identification models through data augmentation, using generative adversarial networks. We focus on three categories of data augmentation approaches: style transfer, pose transfer, and random generation.

Title: MedViT: A Robust Vision Transformer for Generalized Medical Image Classification. (arXiv:2302.09462v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09462
• Code URL: null
• Copy Paste: [[2302.09462] MedViT: A Robust Vision Transformer for Generalized Medical Image Classification](http://arxiv.org/abs/2302.09462) #robust
• Summary:

  Convolutional Neural Networks (CNNs) have advanced existing medical systems for automatic disease diagnosis. However, there are still concerns about the reliability of deep medical diagnosis systems against the potential threats of adversarial attacks since inaccurate diagnosis could lead to disastrous consequences in the safety realm. In this study, we propose a highly robust yet efficient CNN-Transformer hybrid model which is equipped with the locality of CNNs as well as the global connectivity of vision Transformers. To mitigate the high quadratic complexity of the self-attention mechanism while jointly attending to information in various representation subspaces, we construct our attention mechanism by means of an efficient convolution operation. Moreover, to alleviate the fragility of our Transformer model against adversarial attacks, we attempt to learn smoother decision boundaries. To this end, we augment the shape information of an image in the high-level feature space by permuting the feature mean and variance within mini-batches. With less computational complexity, our proposed hybrid model demonstrates its high robustness and generalization ability compared to the state-of-the-art studies on a large-scale collection of standardized MedMNIST-2D datasets.

Title: Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints. (arXiv:2302.09185v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09185
• Code URL: https://github.com/salt-nlp/bound-cap-llm
• Copy Paste: [[2302.09185] Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints](http://arxiv.org/abs/2302.09185) #robust
• Summary:

  The limits of open-ended generative models are unclear, yet increasingly important. What causes them to succeed and what causes them to fail? In this paper, we take a prompt-centric approach to analyzing and bounding the abilities of open-ended generative models. We present a generic methodology of analysis with two challenging prompt constraint types: structural and stylistic. These constraint types are categorized into a set of well-defined constraints that are analyzable by a single prompt. We then systematically create a diverse set of simple, natural, and useful prompts to robustly analyze each individual constraint. Using the GPT-3 text-davinci-002 model as a case study, we generate outputs from our collection of prompts and analyze the model's generative failures. We also show the generalizability of our proposed method on other large models like BLOOM and OPT. Our results and our in-context mitigation strategies reveal open challenges for future research. We have publicly released our code at https://github.com/SALT-NLP/Bound-Cap-LLM.

Title: How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. (arXiv:2302.09210v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09210
• Code URL: https://github.com/microsoft/gpt-mt
• Copy Paste: [[2302.09210] How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](http://arxiv.org/abs/2302.09210) #robust
• Summary:

  Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a comprehensive evaluation of GPT models for machine translation, covering various aspects such as quality of different GPT models in comparison with state-of-the-art research and commercial systems, effect of prompting strategies, robustness towards domain shifts and document-level translation. We experiment with eighteen different translation directions involving high and low resource languages, as well as non English-centric translations, and evaluate the performance of three GPT models: ChatGPT, GPT3.5 (text-davinci-003), and text-davinci-002. Our results show that GPT models achieve very competitive translation quality for high resource languages, while having limited capabilities for low resource languages. We also show that hybrid approaches, which combine GPT models with other translation systems, can further enhance the translation quality. We perform comprehensive analysis and human evaluation to further understand the characteristics of GPT translations. We hope that our paper provides valuable insights for researchers and practitioners in the field and helps to better understand the potential and limitations of GPT models for translation.

Title: Delving into the Adversarial Robustness of Federated Learning. (arXiv:2302.09479v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09479
• Code URL: null
• Copy Paste: [[2302.09479] Delving into the Adversarial Robustness of Federated Learning](http://arxiv.org/abs/2302.09479) #robust
• Summary:

  In Federated Learning (FL), models are as fragile as centrally trained models against adversarial examples. However, the adversarial robustness of federated learning remains largely unexplored. This paper casts light on the challenge of adversarial robustness of federated learning. To facilitate a better understanding of the adversarial vulnerability of the existing FL methods, we conduct comprehensive robustness evaluations on various attacks and adversarial training methods. Moreover, we reveal the negative impacts induced by directly adopting adversarial training in FL, which seriously hurts the test accuracy, especially in non-IID settings. In this work, we propose a novel algorithm called Decision Boundary based Federated Adversarial Training (DBFAT), which consists of two components (local re-weighting and global regularization) to improve both accuracy and robustness of FL systems. Extensive experiments on multiple datasets demonstrate that DBFAT consistently outperforms other baselines under both IID and non-IID settings.

Title: Smoothly Giving up: Robustness for Simple Models. (arXiv:2302.09114v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09114
• Code URL: null
• Copy Paste: [[2302.09114] Smoothly Giving up: Robustness for Simple Models](http://arxiv.org/abs/2302.09114) #robust
• Summary:

  There is a growing need for models that are interpretable and have reduced energy and computational cost (e.g., in health care analytics and federated learning). Examples of algorithms to train such models include logistic regression and boosting. However, one challenge facing these algorithms is that they provably suffer from label noise; this has been attributed to the joint interaction between oft-used convex loss functions and simpler hypothesis classes, resulting in too much emphasis being placed on outliers. In this work, we use the margin-based $\alpha$-loss, which continuously tunes between canonical convex and quasi-convex losses, to robustly train simple models. We show that the $\alpha$ hyperparameter smoothly introduces non-convexity and offers the benefit of "giving up" on noisy training examples. We also provide results on the Long-Servedio dataset for boosting and a COVID-19 survey dataset for logistic regression, highlighting the efficacy of our approach across multiple relevant domains.

Title: Stochastic Approximation Approaches to Group Distributionally Robust Optimization. (arXiv:2302.09267v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09267
• Code URL: null
• Copy Paste: [[2302.09267] Stochastic Approximation Approaches to Group Distributionally Robust Optimization](http://arxiv.org/abs/2302.09267) #robust
• Summary:

  This paper investigates group distributionally robust optimization (GDRO), with the purpose to learn a model that performs well over $m$ different distributions. First, we formulate GDRO as a stochastic convex-concave saddle-point problem, and demonstrate that stochastic mirror descent (SMD), using $m$ samples in each iteration, achieves an $O(m (\log m)/\epsilon^2)$ sample complexity for finding an $\epsilon$-optimal solution, which matches the $\Omega(m/\epsilon^2)$ lower bound up to a logarithmic factor. Then, we make use of techniques from online learning to reduce the number of samples required in each round from $m$ to $1$, keeping the same sample complexity. Specifically, we cast GDRO as a two-players game where one player simply performs SMD and the other executes an online algorithm for non-oblivious multi-armed bandits. Next, we consider a more practical scenario where the number of samples that can be drawn from each distribution is different, and propose a novel formulation of weighted DRO, which allows us to derive distribution-dependent convergence rates. Denote by $n_i$ the sample budget for the $i$-th distribution, and assume $n_1 \geq n_2 \geq \cdots \geq n_m$. In the first approach, we incorporate non-uniform sampling into SMD such that the sample budget is satisfied in expectation, and prove the excess risk of the $i$-th distribution decreases at an $O(\sqrt{n_1 \log m}/n_i)$ rate. In the second approach, we use mini-batches to meet the budget exactly and also reduce the variance in stochastic gradients, and then leverage stochastic mirror-prox algorithm, which can exploit small variances, to optimize a carefully designed weighted DRO problem. Under appropriate conditions, it attains an $O((\log m)/\sqrt{n_i})$ convergence rate, which almost matches the optimal $O(\sqrt{1/n_i})$ rate of only learning from the $i$-th distribution with $n_i$ samples.

Title: Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement. (arXiv:2302.09318v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09318
• Code URL: null
• Copy Paste: [[2302.09318] Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement](http://arxiv.org/abs/2302.09318) #robust
• Summary:

  Many real-world applications require an agent to make robust and deliberate decisions with multimodal information (e.g., robots with multi-sensory inputs). However, it is very challenging to train the agent via reinforcement learning (RL) due to the heterogeneity and dynamic importance of different modalities. Specifically, we observe that these issues make conventional RL methods difficult to learn a useful state representation in the end-to-end training with multimodal information. To address this, we propose a novel multimodal RL approach that can do multimodal alignment and importance enhancement according to their similarity and importance in terms of RL tasks respectively. By doing so, we are able to learn an effective state representation and consequentially improve the RL training process. We test our approach on several multimodal RL domains, showing that it outperforms state-of-the-art methods in terms of learning speed and policy quality.

biometric

steal

extraction

Title: Invertible Neural Skinning. (arXiv:2302.09227v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09227
• Code URL: null
• Copy Paste: [[2302.09227] Invertible Neural Skinning](http://arxiv.org/abs/2302.09227) #extraction
• Summary:

  Building animatable and editable models of clothed humans from raw 3D scans and poses is a challenging problem. Existing reposing methods suffer from the limited expressiveness of Linear Blend Skinning (LBS), require costly mesh extraction to generate each new pose, and typically do not preserve surface correspondences across different poses. In this work, we introduce Invertible Neural Skinning (INS) to address these shortcomings. To maintain correspondences, we propose a Pose-conditioned Invertible Network (PIN) architecture, which extends the LBS process by learning additional pose-varying deformations. Next, we combine PIN with a differentiable LBS module to build an expressive and end-to-end Invertible Neural Skinning (INS) pipeline. We demonstrate the strong performance of our method by outperforming the state-of-the-art reposing techniques on clothed humans and preserving surface correspondences, while being an order of magnitude faster. We also perform an ablation study, which shows the usefulness of our pose-conditioning formulation, and our qualitative results display that INS can rectify artefacts introduced by LBS well. See our webpage for more details: https://yashkant.github.io/invertible-neural-skinning/

Title: Hyneter: Hybrid Network Transformer for Object Detection. (arXiv:2302.09365v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09365
• Code URL: null
• Copy Paste: [[2302.09365] Hyneter: Hybrid Network Transformer for Object Detection](http://arxiv.org/abs/2302.09365) #extraction
• Summary:

  In this paper, we point out that the essential differences between CNN-based and Transformer-based detectors, which cause the worse performance of small objects in Transformer-based methods, are the gap between local information and global dependencies in feature extraction and propagation. To address these differences, we propose a new vision Transformer, called Hybrid Network Transformer (Hyneter), after pre-experiments that indicate the gap causes CNN-based and Transformer-based methods to increase size-different objects result unevenly. Different from the divide and conquer strategy in previous methods, Hyneters consist of Hybrid Network Backbone (HNB) and Dual Switching module (DS), which integrate local information and global dependencies, and transfer them simultaneously. Based on the balance strategy, HNB extends the range of local information by embedding convolution layers into Transformer blocks, and DS adjusts excessive reliance on global dependencies outside the patch.

Title: MultiScale Probability Map guided Index Pooling with Attention-based learning for Road and Building Segmentation. (arXiv:2302.09411v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09411
• Code URL: null
• Copy Paste: [[2302.09411] MultiScale Probability Map guided Index Pooling with Attention-based learning for Road and Building Segmentation](http://arxiv.org/abs/2302.09411) #extraction
• Summary:

  Efficient road and building footprint extraction from satellite images are predominant in many remote sensing applications. However, precise segmentation map extraction is quite challenging due to the diverse building structures camouflaged by trees, similar spectral responses between the roads and buildings, and occlusions by heterogeneous traffic over the roads. Existing convolutional neural network (CNN)-based methods focus on either enriched spatial semantics learning for the building extraction or the fine-grained road topology extraction. The profound semantic information loss due to the traditional pooling mechanisms in CNN generates fragmented and disconnected road maps and poorly segmented boundaries for the densely spaced small buildings in complex surroundings. In this paper, we propose a novel attention-aware segmentation framework, Multi-Scale Supervised Dilated Multiple-Path Attention Network (MSSDMPA-Net), equipped with two new modules Dynamic Attention Map Guided Index Pooling (DAMIP) and Dynamic Attention Map Guided Spatial and Channel Attention (DAMSCA) to precisely extract the building footprints and road maps from remotely sensed images. DAMIP mines the salient features by employing a novel index pooling mechanism to retain important geometric information. On the other hand, DAMSCA simultaneously extracts the multi-scale spatial and spectral features. Besides, using dilated convolution and multi-scale deep supervision in optimizing MSSDMPA-Net helps achieve stellar performance. Experimental results over multiple benchmark building and road extraction datasets, ensures MSSDMPA-Net as the state-of-the-art (SOTA) method for building and road extraction.

Title: Extraction of Constituent Factors of Digestion Efficiency in Information Transfer by Media Composed of Texts and Images. (arXiv:2302.09189v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09189
• Code URL: null
• Copy Paste: [[2302.09189] Extraction of Constituent Factors of Digestion Efficiency in Information Transfer by Media Composed of Texts and Images](http://arxiv.org/abs/2302.09189) #extraction
• Summary:

  The development and spread of information and communication technologies have increased and diversified information. However, the increase in the volume and the selection of information does not necessarily promote understanding. In addition, conventional evaluations of information transfer have focused only on the arrival of information to the receivers. They need to sufficiently take into account the receivers' understanding of the information after it has been acquired, which is the original purpose of the evaluation. In this study, we propose the concept of "information digestion," which refers to the receivers' correct understanding of the acquired information, its contents, and its purpose. In the experiment, we proposed an evaluation model of information digestibility using hierarchical factor analysis and extracted factors that constitute digestibility by four types of media.

Title: Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents. (arXiv:2302.09324v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09324
• Code URL: null
• Copy Paste: [[2302.09324] Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents](http://arxiv.org/abs/2302.09324) #extraction
• Summary:

  While humans can extract information from unstructured text with high precision and recall, this is often too time-consuming to be practical. Automated approaches, on the other hand, produce nearly-immediate results, but may not be reliable enough for high-stakes applications where precision is essential. In this work, we consider the benefits and drawbacks of various human-only, human-machine, and machine-only information extraction approaches. We argue for the utility of a human-in-the-loop approach in applications where high precision is required, but purely manual extraction is infeasible. We present a framework and an accompanying tool for information extraction using weak-supervision labelling with human validation. We demonstrate our approach on three criminal justice datasets. We find that the combination of computer speed and human understanding yields precision comparable to manual annotation while requiring only a fraction of time, and significantly outperforms fully automated baselines in terms of precision.

membership infer

federate

Title: A Federated Approach for Hate Speech Detection. (arXiv:2302.09243v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09243
• Code URL: https://github.com/jaygala24/fed-hate-speech
• Copy Paste: [[2302.09243] A Federated Approach for Hate Speech Detection](http://arxiv.org/abs/2302.09243) #federate
• Summary:

  Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.

fair

Title: Closed-Loop Transcription via Convolutional Sparse Coding. (arXiv:2302.09347v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09347
• Code URL: null
• Copy Paste: [[2302.09347] Closed-Loop Transcription via Convolutional Sparse Coding](http://arxiv.org/abs/2302.09347) #fair
• Summary:

  Autoencoding has achieved great empirical success as a framework for learning generative models for natural images. Autoencoders often use generic deep networks as the encoder or decoder, which are difficult to interpret, and the learned representations lack clear structure. In this work, we make the explicit assumption that the image distribution is generated from a multi-stage sparse deconvolution. The corresponding inverse map, which we use as an encoder, is a multi-stage convolution sparse coding (CSC), with each stage obtained from unrolling an optimization algorithm for solving the corresponding (convexified) sparse coding program. To avoid computational difficulties in minimizing distributional distance between the real and generated images, we utilize the recent closed-loop transcription (CTRL) framework that optimizes the rate reduction of the learned sparse representations. Conceptually, our method has high-level connections to score-matching methods such as diffusion models. Empirically, our framework demonstrates competitive performance on large-scale datasets, such as ImageNet-1K, compared to existing autoencoding and generative methods under fair conditions. Even with simpler networks and fewer computational resources, our method demonstrates high visual quality in regenerated images. More surprisingly, the learned autoencoder performs well on unseen datasets. Our method enjoys several side benefits, including more structured and interpretable representations, more stable convergence, and scalability to large datasets. Our method is arguably the first to demonstrate that a concatenation of multiple convolution sparse coding/decoding layers leads to an interpretable and effective autoencoder for modeling the distribution of large-scale natural image datasets.

interpretability

Title: TAX: Tendency-and-Assignment Explainer for Semantic Segmentation with Multi-Annotators. (arXiv:2302.09561v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09561
• Code URL: null
• Copy Paste: [[2302.09561] TAX: Tendency-and-Assignment Explainer for Semantic Segmentation with Multi-Annotators](http://arxiv.org/abs/2302.09561) #interpretability
• Summary:

  To understand how deep neural networks perform classification predictions, recent research attention has been focusing on developing techniques to offer desirable explanations. However, most existing methods cannot be easily applied for semantic segmentation; moreover, they are not designed to offer interpretability under the multi-annotator setting. Instead of viewing ground-truth pixel-level labels annotated by a single annotator with consistent labeling tendency, we aim at providing interpretable semantic segmentation and answer two critical yet practical questions: "who" contributes to the resulting segmentation, and "why" such an assignment is determined. In this paper, we present a learning framework of Tendency-and-Assignment Explainer (TAX), designed to offer interpretability at the annotator and assignment levels. More specifically, we learn convolution kernel subsets for modeling labeling tendencies of each type of annotation, while a prototype bank is jointly observed to offer visual guidance for learning the above kernels. For evaluation, we consider both synthetic and real-world datasets with multi-annotators. We show that our TAX can be applied to state-of-the-art network architectures with comparable performances, while segmentation interpretability at both levels can be offered accordingly.

Title: Interpretability in Activation Space Analysis of Transformers: A Focused Survey. (arXiv:2302.09304v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09304
• Code URL: null
• Copy Paste: [[2302.09304] Interpretability in Activation Space Analysis of Transformers: A Focused Survey](http://arxiv.org/abs/2302.09304) #interpretability
• Summary:

  The field of natural language processing has reached breakthroughs with the advent of transformers. They have remained state-of-the-art since then, and there also has been much research in analyzing, interpreting, and evaluating the attention layers and the underlying embedding space. In addition to the self-attention layers, the feed-forward layers in the transformer are a prominent architectural component. From extensive research, we observe that its role is under-explored. We focus on the latent space, known as the Activation Space, that consists of the neuron activations from these feed-forward layers. In this survey paper, we review interpretability methods that examine the learnings that occurred in this activation space. Since there exists only limited research in this direction, we conduct a detailed examination of each work and point out potential future directions of research. We hope our work provides a step towards strengthening activation space analysis.

explainability

watermark

diffusion

Title: Exploring the Representation Manifolds of Stable Diffusion Through the Lens of Intrinsic Dimension. (arXiv:2302.09301v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2302.09301
• Code URL: null
• Copy Paste: [[2302.09301] Exploring the Representation Manifolds of Stable Diffusion Through the Lens of Intrinsic Dimension](http://arxiv.org/abs/2302.09301) #diffusion
• Summary:

  Prompting has become an important mechanism by which users can more effectively interact with many flavors of foundation model. Indeed, the last several years have shown that well-honed prompts can sometimes unlock emergent capabilities within such models. While there has been a substantial amount of empirical exploration of prompting within the community, relatively few works have studied prompting at a mathematical level. In this work we aim to take a first step towards understanding basic geometric properties induced by prompts in Stable Diffusion, focusing on the intrinsic dimension of internal representations within the model. We find that choice of prompt has a substantial impact on the intrinsic dimension of representations at both layers of the model which we explored, but that the nature of this impact depends on the layer being considered. For example, in certain bottleneck layers of the model, intrinsic dimension of representations is correlated with prompt perplexity (measured using a surrogate model), while this correlation is not apparent in the latent layers. Our evidence suggests that intrinsic dimension could be a useful tool for future studies of the impact of different prompts on text-to-image models.

Title: When Visible-to-Thermal Facial GAN Beats Conditional Diffusion. (arXiv:2302.09395v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2302.09395
• Code URL: null
• Copy Paste: [[2302.09395] When Visible-to-Thermal Facial GAN Beats Conditional Diffusion](http://arxiv.org/abs/2302.09395) #diffusion
• Summary:

  Thermal facial imagery offers valuable insight into physiological states such as inflammation and stress by detecting emitted radiation in the infrared spectrum, which is unseen in the visible spectra. Telemedicine applications could benefit from thermal imagery, but conventional computers are reliant on RGB cameras and lack thermal sensors. As a result, we propose the Visible-to-Thermal Facial GAN (VTF-GAN) that is specifically designed to generate high-resolution thermal faces by learning both the spatial and frequency domains of facial regions, across spectra. We compare VTF-GAN against several popular GAN baselines and the first conditional Denoising Diffusion Probabilistic Model (DDPM) for VT face translation (VTF-Diff). Results show that VTF-GAN achieves high quality, crisp, and perceptually realistic thermal faces using a combined set of patch, temperature, perceptual, and Fourier Transform losses, compared to all baselines including diffusion.

Title: Distributional Offline Policy Evaluation with Predictive Error Guarantees. (arXiv:2302.09456v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2302.09456
• Code URL: https://github.com/ziqian2000/fitted-likelihood-estimation
• Copy Paste: [[2302.09456] Distributional Offline Policy Evaluation with Predictive Error Guarantees](http://arxiv.org/abs/2302.09456) #diffusion
• Summary:

  We study the problem of estimating the distribution of the return of a policy using an offline dataset that is not generated from the policy, i.e., distributional offline policy evaluation (OPE). We propose an algorithm called Fitted Likelihood Estimation (FLE), which conducts a sequence of Maximum Likelihood Estimation (MLE) problems and has the flexibility of integrating any state-of-art probabilistic generative models as long as it can be trained via MLE. FLE can be used for both finite horizon and infinite horizon discounted settings where rewards can be multi-dimensional vectors. In our theoretical results, we show that for both finite and infinite horizon discounted settings, FLE can learn distributions that are close to the ground truth under total variation distance and Wasserstein distance, respectively. Our theoretical results hold under the conditions that the offline data covers the test policy's traces and the supervised learning MLE procedures succeed. Experimentally, we demonstrate the performance of FLE with two generative models, Gaussian mixture models and diffusion models. For the multi-dimensional reward setting, FLE with diffusion models is capable of estimating the complicated distribution of the return of a test policy.