secure

Title: RANDGENER: Distributed Randomness Beacon from Verifiable Delay Function. (arXiv:2310.12693v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12693
• Code URL: null
• Copy Paste: [[2310.12693]] RANDGENER: Distributed Randomness Beacon from Verifiable Delay Function(http://arxiv.org/abs/2310.12693)
• Summary:

  Buoyed by the excitement around secure decentralized applications, the last few decades have seen numerous constructions of distributed randomness beacons (DRB) along with use cases; however, a secure DRB (in many variations) remains an open problem. We further note that it is natural to want some kind of reward for participants who spend time and energy evaluating the randomness beacon value -- this is already common in distributed protocols.

  In this work, we present RandGener, a novel $n$-party commit-reveal-recover (or collaborative) DRB protocol with a novel reward and penalty mechanism along with a set of realistic guarantees. We design our protocol using trapdoor watermarkable verifiable delay functions in the RSA group setting (without requiring a trusted dealer or distributed key generation).

Title: Trenchcoat: Human-Computable Hashing Algorithms for Password Generation. (arXiv:2310.12706v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12706
• Code URL: null
• Copy Paste: [[2310.12706]] Trenchcoat: Human-Computable Hashing Algorithms for Password Generation(http://arxiv.org/abs/2310.12706)
• Summary:

  The average user has between 90-130 online accounts, and around $3 \times 10^{11}$ passwords are in use this year. Most people are terrible at remembering "random" passwords, so they reuse or create similar passwords using a combination of predictable words, numbers, and symbols. Previous password-generation or management protocols have imposed so large a cognitive load that users have abandoned them in favor of insecure yet simpler methods (e.g., writing them down or reusing minor variants).

  We describe a range of candidate human-computable "hash" functions suitable for use as password generators - as long as the human (with minimal education assumptions) keeps a single, easily-memorizable "master" secret - and rate them by various metrics, including effective security.

  These functions hash master-secrets with user accounts to produce sub-secrets that can be used as passwords; $F_R($s$, w) \longrightarrow y$, takes a website $w$, produces a password $y$, parameterized by master secret $s$, which may or may not be a string.

  We exploit the unique configuration $R$ of each user's associative and implicit memory (detailed in section 2) to ensure that sources of randomness unique to each user are present in each master-secret $F_R$. An adversary cannot compute or verify $F_R$ efficiently since $R$ is unique to each individual; in that sense, our hash function is similar to a physically unclonable function. For the algorithms we propose, the user need only complete primitive operations such as addition, spatial navigation or searching. Critically, most of our methods are also accessible to neurodiverse, or cognitively or physically differently-abled persons.

  We present results from a survey (n=134 individuals) investigating real-world usage of these methods and how people currently come up with their passwords, we also survey 400 websites to collate current password advice.

Title: Tight Short-Lived Signatures. (arXiv:2310.12723v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12723
• Code URL: null
• Copy Paste: [[2310.12723]] Tight Short-Lived Signatures(http://arxiv.org/abs/2310.12723)
• Summary:

  A Time-lock puzzle (TLP) sends information into the future: a predetermined number of sequential computations must occur (i.e., a predetermined amount of time must pass) to retrieve the information, regardless of parallelization. Buoyed by the excitement around secure decentralized applications and cryptocurrencies, the last decade has witnessed numerous constructions of TLP variants and related applications (e.g., cost-efficient blockchain designs, randomness beacons, e-voting, etc.).

  In this poster, we first extend the notion of TLP by formally defining the "time-lock public key encryption" (TLPKE) scheme. Next, we introduce and construct a "tight short-lived signatures" scheme using our TLPKE. Furthermore, to test the validity of our proposed schemes, we do a proof-of-concept implementation and run detailed simulations.

Title: TwinPot: Digital Twin-assisted Honeypot for Cyber-Secure Smart Seaports. (arXiv:2310.12880v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12880
• Code URL: null
• Copy Paste: [[2310.12880]] TwinPot: Digital Twin-assisted Honeypot for Cyber-Secure Smart Seaports(http://arxiv.org/abs/2310.12880)
• Summary:

  The idea of next-generation ports has become more apparent in the last ten years in response to the challenge posed by the rising demand for efficiency and the ever-increasing volume of goods. In this new era of intelligent infrastructure and facilities, it is evident that cyber-security has recently received the most significant attention from the seaport and maritime authorities, and it is a primary concern on the agenda of most ports. Traditional security solutions can be applied to safeguard IoT and Cyber-Physical Systems (CPS) from harmful entities. Nevertheless, security researchers can only watch, examine, and learn about the behaviors of attackers if these solutions operate more transparently. Herein, honeypots are potential solutions since they offer valuable information about the attackers. It can be virtual or physical. Virtual honeypots must be more realistic to entice attackers, necessitating better high-fidelity. To this end, Digital Twin (DT) technology can be employed to increase the complexity and simulation fidelity of the honeypots. Seaports can be attacked from both their existing devices and external devices at the same time. Existing mechanisms are insufficient to detect external attacks; therefore, the current systems cannot handle attacks at the desired level. DT and honeypot technologies can be used together to tackle them. Consequently, we suggest a DT-assisted honeypot, called TwinPot, for external attacks in smart seaports. Moreover, we propose an intelligent attack detection mechanism to handle different attack types using DT for internal attacks. Finally, we build an extensive smart seaport dataset for internal and external attacks using the MANSIM tool and two existing datasets to test the performance of our system. We show that under simultaneous internal and external attacks on the system, our solution successfully detects internal and external attacks.

security

Title: AI Potentiality and Awareness: A Position Paper from the Perspective of Human-AI Teaming in Cybersecurity. (arXiv:2310.12162v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12162
• Code URL: null
• Copy Paste: [[2310.12162]] AI Potentiality and Awareness: A Position Paper from the Perspective of Human-AI Teaming in Cybersecurity(http://arxiv.org/abs/2310.12162)
• Summary:

  This position paper explores the broad landscape of AI potentiality in the context of cybersecurity, with a particular emphasis on its possible risk factors with awareness, which can be managed by incorporating human experts in the loop, i.e., "Human-AI" teaming. As artificial intelligence (AI) technologies advance, they will provide unparalleled opportunities for attack identification, incident response, and recovery. However, the successful deployment of AI into cybersecurity measures necessitates an in-depth understanding of its capabilities, challenges, and ethical and legal implications to handle associated risk factors in real-world application areas. Towards this, we emphasize the importance of a balanced approach that incorporates AI's computational power with human expertise. AI systems may proactively discover vulnerabilities and detect anomalies through pattern recognition, and predictive modeling, significantly enhancing speed and accuracy. Human experts can explain AI-generated decisions to stakeholders, regulators, and end-users in critical situations, ensuring responsibility and accountability, which helps establish trust in AI-driven security solutions. Therefore, in this position paper, we argue that human-AI teaming is worthwhile in cybersecurity, in which human expertise such as intuition, critical thinking, or contextual understanding is combined with AI's computational power to improve overall cyber defenses.

Title: SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models. (arXiv:2310.12665v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12665
• Code URL: null
• Copy Paste: [[2310.12665]] SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models(http://arxiv.org/abs/2310.12665)
• Summary:

  While advanced machine learning (ML) models are deployed in numerous real-world applications, previous works demonstrate these models have security and privacy vulnerabilities. Various empirical research has been done in this field. However, most of the experiments are performed on target ML models trained by the security researchers themselves. Due to the high computational resource requirement for training advanced models with complex architectures, researchers generally choose to train a few target models using relatively simple architectures on typical experiment datasets. We argue that to understand ML models' vulnerabilities comprehensively, experiments should be performed on a large set of models trained with various purposes (not just the purpose of evaluating ML attacks and defenses). To this end, we propose using publicly available models with weights from the Internet (public models) for evaluating attacks and defenses on ML models. We establish a database, namely SecurityNet, containing 910 annotated image classification models. We then analyze the effectiveness of several representative attacks/defenses, including model stealing attacks, membership inference attacks, and backdoor detection on these public models. Our evaluation empirically shows the performance of these attacks/defenses can vary significantly on public models compared to self-trained models. We share SecurityNet with the research community. and advocate researchers to perform experiments on public models to better demonstrate their proposed methods' effectiveness in the future.

Title: knowCC: Knowledge, awareness of computer & cyber ethics between CS/non-CS university students. (arXiv:2310.12684v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12684
• Code URL: null
• Copy Paste: [[2310.12684]] knowCC: Knowledge, awareness of computer & cyber ethics between CS/non-CS university students(http://arxiv.org/abs/2310.12684)
• Summary:

  Technology has advanced dramatically in the previous several years. There are also cyber assaults. Cyberattacks pose a possible danger to information security and the general public. Since data practice and internet consumption rates continue to upswing, cyber awareness has become progressively important. Furthermore, as businesses pace their digital transformation with mobile devices, cloud services, communal media, and Internet of Things services, cybersecurity has appeared as a critical issue in corporate risk management. This research focuses on the relations between cybersecurity awareness, cyber knowledge, computer ethics, cyber ethics, and cyber behavior, as well as protective tools, across university students in general. The findings express that while internet users are alert of cyber threats, they only take the most elementary and easy-to-implement precautions. Several knowledge and awareness have been proposed to knob the issue of cyber security. It also grants the principles of cybersecurity in terms of its structure, workforces, and evidence pertaining to the shield of personal information in the cyber world. The first step is for people to educate themselves about the negative aspects of the internet and to learn more about cyber threats so that they can notice when an attack is taking place. To validate the efficiency of the suggested analysis between CS and non-CS university students, case study along with several comparisons are provided.

Title: TabuLa: Harnessing Language Models for Tabular Data Synthesis. (arXiv:2310.12746v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12746
• Code URL: null
• Copy Paste: [[2310.12746]] TabuLa: Harnessing Language Models for Tabular Data Synthesis(http://arxiv.org/abs/2310.12746)
• Summary:

  Given the ubiquitous use of tabular data in industries and the growing concerns in data privacy and security, tabular data synthesis emerges as a critical research area. The recent state-of-the-art methods show that large language models (LLMs) can be adopted to generate realistic tabular data. As LLMs pre-process tabular data as full text, they have the advantage of avoiding the curse of dimensionality associated with one-hot encoding high-dimensional data. However, their long training time and limited re-usability on new tasks prevent them from replacing exiting tabular generative models. In this paper, we propose Tabula, a tabular data synthesizer based on the language model structure. Through Tabula, we demonstrate the inherent limitation of employing pre-trained language models designed for natural language processing (NLP) in the context of tabular data synthesis. Our investigation delves into the development of a dedicated foundational model tailored specifically for tabular data synthesis. Additionally, we propose a token sequence compression strategy to significantly reduce training time while preserving the quality of synthetic data. Extensive experiments on six datasets demonstrate that using a language model structure without loading the well-trained model weights yields a better starting model for tabular data synthesis. Moreover, the Tabula model, previously trained on other tabular data, serves as an excellent foundation model for new tabular data synthesis tasks. Additionally, the token sequence compression method substantially reduces the model's training time. Results show that Tabula averagely reduces 46.2% training time per epoch comparing to current LLMs-based state-of-the-art algorithm and consistently achieves even higher synthetic data utility.

privacy

Title: PrivacyGAN: robust generative image privacy. (arXiv:2310.12590v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12590
• Code URL: null
• Copy Paste: [[2310.12590]] PrivacyGAN: robust generative image privacy(http://arxiv.org/abs/2310.12590)
• Summary:

  Classical techniques for protecting facial image privacy typically fall into two categories: data-poisoning methods, exemplified by Fawkes, which introduce subtle perturbations to images, or anonymization methods that generate images resembling the original only in several characteristics, such as gender, ethnicity, or facial expression.In this study, we introduce a novel approach, PrivacyGAN, that uses the power of image generation techniques, such as VQGAN and StyleGAN, to safeguard privacy while maintaining image usability, particularly for social media applications. Drawing inspiration from Fawkes, our method entails shifting the original image within the embedding space towards a decoy image.We evaluate our approach using privacy metrics on traditional and novel facial image datasets. Additionally, we propose new criteria for evaluating the robustness of privacy-protection methods against unknown image recognition techniques, and we demonstrate that our approach is effective even in unknown embedding transfer scenarios. We also provide a human evaluation that further proves that the modified image preserves its utility as it remains recognisable as an image of the same person by friends and family.

Title: Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples. (arXiv:2310.12707v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12707
• Code URL: https://github.com/csjunjun/ric
• Copy Paste: [[2310.12707]] Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples(http://arxiv.org/abs/2310.12707)
• Summary:

  With the increasing prevalence of cloud computing platforms, ensuring data privacy during the cloud-based image related services such as classification has become crucial. In this study, we propose a novel privacypreserving image classification scheme that enables the direct application of classifiers trained in the plaintext domain to classify encrypted images, without the need of retraining a dedicated classifier. Moreover, encrypted images can be decrypted back into their original form with high fidelity (recoverable) using a secret key. Specifically, our proposed scheme involves utilizing a feature extractor and an encoder to mask the plaintext image through a newly designed Noise-like Adversarial Example (NAE). Such an NAE not only introduces a noise-like visual appearance to the encrypted image but also compels the target classifier to predict the ciphertext as the same label as the original plaintext image. At the decoding phase, we adopt a Symmetric Residual Learning (SRL) framework for restoring the plaintext image with minimal degradation. Extensive experiments demonstrate that 1) the classification accuracy of the classifier trained in the plaintext domain remains the same in both the ciphertext and plaintext domains; 2) the encrypted images can be recovered into their original form with an average PSNR of up to 51+ dB for the SVHN dataset and 48+ dB for the VGGFace2 dataset; 3) our system exhibits satisfactory generalization capability on the encryption, decryption and classification tasks across datasets that are different from the training one; and 4) a high-level of security is achieved against three potential threat models. The code is available at https://github.com/csjunjun/RIC.git.

Title: PrivInfer: Privacy-Preserving Inference for Black-box Large Language Model. (arXiv:2310.12214v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12214
• Code URL: null
• Copy Paste: [[2310.12214]] PrivInfer: Privacy-Preserving Inference for Black-box Large Language Model(http://arxiv.org/abs/2310.12214)
• Summary:

  Large language models (LLMs), such as ChatGPT, have simplified text generation tasks, yet their inherent privacy risks are increasingly garnering attention. While differential privacy techniques have been successfully applied to text classification tasks, the resultant semantic bias makes them unsuitable for text generation. Homomorphic encryption inference methods have also been introduced, however, the significant computational and communication costs limit their viability. Furthermore, closed-source, black-box models such as GPT-4 withhold their architecture, thwarting certain privacy-enhancing strategies such as splitting inference into local and remote and then adding noise when communicating. To overcome these challenges, we introduce PrivInfer, the first privacy-preserving inference framework for black-box LLMs in text generation. Inspired by human writing, PrivInfer employs differential privacy methods to generate perturbed prompts for remote LLMs inference and extracts the meaningful response from the remote perturbed results. We also introduce RANTEXT, a differential privacy scheme specifically for LLMs that leverages random adjacency in text perturbations. Experimental results indicate that PrivInfer is comparable to GPT-4 in terms of text generation quality while protecting privacy, and RANTEXT provides enhanced privacy protection against three types of differential privacy attacks, including our newly introduced GPT inference attack, compared to baseline methods.

Title: Privacy-Preserving Hierarchical Anonymization Framework over Encrypted Data. (arXiv:2310.12401v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12401
• Code URL: null
• Copy Paste: [[2310.12401]] Privacy-Preserving Hierarchical Anonymization Framework over Encrypted Data(http://arxiv.org/abs/2310.12401)
• Summary:

  Smart cities, which can monitor the real world and provide smart services in a variety of fields, have improved people's living standards as urbanization has accelerated. However, there are security and privacy concerns because smart city applications collect large amounts of privacy-sensitive information from people and their social circles. Anonymization, which generalizes data and reduces data uniqueness is an important step in preserving the privacy of sensitive information. However, anonymization methods frequently require large datasets and rely on untrusted third parties to collect and manage data, particularly in a cloud environment. In this case, private data leakage remains a critical issue, discouraging users from sharing their data and impeding the advancement of smart city services. This problem can be solved if the computational entity can perform the anonymization process without obtaining the original plain text. This study proposed a hierarchical k-anonymization framework using homomorphic encryption and secret sharing composed of two types of domains. Different computing methods are selected flexibly, and two domains are connected hierarchically to obtain higher-level anonymization results in an efficient manner. The experimental results show that connecting two domains can accelerate the anonymization process, indicating that the proposed secure hierarchical architecture is practical and efficient.

Title: Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework. (arXiv:2310.12523v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12523
• Code URL: null
• Copy Paste: [[2310.12523]] Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework(http://arxiv.org/abs/2310.12523)
• Summary:

  The generative Artificial Intelligence (AI) tools based on Large Language Models (LLMs) use billions of parameters to extensively analyse large datasets and extract critical private information such as, context, specific details, identifying information etc. This have raised serious threats to user privacy and reluctance to use such tools. This article proposes the conceptual model called PrivChatGPT, a privacy-preserving model for LLMs that consists of two main components i.e., preserving user privacy during the data curation/pre-processing together with preserving private context and the private training process for large-scale data. To demonstrate its applicability, we show how a private mechanism could be integrated into the existing model for training LLMs to protect user privacy; specifically, we employed differential privacy and private training using Reinforcement Learning (RL). We measure the privacy loss and evaluate the measure of uncertainty or randomness once differential privacy is applied. It further recursively evaluates the level of privacy guarantees and the measure of uncertainty of public database and resources, during each update when new information is added for training purposes. To critically evaluate the use of differential privacy for private LLMs, we hypothetically compared other mechanisms e..g, Blockchain, private information retrieval, randomisation, for various performance measures such as the model performance and accuracy, computational complexity, privacy vs. utility etc. We conclude that differential privacy, randomisation, and obfuscation can impact utility and performance of trained models, conversely, the use of ToR, Blockchain, and PIR may introduce additional computational complexity and high training latency. We believe that the proposed model could be used as a benchmark for proposing privacy preserving LLMs for generative AI tools.

protect

defense

Title: Prompt Injection Attacks and Defenses in LLM-Integrated Applications. (arXiv:2310.12815v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12815
• Code URL: https://github.com/liu00222/open-prompt-injection
• Copy Paste: [[2310.12815]] Prompt Injection Attacks and Defenses in LLM-Integrated Applications(http://arxiv.org/abs/2310.12815)
• Summary:

  Large Language Models (LLMs) are increasingly deployed as the backend for a variety of real-world applications called LLM-Integrated Applications. Multiple recent works showed that LLM-Integrated Applications are vulnerable to prompt injection attacks, in which an attacker injects malicious instruction/data into the input of those applications such that they produce results as the attacker desires. However, existing works are limited to case studies. As a result, the literature lacks a systematic understanding of prompt injection attacks and their defenses. We aim to bridge the gap in this work. In particular, we propose a general framework to formalize prompt injection attacks. Existing attacks, which are discussed in research papers and blog posts, are special cases in our framework. Our framework enables us to design a new attack by combining existing attacks. Moreover, we also propose a framework to systematize defenses against prompt injection attacks. Using our frameworks, we conduct a systematic evaluation on prompt injection attacks and their defenses with 10 LLMs and 7 tasks. We hope our frameworks can inspire future research in this field. Our code is available at https://github.com/liu00222/Open-Prompt-Injection.

Title: Learn from the Past: A Proxy based Adversarial Defense Framework to Boost Robustness. (arXiv:2310.12713v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12713
• Code URL: null
• Copy Paste: [[2310.12713]] Learn from the Past: A Proxy based Adversarial Defense Framework to Boost Robustness(http://arxiv.org/abs/2310.12713)
• Summary:

  In light of the vulnerability of deep learning models to adversarial samples and the ensuing security issues, a range of methods, including Adversarial Training (AT) as a prominent representative, aimed at enhancing model robustness against various adversarial attacks, have seen rapid development. However, existing methods essentially assist the current state of target model to defend against parameter-oriented adversarial attacks with explicit or implicit computation burdens, which also suffers from unstable convergence behavior due to inconsistency of optimization trajectories. Diverging from previous work, this paper reconsiders the update rule of target model and corresponding deficiency to defend based on its current state. By introducing the historical state of the target model as a proxy, which is endowed with much prior information for defense, we formulate a two-stage update rule, resulting in a general adversarial defense framework, which we refer to as `LAST' ({\bf L}earn from the P{\bf ast}). Besides, we devise a Self Distillation (SD) based defense objective to constrain the update process of the proxy model without the introduction of larger teacher models. Experimentally, we demonstrate consistent and significant performance enhancements by refining a series of single-step and multi-step AT methods (e.g., up to $\bf 9.2\%$ and $\bf 20.5\%$ improvement of Robust Accuracy (RA) on CIFAR10 and CIFAR100 datasets, respectively) across various datasets, backbones and attack modalities, and validate its ability to enhance training stability and ameliorate catastrophic overfitting issues meanwhile.

attack

Title: REVAMP: Automated Simulations of Adversarial Attacks on Arbitrary Objects in Realistic Scenes. (arXiv:2310.12243v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12243
• Code URL: https://github.com/poloclub/revamp
• Copy Paste: [[2310.12243]] REVAMP: Automated Simulations of Adversarial Attacks on Arbitrary Objects in Realistic Scenes(http://arxiv.org/abs/2310.12243)
• Summary:

  Deep Learning models, such as those used in an autonomous vehicle are vulnerable to adversarial attacks where an attacker could place an adversarial object in the environment, leading to mis-classification. Generating these adversarial objects in the digital space has been extensively studied, however successfully transferring these attacks from the digital realm to the physical realm has proven challenging when controlling for real-world environmental factors. In response to these limitations, we introduce REVAMP, an easy-to-use Python library that is the first-of-its-kind tool for creating attack scenarios with arbitrary objects and simulating realistic environmental factors, lighting, reflection, and refraction. REVAMP enables researchers and practitioners to swiftly explore various scenarios within the digital realm by offering a wide range of configurable options for designing experiments and using differentiable rendering to reproduce physically plausible adversarial objects. We will demonstrate and invite the audience to try REVAMP to produce an adversarial texture on a chosen object while having control over various scene parameters. The audience will choose a scene, an object to attack, the desired attack class, and the number of camera positions to use. Then, in real time, we show how this altered texture causes the chosen object to be mis-classified, showcasing the potential of REVAMP in real-world scenarios. REVAMP is open-source and available at https://github.com/poloclub/revamp.

Title: Segment Anything Meets Universal Adversarial Perturbation. (arXiv:2310.12431v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12431
• Code URL: null
• Copy Paste: [[2310.12431]] Segment Anything Meets Universal Adversarial Perturbation(http://arxiv.org/abs/2310.12431)
• Summary:

  As Segment Anything Model (SAM) becomes a popular foundation model in computer vision, its adversarial robustness has become a concern that cannot be ignored. This works investigates whether it is possible to attack SAM with image-agnostic Universal Adversarial Perturbation (UAP). In other words, we seek a single perturbation that can fool the SAM to predict invalid masks for most (if not all) images. We demonstrate convetional image-centric attack framework is effective for image-independent attacks but fails for universal adversarial attack. To this end, we propose a novel perturbation-centric framework that results in a UAP generation method based on self-supervised contrastive learning (CL), where the UAP is set to the anchor sample and the positive sample is augmented from the UAP. The representations of negative samples are obtained from the image encoder in advance and saved in a memory bank. The effectiveness of our proposed CL-based UAP generation method is validated by both quantitative and qualitative results. On top of the ablation study to understand various components in our proposed method, we shed light on the roles of positive and negative samples in making the generated UAP effective for attacking SAM.

Title: PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models. (arXiv:2310.12439v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12439
• Code URL: null
• Copy Paste: [[2310.12439]] PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models(http://arxiv.org/abs/2310.12439)
• Summary:

  Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.

Title: Attack Prompt Generation for Red Teaming and Defending Large Language Models. (arXiv:2310.12505v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12505
• Code URL: https://github.com/aatrox103/sap
• Copy Paste: [[2310.12505]] Attack Prompt Generation for Red Teaming and Defending Large Language Models(http://arxiv.org/abs/2310.12505)
• Summary:

  Large language models (LLMs) are susceptible to red teaming attacks, which can induce LLMs to generate harmful content. Previous research constructs attack prompts via manual or automatic methods, which have their own limitations on construction cost and quality. To address these issues, we propose an integrated approach that combines manual and automatic methods to economically generate high-quality attack prompts. Specifically, considering the impressive capabilities of newly emerged LLMs, we propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning. Furthermore, we propose a defense framework that fine-tunes victim LLMs through iterative interactions with the attack framework to enhance their safety against red teaming attacks. Extensive experiments on different LLMs validate the effectiveness of our proposed attack and defense frameworks. Additionally, we release a series of attack prompts datasets named SAP with varying sizes, facilitating the safety evaluation and enhancement of more LLMs. Our code and dataset is available on https://github.com/Aatrox103/SAP .

Title: Automatic Hallucination Assessment for Aligned Large Language Models via Transferable Adversarial Attacks. (arXiv:2310.12516v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12516
• Code URL: null
• Copy Paste: [[2310.12516]] Automatic Hallucination Assessment for Aligned Large Language Models via Transferable Adversarial Attacks(http://arxiv.org/abs/2310.12516)
• Summary:

  Although remarkable progress has been achieved in preventing large language model (LLM) hallucinations using instruction tuning and retrieval augmentation, it remains challenging to measure the reliability of LLMs using human-crafted evaluation data which is not available for many tasks and domains and could suffer from data leakage. Inspired by adversarial machine learning, this paper aims to develop a method of automatically generating evaluation data by appropriately modifying existing data on which LLMs behave faithfully. Specifically, this paper presents AutoDebug, an LLM-based framework to use prompting chaining to generate transferable adversarial attacks in the form of question-answering examples. We seek to understand the extent to which these examples trigger the hallucination behaviors of LLMs.

  We implement AutoDebug using ChatGPT and evaluate the resulting two variants of a popular open-domain question-answering dataset, Natural Questions (NQ), on a collection of open-source and proprietary LLMs under various prompting settings. Our generated evaluation data is human-readable and, as we show, humans can answer these modified questions well. Nevertheless, we observe pronounced accuracy drops across multiple LLMs including GPT-4. Our experimental results show that LLMs are likely to hallucinate in two categories of question-answering scenarios where (1) there are conflicts between knowledge given in the prompt and their parametric knowledge, or (2) the knowledge expressed in the prompt is complex. Finally, we find that the adversarial examples generated by our method are transferable across all considered LLMs. The examples generated by a small model can be used to debug a much larger model, making our approach cost-effective.

Title: Charge Manipulation Attacks Against Smart Electric Vehicle Charging Stations and Deep Learning-based Detection Mechanisms. (arXiv:2310.12254v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12254
• Code URL: null
• Copy Paste: [[2310.12254]] Charge Manipulation Attacks Against Smart Electric Vehicle Charging Stations and Deep Learning-based Detection Mechanisms(http://arxiv.org/abs/2310.12254)
• Summary:

  The widespread deployment of "smart" electric vehicle charging stations (EVCSs) will be a key step toward achieving green transportation. The connectivity features of smart EVCSs can be utilized to schedule EV charging operations while respecting user preferences, thus avoiding synchronous charging from a large number of customers and relieving grid congestion. However, the communication and connectivity requirements involved in smart charging raise cybersecurity concerns. In this work, we investigate charge manipulation attacks (CMAs) against EV charging, in which an attacker manipulates the information exchanged during smart charging operations. The objective of CMAs is to shift the EV aggregator's demand across different times of the day. The proposed CMAs can bypass existing protection mechanisms in EV communication protocols. We quantify the impact of CMAs on the EV aggregator's economic profit by modeling their participation in the day-ahead (DA) and real-time (RT) electricity markets. Finally, we propose an unsupervised deep learning-based mechanism to detect CMAs by monitoring the parameters involved in EV charging. We extensively analyze the attack impact and the efficiency of the proposed detection on real-world EV charging datasets. The results highlight the vulnerabilities of smart charging operations and the need for a monitoring mechanism to detect malicious CMAs.

Title: Notes on Small Private Key Attacks on Common Prime RSA. (arXiv:2310.12572v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12572
• Code URL: null
• Copy Paste: [[2310.12572]] Notes on Small Private Key Attacks on Common Prime RSA(http://arxiv.org/abs/2310.12572)
• Summary:

  We point out critical deficiencies in lattice-based cryptanalysis of common prime RSA presented in ``Remarks on the cryptanalysis of common prime RSA for IoT constrained low power devices'' [Information Sciences, 538 (2020) 54--68]. To rectify these flaws, we carefully scrutinize the relevant parameters involved in the analysis during solving a specific trivariate integer polynomial equation. Additionally, we offer a synthesized attack illustration of small private key attacks on common prime RSA.

Title: CAT: Closed-loop Adversarial Training for Safe End-to-End Driving. (arXiv:2310.12432v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12432
• Code URL: null
• Copy Paste: [[2310.12432]] CAT: Closed-loop Adversarial Training for Safe End-to-End Driving(http://arxiv.org/abs/2310.12432)
• Summary:

  Driving safety is a top priority for autonomous vehicles. Orthogonal to prior work handling accident-prone traffic events by algorithm designs at the policy level, we investigate a Closed-loop Adversarial Training (CAT) framework for safe end-to-end driving in this paper through the lens of environment augmentation. CAT aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios that are dynamically generated over time. A novel resampling technique is developed to turn log-replay real-world driving scenarios into safety-critical ones via probabilistic factorization, where the adversarial traffic generation is modeled as the multiplication of standard motion prediction sub-problems. Consequently, CAT can launch more efficient physical attacks compared to existing safety-critical scenario generation methods and yields a significantly less computational cost in the iterative learning pipeline. We incorporate CAT into the MetaDrive simulator and validate our approach on hundreds of driving scenarios imported from real-world driving datasets. Experimental results demonstrate that CAT can effectively generate adversarial scenarios countering the agent being trained. After training, the agent can achieve superior driving safety in both log-replay and safety-critical traffic scenarios on the held-out test set. Code and data are available at https://metadriverse.github.io/cat.

robust

Title: Mesh Represented Recycle Learning for 3D Hand Pose and Mesh Estimation. (arXiv:2310.12189v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12189
• Code URL: null
• Copy Paste: [[2310.12189]] Mesh Represented Recycle Learning for 3D Hand Pose and Mesh Estimation(http://arxiv.org/abs/2310.12189)
• Summary:

  In general, hand pose estimation aims to improve the robustness of model performance in the real-world scenes. However, it is difficult to enhance the robustness since existing datasets are obtained in restricted environments to annotate 3D information. Although neural networks quantitatively achieve a high estimation accuracy, unsatisfied results can be observed in visual quality. This discrepancy between quantitative results and their visual qualities remains an open issue in the hand pose representation. To this end, we propose a mesh represented recycle learning strategy for 3D hand pose and mesh estimation which reinforces synthesized hand mesh representation in a training phase. To be specific, a hand pose and mesh estimation model first predicts parametric 3D hand annotations (i.e., 3D keypoint positions and vertices for hand mesh) with real-world hand images in the training phase. Second, synthetic hand images are generated with self-estimated hand mesh representations. After that, the synthetic hand images are fed into the same model again. Thus, the proposed learning strategy simultaneously improves quantitative results and visual qualities by reinforcing synthetic mesh representation. To encourage consistency between original model output and its recycled one, we propose self-correlation loss which maximizes the accuracy and reliability of our learning strategy. Consequently, the model effectively conducts self-refinement on hand pose estimation by learning mesh representation from its own output. To demonstrate the effectiveness of our learning strategy, we provide extensive experiments on FreiHAND dataset. Notably, our learning strategy improves the performance on hand pose and mesh estimation without any extra computational burden during the inference.

Title: WeedCLR: Weed Contrastive Learning through Visual Representations with Class-Optimized Loss in Long-Tailed Datasets. (arXiv:2310.12465v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12465
• Code URL: null
• Copy Paste: [[2310.12465]] WeedCLR: Weed Contrastive Learning through Visual Representations with Class-Optimized Loss in Long-Tailed Datasets(http://arxiv.org/abs/2310.12465)
• Summary:

  Image classification is a crucial task in modern weed management and crop intervention technologies. However, the limited size, diversity, and balance of existing weed datasets hinder the development of deep learning models for generalizable weed identification. In addition, the expensive labelling requirements of mainstream fully-supervised weed classifiers make them cost- and time-prohibitive to deploy widely, for new weed species, and in site-specific weed management. This paper proposes a novel method for Weed Contrastive Learning through visual Representations (WeedCLR), that uses class-optimized loss with Von Neumann Entropy of deep representation for weed classification in long-tailed datasets. WeedCLR leverages self-supervised learning to learn rich and robust visual features without any labels and applies a class-optimized loss function to address the class imbalance problem in long-tailed datasets. WeedCLR is evaluated on two public weed datasets: CottonWeedID15, containing 15 weed species, and DeepWeeds, containing 8 weed species. WeedCLR achieves an average accuracy improvement of 4.3\% on CottonWeedID15 and 5.6\% on DeepWeeds over previous methods. It also demonstrates better generalization ability and robustness to different environmental conditions than existing methods without the need for expensive and time-consuming human annotations. These significant improvements make WeedCLR an effective tool for weed classification in long-tailed datasets and allows for more rapid and widespread deployment of site-specific weed management and crop intervention technologies.

Title: FUSC: Fetal Ultrasound Semantic Clustering of Second Trimester Scans Using Deep Self-supervised Learning. (arXiv:2310.12600v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12600
• Code URL: null
• Copy Paste: [[2310.12600]] FUSC: Fetal Ultrasound Semantic Clustering of Second Trimester Scans Using Deep Self-supervised Learning(http://arxiv.org/abs/2310.12600)
• Summary:

  Ultrasound is the primary imaging modality in clinical practice during pregnancy. More than 140M fetuses are born yearly, resulting in numerous scans. The availability of a large volume of fetal ultrasound scans presents the opportunity to train robust machine learning models. However, the abundance of scans also has its challenges, as manual labeling of each image is needed for supervised methods. Labeling is typically labor-intensive and requires expertise to annotate the images accurately. This study presents an unsupervised approach for automatically clustering ultrasound images into a large range of fetal views, reducing or eliminating the need for manual labeling. Our Fetal Ultrasound Semantic Clustering (FUSC) method is developed using a large dataset of 88,063 images and further evaluated on an additional unseen dataset of 8,187 images achieving over 92% clustering purity. The result of our investigation hold the potential to significantly impact the field of fetal ultrasound imaging and pave the way for more advanced automated labeling solutions. Finally, we make the code and the experimental setup publicly available to help advance the field.

Title: Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding. (arXiv:2310.12724v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12724
• Code URL: null
• Copy Paste: [[2310.12724]] Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding(http://arxiv.org/abs/2310.12724)
• Summary:

  The surge in video and social media content underscores the need for a deeper understanding of multimedia data. Most of the existing mature video understanding techniques perform well with short formats and content that requires only shallow understanding, but do not perform well with long format videos that require deep understanding and reasoning. Deep Video Understanding (DVU) Challenge aims to push the boundaries of multimodal extraction, fusion, and analytics to address the problem of holistically analyzing long videos and extract useful knowledge to solve different types of queries. This paper introduces a query-aware method for long video localization and relation discrimination, leveraging an imagelanguage pretrained model. This model adeptly selects frames pertinent to queries, obviating the need for a complete movie-level knowledge graph. Our approach achieved first and fourth positions for two groups of movie-level queries. Sufficient experiments and final rankings demonstrate its effectiveness and robustness.

Title: Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping. (arXiv:2310.12769v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12769
• Code URL: https://github.com/butkej/protomixer
• Copy Paste: [[2310.12769]] Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping(http://arxiv.org/abs/2310.12769)
• Summary:

  Whole-slide image analysis via the means of computational pathology often relies on processing tessellated gigapixel images with only slide-level labels available. Applying multiple instance learning-based methods or transformer models is computationally expensive as, for each image, all instances have to be processed simultaneously. The MLP-Mixer is an under-explored alternative model to common vision transformers, especially for large-scale datasets. Due to the lack of a self-attention mechanism, they have linear computational complexity to the number of input patches but achieve comparable performance on natural image datasets. We propose a combination of feature embedding and clustering to preprocess the full whole-slide image into a reduced prototype representation which can then serve as input to a suitable MLP-Mixer architecture. Our experiments on two public benchmarks and one inhouse malignant lymphoma dataset show comparable performance to current state-of-the-art methods, while achieving lower training costs in terms of computational time and memory load. Code is publicly available at https://github.com/butkej/ProtoMixer.

Title: OODRobustBench: benchmarking and analyzing adversarial robustness under distribution shift. (arXiv:2310.12793v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12793
• Code URL: null
• Copy Paste: [[2310.12793]] OODRobustBench: benchmarking and analyzing adversarial robustness under distribution shift(http://arxiv.org/abs/2310.12793)
• Summary:

  Existing works have made great progress in improving adversarial robustness, but typically test their method only on data from the same distribution as the training data, i.e. in-distribution (ID) testing. As a result, it is unclear how such robustness generalizes under input distribution shifts, i.e. out-of-distribution (OOD) testing. This is a concerning omission as such distribution shifts are unavoidable when methods are deployed in the wild. To address this issue we propose a benchmark named OODRobustBench to comprehensively assess OOD adversarial robustness using 23 dataset-wise shifts (i.e. naturalistic shifts in input distribution) and 6 threat-wise shifts (i.e., unforeseen adversarial threat models). OODRobustBench is used to assess 706 robust models using 60.7K adversarial evaluations. This large-scale analysis shows that: 1) adversarial robustness suffers from a severe OOD generalization issue; 2) ID robustness correlates strongly with OOD robustness, in a positive linear way, under many distribution shifts. The latter enables the prediction of OOD robustness from ID robustness. Based on this, we are able to predict the upper limit of OOD robustness for existing robust training schemes. The results suggest that achieving OOD robustness requires designing novel methods beyond the conventional ones. Last, we discover that extra data, data augmentation, advanced model architectures and particular regularization approaches can improve OOD robustness. Noticeably, the discovered training schemes, compared to the baseline, exhibit dramatically higher robustness under threat shift while keeping high ID robustness, demonstrating new promising solutions for robustness against both multi-attack and unforeseen attacks.

Title: REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models. (arXiv:2310.12362v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12362
• Code URL: null
• Copy Paste: [[2310.12362]] REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models(http://arxiv.org/abs/2310.12362)
• Summary:

  We present REMARK-LLM, a novel efficient, and robust watermarking framework designed for texts generated by large language models (LLMs). Synthesizing human-like content using LLMs necessitates vast computational resources and extensive datasets, encapsulating critical intellectual property (IP). However, the generated content is prone to malicious exploitation, including spamming and plagiarism. To address the challenges, REMARK-LLM proposes three new components: (i) a learning-based message encoding module to infuse binary signatures into LLM-generated texts; (ii) a reparameterization module to transform the dense distributions from the message encoding to the sparse distribution of the watermarked textual tokens; (iii) a decoding module dedicated for signature extraction; Furthermore, we introduce an optimized beam search algorithm to guarantee the coherence and consistency of the generated content. REMARK-LLM is rigorously trained to encourage the preservation of semantic integrity in watermarked content, while ensuring effective watermark retrieval. Extensive evaluations on multiple unseen datasets highlight REMARK-LLM proficiency and transferability in inserting 2 times more signature bits into the same texts when compared to prior art, all while maintaining semantic integrity. Furthermore, REMARK-LLM exhibits better resilience against a spectrum of watermark detection and removal attacks.

Title: Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications. (arXiv:2310.12620v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12620
• Code URL: null
• Copy Paste: [[2310.12620]] Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications(http://arxiv.org/abs/2310.12620)
• Summary:

  Temporal data distribution shift is prevalent in the financial text. How can a financial sentiment analysis system be trained in a volatile market environment that can accurately infer sentiment and be robust to temporal data distribution shifts? In this paper, we conduct an empirical study on the financial sentiment analysis system under temporal data distribution shifts using a real-world financial social media dataset that spans three years. We find that the fine-tuned models suffer from general performance degradation in the presence of temporal distribution shifts. Furthermore, motivated by the unique temporal nature of the financial text, we propose a novel method that combines out-of-distribution detection with time series modeling for temporal financial sentiment analysis. Experimental results show that the proposed method enhances the model's capability to adapt to evolving temporal shifts in a volatile financial market.

Title: Causal-structure Driven Augmentations for Text OOD Generalization. (arXiv:2310.12803v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12803
• Code URL: null
• Copy Paste: [[2310.12803]] Causal-structure Driven Augmentations for Text OOD Generalization(http://arxiv.org/abs/2310.12803)
• Summary:

  The reliance of text classifiers on spurious correlations can lead to poor generalization at deployment, raising concerns about their use in safety-critical domains such as healthcare. In this work, we propose to use counterfactual data augmentation, guided by knowledge of the causal structure of the data, to simulate interventions on spurious features and to learn more robust text classifiers. We show that this strategy is appropriate in prediction problems where the label is spuriously correlated with an attribute. Under the assumptions of such problems, we discuss the favorable sample complexity of counterfactual data augmentation, compared to importance re-weighting. Pragmatically, we match examples using auxiliary data, based on diff-in-diff methodology, and use a large language model (LLM) to represent a conditional probability of text. Through extensive experimentation on learning caregiver-invariant predictors of clinical diagnoses from medical narratives and on semi-synthetic data, we demonstrate that our method for simulating interventions improves out-of-distribution (OOD) accuracy compared to baseline invariant learning algorithms.

Title: Model Merging by Uncertainty-Based Gradient Matching. (arXiv:2310.12808v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12808
• Code URL: null
• Copy Paste: [[2310.12808]] Model Merging by Uncertainty-Based Gradient Matching(http://arxiv.org/abs/2310.12808)
• Summary:

  Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to mismatches in the gradients and propose a new uncertainty-based scheme to improve the performance by reducing the mismatch. The connection also reveals implicit assumptions in other schemes such as averaging, task arithmetic, and Fisher-weighted averaging. Our new method gives consistent improvements for large language models and vision transformers, both in terms of performance and robustness to hyperparameters.

Title: AgentTuning: Enabling Generalized Agent Abilities for LLMs. (arXiv:2310.12823v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12823
• Code URL: null
• Copy Paste: [[2310.12823]] AgentTuning: Enabling Generalized Agent Abilities for LLMs(http://arxiv.org/abs/2310.12823)
• Summary:

  Open large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in the real world. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization, necessitating both fine-grained prompting methods and robust LLMs to achieve satisfactory performance. Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show that AgentTuning enables LLMs' agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct and AgentLM-7B, 13B, and 70B models at https://github.com/THUDM/AgentTuning , serving open and powerful alternatives to commercial LLMs for agent tasks.

Title: Probing LLMs for hate speech detection: strengths and vulnerabilities. (arXiv:2310.12860v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12860
• Code URL: null
• Copy Paste: [[2310.12860]] Probing LLMs for hate speech detection: strengths and vulnerabilities(http://arxiv.org/abs/2310.12860)
• Summary:

  Recently efforts have been made by social media platforms as well as researchers to detect hateful or toxic language using large language models. However, none of these works aim to use explanation, additional context and victim community information in the detection process. We utilise different prompt variation, input information and evaluate large language models in zero shot setting (without adding any in-context examples). We select three large language models (GPT-3.5, text-davinci and Flan-T5) and three datasets - HateXplain, implicit hate and ToxicSpans. We find that on average including the target information in the pipeline improves the model performance substantially (~20-30%) over the baseline across the datasets. There is also a considerable effect of adding the rationales/explanations into the pipeline (~10-20%) over the baseline across the datasets. In addition, we further provide a typology of the error cases where these large language models fail to (i) classify and (ii) explain the reason for the decisions they take. Such vulnerable points automatically constitute 'jailbreak' prompts for these models and industry scale safeguard techniques need to be developed to make the models robust against such prompts.

Title: A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems. (arXiv:2310.12892v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12892
• Code URL: null
• Copy Paste: [[2310.12892]] A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems(http://arxiv.org/abs/2310.12892)
• Summary:

  Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically analyse task performance disparities that exist between multilingual task-oriented dialogue (ToD) systems. We first define new quantitative measures of absolute and relative equivalence in system performance, capturing disparities across languages and within individual languages. Through a series of controlled experiments, we demonstrate that performance disparities depend on a number of factors: the nature of the ToD task at hand, the underlying pretrained language model, the target language, and the amount of ToD annotated data. We empirically prove the existence of the adaptation and intrinsic biases in current ToD systems: e.g., ToD systems trained for Arabic or Turkish using annotated ToD data fully parallel to English ToD data still exhibit diminished ToD task performance. Beyond providing a series of insights into the performance disparities of ToD systems in different languages, our analyses offer practical tips on how to approach ToD data collection and system development for new languages.

Title: Classification-Aided Robust Multiple Target Tracking Using Neural Enhanced Message Passing. (arXiv:2310.12407v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12407
• Code URL: null
• Copy Paste: [[2310.12407]] Classification-Aided Robust Multiple Target Tracking Using Neural Enhanced Message Passing(http://arxiv.org/abs/2310.12407)
• Summary:

  We address the challenge of tracking an unknown number of targets in strong clutter environments using measurements from a radar sensor. Leveraging the range-Doppler spectra information, we identify the measurement classes, which serve as additional information to enhance clutter rejection and data association, thus bolstering the robustness of target tracking. We first introduce a novel neural enhanced message passing approach, where the beliefs obtained by the unified message passing are fed into the neural network as additional information. The output beliefs are then utilized to refine the original beliefs. Then, we propose a classification-aided robust multiple target tracking algorithm, employing the neural enhanced message passing technique. This algorithm is comprised of three modules: a message-passing module, a neural network module, and a Dempster-Shafer module. The message-passing module is used to represent the statistical model by the factor graph and infers target kinematic states, visibility states, and data associations based on the spatial measurement information. The neural network module is employed to extract features from range-Doppler spectra and derive beliefs on whether a measurement is target-generated or clutter-generated. The Dempster-Shafer module is used to fuse the beliefs obtained from both the factor graph and the neural network. As a result, our proposed algorithm adopts a model-and-data-driven framework, effectively enhancing clutter suppression and data association, leading to significant improvements in multiple target tracking performance. We validate the effectiveness of our approach using both simulated and real data scenarios, demonstrating its capability to handle challenging tracking scenarios in practical radar applications.

Title: SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models. (arXiv:2310.12494v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12494
• Code URL: null
• Copy Paste: [[2310.12494]] SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models(http://arxiv.org/abs/2310.12494)
• Summary:

  Understanding the long-term impact of algorithmic interventions on society is vital to achieving responsible AI. Traditional evaluation strategies often fall short due to the complex, adaptive and dynamic nature of society. While reinforcement learning (RL) can be a powerful approach for optimizing decisions in dynamic settings, the difficulty of realistic environment design remains a barrier to building robust agents that perform well in practical settings. To address this issue we tap into the field of system dynamics (SD) as a complementary method that incorporates collaborative simulation model specification practices. We introduce SDGym, a low-code library built on the OpenAI Gym framework which enables the generation of custom RL environments based on SD simulation models. Through a feasibility study we validate that well specified, rich RL environments can be generated from preexisting SD models and a few lines of configuration code. We demonstrate the capabilities of the SDGym environment using an SD model of the electric vehicle adoption problem. We compare two SD simulators, PySD and BPTK-Py for parity, and train a D4PG agent using the Acme framework to showcase learning and environment interaction. Our preliminary findings underscore the dual potential of SD to improve RL environment design and for RL to improve dynamic policy discovery within SD models. By open-sourcing SDGym, the intent is to galvanize further research and promote adoption across the SD and RL communities, thereby catalyzing collaboration in this emerging interdisciplinary space.

Title: Fast Model Debias with Machine Unlearning. (arXiv:2310.12560v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12560
• Code URL: null
• Copy Paste: [[2310.12560]] Fast Model Debias with Machine Unlearning(http://arxiv.org/abs/2310.12560)
• Summary:

  Recent discoveries have revealed that deep neural networks might behave in a biased manner in many real-world scenarios. For instance, deep networks trained on a large-scale face recognition dataset CelebA tend to predict blonde hair for females and black hair for males. Such biases not only jeopardize the robustness of models but also perpetuate and amplify social biases, which is especially concerning for automated decision-making processes in healthcare, recruitment, etc., as they could exacerbate unfair economic and social inequalities among different groups. Existing debiasing methods suffer from high costs in bias labeling or model re-training, while also exhibiting a deficiency in terms of elucidating the origins of biases within the model. To this respect, we propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases inherent in trained models. The FMD identifies biased attributes through an explicit counterfactual concept and quantifies the influence of data samples with influence functions. Moreover, we design a machine unlearning-based strategy to efficiently and effectively remove the bias in a trained model with a small counterfactual dataset. Experiments on the Colored MNIST, CelebA, and Adult Income datasets along with experiments with large language models demonstrate that our method achieves superior or competing accuracies compared with state-of-the-art methods while attaining significantly fewer biases and requiring much less debiasing cost. Notably, our method requires only a small external dataset and updating a minimal amount of model parameters, without the requirement of access to training data that may be too large or unavailable in practice.

Title: Open-World Lifelong Graph Learning. (arXiv:2310.12565v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12565
• Code URL: https://github.com/bobowner/open-world-lgl
• Copy Paste: [[2310.12565]] Open-World Lifelong Graph Learning(http://arxiv.org/abs/2310.12565)
• Summary:

  We study the problem of lifelong graph learning in an open-world scenario, where a model needs to deal with new tasks and potentially unknown classes. We utilize Out-of-Distribution (OOD) detection methods to recognize new classes and adapt existing non-graph OOD detection methods to graph data. Crucially, we suggest performing new class detection by combining OOD detection methods with information aggregated from the graph neighborhood. Most OOD detection methods avoid determining a crisp threshold for deciding whether a vertex is OOD. To tackle this problem, we propose a Weakly-supervised Relevance Feedback (Open-WRF) method, which decreases the sensitivity to thresholds in OOD detection. We evaluate our approach on six benchmark datasets. Our results show that the proposed neighborhood aggregation method for OOD scores outperforms existing methods independent of the underlying graph neural network. Furthermore, we demonstrate that our Open-WRF method is more robust to threshold selection and analyze the influence of graph neighborhood on OOD detection. The aggregation and threshold methods are compatible with arbitrary graph neural networks and OOD detection methods, making our approach versatile and applicable to many real-world applications.

biometric

steal

Title: Towards Understanding and Characterizing the Arbitrage Bot Scam In the Wild. (arXiv:2310.12306v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12306
• Code URL: null
• Copy Paste: [[2310.12306]] Towards Understanding and Characterizing the Arbitrage Bot Scam In the Wild(http://arxiv.org/abs/2310.12306)
• Summary:

  This paper presents the first comprehensive analysis of an emerging cryptocurrency scam named "arbitrage bot" disseminated on online social networks. The scam revolves around Decentralized Exchanges (DEX) arbitrage and aims to lure victims into executing a so-called "bot contract" to steal funds from them.

  To collect the scam at a large scale, we developed a fully automated scam detection system named CryptoScamHunter, which continuously collects YouTube videos and automatically detects scams. Meanwhile, CryptoScamHunter can download the source code of the bot contract from the provided links and extract the associated scam cryptocurrency address. Through deploying CryptoScamHunter from Jun. 2022 to Jun. 2023, we have detected 10,442 arbitrage bot scam videos published from thousands of YouTube accounts. Our analysis reveals that different strategies have been utilized in spreading the scam, including crafting popular accounts, registering spam accounts, and using obfuscation tricks to hide the real scam address in the bot contracts. Moreover, from the scam videos we have collected over 800 malicious bot contracts with source code and extracted 354 scam addresses. By further expanding the scam addresses with a similar contract matching technique, we have obtained a total of 1,697 scam addresses. Through tracing the transactions of all scam addresses on the Ethereum mainnet and Binance Smart Chain, we reveal that over 25,000 victims have fallen prey to this scam, resulting in a financial loss of up to 15 million USD.

  Overall, our work sheds light on the dissemination tactics and censorship evasion strategies adopted in the arbitrage bot scam, as well as on the scale and impact of such a scam on online social networks and blockchain platforms, emphasizing the urgent need for effective detection and prevention mechanisms against such fraudulent activity.

extraction

Title: Weakly Supervised Learning for Breast Cancer Prediction on Mammograms in Realistic Settings. (arXiv:2310.12677v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12677
• Code URL: null
• Copy Paste: [[2310.12677]] Weakly Supervised Learning for Breast Cancer Prediction on Mammograms in Realistic Settings(http://arxiv.org/abs/2310.12677)
• Summary:

  Automatic methods for early detection of breast cancer on mammography can significantly decrease mortality. Broad uptake of those methods in hospitals is currently hindered because the methods have too many constraints. They assume annotations available for single images or even regions-of-interest (ROIs), and a fixed number of images per patient. Both assumptions do not hold in a general hospital setting. Relaxing those assumptions results in a weakly supervised learning setting, where labels are available per case, but not for individual images or ROIs. Not all images taken for a patient contain malignant regions and the malignant ROIs cover only a tiny part of an image, whereas most image regions represent benign tissue. In this work, we investigate a two-level multi-instance learning (MIL) approach for case-level breast cancer prediction on two public datasets (1.6k and 5k cases) and an in-house dataset of 21k cases. Observing that breast cancer is usually only present in one side, while images of both breasts are taken as a precaution, we propose a domain-specific MIL pooling variant. We show that two-level MIL can be applied in realistic clinical settings where only case labels, and a variable number of images per patient are available. Data in realistic settings scales with continuous patient intake, while manual annotation efforts do not. Hence, research should focus in particular on unsupervised ROI extraction, in order to improve breast cancer prediction for all patients.

Title: Product Attribute Value Extraction using Large Language Models. (arXiv:2310.12537v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12537
• Code URL: https://github.com/wbsg-uni-mannheim/extractgpt
• Copy Paste: [[2310.12537]] Product Attribute Value Extraction using Large Language Models(http://arxiv.org/abs/2310.12537)
• Summary:

  E-commerce applications such as faceted product search or product comparison are based on structured product descriptions like attribute/value pairs. The vendors on e-commerce platforms do not provide structured product descriptions but describe offers using titles or descriptions. To process such offers, it is necessary to extract attribute/value pairs from textual product attributes. State-of-the-art attribute/value extraction techniques rely on pre-trained language models (PLMs), such as BERT. Two major drawbacks of these models for attribute/value extraction are that (i) the models require significant amounts of task-specific training data and (ii) the fine-tuned models face challenges in generalizing to attribute values not included in the training data. This paper explores the potential of large language models (LLMs) as a training data-efficient and robust alternative to PLM-based attribute/value extraction methods. We consider hosted LLMs, such as GPT-3.5 and GPT-4, as well as open-source LLMs based on Llama2. We evaluate the models in a zero-shot scenario and in a scenario where task-specific training data is available. In the zero-shot scenario, we compare various prompt designs for representing information about the target attributes of the extraction. In the scenario with training data, we investigate (i) the provision of example attribute values, (ii) the selection of in-context demonstrations, and (iii) the fine-tuning of GPT-3.5. Our experiments show that GPT-4 achieves an average F1-score of 85% on the two evaluation datasets while the best PLM-based techniques perform on average 5% worse using the same amount of training data. GPT-4 achieves a 10% higher F1-score than the best open-source LLM. The fine-tuned GPT-3.5 model reaches a similar performance as GPT-4 while being significantly more cost-efficient.

Title: Time-Aware Representation Learning for Time-Sensitive Question Answering. (arXiv:2310.12585v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12585
• Code URL: https://github.com/sonjbin/tcqa
• Copy Paste: [[2310.12585]] Time-Aware Representation Learning for Time-Sensitive Question Answering(http://arxiv.org/abs/2310.12585)
• Summary:

  Time is one of the crucial factors in real-world question answering (QA) problems. However, language models have difficulty understanding the relationships between time specifiers, such as 'after' and 'before', and numbers, since existing QA datasets do not include sufficient time expressions. To address this issue, we propose a Time-Context aware Question Answering (TCQA) framework. We suggest a Time-Context dependent Span Extraction (TCSE) task, and build a time-context dependent data generation framework for model training. Moreover, we present a metric to evaluate the time awareness of the QA model using TCSE. The TCSE task consists of a question and four sentence candidates classified as correct or incorrect based on time and context. The model is trained to extract the answer span from the sentence that is both correct in time and context. The model trained with TCQA outperforms baseline models up to 8.5 of the F1-score in the TimeQA dataset. Our dataset and code are available at https://github.com/sonjbin/TCQA

membership infer

federate

Title: Equipping Federated Graph Neural Networks with Structure-aware Group Fairness. (arXiv:2310.12350v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12350
• Code URL: https://github.com/yuening-lab/f2gnn
• Copy Paste: [[2310.12350]] Equipping Federated Graph Neural Networks with Structure-aware Group Fairness(http://arxiv.org/abs/2310.12350)
• Summary:

  Graph Neural Networks (GNNs) have been widely used for various types of graph data processing and analytical tasks in different domains. Training GNNs over centralized graph data can be infeasible due to privacy concerns and regulatory restrictions. Thus, federated learning (FL) becomes a trending solution to address this challenge in a distributed learning paradigm. However, as GNNs may inherit historical bias from training data and lead to discriminatory predictions, the bias of local models can be easily propagated to the global model in distributed settings. This poses a new challenge in mitigating bias in federated GNNs. To address this challenge, we propose $\text{F}^2$GNN, a Fair Federated Graph Neural Network, that enhances group fairness of federated GNNs. As bias can be sourced from both data and learning algorithms, $\text{F}^2$GNN aims to mitigate both types of bias under federated settings. First, we provide theoretical insights on the connection between data bias in a training graph and statistical fairness metrics of the trained GNN models. Based on the theoretical analysis, we design $\text{F}^2$GNN which contains two key components: a fairness-aware local model update scheme that enhances group fairness of the local models on the client side, and a fairness-weighted global model update scheme that takes both data bias and fairness metrics of local models into consideration in the aggregation process. We evaluate $\text{F}^2$GNN empirically versus a number of baseline methods, and demonstrate that $\text{F}^2$GNN outperforms these baselines in terms of both fairness and model accuracy.

fair

Title: knn-seq: Efficient, Extensible kNN-MT Framework. (arXiv:2310.12352v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12352
• Code URL: https://github.com/naist-nlp/knn-seq
• Copy Paste: [[2310.12352]] knn-seq: Efficient, Extensible kNN-MT Framework(http://arxiv.org/abs/2310.12352)
• Summary:

  k-nearest-neighbor machine translation (kNN-MT) boosts the translation quality of a pre-trained neural machine translation (NMT) model by utilizing translation examples during decoding. Translation examples are stored in a vector database, called a datastore, which contains one entry for each target token from the parallel data it is made from. Due to its size, it is computationally expensive both to construct and to retrieve examples from the datastore. In this paper, we present an efficient and extensible kNN-MT framework, knn-seq, for researchers and developers that is carefully designed to run efficiently, even with a billion-scale large datastore. knn-seq is developed as a plug-in on fairseq and easy to switch models and kNN indexes. Experimental results show that our implemented kNN-MT achieves a comparable gain to the original kNN-MT, and the billion-scale datastore construction took 2.21 hours in the WMT'19 German-to-English translation task. We publish our knn-seq as an MIT-licensed open-source project and the code is available on https://github.com/naist-nlp/knn-seq . The demo video is available on https://youtu.be/zTDzEOq80m0 .

Title: Building Random, Fair, and Verifiable Games on Blockchain. Raffle smart contract designs on Sui Network. (arXiv:2310.12305v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12305
• Code URL: null
• Copy Paste: [[2310.12305]] Building Random, Fair, and Verifiable Games on Blockchain(http://arxiv.org/abs/2310.12305)
• Summary:

  Randomness plays a pivotal role in modern online gaming, but disputes have arisen over the accuracy of stated winning chances, resulting in legal issues and financial setbacks for gaming companies. Fortunately, blockchain-based games offer a solution to the transparency and fairness issue regarding randomness. Furthermore, emerging blockchain technology like Sui Network enhances the efficiency of smart contracts by eliminating traditional web3 barriers, such as inefficiencies and expensive transaction fees. This unlocks the potential for extensive decentralized gaming applications.

  This paper aims to provide insights into designing a fair, verifiable, and efficient smart contract game on blockchain by the example of building raffles on the Sui Network. We explore efficient methods for implementing randomness on smart contracts, including DRAND committee-based decentralized random beacons and single private-key-based verifiable random functions (VRF). Then, progress from basic to comprehensive smart contract design. We addressed limitations in developing blockchain games in general, such as data input and storage space constraints.

  We propose corresponding solutions, encompassing the utilization of Object Tables, Delegate Object Creation, and Zero-Knowledge Proofs (ZKP) to optimize storage and input efficiency. After testing our designs, we found that the transaction fees for DRAND beacons and private-key-based VRFs are similar. Moreover, Object Tables incur higher overall transaction fees, while the ZKP setup fee is cheap but becomes very expensive during the verification process. Moreover, we identified suitable designs for different application scenarios by comparing the pros and cons of different smart contract implementations. Our findings provide valuable guidance for future researchers and developers in building random, fair, and verifiable games with smart contracts.

Title: Toward Unbiased Multiple-Target Fuzzing with Path Diversity. (arXiv:2310.12419v1 [cs.CR])

• Paper URL: http://arxiv.org/abs/2310.12419
• Code URL: null
• Copy Paste: [[2310.12419]] Toward Unbiased Multiple-Target Fuzzing with Path Diversity(http://arxiv.org/abs/2310.12419)
• Summary:

  In this paper, we propose a novel directed fuzzing solution named AFLRun, which features target path-diversity metric and unbiased energy assignment. Firstly, we develop a new coverage metric by maintaining extra virgin map for each covered target to track the coverage status of seeds that hit the target. This approach enables the storage of waypoints into the corpus that hit a target through interesting path, thus enriching the path diversity for each target. Additionally, we propose a corpus-level energy assignment strategy that guarantees fairness for each target. AFLRun starts with uniform target weight and propagates this weight to seeds to get a desired seed weight distribution. By assigning energy to each seed in the corpus according to such desired distribution, a precise and unbiased energy assignment can be achieved.

  We built a prototype system and assessed its performance using a standard benchmark and several extensively fuzzed real-world applications. The evaluation results demonstrate that AFLRun outperforms state-of-the-art fuzzers in terms of vulnerability detection, both in quantity and speed. Moreover, AFLRun uncovers 29 previously unidentified vulnerabilities, including 8 CVEs, across four distinct programs.

Title: WeaveNet for Approximating Two-sided Matching Problems. (arXiv:2310.12515v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12515
• Code URL: null
• Copy Paste: [[2310.12515]] WeaveNet for Approximating Two-sided Matching Problems(http://arxiv.org/abs/2310.12515)
• Summary:

  Matching, a task to optimally assign limited resources under constraints, is a fundamental technology for society. The task potentially has various objectives, conditions, and constraints; however, the efficient neural network architecture for matching is underexplored. This paper proposes a novel graph neural network (GNN), \textit{WeaveNet}, designed for bipartite graphs. Since a bipartite graph is generally dense, general GNN architectures lose node-wise information by over-smoothing when deeply stacked. Such a phenomenon is undesirable for solving matching problems. WeaveNet avoids it by preserving edge-wise information while passing messages densely to reach a better solution. To evaluate the model, we approximated one of the \textit{strongly NP-hard} problems, \textit{fair stable matching}. Despite its inherent difficulties and the network's general purpose design, our model reached a comparative performance with state-of-the-art algorithms specially designed for stable matching for small numbers of agents.

Title: A Theoretical Approach to Characterize the Accuracy-Fairness Trade-off Pareto Frontier. (arXiv:2310.12785v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12785
• Code URL: null
• Copy Paste: [[2310.12785]] A Theoretical Approach to Characterize the Accuracy-Fairness Trade-off Pareto Frontier(http://arxiv.org/abs/2310.12785)
• Summary:

  While the accuracy-fairness trade-off has been frequently observed in the literature of fair machine learning, rigorous theoretical analyses have been scarce. To demystify this long-standing challenge, this work seeks to develop a theoretical framework by characterizing the shape of the accuracy-fairness trade-off Pareto frontier (FairFrontier), determined by a set of all optimal Pareto classifiers that no other classifiers can dominate. Specifically, we first demonstrate the existence of the trade-off in real-world scenarios and then propose four potential categories to characterize the important properties of the accuracy-fairness Pareto frontier. For each category, we identify the necessary conditions that lead to corresponding trade-offs. Experimental results on synthetic data suggest insightful findings of the proposed framework: (1) When sensitive attributes can be fully interpreted by non-sensitive attributes, FairFrontier is mostly continuous. (2) Accuracy can suffer a \textit{sharp} decline when over-pursuing fairness. (3) Eliminate the trade-off via a two-step streamlined approach. The proposed research enables an in-depth understanding of the accuracy-fairness trade-off, pushing current fair machine-learning research to a new frontier.

Title: Exploring Graph Neural Networks for Indian Legal Judgment Prediction. (arXiv:2310.12800v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12800
• Code URL: null
• Copy Paste: [[2310.12800]] Exploring Graph Neural Networks for Indian Legal Judgment Prediction(http://arxiv.org/abs/2310.12800)
• Summary:

  The burdensome impact of a skewed judges-to-cases ratio on the judicial system manifests in an overwhelming backlog of pending cases alongside an ongoing influx of new ones. To tackle this issue and expedite the judicial process, the proposition of an automated system capable of suggesting case outcomes based on factual evidence and precedent from past cases gains significance. This research paper centres on developing a graph neural network-based model to address the Legal Judgment Prediction (LJP) problem, recognizing the intrinsic graph structure of judicial cases and making it a binary node classification problem. We explored various embeddings as model features, while nodes such as time nodes and judicial acts were added and pruned to evaluate the model's performance. The study is done while considering the ethical dimension of fairness in these predictions, considering gender and name biases. A link prediction task is also conducted to assess the model's proficiency in anticipating connections between two specified nodes. By harnessing the capabilities of graph neural networks and incorporating fairness analyses, this research aims to contribute insights towards streamlining the adjudication process, enhancing judicial efficiency, and fostering a more equitable legal landscape, ultimately alleviating the strain imposed by mounting case backlogs. Our best-performing model with XLNet pre-trained embeddings as its features gives the macro F1 score of 75% for the LJP task. For link prediction, the same set of features is the best performing giving ROC of more than 80%

Title: Detection and Evaluation of bias-inducing Features in Machine learning. (arXiv:2310.12805v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12805
• Code URL: null
• Copy Paste: [[2310.12805]] Detection and Evaluation of bias-inducing Features in Machine learning(http://arxiv.org/abs/2310.12805)
• Summary:

  The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the individual(s). This implies that we can identify how the problems are inherited, rank the causes to help prioritize fixes, simplify a complex problem and visualize them. In the context of machine learning (ML), one can use cause-to-effect analysis to understand the reason for the biased behavior of the system. For example, we can examine the root causes of biases by checking each feature for a potential cause of bias in the model. To approach this, one can apply small changes to a given feature or a pair of features in the data, following some guidelines and observing how it impacts the decision made by the model (i.e., model prediction). Therefore, we can use cause-to-effect analysis to identify the potential bias-inducing features, even when these features are originally are unknown. This is important since most current methods require a pre-identification of sensitive features for bias assessment and can actually miss other relevant bias-inducing features, which is why systematic identification of such features is necessary. Moreover, it often occurs that to achieve an equitable outcome, one has to take into account sensitive features in the model decision. Therefore, it should be up to the domain experts to decide based on their knowledge of the context of a decision whether bias induced by specific features is acceptable or not. In this study, we propose an approach for systematically identifying all bias-inducing features of a model to help support the decision-making of domain experts. We evaluated our technique using four well-known datasets to showcase how our contribution can help spearhead the standard procedure when developing, testing, maintaining, and deploying fair/equitable machine learning systems.

interpretability

Title: Understanding Video Transformers for Segmentation: A Survey of Application and Interpretability. (arXiv:2310.12296v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12296
• Code URL: null
• Copy Paste: [[2310.12296]] Understanding Video Transformers for Segmentation: A Survey of Application and Interpretability(http://arxiv.org/abs/2310.12296)
• Summary:

  Video segmentation encompasses a wide range of categories of problem formulation, e.g., object, scene, actor-action and multimodal video segmentation, for delineating task-specific scene components with pixel-level masks. Recently, approaches in this research area shifted from concentrating on ConvNet-based to transformer-based models. In addition, various interpretability approaches have appeared for transformer models and video temporal dynamics, motivated by the growing interest in basic scientific understanding, model diagnostics and societal implications of real-world deployment. Previous surveys mainly focused on ConvNet models on a subset of video segmentation tasks or transformers for classification tasks. Moreover, component-wise discussion of transformer-based video segmentation models has not yet received due focus. In addition, previous reviews of interpretability methods focused on transformers for classification, while analysis of video temporal dynamics modelling capabilities of video models received less attention. In this survey, we address the above with a thorough discussion of various categories of video segmentation, a component-wise discussion of the state-of-the-art transformer-based models, and a review of related interpretability methods. We first present an introduction to the different video segmentation task categories, their objectives, specific challenges and benchmark datasets. Next, we provide a component-wise review of recent transformer-based models and document the state of the art on different video segmentation tasks. Subsequently, we discuss post-hoc and ante-hoc interpretability methods for transformer models and interpretability methods for understanding the role of the temporal dimension in video models. Finally, we conclude our discussion with future research directions.

Title: Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models. (arXiv:2310.12454v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12454
• Code URL: null
• Copy Paste: [[2310.12454]] Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models(http://arxiv.org/abs/2310.12454)
• Summary:

  Pretrained language models are expected to effectively map input text to a set of vectors while preserving the inherent relationships within the text. Consequently, designing a white-box model to compute metrics that reflect the presence of specific internal relations in these vectors has become a common approach for post-hoc interpretability analysis of pretrained language models. However, achieving interpretability in white-box models and ensuring the rigor of metric computation becomes challenging when the source model lacks inherent interpretability. Therefore, in this paper, we discuss striking a balance in this trade-off and propose a novel line to constructing metrics for understanding the mechanisms of pretrained language models. We have specifically designed a family of metrics along this line of investigation, and the model used to compute these metrics is referred to as the tree topological probe. We conducted measurements on BERT-large by using these metrics. Based on the experimental results, we propose a speculation regarding the working mechanism of BERT-like pretrained language models, as well as a strategy for enhancing fine-tuning performance by leveraging the topological probe to improve specific submodules.

Title: Character-level Chinese Backpack Language Models. (arXiv:2310.12751v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12751
• Code URL: null
• Copy Paste: [[2310.12751]] Character-level Chinese Backpack Language Models(http://arxiv.org/abs/2310.12751)
• Summary:

  The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components. However, Backpacks' reliance on token-defined meaning raises questions as to their potential for languages other than English, a language for which subword tokenization provides a reasonable approximation for lexical items. In this work, we train, evaluate, interpret, and control Backpack language models in character-tokenized Chinese, in which words are often composed of many characters. We find that our (134M parameter) Chinese Backpack language model performs comparably to a (104M parameter) Transformer, and learns rich character-level meanings that log-additively compose to form word meanings. In SimLex-style lexical semantic evaluations, simple averages of Backpack character senses outperform input embeddings from a Transformer. We find that complex multi-character meanings are often formed by using the same per-character sense weights consistently across context. Exploring interpretability-through control, we show that we can localize a source of gender bias in our Backpacks to specific character senses and intervene to reduce the bias.

Title: MuseGNN: Interpretable and Convergent Graph Neural Network Layers at Scale. (arXiv:2310.12457v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12457
• Code URL: null
• Copy Paste: [[2310.12457]] MuseGNN: Interpretable and Convergent Graph Neural Network Layers at Scale(http://arxiv.org/abs/2310.12457)
• Summary:

  Among the many variants of graph neural network (GNN) architectures capable of modeling data with cross-instance relations, an important subclass involves layers designed such that the forward pass iteratively reduces a graph-regularized energy function of interest. In this way, node embeddings produced at the output layer dually serve as both predictive features for solving downstream tasks (e.g., node classification) and energy function minimizers that inherit desirable inductive biases and interpretability. However, scaling GNN architectures constructed in this way remains challenging, in part because the convergence of the forward pass may involve models with considerable depth. To tackle this limitation, we propose a sampling-based energy function and scalable GNN layers that iteratively reduce it, guided by convergence guarantees in certain settings. We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size.

explainability

Title: Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling. (arXiv:2310.12421v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12421
• Code URL: null
• Copy Paste: [[2310.12421]] Detecting and Mitigating Algorithmic Bias in Binary Classification using Causal Modeling(http://arxiv.org/abs/2310.12421)
• Summary:

  This paper proposes the use of causal modeling to detect and mitigate algorithmic bias. We provide a brief description of causal modeling and a general overview of our approach. We then use the Adult dataset, which is available for download from the UC Irvine Machine Learning Repository, to develop (1) a prediction model, which is treated as a black box, and (2) a causal model for bias mitigation. In this paper, we focus on gender bias and the problem of binary classification. We show that gender bias in the prediction model is statistically significant at the 0.05 level. We demonstrate the effectiveness of the causal model in mitigating gender bias by cross-validation. Furthermore, we show that the overall classification accuracy is improved slightly. Our novel approach is intuitive, easy-to-use, and can be implemented using existing statistical software tools such as "lavaan" in R. Hence, it enhances explainability and promotes trust.

watermark

diffusion

Title: DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors. (arXiv:2310.12190v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12190
• Code URL: https://github.com/ailab-cvc/videocrafter
• Copy Paste: [[2310.12190]] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors(http://arxiv.org/abs/2310.12190)
• Summary:

  Enhancing a still image with motion offers more engaged visual experience. Traditional image animation techniques mainly focus on animating natural scenes with random dynamics, such as clouds and fluid, and thus limits their applicability to generic visual contents. To overcome this limitation, we explore the synthesis of dynamic content for open-domain images, converting them into animated videos. The key idea is to utilize the motion prior of text-to-video diffusion models by incorporating the image into the generative process as guidance. Given an image, we first project it into a text-aligned rich image embedding space using a learnable image encoding network, which facilitates the video model to digest the image content compatibly. However, some visual details still struggle to be preserved in the resulting videos. To supplement more precise image information, we further feed the full image to the diffusion model by concatenating it with the initial noises. Experimental results reveal that our proposed method produces visually convincing animated videos, exhibiting both natural motions and high fidelity to the input image. Comparative evaluation demonstrates the notable superiority of our approach over existing competitors. The source code will be released upon publication.

Title: Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping. (arXiv:2310.12474v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12474
• Code URL: https://github.com/fudan-zvg/pgc-3d
• Copy Paste: [[2310.12474]] Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping(http://arxiv.org/abs/2310.12474)
• Summary:

  High-resolution 3D object generation remains a challenging task primarily due to the limited availability of comprehensive annotated training data. Recent advancements have aimed to overcome this constraint by harnessing image generative models, pretrained on extensive curated web datasets, using knowledge transfer techniques like Score Distillation Sampling (SDS). Efficiently addressing the requirements of high-resolution rendering often necessitates the adoption of latent representation-based models, such as the Latent Diffusion Model (LDM). In this framework, a significant challenge arises: To compute gradients for individual image pixels, it is necessary to backpropagate gradients from the designated latent space through the frozen components of the image model, such as the VAE encoder used within LDM. However, this gradient propagation pathway has never been optimized, remaining uncontrolled during training. We find that the unregulated gradients adversely affect the 3D model's capacity in acquiring texture-related information from the image generative model, leading to poor quality appearance synthesis. To address this overarching challenge, we propose an innovative operation termed Pixel-wise Gradient Clipping (PGC) designed for seamless integration into existing 3D generative models, thereby enhancing their synthesis quality. Specifically, we control the magnitude of stochastic gradients by clipping the pixel-wise gradients efficiently, while preserving crucial texture-related gradient directions. Despite this simplicity and minimal extra cost, extensive experiments demonstrate the efficacy of our PGC in enhancing the performance of existing 3D generative models for high-resolution object rendering.

Title: Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation. (arXiv:2310.12583v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12583
• Code URL: null
• Copy Paste: [[2310.12583]] Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation(http://arxiv.org/abs/2310.12583)
• Summary:

  Latent diffusion models excel at producing high-quality images from text. Yet, concerns appear about the lack of diversity in the generated imagery. To tackle this, we introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity, spanning into richer realms, including color diversity.Diverse Diffusion is a general unsupervised technique that can be applied to existing text-to-image models. Our approach focuses on finding vectors in the Stable Diffusion latent space that are distant from each other. We generate multiple vectors in the latent space until we find a set of vectors that meets the desired distance requirements and the required batch size.To evaluate the effectiveness of our diversity methods, we conduct experiments examining various characteristics, including color diversity, LPIPS metric, and ethnicity/gender representation in images featuring humans.The results of our experiments emphasize the significance of diversity in generating realistic and varied images, offering valuable insights for improving text-to-image models. Through the enhancement of image diversity, our approach contributes to the creation of more inclusive and representative AI-generated art.

Title: EMIT-Diff: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model. (arXiv:2310.12868v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12868
• Code URL: null
• Copy Paste: [[2310.12868]] EMIT-Diff: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model(http://arxiv.org/abs/2310.12868)
• Summary:

  Large-scale, big-variant, and high-quality data are crucial for developing robust and successful deep-learning models for medical applications since they potentially enable better generalization performance and avoid overfitting. However, the scarcity of high-quality labeled data always presents significant challenges. This paper proposes a novel approach to address this challenge by developing controllable diffusion models for medical image synthesis, called EMIT-Diff. We leverage recent diffusion probabilistic models to generate realistic and diverse synthetic medical image data that preserve the essential characteristics of the original medical images by incorporating edge information of objects to guide the synthesis process. In our approach, we ensure that the synthesized samples adhere to medically relevant constraints and preserve the underlying structure of imaging data. Due to the random sampling process by the diffusion model, we can generate an arbitrary number of synthetic images with diverse appearances. To validate the effectiveness of our proposed method, we conduct an extensive set of medical image segmentation experiments on multiple datasets, including Ultrasound breast (+13.87%), CT spleen (+0.38%), and MRI prostate (+7.78%), achieving significant improvements over the baseline segmentation methods. For the first time, to our best knowledge, the promising results demonstrate the effectiveness of our EMIT-Diff for medical image segmentation tasks and show the feasibility of introducing a first-ever text-guided diffusion model for general medical image segmentation tasks. With carefully designed ablation experiments, we investigate the influence of various data augmentation ratios, hyper-parameter settings, patch size for generating random merging mask settings, and combined influence with different network architectures.

Title: Closed-Form Diffusion Models. (arXiv:2310.12395v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12395
• Code URL: null
• Copy Paste: [[2310.12395]] Closed-Form Diffusion Models(http://arxiv.org/abs/2310.12395)
• Summary:

  Score-based generative models (SGMs) sample from a target distribution by iteratively transforming noise using the score function of the perturbed target. For any finite training set, this score function can be evaluated in closed form, but the resulting SGM memorizes its training data and does not generate novel samples. In practice, one approximates the score by training a neural network via score-matching. The error in this approximation promotes generalization, but neural SGMs are costly to train and sample, and the effective regularization this error provides is not well-understood theoretically. In this work, we instead explicitly smooth the closed-form score to obtain an SGM that generates novel samples without training. We analyze our model and propose an efficient nearest-neighbor-based estimator of its score function. Using this estimator, our method achieves sampling times competitive with neural SGMs while running on consumer-grade CPUs.

Title: SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation. (arXiv:2310.12508v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12508
• Code URL: https://github.com/optml-group/unlearn-saliency
• Copy Paste: [[2310.12508]] SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation(http://arxiv.org/abs/2310.12508)
• Summary:

  With evolving data regulations, machine unlearning (MU) has become an important tool for fostering trust and safety in today's AI models. However, existing MU methods focusing on data and/or weight perspectives often grapple with limitations in unlearning accuracy, stability, and cross-domain applicability. To address these challenges, we introduce the concept of 'weight saliency' in MU, drawing parallels with input saliency in model explanation. This innovation directs MU's attention toward specific model weights rather than the entire model, improving effectiveness and efficiency. The resultant method that we call saliency unlearning (SalUn) narrows the performance gap with 'exact' unlearning (model retraining from scratch after removing the forgetting dataset). To the best of our knowledge, SalUn is the first principled MU approach adaptable enough to effectively erase the influence of forgetting data, classes, or concepts in both image classification and generation. For example, SalUn yields a stability advantage in high-variance random data forgetting, e.g., with a 0.2% gap compared to exact unlearning on the CIFAR-10 dataset. Moreover, in preventing conditional diffusion models from generating harmful images, SalUn achieves nearly 100% unlearning accuracy, outperforming current state-of-the-art baselines like Erased Stable Diffusion and Forget-Me-Not.

noise learning

data-free

transformer

Title: LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following. (arXiv:2310.12344v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12344
• Code URL: https://github.com/joeyy5588/lacma
• Copy Paste: [[2310.12344]] LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following(http://arxiv.org/abs/2310.12344)
• Summary:

  End-to-end Transformers have demonstrated an impressive success rate for Embodied Instruction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environment. This lack of generalizability is due to the agent's insensitivity to subtle changes in natural language instructions. To mitigate this issue, we propose explicitly aligning the agent's hidden states with the instructions via contrastive learning. Nevertheless, the semantic gap between high-level language instructions and the agent's low-level action space remains an obstacle. Therefore, we further introduce a novel concept of meta-actions to bridge the gap. Meta-actions are ubiquitous action patterns that can be parsed from the original action sequence. These patterns represent higher-level semantics that are intuitively aligned closer to the instructions. When meta-actions are applied as additional training signals, the agent generalizes better to unseen environments. Compared to a strong multi-modal Transformer baseline, we achieve a significant 4.5% absolute gain in success rate in unseen environments of ALFRED Embodied Instruction Following. Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents. The code is available at: https://github.com/joeyy5588/LACMA.

Title: Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps. (arXiv:2310.12616v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12616
• Code URL: null
• Copy Paste: [[2310.12616]] Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps(http://arxiv.org/abs/2310.12616)
• Summary:

  Historical maps provide useful spatio-temporal information on the Earth's surface before modern earth observation techniques came into being. To extract information from maps, neural networks, which gain wide popularity in recent years, have replaced hand-crafted map processing methods and tedious manual labor. However, aleatoric uncertainty, known as data-dependent uncertainty, inherent in the drawing/scanning/fading defects of the original map sheets and inadequate contexts when cropping maps into small tiles considering the memory limits of the training process, challenges the model to make correct predictions. As aleatoric uncertainty cannot be reduced even with more training data collected, we argue that complementary spatio-temporal contexts can be helpful. To achieve this, we propose a U-Net-based network that fuses spatio-temporal features with cross-attention transformers (U-SpaTem), aggregating information at a larger spatial range as well as through a temporal sequence of images. Our model achieves a better performance than other state-or-art models that use either temporal or spatial contexts. Compared with pure vision transformers, our model is more lightweight and effective. To the best of our knowledge, leveraging both spatial and temporal contexts have been rarely explored before in the segmentation task. Even though our application is on segmenting historical maps, we believe that the method can be transferred into other fields with similar problems like temporal sequences of satellite images. Our code is freely accessible at https://github.com/chenyizi086/wu.2023.sigspatial.git.

Title: Heart Disease Detection using Vision-Based Transformer Models from ECG Images. (arXiv:2310.12630v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12630
• Code URL: null
• Copy Paste: [[2310.12630]] Heart Disease Detection using Vision-Based Transformer Models from ECG Images(http://arxiv.org/abs/2310.12630)
• Summary:

  Heart disease, also known as cardiovascular disease, is a prevalent and critical medical condition characterized by the impairment of the heart and blood vessels, leading to various complications such as coronary artery disease, heart failure, and myocardial infarction. The timely and accurate detection of heart disease is of paramount importance in clinical practice. Early identification of individuals at risk enables proactive interventions, preventive measures, and personalized treatment strategies to mitigate the progression of the disease and reduce adverse outcomes. In recent years, the field of heart disease detection has witnessed notable advancements due to the integration of sophisticated technologies and computational approaches. These include machine learning algorithms, data mining techniques, and predictive modeling frameworks that leverage vast amounts of clinical and physiological data to improve diagnostic accuracy and risk stratification. In this work, we propose to detect heart disease from ECG images using cutting-edge technologies, namely vision transformer models. These models are Google-Vit, Microsoft-Beit, and Swin-Tiny. To the best of our knowledge, this is the initial endeavor concentrating on the detection of heart diseases through image-based ECG data by employing cuttingedge technologies namely, transformer models. To demonstrate the contribution of the proposed framework, the performance of vision transformer models are compared with state-of-the-art studies. Experiment results show that the proposed framework exhibits remarkable classification results.

Title: Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers. (arXiv:2310.12755v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12755
• Code URL: null
• Copy Paste: [[2310.12755]] Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers(http://arxiv.org/abs/2310.12755)
• Summary:

  In the wake of Masked Image Modeling (MIM), a diverse range of plain, non-hierarchical Vision Transformer (ViT) models have been pre-trained with extensive datasets, offering new paradigms and significant potential for semantic segmentation. Current state-of-the-art systems incorporate numerous inductive biases and employ cumbersome decoders. Building upon the original motivations of plain ViTs, which are simplicity and generality, we explore high-performance `minimalist' systems to this end. Our primary purpose is to provide simple and efficient baselines for practical semantic segmentation with plain ViTs. Specifically, we first explore the feasibility and methodology for achieving high-performance semantic segmentation using the last feature map. As a result, we introduce the PlainSeg, a model comprising only three 3$\times$3 convolutions in addition to the transformer layers (either encoder or decoder). In this process, we offer insights into two underlying principles: (i) high-resolution features are crucial to high performance in spite of employing simple up-sampling techniques and (ii) the slim transformer decoder requires a much larger learning rate than the wide transformer decoder. On this basis, we further present the PlainSeg-Hier, which allows for the utilization of hierarchical features. Extensive experiments on four popular benchmarks demonstrate the high performance and efficiency of our methods. They can also serve as powerful tools for assessing the transfer ability of base models in semantic segmentation. Code is available at \url{https://github.com/ydhongHIT/PlainSeg}.

Title: 2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision. (arXiv:2310.12817v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12817
• Code URL: null
• Copy Paste: [[2310.12817]] 2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision(http://arxiv.org/abs/2310.12817)
• Summary:

  We present a Multimodal Interlaced Transformer (MIT) that jointly considers 2D and 3D data for weakly supervised point cloud segmentation. Research studies have shown that 2D and 3D features are complementary for point cloud segmentation. However, existing methods require extra 2D annotations to achieve 2D-3D information fusion. Considering the high annotation cost of point clouds, effective 2D and 3D feature fusion based on weakly supervised learning is in great demand. To this end, we propose a transformer model with two encoders and one decoder for weakly supervised point cloud segmentation using only scene-level class tags. Specifically, the two encoders compute the self-attended features for 3D point clouds and 2D multi-view images, respectively. The decoder implements interlaced 2D-3D cross-attention and carries out implicit 2D and 3D feature fusion. We alternately switch the roles of queries and key-value pairs in the decoder layers. It turns out that the 2D and 3D features are iteratively enriched by each other. Experiments show that it performs favorably against existing weakly supervised point cloud segmentation methods by a large margin on the S3DIS and ScanNet benchmarks. The project page will be available at https://jimmy15923.github.io/mit_web/.

Title: Direct Neural Machine Translation with Task-level Mixture of Experts models. (arXiv:2310.12236v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12236
• Code URL: null
• Copy Paste: [[2310.12236]] Direct Neural Machine Translation with Task-level Mixture of Experts models(http://arxiv.org/abs/2310.12236)
• Summary:

  Direct neural machine translation (direct NMT) is a type of NMT system that translates text between two non-English languages. Direct NMT systems often face limitations due to the scarcity of parallel data between non-English language pairs. Several approaches have been proposed to address this limitation, such as multilingual NMT and pivot NMT (translation between two languages via English). Task-level Mixture of expert models (Task-level MoE), an inference-efficient variation of Transformer-based models, has shown promising NMT performance for a large number of language pairs. In Task-level MoE, different language groups can use different routing strategies to optimize cross-lingual learning and inference speed. In this work, we examine Task-level MoE's applicability in direct NMT and propose a series of high-performing training and evaluation configurations, through which Task-level MoE-based direct NMT systems outperform bilingual and pivot-based models for a large number of low and high-resource direct pairs, and translation directions. Our Task-level MoE with 16 experts outperforms bilingual NMT, Pivot NMT models for 7 language pairs, while pivot-based models still performed better in 9 pairs and directions.

Title: Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer. (arXiv:2310.12442v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12442
• Code URL: null
• Copy Paste: [[2310.12442]] Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer(http://arxiv.org/abs/2310.12442)
• Summary:

  Pretrained transformer models have demonstrated remarkable performance across various natural language processing tasks. These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence. However, the (full) attention mechanism incurs high computational cost - quadratic in the sequence length, which is not affordable in tasks with long sequences, e.g., inputs with 8k tokens. Although sparse attention can be used to improve computational efficiency, as suggested in existing work, it has limited modeling capacity and often fails to capture complicated dependencies in long sequences. To tackle this challenge, we propose MASFormer, an easy-to-implement transformer variant with Mixed Attention Spans. Specifically, MASFormer is equipped with full attention to capture long-range dependencies, but only at a small number of layers. For the remaining layers, MASformer only employs sparse attention to capture short-range dependencies. Our experiments on natural language modeling and generation tasks show that a decoder-only MASFormer model of 1.3B parameters can achieve competitive performance to vanilla transformers with full attention while significantly reducing computational cost (up to 75%). Additionally, we investigate the effectiveness of continual training with long sequence data and how sequence length impacts downstream generation performance, which may be of independent interest.

Title: Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights. (arXiv:2310.12462v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12462
• Code URL: null
• Copy Paste: [[2310.12462]] Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights(http://arxiv.org/abs/2310.12462)
• Summary:

  In the realm of deep learning, transformers have emerged as a dominant architecture, particularly in natural language processing tasks. However, with their widespread adoption, concerns regarding the security and privacy of the data processed by these models have arisen. In this paper, we address a pivotal question: Can the data fed into transformers be recovered using their attention weights and outputs? We introduce a theoretical framework to tackle this problem. Specifically, we present an algorithm that aims to recover the input data $X \in \mathbb{R}^{d \times n}$ from given attention weights $W = QK^\top \in \mathbb{R}^{d \times d}$ and output $B \in \mathbb{R}^{n \times n}$ by minimizing the loss function $L(X)$. This loss function captures the discrepancy between the expected output and the actual output of the transformer. Our findings have significant implications for the Localized Layer-wise Mechanism (LLM), suggesting potential vulnerabilities in the model's design from a security and privacy perspective. This work underscores the importance of understanding and safeguarding the internal workings of transformers to ensure the confidentiality of processed data.

Title: Multilingual estimation of political-party positioning: From label aggregation to long-input Transformers. (arXiv:2310.12575v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12575
• Code URL: https://github.com/macleginn/party-positioning-code
• Copy Paste: [[2310.12575]] Multilingual estimation of political-party positioning: From label aggregation to long-input Transformers(http://arxiv.org/abs/2310.12575)
• Summary:

  Scaling analysis is a technique in computational political science that assigns a political actor (e.g. politician or party) a score on a predefined scale based on a (typically long) body of text (e.g. a parliamentary speech or an election manifesto). For example, political scientists have often used the left--right scale to systematically analyse political landscapes of different countries. NLP methods for automatic scaling analysis can find broad application provided they (i) are able to deal with long texts and (ii) work robustly across domains and languages. In this work, we implement and compare two approaches to automatic scaling analysis of political-party manifestos: label aggregation, a pipeline strategy relying on annotations of individual statements from the manifestos, and long-input-Transformer-based models, which compute scaling values directly from raw text. We carry out the analysis of the Comparative Manifestos Project dataset across 41 countries and 27 languages and find that the task can be efficiently solved by state-of-the-art models, with label aggregation producing the best results.

Title: Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model. (arXiv:2310.12611v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12611
• Code URL: null
• Copy Paste: [[2310.12611]] Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model(http://arxiv.org/abs/2310.12611)
• Summary:

  Language models (LMs) exhibit and amplify many types of undesirable biases learned from the training data, including gender bias. However, we lack tools for effectively and efficiently changing this behavior without hurting general language modeling performance. In this paper, we study three methods for identifying causal relations between LM components and particular output: causal mediation analysis, automated circuit discovery and our novel, efficient method called DiffMask+ based on differential masking. We apply the methods to GPT-2 small and the problem of gender bias, and use the discovered sets of components to perform parameter-efficient fine-tuning for bias mitigation. Our results show significant overlap in the identified components (despite huge differences in the computational requirements of the methods) as well as success in mitigating gender bias, with less damage to general language modeling compared to full model fine-tuning. However, our work also underscores the difficulty of defining and measuring bias, and the sensitivity of causal discovery procedures to dataset choice. We hope our work can contribute to more attention for dataset development, and lead to more effective mitigation strategies for other types of bias.

Title: Non-Autoregressive Sentence Ordering. (arXiv:2310.12640v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12640
• Code URL: https://github.com/steven640pixel/nonautoregressive-sentence-ordering
• Copy Paste: [[2310.12640]] Non-Autoregressive Sentence Ordering(http://arxiv.org/abs/2310.12640)
• Summary:

  Existing sentence ordering approaches generally employ encoder-decoder frameworks with the pointer net to recover the coherence by recurrently predicting each sentence step-by-step. Such an autoregressive manner only leverages unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences for ordering. To overcome these limitations, in this paper, we propose a novel Non-Autoregressive Ordering Network, dubbed \textit{NAON}, which explores bilateral dependencies between sentences and predicts the sentence for each position in parallel. We claim that the non-autoregressive manner is not just applicable but also particularly suitable to the sentence ordering task because of two peculiar characteristics of the task: 1) each generation target is in deterministic length, and 2) the sentences and positions should match exclusively. Furthermore, to address the repetition issue of the naive non-autoregressive Transformer, we introduce an exclusive loss to constrain the exclusiveness between positions and sentences. To verify the effectiveness of the proposed model, we conduct extensive experiments on several common-used datasets and the experimental results show that our method outperforms all the autoregressive approaches and yields competitive performance compared with the state-of-the-arts. The codes are available at: \url{https://github.com/steven640pixel/nonautoregressive-sentence-ordering}.

Title: Transformer-based Entity Legal Form Classification. (arXiv:2310.12766v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12766
• Code URL: null
• Copy Paste: [[2310.12766]] Transformer-based Entity Legal Form Classification(http://arxiv.org/abs/2310.12766)
• Summary:

  We propose the application of Transformer-based language models for classifying entity legal forms from raw legal entity names. Specifically, we employ various BERT variants and compare their performance against multiple traditional baselines. Our evaluation encompasses a substantial subset of freely available Legal Entity Identifier (LEI) data, comprising over 1.1 million legal entities from 30 different legal jurisdictions. The ground truth labels for classification per jurisdiction are taken from the Entity Legal Form (ELF) code standard (ISO 20275). Our findings demonstrate that pre-trained BERT variants outperform traditional text classification approaches in terms of F1 score, while also performing comparably well in the Macro F1 Score. Moreover, the validity of our proposal is supported by the outcome of third-party expert reviews conducted in ten selected jurisdictions. This study highlights the significant potential of Transformer-based models in advancing data standardization and data integration. The presented approaches can greatly benefit financial institutions, corporations, governments and other organizations in assessing business relationships, understanding risk exposure, and promoting effective governance.

Title: Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization. (arXiv:2310.12794v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12794
• Code URL: https://github.com/ningyuxu/structural_concepts_correspondence
• Copy Paste: [[2310.12794]] Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization(http://arxiv.org/abs/2310.12794)
• Summary:

  Large language models (LLMs) have exhibited considerable cross-lingual generalization abilities, whereby they implicitly transfer knowledge across languages. However, the transfer is not equally successful for all languages, especially for low-resource ones, which poses an ongoing challenge. It is unclear whether we have reached the limits of implicit cross-lingual generalization and if explicit knowledge transfer is viable. In this paper, we investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability among the spaces of structural concepts within each language for both encoder-only and decoder-only LLMs. We then propose a meta-learning-based method to learn to align conceptual spaces of different languages, which facilitates zero-shot and few-shot generalization in concept classification and also offers insights into the cross-lingual in-context learning phenomenon. Experiments on syntactic analysis tasks show that our approach achieves competitive results with state-of-the-art methods and narrows the performance gap between languages, particularly benefiting those with limited resources.

Title: The Locality and Symmetry of Positional Encodings. (arXiv:2310.12864v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12864
• Code URL: https://github.com/tigerchen52/locality_symmetry
• Copy Paste: [[2310.12864]] The Locality and Symmetry of Positional Encodings(http://arxiv.org/abs/2310.12864)
• Summary:

  Positional Encodings (PEs) are used to inject word-order information into transformer-based language models. While they can significantly enhance the quality of sentence representations, their specific contribution to language models is not fully understood, especially given recent findings that various positional encodings are insensitive to word order. In this work, we conduct a systematic study of positional encodings in \textbf{Bidirectional Masked Language Models} (BERT-style) , which complements existing work in three aspects: (1) We uncover the core function of PEs by identifying two common properties, Locality and Symmetry; (2) We show that the two properties are closely correlated with the performances of downstream tasks; (3) We quantify the weakness of current PEs by introducing two new probing tasks, on which current PEs perform poorly. We believe that these results are the basis for developing better PEs for transformer-based language models. The code is available at \faGithub~ \url{https://github.com/tigerchen52/locality\_symmetry}

Title: Learning to Solve Climate Sensor Placement Problems with a Transformer. (arXiv:2310.12387v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12387
• Code URL: null
• Copy Paste: [[2310.12387]] Learning to Solve Climate Sensor Placement Problems with a Transformer(http://arxiv.org/abs/2310.12387)
• Summary:

  The optimal placement of sensors for environmental monitoring and disaster management is a challenging problem due to its NP-hard nature. Traditional methods for sensor placement involve exact, approximation, or heuristic approaches, with the latter being the most widely used. However, heuristic methods are limited by expert intuition and experience. Deep learning (DL) has emerged as a promising approach for generating heuristic algorithms automatically. In this paper, we introduce a novel sensor placement approach focused on learning improvement heuristics using deep reinforcement learning (RL) methods. Our approach leverages an RL formulation for learning improvement heuristics, driven by an actor-critic algorithm for training the policy network. We compare our method with several state-of-the-art approaches by conducting comprehensive experiments, demonstrating the effectiveness and superiority of our proposed approach in producing high-quality solutions. Our work presents a promising direction for applying advanced DL and RL techniques to challenging climate sensor placement problems.

Title: On the Optimization and Generalization of Multi-head Attention. (arXiv:2310.12680v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12680
• Code URL: null
• Copy Paste: [[2310.12680]] On the Optimization and Generalization of Multi-head Attention(http://arxiv.org/abs/2310.12680)
• Summary:

  The training and generalization dynamics of the Transformer's core mechanism, namely the Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on single-head attention. Inspired by the demonstrated benefits of overparameterization when training fully-connected networks, we investigate the potential optimization and generalization advantages of using multiple attention heads. Towards this goal, we derive convergence and generalization guarantees for gradient-descent training of a single-layer multi-head self-attention model, under a suitable realizability condition on the data. We then establish primitive conditions on the initialization that ensure realizability holds. Finally, we demonstrate that these conditions are satisfied for a simple tokenized-mixture model. We expect the analysis can be extended to various data-model and architecture variations.

generative

Title: Improving SCGAN's Similarity Constraint and Learning a Better Disentangled Representation. (arXiv:2310.12262v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12262
• Code URL: null
• Copy Paste: [[2310.12262]] Improving SCGAN's Similarity Constraint and Learning a Better Disentangled Representation(http://arxiv.org/abs/2310.12262)
• Summary:

  SCGAN adds a similarity constraint between generated images and conditions as a regularization term on generative adversarial networks. Similarity constraint works as a tutor to instruct the generator network to comprehend the difference of representations based on conditions. We understand how SCGAN works on a deeper level. This understanding makes us realize that the similarity constraint functions like the contrastive loss function. We believe that a model with high understanding and intelligence measures the similarity between images based on their structure and high level features, just like humans do. Two major changes we applied to SCGAN in order to make a modified model are using SSIM to measure similarity between images and applying contrastive loss principles to the similarity constraint. The modified model performs better using FID and FactorVAE metrics. The modified model also has better generalisability compared to other models. Keywords Generative Adversarial Nets, Unsupervised Learning, Disentangled Representation Learning, Contrastive Disentanglement, SSIM

Title: Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling. (arXiv:2310.12902v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12902
• Code URL: null
• Copy Paste: [[2310.12902]] Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling(http://arxiv.org/abs/2310.12902)
• Summary:

  The paper proposes a framework that combines behavioral and computational experiments employing fictional prompts as a novel tool for investigating cultural artifacts and social biases in storytelling both by humans and generative AI. The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging methods from narratology and inferential statistics. Both crowdworkers and large language models responded to identical prompts about creating and falling in love with an artificial human. The proposed experimental paradigm allows a direct comparison between human and LLM-generated storytelling. Responses to the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth in the collective imaginary of both humans and large language models. All solicited narratives present a scientific or technological pursuit. The analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more more progressive in terms of gender roles and sexuality than those written by humans. While AI narratives can occasionally provide innovative plot twists, they offer less imaginative scenarios and rhetoric than human-authored texts. The proposed framework argues that fiction can be used as a window into human and AI-based collective imaginary and social dimensions.

Title: Open-Set Multivariate Time-Series Anomaly Detection. (arXiv:2310.12294v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12294
• Code URL: null
• Copy Paste: [[2310.12294]] Open-Set Multivariate Time-Series Anomaly Detection(http://arxiv.org/abs/2310.12294)
• Summary:

  Numerous methods for time series anomaly detection (TSAD) methods have emerged in recent years. Most existing methods are unsupervised and assume the availability of normal training samples only, while few supervised methods have shown superior performance by incorporating labeled anomalous samples in the training phase. However, certain anomaly types are inherently challenging for unsupervised methods to differentiate from normal data, while supervised methods are constrained to detecting anomalies resembling those present during training, failing to generalize to unseen anomaly classes. This paper is the first attempt in providing a novel approach for the open-set TSAD problem, in which a small number of labeled anomalies from a limited class of anomalies are visible in the training phase, with the objective of detecting both seen and unseen anomaly classes in the test phase. The proposed method, called Multivariate Open-Set timeseries Anomaly Detection (MOSAD) consists of three primary modules: a Feature Extractor to extract meaningful time-series features; a Multi-head Network consisting of Generative-, Deviation-, and Contrastive heads for capturing both seen and unseen anomaly classes; and an Anomaly Scoring module leveraging the insights of the three heads to detect anomalies. Extensive experiments on three real-world datasets consistently show that our approach surpasses existing methods under various experimental settings, thus establishing a new state-of-the-art performance in the TSAD field.

Title: Knowledge from Uncertainty in Evidential Deep Learning. (arXiv:2310.12663v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12663
• Code URL: null
• Copy Paste: [[2310.12663]] Knowledge from Uncertainty in Evidential Deep Learning(http://arxiv.org/abs/2310.12663)
• Summary:

  This work reveals an evidential signal that emerges from the uncertainty value in Evidential Deep Learning (EDL). EDL is one example of a class of uncertainty-aware deep learning approaches designed to provide confidence (or epistemic uncertainty) about the current test sample. In particular for computer vision and bidirectional encoder large language models, the `evidential signal' arising from the Dirichlet strength in EDL can, in some cases, discriminate between classes, which is particularly strong when using large language models. We hypothesise that the KL regularisation term causes EDL to couple aleatoric and epistemic uncertainty. In this paper, we empirically investigate the correlations between misclassification and evaluated uncertainty, and show that EDL's `evidential signal' is due to misclassification bias. We critically evaluate EDL with other Dirichlet-based approaches, namely Generative Evidential Neural Networks (EDL-GEN) and Prior Networks, and show theoretically and empirically the differences between these loss functions. We conclude that EDL's coupling of uncertainty arises from these differences due to the use (or lack) of out-of-distribution samples during training.

Title: Fine-Tuning Generative Models as an Inference Method for Robotic Tasks. (arXiv:2310.12862v1 [cs.LG])

• Paper URL: http://arxiv.org/abs/2310.12862
• Code URL: null
• Copy Paste: [[2310.12862]] Fine-Tuning Generative Models as an Inference Method for Robotic Tasks(http://arxiv.org/abs/2310.12862)
• Summary:

  Adaptable models could greatly benefit robotic agents operating in the real world, allowing them to deal with novel and varying conditions. While approaches such as Bayesian inference are well-studied frameworks for adapting models to evidence, we build on recent advances in deep generative models which have greatly affected many areas of robotics. Harnessing modern GPU acceleration, we investigate how to quickly adapt the sample generation of neural network models to observations in robotic tasks. We propose a simple and general method that is applicable to various deep generative models and robotic environments. The key idea is to quickly fine-tune the model by fitting it to generated samples matching the observed evidence, using the cross-entropy method. We show that our method can be applied to both autoregressive models and variational autoencoders, and demonstrate its usability in object shape inference from grasping, inverse kinematics calculation, and point cloud completion.

large language model

Title: Lost in Translation: When GPT-4V(ision) Can't See Eye to Eye with Text. A Vision-Language-Consistency Analysis of VLLMs and Beyond. (arXiv:2310.12520v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12520
• Code URL: null
• Copy Paste: [[2310.12520]] Lost in Translation: When GPT-4V(ision) Can't See Eye to Eye with Text(http://arxiv.org/abs/2310.12520)
• Summary:

  Recent advancements in multimodal techniques open exciting possibilities for models excelling in diverse tasks involving text, audio, and image processing. Models like GPT-4V, blending computer vision and language modeling, excel in complex text and image tasks. Numerous prior research endeavors have diligently examined the performance of these Vision Large Language Models (VLLMs) across tasks like object detection, image captioning and others. However, these analyses often focus on evaluating the performance of each modality in isolation, lacking insights into their cross-modal interactions. Specifically, questions concerning whether these vision-language models execute vision and language tasks consistently or independently have remained unanswered. In this study, we draw inspiration from recent investigations into multilingualism and conduct a comprehensive analysis of model's cross-modal interactions. We introduce a systematic framework that quantifies the capability disparities between different modalities in the multi-modal setting and provide a set of datasets designed for these evaluations. Our findings reveal that models like GPT-4V tend to perform consistently modalities when the tasks are relatively simple. However, the trustworthiness of results derived from the vision modality diminishes as the tasks become more challenging. Expanding on our findings, we introduce "Vision Description Prompting," a method that effectively improves performance in challenging vision-related tasks.

Title: Measuring Pointwise $\mathcal{V}$-Usable Information In-Context-ly. (arXiv:2310.12300v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12300
• Code URL: null
• Copy Paste: [[2310.12300]] Measuring Pointwise $\mathcal{V}$-Usable Information In-Context-ly(http://arxiv.org/abs/2310.12300)
• Summary:

  In-context learning (ICL) is a new learning paradigm that has gained popularity along with the development of large language models. In this work, we adapt a recently proposed hardness metric, pointwise $\mathcal{V}$-usable information (PVI), to an in-context version (in-context PVI). Compared to the original PVI, in-context PVI is more efficient in that it requires only a few exemplars and does not require fine-tuning. We conducted a comprehensive empirical analysis to evaluate the reliability of in-context PVI. Our findings indicate that in-context PVI estimates exhibit similar characteristics to the original PVI. Specific to the in-context setting, we show that in-context PVI estimates remain consistent across different exemplar selections and numbers of shots. The variance of in-context PVI estimates across different exemplar selections is insignificant, which suggests that in-context PVI are stable. Furthermore, we demonstrate how in-context PVI can be employed to identify challenging instances. Our work highlights the potential of in-context PVI and provides new insights into the capabilities of ICL.

Title: Document-Level Language Models for Machine Translation. (arXiv:2310.12303v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12303
• Code URL: null
• Copy Paste: [[2310.12303]] Document-Level Language Models for Machine Translation(http://arxiv.org/abs/2310.12303)
• Summary:

  Despite the known limitations, most machine translation systems today still operate on the sentence-level. One reason for this is, that most parallel training data is only sentence-level aligned, without document-level meta information available. In this work, we set out to build context-aware translation systems utilizing document-level monolingual data instead. This can be achieved by combining any existing sentence-level translation model with a document-level language model. We improve existing approaches by leveraging recent advancements in model combination. Additionally, we propose novel weighting techniques that make the system combination more flexible and significantly reduce computational overhead. In a comprehensive evaluation on four diverse translation tasks, we show that our extensions improve document-targeted scores substantially and are also computationally more efficient. However, we also find that in most scenarios, back-translation gives even better results, at the cost of having to re-train the translation system. Finally, we explore language model fusion in the light of recent advancements in large language models. Our findings suggest that there might be strong potential in utilizing large language models via model combination.

Title: A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4. (arXiv:2310.12321v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12321
• Code URL: null
• Copy Paste: [[2310.12321]] A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4(http://arxiv.org/abs/2310.12321)
• Summary:

  Large language models (LLMs) are a special class of pretrained language models obtained by scaling model size, pretraining corpus and computation. LLMs, because of their large size and pretraining on large volumes of text data, exhibit special abilities which allow them to achieve remarkable performances without any task-specific training in many of the natural language processing tasks. The era of LLMs started with OpenAI GPT-3 model, and the popularity of LLMs is increasing exponentially after the introduction of models like ChatGPT and GPT4. We refer to GPT-3 and its successor OpenAI models, including ChatGPT and GPT4, as GPT-3 family large language models (GLLMs). With the ever-rising popularity of GLLMs, especially in the research community, there is a strong need for a comprehensive survey which summarizes the recent research progress in multiple dimensions and can guide the research community with insightful future research directions. We start the survey paper with foundation concepts like transformers, transfer learning, self-supervised learning, pretrained language models and large language models. We then present a brief overview of GLLMs and discuss the performances of GLLMs in various downstream tasks, specific domains and multiple languages. We also discuss the data labelling and data augmentation abilities of GLLMs, the robustness of GLLMs, the effectiveness of GLLMs as evaluators, and finally, conclude with multiple insightful future research directions. To summarize, this comprehensive survey paper will serve as a good resource for both academic and industry people to stay updated with the latest research related to GPT-3 family large language models.

Title: The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions. (arXiv:2310.12418v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12418
• Code URL: null
• Copy Paste: [[2310.12418]] The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions(http://arxiv.org/abs/2310.12418)
• Summary:

  Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks. However, it remains unclear whether the existing focus of NLP research accurately captures the genuine requirements of human users. This paper provides a comprehensive analysis of the divergence between current NLP research and the needs of real-world NLP applications via a large-scale collection of user-GPT conversations. We analyze a large-scale collection of real user queries to GPT. We compare these queries against existing NLP benchmark tasks and identify a significant gap between the tasks that users frequently request from LLMs and the tasks that are commonly studied in academic research. For example, we find that tasks such as ``design'' and ``planning'' are prevalent in user interactions but are largely neglected or different from traditional NLP benchmarks. We investigate these overlooked tasks, dissect the practical challenges they pose, and provide insights toward a roadmap to make LLMs better aligned with user needs.

Title: MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models. (arXiv:2310.12426v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12426
• Code URL: null
• Copy Paste: [[2310.12426]] MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models(http://arxiv.org/abs/2310.12426)
• Summary:

  Language Models (LMs) have shown impressive performance in various natural language tasks. However, when it comes to natural language reasoning, LMs still face challenges such as hallucination, generating incorrect intermediate reasoning steps, and making mathematical errors. Recent research has focused on enhancing LMs through self-improvement using feedback. Nevertheless, existing approaches relying on a single generic feedback source fail to address the diverse error types found in LM-generated reasoning chains. In this work, we propose Multi-Aspect Feedback, an iterative refinement framework that integrates multiple feedback modules, including frozen LMs and external tools, each focusing on a specific error category. Our experimental results demonstrate the efficacy of our approach to addressing several errors in the LM-generated reasoning chain and thus improving the overall performance of an LM in several reasoning tasks. We see a relative improvement of up to 20% in Mathematical Reasoning and up to 18% in Logical Entailment.

Title: Contrastive Learning for Inference in Dialogue. (arXiv:2310.12467v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12467
• Code URL: https://github.com/hltchkust/contrastive_inference_dialogue
• Copy Paste: [[2310.12467]] Contrastive Learning for Inference in Dialogue(http://arxiv.org/abs/2310.12467)
• Summary:

  Inference, especially those derived from inductive processes, is a crucial component in our conversation to complement the information implicitly or explicitly conveyed by a speaker. While recent large language models show remarkable advances in inference tasks, their performance in inductive reasoning, where not all information is present in the context, is far behind deductive reasoning. In this paper, we analyze the behavior of the models based on the task difficulty defined by the semantic information gap -- which distinguishes inductive and deductive reasoning (Johnson-Laird, 1988, 1993). Our analysis reveals that the disparity in information between dialogue contexts and desired inferences poses a significant challenge to the inductive inference process. To mitigate this information gap, we investigate a contrastive learning approach by feeding negative samples. Our experiments suggest negative samples help models understand what is wrong and improve their inference generations.

Title: Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models. (arXiv:2310.12481v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12481
• Code URL: null
• Copy Paste: [[2310.12481]] Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models(http://arxiv.org/abs/2310.12481)
• Summary:

  In this paper, we identify a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g. ChatGPT). LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages. To systematically evaluate the cultural dominance issue, we build a benchmark that consists of both concrete (e.g. holidays and songs) and abstract (e.g. values and opinions) cultural objects. Empirical results show that the representative GPT models suffer from the culture dominance problem, where GPT-4 is the most affected while text-davinci-003 suffers the least from this problem. Our study emphasizes the need for critical examination of cultural dominance and ethical consideration in their development and deployment. We show two straightforward methods in model development (i.e. pretraining on more diverse data) and deployment (e.g. culture-aware prompting) can significantly mitigate the cultural dominance issue in LLMs.

Title: Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong. (arXiv:2310.12558v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12558
• Code URL: null
• Copy Paste: [[2310.12558]] Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong(http://arxiv.org/abs/2310.12558)
• Summary:

  Large Language Models (LLMs) are increasingly used for accessing information on the web. Their truthfulness and factuality are thus of great interest. To help users make the right decisions about the information they're getting, LLMs should not only provide but also help users fact-check information. In this paper, we conduct experiments with 80 crowdworkers in total to compare language models with search engines (information retrieval systems) at facilitating fact-checking by human users. We prompt LLMs to validate a given claim and provide corresponding explanations. Users reading LLM explanations are significantly more efficient than using search engines with similar accuracy. However, they tend to over-rely the LLMs when the explanation is wrong. To reduce over-reliance on LLMs, we ask LLMs to provide contrastive information - explain both why the claim is true and false, and then we present both sides of the explanation to users. This contrastive explanation mitigates users' over-reliance on LLMs, but cannot significantly outperform search engines. However, showing both search engine results and LLM explanations offers no complementary benefits as compared to search engines alone. Taken together, natural language explanations by LLMs may not be a reliable replacement for reading the retrieved passages yet, especially in high-stakes settings where over-relying on wrong AI explanations could lead to critical consequences.

Title: Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing. (arXiv:2310.12664v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12664
• Code URL: null
• Copy Paste: [[2310.12664]] Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing(http://arxiv.org/abs/2310.12664)
• Summary:

  The emergence of Large Language Models (LLMs), such as ChatGPT, has revolutionized general natural language preprocessing (NLP) tasks. However, their expertise in the financial domain lacks a comprehensive evaluation. To assess the ability of LLMs to solve financial NLP tasks, we present FinLMEval, a framework for Financial Language Model Evaluation, comprising nine datasets designed to evaluate the performance of language models. This study compares the performance of encoder-only language models and the decoder-only language models. Our findings reveal that while some decoder-only LLMs demonstrate notable performance across most financial tasks via zero-shot prompting, they generally lag behind the fine-tuned expert models, especially when dealing with proprietary datasets. We hope this study provides foundation evaluations for continuing efforts to build more advanced LLMs in the financial domain.

Title: GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents. (arXiv:2310.12821v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12821
• Code URL: null
• Copy Paste: [[2310.12821]] GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents(http://arxiv.org/abs/2310.12821)
• Summary:

  Current gesture recognition systems primarily focus on identifying gestures within a predefined set, leaving a gap in connecting these gestures to interactive GUI elements or system functions (e.g., linking a 'thumb-up' gesture to a 'like' button). We introduce GestureGPT, a novel zero-shot gesture understanding and grounding framework leveraging large language models (LLMs). Gesture descriptions are formulated based on hand landmark coordinates from gesture videos and fed into our dual-agent dialogue system. A gesture agent deciphers these descriptions and queries about the interaction context (e.g., interface, history, gaze data), which a context agent organizes and provides. Following iterative exchanges, the gesture agent discerns user intent, grounding it to an interactive function. We validated the gesture description module using public first-view and third-view gesture datasets and tested the whole system in two real-world settings: video streaming and smart home IoT control. The highest zero-shot Top-5 grounding accuracies are 80.11% for video streaming and 90.78% for smart home tasks, showing potential of the new gesture understanding paradigm.

Title: StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding. (arXiv:2310.12874v1 [cs.CL])

• Paper URL: http://arxiv.org/abs/2310.12874
• Code URL: null
• Copy Paste: [[2310.12874]] StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding(http://arxiv.org/abs/2310.12874)
• Summary:

  Analogy-making between narratives is one of the most critical abilities in natural language understanding. In this paper, we evaluate the ability to identify and generate analogy by building a first-of-its-kind large-scale story-level analogy corpus, StoryAnalogy, which contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We design a set of tests on StoryAnalogy, presenting the first evaluation of story-level analogy identification and generation. Interestingly, we find that the analogy identification tasks are extremely challenging not only for the sentence embedding models but also for the recent large language models (LLMs) such as ChatGPT and LLaMa, where ChatGPT only achieved around 30% accuracy in multiple-choice questions (> 85% accuracy for humans). Finally, we find that data in StoryAnalogy can improve LLMs analogy generation quality, where a fine-tuned FlanT5-xxl model yields comparable performance to zero-shot ChatGPT.

segmentation

Title: Deep Learning Techniques for Video Instance Segmentation: A Survey. (arXiv:2310.12393v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12393
• Code URL: null
• Copy Paste: [[2310.12393]] Deep Learning Techniques for Video Instance Segmentation: A Survey(http://arxiv.org/abs/2310.12393)
• Summary:

  Video instance segmentation, also known as multi-object tracking and segmentation, is an emerging computer vision research area introduced in 2019, aiming at detecting, segmenting, and tracking instances in videos simultaneously. By tackling the video instance segmentation tasks through effective analysis and utilization of visual information in videos, a range of computer vision-enabled applications (e.g., human action recognition, medical image processing, autonomous vehicle navigation, surveillance, etc) can be implemented. As deep-learning techniques take a dominant role in various computer vision areas, a plethora of deep-learning-based video instance segmentation schemes have been proposed. This survey offers a multifaceted view of deep-learning schemes for video instance segmentation, covering various architectural paradigms, along with comparisons of functional performance, model complexity, and computational overheads. In addition to the common architectural designs, auxiliary techniques for improving the performance of deep-learning models for video instance segmentation are compiled and discussed. Finally, we discuss a range of major challenges and directions for further investigations to help advance this promising research field.

Title: Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing. (arXiv:2310.12452v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12452
• Code URL: https://github.com/hanbobizl/dmnet
• Copy Paste: [[2310.12452]] Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing(http://arxiv.org/abs/2310.12452)
• Summary:

  Few-shot segmentation (FSS) is proposed to segment unknown class targets with just a few annotated samples. Most current FSS methods follow the paradigm of mining the semantics from the support images to guide the query image segmentation. However, such a pattern of `learning from others' struggles to handle the extreme intra-class variation, preventing FSS from being directly generalized to remote sensing scenes. To bridge the gap of intra-class variance, we develop a Dual-Mining network named DMNet for cross-image mining and self-mining, meaning that it no longer focuses solely on support images but pays more attention to the query image itself. Specifically, we propose a Class-public Region Mining (CPRM) module to effectively suppress irrelevant feature pollution by capturing the common semantics between the support-query image pair. The Class-specific Region Mining (CSRM) module is then proposed to continuously mine the class-specific semantics of the query image itself in a `filtering' and `purifying' manner. In addition, to prevent the co-existence of multiple classes in remote sensing scenes from exacerbating the collapse of FSS generalization, we also propose a new Known-class Meta Suppressor (KMS) module to suppress the activation of known-class objects in the sample. Extensive experiments on the iSAID and LoveDA remote sensing datasets have demonstrated that our method sets the state-of-the-art with a minimum number of model parameters. Significantly, our model with the backbone of Resnet-50 achieves the mIoU of 49.58% and 51.34% on iSAID under 1-shot and 5-shot settings, outperforming the state-of-the-art method by 1.8% and 1.12%, respectively. The code is publicly available at https://github.com/HanboBizl/DMNet.

Title: Lidar Panoptic Segmentation and Tracking without Bells and Whistles. (arXiv:2310.12464v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12464
• Code URL: https://github.com/abhinavagarwalla/most-lps
• Copy Paste: [[2310.12464]] Lidar Panoptic Segmentation and Tracking without Bells and Whistles(http://arxiv.org/abs/2310.12464)
• Summary:

  State-of-the-art lidar panoptic segmentation (LPS) methods follow bottom-up segmentation-centric fashion wherein they build upon semantic segmentation networks by utilizing clustering to obtain object instances. In this paper, we re-think this approach and propose a surprisingly simple yet effective detection-centric network for both LPS and tracking. Our network is modular by design and optimized for all aspects of both the panoptic segmentation and tracking task. One of the core components of our network is the object instance detection branch, which we train using point-level (modal) annotations, as available in segmentation-centric datasets. In the absence of amodal (cuboid) annotations, we regress modal centroids and object extent using trajectory-level supervision that provides information about object size, which cannot be inferred from single scans due to occlusions and the sparse nature of the lidar data. We obtain fine-grained instance segments by learning to associate lidar points with detected centroids. We evaluate our method on several 3D/4D LPS benchmarks and observe that our model establishes a new state-of-the-art among open-sourced models, outperforming recent query-based models.

Title: RecolorCloud: A Point Cloud Tool for Recoloring, Segmentation, and Conversion. (arXiv:2310.12470v1 [cs.CV])

• Paper URL: http://arxiv.org/abs/2310.12470
• Code URL: null
• Copy Paste: [[2310.12470]] RecolorCloud: A Point Cloud Tool for Recoloring, Segmentation, and Conversion(http://arxiv.org/abs/2310.12470)
• Summary:

  Point clouds are a 3D space representation of an environment that was recorded with a high precision laser scanner. These scanners can suffer from environmental interference such as surface shading, texturing, and reflections. Because of this, point clouds may be contaminated with fake or incorrect colors. Current open source or proprietary tools offer limited or no access to correcting these visual errors automatically.

  RecolorCloud is a tool developed to resolve these color conflicts by utilizing automated color recoloring. We offer the ability to deleting or recoloring outlier points automatically with users only needing to specify bounding box regions to effect colors. Results show a vast improvement of the photo-realistic quality of large point clouds. Additionally, users can quickly recolor a point cloud with set semantic segmentation colors.